ACW bulk import not progressing through ID after 2 days

What Wildbook are you working in? ACW

What is the entire URL out of the browser, exactly where the error occurred?

Can you describe what the issue is you’re experiencing?
User sent this bulk import to ID 2 days ago and there’s been no progress. I’ve checked Grafana and the Data Integrity IA queue details and all appear normal. I’ve also checked the user’s spreadsheet and it looks fine too; I checked for extra spaces at the end of the species details and other columns and didn’t see anything unusual there, although I may be missing something. Anyway, I’m not sure what could be causing the delay on this batch so if someone could investigate, I’d appreciate it.

I’ve sent the upload spreadsheet to services@wildme.org

Thanks!
Maureen

I’m looking into the wider issue of why ACW is slower than usual. I’ll follow up as soon as I have more info.

1 Like

The Grafana dashboard reporting won’t be accurate until Jason finishes reworking the metrics. Unfortunately that will be delayed until after this week due to grant deadlines.

The spreadsheet looks fine to me, too.

I resumed stalled tasks on this import, which is a little different from re-sending the entire thing to ID again. If the status is queued and it’s stuck, it should force just the stuck ones back in line while leaving the ones that are behaving correctly alone. Let me know if this one still isn’t cooperating by tomorrow.

1 Like

Hello Anastasia

There does not appear to be any progress on this bulk upload.

My next step will be to delete it and rerun.

I will let you know what happens then.

Thanks for your effort on this.

Regards

Paul

cc: @ACWadmin1

1 Like

Thanks for the update! Let me know if things are still weird after re-importing so I can keep digging in.

Hi @Anastasia, in fact we’re not deleting and re-running this bulk import. Paul replied to the wrong post. The import in this post belongs to another user and so we can’t just delete and re-run it for them as we don’t have the images to re-import.

And now that I’ve checked it again it still hasn’t progressed at all. As I noted, I checked the import file for issues that I know might cause problems but I couldn’t find anything.

Could you please look into why this bulk import isn’t getting processed by ID at all? We’re now on day 5 since it was sent to ID.

thanks
Maureen

Manually re-running ID, forcing stalled “queued” jobs to re-run, and even just manually running a new match from the encounter page on this import hasn’t made any visible progress on its ID status. Can you let the user know to try deleting and re-importing it and letting us know if there are still issues with it?

Hi @Anastasia - the bulk import was deleted and re-imported last evening, Pacific time. Here’s the new import: https://africancarnivore.wildbook.org/import.jsp?taskId=56d0e8c7-39df-427a-820a-7748497bf58b
While it said that detection jobs were averaging 4 minutes each, and there were no other detection jobs running, after half an hour in detection, there was no progress.
This morning I see it’s at 56/57 detections completed. So it looks like another bulk import stuck on the last detection job. If you can fix that for us, then we can once again kick off ID and see what happens.

Thanks
Maureen

1 Like

Thanks for the updated link! I’ve re-sent it to detection.

The bulk import is again stuck at the last detection. So no improvement since the previous push through detection.

Thanks
Maureen

Thank you. I’m taking this feedback to the team as we keep working on why detections aren’t completing.

Detections completed yesterday so I kicked off identification yesterday as well. This morning, there is again no progress through the ID queue.

Please help.

Thanks
Maureen

We’ve applied a new build for ACW that seems more stable, but there are still some data issues we’re working out. I’ve forced this import back through the queue in the meantime.

1 Like

The queued jobs in ACW are going to take a while to complete because there are around 600 right now, but things are looking good so far on our end.

Let me know if you see other examples of bulk imports that get stuck in detection where it’s one image short of being complete.

Resolving as the bulk import ID status is showing as complete.

1 Like