Bulk import fails send images to detection and return annotations

What Wildbook are you working in?
IoT

What is the entire URL out of the browser, exactly where the error occurred?

Can you describe what the issue is you’re experiencing?
1512 encounters were bulk imported, along with 4362 associated images. The images were not sent to the detection pipeline and no annotations were returned. No sighting IDs were added either. I would also like to add the records from the bulk import to a project.

Can you provide steps on how to reproduce what you’re experiencing?
I will email the spreadsheet
If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report

Hi @Justin

I am resending the assets to detection. I will report back when they’re done.

Thanks,
Jason

Hello @jason and @tanyastere,

As suggested, I have now deleted the original batch import that was too large for the system to process, and have added a new batch import of 367 encounters/971 images. Can you please start the detection process on these encounters? The import ID is 21bbb1ea-5d75-4ed3-ad3f-bb5fb6c08499

Regards, Justin

Hi @Justin

Do you see the “send to detection” button at the bottom of the bulk import page?

That should kick off the detection pipeline and start drawing the bounding boxes around your animals in preparation for the next step of individual ID, which will appear as a new button when detection completes.

Thank you,
Jason

Thanks @Jason. I don’t get the option to ‘send to detection’. See screenshot below

Hi @Justin

I have boosted your permissions. Can you please try again?

Thanks,
Jason

Hi @Jason,

Thanks for that. With the elevated permission I was able to send the bulk import to detection, but it still doesn’t allow me to match the detected turtles against the catalogue (i.e. no ‘match results’ option).

Regards, Justin

Hi @Justin Since this thread was closed due to inactivity, can you start a new one and include a link to your bulk import as well as send a copy of your spreadsheet to services at wildme dot org so we can troubleshoot? Thanks!

Reopening thread per email discussion.

I’m including the link to your latest bulk import for reference as well: Internet of Turtles

I’ll follow up soon with next steps after I’ve looked more carefully at the spreadsheet and encounters.

Hi @Justin

Thanks for your patience! A few quick notes on your spreadsheet:

  • Three of your file names have parentheses. These are considered special characters. If it’s not A-Z, a-z, 0-9, space, or period, it’s a special character and may not play nice when uploaded.
  • The field Encounter.submitterID should be your IoT username. Without it, the system doesn’t realize who to assign these encounters to and won’t let you edit them.
  • I don’t think this is creating issues in this instance, but I’m listing it here as a best practice: we recommend removing any unused fields from your spreadsheet before uploading it. The system will occasionally detect an empty field and then ignore the data in other similar fields.

I recently pinned a Bulk Import Cheat Sheet to the Community that covers the most frequent issues we see with spreadsheets.

As far as next steps, we can reset your bulk import, so we’re starting from a clean slate while not affecting any of your past manually-entered encounters. After that, you can submit your updated spreadsheet with the above points corrected (feel free to email it to us for review before you submit it for upload). If any issues crop up after that new upload, we should be able to quickly pinpoint a fix.

Let us know if that works for you and I’ll let you know when we’ve reset the import and you can start preparing your submission again!

Hi @Justin

Do you still need help with this? It’s been a while since we heard from you. We’re waiting on you to confirm if you want us to reset your bulk import before you try a new upload.

Hello @Anastasia, I made the requested changes to the spreadsheet and tried again (6f769fa8-d6b0-41fc-a959-dbe909b2245e). It is still sending to detection. Please check if it has worked this time, if not both imports will need to be deleted. Regards, Justin

I checked your import and it looks like it’s completed identification on all but one image:

I’m not sure why that last image is stuck, considering that this was uploaded on Feb 8th.

I’ll follow up soon about the stuck image.

Hey @Justin

Sorry for the late follow-up. We tried resetting your imports but have had mixed results on the matches. I noticed that in your spreadsheet, the Encounter.locationID isn’t using any of the Location IDs that exist in IoT. Because identification is run by narrowing match results to existing regions, it’s not running matches because entires like “CB” or “H” aren’t valid Locations IDs in IoT. You can find the available locations in the drop-down Location ID menu on the Report an Encounter page. These will always be your point of reference for entering the location:

And here I’ve scrolled down to the currently available Australian options:

When you update your spreadsheet, try using fewer encounters per bulk import to simplify troubleshooting and to reduce strain on the servers. Like 100-200. As I said before, send over your updated spreadsheet before you make a new upload so we can rule out any data issues before uploading and hopefully resolving this soon.