Flukebook bulk imports stuck in detection

What Wildbook are you working in?

  • Flukebook

What is the entire URL out of the browser, exactly where the error occurred?

Can you describe what the issue is you’re experiencing?

  • I uploaded 17 bulk imports 2 weeks ago. Seven of these bulk imports have not undergone detection.
    The links to these bulk imports:

https://www.flukebook.org/react/bulk-import-task?id=c8b05993-7d61-4439-bba3-e738ebe715c9

https://www.flukebook.org/react/bulk-import-task?id=a4e8c0ef-9ad3-4b4a-8e68-714af6b4fe8b

https://www.flukebook.org/react/bulk-import-task?id=8098ae62-93f6-4320-980d-6d4e9d7fbe65

https://www.flukebook.org/react/bulk-import-task?id=91a96202-c72e-4e9b-bfb1-adcfdff0fb07

https://www.flukebook.org/react/bulk-import-task?id=4e6fa47c-2526-4cf4-8b78-41ceb6ef0636

https://www.flukebook.org/react/bulk-import-task?id=2a83bc5c-7c7a-40fe-b27a-7dcda1a1a278

https://www.flukebook.org/react/bulk-import-task?id=8537fbe1-e611-4d66-9913-1c5eb815dfe7

Can you provide steps on how to reproduce what you’re experiencing?

  • Uploading of the documents did not produce errors, yet for these seven bulk imports the detection is still at 0% after two weeks.

If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report

Hi @phocoena

Thanks for sending your spreadsheets!

I’ve resent your imports to detection. It looks like Flukebook is working through a backlog right now, so it may take a little longer than usual for detection and identification to complete.

Hi Anastasia,
Thank you very much for your help!

Dear Anastasia,
Thank you again for your assistance and prompt response. The seven imports are still awaiting detection. Do you have any indication of how long this might take? Will it be another day, a week, or longer?
My apologies for reaching out again, but my student has already been waiting for the results for three weeks, and her deadlines are approaching. It would be helpful to know whether it is worth waiting, or if we should consider using alternative software or methods. Thank you once more, and I wish you a nice weekend!

No need to apologize! Backlogs usually shouldn’t take more than a day to clear. I’ve got the devs looking into why these particular imports aren’t completing. I’ll let you know as soon as I have an update.

1 Like

Quick update: It looks like Flukebook is trying to use the older PIE v2 algorithm for this species which are meant to be used with prominent fins in the photo. Some of these encounters have tiny little fins far away. We may not be able to match such indistinct fins.

We’re also going to update the primary algorithm for this species to be MiewID which should return results faster and tends to have a better track record with images that have been historically more difficult to identify. I should have more info by tomorrow.

1 Like

Thank you so much for looking into this so quickly and for the clear explanation.
Great to hear that switching to MiewID may improve both speed and match quality. I really appreciate the update and will wait for further information tomorrow.

Thanks again for your efforts!

1 Like

Thanks for your patience. We’ve enabled MiewID for this species. Going forward, you should see fewer issues with identification wait times. We’ve re-sent to ID any imports that had incomplete identifications. Let me know if you find any issues as you review the matches.

There’s also a separate issue we’re tracking in this ticket where the status boxes at the top of the bulk import page aren’t accurately reflecting an import’s detection and identifications status. Using your first bulk import link as an example, it shows detection at 0% but identification at 100%, despite the fact that identification can’t begin unless detection has already completed. Usually a more reliable indicator of identification status is the Class column on the import task page:

“Completed” means it finished the identification process (more info on statuses are in our bulk import FAQ). Where there’s a dash instead of a status means that the detector couldn’t find an animal in the picture. For those images, you’ll want to go to the encounter and add a manual annotation and then manually start a match after the annotation is saved.

Thank you very much for all your help! I’m excited to start the matching process.

I did have one more question.
How can I assign the images from all my bulk imports to my project ‘Photo ID Sara’?
My Excel documents did include the column ‘Encounter.project0.researchProjectName’, but I forgot to add the column ‘Encounter.project0.projectIdPrefix’.

I tried to add the images by following the advice you gave someone else.

  1. Start an Encounter Search

  2. Click on Metadata in the left nav and enter your spreadsheet name in the Bulk Import File Names field.

  3. Submit your search

  4. Click on the Project Management tab of the search results page

  5. Select your project name from the Select Projects To Add To list and click the Add to Project(s) button.

Except, after selecting my bulk imports, it said there were no records to display. In the Metadata section of the encounters from my bulk imports, there is no ‘Project’ displayed.

Can you email me your spreadsheet so I can take a look at why it didn’t added these encounters to your project? services@wildme.org

This sounds like the issue reported in this ticket: Encounter React search doesn't display results for Bulk Import File name field · Issue #1277 · WildMeOrg/Wildbook · GitHub

I’ll look into alternate ways to get the bulk import into your project.

Thanks for sending your spreadsheets! Unfortunately, both Encounter.project0.projectIdPrefix and Encounter.project0.researchProjectName need to be present on your spreadsheet otherwise it won’t save. I tested importing one of your spreadsheets and see that the validation screen didn’t flag this for you, so I’ve created a ticket for it: If project field is missing from bulk import, it doesn't get flagged during validation · Issue #1314 · WildMeOrg/Wildbook · GitHub

Can you let me know if all of your Wildbook data should be in the project or is it just the encounters in these imports? That will help me figure out the fastest way to get these into your project.

Thank you for the quick and clear explanation!

The encounters of all of my bulk imports should be in the project, along with the encounters I’ve already added to the project.

Thanks again for all your hard work!

Ok, so not the entirety of your uploaded encounter data belongs in the project, just the encounters from the 17 imports. I’ll work on this today and let you know when I have an update.

Hi Anastasia, I was just wondering if you have an update for me? Thanks

Thanks for hanging in there. This should be wrapped up by end of day today. I’ll let you know if that timeline changes.

Hi @phocoena

I’m sorry for the delay, I hope to have these finished by tomorrow.

Your encounters should all be in your project now. Thank you!

Thank you so much for all your work!

1 Like