Incorrect annotation area and not recognizing head in IOT

Bug Reports
,
1

What Wildbook are you working in?
IOT
What is the entire URL out of the browser, exactly where the error occurred?
https://iot.wildbook.org/encounters/encounter.jsp?number=0cee16d7-b586-4072-a1ce-52c1a3a223f3
https://iot.wildbook.org/encounters/encounter.jsp?number=c2212658-6093-4285-8c50-8c87eb311258
https://iot.wildbook.org/encounters/encounter.jsp?number=c2212658-6093-4285-8c50-8c87eb311258
https://iot.wildbook.org/individuals.jsp?number=a431d8d9-f628-407e-bcc9-e2305dc208bd
https://iot.wildbook.org/encounters/encounter.jsp?number=03072479-673d-466a-a652-5000b926aa39
https://iot.wildbook.org/encounters/encounter.jsp?number=c0dee2b4-b4d7-4691-9fb7-614ed59930b1
https://iot.wildbook.org/individuals.jsp?number=e608ae6d-a543-4ad7-bc56-f9e428851aa0

Can you describe what the issue is you’re experiencing?
The incorrect area is selected or the head is not recognized. (I know for some it might be more difficult because of image quality…)
Can you provide steps on how to reproduce what you’re experiencing?

If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report

Thank you again for your help!

2

Hello,

I can see the problem you mention on the encounter pages. It may be related to image
dimensions, or orientation provided by the Exif metadata. We have been doing other work
in this portion of the software, and will keep you updated as changes are made.

I’m tracking the issue as work ticket WB-743, and will investigate further once we complete the scheduled upgrade of the platform we discussed to bring on the manual annotation tool.

Thanks for reporting this.

3

Hello @noaahawaii,

The two fixes we discussed in the other thread should take care of these issues.

I believe that IOT wasn’t handling the horizontal image metadata well and this should be fixed. If there is a new set of images where you see problems with bounding box placement send them our way and we’ll investigate for other cases.

Some of the heads being missed is likely due to our detection model being trained on single animal images with little exception, and with on-land images in the minority, though they typically do well. If we get enough multiple animal / on land images we should open up a discussion on retraining the model using these as a larger component.

The solution for now is using the manual annotation tool as you have been. Hopefully with the new ability to remove errant bounding boxes along with their associated encounter, and better image metadata parsing to ensure the bounding boxes are in the right place this will be more useful to you.

Once again, thanks for reporting these issues so we can fix them here and on other platforms.

4

Would it help you to retrain the model if I send you images of multiple turtles together?

5

Short answer is yes, though we wouldn’t know how much until retraining is complete. We would need to chat with @tanyastere on a timeline for the work, and @parham to see about how many images would be required.

Any retraining of a detection model requires the step of manually adding bounding boxes to the new data- much as you have been with the manual annotation tool in IOT. Generally a few hundred images are needed and detection performance for the specific photo characteristics are improved.

It would be great to improve detection for this type of image if you want to start coordinating on this.