IoT matching issues

Support
,
1

Hello Anastasia,

To follow up from previous issues, we are still having problems with matching. When we uploaded all images to the Internet of Turtles and viewed our submissions there were many that had been duplicated. We checked them all and there were multiple errors such as new annotations. We have recorded these and so have on record which ones to ignore and the original uploads can be searched.

All uploads reported “no match found” even though we know turtles were reencountered over multiple years due to presence of flipper tags, so we know some should match.

We were advised previously the problem was due to there being no “location ID” selected during upload and instead, we just input the name of the place and the coordinates. We then went through all uploads and edited the uploads to include location ID.

As the uploads had then been edited, we had to select “start new match” rather than “match results” as “match results” was not updated with the new location ID.

When we selected “start new match” they look a very long time to load (in fact they never finished loading and the system stated “attempting to fetch results”)

We contacted the community forum again and were told this was due to the program Hotspotter contributing to long queues and taking up disc space thus causing jobs to fail, they disabled Hotspotter as a default algorithm from all the uploads which left the Internet of Turtles just using MiewID for photo recognition.

We sent over a handful of encounters to the community forum for you to investigate and only a few worked, the majority of my encounters do not. There are approx 200 that did not match.

Below are 10 randomly selected encounters from all our uploads. If you click the little menu by the side of an image and select “start new match” you should see the issue.
It seems it is the system itself that is not loading results.

Could you please advise what to do? Should we re-load everything in a new account as this account does not seem to work?

Thanks Nicole

https://iot.wildbook.org/encounters/encounter.jsp?number=d8e769f8-761a-4bc3-ba6a-f9d521819782

https://iot.wildbook.org/encounters/encounter.jsp?number=88fa3a50-162b-4d22-afbf-4eb02033d75a

https://iot.wildbook.org/encounters/encounter.jsp?number=385cfaf3-c96d-477a-932e-f490ead0ac92

https://iot.wildbook.org/encounters/encounter.jsp?number=7c13759a-3121-4faa-82b5-91828874e65a

https://iot.wildbook.org/encounters/encounter.jsp?number=9c017ada-2495-41de-a8d6-7ceb61834a04

https://iot.wildbook.org/encounters/encounter.jsp?number=abd621c5-d0e3-49a9-b177-363591e38d21

https://iot.wildbook.org/encounters/encounter.jsp?number=e3be0018-36ab-49cf-8c30-c4a57d388d7b

https://iot.wildbook.org/encounters/encounter.jsp?number=04373fe1-2d3f-4151-9465-e821a447cf3d

https://iot.wildbook.org/encounters/encounter.jsp?number=c929057e-a574-4fc9-93e5-0bbc8b9123c1

https://iot.wildbook.org/encounters/encounter.jsp?number=8884caaf-6adc-408a-affa-964f8191c883

2

Hi @nesteban

I’m going to move this to its own topic as the previous one was already marked as resolved.

It sounds like there are multiple issues you want to address:

  • Uploading a single encounter results in a duplicate encounter being created
  • No match candidates are displayed on some encounters even though you’re confident these turtles are known individuals in IoT
  • You expected that updating a location ID on an encounter would automatically re-run ID against the updated location ID
  • Long wait for ID to complete/ID does not complete at all

Is this accurate? Typically, we tackle one issue per support topic to streamline troubleshooting. I mainly want to clarify if you’re asking for our help with the match results of the linked encounters you provided plus the above bulleted issues, or if you were summarizing previous issues you’ve had in IoT that are now resolved.

3

Hello, thanks for getting back to me. I was summarising all the issues over the last few months and how we have tried to address them. The key issue is: ID does not complete at all, and No match candidates are displayed on some encounters even though you’re confident these turtles are known individuals in IoT. The links are for examples of some of the uploads where matching process is incomplete.
Thanks, Nicole

4

Thanks for clarifying! For some of these images with the top down view of the shell or partial captures on land can make the annotations imprecise as these viewpoints weren’t well represented in our training data for hawksbills.

For example, the 4 images in your first encounter link where the annotation isn’t fully around the turtle’s body or face would mean that ID couldn’t successfully run from these annotations because they’ve cropped the turtle. In those instances, we’d recommend deleting the existing annotation and adding a new one from the same menu so you can draw it manually. And restart a match from it. The caveat being that match results from these viewpoints may not be as varied because of their lack of representation in our training data.

The rest of the match result for that encounter haven’t been re-run again since January 2024, before the location ID was added in March 2024. I’ve resubmitted those annotations to ID and see match results for them now.

In terms of wait times, if you select Hotspotter when you re-run a match, it will take much longer to get those results. It’s off by default since MiewID is faster, but it will still take MiewID a few minutes for each match to complete. “Attempting to fetch results” may appear for a few minutes until the results are ready.

Looking at the second encounter link you shared, I do see a few empty Hotspotter results from January 2024, so I’ve re-run those with MiewID and I see match candidates appear now. I did not re-run results on annotations that cropped the turtle’s face or shell or ones where match candidates were already available. I prefer to let users delete and manually redraw their own annotations unless I’m asked to.

I’ll continue going through the rest of your links and resending annotations with missing matches to ID. I also want to take this chance to point out that match results are only a snapshot in time and represent candidates based on what was available in IoT at the time it was run. Match results can quickly become outdated as new data is uploaded since old results haven’t factored in any of the new uploads that occurred since it was run.

With your permission, we’d like to use the images you’ve uploaded to help retrain our detector with these viewpoints so that we can improve detection and ID for this species.

5

Hi @Anastasia,

Thank you for your help. I have been continuing to summarise issues over the last few months. The key issue I have been working on are the individuals we know have matches on the system due to tagging but have been uploaded on separate encounters as the pictures were taken at different times. We have a total of 47 individuals (encounters) we know have matches but only 19 of have been matched correctly, I have marked these individuals to the same encounter number as the website suggests.

The rest result in errors with the ID not completing at all or simply do not show their correct match. I have linked the encounters with such errors for your assistance.

Correct match not identified:

Error given: Error initiating IA job

Error given: attempting to fetch results Internet of Turtles

Error given: attempting to fetch results Internet of Turtles

Correct match not identified

Error given: Error initiating IA job

Error given: attempting to fetch results Internet of Turtles

Correct match not identified

Error given: error initiating IA job

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Correct match not identified

Error initiating IA job

Attempting to fetch results

attempting to fetch results

correct match not identified

correct match not identified

correct match not identified

correct match not identified

Could you please advise on how we should go forward with this? For the encounters that do show matches but not their correct one, is this a problem with the image itself or the system?

Many thanks

6

I don’t think we got a response from you about this yet, but since you seem to have a lot of the underrepresented hawksbill viewpoints in your data, this will help with some of the pain points you’ve described.

Which individuals are these supposed to be? I suspect if a correct match doesn’t appear in the match results, it’s because we don’t an previous encounter of the individual that shares these viewpoint already. I can’t say for sure without looking at the encounters already associated with the individual.

Have you restarted matches for the ones that have this error? If this doesn’t work, sometimes deleting the existing annotation and then redrawing it and starting a match from the new annotation will work.

Separately, I also think the image files you’re working with are too large for Wildbook. I looked at one of your examples and the image is 2.6MB and 4000 x 3000 pixels. Our photography guidelines recommend you use images no larger than 1600 pixels and no less than 480 pixels. Larger images take longer to go through ID and when you run into matching errors like you’ve described, it also takes longer to re-send those images back through image analysis.

7

Dear Anastasia

Thanks for your response and I appreciate your time with following up.

We would be happy for you to use our images for retraining the detector. We would like to resolve the issues and hope this will help. Some of the turtles have flipper tags and have been recaptured repeatedly so we can also test the detections with turtles captured on multiple occasions.

We thought the images were below required size. Do we need to delete those images and re-upload with smaller resolutions?

We will try to retry the match for the error as you suggested below.

Please let us know if there is something else we should do.

Regards
Nicole

8

Great; thank you! I’ll let you know if I have any follow-up questions about the data.

No, you don’t have to delete them. This is just advice to keep in mind going forward.

9

Dear Anastasia,

We have taken your advice on uploading images with a smaller size and pixels to the encounters in which we previously experienced errors.
This seemed to work for two encounters, which have now been correctly matched.

However, we still seem to be encountering errors on the following individuals to which we have also reduced the size and number of pixels. These are some examples in which we encounter the error message “attempting to fetch results”.

We appreciate your time following up. Please let us know how we could go about solving these issues.

Kind Regards,

10

Hotspotter was disabled for hawksbill and green sea turtles back in April. I see a few match results that were last run in February (such as this one), which would explain why nothing’s displaying for you.

You would need to re-run ID on outdated match results, which is a good idea anyway since match results are just a snapshot in time and wouldn’t display match candidates that had been uploaded to IoT since the match was run.

As a reminder for the top-down images, the hawksbill model wasn’t trained on that viewpoint, so those viewpoints remain unmatchable for now.

11

By the way, I’m waiting to hear back from our machine learning team about the duplicate annotations. In the meantime, you can go to Administer > Data Integrity > Annotations Duplicated in Two or More Encounters for a quick way to find encounters that share the same photo if you want to start cleaning up duplicate encounters and annotations.

12

The detector will be retrained on your hawksbill data soon so you should see improvement in your annotations when that’s complete. I don’t have a timeline for this yet, but this should also fix the detections on your existing encounters as well.

I’ll let you know when I have an update, but feel free to check in if it’s been a while and you haven’t heard any news yet.