What operating system were you using? (eg. MacOS 10.15.3)
Windows 10
What web browser were you using? (eg. Chrome 79)
Chrome Version 91.0.4472.124,
What is your role on the site? (admin, researcher, etc)
Researcher, Rest, Machinelearning
What happened?
Matching process (again unfortunately, with or without hotspotter) is very slow or not working. Stuck on ‘attempting to fetch results’. This happens after uploading a new encounter, as well as after trying to ‘start another match’ on old entries
Match results will display results from same event. (comparing images from same encounter) Internet of Turtles
Sometimes it will try to match against encounters from outside ‘my data’ Internet of Turtles
results for known turtles, turtles that we have multiple encounters from, will not be matched up or listed in results. In these instances I have to assign the correct ID manually. But even after running another match it will still not match up with images of different encounters of that individual.
What did you expect to happen?
For the correct matches from my data to be listed
What are some steps we could take to reproduce the issue?
If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report
Hi, @noaahawaii !
These issues sound eerily similar to some that we have solved recently on other platforms. I’ll plan on updating IoT this week, and then can we see whether that fixes your issues?
Cheers,
Mark
Hi, @noaahawaii !
Still on the docket! There’s been a lot of activity on community lately, so the backlog has been growing. But I have not forgotten you, and you’re definitely on the queue!
-Mark
Should I keep uploading or pauze until there is a proper fix since issues are persisting?
If there is no properly functioning matching process and I only assign (temporary) ID’s to individuals would that still work, or am I then only creating a lot more extra work for myself because I would have to go back in later once there is a fix and run the matching process again to properly compare individuals?
Is there also a way for me to track the internal issue or would I be notified of the progress once ready?
Should I keep uploading or pauze until there is a proper fix since issues are persisting?
Yes. Adding media/encounters to the system will not be affected, so carry on with this.
If there is no proper functioning matching process and I only assign (temporary) ID’s to individuals would that still work, or am I then only creating a lot more extra work for myself because I would have to go back in later once there is a fix and run the matching process again to properly compare individuals?
I’m not super clear what you mean here. If you are able to find and confirm a match with PIE or hotspotter using the manual annotation tool, you can go ahead and assign the ID (perhaps for an individual that already exists in the database, for instance) that way. If you find yourself assigning “temporary” IDs (this is the part of your response that I’m a little confused by), I’d say just hold off on doing that until matching the way you’re used to is fully restored. If you haven’t used the manual annotation tool before, it might be worth reading about and trying! I’d love for @tanyastere to weigh in on this one if her opinion is different from mine.
Is there also a way for me to track the internal issue or would I be notified of the progress once ready?
Unfortunately, not yet. But it’s part of the protocol for us to loop back around when the task is completed and let you know here!
I believe this issue has been resolved (partially or fully), though you’ll need to re-run matching or submit new data to see more consistent results sets.
TLDR; Our machine learning detector model predicts species and feature (body or head). When it gets species wrong, it will then match against only that type of species and feature. At the links you provided above, I saw weird/mixed matching sets, and I traced the issue to missed species predictions. I add a set of configurations that will help prevent this in the future and migrated past, missed species predictions to their correct values. This should provide better matching results across IoT for the future. Thanks for bringing this to our attention.
Would you be able to test with new data or re-run some matches and confirm here that things are better?
For #2, that is normal behavior. The system is reporting that it ran successfully, and it did not think that anything met a minimum threshold (score 0) for matching.
For #1, can you describe how you achieved this result? I can absolutely see what you saw, but whenever I re-run matching with “start another match” I can’t reproduce it. Same Encounter matches are excluded.
I normally uploaded a new entry and ran a matching process. After upload I started ‘start another match’ and this was produced. I have seen this before on other entries I had done over the past year or so. But yes, I see that after re-running, it does not show up anymore.
For #2 could it be that the system has a hard time detecting patterns on these images due to low/poor contrast or quality of the image? I do tend to see it happen more often in those cases.
For #2, it could be related to quality. HotSpotter has a minimum threshold of matchability, and if no matches rise above it (i.e. there is very little texture correspondence), we do get a response like this, and it is normal. Unless you know there is a match in the database, generally this is a good sign that this individual is unmatched and unique.