Can you describe what the issue is you’re experiencing?
After checking the match results in the project “Botanical Garden Bielefeld Adults”, a match candidate was shown:
The number shown is relatively high but I noticed the annotation was not perfectly covering the body of the salamander. So, I deleted the annotation, drew a new one (viewpoint: up, IA class. salamander_fire_adult) and started a new match. The new match does not suggest the match candidate anymore.
I drew the new annotation partly also because of curiousity as we have noticed that in some instances, drawing a new (better) annotation provides different match results in both ways: sometimes a match candidate is provided that was not suggested before or like in this case where a match candidate was provided before but not anymore.
So, I’m just trying to understand what happens here. What is the significance of the annotation for the match?
If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report
It’s likely that even though the original annotation is “close enough”, drawing a new one that doesn’t cut off any part of the salamander means the new results aren’t excluding relevant hot spots that were previously outside of the annotation box.
I can check with my teammates if you’re looking for a more sophisticated answer than that.
Hi Anstasia.
thanks. It would be nice to have a little more inside into how the matching works with the annotation as I’m also often explaining the Wildbook functionalities to others.
Thanks for your patience! I confirmed that the difference between the match results is because redrawing the annotation you’ve changed the available data it’s working with.
When there is a clear match candidate, it should still be in the same place in the rankings or close to it. When there is no clear match, then all the candidates are likely low scores and they’re getting shuffled around in the ranking placement.
Hotspotter uses spatial verification to not only find similar patterns in images, but also match the orientation of those patterns. So when you’ve drawn an annotation correctly and it determines that no match is available, that comes with a strong degree of confidence.
In your encounter example above, with the imperfect annotation, Hotspotter thought, “Maybe this one is your salamander?” But once you redrew the annotation so that it was placed correctly, it had enough information to determine there was no match for this salamander.
Ok, this is unfortunate because it was actually a match! Of course, I matched the encounter to the individual after seeing that the match candidate was not shown anymore after starting the new match.
So I guess, the score for this match candidate was very low and close to zero after drawing the new annotation. Therefore, the match candidate was not shown anymore. Is there a way to increase the threshold for the score, so that match candidates even with very low scores are being shown? Sometimes, we get match candidates with a very low score (even 0!) shown, but in most cases, we don’t get a match candidate shown at all!
Thanks for your patience. I spent time looking into this more with my teammates today and it looks like for the old match link the annotation was deleted. That leaves a task/match attempt with no reference annotation and the original match result candidate has also had its annotation deleted.
The issue is that these were sent for ID in November but not reviewed until recently. In that time, annotations have been deleted (and potentially not re-drawn) and that means unless the match is re-run, the match page is trying to load data that’s not there anymore and not being able to display that info in a way that’s meaningful.
I know with the amount of time it takes Hotspotter to run jobs it might not always be possible to review matches right away, but we typically recommend reviewing matches within 2 weeks to avoid issues with the data changing before you’ve had a chance to review it.
Ok, that’s interesting! That means we should try and view match results as soon as possible after identification has finished (not waiting more than 2 weeks)?
We are a little bit behind with the “matching” (viewing match results) as we’re very lucky to have a dense and healthy population of fire salamanders here with regular nights of 300+ encounters. The checking for match results usually takes a lot more time than taking the photos and after a good season (spring or autumn) we’re usually 2,000 - 4,000 encounters behind. But I think we could just sequentially start the identification process, so that we will check the match result in 2 weeks or earlier after sending to identification.
What I don’t understand yet is that you say the original match result candidate also had its annotation deleted. The original match candidate is encounter “BGBI_23-5573” (see screenshot) and when I check the Audit Trail of that encounter, it just says:
" David.Kupitz on Tue Oct 31 20:02:31 UTC 2023
added this encounter to Project Botanical Garden Bielefeld Adults
"
no mentioning of removing or adding an annotation. Whereas for the second encounter of the individual, that was identified during the original identification process but was not suggested after redrawing the annotation (i.e., the encounter the match was based on) and starting a new match, the Audit trail says:
" David.Kupitz on Sun Nov 05 15:10:41 UTC 2023
added this encounter to Project Botanical Garden Bielefeld Adults
new Annotation manually added by Max.Muehlenhaupt
Max.Muehlenhaupt on Fri Jan 26 09:52:14 UTC 2024
Removed from b6d43211-9893-4cf3-a366-c09ee606d29c.
Max.Muehlenhaupt on Fri Jan 26 09:52:28 UTC 2024
Added to d9cc2a5f-89f4-4f4f-abcb-873c66810a24.
Max.Muehlenhaupt on Fri Jan 26 09:52:29 UTC 2024
Changed name to: BGBI-06410 for encounter: 872da034-d151-49ef-8128-4dd0437b47ea, which is individual: d9cc2a5f-89f4-4f4f-abcb-873c66810a24
"
(no mention, of removing the original annotation, though).
If I understand correct, with what you’re saying in your post, you basically suggest that because annotations were removed and redrawn for both encounters, the match could not be found but we only re-drew the annotation of the second encounter.
We frequently remove old (not so good) annotations and manually add new annotations that frame the fire salamander better. In some instances after running the match we get match candidates that were not provided before! In other cases, we “lose” match candidates that were provided before. Of course, when we “lose” the match candidate, we have already taken a note so that we can change the identity of the individual in the encounter manually.
Ideally, yes. We know this isn’t always possible. Additionally, the inspection images that show the Hotspotter heat maps typically get cached after a short amount of time (I think it was 2 weeks, but I’m not 100% certain on that timeline). This means that the match has to be re-run in order to see the inspection image again and that just adds more work for the researcher and on the ML side than if the matches had been reviewed sooner.
Removing an annotation doesn’t create a note in the Audit trail, but adding one does. Not all actions taken on an Encounter are recorded here, but things like adding annotation, changing the location, owner, review status, identity, and species of the animal are all logged.
I see new Annotation manually added by Max.Muehlenhaupt which would mean an annotation was added because the detector couldn’t do it automatically or that the original annotation was deleted and then manually added.
I’m not sure how comfortable you are digging into the obrowse tool on the site (there are links to it in the image preview of the encounter). It’s a development tool, though users can access it here.
If I take your old match url: https://amphibian-reptile.wildbook.org/iaResults.jsp?taskId=5468afb6-8343-461d-b0d0-3b4219273cb9&projectIdPrefix=None%20Selected
And change the URL to search for that task ID in obrowse: https://amphibian-reptile.wildbook.org/obrowse.jsp?type=Task&id=5468afb6-8343-461d-b0d0-3b4219273cb9&projectIdPrefix=None%20Selected
I can see that there are zero media assets and zero annotations associated with this match. This indicates that both the Encounter photo and annotation that match was made on were deleted.
As I mentioned in this post, having a good data management workflow will help reduce the likelihood that you’ll see these significant changes in your match results because there will be less time between when the match was run and when it gets reviewed. I’m not able to shed more light on what the old matches used to look when the annotations that those matches were based on are gone because I can’t access what’s been deleted.
Alright, I understand that Viewing Match Results should be done in a timewise manner and identification should likely be started consecutively for the bulk imports to make sure that Viewing Match Results will happen within 2 weeks after identification.
Thanks for the helpful explainations!
