ARW - High frequency of no matches provided and no matches found despite known recaptures in a large dataset of adult fire salamanders

Support
, ,
1

What Wildbook are you working in?
Amphibian And Reptile Wildbook

What is the entire URL out of the browser, exactly where the error occurred?

Can you describe what the issue is you’re experiencing?

Hi,
I have a large dataset of 1499 fire salamander photos with ~30 known recaptures from 2007 to 2010 and another ~400 photos of fire salamanders from 2018-2022 in the project “Uni Bielefeld - Adults Kottenforst”. We want to find out if fire salamanders captured in 2007 to 2010 were also encountered in 2018-2022.
First, I uploaded the 2007-2010 pictures. I have not marked the known recaptures in the dataset when running the bulk import (bulk import sheet attached to eMail). First, I checked if the known recaptures could be detected by the software. Unfortunately, most recaptures were not detected by the software. For example, 172 (Amphibian and Reptile Wildbook) is a known recapture of 81 (Amphibian and Reptile Wildbook) but was not detected as a match. Instead “Image Analysis has returned and no match was found.” for both encounters. Same for 526 (Amphibian and Reptile Wildbook) that is a recapture of 91 (Amphibian and Reptile Wildbook). In this case, there are actually match candidates suggested (https://amphibian.hydra.dyn.wildme.io/api/query/graph/match/thumb/?extern_reference=jqnhsoognunqbemo&query_annot_uuid=f0deb717-633b-4caa-8f48-705a52b55ddf&database_annot_uuid=4b863013-4d37-4afd-bb52-bdcdea632a7d&version=heatmask) but only one each. In many cases, no match is suggested at all.
In some cases, where an individual was recaptured two or more times (three or more encounters with the same individual), a match could be found but not for all encounters. For example, 17, 110, 716 and 921 are all encounters of the same individual but 110 didn’t match with any (Amphibian and Reptile Wildbook - no match found) but 17 and 716 matched with 921 (Amphibian and Reptile Wildbook).
So, clearly the software is having a hard time to find matches. I noticed that sometimes, either the numbers on the tags in the photos (e.g., https://amphibian.hydra.dyn.wildme.io/api/query/graph/match/thumb/?extern_reference=jqnhsoognunqbemo&query_annot_uuid=642a4ab2-dcc9-4385-aab0-e8f540db5535&database_annot_uuid=845919ea-4e03-4154-91a6-79218cfa342f&version=heatmask) or the grid paper background in the photos is suggested as matching area (e.g., https://amphibian.hydra.dyn.wildme.io/api/query/graph/match/thumb/?extern_reference=jqnhsoognunqbemo&query_annot_uuid=588d5c0d-79ef-455d-9bb4-29d945596a83&database_annot_uuid=897def2f-590c-477c-b83b-232194689d0c&version=heatmask). However, this only happens a couple of times. Most often, I get the message “Image Analysis has returned and no match was found.”.

Next, I uploaded another 150 photos from 2022 to compare them with other photos from 2007-2010 and 2018-2021 (bulk import excel sheet attached to eMail). In almost all cases, I get the message “Image Analysis has returned and no match was found.” (e.g., Amphibian and Reptile Wildbook).

So, I wonder how I can increase the chances of finding matches. I used “KF” as locationID for all photos in the project and all imported photos are matched against KF only. Still, I get a lot of encounters without any matches suggested which is weird considering that my colleagues working in ARW usually get suggested matches for all their encounters. I could of course, cut out the fire salamander individual in the photos, and therefore remove the grid paper in the background. However, I’m not sure if that’s the main problem.

Can you provide steps on how to reproduce what you’re experiencing?
see above

If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report
Done.

2

Hi @Max.Muehlenhaupt

Spot-checking the match results, I can see that the system is trying to match against a robust set of annotations. See number circled in red below, so your bulk import went through successfully and matching was attempted. I see the expected annotations in the match-against list, so that also is good. Overall, HotSpotter is running correctly and simply not making the match.

Things that could affect this:

  • pose changes in the animal (spread out/long) versus curled

  • poor rotation predictions by the ML detector

  • changes in the animal overall

One thing could try is turning “spatial verification” off for HotSpotter (it removes false positives by making sure matched pattern areas are aligned spatially with each other and not random on the body). It would increase false positives, but it may be what is needed to get correct matches to begin appearing. In this case, where there a lot of instances of nothing matching, it may be worth trying. Want me to turn it off?

Thank you,
Jason

3

Hi @jason ,
I see. In the future, I will try to get pictures of the animals as straight as possible.

Yes, can you please try turning off spatial verification? Will I need to resend the pictures to identification again after you turned off spatial verification? False positives might not be such a big problem since I’m checking all possible matches anyways but if there’s too many false positives, they might “push out” the true positive from the list of possible matches.

Do you think light conditions might affect match results? Since fire salamanders emerge from their hides only after dark in our study area, we are alsways using flashlights when taking pictures. However, the “spotlight” could be on the animals body or next to it which might affect the shading and colors of the picture.

Cheers,
Max

4

Hi @Max.Muehlenhaupt

I gave reissued all Encounters from this bulk import to ID:

I am seeing many more results appearing, but I don’t know their accuracy. Please review them as they complete (still many in the queue right now) and let me know if you’re seeing matches appear.

Thanks,
Jason

2 Likes
5

Hi @jason ,

Thank you. Could it be that you turned of “spatial verficiation” only for the encounters from my second, smaller bulk import (encounters from 2022 March-May; bulk import log: Amphibian and Reptile Wildbook | Login)? I receive 12 matching candidates per encounter now. I was also able to find a previously undiscovered recapture, so it might be working better!

However, for my larger bulk import (encounters from 2007-2010; bulk import log: Amphibian and Reptile Wildbook | Login) not much has really changed. I’m still receiving only very few candidates per match result and known recaptures are not detected by the software.
Could you please turn off spatial verification within this dataset as well? This way, I could test for recaptures between the dataset 2007-2010 and datasets 2019-2021 by having more possible candidates despite spatial verification turned off and thus, having more false positives. Also, is there a way to force matches? It might be helpful to mark the known recaptures within the 2007-2010 data set.

Cheers,
Max

6

Spatial verification is off for adult fire salamanders across the Wildbook. However it only applies to new match results. It does not convert previously run matching tasks.

On the larger bulk import page, do you see an option to resend the bulk import to ID? If so, you can rerun matching there. If you don’t see that button, I can resend them. It will occupy the matching queue for a long time, so other users in your lab may experience significant delays.

Thanks
Jason

7

Alright, I have resent encounters to identification against “KF”. Thanks!

1 Like
8

Hi @jason ,
turning off the spatial verification made many more matching candidates appear under mtach results, especially for that 2022 dataset I uploaded (Amphibian and Reptile Wildbook | Login). As a result, more matches were detected! Also for the 2007-2010 dataset (https://amphibian-reptile.wildbook.org/import.jsp?taskId=800ca2ec-fc56-4f43-926f-cb0d4e1fbf9e), matches of known recaptures were now found that were not found before.
However, for the 2007-2010 dataset, known recaptures are still sometimes not detected via the software. For example, one individual was captured 4 times (as 17; 110; 716 and 921). 17, 716, 921 were merged (Amphibian and Reptile Wildbook) but 110 was not detected as a match (https://amphibian-reptile.wildbook.org/iaResults.jsp?taskId=5e8de77f-69f3-4130-bb5f-00bb100fd265&projectIdPrefix=KF%20Adult%20re). 73 was merged with 163 (Amphibian and Reptile Wildbook) but did not detect 374 (Amphibian and Reptile Wildbook) as a match.
As a last example, 526 (Amphibian and Reptile Wildbook) that is a known recapture of 91 was also not detected as a match (https://amphibian-reptile.wildbook.org/iaResults.jsp?taskId=224bebd7-b196-4816-8eab-ed8edce6cd60&projectIdPrefix=KF%20Adult%20re). There are some more example of not-detected recaptures.
It doesn’t seem like the grid as an underground is the problem, as cutting the background out of a fire salamander photo and uploading the photo again, did not improve match results. Do you think, we can somehow improve the accuracy of the matcher even more? Of course, I could force-match (how exactly?) the encounters of known recaptures. However, for future (unknown) recaptures, this is of course not possible.

9

Hi @Max.Muehlenhaupt

We have likely maxed out HotSpotter for matchability, though it has some amount of additional tuning we can explore in the future. Switching to the deep learning-based PIE algorithm is also a possibility, which generally has higher accuracy but requires 3+ photos per viewpoint of at least 200+ inidividuals accurately identified. That would be a new project and require new funding to train. We’re happy to discuss it in a meeting.

Of course, I could force-match (how exactly?) the encounters of known recaptures.

Just set the ID on the encounter’s page. You don’t have to have a match result from HotSpotter.

image
image2367×1539 295 KB

Thanks,
Jason

1 Like
10

Alright, Thanks Jason!

1 Like