Detection on AWD - only 1 annotation per media asset regardless of # of individuals

What Wildbook are you working in? ACW

What is the entire URL out of the browser, exactly where the error occurred?
Sighting examples:

Encounter examples:
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=456c741e-b1a6-4742-83d6-179cc14f18dd
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=3cf5203e-5f17-415c-b63e-ec68ee9400c6

Can you describe what the issue is you’re experiencing?
I’m noticing that the detection of multiple animals into separate encounters seems to have degraded. I did a batch of bulk imports recently and I’m finding that where there are multiple animals of the target species in a media asset, only 1 gets annotated.

Also, often, where there’s more than one animal in the media asset, I’m seeing the annotation placed around multiple individuals instead of just one.

Where there’s only 1 individual in the media asset, the detector is working GREAT - one dog, one tail.

But there are instances where I see only a tail annotated and no body and nothing else detected (see examples in sighting links above)

This I think, is impacting matching as in the following example - it detected a right viewpoint in the match image and attached that keyword, but it has included 3 individuals in the annotation - the main one having a left profile not a right:
https://africancarnivore.wildbook.org/iaResults.jsp?taskId=8cf6f1ca-62a7-4b2d-9b80-735dd7f0de70

I’m seeing a high percentage of this in the sightings that I’m reviewing. Not sure if the most recent update, the tail to body assigner, has impacted detection???

Hi, thanks for sharing this bug. This does seem related to the new assigner, specifically the assigner in conjunction with the bulk uploader. We are investigating this now and will keep you updated.

Thanks,
Drew

1 Like

Awesome, thanks Drew! I’ll keep reviewing the uploads and will let you know if I see anything else that doesn’t look right. So far, this is the only problem I’ve seen. I’ll try to run more matches too; haven’t done many of those yet.

cheers
Maureen

1 Like

Could you share the excel sheet that generated https://africancarnivore.wildbook.org/occurrence.jsp?number=BPC_OvrvwWtd_Fldr161_2020_3_24 please? Either here or via email. No need to include the images for now. Thank you!

Hi @Drew, I’ve just emailed it to you. Let me know if there’s anything else I can do or send.

thanks!
M

Received, thank you. One last question for now: are you pressing the ‘send to detection (no identification)’ or the ‘send to identification’ button at the end of your bulk upload process?

Good question - for all bulk imports, we use the ‘send to detection (no identification)’ option ALWAYS.

thanks!
M

Good morning @Drew, I have some additional findings that I think might be related to the same source bug or issue.

I’ve just looked at sighting #305: https://africancarnivore.wildbook.org/occurrence.jsp?number=BPC_OvrvwWtd_Fldr305_2019_11_10

There appear to be many duplicates that have been created - both within a single encounter record (ex’s. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=1f5248e9-6d5c-4943-858d-1be50a45737b and https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=7719ca29-c84e-4462-a25a-1a4a93c5fd07) as well as in newly generated encounters from this sighting. I’ll send the original upload file to you separately; if you’d like the original images as well, please let me know.

I also checked the list of “Annotations Duplicated in 2 or more Encounters” and it has ballooned to a total of 928.

Looking at the sightings more recently uploaded, I found examples where the # of encounters is massively above what was uploaded and unexpectedly so:

Ex1: folder #309 - 81 media assets uploaded / 721 encounters listed
Ex.2: folder #311 - 118 media assets uploaded / 590 encounters listed

From the #311 batch, this encounter has 3 extra/duplicate annotations:
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=b3a65620-8229-4989-b9de-df654dd75253

I just also ran a match from 1 annotation in 1 encounter record and got this interesting result (link to results: Wildbook for Carnivores) - multiple duplicate encounter records; i.e. exact duplicate encounter IDs listed as distinct duplicate encounters, if that makes sense:

Hope this helps!

(bookkeeping: tracking under WB-1360 and WB-1381, resume ignoring me)

1 Like

I’ve been making progress on this but am not done yet. The issue in your original comment should be resolved but I am still looking into those described in your second longer comment, so no need to do further testing until I’ve addressed that as well. Thanks for all the information and links, and you can expect another update tomorrow.

1 Like

good morning @Drew. Thanks so much for figuring out and fixing the original issue. That’s a huge relief!!! Regarding the 2nd issue, I’m wondering if it might be related to or similar to a problem @colin worked on before the holidays? - https://community.wildbook.org/t/missing-occurrence-id-in-encounter-records/438/12?u=acwadmin1
Because of that issue, I’ve been VERY careful about making sure I’m only running detection once for each import - I open each import task in a new window, click on the ‘detection only’ button then wait until I see it completed after a few minutes of refreshing the original list of imports. So I don’t believe this latest issue is caused by me accidentally running detection twice on a given batch, as Colin suggested would be the only way it could happen after he fixed the original issue.

So maybe not related to that earlier issue but I thought I’d mention it bec of the same symptom - many duplicate encounters being generated.

Hope that helps (and isn’t a red herring!!!)

thanks again!
Maureen

Thanks Maureen! That does sound super similar, whether it’s just the same symptom or the same cause I am not sure. I did wonder if you had pressed the button twice, so it’s good to hear that’s not the case. I’ll keep you updated as I deduplicate the data today, and hopefully prevent this from happening again.

Hi again @ACWadmin1,

These data issues should now be fixed and you should proceed with more using/testing of the platform! Keep your ear to the ground though, and I’ll keep this ticket open because there’s still one issue we don’t understand.

  1. The only-one annotation-per-image issue in your original comment was, I believe, just an error in how we had rerun detection+assigner on your old images. Correct me if I’m wrong, but we never saw this specific issue on new data uploaded + detected after the assigner was added. Regardless, I cleaned that data and reran it through the assigner, and looking at your Sightings above I see the annotations and encounters we’d expect. If you have old Import Tasks that are still corrupted in a unique rather than systematic way, consider if you can re-upload that data, as the compatibility between old imports and the new assigner is… complicated. But let me know if you see widespread problems still unaddressed.
  2. The second big issue, seeing duplicate annotations and duplicate encounters is the one that’s still a bit of a mystery. It is a nondeterministic issue, meaning I cannot replicate it myself by copying what you did, but I can see in the logs when the error(s) happened for you. The problem is that sometimes detection is called 2 or 3 times instead of just once when you send detection for a bulk import, though we don’t know why. This leads to 2 or 3 copies of all the annotations (and in the right scenarios, encounters) being created as a result. I have deleted these duplicates, and the links you’ve shared are now clean. There were a total of 17 import tasks where this duplication had occurred. Let us know if you see it again.

So we will be monitoring this issue. I’ve built scripts to clean up these scenarios and can clean your data more quickly if we see either again. Meanwhile we are tracking the detection-duplications and will work on preventing those in the future.

Thank you SO much for your patience and detailed notes!
-Drew

1 Like

@Drew - you just MADE MY WEEK!!! This is awesome. It seemed crazy complicated to me and I really didn’t expect a fix as quick as this. My lack of faith is NOT a reflection on you, just that technology sometimes hates me so I expect things like this to be weird and hard to resolve. Although I’m sure it wasn’t a cake walk.

We’ll keep our eyes peeled for any other signs of problems and keep you posted.

thank you again SO MUCH! And have a wonderful weekend.

best regards,
Maureen