Multiple "this encounter" annotations in a single encounter record

Makes sense. Unfortunately, I’m away and out of internet reach for the whole of next week so can I suggest leaving it open for 2 weeks? I don’t expect Paul will have time to get to it much next week so we likely wouldn’t find any edge issues until after that. thanks

HA! Yea, we can leave it open two weeks :smiley:

1 Like

Okay! We are officially newly updated and released on ACW. We should no longer be making unreasonable assumptions about what goes in an single encounter. If there are two bodies in an encounter, every annotation gets split into a separate encounter.

1 Like

Thanks to everyone who worked on this. We really appreciate it. Have a great weekend!

Do we need to run everything through detection again to see this working?

I’ve been using a particular sighting to check this bug and I still see some encounter records within it that have multiple individuals boxed and showing as this encounter.

Here’s the sighting ID: https://africancarnivore.wildbook.org/occurrence.jsp?number=BPC_OvrvwWtd_Fldr3_2019_11_06
Here are a few of the encounters where I’m seeing this:

  1. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=23455a98-f394-42e8-9028-b1df028be24a
  2. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=37e7161a-6dc7-4f53-ad57-e35af1b80d2f
  3. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=474a27a5-a6fb-4edd-8f05-7b3f826aa5fe
  4. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=dae7298f-93f1-46b3-9877-d3fc9cf306a4
  5. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=9ae7ea48-9f1b-43c4-9fbf-604c02f8d3a3

Thanks
M

Hey @ACWadmin1,
Actually, this only impacts new images that are run through detection.

We are working on a clean-up script that will do the same work to existing encounters, but kind of want to make sure that new work going forward is doing what it should.
(@PaulK for reference since M is out of town this week)

Fastest way to confirm the changes would be to delete one of these encounters and re-upload it to make sure it is doing what it should.

Hi Tanya @ACWadmin1 FYI

I have some of our own data I can upload and run thru detection to test.

I shall attempt to do so today and advise outcomes.

We prefer not to delete any of the Encounters in system belonging to our researchers.

Thanks

Paul

1 Like

Hello all @ACWadmin1

I have tested the modified code with a single new bulk upload with mixed results;

https://africancarnivore.wildbook.org/occurrence.jsp?number=Selinda_Explorers_Pack_1

11 Original images resulting in 20 encounters, no issues with single dogs in foregrounds
Some problems detecting dogs in the distance?
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=99ef2141-805e-4704-89ad-f43daed3a7b1

Some of the results are really poor detection… this being the worst
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=f8a87cda-7834-4363-9541-2256075a1be4

Some of the encounters appear to be a case of tails not being kept with the same body
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=d0f5a247-f3d4-4d63-981b-fc6841046f53

And we seemed to have lost all of our keywords
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=ef6b1118-73b6-4f6b-bd16-efabf330b83b#&gid=1&pid=2

Sorry for the bad news… let me know if I can do anything else to help out the testing.

Thanks

Paul

The news is…not great, but it lets us know some things to investigate. Our next steps:

  • We have some ideas on what is happening with the keywords. That’s first thing to reclaim since we have direction
  • The detection issue is off-the-charts bad. Top priority to get it resolved. We have two ideas of where to start looking for the issues
  • The tails and body issue is a result of the detection being terrible. We are trying to keep the case of a single tail and single body being kept in the same encounter, which is normally pretty trustworthy when detection is so wildly off-base.
    I think what we’ll need to do is keep these images available for reference as we dig, then to delete and re-run them once we get a new fix in. I can copy the images and run them myself if you’re okay with me using them as test data, or we can ping you to run when we get fixes into the platform.

Thanks Tanya

Go ahead and use these images for testing - they are our own personal pics.

Good luck.

paul fyi @ACWadmin1

1 Like

Hey @PaulK,
We think we found the issue for both the labeling issue and the missed detections. If you have time, we’d appreciate a test run from you.

Hi Tanya

I can do a test tonight. Did you delete the sighting I used earlier or should I use another one ?

Thanks

Paul

Hello all

Redid the upload (after deleting all prior encounters) and definite improvement

Key words are back

No multiple “this encounter” except for tail with body
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=6ae7bb87-de68-4411-80eb-c61a8cd0405b

Still some bad detection misses
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=eeef88a7-fdb0-4ae2-b779-5ce57da0d334

One super good detection/labelling
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=8fa22068-efc0-48a7-9b6c-1dc7202fa6ab

I shall prep a completely new sighting upload and do another test - either tonight or in the AM.

Thanks

Paul fyi @ACWadmin1

One other item I did notice as I was leaving…

Although a number of new encounters have been created from the original 11 images uploaded, the actual Sighting list of Encounters has not been updated.
https://africancarnivore.wildbook.org/occurrence.jsp?number=Selinda_Explorers_Pack_1

Can/Should it be?

And, because of this, I did not delete all of the Encounters created on the 27th.

I shall go do that now and then we have a clean data set.

Thanks

Paul

Apparently I have sent too many replies… so I keep editing this one.

Totally new sighting uploaded last night

https://africancarnivore.wildbook.org/occurrence.jsp?number=MR_PK_Selinda_2017_09_29_AM

Good results overall

One multiple “This Encounter”
https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=074cec64-fa26-4137-b330-ffea46e4f8b2

Couple of weird detections where only a tail was captured for “this Encounter”

https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=de663e03-38b7-4d1b-aab8-f79a4fe4a844

https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=f4b822b6-9aaa-4e4f-9556-90dbb6ea8072

https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=17b46699-713c-414c-9024-75fef856d725

Okay, the “tail only” detections are expected for now. What’s happening there is, if there’s two bodies, we’re separating out annotations and not making assumptions. The one with multiple “This Encounter” however, is a significant problem that we need to delve into because that is definitely a problem.

Hey there @PaulK and @ACWadmin1,
We’ve deployed an update to the fix. This should address two problems you were seeing:

  1. Different tail colors were not being recognized correctly, which caused strange encounters to be grouped together.
  2. We have now added a check for part/body overlap. This will allow a greater number of tail and body assignments to be kept together appropriately.

This addresses the found-cases that came up when you were testing Paul, so we think this is really close, if not done.

Hi @tanyastere; cc @PaulK

Note: in addition to the results below, I’m waiting for another upload to complete detection - that batch has 34 encounters so a few more examples to review. I’ll submit an update to this reply with my findings from that once I have them.

Meanwhile, here are the initial re-test results:

I deleted and re-uploaded the same partial sighting (19 encounters) used by Paul in previous tests on this issue: https://africancarnivore.wildbook.org/occurrence.jsp?number=MR_PK_Selinda_2017_09_29_AM

IA - ran through detection without identification.

All in all, great results. Key findings:

  1. No multiple “this encounter” instances - yay!

  2. Weird detection instances:

  • a fairly obvious missed detection
  • annotations seem offset more than usual

Update with 2nd test upload of 34 encounters -

I’ve reviewed 30 encounters of the 84 and the results match (no pun intended) what was seen in the first batch above -

  1. NO multiple this encounters in any encounters reviewed so far - YAY!!!
  2. Some tail detections without a corresponding body annotation - ex. https://africancarnivore.wildbook.org/encounters/encounter.jsp?number=dddd8441-892c-4945-8c7c-1d350135412c
  3. Some annotations are a little off - missing heads or top of back or bum end of the animal

And then there are some really GREAT examples of GREAT detection -

I’ll continue to review the rest of this 2nd test upload but it looks like you’ve beaten this bug beast into the dust! Fingers crossed for the remaining encounters to be reviewed but it’s looking GREAT!!!

Stay tuned and THANK YOU to everyone who worked on this.

1 Like

2 posts were split to a new topic: Improve detection when object in foreground

Hi @tanyastere cc: @PaulK

Great news - I’ve done a few more uploads to verify the fixes on this issue and all look great! I’ve not found a single example of multiple this encounters in a single encounter record and everything else is as expected.

Please go ahead anytime with applying the script to fix the existing data in the system. Our researcher is taking time off next week to work on his data so the sooner the better from our standpoint re: script.

Thanks again! We’re really excited to get going again with this species.

1 Like