I donât think this is related to the bulk import spreadsheet used to do the upload since the OccID is showing up in the search results, so Iâm not sending the spreadsheet for now. Please let me know if you need it.
Since the import was partially successful I will need to inspect the excel sheet to see any differences between rows that were assigned to the sighting and those that werenât, or at least try to replicate this behavior.
Please send along the excel sheet when you have a chance. Thanks!
It appears that the 12 encounters associated with the import are the ones referenced in the excel file, and the ones you see not connected to the sighting are sibling encounters generated from images with multiple animals.
I have created issue ticket WB-1223 to find a solution. We donât usually see this happen, it is a unique case to investigate.
I recommend adding the sighting ID manually to these in the encounter page âIdentityâ menu since there are only a couple, and we will provide an update when this is fully diagnosed and resolved.
Hi @colin, Iâve actually been noticing quite a few examples of this as Iâve been going through our list of âAnnotations Duplicated in Two or More Encountersâ. I can dig up and send other examples later today if that helps.
Hi @colin
Here are some other examples - all say âcloneWithoutAnnotationsâ in the Audit trail section, which I assume means itâs an annotation that was generated from images with multiple animals?
@colin - I thought this example might also be a useful example - I found this while working through the âAnnotations Duplicated in 2 or more encountersâ list. The first encounter of the 2 duplicates (#1 below) has the correct Occurrence ID and the other (#2 below) has none assigned. Encounter #2 below also shows as âcloneWithoutAnnotationsâ in the audit trail.
There is now a fix deployed that should remedy this behavior. Iâve tested the new code with truncated copies of your imports where the behavior was observed, and seen successful occurrence assignment in each one.
Please confirm that this the solves the problem for you with future imports or re-runs of these.
Hi @colin, glad to have a fix for future uploads, thanks. I have a few follow up questions:
Does this mean you canât fix the encounters in the system that currently do not have OccIDs assigned where they should have done?
Are these encounters with missing OccIDs actually somehow still connected to the sighting it was originally connected to in the upload, even if the OccID is missing in the record? In other words, is this just a display issue or are these encounters actually disconnected from their original sighting until we find them and re-enter the OccID?
If I go to a sighting that I know has encounters with this issue and open each encounter, will I find the ones that are missing the OccID? If not, how do I find the encounters with this issue and fix them all?
In the last pair of examples I sent above, can you explain why this is happening? Iâve seen a lot of these examples in the duplicates list. That is, a duplicate annotation, that is missing its occurrence ID, that has the same Import task ID as its duplicate with an occurrence ID, and the same file name.
But the import did not have 2 copies of the same file so it appears that the duplicate was created by the system (âcloneWithoutAnnotationsâ). Is this a separate bug?
Creating a new scripted way to repair these encounters would at this point be slower.
The interface for assigning the occurrence ID to encounters where it is missing, or removing and
re-running an import is a shortcut to the actions that would need to be taken and has already been
tested.
Only through the encounter. All the encounters referenced are connected to the appropriate sighting, it is only the extra ones generated from multi animal images that are not connected. Itâs not a display issue, they were not assigned the sighting ID.
Yes. You could visit each encounter in the sighting with a multi animal image, visit the cloned encounters through the blue arrows on the annotation boxes and assign ID. This would reach all that were affected. On many sightings this is likely this quickest method. On a larger import removing and resubmitting may be faster.
Iâll poke around a little more at those two, they appear to be a bit different. The duplicate encounters were created two days after the original from the import.
Re: #2 - the encounters that are missing OccIDs must be linked to the sighting somehow because in the first set of examples I sent above, I got to these encounters by using the sighting ID as the search filter and opened them from the resulting list. So the sighting ID is linked to these encounters somehow, otherwise how would it know to display it in the search results?
The outstanding concern I have then is, how do I find all of the instances where this has happened?
Do you know when the problem started? We have thousands of encounters and hundreds of sightings in the system, most of which were uploaded in the past 3 months. Was the bug present from the start of our use of ACW? Or was it introduced in a later release? Knowing that would at least help me know where to start looking.
Also, does this only affect one species or is it across all species in ACW? If itâs all species then for at least one, deleting and re-importing isnât an option due to the amount of matching work already done.
Lastly, re:#4 - I have many other examples in the duplicates list if you need more. I deleted a bunch of them already, before I noticed the pattern. Let me know if you want me to send additional examples.
First of all, for the encounters referenced in bullet point #4 there is now a fix deployed to prevent it happening again. This behavior would only be triggered when something was accidentally sent to detection twice. This was something that emerged on Oct 19 and was fixed today.
When I search for these sightings by name I only see the one sighting created by the import. Visiting that sighting shows a list of the entries on the import sheet, but not any cloned encounters generated by detection. If you have an example search result that shows otherwise you could link me so I can take a look.
The other behavior- cloned encounters not receiving the occurrence ID from the original encounter would effect all multi-animal images from an import where an occurrence ID is specified.
Since this original issue appears to have a broader effect than we thought we are evaluating a scripting solution.
Hi @colin,
Firstly, a script would be VERY welcome! I have no idea how we would otherwise ensure weâve fixed all the instances and keeping all encounters from a sighting linked together is extremely important to us, not least because it helps capture both left and right profiles for individuals.
Next, we are aware of the issue where the Sighting record does not include cloned encounters and I believe Paul reported that bug awhile ago. So we donât use the Sighting record to find all of the encounters in a sighting. Steps below are what we follow and how I found the OccID displaying in search results that link to encounters where the OccID is missing:
Select âView Unapproved Encountersâ from top menu
Apply filter âMR_PK_Duba_2019_11_20_PMâ
Open each of the 26 encounter records - youâll find the encounters that have their OccID field showing as ânone assignedâ even though the link from the search results clearly showed the correct OccID
Iâve just run a script to add all the âsiblingâ encounters generated from detection to the occurrence referenced in the import sheet for an encounter and image. This should repair all instances
of this disconnect since detection on ACW was enabled. Iâve checked the encounters from your original post and several others and can see the additions to the sighting record.
This doesnât effect the doubled encounters from #4 above, those hopefully are minimal since the bug was short lived and only occurred on when an import was sent to detection twice. There should be little effect on the system from these and they can be deleted when found.
Hi @colin, thatâs terrific, thanks so much! It would have been a nightmare to try to find these and clean them up manually so you saved us a huge amount of work.
Itâs unfortunate about the dupes not getting fixed; there are, I believe, about 200-300, based on what Iâve been finding in the duplicates list in the Data Integrity tool. But Iâve already fixed a few hundred of those anyway so Iâll just keep plowing through them.
Have a wonderful New Yearâs and a vaccine-filled, healthy 2021 to us all!!!
