Bulk import with IDs creates multiple individual records with the same ID

What Wildbook are you working in? ACW

What is the entire URL out of the browser, exactly where the error occurred?

Can you describe what the issue is you’re experiencing?
A user uploaded 6 bulk imports that contained both ID’d and un-ID’d images. Her process was as follows: “I uploaded them one right after the other, and once they were all uploaded I sent them to detection one after the other - not all at once to avoid completely clogging up the system but definitely with some overlap.”
The affected IDs are LL_TZ_SN_leop_0001 to LL_TZ_SN_leop_0013 (13 individuals).

This appears to be similar to an issue reported here by a Flukebook user: Questions about matching - #7 by CMKonrad

I’ve just sent the 6 spreadsheets to services@wildme.org

If it helps, the user’s platform is:
Windows 11 Home 22621.1265 and Chrome Version 110.0.5481.104

cheers
Maureen

Thanks for flagging this as a wider-spread issue, Maureen. I have this on our bug-report to-do list already, but I’m updating it to reflect that it’s not limited to Flukebook.

1 Like

This was found to be a pretty widespread issue in ACW and we’re actively working on a resolution. I’ll let you know when there’s an update on a fix.

1 Like

Also posting this here:

I recently learned we can force a merge between individuals. I’ve added instructions on how to do it in our new Merge FAQ.

1 Like

Hi Maureen,

I’m updating this to include the link to the ticket where we’re tracking this. I know there are multiple posts that report the same issue, so I apologize in advance for the spam.

1 Like

Hi @ACWadmin1

While we’re looking for the root of this (and it may be a thorny security/visibility check is needed), we do have the power to fix duplication of individuals. If you know of blocks of bulk imports or blocks of search results (e.g., send us the links), I can merge individuals with the same name via script in bulk.

1 Like

Hi @jason - thanks for the update. The case mentioned above involves the following 6 bulk imports:

The duplicated IDs are:
LL_TZ_SN_leop_0001
LL_TZ_SN_leop_0002
LL_TZ_SN_leop_0003
LL_TZ_SN_leop_0004
LL_TZ_SN_leop_0005
LL_TZ_SN_leop_0006
LL_TZ_SN_leop_0007
LL_TZ_SN_leop_0008
LL_TZ_SN_leop_0009
LL_TZ_SN_leop_0010
LL_TZ_SN_leop_0011
LL_TZ_SN_leop_0012
LL_TZ_SN_leop_0013

I’ll check with other users to find out if anyone else has had this issue. I don’t think there are a lot of ID’d bulk imports done recently but I’ll have a look.

Meanwhile, I know that a few users on Whiskerbook is preparing a number of ID’d bulk imports. Should I tell them to hold off until this is fixed?

thanks again.
Maureen

Here are a couple of other examples that I’ve tracked down:
LP-BW-05M:

https://africancarnivore.wildbook.org/individuals.jsp?id=b052d5eb-91a9-470f-bdcf-bd67edd7f68f

Ke_SpH024_OJ:

Thanks!
Maureen

Hi @ACWadmin1

These bulk imports seems internally consistent on ID. To de-duplicate, I need to understand the superset of 1) these imports plus 2) what they duplicated (allowing the pre-existing and bulk imported IDs to get merged). Is there an Encounter Search that defines the superset of this catalog, such as filtering by a specific user or species or project? Once I have that, I can have a script merge duplicate IDs.

Thanks,
Jason

Hi there,

I’ve just had a WKB user report an issue with duplicate IDs created after the IDs were assigned in the system - the data was uploaded as un-ID’d. I’ve reported it here: WKB user reports duplicate IDs

It reminded me of this report and the fact that it’s still not resolved.

I’m not clear on what else is needed to create the script to de-duplicate /merge the IDs listed above; I’m not sure that these were all linked to a bulk import or if these happened in some other way, like the Whiskerbook example.

So I’ve created search criteria for each of the duplicate ID’s listed above. Hopefully this will enable @jason to merge the records and create a single instance of each Marked Individual page:

Ke_SpH024_OJ: https://africancarnivore.wildbook.org/individualSearchResults.jsp?noDate=noDate&approved=acceptedEncounters&unapproved=allEncounters&unidentifiable=allEncounters&male=male&female=female&unknown=unknown&alive=alive&dead=dead&biomeasurement13C(operator)=gteq&biomeasurement15N(operator)=gteq&biomeasurement34S(operator)=gteq&individualID=Ke_SpH024_OJ&resightGapOperator=greater&organizationId=None&projectId=None&submitSearch=Search

LL_TZ_SN_leop_0001: https://africancarnivore.wildbook.org/individualSearchResults.jsp?noDate=noDate&approved=acceptedEncounters&unapproved=allEncounters&unidentifiable=allEncounters&male=male&female=female&unknown=unknown&alive=alive&dead=dead&biomeasurement13C(operator)=gteq&biomeasurement15N(operator)=gteq&biomeasurement34S(operator)=gteq&individualID=LL_TZ_SN_leop_0001&resightGapOperator=greater&organizationId=None&projectId=None&submitSearch=Search

LL_TZ_SN_leop_0002: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0003: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0004: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0005: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0006: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0007: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0009: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0010: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0011: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0012: Wildbook for Carnivores | Login

LL_TZ_SN_leop_0013: Wildbook for Carnivores | Login

It would be great to finally get these cleaned up. Hopefully the above is sufficient to make that possible but if not, please let me know what else is needed.

thanks!
Maureen

1 Like

@ACWadmin1 Thanks for your patience. The duplicate individuals have been merged.

1 Like