Bulk import stuck at last detection +

Thanks! We’ve actually been waiting for an example like this to help troubleshoot why we’d been seeing detection get stuck at the final image. I’ll let you know when this is fixed.

2 Likes

Quick update: One of the images is a duplicate and appears twice in the same import. Because they’re part of the same detection job and there are only 198 unique acmIDs but 199 images, it’s getting hung up on searching for that last unique ID when there isn’t one (the duplicate images have the same acmID).

Well that’s fascinating! I would have guessed that we’ve had duplicate images within previous bulk imports but never saw this causing the problem of not completing detection.

What do we do to fix this?

Thanks
Maureen

We’re working on some code updates from our end. I’ll post here when that’s complete.

Hi Anastasia,

Thank you for pointing that out. It is not technically a duplicate but indeed the files’ names are very similar: one is KNP_20230819_0800. (with the dot) and the other one is KNP_20230819_0800 (without the dot).
That was an error when renaming the images.

I have launched another import yesterday morning and detection has not started but let’s see if it gets stuck as well or not. Here is the link: https://africancarnivore.wildbook.org/import.jsp?taskId=f632f1bb-2e77-4617-9f0f-1cf2f9b13b92

Hi again,

Under Bulk Import Logs, I have access to all the bulk imports of Gabriele Cozzi and Laura Gigliotti under my profile. I can access all their data.

I had requested a collab with Laura regarding one image but I did not expect to have access to all her information. I did not request a collab with Gabriele. There must be an issue.

From what I understand, it’s not the file name similarity that’s the issue, it’s that WBIA recognizes pixel by pixel when an image is the same as another it’s seen before.

I just resent this import to detection.

Collaborations work by giving another user access to all of your data. The only degree of restriction we offer is whether you give them permission to edit or only view that data. It’s not possible to set a collaboration permission for a single image. If you change your mind about a collaboration later, you can always revoke a user’s permissions from the My Account page.

Wildbook doesn’t let me see which users have collaborations with each other or when the request was initiated, so I don’t have any insight for how it happened if you don’t remember accepting or initiating it. If you don’t want the collaboration in place anymore, you can revoke Gabriele’s permission from the My Account page.

1 Like

Hi @Anastasia, @jason & @Marine_Ingwe - I thought we’d fixed the bulk import access issue that I believe Marine is referring to, a while back but it looks like I’m mistaken. Here’s a link to the ticket I submitted on the topic last year, with a follow up in April this year: Access to other users' bulk imports

@anastasia & @jason - I originally posted this as an issue/bug because if there’s no collaboration between users, there should be no access to each other’s bulk imports. But it got converted to a feature request due to some follow up questions from Jason.

But I still feel it’s a security issue that needs to be fixed urgently, not a feature request.

What I believe Marine is referring to can be reproduced as follows:

  1. Log in using a researcher-only level account
  2. Go to the Bulk import logs
  3. Click on the bulk import of a user that you do not have a collaboration with
    The result is that you can access the detailed bulk import page & you can open all of the match results pages.
    I tested this and it works - let me know and I can share via email the user IDs I used for this test. When I try to confirm a match or open an encounter record or sighting page related to the other user’s bulk import and/or match results page, I get the access is unauthorized -type message. So I don’t have full access to the other user’s data but I have access I don’t believe a researcher-level user should have - I can open and review their bulk import details & related match results pages.

This problem is aggravated by something I think is fairly new because I haven’t noticed it before today.

Previously, the bulk import logs were separated into 2 different screens:

  1. My bulk imports
  2. Other users’ bulk imports - accessible by clicking on a link at the top of the page for (1) above

Today, I see that all bulk imports are combined on the same landing page when you click to go to Bulk import logs. Which makes other users’ bulk imports more immediately available than previously. I’m guessing this is why Marine discovered that she can access other user’s bulk imports (users with whom she does not have a collaboration) when she hadn’t noticed previously.

Other users will find this quickly as well and so I suspect we’ll start to hear from others shortly, reporting the same issue.

All the users I know will not want to see that users they don’t have collaborations with can access their bulk imports, even if they can’t edit it, etc.

So a fix would be very much appreciated.

thanks
Maureen

cc: @PaulK

1 Like

I tested this with a researcher account and though I could see a list of imports, I got an Access Denied message when I tried to click through to any of them. Then when I added an org to the researcher test account and tried to open the import of someone else in the same org, it let me click through to it and view match results.

We’re debugging this now and I’ve delisted this post so it’s not currently visible to other forum users while we address it.

Edited to add: This actually was only reproducible when I had the orgAdmin role on the researcher account. Once I removed it, I couldn’t see anyone’s imports, even within my assigned org.

Can you email me those IDs so we can keep testing?

Quick update to share that there are about 20 detection jobs ahead of this. It’ll be a little while longer due to the backlog.

I’ve confirmed the fix was put in place last night and that the bulk import looks to have completed detection and is queued for identification. I’m going to mark this post as resolved since the original issue has now been addressed.

@ACWadmin1 @PaulK We can follow up on the bulk import security testing via email.