I and another user are seeing several different errors in various places related to Bulk Import IA processing.
My large lion batch - it’s continuing to process as of noon PT on Friday, Aug. 12. However, I’m seeing a number of this kind of error on several of the lion batch’s match results pages:
Examples of this:
Also, a user who has some wild dog bulk imports that she’s trying to re-send to ID, has reported a few different IA processing issues:
She is getting an error message once she clicks on “Re-send to identification”:
Example: Wildbook for Carnivores | Login
I think this is a user permissions issue because I’m able to re-send to ID her bulk imports but her user ID isn’t allowing her to do the same. She has OrgAdmin level access.
URL and additional details still to come from the user on this issue: She is also getting a message on her match results that says “Error. Something went wrong fetching the inspection image.” And the Inspect button has been missing on some results (I’ve asked if she was sure she was still logged in & am waiting to hear back).
Hi @jason - any news on this? We’ve seen more errors show up on a new import so I’m aware the issue isn’t resolved yet but I was hoping there’s been some progress on determining the problem and/or fix?
We have seen two crashes on that server, which is a clone of the other two in our colocated server cabinet. It is currently running normally, but we have not identified what the issue is. It may be something as subtle as a high workload heating up a server component and revealing a flaw such as a crack.
Do you have a small or medium size bulk import you could send through? Those are likely to be safe as we explore the issue.
Hi @jason - do you want a brand new upload? I can arrange one but in the meantime, we uploaded this one on Tuesday (16th) - 106 media assets, subject: cheetah, so not often more than 1 animal in each photo, so not too many extra encounters. It is the source of the 2 latest match results links with errors that I sent above, earlier today.
If you prefer a new bulk import, do you want me to notify you before I hit “commit”?
OK, that looks like it went well and did not cause any kind of crash. We will keep monitoring, but please use the system normally, and we will monitor on our side. We have addressed at least one issue (a server component allowed to automatically upgrade and restarte), which we have resolved. That still might leave a second issue, but we can only monitor for it at the moment.
Okay, thanks. We have a few people looking to upload new bulk imports and/or re-send existing through ID so I’ll let you know if I hear of any further issues.
Hi @jason - another user has reported errors and other issues with a bulk import uploaded and pushed through IA yesterday, Aug. 22: Wildbook for Carnivores | Login
She reports that she “a batch of hyena photos early yesterday and it worked perfectly. Later in the afternoon I uploaded another batch with 109 photos [which are] getting the unknown error message (e.g. “there was an error with task ae0ed77b-6b9c-4f4a-8941-da5e6daba49f: unknown error”)”.
This morning, I looked at a sampling of match results from the batch she reported as having issues and I’m seeing the same thing - either an unknown error message or “attempting to fetch results”.
I can see 404 connection errors when Wildbook tried to send those example ID jobs to WBIA. We’re investigating “Why” that happened when it did (time of day and reason for interruption), but currently WBIA is receiving requests and processing them in the queue. For now, I think it is safe to simply resend them from the bulk import, and they should get picked up again without issue.
