What is the entire URL out of the browser, exactly where the error occurred?
There have been a number of bulk imports sent to detection in the past 12+hours that are not progressing at all. When I look at the bulk imports sent through ID since the 16th, they’ve all pretty much completed except for the few exceptions and the single errored jobs I reported here.
Is there something holding up the detection queue?
Here are the links to some of the many bulk imports that haven’t made any progress, the first one having been kicked off @ ~7am PT Nov 22:
In fact, I count 25 bulk imports uploaded on the 22nd that were sent to detection and none have completed a single detection.
I feel like there’s maybe an ID job that’s failed or is a problem that’s causing the detection queue to not move?
Any insights and assistance would be much appreciated.
thanks
Maureen
I can’t figure out why there hasn’t been any progress on any of the bulk imports sent to detection today.
There’s definitely something wrong with ACW’s detection queue. Here are the latest victims - these 2 bulk imports were sent to detection on Thurs, Nov 23 before 9am PT (so almost 30hrs ago as I write this):
These 2 larger bulk imports were sent to identificationafter the above 2 bulk imports were sent to detection:
These 2 bulk imports have both completed identification while the 2 imports above are still showing zero completed detections.
If it’s the case that certain HotSpotter jobs are interrupting or blocking detection jobs, @PaulK and I would prefer to give detection jobs their own lane, separate from any and all other ID jobs and lanes. The other option, of removing these certain HotSpotter jobs from the lane/queue that the detection jobs also run on, isn’t ideal since we have users who are in the field and need to be able to quickly run a single encounter or two through both MiewID and HotSpotter to find out as quickly as possible who it is - this is often because it’s an animal with a snare on it that needs to be found urgently to have the snare removed. So removing the ability for users to be able to kick off individual encounters against HS & Miew isn’t a great option.
So if we could segregate detection jobs into their own dedicated queue so that they no longer get impacted by one-off HotSpotter jobs, that would be our preference.
Thanks in advance for your assistance with this. As always, it’s very much appreciated.
These detection jobs are sitting behind slow, large HotSpotter jobs running in the fast lane that detections run in. Might be a good case for not allowing Hotspotter jobs into the fastlane. There are only 9 jobs left in the fast lane (including the detections), so those should be completing soon. I can see all worker threads are alive and working.
Overall, this might reinforce the idea of a ban on Hotspotter in the fastlane.
Hi @jason - I understand why you are suggesting banning HS jobs from the fast lane but that removes the ID for urgent reasons capability, since HS can find matches that MiewID doesn’t and vice versa. So to get the best chance of getting an ID, users with an urgent need to ID a, for example, snared animal, they want to run both HS & Miew on a single encounter or 2 (one for each side viewpoint).
So we’re reluctant to remove that capability from the system. Would it not be possible to give detection jobs their own lane?
It’s a good idea and something we could look at in the new year. It may or may not be a large change, but because it affects the critical queuing system, it would take a lot of time to test, so we would do it slowly and carefully.
Okay, thanks @jason. Then for now, if you could pull HotSpotter jobs out of the fast lane, that will be much appreciated.
We’ll communicate with our users about the change and monitor the impact. We can circle back with you in the new year on how it’s going and if we want to pursue the proposed change above.