ACW - AWD matching taking an enormous amount of time per match job from bulk imports

In which Wildbook did the issue occur? ACW

What operating system were you using? (eg. MacOS 10.15.3) WIN 11

What web browser were you using? (eg. Chrome 79) Chrome latest

What is your role on the site? (admin, researcher, etc) ADMIN

What happened? I have been trying to rerun a number of EWT bulk import jobs to refresh matching.

Individual bulk imports (see below - 183 and 344 matcheable encounters each) are taking many days or even weeks to complete processing.

Also, when accessing Match Results for encounters that do complete - often results will not present or take minutes to show up.

I have noticed that when the system finds no match encounters that message shows up immediately.

What did you expect to happen? Bulk import matching to be completed in a reasonable amount of time. Overnight or within a day or two at most

What are some steps we could take to reproduce the issue? Check the results and turnaround times for jobs flowing from ACW to WBIA lately - something is wrong.



Appreciate your input on this.
Thanks

Paul

If this is a bulk import report, send the spreadsheet to services@wildme.org with the email subject line matching your bug report

I’m looking into the wider issue of why ACW is slower than usual. I’ll follow up as soon as I have more info.

Looks like ACW/WBIA ID queue(s) are down.

Not getting any jobs going through?

Thanks

Paul

@ACWadmin1
@jason

WBIA is a bit slower than usual while we work on addressing what’s causing it.

The results page has a dependency on WBIA where it’s looking for those rendered match images and has to check WBIA if they’re available. So if WBIA is slow, it can slow down the match results page, even for old matches.

Thanks for sending the spreadsheet! I didn’t see anything unusual with it.

Jason mentioned that the dashboards aren’t reporting accurately and he won’t be able to fix the metrics reporting on it until at least next week. I don’t know which metrics specifically aren’t reporting accurately, just broadly that the dashboards aren’t accurately modeling what they should.

That said, I resumed stalled tasks on these imports, which is a little different from re-sending the entire thing to ID again. If the status is queued and it’s stuck, it should force just the stuck ones back in line while leaving the ones that are behaving correctly alone. Let me know if these still aren’t cooperating by tomorrow.

Thank you Anastasia

I will let you know.

Regards

Paul

@ACWadmin1

1 Like