In which Wildbook did the issue occur? Whiskerbook
What operating system were you using? Mac OS 11.5.2
What web browser were you using? Safari
What is your role on the site? researcher
What happened? When doing matching with hotspotter, I’m seeing two issues happening.
It looks like there are a few instances of comparisons being done between the wrong orientations of the individuals in the images, i.e. right side compared to left side. Example at link below
I’m seeing instances of the background being matched in some of the images. A few examples below; the second link also shows another instance of comparing wrong orientations.
TLDR; The viewpoint predictor made a few bad predictions (e.g., back instead of right), causing a wider or different set of annotations to be considered. In the short term, you can manually remove and redraw the annotations, setting the correct viewpoint in the process. If this happens frequently, we may in the future need to retrain the leopard detector with Jhalana Reserve Forest data too. Also recommended: pre-rotate leopard photos to the horizontal to allow for Hotspotter matching (we don’t have an orientation-correcting ML network for leopards (yet), though we can generate one in the future.)
Here are links to the corresponding Encounters and the machine learning viewpoint predictions:
In essence, it looks like the ML predicted “back” for 2/3 cases, which causes a different query against database annotations. Here’s what a “back” viewpoint matching query looks like:
So what we consider to match against does vary by the viewpoint predicted for an annotation. The short-term fix is to review your data for annotations with bad viewpoint predictions and manually correct them. Here is a docs page about that process:
and this link to your data may be the fastest way to roll over and check those viewpoints:
In the longer-term, if we see a lot of these, retraining the African leopard detector to include this subspecies data may improve the detector all around with additional, more diverse data.
Also quick note: neither the current detector nor Hotspotter autorotate images for comparison, the first example above would likely not yield a match in Hotspotter even if “right” were predicted as the viewpoint. I recommend manually rotating and reuploading the image to the Encounter. We do have an orientation network that would be a great addition to this detector, but it wasn’t around when we did the first training of this detector. I’m hoping in the future we can have the system autorotate images.
