We discovered under heavy load that the machine learning server hosting the GPUs for Flukebook.org, GiraffeSpotter.org, and the Internet of Turtles did not have the right CUDA drivers, and as a result ML tasks were taking much longer than normal due to not properly engaging the GPUs.
We are in the process of upgrading the server drivers and will restart them as soon as possible (and with the GPUs properly engaged).
Update: The machine learning server behind Flukebook, Internet of Turtles, and GiraffeSpotter is back online and using its GPU! We’re excited to see it processing detection jobs more quickly.
One final piece needs to be addressed: the finFindR algorithm for dorsal fin matching needs upgrading for compatibility with the new CUDA drivers. We do not have a timeline on this fix. The current workaround is to use the PIE v2 and CurvRank algorithms for dorsal fin matching for affected species (largely bottlenose dolphins and Pacific white-sided dolphins).