Hello,
There will be some very limited downtime for machine learning (ML) requests tomorrow morning (9AM Pacific on August 9th, 2022) on African Carnivore Wildbook.
The backend Wildbook IA (WBIA) system that supports all detection, classification, and ID job requests will be migrated to a larger and more powerful machine. The entire WBIA database and its complementary caches have already been fully transferred, but we anticipate approximately 30 minutes of additional reconfiguration and final verification to transfer the service and restart the main public site fully. All systems may experience restarts during this process. If you cannot visit the site or execute ML jobs, please try again momentarily.
The new machine has 64 compute cores, 256 GB of system memory, and will share that machine with a handful of (but much smaller) projects. This will constitute a major upgrade in ML response times – to reflect the project’s growth (and anticipated growth) – and is done with no additional cost to ACW. This change will also stabilize the performance of the Flukebook project since it currently shares resources with ACW.
We apologize for the short notice of this migration. We need to take somewhat emergency action to stabilize one of our busiest machine learning machines, and ACW is the best candidate for migration to our largest and fastest machine.
Time offline: 08/09/2022 at 09:00 Pacific
Time complete: 08/09/2022 at 09:30 Pacific
Actual time offline: 08/09/2022 at 08:57 Pacific
Estimated time complete: 08/09/2022 at 14:30 Pacific
Actual time complete: 08/09/2022 at 14:18 Pacific
Thanks!
Jason Parham
Senior Computer Vision Research Engineer
Wild Me
CC @ACWadmin1