Hello,
There will be some very limited downtime for machine learning (ML) requests tomorrow afternoon (4PM Pacific on January 26th, 2021) on African Carnivore Wildbook.
The backend Wildbook IA (WBIA) system that supports all detection, classification, and ID job requests will be migrated to a larger and more powerful machine. The entire WBIA database and its complementary caches (660 GB) have already been fully transferred, but we anticipate approximately 30 minutes of additional reconfiguration and final verification to fully transfer the service and restart the main public site. All systems may experience restarts during this process. If you are unable to visit the site or execute ML jobs, please try again momentarily.
To provide a comparison, the backend machine supporting all current ACW ML requests has 6 compute cores, 1 GPU with 12GB of ML memory, 128 GB of system memory, and shares that machine with 9 other projects. The new machine has 20 compute cores, 3 GPUs with 48 GB of total ML memory, 256 GB of system memory, and will share that machine with 4 (but larger) projects. This will constitute a major upgrade in ML response times – to reflect the growth (and anticipated growth) of the project – and is done with no additional service cost to ACW.
Time offline: 01/26/2021 at 16:00 Pacific
Time complete: 01/26/2021 at 16:30 Pacific
New Time complete: 01/26/2021 at 17:00 Pacific
Time Completed: 01/26/2021 at 16:38 Pacific
Thanks!
Jason Parham
Senior Computer Vision Research Engineer
Wild Me
CC @ACWadmin1