RESOLVED: MAINTENANCE: OUTAGE: Machine Learning Server Behind Several Wildbooks/Codex 3/29-3/31

We have a RAID5 disk drive malfunction (1 of 6) on the Hydra machine learning server. This affects the backend machine learning functions on the following platforms:

  • African Carnivore Wildbook
  • SeadragonSearch
  • Sharkbook
  • MantaMatcher
  • Whiskerbook
  • Wildbook for Lynx
  • Amphibian Wildbook
  • Codex for Seals

This does not affect front-end data submission, storage, searching, or export. There has been no data loss, and this only affects detection and ID functions.

We are working on a two-part solution:

  1. Bring the RAID array back online in a degraded capacity (RAID5 allows for a single drive failure). We will be evaluating this solution today 3/30.
  2. Replace the damaged drive. This drive is being express shipped but won’t arrive until Saturday, 4/1. We will then be replacing it in the data center as soon as it arrives.

I apologize for the inconvenience. We will let you know as soon as these services are back online.

Thank you,
Jason

@ACWadmin1 @Anastasia @NeridaWilson

2 Likes

UPDATE: all machine learning functions are restored EXCEPT for Whiskerbook.org. Work is underway to restore the machine learning for that Wildbook.

Whiskerbook.org machine learning functions are back up.

1 Like