published 02/06-26 by freddo
Earlier today one of the Ceph monitor nodes was brought down for maintenance. Unfortunately, despite having four others, it failed to do a graceful failover causing some CephFS dependent services to lock up, amongst them our databases.
The issue should be getting resolved now but may remain unstable for a while longer.