Thu Apr 12 17:29:24 PDT 2007 — Power maintenance. At 8:00am on Saturday, April 14th one of the San Francisco colocation facilities we use for customer termination will experience a planned power outage. A number of circuit breakers inside power distribution units located on the colocation floor have failed infrared testing and are being proactively replaced before they can cause unplanned downtime. While we have been given a 4 hour outage window, we expect the total power loss to be under 2 hours.
All of our critical equipment is dual-fed from redundant feeds, and as such we do not expect any customer impact. However, there is the possibility for cascading power failure as well as the failure of our transit and transport providers at that facility. We will have staff on-site to facilitate restoration in the event of a power failure, and our Network Operations Center will be fully manned to deal with any networking problems that arise.
We’ll keep the MOTD up to date in the event of any problems, but expect things to go smoothly.
-Nathan, the Network Operations Center and Tech Support
Update Tue Apr 17 09:44:20 PDT 2007 — The power outage was a non-event. Despite half of the colo facility going dark, Sonic’s gear remained up and fully operational as the PDU breakers were replaced. It’s a joy to see our carefully planned redundancy work as expected, and quite a thrill to watch large circuit breakers tripped on live loads! -Nathan and Matt