Fri Jan 18 01:09:33 PST 2008 — Santa Rosa Transport Failure. Earlier today one of our two redundant GigaMAN transport links out of our Santa Rosa datacenter flapped. We took the link out of routing and began troubleshooting the problem with AT&T. Customers saw no disruption as all our Santa Rosa traffic took advantage of our redundant GigaMAN link to go around our Bay Area ring. Then, this evening, our colocation provider in San Jose began emergency maintenance on their cabling plant. This emergency maintenance took down our redundant Santa Rosa GigaMAN link, effectively severing our Santa Rosa datacenter from the rest of our network and the Internet. We are currently working with both AT&T and our colocation provider to restore service as quickly as possible. This outage affects all of our colocation, web hosting, and WBA customers, as well as T1 customers in the Santa Rosa area. All access to Sonic.net email services are affected as well. All of our dial-up, DSL, Business-T and T1 customers in the San Francisco area have complete access to the Internet, thanks to our redundant DNS and authentication server clusters in our San Francisco POP. We will continue to provide updates as the situation develops. -Jared and Nathan
Update Fri Jan 18 01:46:19 PST 2008 — Our San Jose colocation provider has completed their emergency maintenance, thus restoring one of our GigaMAN connections. All services and customers are back online at this time and we do not foresee any further problems. Sonic.net’s network is built to be extremely redundant and protected from most failures, but even the most protected network can be laid low by the right combinations of the right failures at the wrong time. -Jared and Nathan