Tonight, Wednesday, August 1, at 12:01AM we will be performing invasive maintenance on equipment serving Fusion and FlexLink products in San Bruno. Customer down time is expected to be less than 20 minutes.
-Sonic.net NOC
LA area legacy DSL maintenance
This Friday morning, at 12:00AM on July 27th, we will be performing intrusive maintenance on DSL aggregation equipment serving customers in the LA area. Expected downtime is less than 15 minutes.
Update: The maintenance was completed successfully and all affected customers are back online.
– Robbie and Tomoc
Legacy DSL Outage
A port on one of our ATM aggregation switches appears to have failed and has taken a small subset of legacy DSL customers in the bay area offline. We are working on rerouting this traffic and bringing these customers back online ASAP.
-Robbie and Tomoc
Update: All affected customers have been moved to a new port and should be back online. If you are still experiencing trouble with your DSL connection please call our support department as usual.
Legacy DSL Maintenance in LA
This Friday morning, at 12:00AM on July 20th, we will be performing intrusive maintenance on DSL aggregation equipment serving customers in the LA area. Expected downtime is less than 15 minutes.
– Robbie and Tomoc
Network Maintenance
This coming Monday, July 16th, from 9:00PM to 3:00AM, we will be performing major intrusive work within one of our core points of presence in San Jose. We do not expect any service impact to our customers as a result of this operation. Fusion voice users may notice a very short delay in their calls when initially dialing out during a portion of this window.
– NOC
Update: Maintenance completed without a hitch, customers should not have experienced any service interruption.
Sonic.net Router Maintenance
Tonight, July 8th, at 12:00am, we will be performing maintenance on routers that aggregate Business-T and Frame-Relay connections. Â The procedure will require a reboot of the routers. Expected downtime is less than 15 minutes.
– Robbie and Tomoc
Update: As some of you may have noticed, 12:00am tonight is actually July 6th. I apologize for any confusion this may have caused.
Update 2: The router maintenance has been completed and all affected customers are now be back online. Total downtime was about 3 minutes.
Sonic Telecom Emergency Backbone Maintenance
Due to a hardware failure in the Sonic Telecom backbone we will need to perform emergency router maintenance over the next hour. We do not believe this work will have any impact on customer traffic but there is the potential for a brief service interruption. Further updates will follow shortly.
-Tim and Nathan
Update: Maintenance work has been completed. There should have been no impact to customer traffic. This evening, from 12:00AM until 1:00AM, we will be performing further maintenance of a similar nature. Again we do not expect any impact to customer traffic but there is a possibility of brief service interruptions during this time.
Fusion Outage in San Francisco
This evening during routine network maintenance, around 5:10 PM,  a configuration error caused one of our DSLAMs in San Francisco to destabilize.  This disrupted service for all Fusion customers on that particular DSLAM. Service has been restored to all affected customers as of  around 5:50 PM. If you are still having any trouble with your Fusion service please contact our support department as usual.
– Robbie and the NOC
Coming soon: New Fusion equipment!
I’m very excited to announce that on July 1st, we will be launching a new equipment program for our residential Fusion Broadband+Phone service. The new equipment is a super-fast ASDL2+ modem/router with four Fast Ethernet ports, high-powered WiFi “N” and an easy-to-use web interface. Built-in Firewall protection for your home network is also included.
Our new Fusion equipment will also streamline our support systems, and allow us to upgrade firmware, manage configuration and roll out new features. For example, in future this capability will be used to offer IPv6 to customers.
The new equipment will be owned by Sonic.net and rented to customers as part of the Fusion home service, for $6.50 per month. This assures you’ll always have working equipment – we can remotely troubleshoot it, and replace it for you at no charge if the equipment fails.
Existing customers are of course welcome to continue to use the equipment that they already own, or you will be able to upgrade (starting next week) to new rental equipment if you find the features compelling.
Webcluster Service Interuption
A number of hosted websites (including www.sonic.net) were offline for a approximately seven minutes this afternoon. We’re investigating the initial cause that caused the servers to drop out but believe a known bug in our load balancer configuration was also triggered that prevented all of the sites from being automatically brought back online.
Update: Thu Jun 28 10:16:44 PDT 2012. We’ve seen a couple more events since this happened yesterday and we believe we have identified the DoS attack and are working to mitigate its affects now.
Update: Thu Jun 28 13:13:41 PDT 2012. We’ve installed a custom apache module that should eliminate or, at least, reduce the affects of the DoS attack on our web cluster and are hopeful that this will prevent this issue from causing problems again.
-Kelsey, William and Kevan