UPDATE: Work is complete.
Tonight at 11:59 pm, we will be updating several customer facing systems. The services impacted will include:
- IMAP
- Web Hosting
- MySQL
- IPv6 Tunnels
Interruptions are expected to be brief.
-Grant and SOC
Fusion/FlexLink Intrusive Maintenance
Update( 1:00AM ): This maintenance is now complete.
This evening, beginning at 11:59PM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in San Francisco. Expected downtime is less than 15 minutes. A small portion of customers may need to reboot their DSL modems after these code upgrades take place.
– Robbie
Non-Intrusive Bay Area Backbone Maintenance
Tonight beginning at 11:59PM PDT we will begin performing software upgrades on backbone routers in the Bay Area. No customer impact is expected outside of traffic re-routing prior to the upgrades.
-Tim J.
Fusion/FlexLink Intrusive Maintenance
Update (1:00AM): This maintenance is now complete.
This evening, beginning at 11:59PM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Sebastopol, Occidental, and Guerneville. Expected downtime is less than 15 minutes. A small portion of customers may need to reboot their DSL modems after these code upgrades take place.
– Robbie
ATM Customer Aggregation Router Reload
Tonight, July 28, starting at 11:59pm, we will be performing a maintenance reload on our ATM customer aggregation routers. This will result in 5-10 minutes of downtime for Business-T and FRATM customers.
Update: Maintenance complete
-Tomoc
Fusion/FlexLink Intrusive Maintenance
7/29/14 – 12:14AM – This maintenance is now complete.
This evening, 7/28/14, beginning at 11:59PM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Eastern portions of Santa Rosa. Expected downtime is less than 30 minutes.
-Tim J.
Non-Intrusive Voice Network Maintenance
7/29/14 – 12:13AM This maintenance is now complete.
Tonight 7/28/2014 starting at 11:59PM PDT we will be finishing a software upgrade on portions of our voice network in Santa Rosa. This is expected to take 10-15 minutes and no customer impact is expected from this maintenance.
-Tim J.
Fusion/FlexLink Intrusive Maintenance
Update(1:22AM): We were able to remotely resolve the issue with the problematic card and all affected customers are now back online.
Update(12:51AM): This maintenance is now complete. A single card within a dslam in Mendocino failed to upgrade properly, leaving the customers served from that card offline for the moment. We are working towards a resolution and will update this post when the issue has been resolved.
This evening, beginning at 11:59PM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Fort Bragg, Mendocino, and Cloverdale. Expected downtime is less than 15 minutes. A small portion of customers may need to reboot their DSL modems after these code upgrades take place.
– Robbie
Outbound mail filter issue
A bug introduced this afternoon in our outbound mail filters would, under certain circumstances, add a non-compliant header to messages sent through mail.sonic.net. While most servers we’re happy to accept these messages some systems refused to accept the messages. A fix for the bug has already been released into production. –Kelsey
DNS Server Issues
For the past couple of hours we’ve been having issues with our DNS servers. We just identified the source of the problem and should have it completely resolved in a few minutes. As it turns out, this was a variance on the attacks that have caused us trouble in the past and was escaping the notice of our automated systems. We’re sorry for any inconvenience this may have caused!
-Kelsey, William, Tomoc and Tim