Tonight, September 7 2022, we will be rebooting several public-facing systems including VPN for updates. Downtime for each system is expected to be no more than a few minutes, and all work should be completed within two hours.
Update 12:10am:
Maintenance complete.
Update(12:30) Maintenance is now complete.
Wednesday night (8/31/22), starting at 11:59PM, a small subset of customers in the Sebastopol area will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected down time is about 10 minutes. The maintenance window will be 1 hours.
– Network Engineering
Update 8/28/2022 3:16 AM: The failed hardware has been replaced and full redundancy has been restored.
Today 8/27/2022, around 6:20 PM Sonic experienced a hardware failure on one of our main edge routers. The symptoms indicated a restart of the affect line card would resolve the issue, so we did so. Upon booting, the previously failed interfaces began discarding all outbound traffic, causing routing failures towards some internet destinations from 7:00 PM to 7:15 PM. We have taken the affected piece of hardware out of service and our redundant edge router is taking over the traffic. As of now, we appear to be stable. We are working on dispatching out to replace the failed hardware to restore full redundancy.
-Network Engineering
UPDATE: 10:59pm Mail access restored.
During maintenance this evening starting at roughly 10:40 pm, our IMAP/POP3 cluster suffered a service impacting failure. We are working to restore mail access as soon as possible.
– SOC
Update(3:05) Maintenance is now complete.
Friday morning (8/26/22), starting at 1:00AM, a subset of fiber customers in the Albany and Berkeley area will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected downtime is about 45 minutes. The maintenance window will be 4 hours.
– Network Engineering
At 3:42 PM this afternoon one of the UPSes that supplies power to our Santa Rosa data center had it’s load disconnected during routine maintenance. Power was restored to approximately 3 minutes later at 3:45 PM. The affected UPS handles both our own internal loads as well as our colocated customers. The overwhelming majority of our systems are supplied with redundant power fed from two independent power systems and no public services experienced anything more than a transient failure. More information will be forthcoming once our internal investigation of the procedural failure has completed. – Kelsey
Beginning at 11:59PM on Wednesday, August 24th, we will be carrying-out network upgrades on a portion of our core network in Southern California. Sonic enterprise SSE customers in the region will experience a 1 or 2 hour downtime as we move equipment that is essential to that particular service. All other services may notice some routing changes but should not experience any downtime.
Update(9:15PM): A latent misconfiguration causing traffic loss for many subscribers in the Southern California area was exposed around 8PM when draining transit traffic in preparation for this operation. Due to the nature of the traffic loss our network monitoring did not catch the issue immediately. We were able to restore all traffic by 9:05PM and apologize for the inconvenience. We will be taking steps to protect against this type of failure in the future.
Update: This outage has been marked clear. If you are still having issues please give our Tech Support Team a call at 1-855-394-0100 8am-10pm 7 days a week.
Hello Everyone!
We are currently aware of an outage in SF that is affecting a subset of customers. If you have not already please power cycle the ONT and router and see if this resolve the issue. We are working towards resolving this issue as quickly as possible.
-Sonic Support
Customers on our Fusion DSL platform were unable to leave voicemail messages or listen to them between approximately 10:00AM and 1:30PM PST today. This was due to a configuration error which was applied after routine maintenance. We have fixed the underlying configuration and will be adding monitoring to ensure that we catch this failure state in the future, should it re-occur.
— Joe M
Sonic System Operations
Update 3:25AM: This maintenance is now complete.
Friday (8/18/22), starting at 2:00AM, a subset of customers in the San Francisco area will experience a brief service interruption while we perform some maintenance. The expected down time is about 30 minutes while the equipment upgrades. The maintenance window will be 2 hours.