Update 9/14/22 1:55AM: Tonight’s maintenance is successfully completed.
Wednesday (9/14/22), starting at 1:00AM, a subset of customers in the Berkeley area will experience a brief service interruption while we perform some maintenance. The expected down time is about 30 minutes while the equipment upgrades. The maintenance window will be 2 hours.
– Network Engineering
Tonight, September 7 2022, we will be rebooting several public-facing systems including VPN for updates. Downtime for each system is expected to be no more than a few minutes, and all work should be completed within two hours.
Update 12:10am:
Maintenance complete.
Update 9/2/22 2:20AM: Tonight’s maintenance is successfully completed.
Friday (9/2/22), starting at 1:00AM, a subset of customers in the Berkeley area will experience a brief service interruption while we perform some maintenance. The expected down time is about 30 minutes while the equipment upgrades. The maintenance window will be 2 hours.
– Network Engineering
Thursday (9/1/22), starting at 12:00AM, we will be performing maintenance on equipment serving a subset of Berkeley fiber customers. No customer impact is expected from this work. The maintenance window will be 2 hours.
– Network Engineering
Update(12:30) Maintenance is now complete.
Wednesday night (8/31/22), starting at 11:59PM, a small subset of customers in the Sebastopol area will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected down time is about 10 minutes. The maintenance window will be 1 hours.
– Network Engineering
Update(1:19am) This maintenance is now complete.
Friday morning (9/2/22), starting at 12:00AM, a subset of fiber and Business fiber customers in the Albany and Berkeley areas will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected downtime is about 45 minutes. The maintenance window will be 4 hours.
– Network Engineering
Update 8/28/2022 3:16 AM: The failed hardware has been replaced and full redundancy has been restored.
Today 8/27/2022, around 6:20 PM Sonic experienced a hardware failure on one of our main edge routers. The symptoms indicated a restart of the affect line card would resolve the issue, so we did so. Upon booting, the previously failed interfaces began discarding all outbound traffic, causing routing failures towards some internet destinations from 7:00 PM to 7:15 PM. We have taken the affected piece of hardware out of service and our redundant edge router is taking over the traffic. As of now, we appear to be stable. We are working on dispatching out to replace the failed hardware to restore full redundancy.
-Network Engineering
UPDATE: 10:59pm Mail access restored.
During maintenance this evening starting at roughly 10:40 pm, our IMAP/POP3 cluster suffered a service impacting failure. We are working to restore mail access as soon as possible.
– SOC
Update(3:05) Maintenance is now complete.
Friday morning (8/26/22), starting at 1:00AM, a subset of fiber customers in the Albany and Berkeley area will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected downtime is about 45 minutes. The maintenance window will be 4 hours.
– Network Engineering
At 3:42 PM this afternoon one of the UPSes that supplies power to our Santa Rosa data center had it’s load disconnected during routine maintenance. Power was restored to approximately 3 minutes later at 3:45 PM. The affected UPS handles both our own internal loads as well as our colocated customers. The overwhelming majority of our systems are supplied with redundant power fed from two independent power systems and no public services experienced anything more than a transient failure. More information will be forthcoming once our internal investigation of the procedural failure has completed. – Kelsey