Update(1:19am) This maintenance is now complete.
Friday morning (9/2/22), starting at 12:00AM, a subset of fiber and Business fiber customers in the Albany and Berkeley areas will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected downtime is about 45 minutes. The maintenance window will be 4 hours.
– Network Engineering
Update 8/28/2022 3:16 AM: The failed hardware has been replaced and full redundancy has been restored.
Today 8/27/2022, around 6:20 PM Sonic experienced a hardware failure on one of our main edge routers. The symptoms indicated a restart of the affect line card would resolve the issue, so we did so. Upon booting, the previously failed interfaces began discarding all outbound traffic, causing routing failures towards some internet destinations from 7:00 PM to 7:15 PM. We have taken the affected piece of hardware out of service and our redundant edge router is taking over the traffic. As of now, we appear to be stable. We are working on dispatching out to replace the failed hardware to restore full redundancy.
-Network Engineering
UPDATE: 10:59pm Mail access restored.
During maintenance this evening starting at roughly 10:40 pm, our IMAP/POP3 cluster suffered a service impacting failure. We are working to restore mail access as soon as possible.
– SOC
Update(3:05) Maintenance is now complete.
Friday morning (8/26/22), starting at 1:00AM, a subset of fiber customers in the Albany and Berkeley area will experience a brief service interruption while we perform maintenance on hardware serving their connections. The expected downtime is about 45 minutes. The maintenance window will be 4 hours.
– Network Engineering
At 3:42 PM this afternoon one of the UPSes that supplies power to our Santa Rosa data center had it’s load disconnected during routine maintenance. Power was restored to approximately 3 minutes later at 3:45 PM. The affected UPS handles both our own internal loads as well as our colocated customers. The overwhelming majority of our systems are supplied with redundant power fed from two independent power systems and no public services experienced anything more than a transient failure. More information will be forthcoming once our internal investigation of the procedural failure has completed. – Kelsey
Beginning at 11:59PM on Wednesday, August 24th, we will be carrying-out network upgrades on a portion of our core network in Southern California. Sonic enterprise SSE customers in the region will experience a 1 or 2 hour downtime as we move equipment that is essential to that particular service. All other services may notice some routing changes but should not experience any downtime.
Update(9:15PM): A latent misconfiguration causing traffic loss for many subscribers in the Southern California area was exposed around 8PM when draining transit traffic in preparation for this operation. Due to the nature of the traffic loss our network monitoring did not catch the issue immediately. We were able to restore all traffic by 9:05PM and apologize for the inconvenience. We will be taking steps to protect against this type of failure in the future.
Today on August 22, 2022, we added a default SPF (Sender Policy Framework) record to every customer domain name that has Sonic DNS hosting and email hosted with Sonic.
On August 15th, we emailed everyone who would be impacted by this change. The email subject was “Important information about your Sonic domain email hosting” and included a list of domain names that would be updated.
If you did not receive the email or do not have domain email and DNS hosting with Sonic, this notification does not apply to you.
If you received the above notification email and did not opt out or add an SPF record before today, the following SPF record was added to your domain:
"v=spf1 include:mail.sonic.net ~all"
For more information, please see our help document SPF and Sonic. To learn more about SPF, see Sender Policy Framework on Wikipedia.
We made this change to fight spam & phishing and to help ensure reliable email delivery for our domain email customers. SPF tells other email providers that Sonic is authorized to send email on behalf of these hosted domain names.
Update: This outage has been marked clear. If you are still having issues please give our Tech Support Team a call at 1-855-394-0100 8am-10pm 7 days a week.
Hello Everyone!
We are currently aware of an outage in SF that is affecting a subset of customers. If you have not already please power cycle the ONT and router and see if this resolve the issue. We are working towards resolving this issue as quickly as possible.
-Sonic Support
Update(1:23am) Maintenance is now complete. IPv6 has been turned on for this location and people are binding properly. Please reach out to support should you encounter anything out of the ordinary.
–Network Engineering
We are pleased to announce that on Thursday evening we will be enabling native IPv6 support in Sebastopol for our fiber customers. This will be deployed in a dual-stack configuration that supports both SLAAC and IA-PD protocols. The process will start on 8/18/22, at 11:59PM. Fiber customers in the Sebastopol area will experience a brief service interruption while we perform configuration changes necessary to bring up IPv6. The expected downtime is about 20 minutes. The maintenance window will be 4 hours.
IPv6 is an important deployment as it provides significantly expanded address space. As the number of devices that need access to the Internet continues to grow, so does the need for additional addressing. For more information about IPv6, check out this link:
https://help.sonic.com/hc/en-us/articles/8242850487575-IPv6
Over the coming weeks and months, we will be deploying IPv6 to all our fiber customers. Our SmartRG and eero rental devices are fully compatible and will work in their standard configuration.
We conducted thorough internal testing but are unable to test for every scenario. We don’t expect any issues, however, if you encounter anything out of the ordinary, don’t hesitate to reach out to our support team.
– Network Engineering
Customers on our Fusion DSL platform were unable to leave voicemail messages or listen to them between approximately 10:00AM and 1:30PM PST today. This was due to a configuration error which was applied after routine maintenance. We have fixed the underlying configuration and will be adding monitoring to ensure that we catch this failure state in the future, should it re-occur.
— Joe M
Sonic System Operations