Brentwood – Emergency Fusion Fiber Maintenance

Update 6:15pm (6/9/19) – As of around 5PM PST, all impacted Brentwood Fusion fiber customers should be fixed.  Please contact our support department if you believe you were part of this outage and are still not online.

 

Update 4:25pm (6/9/19) – We have identified and just successfully tested a fix for the connectivity issue.  We are presently working to ready and deploy it to all impacted Brentwood fiber customers as quickly as possible.

 

Update 9:40am (6/9/19) – We are aware of the small group of Brentwood customers experiencing lack of connectivity and is working as fast as possible to resolve the issue.  Last night’s maintenance was unsuccessful in providing the intended fix, and we are actively perusing alternate means of repair.  We will provide an ETR as soon as we believe it can be accurately given.

 

Tonight at 11:59pm we will be performing intrusive maintenance on a small subset of Fusion fiber customers located in Brentwood.  Actual customer downtime should be less than 60 minutes, but the maintenance window extends for 4 hours.

-Network Engineering

System Maintenance

Update: All services are back online.

Tonight during system maintenance we encountered an issue that is effecting some services including webmail. We are working to bring the services back online as quickly as possible.

Phone Switch Maintenance

Update: 12:33 AM Maintenance is complete.

Tonight 6/5/2019 at 11:59 PM we will be performing maintenance on our phone switch. No service interruptions is expected as a result of this maintenance. The window for this maintenance is 2 hours.

~Network Engineering

Brentwood – Fusion Fiber/Business Park Fiber Intrusive Maintenance

Update(2:47am): This maintenance is now done

Tomorrow night(6/6/19), starting at 11:59 PM, we will be performing intrusive maintenance affecting all Fusion Fiber and Business Park Fiber customers in the Brentwood area. Expected down time is approximately 60 minutes. The maintenance window is 4 hours.

– Network Engineering

Albany Fiber Outage

(Update: As of 9:58 AM, all backbone repairs have completed. A few individual customers need the drop to their premises need to be re-done, and we are following up with them).

 

A small subset of our customers in Albany are experiencing a Fiber outage. A tree limb brought down the PG&E lines and our Fiber backbone in the area.

We have dispatched a repair team to replace the downed Fiber Strand.

 

ETR is 5-9 hours for complete repairs.

Critical Systems Update

Update: All updates complete.

Various systems are receiving a critical kernel update and will need to be rebooted. Downtime for each system is expected to be brief, but work will progress throughout the day until updates are complete.

East Bay Fiber Connectivity Issues

UPDATE: As of 12:16 PM, connectivity issues have been resolved.
We’re currently investigating connectivity issues on our fiber in parts of the East Bay with customers using eero routers. Unfortunately, we do not have an ETR at this time.

Sunset District – Non-intrusive Fiber Maintenance

UPDATE 3:00AM – Maintenance successfully completed.

Thursday (5/16/19) at 11:59PM, we will be performing upgrades on equipment serving Fusion Fiber and FlexLink Fiber customers in the Sunset District of San Francisco. We do not expect any customer impact during this work. Our maintenance window will last up to 4:00AM. An update will be provided on completion.

-Network Engineering

Northern California Edge Maintenance

Update(2:35am): An unexpected router bug caused a routing loop affecting traffic in our Santa Rosa data center for approximately 20 minutes. This would have affected local colo customers, as well as some internal Sonic infrastructure. Maintenance is complete and all services are restored.

 

Tonight, May 14, starting at 11:59pm, we will be performing maintenance on network edge equipment serving the Northern California Sonic network. Customers may experience brief intervals of routing instability as traffic is shifted to backup paths, however this maintenance should not cause any customer down time. This maintenance window is 4 hours.

-Network Engineering

System Maintenance

This maintenance has been postponed and will be rescheduled at a later date.

Update: This maintenance has been rescheduled for Tuesday, May 14th.

Tonight beginning at 11pm we will be performing maintenance on several systems including our customer shell server. We expect any downtime to be minimal and for the maintenance to be completed within 2 hours.