Category: DSL

Northern California Network Core Maintenance

Update(1:10AM): Maintenance successfully completed.

Tomorrow (9/17/20) at 11:59PM we will be performing maintenance on core network hardware in our Santa Rosa data-center.  We expect this maintenance to be non-intrusive, but has the potential to impact a wide range of enterprise and Sonic hosted services.  The maintenance window will be 4 hours.

– Network Engineering

Core Router Maintenance

Update(3:30AM): Maintenance Complete

Starting at 11:59PM on Wednesday, September 16th, we will be performing maintenance on core routing equipment that serves all legacy DSL customers as well as Fusion/FlexLink customers in the Sacramento area. Expected downtime for this operation is under 30 minutes. The maintenance window is open until 4am.

Legacy DSL Emergency Maintenance

Update(12:42am): Maintenance complete

Tonight, September 10th, starting at 11:59pm, we will be performing an emergency migration of a small subset of Legacy DSL subscribers to different aggregation equipment. This migration is to mitigate a small amount of packet loss to the affected equipment. Customer down time is expected to be less than 10 minutes.

-Network Engineering

Core Router Maintenance

Starting at 11:59PM on Monday, August 24th, we will be performing maintenance on core routing equipment that serves all legacy DSL customers as well as Fusion/FlexLink customers in the Sacaramento area. Expected downtime for this operation is under 30 minutes. The maintenance window is open until 4am.

Northern California – Legacy DSL Maintenance

Update(5:17AM): Maintenance complete

Beginning at 11:59PM on 7/15/20 we will be performing maintenance on legacy DSL customers in northern California. The maintenance window for this operation is 6 hours, but we expect less than 2 hours downtime for affected customers.

Sacramento Emergency Fiber Maintenance

Tonight, beginning at about 9pm, a vendor of ours will be performing emergency maintenance on outside plant that provides connectivity for our network in the Sacramento and surrounding area for both legacy DSL and Fusion/FlexLink services. They expect the downtime to be around 30 minutes within the 9 hour maintenance window.

Legacy DSL Maintenance

Update: This has been postponed until tomorrow night, 2/12/2020.

Tomorrow night, 2/11/2020, beginning at 11:59PM, we will be performing maintenance on equipment serving a small subset of legacy DSL customers in northern California. Expected downtime is less than 45 minutes.

Lata 1 Northern California Legacy DSL Outage

Update: After a technician was dispatched for hardware replacement, this issue should have been resolved as of 3:30PM today.

 

Today just after 12:00pm, a piece of network infrastructure serving a subset of legacy DSL customers in northern California appears to have failed. We are currently identifying the exact source of the failure.  Update will be provided when we have more info and/or an ETR.

-Network Engineering

Lata 1 Northern California Legacy DSL Outage

Update 12:40AM: Replacement of failed hardware has resolved the issue and all impacted customer connectivity should be restored.

Tonight just after 6:00pm, a piece of network infrastructure serving a subset of legacy DSL customers in northern California appears to have failed.  After lengthy investigation, we have identified the source of the issue and are currently working on a repair as quickly as possible. An update will provided when we have a timeline for resolution.

-NOC Engineering

Sacramento Area – Fusion and legacy DSL outage

Update(2:11pm): The second replacement card has resolved the issue. All service restored.

Update(1:55pm): The replacement card has failed to boot, so another replacement was located and is being installed. New ETR of 3:00pm

Update(12:46pm): The issue has been tracked to a failed 100G card in our provider’s network. Dispatch for replacement has been made and we have an ETR of 1:30pm

Update(10:53am): Further investigation has narrowed the problem down to an equipment failure in our transport provider’s network. We are working with them to restore service as quickly as possible but don’t have an ETR as of yet. We would also like to take this opportunity to update the scope of the outage, which we have determined affected Fusion and FlexLink services in the Sacramento area, as well as Legacy DSL subscribers in the Sacramento and Stockton areas.

At 8:05AM this morning, a hardware serving a backbone link to Sacramento failed.  We are currently assessing the situation, but presently believe connectivity is impacted to Sonic Fusion and legacy DSL customers in the Sacramento area.  We will update with an ETR when it is available.

-Network Engineering