Month: April 2012

Legacy DSL Outage

This morning, Wednesday, April 25 at 7:00AM, equipment serving a subset of customers in (LATA2) Chico performed a soft reboot. We are investigating the cause of this reload with our vendors to ensure this does not happen again. All affected customers should be back online, estimated down time of less than 5 minutes.

-Tomoc

Update: We are experiencing further trouble with this device and are currently investigating

Update: We have narrowed down the extended troubles to a failing card in one of our pieces of equipment. The problematic card has been replaced and all affected customer’s service has been restored.

Storage cluster maintenance failure.

At 3:50pm today we suffered a storage cluster failure while performing maintenance on one of our storage arrays, all services have been restored, but for a brief window customers may have seen e-mail and web site errors.

–Augie

Emergency Legacy DSL Maintenance

Tonight, Tuesday, April 24th, starting at midnight, we will be performing maintenance on an ATM switch serving some of our (LATA1) Bay Area subscribers. Expected down time is less than 10 minutes.

-Tomoc and Robbie

Update: All work has been completed. A small subset of subscribers experienced approximately 5 minutes of down time during this procedure.

Emergency Maintenance on Legacy DSL

We are preforming emergency maintenance on one of the ATM switches serving customers in the Bay Area (LATA 1). A software malfunction has left the device unable to pass traffic for a significant number of customers, and we’ve elected to reload the affected unit to restore service.

This reload will cause 5 to 10 minutes of additional downtime for customers aggregated through that device.

-Nathan and Tomoc

DSL Aggregation Router Reload

This Thursday, April 19 at 12:01 AM we will be performing maintenance reloads of two Redbacks that terminate traditional DSL service. This will affect some of our Bay Area DSL subscribers. Expected downtime is 5 minutes.

-Tomoc

Update:

Reloads completed successfully, all service restored.

Legacy DSL Outage

Earlier tonight, around 11:30PM, we experienced a failure on equipment servicing legacy dsl customers in the Chico area. The affected circuits have been migrated to backup equipment and are now back online.

Legacy DSL Maintenance

This Wednesday, April 11th, starting at 12:01 am, we will be performing maintenance on equipment servicing legacy DSL customers in northern California. Some customers may experience a loss of connectivity for up to 5 minutes during this maintenance window. Fusion and Flexlink customers will be not be affected by this operation.

– Robbie, Tomoc and Jared

Update: Maintenance completed successfully, all service has restored.

Legacy DSL Maintenance

This Thursday, April 5th, starting at 11pm, we will be performing a variety of maintenance operations on equipment servicing legacy DSL customers in northern California. We will be avoiding downtime as much as possible but customers may experience a loss of connectivity for up to 15 minutes during this maintenance window.

– Robbie, Tomoc and Jared

Update:  All work has been completed. A small subset of customers would have experienced 5-10 minutes of downtime during parts of the work.

Intrusive Network Maintenance

This Tuesday, April 3 at 12:01 AM, we will be performing maintenance reloads on equipment serving Legacy DSL in San Francisco/bay area, Sacramento, Fresno and San Diego. Expected down time is between 5-10 minutes.

-Tomoc and Robbie

Update: The maintenance work was completed successfully and all impacted customers are back online.