Partial LATA1 DSL Outage

We have received confirmation from our vendor regarding a DSL outage that is affecting parts of LATA1.   We are actively working with our vendor to resolve this outage as soon as possible.  At this time we have no estimated time of repair.

Customers on our Fusion platform are unaffected by this outage.

Update: Our vendor reports that the outage was a result of a failing ATM switch within their network.  They have addressed the problem and service should now be restored to all affected users.

If you are still experiencing problems with your DSL service, rebooting your modem may help.  If not, please contact our support group for assistance.

-Chris

Emergency ATM Switch Reload

Today, Saturday September 24, at approximately 10:55 AM, we will be performing an emergency reload of one of our ATM switches. The ATM switch appears to have suffered an error which is affecting traffic flow through the box, and the only possible resolution is a full reload of the switch. All customers served by this switch will experience 5-10 minutes of downtime.

-Jared and Nathan

Update: The ATM switch has been reloaded and all affected customers should be back online at this time. If you are still experiencing problems with your legacy DSL connection, please contact our support team.

Legacy DSL Outage

We are in the process of diagnosing a Legacy DSL outage that appears to be affecting much of Northern California.  We are actively working with our vendor to resolve this outage as soon as possible.  At this time there is no estimated time of repair.

Update: Our vendor has diagnosed the cause of this outage as a defective ATM card within their network.  The card has been replaced and service has been restored to all affected users.

-Chris

Emergency Legacy DSL Maintenance

We are currently performing emergency maintenance on equipment serving Legacy DSL customers in the San Diego area. We expect 5 to 10 minutes of downtime while the work is performed.

-Tim and Jared

Update: The intrusive maintenance has been completed, and all affected customers should be back online. The trouble found was a failing ATM port in our ATM core that was causing degraded performance for all customers in the San Diego area.

Graymail Nightly Reports

Due to a recent database upgrade the nightly graymail reports were not being delivered to users. We have corrected the issue and are now processing and delivering reports to users. We apologize for any inconvenience this may have caused. -William

Non-Intrusive Backbone Maintenance

Tonight, September 7 at 12:01 AM we will be performing several non-intrusive network maintenance tasks on our Telecom backbone. The work should be completed by 1AM, and no impact is expected to customer traffic.

-Jared and Nathan

News Server Downtime

The primary server in the NNRP cluster, news.sonic.net, lost power briefly and was rebooted.  It’s currently checking its filesystems and should be back in service shortly but news.sonic.net is unavailable in the meantime.

Brief FlexLink and Fusion outage

This afternoon, at approximately 11:59AM, a network misconfiguration caused a brief outage affecting all FlexLink and Fusion customers in the Pacific Heights area of San Francisco. The cause was isolated and fixed, and service was restored by approximately 12:10PM. We are investigating the cause of the misconfiguration further so that we can prevent it from occurring again in the future.

-Tim and Jared

Non-disruptive NetApp Filer Maintenance.

Tonight ( 29, August ) at 11:00 PM we will be performing a set of software upgrades across our NetApp Filers.

These Filers store customers’ e-mail, web files, and other data.

There will be no impact on customers ability to access their information during this maintenance, and we expect it to take no more than an hour to complete.

–Augie

Update: Maintenance completed at 12:30 AM Tuesday morning. Not all of the software updates were applied during the maintenance window, another maintenance window is scheduled for tonight ( 30, August ) at 11:00 PM to complete the software updates.

Update: 31, August The final round of software upgrades will take place tonight at 11:00 PM.

Update: 1, September Maintenance continues tonight at 11:00 PM as last night’s upgrade did not proceed as planned, tonight will hopefully be the final round of upgrades, again there will be no customer impact during the maintenance. –Augie

FINAL UPDATE: 2, September 00:01 Maintenance complete all NetApp filers have been updated.