Month: September 2013

Network Maintenance

Tonight, September 30, starting at 11:59PM, we will be performing minor maintenance on routers that serve Biz-T and FRATM customers in both northern and southern California. Expected down time is less than 10 minutes.

Update: Maintenance complete, approximate downtime was 6 minutes.

-Tomoc

Slow DNS Queries

Update:

The small DOS attack has been blocked, and service should be stable.

 

Starting at approximately 4am till now, our DNS servers have been  plagued by small DNS amplification attack. Any problems with slow DNS queries could be attributed to this. We are working on a fix to prevent problems like this in the future.

-Grant and the SOC

Slow DNS responses this morning

This morning approximately between 6:30 AM and 7:00 AM, users may have noticed slow or dropped responses to our recursive dns servers. Further investigation discovered this interruption was caused by a small reflective DOS attack. We are actively working on a solution to fix these DOS attacks that affect our dns servers and hope to have a solution soon. We apologize for not getting this notice out sooner when the problem occurred.

-William and the SOC

Fusion/FlexLink Equipment Maintenance

This evening, beginning at 12:00AM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Fairfield, Vallejo, Ignacio, San Rafael, Larkspur, Mill Valley, and Sausalito. Expected downtime for affected customers is less than 15 minutes.

-Tim

Fusion/FlexLink Equipment Maintenance

This evening, beginning at 12:00AM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Napa, Petaluma, and Windsor. Expected downtime for affected customers is less than 15 minutes.

-Tim

Fusion/FlexLink Equipment Maintenance

This evening, beginning at 12:00AM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Cotati, Saint Helena, and Yountville. Expected downtime for affected customers is less than 15 minutes.

-Tim

Legacy DSL Outage

This morning, September 24, starting at 2:51am, equipment serving a small subset of legacy DSL subscribers in the San Francisco/Bay Area stopped passing traffic. A reload of the affected equipment restored connectivity by 3:12am, but we will continue to investigate the root cause of the issue.

 

-Tomoc

Fusion/FlexLink Equipment Maintenance

This evening, beginning at 12:00AM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink Ethernet customers in the Santa Rosa area. Expected downtime for affected customers is less than 15 minutes.

UPDATE: Due to unexpected complications in routine preparation for this maintenance,  a small subset of Fusion customers were taken offline. We are currently working to restore connectivity to these users.

Update – 19:09 – All affected customers should be back up. We are investigating why routine non-intrusive preparation work caused this outage but plan to proceed with the maintenance window as scheduled.

-Tim

 

Issues with Outbound Faxing Through Member Tools

Due to some outbound routing changes made on Friday 9/13/2013, the “Send A Fax” (https://members.sonic.net/fax/) tool was not completing outbound faxes to most destinations. This issue has been resolved as of 10:35AM PDT today and all faxes should be completing properly now.

-Tim

Network Instability in the North Bay

A malfunctioning fiber optic connection that is part of our core network between the north bay area telecom network, our Santa Rosa datacenter, and the rest of the bay area caused some network instability over the last 45 minutes or so. The problem has been located and mitigated at this point.

– Robbie and the NOC