Month: April 2016

Sonic Fiber Maintenance – San Francisco

Starting tonight at 11:59pm we will be performing maintenance in San Francisco on a small subset of customers in the sunset district for Fusion Fiber services.  Estimated downtime is 10 – 15 minutes.

-Brandon

Santa Cruz – Fusion/FlexLink Intrusive Maintenance (April 13th and 14th)

Update(1:20AM): This maintenance is now complete.

Both tonight (April 13th, 2016) and tomorrow night (April 14th, 2016) beginning at midnight we will be performing upgrades on equipment serving a subset of Fusion and Flexlink customers in Santa Cruz. Expected downtime for each operation can be up to 1 hour, but is typically under 30 minutes for most customers.

-Michael and Robbie

Chico ATM Outage

This morning, April 8, at 5:30am, an aggregation circuit serving legacy DSL and Business T subscribers in the Chico area went down. We are currently investigating the issue.

Update(10:32am): We are working with our transport vendor to restore traffic as soon as possible. No ETA at this time

Update(4:26pm): An upstream provider has located an equipment failure and is working to replace the failed equipment. Still no ETR.

Update (4/9/2016 9:40AM): The upstream provider now has the replacement parts on site at the location of the failure, and an engineer is currently working on replacing the failed equipment. We will update as soon as we have more information.

Update (4/9/2016 1:21PM): The failed equipment has been replaced by the upstream provider, but the circuit is still down. We are continuing to work with our upstream provider to resolve this.

Update (4/9/2016 3:28PM): Our upstream provider has identified a second failure, and is actively working to replace the failed equipment. We hope to have an ETR soon.

Update (4/9/2016 6:04PM): Our upstream provider has the replacement part in transit, and the ETR is now 3AM PST.

Update (4/10/2016 5:58AM): Our upstream provider has experienced delays in transporting the replacement equipment to the failure area, and they have given us a new ETR of today at 1:00PM PST.

Update (4/10/2016 1:33PM): The replacement equipment is now on site, and technicians are currently working on installing it.

Update (4/10/2016 3:45PM): The equipment has been replaced, but the issue persists. Our transit provider is continuing to troubleshoot the cause of the outage. There is no ETR at this point.

Update (4/10/2016 6:35PM): The upstream provider has identified and resolved multiple equipment failures in their network causing this outage.  All services were restored as of 6:00PM PST.

-Tomoc and Michael

San Francisco Outage

A small subset of Fusion customers served in the San Francisco area experienced an interruption of service this morning. The issue appears to have stemmed from a code upgrade that was done on our equipment last night. We are working with our vendors to ensure that upgrades in the future will no longer cause an issue.

 

  • NOC

System Maintenance

Update: All work has been completed.

The Operations team will be performing software updates to the customer database servers tonight, April 6th, at 11:59pm. The update process should take no more that 30 minutes. During this time, MySQL and PostgreSQL databases and any websites that use them (such as WordPress) may be unavailable.

— Joe and the System Operations Center

Webmail Upgrade

On Friday, April 15th at approximately 11 PM, webmail.sonic.net will be temporarily unavailable while it is upgraded.  We expect this to take approximately 1-1.5 hours to complete.  Mail services such as IMAP and POP will remain available and webmail-beta.sonic.net can be used during this time to retrieve mail.

Please see this forum post for more information.  If you haven’t already, we encourage you to try the beta and report any issues regarding features or functionality that you use in the current version.  Please note that settings between beta and production are separate, but all mail and folders are synchronized between the two.

SOC

Fusion/FlexLink Router Maintenance

Tonight, April 6, starting at 11:59pm, we will be performing maintenance on equipment serving subsets of Fusion and FlexLink customers in the following areas. Expected downtime is 20 minutes.

Affected areas include Forestville, Pittsburg, Oakley, San Carlos

Update(12:54am): Maintenance complete

-Tomoc

System Maintenance

UPDATE: Maintenance complete.

Beginning at 11:59pm this evening, Operations will be performing routine system maintenance. The following services may experience brief interruptions:

  • OpenVPN
  • IMAP/POP3
  • IPv6 Tunnels
  • Membertools
  • Custom Hosting Containers
  • Custom Web

Thanks,

SOC