Month: February 2017

System Maintenance

Update: Maintenance complete.

 

Tonight at 11:59pm, System Operations will be running maintenance updates on several customer facing systems. The following services may experience brief interruptions:

  • IMAP/POP3/SMTP
  • Customer hosted websites
  • VPN

The maintenance period is expected to last 1 hour.

-SOC

Jasper Building Outage

Update(5:40PM): A power issued took some equipment offline. The affected equipment has been moved to a different power source.

We are currently investigating an issue affecting fiber customers located within the Jasper building. We will update this message once we have more info

  • Noc

San Francisco (Richmond District) – Fusion Fiber Intrusive Maintenance

Update(2:33AM): Maintenance is now complete. A small subset of customers may have experienced interruption in voice service during the maintenance.

Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion Fiber customers in the Richmond District of San Francisco. Expected downtime for this operation is about 5 minutes.

-Network Engineering

Core Accounting and CRM Database Server Maintenance

Tonight, starting at 23:00 we will be shutting down the SQL Database server to complete a copy of its data to a new slave server.  The final synchronization is expected to take less than an hour.  During this time, membertools and new service qualifications and signups will be unavailable.

– Sonic System Operations

Santa Rosa Datacenter UPS Maintenance

The UPSes for our Santa Rosa Datacenter are scheduled for their regular maintenance on Tuesday, Feb. 14th starting at 10:00AM.  Maintenance is fully scripted with our vendor and should not impact any services.  However, the maintenance does involve bypassing the UPS and running loads directly on utility power.  The backup generator will be kept running for the duration of the maintenance to protect against utility power failure.

– Sonic Facilities and System Operations

San Francisco Fusion Fiber Outage

Today, starting at 12:08am, equipment serving a subset of Fusion Fiber customers in the San Francisco area appears to have suffered a hardware failure. We are working to replace the equipment and restore service as quickly as possible.

Update(2:40am): ETR 2 hours

Update(4:59am): Affected equipment has been replaced and we believe service has been restored to all affected customers

-Tomoc

Intermittent connectivity for copper based customers in San Francisco

As of approximately 10:40am, a small subset of copper based customers in San Francisco may be currently experiencing intermittent connectivity issues.  We are working to identify the cause of this problem and repair it.  More information to follow.

Update(1:02pm): We have narrowed this issue down to a single device in our network and are no longer experiencing any instability. We are continuing to investigate the root cause.

-Sonic NOC