Category: Uncategorized

Tonight, 5/3/2016 starting at 11:59 we will be preforming maintenance on a small subset of Fusion Fiber customers in the San Francisco Area.  Expected downtime is 10 minutes.

-Brandon

Update: 12:01am – Maintenance is now complete

-Brandon

Early this morning, we updated firmware on equipment serving some San Francisco Fusion Fiber subscribers. This new firmware has proven unstable and has caused intermittent service interruptions throughout the day. We have been working with our vendor to resolve the issue as quickly as possible and ensure it does not continue in the future.

-Tomoc and Brandon

Update (10:00pm)  We have isolated and fixed the issue, service should now be restored/stable.

-Brandon

Tonight, April 27, starting at 11:59pm, we will be performing software updates on equipment serving Fusion Fiber customers in Sebastopol.  Estimated time of completion is 3am.

Update(12:29am): Maintenance complete

-Tomoc

Tonight, April 25, starting at 11:59pm we will be performing software updates on equipment serving a subset of Fusion Fiber customers in the Sunset area.  Estimated time of completion is 3am.

Update(3:42am): Maintenance complete.

-Tomoc

This morning, April 8, at 5:30am, an aggregation circuit serving legacy DSL and Business T subscribers in the Chico area went down. We are currently investigating the issue.

Update(10:32am): We are working with our transport vendor to restore traffic as soon as possible. No ETA at this time

Update(4:26pm): An upstream provider has located an equipment failure and is working to replace the failed equipment. Still no ETR.

Update (4/9/2016 9:40AM): The upstream provider now has the replacement parts on site at the location of the failure, and an engineer is currently working on replacing the failed equipment. We will update as soon as we have more information.

Update (4/9/2016 1:21PM): The failed equipment has been replaced by the upstream provider, but the circuit is still down. We are continuing to work with our upstream provider to resolve this.

Update (4/9/2016 3:28PM): Our upstream provider has identified a second failure, and is actively working to replace the failed equipment. We hope to have an ETR soon.

Update (4/9/2016 6:04PM): Our upstream provider has the replacement part in transit, and the ETR is now 3AM PST.

Update (4/10/2016 5:58AM): Our upstream provider has experienced delays in transporting the replacement equipment to the failure area, and they have given us a new ETR of today at 1:00PM PST.

Update (4/10/2016 1:33PM): The replacement equipment is now on site, and technicians are currently working on installing it.

Update (4/10/2016 3:45PM): The equipment has been replaced, but the issue persists. Our transit provider is continuing to troubleshoot the cause of the outage. There is no ETR at this point.

Update (4/10/2016 6:35PM): The upstream provider has identified and resolved multiple equipment failures in their network causing this outage.  All services were restored as of 6:00PM PST.

-Tomoc and Michael

A small subset of Fusion customers served in the San Francisco area experienced an interruption of service this morning. The issue appears to have stemmed from a code upgrade that was done on our equipment last night. We are working with our vendors to ensure that upgrades in the future will no longer cause an issue.

• NOC

Tonight, April 6, starting at 11:59pm, we will be performing maintenance on equipment serving subsets of Fusion and FlexLink customers in the following areas. Expected downtime is 20 minutes.

Affected areas include Forestville, Pittsburg, Oakley, San Carlos

Update(12:54am): Maintenance complete

-Tomoc

UPDATE: Maintenance complete.

Beginning at 11:59pm this evening, Operations will be performing routine system maintenance. The following services may experience brief interruptions:

• OpenVPN
• IMAP/POP3
• IPv6 Tunnels
• Membertools
• Custom Hosting Containers
• Custom Web

Thanks,

SOC

This morning Apr 05 at 7:20am a router aggregating Flexlink Long Range customers out of Albany, CA developed problems. We have dispatched a technician to resolve the issue.

Michael

Update: 10:25am – Service is now restored.

-Brandon