Tonight at midnight a very small subset of FlexLink Ethernet customers located in San Francisco will experience about 10 minutes of downtime due to emergency system maintenence needed to restore services to network features.
-Update – 10/15/2015
The same equipment is having extra work done tonight at midnight for a likely final round.
We are experiencing a connectivity problem that is affecting Fusion customers in the Pittsburg area. We are dispatching technicians to troubleshoot the problem with an ETA of 10:00am
-Michael
Update: Â The router encountered a software defect which caused the router to stop responding. Â Our technicians restarted the router and service has been restored.
-Brandon
At 8:30 am we started experiencing a large DDOS attack, causing most of our systems to become unaccessable. Our Network Operations Center is currently working to resolve the issue.
Update:9:40 am
We have blocked the large DDOS attack to our Santa Rosa Data Center. All services are back up.
Update: 10:45 AM
We have experienced some further trouble from this attack and have blocked the malicious traffic at this point.
NOC
Beginning tonight at midnight we will be performing maintenance on equipment serving legacy DSL customers in Northern California. Users may notice their connection go up and down for 10-20 minutes at a time during this maintenance window.
– Robbie and Tomoc
Today at 8:35 there was a network configuration error that caused a brief interruption of service for some of our Flexlink Long Range customers served out of SNRSCA01. The interruption lasted for about 10 minutes.
–Michael
Beginning tonight at Midnight, we will be rebooting a back-up part of our phone switch. Â This will have no impact on voice and should take around 10 minutes to complete.
Thanks – Brandon
Work has now been completed
-Brandon
System Operations will be performing updates tonight starting at 11:59pm. Â We do not anticipate any impact to customer services and maintenance should be completed within 1 hour.
Update(7:30AM): Customers affected by this outage should now be back online.
Update(12:10AM): Unfortunately this problem wasn’t as isolated as previously thought and another technician is being dispatched out to clear more equipment. More info once I receive it.
Update(8:30PM): Our provider is still working with their vendor to resolve this issue. It has been heavily escalated at this point and we are getting updates as quickly as we can.
Update (10:55AM): Our provider indicates that they have isolated the problem and will be dispatching a technician to perform the repair. No ETR given quite yet.
Update (7:25AM): We are still working with our transit provider for this area to locate and repair the problem.
We are currently experiencing an outage affecting legacy DSL users in the Chico area. The issue is still being investigated and we will update this message once we have an ETR.
Update (12:30AM): This maintenance is now complete.
Beginning tonight at midnight I will be performing software upgrades on equipment serving Fusion and FlexLink customers in Mill Valley. Expected downtime is around 15 minutes.
– Robbie
Update (11:40am): All service has been restored. We are investigating the problem with our hardware vendor to prevent this problem from happening in the future.
Today, September 17, equipment serving a subset of subscribers in the San Francisco area stopped passing traffic after software upgrades during scheduled maintenance. We are working to restore service to all affected customers.
-Michael