Tomorrow (8/13/19) at 8:00 PM we will performing intrusive upgrades on our faxback server, which provides faxes for our Enterprise customers. Downtime is expected to be about an hour.
-Network Operations
Month: August 2019
Nor-Cal Non-Intrusive Network Maintenance
Update(12:26am): Maintenance complete
Tuesday, August 13, starting at 11:59pm, we will be performing non-intrusive maintenance on equipment serving the edge of our network. Maintenance window is 4 hours.
-Network Engineering
Sebastopol – Emergency Intrusive Maintenance for Fusion Services
1:58am Maintenance Complete.
1:11am Maintenance to proceed in 30 min same expected downtime 10 min
12:09pm Maintenance postponed
Tonight, starting at 11:59 PM, we will be performing intrusive maintenance affecting a small subset of Fiber customers in the Sebastopol area. Downtime is expected to be 10 min.
-Network Engineering
Sebastopol – Fusion Fiber Maintenance
Update(12:37pm) – Maintenance is now complete
Tomorrow (8/8/19) at 11:59PM, we will be performing upgrades on equipment serving residential and enterprise fiber customers in Sebastopol. Connectivity will be impacted for all customers during the course of this work. Our maintenance window will commence until 4:00AM, but we expect actual customer downtime of fewer than 30 minutes. An update will be provided on the completion of our work.
-Network Engineering
Sunset District – Fiber Maintenance
Update: 2:00AM – Our work was successful and customer downtime was less than the expected 60 minute window.
Tomorrow (8/7/19) at 11:59PM, we will be performing upgrades on equipment serving residential fiber customers in the Sunset district of San Francisco. Connectivity will be impacted for a subset of customers during the course of this work. Our maintenance window will commence until 4:00AM, but we expect actual customer downtime of less than 60 minutes. An update will be provided on the completion of our work.
-Network Engineering
System Maintenance
Update: Maintenance complete.
Tonight begining at 11pm we will be upgrading a range of systems, including webmail and our VPN server. Any downtime to effected systems should be brief and we expect to complete the maintenance within 2 hours.
Multiple Systems Outage
Due to a switch configuration issue, various customer-facing systems are currently offline or not functioning as intended. System Operations and Network Engineering are working to resolve the issue. This post will be updated when more information is available.
Update: Network Engineering has isolated the problem to a code upgrade in a core switch cluster. After rolling the upgrade back functionality has been restored. Work to determine the specific failure mode and future mitigation steps will be conducted during business hours.
System Maintenace
Tonight beginning at 11pm we will be installing updates on various systems, including our VPN server. Any downtime to services is expected to be brief and we hope to complete the maintenance within 2 hours.
Update: Due to an unrelated issue impacting our ability to complete work during the window provided, this maintenance has been postponed until a future date. We will post a new MOTD when that maintenance is to occur.