Wednesday, starting at 11:59pm, we will be performing non-intrusive maintenance on core equipment throughout the Northern California Sonic Telecom core network. Maintenance window is 4 hours.
Update (2:10am): Maintenance is now complete, with no customer impact.
– Network Engineering
Voicemail messages on our fusion voices platform experienced a delay in delivery today between 9am and 2pm. This was due to a problem that arose during a hardware and software upgrade on the servers which host the service. We have rolled back our upgrades for the time being while we diagnose the problem further. All delayed messages have been delivered and the service is currently functioning normally.
Update: Further voicemail service interruptions have been narrowed down to issues with a text to speech module in the voicemail platform. Service should be stable.
Update: Unfortunately, a second issue with the new platform caused some voicemails to be queued and not delivered again over the weekend. All of the queued messages have been delivered now. While we investigate and resolve the underlying issue we’re also adding additional monitoring to ensure that should it happen again it doesn’t go unnoticed. We’re sorry for any trouble this may have caused, prompt delivery of voicemail message is obviously very important.
-SOC
Update (2:29AM) Maintenance complete
Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Concord and Fairfield. Expected downtime for this operation can be up to 2 hours, but is typically less than 15 minutes for most customers.
-Network Engineering
Update (12:22AM): Maintenance complete
Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Saint Helena. Expected downtime for this operation can be up to 2 hours, but is typically less than 15 minutes for most customers.
-Network Engineering
Update (2:45AM): Maintenance complete.
Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Vallejo and Napa. Expected downtime for this operation can be up to 2 hours, but is typically less than 15 minutes for most customers.
-Network Engineering
Update (1:38AM): Maintenance complete
Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Vallejo. Expected downtime for this operation can be up to 2 hours, but is typically less than 15 minutes for most customers.
-Network Engineering
Update (3:08AM): Maintenance complete
Beginning tonight at midnight, we will be performing upgrades on equipment serving a subset of Fusion and Flexlink customers in Palo Alto and Santa Clara. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.
-Network Engineering
Update: Maintenance was completed at approximately 2:00AM
Beginning tonight at midnight, we will be performing upgrades on equipment serving a small subset of Fusion and Flexlink customers in San Francisco. This maintenance will not affect any Gigabit Fiber customers. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.
-Network Engineering
Update: Last night at about 12:21am, we uncovered a pre-existing routing issue that began affecting (as a result of this maintenance) a much larger customer pool than was originally anticipated. We were able to resolve the issue within about 30 minutes, but some customers may have experienced slow speeds and packet loss during this time. We do not anticipate this issue reoccurring.
Update (2:50AM): Maintenance complete
Update: This maintenance has been postponed until tomorrow night (7-20-2017) at midnight.
Beginning tonight at midnight, we will be performing upgrades on equipment serving a small subset of Fusion and Flexlink customers in San Francisco. This maintenance will not affect any Gigabit Fiber customers. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.
-Network Engineering
Update (2:25AM): Maintenance complete
Beginning tonight at midnight, we will be performing upgrades on equipment serving Fusion and Flexlink customers in Larkspur. Expected downtime for this operation can be up to 2 hours, but is typically less than 30 minutes for most customers.
-Network Engineering