Tonight, beginning at 11:59, we will be performing maintenance on equipment serving Fusion/Flexlink customers in the Albany area. We do not expect any customer downtime for this operation.
– Robbie and Tim
Fiber DIA Intrusive Maintenance
On Friday, 8/30/2013, between the hours of midnight and 6AM, the telco will be performing intrusive maintenance on equipment serving some Fiber DIA customers in Northern California. Actual down time is expected to be less than three hours for affected customers.
-Tim
Update: The planned maintenance work was not performed last night due to unforeseen circumstances. A new date will be provided once the work has been rescheduled.
Fusion Outage – Albany
A issue with one of our DSLAMs in Albany had caused a subset of our customers in the area to go offline. We believe the issue to be resolved at this point.
– Robbie
Temporary Fusion/Flexlink Outage in San Francisco
At around 11:32 AM a small subset of Fusion/Flexlink customers in San Francisco were taken offline due to a misconfigured router. The mistake was quickly recognized, corrected and service was restored by 11:36 AM.
– Robbie
Database Server Upgrade
*Update* All work is complete – customer databases should now be back up and functional.
This Thursday, Aug 22, from 12am to 1am, System Operations will be performing a memory upgrade to the customer SQL server. During the maintenance window, customer databases will not be reachable.
— Joe & the SOC
Customer MySQL Database Servers Maintenance
Tonight at Midnight, our customer database server will be taken off-line to replace two failed disks as well as repair some suspected bad cabling on the raid controller. While the server is off-line any hosting accounts requiring database access will be off-line as well. It is expected to take less that 30 minutes to complete the replacement and repair and have the system back in on-line and in service.
Services were restored on time just before 00:30.
-Kelsey
Support Closed Sunday
Our team will be taking time off this coming Sunday to celebrate Sonic.net’s 19th birthday. Customer Support will be closed on 8/11/2013 from 11am to 6pm. Support will re-open at 6pm on Sunday and remain available until 10pm.
Our normal 24x7x365 schedule is still applicable for our on-call network operations team for dedicated business and colocated customers.
-Adam and Support
FlexLink Equipment Maintenance
On Thursday the 8th, beginning at 12:00AM, we will be performing intrusive maintenance on equipment serving FlexLink Long Range customers in the Burlingame area. Expected downtime for affected customers is less than 15 minutes.
-Tim and Monroe
Update: We will also be performing maintenance on equipment serving a very small number of Private Ethernet Transport customers in the San Francisco area. Expected downtime for this operation is less than 10 minutes.
Customer Database Load
Due to excessive load on our customer database servers, databases connection may intermittently time out. We are working to fix the problem and prevent further interruption.
Update: Currently both customer mysql server instances are off-line, we’re investigating what appears to be a hardware related issue and hope to have services restores shortly.
Update: It appears that these problems were caused by an issue with the raid controller not properly handling IO at anywhere near its expected performance levels. A reboot seems to have restored performance back to normal. We will continue to monitor the system closely.
Custom Hosting updates
Update: Reboot was complete as of 12:05 am.
Tonight, at 11:59 pm, we will be rebooting our custom hosting platform. Any sites hosted on that platform will be unreachable during this time. Expect service to be available at 12:15 am.
Grant and SOC