Non-Intrusive Network Maintenance

Friday, March 23 at 12:01 AM we will be performing non-intrusive network maintenance on one of our LA POPs. This maintenance should not affect connectivity, and should last no more than 10 minutes.

-Tomoc

DSL Aggregation Router Reload

This Wednesday, March 21, at 12:01 AM we will be performing maintenance reloads of six Redbacks that t
erminate traditional DSL service. This will affect some of our Bay Area DSL subscribers.
Expected downtime is 5 minutes.

-Tomoc and Robbie

Update: The scheduled maintenance was completed successfully and all affected customers should be back online.

FlexLink and Fusion Outage

We are currently experiencing an outage affecting FlexLink and Fusion customers in the Menlo Park area. We are investigating the cause of this outage and attempting to restore service as soon as possible. Further updates will be provided shortly.

-Tim and Nathan

Update: A technician has been dispatched to restore service. The current estimated time to restoration is under two hours.

Update #2: Service has been restored as of 1:21PM. The issue was isolated to a piece of IP routing equipment. We will be investigating the root cause of the issue to avoid future occurrences.

Sacramento Network Reconfiguration Redux

Tonight, Friday March 2 at 12:01 AM, we will be performing intrusive network maintenance at our main Sacramento CO. This maintenance will cause 5-10 minutes of downtime for all Fusion and Flexlink customers served by that CO, and potentially longer outage for our Fiber-served customers. This maintenance will tie our old Sacramento services into the new expansion we’ve made in the Sacramento area, providing better reliability and stability for all services.

-Jared and Matt

Update: All network maintenance is complete, and all services should be restored at this time. Fusion and Flexlink customers should have only seen about 5 minutes of downtime.

Sacramento Network Reconfiguration

Tonight, Tuesday, Frebruary 28 at 12:01 AM we will be performing some potentially intrusive network configuration changes to our Telecom network in the Sacramento area. We do not expect any downtime, but there may be some brief outage of network connectivity for Fusion and Flexlink customers in the Sacramento area.

-Jared

ADSL Outage

This afternoon we are experiencing an outage on our ADSL network that manifests as end-users retaining sync but getting either extremely-slow page loads or a complete loss of the ability to surf. Our network engineering staff is currently investigating.  Fusion, dial-up, and other services should be unaffected. — John F

Update: this outage was caused by a card failure. We have switched over to replacement equipment and all affected circuits should be fixed. — Nathan and John F

Customer Database Server Upgrades

Tonight, February 10th at 12:01 AM, the shared customer database servers will undergo routine maintenance which requires a reboot of the servers.  Expected downtime for customer websites that use the shared database service is less than 10 minutes.  This work is in addition to the internal database server upgrades scheduled tonight, but is unrelated.

-Kevan

Update:  All upgrades were completed without event and customer database services are working correctly.

Internal Database Server Upgrades

Tonight, after Support closes, we’re going to begin an invasive upgrade on our primary internal database servers.  The upgrade necessitates a complete dump and restore to new hardware and is expected to take several hours to complete.  The new servers are substantially more powerful and many of our internal systems should see a marked performance improvement.  While the data is being migrated and imported, all signup and member tools will be unavailable but all other services will otherwise remain unaffected.  — Kelsey and William

Update:  The upgrade appears to have gone according to plan and all internal systems are back online at this time.

Changes to our DNS servers ns1/ns2.sonic.net

Starting today ( 8, Feb ) the name servers we hand out to our customers for DNS resolution ( ns1.sonic.net and ns2.sonic.net ) are going to be updated to exclude authoritative data, and be recursive only name servers.

This means that our name servers will behave more like other name servers on the Internet, prior to this change you may have received confusing results when asking our name server for a domain name, and asking Google’s open recursive DNS servers ( for example ) — now you will receive the same results.

Note: customers do not need to update their DNS resolver settings.

–Augie

Intrusive Network Maintenance

Tonight,  Wednesday 8 at 12:01 AM we will be performing some intrusive network maintenance on our backhaul circuits to the Sacramento area. Customers in that area may experience 5-10 minutes of downtime.

-Jared and Nathan