Fusion Outage – Eastern Santa Rosa

This morning, roughly for the hour between 8:00am-9:00am, Fusions users in eastern Santa Rosa where unable to use their connection due to an issue with DHCP. We have located the problem and will be working with our vendor to prevent this issue from coming up again.

– Noc

Fusion/FlexLink Intrusive Maintenance

Update (3:20AM): This maintenance is now complete. A very small number of customers in the Berkeley area are currently offline due to a failed ADSL2+ card. We are dispatching a technician first thing in the morning to replace the problematic card.

This evening, beginning at 11:59PM, we will be performing intrusive maintenance on equipment serving Fusion and FlexLink customers in Richmond, Albany, Alameda, Oakland and Berkeley . Expected downtime is less than 15 minutes. A small portion of customers may need to reboot their DSL modems after these code upgrades take place.

– Robbie

Webcluster Instability

We are currently investigating the cause of an instability issue on our web cluster. At 10:45 this evening, some of our hosted websites became intermittently reachable, and continued to have problems for roughly 5 minutes. At this time we have restored reach-ability, and we are looking for the source of the problem to prevent further interruptions.

Grant and William, SOC

Legacy DSL Outage LA Area

Update( 7:58 PM): While the issue turned out to be bigger than expected, all affected customers should now be online.

A small portion of legacy DSL customers in the LA area are currently offline due to a possible atm port failure. We are currently in the process of migrating customers off of the affected device.

– Robbie

Legacy DSL Maintenance

Update( 12:12AM ): This maintenance is now complete.

Tonight, beginning at 11:59 PM, we will be performing maintenance on equipment serving legacy DSL customers in the LA area. Expected downtime is around 15 minutes.

– Robbie

Fusion/FlexLink Outage

Today, August 3, starting at 5:20pm, links serving Fusion/FlexLink customers in the Fort Brag and Mendocino areas went down. We are currently investigating the issue and will update as soon as we have more information.

Update: A traffic accident in Mendocino caused a fiber cut that appears to be the root cause of this outage. Still no ETR, but we will update as more information becomes available.

Update: AT&T has found the cable cut and has dispatched to start repairs. They will attempt to pull slack in the cable in order to restore service tonight, however there is no ETR until the extent of the damage has been fully assessed.

Update: A splice team is on site with an ETR of 7:15am

Update: AT&T has encountered trouble while restoring service with a new ETR of 10:45am

Update: AT&T splicers are still working in an effort to restore service, currently no ETR

Update: AT&T has discovered that the affected fiber cable was damaged in more than one place. They are currently working to repair the additional damage.

Update (9:25pm): AT&T has notified us that they are currently working on our fiber spans

Update: (12:52am): AT&T was unable to use existing damaged cable, and has an ETA of 3:00am for 6500ft of new cable so they can begin splicing

Update (9:57am): AT&T has located cable and brought it onto the job site. Latest ETR is 4:00pm

Update (2:53pm): Service appears to be restored, however we have not received confirmation from AT&T that our circuits are stable. We will update again as soon as any additional information is available.

Update (11:26pm): At 11:15pm, connectivity was lost for approximately 10 minutes while finalizing repairs. Redundant circuits have been restored and we do not anticipate any further service interruption. We apologize for this inconvenience and appreciate your patience. – Tomoc and the Sonic Team

-Tomoc

FlexLink Outages – San Francisco

We are currently investigating an outage involving some FlexLink customers in the San Francisco area following last nights code upgrades. We will update this post when we believe we have found the cause and solution to this issue.

– NOC

Sonic.net Company Picnic

Sonic.net will close early for our 20th anniversary party, Monday August 4th.

The domain name Sonic.net was registered on July 26th 1994. Bill Clinton was president. The Channel Tunnel opened between France and England. Brazil won the World Cup. Forrest Gump was in theaters. Ace of Base was on the radio. The video game Warcraft was first released. Eight megabytes of RAM was pretty impressive. VHS was a thing. We’ve see a lot of changes in two decades, but with the support of our customers, coworkers, and community we look forward to even more.

Our offices and phone center will be closed Monday at 4:30pm for our company picnic and 20th anniversary celebration. We are sorry for any inconvenience, and will be back on duty at 6:00am Tuesday morning. Thank you for your continued support.

System Maintenance

UPDATE: Work is complete.

Tonight at 11:59 pm, we will be updating several customer facing systems. The services impacted will include:

  • IMAP
  • Web Hosting
  • MySQL
  • IPv6 Tunnels

Interruptions are expected to be brief.

 

-Grant and SOC