Wed Feb 28 15:52:46 PST 2007 — DSL Termination Upgrades. Tonight, shortly after Midnight, some DSL circuits terminated on our equipment in San Francisco may experience a 5 to 10 minute outage as we perform some upgrades and regular maintenance on the termination equipment. -Kelsey and Nathan
Update Thu Mar 1 01:07:21 PST 2007 — We’re done! Maximal customer downtime was 4 minutes, with most folks seeing only a minute or two of connectivity loss. -Nathan and Kelsey
Wed Feb 21 13:37:08 PST 2007 — Planned down time for maintenance. Tonight at midnight one of our core switches will be taken down briefly; during this time reachability to Sonic.net services may be interrupted briefly. Also during this time the customer Database System (custsql.sonic.net) will be taken down for hardware maintenance; expected down time is less than 15 minutes. -Augie and Jared.
Update Thu Feb 22 01:31:14 PST 2007 — Maintenance complete and all services are now restored. -Augie and Jared.
Tue Feb 20 18:48:53 PST 2007 — Brief Web Cluster Instability. For a short period some issues in our webcluster caused some sites to fail to load. Service were restored quickly. -Kelsey and Nathan
Fri Feb 16 18:30:26 PST 2007 — DSL Service disruption. One of the OC-12s that terminates DSL subscribers in LATA 1 (most of Northern California) is offline. Our NOC staff is already onsite at the POP and is working with AT&T and the facility providers to ensure a speedy fix. -Kelsey, Alan and the 7065 NOC.
Update Fri Feb 16 19:40:40 PST 2007 — Service restored. As of approximately 7:30pm all services on the affected OC-12 have been restored. The problem was determined to be faulty cross-floor fiber patching at our 200 Paul Ave. facility. To expedite service restoration we brought some otherwise idle gear into service, which allowed us to bring customers back by eliminating intermediate fiber connections. We will be scheduling a maintenance window in the near future to clean the cross-floor run and return the circuit to its original configuration. We apologize for the late update to this entry, not to mention the downtime! -Nathan, Zeke, Matt and Juston
Thu Feb 15 21:33:58 PST 2007 — Transport bounce. Between 9:05 and 9:20pm this evening, we experienced a number of brief transport outages resulting in packet loss. Customers may have experienced transient connectivity problems to some sites. We are currently working to mitigate and abate this. — Nathan, Zeke, Matt, and John F
Thu Feb 15 17:40:41 PST 2007 — Changes to SuperNews NNRP Services. Today we have implemented the authentication changes to our SuperNews NNRP access that we announced on Feb 6. Users connecting to supernews.sonic.net will need to supply their username in the format of ‘username@sonic.net’ and normal account password. -Kelsey and Jared
Tue Feb 13 09:06:29 PST 2007 — More traffic disruption. The same problem we experienced with one our our upstream providers network dropping large packets on Friday has cropped back up. We’re currently working with the provider now and hope that they are able to have it resolved quickly. -Kelsey and Nathan
Update Tue Feb 13 10:29:43 PST 2007: Pending full resolution from the upstream provider we rerouted affected traffic. We’ll continue to keep a very close eye on things. -Nathan and Kelsey
Fri Feb 9 11:53:08 PST 2007 — Internet traffic disruption. We’re currently tracking an issue inside one of our upstream provider’s networks that manifests itself as an inability to receive packets larger than a certain size. 1476 byte packets appear to be forwarded correctly, while 1477 byte and longer packets are dropped. We’ve identified the broken equipment in the provider’s network and have re-routed traffic to mitigate the problem. We are also working with the provider to solve the issue. We’ll post an update when more information is available. -Nathan and Kelsey
Update Fri Feb 9 14:42:10 PST 2007: Our upstream provider has fixed the trouble, which turned out to be a partially failed router in San Jose. -Nathan
Fri Feb 9 09:15:51 PST 2007 — Inbound mail delays. We are seeing delays in the delivery of inbound mail due to unusually high spam volume overnight. We are bringing additional resources to bear, and expect resolution over the course of the day. – Eli, Operations
Tue Feb 6 11:34:04 PST 2007 — Changes to SuperNews NNRP Services. Friday morning we are going to change our SuperNews NNRP access to use username based authentication in place of the existing IP based access control lists. Users connecting to supernews.sonic.net will need to supply their username in the format of ‘username@sonic.net’ and normal account password. For more information please see www.sonic.net/features/news/ -Kelsey