Thu Feb 15 21:33:58 PST 2007 — Transport bounce. Between 9:05 and 9:20pm this evening, we experienced a number of brief transport outages resulting in packet loss. Customers may have experienced transient connectivity problems to some sites. We are currently working to mitigate and abate this. — Nathan, Zeke, Matt, and John F
Changes to SuperNews NNRP Services.
Thu Feb 15 17:40:41 PST 2007 — Changes to SuperNews NNRP Services. Today we have implemented the authentication changes to our SuperNews NNRP access that we announced on Feb 6. Users connecting to supernews.sonic.net will need to supply their username in the format of ‘username@sonic.net’ and normal account password. -Kelsey and Jared
More traffic disruption.
Tue Feb 13 09:06:29 PST 2007 — More traffic disruption. The same problem we experienced with one our our upstream providers network dropping large packets on Friday has cropped back up. We’re currently working with the provider now and hope that they are able to have it resolved quickly. -Kelsey and Nathan
Update Tue Feb 13 10:29:43 PST 2007: Pending full resolution from the upstream provider we rerouted affected traffic. We’ll continue to keep a very close eye on things. -Nathan and Kelsey
Internet traffic disruption.
Fri Feb 9 11:53:08 PST 2007 — Internet traffic disruption. We’re currently tracking an issue inside one of our upstream provider’s networks that manifests itself as an inability to receive packets larger than a certain size. 1476 byte packets appear to be forwarded correctly, while 1477 byte and longer packets are dropped. We’ve identified the broken equipment in the provider’s network and have re-routed traffic to mitigate the problem. We are also working with the provider to solve the issue. We’ll post an update when more information is available. -Nathan and Kelsey
Update Fri Feb 9 14:42:10 PST 2007: Our upstream provider has fixed the trouble, which turned out to be a partially failed router in San Jose. -Nathan
Inbound mail delays.
Fri Feb 9 09:15:51 PST 2007 — Inbound mail delays. We are seeing delays in the delivery of inbound mail due to unusually high spam volume overnight. We are bringing additional resources to bear, and expect resolution over the course of the day. – Eli, Operations
Changes to SuperNews NNRP Services.
Tue Feb 6 11:34:04 PST 2007 — Changes to SuperNews NNRP Services. Friday morning we are going to change our SuperNews NNRP access to use username based authentication in place of the existing IP based access control lists. Users connecting to supernews.sonic.net will need to supply their username in the format of ‘username@sonic.net’ and normal account password. For more information please see www.sonic.net/features/news/ -Kelsey
CRAC Testing and Evaluation.
Wed Jan 31 10:25:29 PST 2007 — CRAC Testing and Evaluation. We will be conducting some tests on our current data center and colo facility CRAC (Computer Room Air Conditioning) to see how our redundant systems will behave in a failure scenario. This should not have any impact on our customers. -Kelsey, Nathan and Russ
Hotspot DSL outage.
Sun Jan 28 19:18:00 PST 2007 — Hotspot DSL outage. Currently, we are experiencing difficulties with the hardware that terminates our Hotspot-enabled DSL lines. Customers who have Hotspot-enabled DSL lines will have no connectivity at this time. We are working on repair or replacement of the affected hardware, estimated time of repair is one hour. We apologize for the inconvenience. -Nathan and Jared
Update Sun Jan 28 19:53:52 PST 2007 — Hotspot service has been restored. Sorry for the downtime! -Nathan and Jared
Customer router reload.
Wed Jan 17 21:14:30 PST 2007 — Customer router reload. One of our customer routers in Santa Rosa crashed and reloaded at approximately 9:10 this evening. Only customers with T1s served by this router would have seen 5 minutes of downtime. We will be investigating the cause of this crash and taking action as necessary. -Jared
Minor router maintenance.
Tue Jan 9 18:08:14 PST 2007 — Minor router maintenance. We will be removing some interface cards from two of our core routers in Santa Rosa tonight shortly after 11 PM. There may be up to 30 seconds of unreachability to a few servers located in our Santa Rosa datacenter as we remove these cards. -Jared, Zeke, and Matt
Update Tue Jan 9 23:41:51 PST 2007 — Maintenance complete. We have completed the maintenance without incident. Any service disruptions would have been minimal. -Jared and Zeke