Night Operations.

Thu Feb 1 11:15:04 PST 2001 — Night Operations. Tonight at 2am we will be performing maintenance and upgrades on our backbone. This will affect DSL customers and ATM T1 customers. We also _might_ have the same problem we had yesterday afternoon when moving an etherchannel interface, but think that we have worked out the problems with that operation. Downtime should be less than 10 minutes for the SMS 1800 maintenance and upgrade; and less than a second for the etherchannel move — if the load balancing switch doesn’t require a reset. If Piggy does require a reset, downtime could be as much as a minute for load-balanced redundant services, such as email and web. -Scott and Kelsey

Covad scheduled maintenance.

Thu Feb 1 16:32:13 PST 2001 — Covad scheduled maintenance. On Tuesday February 6th between 2 and 4 AM PST, Covad will be doing card upgrades in their San Francisco facility. End users served from this facility (which includes virtually all Sonic.net Covad customers) will be subject to a brief outage during this maintenance. -Dane and Covad support

Hiccup with load balancing.

Wed Jan 31 16:57:20 PST 2001 — Hiccup with load balancing. During a routine change in our network topology that _should_ have gone unnoticed, our load-balancing switch, Piggy, decided it didn’t want to do its job anymore. This is a change that we have made in the past with no problems whatsoever. A reset of the switch brought back mail and web services. Downtime was a very long and frustrating 5 minutes while we tracked down the problem — we apologize for the trouble, and rest assured we won’t be moving etherchannel interfaces between switch blades in the middle of the day anymore. -The Sonic Operations Team and many screaming voices from Tech Support

We experienced a brief DOS attack over the…

Tue Jan 30 13:23:22 PST 2001 — We experienced a brief DOS attack over the last few minutes. It appears that one of our colo customer’s servers had been compromised. They’ve been taken off of the network and the issues have been resolved. -Kelsey, Eli and Scott.

We’ve made some fixes to ‘twig’, our new web…

Mon Jan 29 12:52:34 PST 2001 — We’ve made some fixes to ‘twig’, our new web based email reader client to address issues with it moving email to a location where POP clients could find it. This resolves compatibility issues for customers who sometimes use the web-mail tool, and sometimes use POP email clients such as Outlook or Eudora. Please let us know if you observe any troubles by posting to the local twig discussion group:

news://news.sonic.net/sonic.help.twig

You can access twig at:

www.sonic.net/twig/

-Nathan and Dane

msn.com nameservers down.

Wed Jan 24 10:03:25 PST 2001 — msn.com nameservers down. We’ve received quite a few calls about msn.com not answering — investigation reveals that msn’s nameservers are unreachable. This also affects microsoft.com. We’ve confirmed that the problem isn’t on our network, as the same non-results occur on two other remote networks. (This is also confirmed by a rather wry message on slashdot.org.) UPDATE: Many folks use msn.com or microsoft.com as their startup page in their browser. We recommend changing this to www.sonic.net, as this will allow one to view status messages (like this one), when these significant events occur. -Scott, Randy, the other Scott, Eli, Dane, Steve, and Kevan

msn.com nameservers down.

Wed Jan 24 09:39:25 PST 2001 — msn.com nameservers down. We’ve received quite a few calls about msn.com not answering — investigation reveals that msn’s nameservers are unreachable. We’ve confirmed that the problem isn’t on our network, as the same non-results occur on two other remote networks. (This is also confirmed by a rather wry message on slashdot.org.) -Randy, Scott, the other Scott, Eli, Dane, and Kevan

Night Operations Complete.

Wed Jan 24 01:14:20 PST 2001 — Night Operations Complete. Our core switch has been upgraded and the issues with the netapp cluster have been resolved. We did not experience more than a few minutes of downtime. -Kelsey & Matt