Month: January 2001

Hiccup with load balancing.

Wed Jan 31 16:57:20 PST 2001 — Hiccup with load balancing. During a routine change in our network topology that _should_ have gone unnoticed, our load-balancing switch, Piggy, decided it didn’t want to do its job anymore. This is a change that we have made in the past with no problems whatsoever. A reset of the switch brought back mail and web services. Downtime was a very long and frustrating 5 minutes while we tracked down the problem — we apologize for the trouble, and rest assured we won’t be moving etherchannel interfaces between switch blades in the middle of the day anymore. -The Sonic Operations Team and many screaming voices from Tech Support

We experienced a brief DOS attack over the…

Tue Jan 30 13:23:22 PST 2001 — We experienced a brief DOS attack over the last few minutes. It appears that one of our colo customer’s servers had been compromised. They’ve been taken off of the network and the issues have been resolved. -Kelsey, Eli and Scott.

We’ve made some fixes to ‘twig’, our new web…

Mon Jan 29 12:52:34 PST 2001 — We’ve made some fixes to ‘twig’, our new web based email reader client to address issues with it moving email to a location where POP clients could find it. This resolves compatibility issues for customers who sometimes use the web-mail tool, and sometimes use POP email clients such as Outlook or Eudora. Please let us know if you observe any troubles by posting to the local twig discussion group:

news://news.sonic.net/sonic.help.twig

You can access twig at:

www.sonic.net/twig/

-Nathan and Dane

msn.com nameservers down.

Wed Jan 24 10:03:25 PST 2001 — msn.com nameservers down. We’ve received quite a few calls about msn.com not answering — investigation reveals that msn’s nameservers are unreachable. This also affects microsoft.com. We’ve confirmed that the problem isn’t on our network, as the same non-results occur on two other remote networks. (This is also confirmed by a rather wry message on slashdot.org.) UPDATE: Many folks use msn.com or microsoft.com as their startup page in their browser. We recommend changing this to www.sonic.net, as this will allow one to view status messages (like this one), when these significant events occur. -Scott, Randy, the other Scott, Eli, Dane, Steve, and Kevan

msn.com nameservers down.

Wed Jan 24 09:39:25 PST 2001 — msn.com nameservers down. We’ve received quite a few calls about msn.com not answering — investigation reveals that msn’s nameservers are unreachable. We’ve confirmed that the problem isn’t on our network, as the same non-results occur on two other remote networks. (This is also confirmed by a rather wry message on slashdot.org.) -Randy, Scott, the other Scott, Eli, Dane, and Kevan

Night Operations Complete.

Wed Jan 24 01:14:20 PST 2001 — Night Operations Complete. Our core switch has been upgraded and the issues with the netapp cluster have been resolved. We did not experience more than a few minutes of downtime. -Kelsey & Matt

Night Operations.

Tue Jan 23 14:50:47 PST 2001 — Night Operations. Tonight we are going to have a few brief service outages while we upgrade our core Extreme Networks BlackDiamond switch’s management modules. The upgrade affords us a higher port density and an increased feature set including L4 switching capabilities. While the SE from Extreme Networks is working on the switch we will also be taking Freezer and Icebox down to resolve a cluster configuration problem that we introduced when we added the second shelves last week. We will begin work shortly after Midnight and don’t anticipate more than a few minutes of downtime to complete both projects. Please note that while the switch is offline (about 90 seconds) Sonic.net will be off of the net. While the two NetApps are being rebooted (about 120 seconds), email, shell and web hosting services will pause. -Kelsey