Month: July 2001

Thu Jul 19 14:26:39 PDT 2001 — “Code Red” worm results. As you may have already heard, the tenacious “Code Red” worm is winding its way through the Internet. The worm propagates via Microsoft IIS web servers, so Sonic.net web servers are invulnerable to the worm.

However, that doesn’t prevent the worm from trying to break into Sonic.net’s web servers. We are currently logging about two such attempts a second, and there have been over 33170 such attacks since midnight. Again, Sonic.net servers are unaffected by these attacks. This is obviously a very tenacious worm, and we recommend that IIS administrators ensure that their installations are up-to-date.

For more info about the “Code Red” worm, please visit the following news story:

news.cnet.com/news/0-1003-200-6616583.html

Update: as of now (3:10pm), 32840 unique hosts have attempted to break into our web servers, each which is almost certainly running a copy of the worm.

-Scott, Kelsey, Eli

Wed Jul 18 17:58:59 PDT 2001 — Strange routing problem. The issue earlier today turned out to be due to a strange denial of service attack (DoS) sourced from a colocated customer. Bandwidth used by the attack, at times, exceeded 60 megabits/sec. Neither our DoS detector or the usual graphs showed the DoS, as the traffic took a path through new parts of our redundant network. We know now where to look for this type of attack, and will be watching for it in the future.

As part of troubleshooting this issue, we rebooted our core switch — unfortunately, the Black Diamond crashed on boot. Booting the other image stored in the switch worked — however, it is newer code, from which we’ve seen some problems before related to icmp. (It is, however, a general deployment release of the BD’s OS.) We also discovered some curious misbehaviors that it exhibits with our server vlan, discovered a misconfiguration in one of the core router’s Fast Ethernet interfaces, optimized routing for our mail servers, and generally gave the network a serious once-over.

-Scott, Kelsey, Eli, Nathan, Russ

Wed Jul 18 13:36:38 PDT 2001 — Strange routing problem. We have disabled one of our edge routers in response to intermittent packet loss between our internal network and the edge. We are still determining the source of the trouble. For the time being, Internet performance may be impacted. We will update as soon as we have a resolution or ETR. – Scott, Eli, Kelsey

Fri Jul 13 13:31:45 PDT 2001 — Our MySQL accounting server is offline right now for emergency maintenance. While it is down our member tools are inaccessible but all of our services should otherwise be unaffected. We should have it back online in 10 to 20 minutes. -Russ and Kelsey

Wed Jul 11 13:08:08 PDT 2001 — We received a response from RedBack about a the crash yesterday. Apparently we tickled a known bug in the new OS release which is slated to be fixed in the ‘next major release.’ We are currently investigating to see if there are any workarounds so we can prevent the same crash from occurring until we have a new OS. -Kelsey

Tue Jul 10 16:26:51 PDT 2001 — The RedBack ATM switch crashed and rebooted, incurring about ten minutes of downtime for many broadband connected customers. We’ve got a crash dump file, and will forward it to RedBack for diagnostics. -Dane

Sun Jul 8 00:34:21 PDT 2001 — The upgrade to the RedBack ATM switch was performed successfully. Downtime was 10 minutes. Russ and Kelsey

Sat Jul 7 20:40:42 PDT 2001 — The VeriSign certificate for ssl.sonic.net has expired, we are in the processes of getting this certificate renewed. VeriSign assures us the new certificate will be available in 2 days. We will post updates to this here in the MOTD. -Steve and Scott

UPDATE: The new SSL Certificate has been installed, all is good -Steve

Fri Jul 6 18:47:31 PDT 2001 — We will be performing a software upgrade to our Red Back ATM Switch on Saturday night at or shortly after midnight. This should resolve some minor compatibility issues with Pac Bell’s ATM network.

Expected downtime will be 10 minutes. This will affect DSL and WDSL connections. -Russ and Kelsey