Wed Jul 18 17:58:59 PDT 2001 — Strange routing problem. The issue earlier today turned out to be due to a strange denial of service attack (DoS) sourced from a colocated customer. Bandwidth used by the attack, at times, exceeded 60 megabits/sec. Neither our DoS detector or the usual graphs showed the DoS, as the traffic took a path through new parts of our redundant network. We know now where to look for this type of attack, and will be watching for it in the future.
As part of troubleshooting this issue, we rebooted our core switch — unfortunately, the Black Diamond crashed on boot. Booting the other image stored in the switch worked — however, it is newer code, from which we’ve seen some problems before related to icmp. (It is, however, a general deployment release of the BD’s OS.) We also discovered some curious misbehaviors that it exhibits with our server vlan, discovered a misconfiguration in one of the core router’s Fast Ethernet interfaces, optimized routing for our mail servers, and generally gave the network a serious once-over.
-Scott, Kelsey, Eli, Nathan, Russ