Mon Dec 27 16:39:13 PST 2004 — SpamAssassin Upgrades. We upgraded SpamAssassin to version 3.0.2 this afternoon. This upgrade should clear up some occasional problems we’ve had on the dedicated SpamAssassin servers as well as increase it’s accuracy at properly tagging spam. We are currently blocking roughly 56% of all inbound email with or user configurable MTA level RBL filters. SpamAssassin marks roughly another 55% of the remaining messages as spam and routes them to our graymail interface. All users can update their settings for both the MTA filters and SpamAssassin in the membertools. -Kelsey

Sat Dec 25 06:38:27 PST 2004 — DOS Attack. Starting at 5:52am and ending at 6:07am we experienced a relatively large DOS attack. Customers may have noticed some packet loss and latency to parts of the Internet during that window. The attack has stopped, and we’ll be monitoring our network closely for further signs of attack. -Nathan

Thu Dec 23 15:30:52 PST 2004 — Update on DSL problems in Santa Rosa. We are going to reboot and replace the malfunctioning interface with a spare at 15:40 PST today. This, unfortunately, is our best option to resolve the issues as quickly as possible. Total downtime should be a few minutes at most unless we encounter an unexpected problem. -Kelsey, Nathan, Zeke, Jared, Dane and John

Thu Dec 23 14:51:57 PST 2004 — DSL Packet Loss on Santa Rosa SMS. We’ve identified what appears to be a bad ethernet interface in the SMS thats causing as much as 10% packet loss for DSL customers terminated in Santa Rosa. We are currently investigation solutions to resolve this problem that do not involve replacing the card which would require that we reboot equipment potentially leading to an extended complete outage. -Kelsey, Nathan and John.

Fri Dec 17 11:34:46 PST 2004 — Sonic.net is closed today from 12 noon until 3pm for our staff year-end recognition party. We’re sorry for any inconvenience this may cause – please phone us after 3pm today if you need anything! Happy Holidays! -Dane

Fri Dec 17 11:07:54 PST 2004 — problems with dial-up numbers ending in 9606. Equipment failure caused connectivity problems on dial-up numbers ending with 9606. The equipment was rebooted and is now working properly. – Russ

Fri Dec 17 17:00:54 PST 2004 — DSL Problems in Stockton. Our network provider in Stockton where our DSL termination equipment is colocated is currently experiencing significant packet loss. They have been informed and will have the trouble resolved shortly. -Kelsey, Nathan

Thu Dec 16 10:15:44 PST 2004 — Internal Packet Loss. Last night we discovered some rather severe packet loss in our server backend SAN network that affected a number of hosts. In particular, two of our SpamAssassin servers and both FTP servers were affected by this loss. This may have exhibited in problems using our FTP servers and some mail messages not being filtered by SpamAssassin. The other hosts affected by the problem do not support public services. We resolved the problem this morning when we found one of the SAN switches was flopping back and forth between it’s primary and secondary uplinks to the other switches. -Kelsey and Nathan

Wed Dec 15 17:21:31 PST 2004 — Kernel Security Updates. We are upgrading the Linux kernels on all of our production servers to resolve two recently announced kernel exploits. While each server is rebooted some customer may notice small server outages while the clusters failover. -Kelsey, Scott, and Kevan.

Wed Dec 15 16:24:49 PST 2004 — Mail Issues. For a short time yesterday a small percentage of mail was incorrectly rejected due to being on the SBL spamhaus.org RBL. We disabled the use of this RBL in our MTA filters so we could track down the problem which appears to either be a bug in our custom software or our internal DNS servers. It was not due to hosts being incorrectly listed in the SBL. This only affected people using the SBL to filter their inbound mail – part of our default filter set – and did not affect anyone who had specifically turned it off. We’re going to re-enable the SBL once we’ve added some additional checks to our code. -Kelsey, Russ and Kevan