Month: November 2003

Sun Nov 30 16:45:22 PST 2003 — Usenet Services Restored: news.sonic.net is fully functional again. As it turns out, our NNTP feed server had a corrupted spool. That spool is dedicated to all non-binary messages. Until the problem was tracked down and resolved it would not transit non-binary posts. It appears that all locally posted articles during this time have been handed out to our peers and that missing posts are being filled in. It’s unlikely that 100% of the old inbound messages will reach us. -Kelsey

Sat Nov 29 21:16:29 PST 2003 — Something strange with Usenet. Folks have been reporting a lower Usenet volume than usual — so far, we haven’t found a definitive cause for this. However, one change on our news reader box might have solved the problem.

We apologize for any Usenet strangeness you might encounter, and we continue to investigate. -Dane, Kelsey, and Scott

Sat Nov 29 16:28:52 PST 2003 — Sebastopol busy signals: Equipment failure and problems with the hunt sequence on our Sebastopol dial-up system caused busy signals on 707-823-8812. The problem has been repaired and the number is no longer giving busy signals. This system will soon be replaced with a new, more fault tolerant, system which will prevent this type of problem. The new equipment has already been delivered and the new lines are slated for activation next week. Transition to the new system will take place after a week or two of testing. Our other dial-up numbers have already be upgraded to fault tolerant systems – Russ

Wed Nov 26 10:31:23 PST 2003 — Web, SSL and FTP service issues: For a yet undetermined cause, the tool which controls users bandwidth allocation and that locks out access to a user’s files if that user has enabled quota protection, locked out the root directories of our public web, ssl and ftp servers. We’ve taken steps to assure that this will not happen again, and are continuing to investigate why the tool failed. -Kelsey

Wed Nov 26 08:44:27 PST 2003 — SSL and Web Mail issue: We are experiencing an issue with our SSL server that drives our web mail application. Currently, the tool is off-line while we make repairs. -Matt

Tue Nov 25 10:23:47 PST 2003 — SpamAssassin Changes: In order improve the overall quality and performance of our primary anti-spam filters we’ve disabled one of it’s features; The AWL, or auto-whitelist, keeps track of all addresses that mail has been received from along with statistics about that senders mail profile. This information is used to shift a given message’s score to the senders mean. This can be an effective tool in helping reduce false positives, but, as implemented currently, it has a number of flaws. The most significant flaw is run away growth which eats into users’ disk quotas; it never deletes records from the database.

One of the side effects of the ever-increasing AWL database size is increased server IO load. This morning, the IO load generated by the AWL on freezer, the NetApp filer that they are stored on, was approaching 90% of the filer’s total capacity. As freezer slows down under these high load conditions it cascades to high load on the SpamAssassin servers, mail servers and results in unfiltered mail.

We are working on a major SpamAssassin upgrade that we expect to launch around the New Year. Hopefully, if we feel that the AWL is still a valuable feature of SpamAssassin, we’ll be able to improve and re-enable it at this time. -Kelsey

Sun Nov 23 19:24:00 PST 2003 — Limited dialup service interruption. Failure of a network access server caused temporary interruption of dialup service to approximately 1% of Sonic.net dialup customers. The server has been re-booted and service has been restored. The server will be replaced as soon as possible as a precaution. – Kelsey, John and Russ

Wed Nov 19 13:05:34 PST 2003 — Brief outbound mail delivery interruption. Our outbound mail servers required a reboot. During this time customers attempting to send mail will have had their connection to the mail server refused. The outage lasted only a few minutes; normal service has been restored. – Operations

Tue Nov 18 18:08:28 PST 2003 — We are having problems with our Voice trunk provider, which is causing a number of dropped calls to our Tech Support lines. Focal is working with SBC to determine the source of the problem, and we do not have an estimated time of repair. If you have any problems reaching Support, please try your call again – We also have Online Support available, and plenty of reps standing by – www.sonic.net/livehelp — Eli, Jared, Support Staff

Mon Nov 17 10:26:47 PST 2003 — There are still some Broadlink customer outages. Broadlink service is partially restored, and the Broadlink staff are working hard to get service restored everywhere. -Operations.

Update: Mon Nov 17 12:26:02 PST 2003 — Most Broadlink issues have been resolved. There is still a small isolated network with problems. More info will be posted later. If you are still having problems with your Broadlink service, please call tech support. -Operations

Update: Mon Nov 17 13:55:08 PST 2003 — Broadlink is now showing all customers up and running. -Operations