Wed Sep 3 09:27:02 PDT 2003 — Storm outage on Broadlink’s Barham tower. It appears the thunderstorm that rolled through the Santa Rosa area this morning has caused a problem with Broadlink’s Barham tower. This has resulted in some Broadlink outages. PG&E can not “confirm nor deny” power problems to the site. A Broadlink engineer is one his way to the tower. -Scott, tech support, and Broadlink

Wed Sep 3 20:45:18 PDT 2003 — 522-1003 and other numbers served by our one of our San Francisco POPs have been Intermittently returning all circuits are busy messages since earlier this evening. This is a failure due to a failure on one of the eight router cards which handles three of PRI’s of modems. We are working to resolve the problem and hope to have it resolved shortly. -Operations

Tue Sep 2 15:26:43 PDT 2003 — Mail.sonic.net MX record removed. Last week, in an attempt to compensate for some broken email clients, we put an MX record for mail.sonic.net pointing at mx.sonic.net. This apparently broke sendmail “smarthosting” for some of our customers. We have now removed that MX record. Customers are reminded that “customer@mail.sonic.net” is not a valid return address to include in email. Please check your configuration. -Operations

Tue Sep 2 14:52:57 PDT 2003 — Dialup authentication failing. The authentication process for dialup users has failed. We are switching all of our dialup servers to new authentication servers and they are gradually returning to service. We expect them to all be working by 3 PM today. -Russ, Nathan and John

Update: This problem is resolved as of 3:20 PM.

Thu Aug 28 17:56:00 PDT 2003 — Delayed Email: We discovered ~2,500 inbound messages that had gotten wedged in our mail queues. For some unknown reason these messages were not processed by sendmail’s queue running processes but delivered correctly when explicitly sent from the queue. The overwhelming majority of the messages delayed were SPAM. -Sonic Operations

Mon Aug 25 08:35:26 PDT 2003 — Our San Francisco POP is back up. Power was restored at about 8:10 this morning. The dialup numbers served from that facility are currently in operation. -John

Mon Aug 25 03:00:27 PDT 2003 — POP outage. The 200 Paul POP in San Francisco is currently offline. We are working to restore connectivity as soon as possible. -Nathan

Update: This appears to be a facility-wide power outage in SF. We have re-routed all traffic that was entering and leaving Sonic’s network via 200 Paul to our other Internet links. Currently, dialup numbers served out of 200 Paul (707-522-1003, among others) are returning busy signals. However, backup numbers are available. To get a list of POPs in your area, visit:

www.sonic.net/cgi-bin/pops.pl

-Nathan

Sun Aug 24 13:23:13 PDT 2003 — Altnews problems. One of our Usenet news services is having problems. The system reachable as altnews.sonic.net had hardware problems during a recent system move and is waiting for repairs. Article retention and completeness has been affected. Until altnews is repaired, please use our other Usenet servers: news.sonic.net and supernews.sonic.net. -John

Sat Aug 23 09:57:09 PDT 2003 — Mail server upgrade: The mail server conversion will start at 10:30 am PDT today. This migration is expected to cause some delays in delivery of messages. If you experience this, please wait and try again in a short time. Tech support can not expedite delivery of email to your mail box during the migration.

For more information, please see:

www.sonic.net/whatsnew/mailservers/

If your email client reports an actual error when downloading your messages, please call tech support. We may be experiencing higher than usual call volumes, so please be patient. -Operations and Tech Support

Update 11:11:24 PDT 2003 — We have migrated to the new POP servers and new email is being delivered on the new servers. We have corrected one small problem and are working on another one involving mixed case logins. We expect to resolve that shortly.

Update 12:18:57 PDT 2003 — So far everything looks good. We are beginning the migration of old email spools to the new system now. -Operations

Update 14:08:50 PDT 2003 — 98% of mailboxes converted. We are converting the mailboxes in smallest to largest size order. Only the customers with the largest mailboxes are left to do. -Operations

Sat Aug 23 15:22:12 PDT 2003 — Mailbox conversion done. The regular mailboxes have all been converted to the new format. We are now converting folders from webmail. We estimate this will take about three hours. -Kevan, Kelsey and John