Fri Dec 1 14:20:05 PST 2006 — Core router maintenance. Tomorrow morning at 12:01AM we will be performing maintenance on one of our core routers located in San Francisco. This operation may interrupt some amount of Internet traffic for a few minutes. -Nathan and Zeke

Tue Nov 21 11:10:30 PST 2006 — Some AT&T DSL customers of all ISPs (including Sonic.net) are currently experiencing slow performance that appears to be related to a network congestion issue in AT&T’s ATM network in California.

The congestion appears to have begun this morning around 3am, apparently when a number of customer routes were moved to a pair of circuits which are over capacity. This has resulted in packet loss and high latency for end-users whose links are being routed over the congested links. Estimated time to repair is one to three hours. – Eli, Nathan **I have now mailed this out with incorrect formatting twice. Please accept my apologies for the noise, though I might add that my co-workers are tremendously amused by this. – Eli

Mon Nov 13 18:13:21 PST 2006 — Santa Rosa DSL Migration. At 12:01am on Wednesday, November 15th we will be migrating all of our DSL subscribers terminating in Santa Rosa to a new aggregation circuit landing in San Francisco. The new OC-12 is four times faster than the existing OC-3. In addition, this change brings our subscribers closer to AT&T’s network core. Downtime should be under 15 minutes, with most folks noticing only a 5 minute outage. -Nathan, Kelsey, Jared and Zeke

Update Wed Nov 15 02:21:27 PST 2006 — The DSL migration was completed successfully. -Nathan, Kelsey, Jared and Zeke

Fri Nov 10 14:24:55 PST 2006 — SpamAssassin Upgrades and Changes. Over the last couple of weeks we began testing a new SpamAssassin module that uses OCR (Optical Character Recognition) to scan attached images for known spam phrases resulting in a silent deployment last week. So far it, along with another 3rd party rule and score set, has proven to be very affective at catching most of the spam that was leaking through with an insignificant number of false positive matches. Also, in response to a user’s request, the ‘Drop Blacklisted Messages’ feature now also will drop messages matching Subject and ‘User in Blacklist To’ lists. -Kelsey

Wed Nov 8 03:18:10 PST 2006 — Mail Service Interruption. For one hour and thirty four minutes this morning inbound mail delivery and retrieval was unavailable do to a failure on one of our four backend message stores. The exact cause of the failure is still unknown and now that service is restored we’re continuing to investigate the situation. This is the first time we’ve ever experienced a failure like this in the 10+ years that we have been using Network Appliance Filers. At this time all mail queued in our systems is delivering to customers mail spools. No mail was lost or not accepted for delivery for any reason related to this event by our MX servers during this outage. -Kelsey

Fri Oct 27 14:13:25 PDT 2006 — Focal POP maintenance. On Tuesday, October 31 at 12:01 AM we will be performing router maintenance at our Focal POP in San Francisco. This maintenance will temporarily disrupt all dial to all our numbers ending in 9811 and 9606. Expected loss of connectivity for those numbers is 5-10 minutes. You can find alternate dial-up numbers at www.sonic.net/popf/ -Jared, Zeke, and Nathan

Update Tue Oct 31 01:59:33 PST 2006 — Focal POP maintenance completed. We have completed our scheduled maintenance at our Focal POP. Total downtime for dialup connectivity served from this POP was approximately 15 minutes beginning at 1AM. -Zeke and Jared

Tue Oct 24 09:41:06 PDT 2006 — LA ATM Maintenance. On Thursday, October 26th at 12:01am we will be performing routine maintenance on the equipment that aggregates our Los Angeles DSL customers. This operation is intended to increase performance and reliability for all our LA members. Downtime will be approximately 15 minutes. -Nathan and Kelsey

Update Wed Oct 25 18:18:16 PDT 2006 — Due to circumstances outside of our control, we are postponing this maintenance until 12:01am on Friday, October 27th. We apologize for the inconvenience. -Nathan and Kelsey

Update Thu Oct 26 19:50:05 PDT 2006 — More delays. Do to some miscoordination with another provider we have to put this maintenance off for another night. It is now scheduled for 12:01AM, Saturday. -Nathan

Update Sat Oct 28 01:50:18 PDT 2006 — This maintenance was completed successfully. -Nathan and Kelsey

Sun Oct 15 21:30:40 PDT 2006 — Core NFS Filer Maintenance. Tonight, shortly after Midnight, we are going to perform some upgrades on the NetApp filers that handle all web, ftp and home directories. This is a multi-stage upgrade that will include a few service interruptions during which all dependent services will be unavailable. These upgrades are being conducted to resolve some issues raised by the software and capacity upgrades done last Friday. -Kelsey and Nathan

Thu Oct 12 16:14:36 PDT 2006 — Core NFS Filer Maintenance. Tonight, shortly after Midnight, we are going to perform software upgrades on the NetApp filers that handle all web, ftp and home directories. This is a multi-stage operation that will include a number of short service interruptions during which all dependent services will be unavailable. -Kelsey and Nathan

Update Fri Oct 13 02:46:59 PDT 2006 — Maintenance completed. Both clustered filers are now running the latest code and have had additional capacity added. -Kelsey and Nathan

Mon Oct 9 12:37:31 PDT 2006 — Planned ATM switch maintenance. Tomorrow evening at 10PM we will be installing an additional linecard in one of our ATM switches. We do not anticipate this operation to be service-affecting, however in the event an issue does arise NOC staff will be at the ready. -Network Operations