Mail Server Upgrades.

Thu Sep 8 15:15:02 PDT 2005 — Mail Server Upgrades. We deployed four new POP3 servers yesterday in order to maintain 2-N redundancy behind our load balancers. While our previous servers were not experiencing load related issues, these upgrades should provide improved interactive performance. -Kelsey, Nathan and Augie

Night Ops.

Wed Sep 7 19:00:53 PDT 2005 — Night Ops. Tonight from approximately Midnight to 3:00 AM membertools will be unavailable as we work on upgrading sonic.sonic.net’s hardware. We’ll also be briefly working on the load balancers that handle mail services for customers – some normal maintenance done earlier today left them in a odd condition. We have reason to believe that a _very_ small percentage of requests for pop and smtp may be dropped at this time. We anticipate only a minute of downtime while we reboot the load balancers to clear the issue. -Kelsey and Nathan

Update Thu Sep 8 13:07:12 PDT 2005 — The work on the load balancers went as expected with only 14 seconds of downtime. The upgrade of sonic, however, was aborted due to issues with it’s replacement hardware. -Kelsey and Nathan

Broadlink outage on Rosenberg Tower.

Tue Sep 6 12:19:28 PDT 2005 — Broadlink outage on Rosenberg Tower. Sonic.net has been informed of an outage affecting all customers whose Broadlink Wireless connections are routed through the Rosenberg Tower in downtown Santa Rosa. Customers on the Wally Moore tower, and customer connected by non-Broadlink connections should be unaffected. Broadlink technicians are working on this issue right now. – John F

Update Tue Sep 6 12:43:17 PDT 2005 — Broadlink reports a full restoral of service at 12:37pm. – John F and Broadlink

Pop server downtime this morning.

Mon Sep 5 07:43:26 PDT 2005 — Pop server downtime this morning. Between approximately 4:30am and 5:00am this morning, one of the file servers serving the mail cluster lost its beady little mind, and the expected fail-over did not take place. This affected both popmail and mail delivery. No mail was lost.

Resetting the filer’s interface cleared up the problem. We have seen this before, and will be submitting more information to the vendor regarding the failure, in hopes of a speedy resolution. -Scott and Kelsey

Shell Server Policy Change.

Wed Aug 31 16:53:35 PDT 2005 — Shell Server Policy Change. Shell server users are now allowed access to cron on shell.sonic.net. Please use the service appropriately; cron jobs running too frequently or otherwise using excessive system resources will be disabled. -Kelsey and Scott

Phone system problems.

Thu Aug 25 12:32:32 PDT 2005 — Phone system problems. We are having intermittent problems with our phone system causing us to drop calls; we are working to resolve the problem. -Tech Support

Update Fri Aug 26 10:13:33 PDT 2005: Phone system problems persist. We are working with our phone system and telco providers to resolve the dropped-call issue. We have no ETR at this time, but no Internet services are affected. -John F and Jasper C

Update: The phone system issues have been resolved. -Tech Support

LATA 9 DSL Issues.

Fri Aug 19 15:36:07 PDT 2005 — LATA 9 DSL Issues. We have been tracking a capacity problem within our provider’s Stockton network that manifests as light latency and packet loss to some destinations. Only DSL customers in the Stockton area are affected. Our vendor has isolated the trouble and is working to repair the problem. While they have not provided us with an ETR, we expect normal service to be restored within the next couple of days. -Network Operations

Customer Router Maintenance.

Thu Aug 18 18:14:28 PDT 2005 — Customer Router Maintenance. On Saturday, 8/20 @ 12:01AM we will be performing maintenance on one of our customer routers at our 200 Paul Ave. POP in San Francisco. This maintenance will cause a brief loss of connectivity for T1 and dialup customers served from this facility. We expect service to be fully restored within a few minutes following the maintenance. -Network Operations