Month: September 2005

Night Ops.

Wed Sep 7 19:00:53 PDT 2005 — Night Ops. Tonight from approximately Midnight to 3:00 AM membertools will be unavailable as we work on upgrading sonic.sonic.net’s hardware. We’ll also be briefly working on the load balancers that handle mail services for customers – some normal maintenance done earlier today left them in a odd condition. We have reason to believe that a _very_ small percentage of requests for pop and smtp may be dropped at this time. We anticipate only a minute of downtime while we reboot the load balancers to clear the issue. -Kelsey and Nathan

Update Thu Sep 8 13:07:12 PDT 2005 — The work on the load balancers went as expected with only 14 seconds of downtime. The upgrade of sonic, however, was aborted due to issues with it’s replacement hardware. -Kelsey and Nathan

Broadlink outage on Rosenberg Tower.

Tue Sep 6 12:19:28 PDT 2005 — Broadlink outage on Rosenberg Tower. Sonic.net has been informed of an outage affecting all customers whose Broadlink Wireless connections are routed through the Rosenberg Tower in downtown Santa Rosa. Customers on the Wally Moore tower, and customer connected by non-Broadlink connections should be unaffected. Broadlink technicians are working on this issue right now. – John F

Update Tue Sep 6 12:43:17 PDT 2005 — Broadlink reports a full restoral of service at 12:37pm. – John F and Broadlink

Pop server downtime this morning.

Mon Sep 5 07:43:26 PDT 2005 — Pop server downtime this morning. Between approximately 4:30am and 5:00am this morning, one of the file servers serving the mail cluster lost its beady little mind, and the expected fail-over did not take place. This affected both popmail and mail delivery. No mail was lost.

Resetting the filer’s interface cleared up the problem. We have seen this before, and will be submitting more information to the vendor regarding the failure, in hopes of a speedy resolution. -Scott and Kelsey