Month: August 2006

Tue Aug 29 09:42:31 PDT 2006 — Broadlink Communications outage. Some customers are being affected by an outage on a sector of the Barham tower. Broadlink is currently working on this issue but does not currently have an estimated time of repair.

Update: Tue Aug 29 10:29:41 PDT 2006 — The outage has been traced to a faulty antenna serving the affected sector. Broadlink is currently replacing this and expect completion by this afternoon. – Jasper C.

Update: Tue Aug 29 16:36:11 PDT 2006 — The antenna has been replaced, and service should be restored to all affected customers.

Mon Aug 21 11:53:22 PDT 2006 — Inbound Mail Server Delays. Our inbound mail complex has been experiencing some trouble over the last hour or two this morning. Although we have yet to identify the root cause we have some work arounds in place. All locally queued mail is being processed now and normal mail flows have been restored. -Kelsey, Jared and Dan

Wed Aug 16 12:47:23 PDT 2006 — DoS Attack Featuring Our Webcluster. A brief but large DoS has destabilized our web server cluster which handles most customer sites and includes www.sonic.net. While it appears the DoS attack is already over it takes a few minutes for the complex of servers to get back into order. We will continue to pay special attention until the system stabilizes. -Kelsey, Nathan, Zeke and Jared

Mon Aug 14 15:19:29 PDT 2006 — Broadlink Communications outage. In the Santa Rosa market, some Broadlink wireless customers are being affected by an outage in their network. Broadlink tells us that they’re working to resolve this issue as quickly as possible.

Update: This issue is resolved. -Dane

Fri Aug 11 15:10:48 PDT 2006 — Internal DNS Server Upgrades. We’ve upgraded the hardware on two of our internal DNS servers to better accommodate the current load they are under. These two servers handle over 10,000 DNS requests per second serving both public and private DNS based blacklists. This upgrade, combined with the hardware upgrades currently in progress on two of our other internal recursive DNS servers, will help prevent a single system failure from adversely affecting our services as a whole. -Kelsey, Augie and Dan

Thu Aug 10 18:51:18 PDT 2006 — Internal DNS Server Outage. One of the many DNS servers that we employ internally started failing early this afternoon. We finished getting it’s replacement online and things are back to normal now. The most notable affect of this failure is in delayed email flows both in and out of our network and skipped SpamAssassin checks. -Kelsey, Augie and Dan

Thu Aug 10 15:49:56 PDT 2006 — Intermittent packet loss. One of our upstream transit providers encountered a router bug this morning that caused packet loss to specific Internet destinations. Among the affected routes were several of SBC’s netblocks. The issue was fixed around 10:30am and traffic is flowing normally. -Nathan and Kavan

Tue Aug 8 10:59:06 PDT 2006 — Customer router downtime. One of our T1 aggregation routers reloaded this morning, causing approx. 3 minutes of downtime for all attached customers. We are investigating the cause of the router reload. -Zeke, Jared and Nathan

Sun Aug 6 08:12:58 PDT 2006 — Sonic.net company picnic. Technical support will be closed today from 11AM until 5PM for our company picnic and 12th anniversary celebration. We’re sorry for any inconvenience, and will be back on duty from 5PM on this evening. -John F

Thu Aug 3 00:16:24 PDT 2006 — DSL maintenance complete. The planned code upgrade to our DSL aggregator in Santa Rosa went smoothly. Total downtime for affected customers was less than 5 minutes, and all affected customers should be online again at this time. -Jared