Month: February 2005

Multicast and IPv6 outage.

Mon Feb 21 17:11:22 PST 2005 — Multicast and IPv6 outage. The router which provides both multicast and IPv6 service is currently down. We’re working on restoring service, but don’t have an ETR at this time. -Nathan and John

Update Mon Feb 21 22:37:58 PST 2005 — Multicast and IPv6 service is restored. -Nathan

DSL outage in eastern Santa Rosa.

Fri Feb 18 17:22:42 PST 2005 — DSL outage in eastern Santa Rosa. ASI/SBC has acknowledged that there is a widespread problem with DSL services in the Eastern Santa Rosa coverage area. They have deployed repair technicians, and service should be restored shortly. -Kavan

Latency and loss to San Jose, resolved.

Fri Feb 18 01:36:27 PST 2005 — Latency and loss to San Jose, resolved. One of our links from Santa Rosa to San Jose has started showing latency and packet loss. Fortunately, our redundant ring architecture has allowed us to move traffic off the impacted link until the vendor can repair it. -Nathan

Another SAN related mail server event.

Mon Feb 14 07:01:24 PST 2005 — Another SAN related mail server event. Gremlins continue to plague the SAN network. This morning an interface on a different NetApp locked-up and stopped passing traffic. This may indicate that the failures are actually being caused by a common shared gige switch as two back to back interface lock ups would be very unusual otherwise. The event occurred between approximately 5:30 and 6:50AM, during this period most people would have experienced timeouts or other errors while trying to retrieve their mail from the POP3 or webmail servers. -Kelsey

POP3 Server Outage.

Sun Feb 13 08:21:50 PST 2005 — POP3 Server Outage. Due to a failure in one of the four clustered NetApp filers network interfaces our POP3 servers went unavailable early this morning. We’ll investigate why the failure occurred; why the redundant network interface wasn’t able to take over the failed one. All services have been restored. -Kelsey

Broadlink TMAC and Above outage this morning.

Fri Feb 11 11:20:55 PST 2005 — Broadlink TMAC and Above outage this morning. Broadlink customers serviced by the TMAC and Above towers went offline for approximately 35 minutes this morning due to a physical power problem on-site. Service was restored at 11:03am. — John F, Jason, and Tim

The Accelerator is now available…

Thu Feb 10 00:34:20 PST 2005 — The Accelerator is now available for Macintosh. Our innovative dialup web accelerator, previously only supporting Windows, is now available for Mac OS 10.2 and above.

See for a link to more details and the software download page. The web accelerator delivers web browsing that can be as much as five times as fast as standard dialup. A single ten minute download of the accelerator software could save you hours every month!

The Accelerator also includes options to block banner ads and pop-up ads, two of the most annoying “features” of the Internet. These ads take bandwidth and slow your browsing, so this is another way our accelerator speeds up your use of the web.

For members, there’s no additional charge for the Accelerator, so if you’re using dialup but you’d like to browse faster, give it a try! -Dane

More Storage Network Failures.

Wed Feb 9 11:17:22 PST 2005 — More Storage Network Failures. This morning our SAN network failed again. This time we were able to identify that the cause of the generalized problems was a packet storm most likely caused by a software failure on one of the switches. After numerous attempts at getting the SAN to be responsive through less invasive means we finally rebooted all of the switches which cleared the storm and restored services. The trouble this morning was far more severe than what happened last night. -Kelsey

Storage Network Fixed.

Wed Feb 9 03:28:35 PST 2005 — Storage Network Fixed. It appears that one of the storage network switches hit a software bug and went into a state where it only selectively forwarded traffic. Reloading the switch corrected the problem. We’ll continue to investigate the cause of the fault and work with our vendor to correct it. -Kelsey

Storage Network Failure.

Wed Feb 9 02:20:26 PST 2005 — Storage Network Failure. At least one switch in our NFS SAN is currently off-line. The scope of this failure is currently unknown but it is not believed to have affected basic service availability. Someone is en route now to investigate and repair. -Kelsey