Month: December 1999

Fri Dec 3 19:07:17 PST 1999 — Covad communications is having a massive outage which is affecting most of their DSL customers in the state. They say that they’re working fast and hard to resolve this, and I’m sure the stress over there is pretty extreme. They had no ETR available when we spoke with them. -Dane, Jen and Eli

Fri Dec 3 15:15:46 PST 1999 — Now that the web logs for multihomes are centralized you can tail your logfiles. We have provided a simple command for you to do this, ‘twl’ To use ‘twl’ just enter ‘twl yourdomain’ at the shell and then ‘CTRL-C’ to stop tailing the logs. This command will continue to work once the we have the Alteon switch up and running with load balanced web servers. -Kelsey

Thu Dec 2 21:36:21 PST 1999 — Snapshot data filled up the Network Appliance just now. We are adding another disk to the array tonight, which should prevent further problems. -Scott and Dane

Thu Dec 2 17:45:51 PST 1999 — At around 10am this morning, snapshot data on our Network Appliance file server filled up the disk array, causing disk operations to report ‘disk full’ for about 20 minutes. Normal operation was restored by removing snapshot data.

Thu Dec 2 00:04:32 PST 1999 — Pacific Bell will be performing maintenance on our primary T3 between 3:00am and 3:15am this morning. Expected impact is a slight slowdown of overall Internet performance during this time as our backup circuits take up the load. PacBell will be swapping in a new card in the SMDS switch in an attempt to clear our performance and latency concerns. In the worse case, if they have issues with the swap and have to backpedal, the circuit will be back online by 3:30am. -Dane

Wed Dec 1 12:01:54 PST 1999 — During network reconfiguration that should have been non-intrusive, our core gateway router, mega.sonic.net crashed and reloaded. We were changing the pathing to re-balance load on our circuit to Cable and Wireless as we work on ongoing latency troubles on our UUNet T3, and this isn’t an operation that should cause any problems in the router.

We’ve opened a support ticket with Cisco, and have asked their tech group to come up with some sort of reason for what happened. Downtime was four minutes and 42 seconds, during which Internet connectivity was now available. We’re very sorry for the interruption, and we’ll work with Cisco to come up with an explanation and to try to come up with some way to achieve more redundancy than we already have in our core structure. -Dane, Scott and Co.

Wed Dec 1 19:52:11 PST 1999 — Urchin, the new web log analysis tool, is up and running. You may take a look at your multihome’s urchin report by going to www.sonic.net/stats and filling out the form. You will be prompted for the username and password associated with the multihome. The raw web logs have also been updated to include November 30th. Both Urchin and the raw logs are updated once a day around 2:00am. -Kelsey