Month: December 1999

Tue Dec 14 01:25:34 PST 1999 — We had some problems around 1am with our main outbound mail server. I restarted the mailer software at 1:15am, and all is well now. If you had trouble sending email, please try again. -Dane

Mon Dec 13 18:41:12 PST 1999 — UUNet and PacBell are still trying to find and eliminate the performance trouble on our primary T3. We’ll keep you posted here when we have more details! -Dane

Sat Dec 11 09:07:38 PST 1999 — UUNet is still working with PacBell and ADC Kentrox on the trouble on the SMDS T3. We’ve offloaded some traffic, so it’s performing pretty well today, but we still don’t have a final solution. We’ll continue to press forward on this. -Dane

Sat Dec 11 09:05:49 PST 1999 — We’ve found two cards in nas21.sonic.net that were often answering with a modem tone that would never synchronize. Dave has written some testing tools that did a great job at nailing this trouble down. We’ll be looking at nas22 through nas25 as well to see if we can find any other trouble. We’ve taken the bad cards out of service, and this should make the 1003 dialup group much more reliable! -Dane and Dave

Sat Dec 11 09:04:21 PST 1999 — mailbox.sonic.net, the machine that handles add-on mailboxes had trouble this AM. It’s had intermittent problems with it’s Ethernet interface, and we’ll take steps to avoid this in the future. It’s back online now, sorry about the unavailability of POP for these users for a time. -Dane

Fri Dec 10 09:48:52 PST 1999 — Thunder’s web logs were corrupted last night and there is no way to recover them. The logs were most likely damaged during the copy from thunder to the log host, timber. All multihomed web site stats are affected by this including the Urchin reports. We have taken steps to prevent this from happening again. -Kelsey

Fri Dec 10 19:08:31 PST 1999 — After this AM’s card swap at PacBell, as the circuit load went up during the day, we’ve been seeing poor performance on the SMDS T3 here. I spent most of the day on the phone troubleshooting with Cisco, ADC Kentrox, PacBell and UUNet, and we seem to be reaching a consensus that the ADC CSU/DSU at the UUNet end of the circuit appears to be the source of the problem. We brought additional T1 capacity online today, and this relieved the circuit quite a bit, so performance today was a bit better overall than yesterday, with packet loss at around 3% and latency averaging 50ms.

I’m testing the circuit right now however, and I’m seeing performance around 11ms with no loss, which is wonderful. UUNet is supposed to be working with ADC right now, so it may be that they’ve found the trouble. I’ll put an update in the MOTD when I have full information. -Dane

Fri Dec 10 19:05:08 PST 1999 — We are working to isolate trouble in our 522-1003 dialup group which is causing calls to be answered by a modem with an odd tone which will not sync. Dave has been working much of the day on testing and trouble isolation, and we’re trying to pin this down. We’re stress testing nas21.sonic.net right now to try to isolate the trouble. If we can find it, we’ll eliminate it, and if we can’t, we’ll be rebooting the equipment a bit past midnight tonight. In our testing, we’ve found failure to connect about 30% of the time in this group, and fixing this is a very high priority for us. -Steve and Dave (dj:dj)

Fri Dec 10 19:03:54 PST 1999 — We have fixed the line trouble in our xxx-0174 dial group. It turned out to be a hardware problem in our Lucent/Livingston equipment in Oakland. This should eliminate a problem where 1 in five dial attempts would not be answered by a carrier in this dial group. -Steve (dj:dj)

Thu Dec 9 17:31:59 PST 1999 — One of the five T1 circuits in our xxx-0174 dial group is out of service. PacWest, the phone company that serves these lines out of Oakland is looking at the trouble, and will work to remedy it. While we haven’t seen any busy signals or other problems that we can duplicate here, we suspect that this may be causing some issues in that group. We’re hoping that PacWest will get this resolved this evening. -Dane and Steve