Fri Aug 4 00:10:32 PDT 2000 — Our T3 to Cable & Wireless went down unexpectedly a few minutes ago. All traffic shifted to our UUNet T3, which was more than capable of handling the load. The circuit came back up at 23:59. Cable and Wireless has no explanation for why the T3 went down. We have opened a medium-priority ticket to determine a reason for outage, which will include inspection of CW’s router to ensure it is operating properly. -Scott

Thu Aug 3 14:49:23 PDT 2000 — UUNet’s latency and loss are all better, it looks like it was just impact from the session clearing. They were unable to find a reason for the session clear on their end – it didn’t actually go down, but the BGP session cleared, probably due to loss of the heartbeat. There’s no indications on either end about why the heartbeat couldn’t make it, so we’ll call this an anomaly. -Dane

Thu Aug 3 13:52:39 PDT 2000 — Our T3 to UUNet bounced a few minutes ago, causing some network slowdown. We’re currently seeing greater than average latency and packet loss on that link, and we are opening a trouble ticket with UUNet’s network operations center to get their feedback. -Dane and Scott

Wed Aug 2 17:17:15 PDT 2000 — Yeah RAID5! The internal MySQL server is back online and the DPT raid controller is rebuilding the array in the back ground and access to the member tools and our internal tools has been restored. We are very happy that our disaster recovery plan worked exactly as we had intended. -Kelsey, Nathan, Eli, Scott, Steve, and Dane.

Wed Aug 2 16:14:13 PDT 2000 — The member tools section is currently down for emergency maintenance due to a raid error on our database server we are in the process of repairing this now and it should be up shortly -Kelsey, Nathan, Scott, Steve

Tue Aug 1 09:26:03 PDT 2000 — this is a test entry. — test — – Kelsey

Tue Aug 1 09:18:25 PDT 2000 — This is a test entry. — test — test. -Kelsey

Tue Aug 1 15:49:32 PDT 2000 — The path on CW’s (Cable and Wireless’) network that was exhibiting high latency and packet loss is fine now. We are still awaiting a call from CW to see if this was the same problem we resolved earlier this morning. -Scott

Tue Aug 1 15:12:51 PDT 2000 — While checking paths to various networks today, we discovered another (little-used) path on CW’s network that is showing high latency and packet loss. We have pointed out the problem to CW, and expect a resolution shortly. -Steve, Nathan, Scott

Tue Aug 1 02:07:25 PDT 2000 — Latency and packet loss issue on CW’s network solved. As it turns out, traffic bound for our network from the PAIX router was taking a secondary route to reach us. CW engineers fixed the routing problem by moving our traffic to a more direct circuit. They are now investigating why the other path had a problem — but as far as we are concerned, the problem is resolved. -Scott