Mon Jun 25 09:47:19 PDT 2001 — The 522-1001 dialup group was having issue authenticating this morning. This required a reboot of the gear. Once we rebooted the gear all started working again. Sorry for any problems this may have caused. -Steve
Correction: The dialup group affected was 522-1003 -Steve
Sat Jun 23 02:09:26 PDT 2001 — Redundancy work completed with no downtime. The second switch (Giga) for our fault-tolerant network design is online. We have activated redundant gigabit links from both Network Appliances to both core switches. Not only will this improve performance to the filers, but will ensure that the filers remain available even if a switch fails. A test of the failover capability worked flawlessly.
We have also moved one of our Cisco 7206 border routers to Giga. Both border routers are already injecting redundant default routes into our OSPF routing domain, which means if one border router fails, we won’t lose the Internet.
Kermit, our second Alteon AD3 load-balancing switch, is now alive and well on Giga. We will use Kermit to build out a new active-active load balancing configuration, which we will then synchronize with Piggy, our other load-balancing switch. This will ensure that web and mail remain up even if a switch fails.
Now that we’ve worked around and solved all of our problems with the Black Diamond switch (knock on wood), our redundant design will finally achieve fruition. -Scott and Kelsey
Fri Jun 22 06:56:07 PDT 2001 — Timber’s maintenance was completed as scheduled. Everything looks good. Total downtime for the SQL database was approx. 45 minutes. -Steve
Fri Jun 22 00:36:42 PDT 2001 — Redundancy work. The second switch for our fault-tolerant network design is online. At 1am tomorrow (Saturday) morning, we will be turning up redundant gigabit links from both Network Appliances to both core switches. Not only will this improve performance to the filers, but will ensure that the filers remain available even if a switch fails. After this, we will move one of our Cisco 7206 border routers to the second switch. Both border routers are already injecting redundant default routes into our OSPF routing domain, which means if one border router fails, we won’t lose the Internet. We don’t anticipate any of this work to cause downtime, but we thought we’d better mention it in case something happens. -Scott and Kelsey
Thu Jun 21 16:18:47 PDT 2001 — We will be performing disk maintenance on timber.sonic.net. This is our customer MySQL server. SQL access will be unavailable for approximately one hour starting at 6:00am on June 22, 2001 and will be back up at approximately 7:00am -Steve
Thu Jun 21 15:57:50 PDT 2001 — Redundancy work. The second switch for our fault-tolerant network design is online. At 1am tomorrow (Friday) morning, we will be turning up redundant gigabit links from both Network Appliances to both core switches. Not only will this improve performance to the filers, but will ensure that the filers remain available even if a switch fails. After this, we will move one of our Cisco 7206 border routers to the second switch. Both border routers are already injecting redundant default routes into our OSPF routing domain, which means if one border router fails, we won’t lose the Internet. We don’t anticipate any of this work to cause downtime, but we thought we’d better mention it in case something happens. -Scott and Kelsey
Wed Jun 20 17:38:09 PDT 2001 — Gale, our new news peering server had a disk failure in it’s RAID earlier today. Under normal operations this should not have caused a failure in the system however, when we replaced the failed disk the OS crashed causing a corruption in the filesystem. It’s been repairing it’s filesystem for the past hour and during this time we have not received or transmitted news on our network. We expect that it should be back up soon but also suspect that we will have a low article completion rate for the next few hours until the server catches up. We’ll be talking to the RAID vendor to find out why the OS crashed when we hot-swapped the failed disk. -Kelsey and Scott
Wed Jun 20 16:15:05 PDT 2001 — Colos down for 5 minutes. A new routing policy on our core layer 3 switch interfered with colocated customer routes. Downtime was about 5 minutes. We apologize for the problem. -Scott
Mon Jun 18 18:07:43 PDT 2001 — BroadLink is back online. They had some software problem with a bandwidth management box that caused it to behave badly. All’s well now, downtime was about 30 minutes. BroadLink says “sorry!” -Dane, Eli and Shane
Mon Jun 18 17:40:48 PDT 2001 — BroadLink customers are currently offline. We’re coordinating with BroadLink to find out what the issue is, and will post detail and an ETR here as soon as we have full information. -Dane and Eli