Month: November 2000

The maintenance window was successful, and…

Tue Nov 28 15:08:10 PST 2000 — The maintenance window was successful, and also much shorter than expected. PacBell DSL and Broadlink customers were back in service within a couple of minutes, and we are pleased with the outcome. – Eli, Kelsey, Chris, Matt, and Tom!

EMERGENCY DOWNTIME- This afternoon we will be

Tue Nov 28 14:11:05 PST 2000 — EMERGENCY DOWNTIME- This afternoon we will be taking our Redback Router offline briefly to replace a failing component. Estimated downtime is 10 to 15 minutes. During this period all Pacbell DSL and Broadlink Wireless connections will be unresponsive. We apologize for this unscheduled downtime. Normally we would perform a maintenance task like this in the early morning, but circumstances dictate that it has to happen right away. Thank you for your patience! – Eli, Kelsey, Chris, and Tom from Redback

A Denial of Service attack made it necessary…

Sat Nov 25 20:45:40 PST 2000 — A Denial of Service attack made it necessary to reboot one of our core network switches. This would have been noticed by customers as slow/no connectivity to both internal servers and the Internet. The source of the attack is under investigation. – Eli, Kelsey

Night Operations Complete.

Sun Nov 19 04:20:00 PST 2000 — Night Operations Complete. I’ve finished both the migration of web data to icebox and the removal of the faulty card from lily. We are all very excited about the new NetApp. Not only does it effectively double our NFS storage capacity, it also should noticeably increase performance. The next big project with the two NetApp filers is to configure them in an active-active cluster. When configured in a cluster, if one filer fails or needs to be maintained, the other filer will take over the other’s disk array until it is brought back online. We are also going to migrate both of the filers to dual gigabit fibre interconnects to our two core switches such that if any single piece of hardware is to fail in the matrix, services will not be interrupted. -Kelsey

Night Operations Update.

Thu Nov 16 02:02:39 PST 2000 — Night Operations Update. Typhoon’s upgrade went as planned and it was brought back online at 1:15AM. There are public stats on the news server available at news.sonic.net/typhoon As always, I welcome any comments you have about our news server performance.

We decided not to work on lily, the T3 mux, tonight and have postponed the maintenance until Sunday AM.

The final syncing of all of the WWW data is taking exponentially longer than anticipated, as such we are also postponing the final migration to icebox until Sunday too. – Kelsey

DSL Outage.

Wed Nov 15 11:31:57 PST 2000 — DSL Outage. We’ve just received word from our SBC (PacBell) reps that one of the DSLAMs in Sebastopol has failed. SBC has not provided us with an estimated time for it being fixed but has promised us an update in a few hours. Until DSL services are restored don’t forget that DSL users can still dial-up using their modems. – Dave, Chris & Kelsey

Night Operations.

Wed Nov 15 10:32:37 PST 2000 — Night Operations. Thursday morning starting at 1:00AM we will be performing a series of upgrades to our network. First we will be installing an additional GB of RAM in typhoon, our news server, which will bring it’s total RAM to 2GB (thanks Ken!) There should only be a few minutes of downtime while we install the new RAM. This upgrade should result in a marked increase in Usenet performance.

We will also be removing one of the redundant modules in lily, our T3 mux, for service. There should be no downtime if the fail-over module properly takes over service. However, if it fails to take over properly, access to some of our pops may be unavailable while the mux resyncs and the T1’s come back online.

At 2:00AM we plan to finish the migration of all web data onto the new Netapp, icebox. For approximately 30-45 minutes all web data will be read-only while we do a final sync with the existing filer. Some CGIs may not function properly during this period and users will not be able to update any webpages or cgi. -Kelsey & Scott

Broadlink Maintenance.

Wed Nov 15 14:53:30 PST 2000 — Broadlink Maintenance. Thursday, November 16th at 11:59 p.m., BroadLink Communications will be performing scheduled maintenance on one of the tower site that provides wireless broadband service. Downtime is expected to be approximately ninety minutes. We apologize for any inconvenience this may cause. If you experience degraded service levels outside this time frame please email Broadlink Support at support@broadlink.com – Broadlink Communications