Month: October 2007

Upgrades to Email storage complete.

Sat Oct 27 03:17:56 PDT 2007 — Upgrades to Email storage complete. We have finished our scheduled maintenance on the Email storage system and Email is flowing again. Total downtime was around one hour. We also took the opportunity to run some software updates on many of our other systems. –Augie, Kelsey, Christopher, and Don.

Upgrades to Email storage.

Thu Oct 25 11:32:30 PDT 2007 — Upgrades to Email storage. This Friday at 12:00 AM we will be adding several new shelves to the NetApp Filers that store customer Email. This upgrade will provide greater capacity for future growth and improved performance for all of’s Email customers. Downtime is expected to be one hour while we install the shelves and flush mail queues. -Augie, Kelsey, Don, and Christopher.

Core Router Maintenance.

Wed Oct 24 12:37:07 PDT 2007 — Core Router Maintenance. This Friday at 12:01 AM we will be performing a software upgrade on one of our core routers in our San Francisco POP. Some customers may notice a brief interruption in Internet connectivity as traffic is re-routed around this router. -Jared

Update Fri Oct 26 01:08:13 PDT 2007 — Router Maintenance Complete. We have completed the software upgrade on our core router in San Francisco. The entire operation went as planned and there was no customer traffic interruption. -Jared

Upstream Network Issue.

Tue Oct 23 14:58:05 PDT 2007 — Upstream Network Issue. At 2PM today one of our upstream network providers suffered a problem in their network. We have routed traffic away from this provider as we work with them to resolve their issue. Customers may have seen 10-15 minutes of spotty or slow connectivity to sites on the Internet. -Jared and Nathan

LA POP Migration complete.

Tue Oct 23 01:59:18 PDT 2007 — LA POP Migration complete. We have completed the migration of all DSL and Business-T customers in the Los Angeles LATA to our new LA POP. Downtime was 45 minutes at the worst, though most customers would have seen 20 minutes or less of downtime. Welcome to LA! -Jared, Nathan, Dusty, Matt and Tim

LA POP Migration.

Fri Oct 19 16:23:05 PDT 2007 — LA POP Migration. The Network Operations team has spent the last couple of weeks deploying a new Point of Presence in Los Angeles. This huge new expansion, which includes redundant border routers, switches, and DSL termination hardware is now on-line and ready to accept customers. We will be swinging all of our LA DSL subscribers as well as our Business-T customers in the Los Angeles area to this new POP at 12:01am on Tuesday, October 23rd. We expect approximately one hour worth of downtime for affected customers as circuits are migrated. This deployment brings with it decreased latency to the Internet for our LA customers as well as a more redundant, scalable network deployment. -Nathan, Dusty, Jared and Tim