Tue Feb 28 17:11:07 PST 2006 — We will be migrating our 707-522-1002 dial-up number to a new provider at 11:00am on Thursday, March 2nd. We expect that this change will be transparent to our customers. This move provides expanded coverage and increased reliability by moving the serving gear closer to the core of our network. -Nathan

Update Thu Mar 2 10:24:27 PST 2006: Our carrier has rescheduled this migration at the last minute. The change will now take place on Monday, March 6th at 11:00am.

Thu Feb 16 14:48:51 PST 2006 — Saturday shortly after 12:00AM we are going to have a large scale night ops in Santa Rosa. The primary purpose of the maintenance will be to migrate our core networking infrastructure to properly redundant A+B power feeds from our two UPSes. The power migration by itself should not be customer affecting. However, we are taking advantage of the maintenance window and performing software upgrades on some of our networking equipment and will also be making a couple of small but potentially service affecting configuration changes. While the networking folks are busy the rest of Operations will be migrating servers to the new UPS to balance power draw between the two UPSes. Very little customer impact is anticipated here – most of these servers are grouped into redundant clusters so while one server is down another should be online handling it’s services. We will also be moving the two Network Appliance filer clusters that handle mail stores in their cabs to make room for 8 additional disk shelves to handle growth as well as redistributing their power feeds. -Kelsey, Nathan, Zeke, Jared, Clay, Augie, Dan and Russ.

Wed Feb 15 17:32:58 PST 2006 — DSL Outage. One of our DSL concentrators in our San Francisco POP went down for 10-15 minutes this evening. We are working with the vendor on analyzing the software core dump so that we can prevent such failures in the future. All affected customers should be back online at this time. – Nathan and Jared

Fri Feb 10 15:14:18 PST 2006 — Planned Dialup Maintenance. On Tuesday 2/14 at approximately 10AM we will be migrating dialup connectivity served from our 200 Paul Ave. POP in San Francisco to a new Cisco Access Server architecture. This upgrade is expected to enhance performance while reducing the number of disconnects. This will cause a brief service interruption for customers dialing 707-823-8812 as well as any of our dialup numbers ending in 5. Typically we would perform such maintenance during off-peak hours, however we feel it is important to perform this maintenance when our Technical Support and Operations staff are available. As always, alternate dialup numbers may be obtained via our Pop Finder tool located at www.sonic.net/popf/ – Russ and Network Operations

Tue Feb 7 21:39:19 PST 2006 — DSL Outage in Santa Rosa. The DSL terminator in Santa Rosa just rebooted. Most customers should already have service restored by the time this MOTD is finished. We will investigate the cause of the failure and take the required steps to resolve it. -Nathan, Jared and Kelsey

Thu Feb 2 23:22:18 PST 2006 — DSL Outage in Santa Rosa. Tonight at approximately 10:55pm, we started seeing DSL customers in the Santa Rosa area reporting sync-no-surf. This appears to be on AT&T’s side of the network, but we are monitoring the situation. We have no ETR at this time. -John F and Technical Support

Thu Feb 2 10:14:34 PST 2006 — Additional DSL Maintenance in San Francisco. Tonight shortly after Midnight we are going to reconfigure the other OC-3 uplinked RedBack SMS 1800 to the new architecture to eliminate the performance issues that have started to affect it as well. Downtime for customers will be brief and the total operation should take less than 30 minutes. -Kelsey and Nathan

Mon Jan 30 13:37:31 PST 2006 — DSL Outage Redux – All affected customers appeared to come back up last night, but we never received an official cause from SBC/AT&T – At approx. 13:10hrs today, we received reports that a significant number of DSL customers in Santa Rosa went down again, and while we have once again received confirmation from SBC/AT&T that it is a problem on their end we still do not have an ETR or official cause. We will continue to contact SBC/AT&T until we have an answer. – Eli, Support

UPDATE: SBC/AT&T appear to have successfully resolved this afternoon’s DSL outage.

Sun Jan 29 21:45:35 PST 2006 — DSL Outage. There appears to be a widespread DSL outage that is affecting a large number of subscribers. We’ve traced it back to SBC/AT&T’s network and are in the process of contacting them now. There is no ETR at this time. -Kelsey, Nathan and Jared

Sat Jan 28 14:20:32 PST 2006 — DSL Maintenance. Tonight, shortly after Midnight, we are re-architecting the termination of the DSL customers that are currently experiencing degraded performance in San Francisco to take advantage of equipment we put in place to terminate the new ATM OC-12 circuit. This is a complex procedure that will result in the subscribers split between two new RedBack SMS 1800s. We will do everything possible to minimize downtime but anticipate some customers may experience outages as long as 30 minutes while the work is underway. Once finished, there will be no more load related performance issues for the affected customers. We apologize for the short notice. -Kelsey and Nathan

Update Sun Jan 29 04:11:38 PST 2006 — The migration has gone surprisingly well. We’ve distributed enough load onto the new DSL termination equipment to mitigate the performance issues. We may do additional maintenance over the next few days if warranted. -Kelsey and Nathan