Month: January 2006

Mon Jan 30 13:37:31 PST 2006 — DSL Outage Redux – All affected customers appeared to come back up last night, but we never received an official cause from SBC/AT&T – At approx. 13:10hrs today, we received reports that a significant number of DSL customers in Santa Rosa went down again, and while we have once again received confirmation from SBC/AT&T that it is a problem on their end we still do not have an ETR or official cause. We will continue to contact SBC/AT&T until we have an answer. – Eli, Support

UPDATE: SBC/AT&T appear to have successfully resolved this afternoon’s DSL outage.

Sun Jan 29 21:45:35 PST 2006 — DSL Outage. There appears to be a widespread DSL outage that is affecting a large number of subscribers. We’ve traced it back to SBC/AT&T’s network and are in the process of contacting them now. There is no ETR at this time. -Kelsey, Nathan and Jared

Sat Jan 28 14:20:32 PST 2006 — DSL Maintenance. Tonight, shortly after Midnight, we are re-architecting the termination of the DSL customers that are currently experiencing degraded performance in San Francisco to take advantage of equipment we put in place to terminate the new ATM OC-12 circuit. This is a complex procedure that will result in the subscribers split between two new RedBack SMS 1800s. We will do everything possible to minimize downtime but anticipate some customers may experience outages as long as 30 minutes while the work is underway. Once finished, there will be no more load related performance issues for the affected customers. We apologize for the short notice. -Kelsey and Nathan

Update Sun Jan 29 04:11:38 PST 2006 — The migration has gone surprisingly well. We’ve distributed enough load onto the new DSL termination equipment to mitigate the performance issues. We may do additional maintenance over the next few days if warranted. -Kelsey and Nathan

Wed Jan 25 20:43:59 PST 2006 — Broadlink TMAC outage. Broadlink reports an problem with their TMAC site — they report estimated time of repair should be sometime between 9pm and 11pm. -Scott, Tony, and Jason from Broadlink

Wed Jan 25 16:26:26 PST 2006 — DSL Latency and Loss. We continue to have ongoing problems with some DSL customers served out of San Francisco. The issue manifests itself as intermittent latency and loss, which often results in poor download performance or choppy streaming media. We have identified that the issue is caused by an overloaded router inside of our San Francisco network. Our attempts to mitigate the overload have met with limited success despite repeated calls to the router vendor and extensive in-house troubleshooting. This router was scheduled to be replaced on 12/30 as part of our upgrade to an ATM OC-12 from AT&T. Unfortunately, AT&T was unable to deliver the ATM OC-12 on time, and has now given us a 1/30 date of installation for the new circuit. Shortly after this link is installed we will be moving all of the affected customers over to new routers with ample capacity.

We apologize for the delay in the upgrade and will continue to work diligently with AT&T to improve service quality.

-Nathan, Dane, Eli and Kelsey

Sat Jan 21 15:59:19 PST 2006 — www.sonic.net was un-available for a few minutes as an internal monitoring process consumed all available resources. The process has been disabled and we will be increasing the available resources for those machines shortly. -Augie

Wed Jan 18 22:55:29 PST 2006 — DSL IP Latency, continued. The problems that we were having with some DSL subscribers served from San Francisco have not yet been resolved to our satisfaction. One of our DSL concentrators continues to cause light packet loss on traffic passing through it. We are working to resolve the issue both with the device vendor as well as our own tests, and we expect to have the problem resolved shortly. -Nathan

Tue Jan 17 11:14:14 PST 2006 — SpamAssassin Upgrade. We just completed upgrading SpamAssassin to the latest stable version. The new version should offer improved filtering with lower false positive and negative events. There have been a few small changes made to the SpamAssassin GUI in our membertools. Users that have customized their config may want to review the changes and consider resetting their scores back to default. For more information please see news://news.sonic.net/sonic.antispam and www.spamassassin.org -Kelsey and Chris

Tue Jan 17 15:44:52 PST 2006 — Customer MySQL Upgrade. We have updated our customer database management software (MySQL) to the latest 4.1.x . This will offer quite a few new features and additional functionality to our customers. A full list of the new features can be found here:

dev.mysql.com/doc/refman/5.0/en/nutshell-4-1-features.html

While we do not anticipate that any of our customers will encounter problems because of this upgrade we do encourage those customers with MySQL databases to confirm that their applications are functioning properly. A full list of changes to the underlying database system can be found here:

dev.mysql.com/doc/refman/4.1/en/news-4-1-x.html

Remember that anyone can add MySQL Database Hosting to their account, and that a database is an excellent way to store any and all data; for more information please see our MySQL FAQ:

www.sonic.net/support/faq/addon/mysql.shtml

-Augie and Kelsey

Mon Jan 16 23:47:05 PST 2006 — DSL IP Latency. Some DSL customers served out of San Francisco may have noticed 10-15ms of additional latency on their connections during the majority of today. The additional delay in handling these packets was caused by a malfunctioning router. We have performed maintenance to correct the router’s behavior and latency has returned to normal. -Nathan