Sun Feb 6 12:38:47 PST 2005 — Broadlink report on primary backhaul circuit. Broadlink is still experiencing problems with their primary backhaul circuit. They report that they are in the process of making adjustments to rectify the condition. Steps taken so far have not effected a resolution. They are continuing to work the problem, and will provide an update in the next two hours. Thanks to Jason and Tim for the long hours they’ve put into a resolution so far. -Scott, and Tim and Jason from Broadlink

Sun Feb 6 09:28:40 PST 2005 — Packet loss problem isolated on Broadlink network. Broadlink has isolated the problem with their backhaul, and they are working to effect a fix. ETR is about 1.5 hours. Please note that link will occasionally drop while they are effecting these repairs. -Scott and Tim with Broadlink

Sat Feb 5 12:39:33 PST 2005 — Packet loss on the Broadlink network. Severe packet loss has been detected on the Broadlink networks backhaul. Work is in progress to restore stability, we have minimized the loss to about 5% until full diagnosis and repair can be completed. -Scott, Kavan, and Tim with Broadlink

Wed Feb 2 10:30:13 PST 2005 — Authentication Server Upgrades. We migrated our four primary authentication servers yesterday to newer, faster, hardware. The new hardware is nearly twice as fast. We’re also going to be deploying a cluster of authentication servers in San Francisco in the near future to provide additional redundancy during network events. -Augie and Kelsey

Wed Feb 2 17:16:54 PST 2005 — Testing of the new mailman system. We are moving the motd-list to our new Mailman server, and this represents the first post to the new system. No problems are expected, and this should be a seamless upgrade. -Kevan

Sat Jan 29 10:14:28 PST 2005 — User Database Issue. Between 8:54 and 9:13 this morning it appears the system that creates and manages the unix passwd databases encountered an unexplained error and pushed a truncated file to our servers. The next automated update corrected the situation. During this period mail to many sonic customers was returned with ‘User Unknown’ errors. Similarly, many users would have received errors when retrieving their mail, trying to access FTP, and in specific cases, some users’ websites may have returned errors. -Kelsey

Thu Jan 27 11:22:26 PST 2005 — New DSL Reverse DNS Tool. DSL customers with static IP addresses can now set their own reverse DNS records with the new member tool. The only requirement imposed on the RDNS records is that the forward already matches the requested reverse record. All manually configured and currently valid records were preserved. sonic.sonic.net/members/dsldns/ -Kelsey and Chris B.

Tue Jan 25 21:31:07 PST 2005 — Core4 FastEthernet failure. A FastEthernet interface in our Santa Rosa Core4 switch failed, which caused loss of connectivity with our old backbone network. The interface was replaced with an onsite spare, downtime was approximately 40 minutes.

A router used as part of our web server migration was on the old network, and the result was that our web cluster was unavailable during the outage. We are removing this single-point-of-failure as soon as possible. -Scott, Kelsey, and Nathan

Tue Jan 25 16:12:58 PST 2005 — Broadlink backhaul problem. Broadlink reports a backhaul circuit down between their Tmac and Rosenberg sites. Jason is working on a fix. No ETR yet… -Scott, and Jason from Broadlink

Update Tue Jan 25 17:17:38 PST 2005: Jason has repaired the backhaul by replacing a cable. The problem was weather-related. -Scott, and (thanks to) Jason of Broadlink

Mon Jan 24 15:30:34 PST 2005 — SMTP Authentication hiccough and port 25 blocking. Earlier today our POP before SMTP authentication system was accidentally disabled when installing and configuring a fresh compilation of sendmail to support SMTPS (SSMTP – SMTP over SSL on port 465) for mail clients that do not correctly support STARTTLS on port 587. We require password authentication when using SMTPS. Additionally, at some point in the future we will also require password authentication on port 587. At this time everything is working correctly despite a few bumps along the way.

NOTE: Many users connecting to our outbound mail servers from SBC/Yahoo DSL lines (not Sonic.net DSL lines) or Comcast cable Internet services may need to update their server settings to work around port 25 filters both of these providers appear to be enabling in their networks. -Kelsey