Thu Jun 5 11:27:34 PDT 2003 — Shell server bugfix. We corrected a minor configuration issue that was preventing /tmp on the shell server from getting cleaned on a regular basis. Files left in /tmp will now be removed after 7 days from their last modification time. -Kelsey and Nathan
DNS server hiccup.
Thu Jun 5 22:17:15 PDT 2003 — DNS server hiccup. Our primary DNS server, ns1.sonic.net, stopped answering queries for approximately two minutes. However, since most users have two DNS servers configured, impact was negligible. If you saw name resolution failures, you may want to check your DNS configuration. Information on DNS setup can be found at:
www.sonic.net/support/index.shtml
under the “Quick Reference Guide” and machine-specific “Setup Guide” links. -Nathan
Routing issues at Sebastopol POP.
Wed Jun 4 21:16:10 PDT 2003 — Routing issues at Sebastopol POP. The last portmaster in the dial group that servers Sebastopol had an apparent misconfiguration that was just uncovered. It has been fixed and now customers using this box are fully functional. -Kelsey, Nathan and Russ
Broadlink head-end router reboot.
Tue Jun 3 17:41:15 PDT 2003 — Broadlink head-end router reboot. The Broadlink router which connects their subscribers to the Sonic network rebooted ten minutes ago. At this time, all subscribers should be up and functional. -John and Nathan
News server lockup.
Tue Jun 3 15:58:35 PDT 2003 — News server lockup. Our news server, news.sonic.net, locked up a few minutes ago. We are working to restore service, but in the mean time, altnews.sonic.net and supernews.sonic.net remain available. -Kelsey and Nathan
Update: The server hardware for news.sonic.net appears to be having some difficultly: the box has rebooted itself a few times. We are investigation options we have to replace the hardware. At this time it is back online, no wait, it’s down. -Kelsey and Nathan
Update: News.sonic.net is back online, one of it’s CPU had a failed cooling fan, it has been replaced. -Kelsey, Kevan, Zeke and Nathan
Name server issue.
Tue Jun 3 00:30:36 PDT 2003 — Name server issue. We experienced an issue with one of our primary name servers where an old configuration directive caused the server to shutdown. The problem has been fixed. Sonic.net has multiple name servers and customers should not have noticed the issue. -Matt and Kelsey
Sundry Updates to DNS, Authentication and…
Mon Jun 2 16:52:13 PDT 2003 — Sundry Updates to DNS, Authentication and SpamAssassin. We were able to track down the sources of the periodic problems with our DNS servers and made some changes that appear to have resolved the problems. The DNS server failures were also causing cascading failures on our mail and authentication servers. We’ve taken additional measures to insulate these services from failures of our core DNS servers. We’ll still be going ahead with the DNS server upgrades and replacement.
We also did some profiling and performance tuning on our SpamAssassin cluster and were able to drive the average time to inspect a message down to half a second from between 2 to 6 seconds. This marked increase in performance should result in snappier email delivery and less load on our core mail servers. -Kelsey and Nathan
Delayed Mail Server Migration: We’ve delayed…
Mon Jun 2 16:45:06 PDT 2003 — Delayed Mail Server Migration: We’ve delayed the migration of ‘mail.sonic.net’ onto the new mail servers until we can resolve some issues surrounding the support of POPB4SMTP authentication. Our logs had indicated that not a single user was requiring POPB4STMP auth to send mail, upon reexamination, we turned up several hundred users. At this time we are unsure if we will enable support for POPB4SMTP auth or if we will assist these customers in transitioning to SMTPAUTH. -Kelsey, Nathan, John and Russ.
Name server issues.
Fri May 30 09:57:32 PDT 2003 — Name server issues. We’ve had occasional issues with our nameservers, which can cause slow queries as the queries fail over to redundant nameservers. We think that doubling the 1GB of memory to 2GB will greatly improve the situation, and toward that end we have three 2GB servers that will shortly become our new nameserver cluster. -Scott, Kelsey, and Operations
Mail Server Migration: On Monday the 2nd of…
Tue May 27 10:08:52 PDT 2003 — Mail Server Migration: On Monday the 2nd of June we will be migrating ‘mail.sonic.net’ to a new cluster of servers. This is the first stage in the complete re-architecture of our mail server infrastructure. There are a number of significant changes on the new servers which may affect some customers. We have replaced POPB4SMTP authentication with SMTPAUTH and enabled STARTTLS. SMTPAUTH and STARTTLS are well defined standards that allow users to securely authenticate and relay mail off our servers while they are not directly connected to our network. Some users, connecting from remote networks may need to make a few minor configuration changes to their email clients. If you have any questions please contact Technical Support or post them to news://news.sonic.net/sonic.net -Kelsey