Fri Sep 7 16:53:18 PDT 2001 — Due to an ongoing network attack, we’re blocking inbound POP checking from SplitRock dialup services. If you’re a Sonic.net customer who is using another ISP’s dialup resources, and if those dialup resources are actually provided by SplitRock, you will not be able to check email via POP until SplitRock security completes their work. -Kelsey, Nathan, Eli and Dane

Wed Sep 5 12:12:31 PDT 2001 — Sonic.net is now providing additional simple addresses for websites. All basic Sonic.net accounts include the ability to publish your own home page, and now it’s easier to publish the address for your pages.

Where previously we provided customers with an address with the “tilde” format, (non-working sample below):

www.sonic.net/~username/

We now also provide the following options:

www.sonic.net/users/username/ www.username.home.sonic.net/ www.username.users.sonic.net/ www.username.members.sonic.net/

You can choose to use and publish any of these addresses when you promote your own website; just replace “username” with your own Sonic.net login name.

Of course, for a business, identity is important, and multihomed “Virtual Server” hosting for your own company domain is a great option. For more information on obtaining your own domain and multihomed hosting, please see the sales page at www.sonic.net/sales/. Note also that we’ll be releasing a new domain name registration tool in the next few days, so if you’re interested in that, wait a little to save some money!

-Kelsey and Dane

Wed Sep 5 11:06:55 PDT 2001 — Tsunami slowdown. Our mailbox server tsunami suffered an NFS bottleneck this morning. Ape’s port configuration for tsunami negotiated a 10Mbps connection rather than 100Mbps when tsunami was rebooted last night. This caused some customers to receive errors when checking email. Tsunami is now cruising along at 100Mbps. -Russ, Kelsey, Eli, Dane and Matt

Wed Sep 5 18:06:13 PDT 2001 — We just upgraded Sonic’s web counter to work with the new home page URLs. Users also no longer need to request that we add new domains to the configuration files as this is now handled internally. For information about the web counter or for instructions on how to add one to your home page, take a look at www.sonic.net/faq/web/counter.shtml -Nathan and Kelsey

Wed Sep 5 03:33:33 PDT 2001 — Night Operations Complete: All scheduled upgrades and maintenance has been completed with minimal customer impact. We appear to have restored FC-AL loop integrity on the NetApp filers and have restored full clustering support. Tsunami has been upgraded and is working well on its new hardware. -Nathan and Kelsey

Tue Sep 4 16:33:44 PDT 2001 — Night Operations: Tonight starting at midnight we will be performing some maintenance and upgrades to icebox, one of our NetApp filers, and tsunami, the server which handles mailbox accounts. We will also be upgrading and rebuilding the RAID in gale, our news feeder server. This is not a customer impacting event. Tsunami is getting a new motherboard and more RAM to resolve its ongoing performance issues. After the upgrade is complete, mailbox users should see a significant increase in performance. The upgrade should only take approximately 15 minutes to complete. Icebox is getting a new FC-AL adapter which will hopefully resolve the Fibre Channel Loop problems that we began seeing last week. The new LRC, which we replaced on one of the disk shelves on Saturday did not resolve the trouble. While we are replacing the FC-AL adapter icebox will be shutdown and all web data which it serves will be unavailable. Additionally, our administrative SQL server, which uses icebox for it’s database storage will offline for the duration of the maintenance. We expect that it should take approximately 20 minutes to replace and test the new FC-AL adapter. We will start the maintenance on icebox at 1:00AM. -Kelsey and Nathan

Tue Sep 4 16:07:16 PDT 2001 — Routing hiccup. We were resolving a minor BGP configuration error in our three core routers, mega, gamma, and delta, and unexpectedly prevented customers connected via mega from being able to access anything outside of our network. The problem existed for approximately ten minutes before it was, brought to our attention. This effected T3, T1, Frame Relay, and remote pop users. We’ve identified the problem that led to this and are working to fix it so we can go back to fixing BGP. We apologize for this brief service interruption. -Nathan, Scott, Eli and Kelsey.

Mon Sep 3 18:33:35 PDT 2001 — News Server downtime. News.sonic.net was offline between 5:25p and 6:25p this afternoon. The NNTP software package has been failing intermittently over the last 2-3 days, and in this case a delay in our notification package led to excessive downtime. We’ve tightened up monitoring, and will continue to work out the issues with the Typhoon software. – Eli, Kelsey

UPDATE: Typhoon, the news server software, has decided that it’s best action is to exit and dump core. We are in contact with the software vendor and waiting for a call back. Gale, our news feeder server is capable of back-logging about 6 hours of news for typhoon so as long as we are able to get it up within that time frame, no news posts should be lost. – Kelsey and Eli

UPDATE: We are seeing what appears to be some overview database corruption on typhoon. This manifests itself as garbled subject lines and/or the wrong post being returned for a given message id. In the two years that we’ve been running typhoon we’ve never seen anything like this before, in fact, the server had an uptime of 282 days until tonight. We are going to continue to work with the software vendor for a prompt resolution to this problem which will hopefully not involve the loss of our historical news spools. In the meantime if a group that you are reading exhibits the corruption use our backup news server, “supernews.sonic.net” in place of “news.sonic.net” -Kelsey, Dane, Nathan and Eli.

Fri Aug 31 18:18:07 PDT 2001 — Emergency maintenance on NetApp filer. One of the loop resiliency circuit cards in one of our NetApp filers has failed. While this is redundant equipment and didn’t cause any immediate impact, our redundancy is compromised, so we consider this a very high priority repair.

Saturday morning at 1:00AM, we will be shutting down freezer.sonic.net, our NFS filer hosting all storage for email, shell and ftp. This means that these services will not be available for about five to ten minutes. Normal maintenance of these filers does not require any downtime, and a second CPU/head allows for full redundancy in a failure or normal maintenance, however, in this case, the LRC card is what supports the redundant filer CPU/head. -Kelsey, Nathan and Dane

Fri Aug 31 17:14:54 PDT 2001 — Sonic.net has deployed a new network time protocol (NTP) server for customer use. You may synchronize your system clocks with the NTP protocol to time.sonic.net, aka ntp.sonic.net. “time” is a stratum 2 atomic synchronized time server for use by Sonic.net customers.

Client software is available for a variety of platforms. For Windows users, we recommend AboutTime, available at the following URL:

www.arachnoid.com/abouttime/index.html

For additional NTP resources, see www.ntp.org/.

-Eli and Dane