Thu May 24 22:33:36 PDT 2001 — We’ve made some software changes on the shell server which have broken a few utilities temporarily. The items which are currently unavailable include ping, traceroute, ssh and screen.

Sorry for the inconvenience, we will work to get these back online shortly. Please use your own workstation for these tools in the mean time.

In other shell server news, we found that a cable was causing some network errors, causing slow performance, particularly with NFS recently. After some investigation, the cable has been replaced, and we’re back up to full speed. -Dane, Scott and Nathan (the cool cable sleuth)

Thu May 24 20:55:43 PDT 2001 — We will be performing maintenance on the 1003 dialup group this evening starting at 11:30pm. Some users who are connected to the 1003 dialup number may be disconnected. The window for this work is between 11:30pm and 12:30am. This maintenance includes moving a number of T1 PRI circuits from copper to our fiber facilities.

Update: All completed. -Steve and Kevan

Thu May 24 17:03:30 PDT 2001 — Due to a cable connector problem, we’ve just suffered about 15 minutes of intermittent performance. The connection between our core switch and one of our two redundant core routers was severed temporarily, apparently due to problems with a cable connector latch in the patch bay. In theory, the second connection to the second core router should have taken on all of the traffic, and it’s redundant connection to the affected router would have carried the traffic. This deployment is still in progress, and hasn’t been fully tested.

We’re sorry about the poor performance, and we’ll take steps to wrap up the testing of the redundancy to assure that a minor failure like this can’t impact things badly. -Dane, Nathan, Eli, Steve and Kevan

Wed May 23 02:09:26 PDT 2001 — Since Friday afternoon ape, the core switch, had been intermittently exhibiting some strange behavior delaying ICMP traffic (pings and traceroutes) on some ports while still running at wire speed for other IP traffic. In some cases the delay introduced into ICMP traffic could be as much as 2 seconds. Because of the problem did not appear to be affecting normal traffic, we left the switch in this state in order to facilitate the troubleshooting and debbuging process with Extreme Networks. Extreme Networks finished the information gathering process tonight and we went ahead and rebooted the switch at 1:25AM to clear the problem. The switch booted cleanly and appears to be fully functional. We are working closely with Extreme Networks to resolve the ongoing problems that we are having with their equipment. -Kelsey

Tue May 22 12:01:29 PDT 2001 — Some non-intrusive testing of our 1003 dialup gear turned out to be more intrusive then we expected. This caused the 1003 dialup group to return either a fast busy or a ‘All circuits are busy’ message. In order to clear up this issue we needed to reset some modems. This caused a few people to be disconnected. Sorry for any problems this may have caused. -Steve

Wed May 16 15:19:40 PDT 2001 — Sonic.net is now offering Sun Netra X1’s on our leased colocation plan! For as low as $195 a month, you can colocate a Sun server with a 400 MHz UltraSPARC processor and from 128-512 Megabytes of RAM. For more details, see our sales page at www.sonic.net/sales/sun/

-Kevan

Wed May 16 14:16:13 PDT 2001 — We completed the upgrades of our local DNS servers yesterday afternoon. We now have three new dedicated DNS servers on our local network and are currently securing a facility for a fourth off-site server on the East Coast to increase the resiliency of our DNS services, especially in the event of network outages here at our NOC — or the West Coast in general.

The new DNS servers are considerably more responsive than the two old servers and have significantly more robust hardware beneath them. You may notice faster DNS resolution times and more responsive web browsing. We’ve published a brief white paper on the new DNS servers at www.sonic.net/network/ along with a link to our public network statistics. If you have any questions we’ll be happy to respond to them in news:sonic.net

-Kelsey

Tue May 15 18:24:51 PDT 2001 — A transient problem with our mail routing system caused e-mails to some address to be rejected for 1 hour and 20 minutes today.

The problem began at 11:47 A.M. and lasted until 1:06 P.M. affecting 101 specific addresses.

We will notify the individual senders and recipients by email so that they can re-send the messages.

-The Sonic Ops Staff

Sun May 13 03:18:42 PDT 2001 — Night Operations: Completed. Extreme Networks worked on site with us tonight to replace our core switch’s faulty MSM modules with two new MSMs. Both of the new MSMs appear to be fully functional: the switch now passes all of its diagnostics and boots without any errors. While we were installing and testing the new modules the switch was rebooted several times over a 30 minute period which resulted in intermittent network outages. Thanks for the help Dennis!

We also finished migrating sonic.sonic.net, or core administrative server and primary DNS server, from 208.201.224.11 to it’s new IP address. Now that 208.201.224.11 is no longer tied to sonic.sonic.net we are free to complete the migration to our new DNS server architecture.

-Kelsey, Scott, Chris, Nathan, and Dennis from Extreme Networks.

Thu May 10 13:05:06 PDT 2001 — ‘Homepage’ virus outbreak. We’ve got filtering in place for the latest Microsoft Visual Basic (.vbs) virus. For more information on the virus here is a link to Symantec’s website: www.symantec.com/avcenter/venc/data/vbs.vbswg2.x@mm.html -Kelsey