Author: admin

Mon Jul 24 11:34:29 PDT 2000 — Nas22.sonic.net rebooted this morning at around 11:30am this would have caused some people who where dialed into the 707-522-10003 dialup group to be disconnected. I am currently investigating as to why this happened. -Steve

Sun Jul 23 12:03:05 PDT 2000 — The remote access equipment that serves the 522-1002 dial-up group blew a circuit breaker a half and hour ago as the load grew this morning. Eli made it down to our NOC to redistribute the equipment across a second circuit and has restored all access to the 522-1002 number. PostOffice, our web based email client, was broken in the ssl server upgrade. Nathan and Scott are working on it now and should have both PostOffice and its replacement, Twig, working shortly. -Kelsey

Sun Jul 23 03:54:10 PDT 2000 — Night operations are complete, and went well. Kelsey and Scooter upgraded the news server to 1 gig of RAM, and it has been moved into a new cabinet location. They also moved mistral, our third web server, into a cabinet and rearranged power for our core LAN rack so that we now have diverse dual-UPS power for the load balancing switches. Steve moved the older analog dial group to new power outlets from our second huge APC Symmetra UPS. Nathan and Scott worked to upgrade our SSL server with a new OS on a new disk. The BroadLink guys moved their DSLAM into their dedicated rack here in the data-center. Then we had doughnuts, but the BroadLink folks went missing before eating theirs, and we’re not quite sure why. -Dane (and a bunch of sleepy ops)

Thu Jul 20 13:59:16 PDT 2000 — This coming Sunday morning at 12:30am we will be performing maintenance and upgrades on a number of facilities here.

The older analog 522-1001 dialup group will be moving to our new power plant, and any connected customers will be disconnected. It’ll be down for just a few minutes as we move power cables.

The news server will be moved to a new location in the data center, and we’ll be putting a full gig of RAM in it. In addition, we’re hoping to get a new disk array online for faster performance.

Broadlink will be doing DSLAM maintenance and upgrades, and wireless DSL customers will be offline for a little while beginning at 1am.

We’ll be working on the configuration in our SMS DSL ATM router, and all DSL customers will be offline briefly while we reset the equipment with the new configuration.

SSL hosted customers on ssl.sonic.net will be down briefly while that system is upgraded with new hardware. The new deployment has been in test at ssl2.sonic.net, and all customers have been notified to test their CGI configurations.

We hope to be finished in three to four hours with all of these tasks. -Dane, Scott, Eli, Steve, Kelsey, Nathan and Scooter

Sat Jul 8 15:12:26 PDT 2000 — We’re continuing to deploy broadband content for DSL customers, and we’ve added a new section to our home page for this material. We’re continuing to work with our content partners to bring additional streams online. The content partnerships and hardware deployments in place with Akamai, iBeam, Multicast ISP and Yahoo Broadcast have allowed us to bring quite a bit of new content in on satellite in into caching proxies. With three satellite dishes on the roof now, and two more in negotiations, Sonic.net is ramping up lots of new streaming media in the near future for broadband customers. -Dane

Sat Jul 8 15:04:43 PDT 2000 — To improve performance of our mail server array, we’ve turned off GCOS matching for incoming email. Previously, if an email came in for an address that was not a Sonic.net username, the mail servers would attempt to match that username against the GCOS, which contains the user’s full name. So, for every SPAM or misdirected email, the mail servers would search the entire passwd’s GCOS looking for that name. During times when we’re bounding piles of SPAM, this was a significant impact on the mail servers. This impact was causing rare cases where YP would fail, and we’d bounce some legitimate customer email back to the sender as ‘user unknown’.

Our concern was that turning off the GCOS matching would break first.last@sonic.net delivery addresses which customers might have been using. Users could previously receive email as John.Doe@sonic.net for example. An analysis of the few million emails we’ve processed here in the last 10 days was done by Nathan and shows that no one uses this feature, so we’ve turned it off to eliminate the impact on the servers. If you’d like an alias for first.last@sonic.net, or any other email alias, just email support@sonic.net and we’d be happy to set one up for you. -Dane, Scott and Nathan.

Fri Jul 7 01:15:05 PDT 2000 — After 12 hours of non-stop work the news server is back online. We apologize for the extended downtime. Many thanks to our friend Ross for his assistance with Solaris. -Scott, Eli Kelsey and Dane

Thu Jul 6 12:15:48 PDT 2000 — We’re taking the News Server offline for 10 minutes to upgrade its BIOS. This hopefully will fix the problems we’ve been having integrating our new RAID array. The new RAID array will hold ‘overview spools,’ which will substantially speed up the time to open a new newsgroup. -Kelsey, Eli, and Scott

Thu Jul 6 23:32:40 PDT 2000 — After much bad luck with system corruption and attempted upgrades gone wrong, we’re in the process of a new installation onto a fresh hard disk for our Usenet news server. Once that’s complete, the /opt filesystem with the Typhoon news software, databases and spools will be mounted to bring news back online.

Meanwhile, we’ve of course got our redundant backup news system at supernews.sonic.net that you can use. It doesn’t have the sonic.* local groups, but you’ll find it to be quite complete and responsive.

Scott, Eli and Kelsey are pulling the night watch to bring this system back online, and I’ll be covering the early AM shift while they get some much needed rest Friday. -Dane

Thu Jul 6 19:35:13 PDT 2000 — After many Solaris nightmares, we can confirm that we most definitely love Linux much more than Solaris x86. We’re escalating with Sun’s engineering group to get additional assistance, and we hope to wrap this up as soon as possible. Basicly, the BIOS upgrade caused the PCI bus to enumerate the devices differently, and this confused the operating system.