Month: October 1997

Mon Oct 20 08:52:35 PDT 1997 — Brian and I have just finished a set of upgrades and fixes. Some services were offline between 2:30am and 7am. Notably, web services were offline from 3am to 6am. The FTP server’s filesystems were available for read only use between 2:30 and 4am, and at the end of this time, we put the NetApp filer online for ftp. All terminal servers got a RAM upgrade between 6am and 7am. This will fix short memory problems which were causing inconsistent connection performance last week. Unfortunately, due to problems with the new motherboard, Thunder’s CPUs, RAM and networking were not upgraded, and we were unable to move the filesystems to the NetApp. This was a major goal of this maintenance, and we’ll be doing some preparation today in hopes of trying again tomorrow. -Dane & Brian

Fri Oct 17 12:01:08 PDT 1997 — We recently upgraded all of our Livingston terminal servers to a new OS version, 3.7. This version fixes quite a few problems which we were having, and it’s been quite stable. Unfortunately, it uses a bit more RAM than the old version, and many of the terminal servers need to be upgraded. The RAM is here, but the devices are always in use, so it’s difficult to schedule the work. This became a problem this morning when two of the devices (out of 18) ran short on memory and started behaving erraticly. We have rebooted them, and things look much better now. Observed behavior included failure to authenticate users, termination of connections and slow performance. The affected devices were pm2.sonic.net and pm4.sonic.net. -Dane

Fri Oct 17 17:15:42 PDT 1997 — Maintenance is scheduled for Monday morning at 1am. We will be working on upgrades to Thunder and Ultra, plus upgrading RAM in the remainder of the Livingston terminal servers. In addition, we are planning a test deployment of the Network Appliance filer. Expect intermittent availability of some services until about 6am.

Fri Oct 17 17:12:08 PDT 1997 — We’ve just completed a set of LAN backbone upgrades here. All LAN connections are now via three large switches which are redundantly connected at 200Mbps. Major devices are in the process of having their connections upgraded. Here’s a list of some of the devices and connection speeds. At 200Mbps we’ve got all of the switches plus Mega and the NetApp. At 100Mbps we’ve got Thunder, Ultra, Slam, Blam and Bolt.

Thu Oct 16 15:24:51 PDT 1997 — A brief power outage caused some problems with a few bits of our network, and some users were cut off or may have had problems getting to the Internet for a few minutes. When this happens, the authentication server has a tendency to fall behind, and some of the less intelligent networking gear makes things worse. -Dane

Sat Oct 11 10:14:53 PDT 1997 — Our new Network Appliance disk array has arrived, and we’re working on testing and deploying it now. The Network Appliance will allow us to have a large shared redundant filesystem which will be mounted on multiple web server, the ftp and shell servers, etc. This means we can scale up our web hosting and serve even more traffic and sites. The NetApp’s filesystem can be grown by adding new disks as we need more space. Currently it’s configured with seven four-gig disks: one for parity, one hot spare and five data disks. This yields about 16 usable gigabytes of storage. Oh ya – we could have bought a Z3. -Dane

Tue Oct 7 17:48:13 PDT 1997 — We currently are experiencing problems with our X2 hunt group. Basicly, one line is bad in the group, and this is yielding a busy signal because it can’t get past that line. You can dial over this trouble by calling 547-3100 if you’d like, or dial the non-X2 number at 522-1001. PacBell has escalated the trouble ticket, and we’re expecting repair shortly. No, we’re not really busy – yes, we really do have enough lines! -Dane

Sun Oct 5 18:07:30 PDT 1997 — We’re beginning to enforce the 12 meg quota in the user home directory area, as that filesystem is getting full. There is a 12 meg soft quota, and a 24 meg hard quota. The soft quota has a three day grace period. Please make sure that your usage is below 12 megs, or you may suddenly find that you can’t write files when we turn this on late Monday afternoon. -Dane

Sun Oct 5 15:51:35 PDT 1997 — We’ve added 36 new modems and lines in the past two days. This extra capacity will help us during particularly busy times on Monday and Tuesday evenings. See the line utilization graph for details on usage: www.sonic.net/lines/

Sixteen more lines are scheduled for addition Tuesday or Wednesday, and more are on order with Bell. -Dane and Brian

Wed Oct 1 21:17:19 PDT 1997 — On October first we brought up a brand new T1 circuit direct to Sprintlink. This rounds out our direct connections, as we’ve now got direct fast links to the three largest backbones. This means low latency and packet loss, which translates to snappy, reliable Internet performance. This triad of circuits also means that our connectivity is sized so that an outage of any one circuit will not impact our network. Next time you’re shopping for ISPs, ask ‘are you connected directly to the big three?’