Month: May 2002

Fri May 31 12:22:32 PDT 2002 — Sonic.net is moving our offices today at 5:00PM. We’re very excited to be moving, and everyone here is madly packing and preparing now. See the MOTD from Friday May 24th for some details on our new location including the address.

Technical support will be closed Saturday, and should open on Sunday at 9:00AM as usual. We expect ring-no-answer on support this evening, and an after hours message tomorrow. Please try us on Sunday if you need assistance! -Dane, and the whole staff

Thu May 30 13:02:19 PDT 2002 — One of our frame relay circuits is down. We are working with ASI to get this circuit back online. This is affecting a few frame relay customers. – Steve, Chris M. and Eli

UPDATE : Fri May 31 11:10:45 PDT 2002 – The Frame relay circuit was brought back up at 2:00pm Thursday after noon. Total down time was about an hour. – Steve

Wed May 29 10:15:22 PDT 2002 — TWIG, Sonic.net’s web mail program, will be taken off line for a software upgrade on 2002-05-29 at 2:00pm PDT.

We expect the upgrade to take 10 to 15 minutes to install but it does include the addition of a new database structure. Because of this, we must ensure no one is using the software while we migrate your data to the new database. At exactly 2:00pm, TWIG will be taken down and anyone using the software will unable to continue. We will install visible warnings in the TWIG software to remind those still on-line of the impending outage.

This software upgrade will bring TWIG to version 2.7.5 and includes the addition of several new features as well as some major and minor bug fixes.

-Chuckc, Kentrak

Wed May 29 14:32:58 PDT 2002 — TWIG upgrade complete! The TWIG upgrade went smoothly and is now complete. All customer data has been successfully migrated to the new database structure.

Thank you for your patience during this upgrade and we apologize for any inconvenience the down time may have caused you.

-Chuckc, Kentrak

Sun May 26 01:04:35 PDT 2002 — We’re doing a variety of small upgrades tonight. The SQL database server is currently under construction, and we’re doing some work on the SSL server as well. Both of these are hardware changes to keep ahead of the performance curve.

The big upgrade of the night is the Network Appliance filers. We’ll be bringing both of our massive redundant storage arrays up to newer operating systems with new features and important bug fixes. We expect five to ten minutes of local storage blocking during the upgrade. -Dane, Kelsey and Matt

Update, Sun May 26 03:56:46 PDT 2002, all upgrades are complete. -Dane

Fri May 24 14:16:48 PDT 2002 — Sonic.net is moving, and telephone technical support will be closed on June 1st. We will also be shutting down early on Friday, at 5pm to begin the move. Support will re-open on Sunday the 2nd at 9am. Regular office hours for administrative staff will not be impacted, and our office will open at the new location at 8am on Monday the 3rd. The new address is 2260 Apollo Way, Santa Rosa, CA 95407.

Our new office offers a lot of new features for customers and staff. New geographically diverse fiber rings provide even more redundancy than our current config, and there’s a lot more bandwidth headroom for growth.

At the new building, three fiber rings with diverse building entrances on three different paths from two different carriers offer speeds into the Nx-Gigabit range. They circle the bay three different ways, diverse of any city or region, leaving Santa Rosa via multiple paths, North, South and East.

The data center is raised floor, with Leibert cooling and UPS systems, and a 23.9 liter v-12 twin-turbocharged diesel keeps it all running in case of a PG&E outage. Initially, the datacenter will hold five times as many cabinets as the current downtown location, but a second undeveloped space provides for ten times as much capacity (10,000 square feet total).

Actual datacenter and service moves will begin around June 13th, and will continue into late July. Our expectation is that various services will be briefly interrupted during migration, and we will post advance notice here in the MOTD when the dates are solidified.

You can see the lobby at the new building at neteye.sonic.net/. We’ll be bringing additional cameras in the datacenter online and publishing more information as time allows. Meanwhile, please direct questions or discussions to news:sonic.net

-Dane, Scott, Eli, Jen and everyone else!

Tue May 21 12:43:59 PDT 2002 — Broadlink situation has been resolved. They were under a Denial Of Service from an off-site network, and the source has been isolated. – Eli, Support

Tue May 21 09:44:13 PDT 2002 — BroadLink outage. BroadLink is experiencing a network wide outage. They are working to have their network back up as soon as possible. Every Sonic.net account comes with backup dialup server. To find a local dialup number for your area you can visit the sonic.net pop finder tool at:

www.sonic.net/cgi-bin/pops.pl

We will keep you informed as to the status of this outage via the MOTD. – Sonic.net Staff

Thu May 16 09:54:42 PDT 2002 — Our domain registration and transfer tools will be taken off line for a registry imposed maintenance window.

Date: Saturday, May 18th Time: 6:00pm to 10:00pm

This maintenance window has been set by the gTLD registry operator and will affect all ICANN accredited registrars equally. We apologize for any inconvenience this may cause you.

-Chuckc

Tue May 14 12:30:22 PDT 2002 — NetApp problems: At around 8:00AM this morning a run-away process filled the mail spool partition. Some cruft was cleared and additional space was allocated to the mail spool after the problem was brought to our attention. However, this has not resolved the problems with the mail spool directory. At this point, although there is plenty of space available, the NetApp is returning ‘No Space Available’ when a process tries to create a file name of greater than 16 characters. Files with names shorter than 8 characters can be created just fine. This is only affecting the mail quota tree on the NetApp, and not the home or ftp trees. We’ve been working with NetApp for the past few hours providing their engineers with proper debugging output and at this point we are waiting to hear back from them.

At this time, we are accepting and delivering inbound mail and users should be able to read and send their email normally. However, some shell based mail applications are reporting errors that they are unable to create lock files. Using the shell based email clients at this time could potentially cause some mailbox corruption, but it is highly unlikely to occur. -Kelsey, Nathan and Matt.

UPDATE: We are still waiting to hear back from NetApp on the analysis of our debugging output. We are reluctant to take any action without NetApp approval. Mail delivery is slowed down at this time but no email has been, or will be, lost.

UPDATE: We think we might have solved the problem but are running some tests to confirm that everything is back to normal.