Author: admin

MySQL5 support added.

Our Database Member-Tool now offers you the choice between MySQL4, MySQL5 or PostgreSQL.

We encourage you to try it out and even try migrating your data from your MySQL4 instance to MySQL5 as this will enable all the features and performance increases in MySQL5 as well as landing you on a brand new piece of hardware specifically designed to host Customer SQL Data.

For future flexibility we have provided each Customer with a DNS entry in the form of [database name].db.sonic.net; giving you the ability to configure your programs once with one Host Name entry and be able to switch between various database back-ends simply by pointing DNS at the new server.

Please let us know what you think by sending feedback and comments to support@sonic.net .

Datacenter was on diesel

Our Santa Rosa datacenter facility is currently running on our backup generator due to problems with the automatic power transfer system. While we do not anticipate this will cause any impact for customers, it’s certainly a failure of a sort. Normally, the transfer switch starts the generator during a utility failure, but today it has triggered without cause. We are working to get back onto primary utility now.

Update: We’re back on utility, but managed to cause a fault our air handlers in the process of going back and forth multiple times. This bumped temperature in the datacenter from the typical 69 degrees up to over 90. All systems are back online now, and temperatures are now close to normal. There was no customer impact during this partial failure. -Dane, Russ, Kelsey, Don, Nathan and Jen

Update: ATS Fault Analysis and Repair.  After reviewing the situation with a GE/Zenith support tech last night it was concluded that the modular timer responsible for automatic genset exercising and transfer tests was triggering the erroneous transfers to emergency power.  This timer was ‘off’ when it failed and, indeed, never been used as we prefer to manually initiate our weekly genset exercise tests.  The faulty timer has been removed and we have every confidence that our ATS will work as expected from now on.  -Kelsey, Russ and Nathan

Transport Link Upgrade

Tonight at 12:01 AM we will be performing an upgrade on our transport link which connects our San Francisco POP to our San Jose POP. We will be moving to a new physical transport link which will provide us with the ability to upgrade the capacity of this link in the future.

All customer traffic will be routed away from this link prior to the operation, so there should be no customer impact. The operation is expected to take 30 minutes to complete.

-Jared and Nathan

Update: The transport link upgrade has been completed. Due to a tech in San Francisco accidentally moving an incorrect piece of fiber, there was a 5-10 minute interruption of traffic from one of our core routers. We apologize for this interruption. All network links have been restored to productivity and appear to be functioning normally at this time.

Momentary ATM Outage

While troubleshooting our new ATM OC-12 circuit in San Francisco, an AT&T tech inadvertently removed the wrong fibers from a multiplexer card in the SNFCCA17 central office. This caused two of our other ATM OC-12 circuits to go off-line. These links serve DSL, Business-T and Frame Relay customers. The AT&T tech quickly realized his error and fixed the problem. Total customer downtime was approximately 75 seconds, lasting from 1:29:05pm to 1:30:20pm. We’re currently in touch with management at AT&T to ensure that this does not happen again. -Nathan and Jared

Customer SQL v4 system maintenance

Tonight I will be performing maintenance on custsql.sonic.net, our mySQL 4x host. Just after 12:01AM I will be performing a necessary reboot of the system. Estimated customer impact is approximately 10 minutes at most. This does not effect mySQL v5 databases. -Don

Update: Maintenance completed. Customer impact was just under three minutes.

Squirrelmail config database maintenance

This morning at 12:01AM I performed some needed maintenance on our Squirrelmail configuration database. Some customers may have been unable to update their settings in Squirrelmail during this time. Customer impact was approximately 3 minutes. During this time, I also upgraded our Squirrelmail installation to the latest stable production version.  -Don

Sebastopol DSL Outage

Hardware failure on the Sebastopol Central Office has caused DSL customers to lose connectivity. We are working with AT&T to resolve the issue, and hope to have an estimated time of repair shortly.

-Adam, John and Steve

Update: As of 5:00PM, service appears to be restored to all affected customers.

DSL Aggregation Router Reboot

The DSL aggregation router that serves DSL to the Chico area rebooted itself approximately 20 minutes ago, causing about 5 minutes of downtime for all DSL customers in that area. Currently all traffic levels and customer connectivity look normal at this time, and we will continue to monitor the router, as well as investigate the cause of the spontaneous reboot.

We apologize for any inconvenience this outage may have caused.

-Jared

Trouble with the Sonic.net Website.

Fri Oct 10 10:05:08 PDT 2008 — Trouble with the Sonic.net Website. We are currently experiencing trouble with our corp.sonic.net Web Server that serves part of the Sonic.net Home Page and all of our Blogs. We are working on the problem and will have it back soon. -Augie, William, and Kelsey.

Update: We have restored services and determined the problem being a hardware failure. We apologize for the inconvenience this may have caused. -Don and William

Mail Storage Maintenance.

Tonight at Midnight we will be performing maintenance on the NetApp Filers that store Customer E-Mail; there is no expected downtime during this period and will take less than an hour to be completed.

During this period we will be adding additional capacity to these Filers; this capacity will allow for future growth of Customer E-Mail storage and improved performance for all of Sonic.net’s E-Mail Customers.

–Augie, Don, William, Sal, and Kelsey.

Update: maintenance has been completed; no problems were encountered (other than starting later than scheduled); capacity was nearly doubled on our E-Mail Storage System.  –Augie