Update:
All work has been completed as of 1:22am, April 12th.
Tonight, April 11th, 2013, at 11:59pm, we will be performing maintenance and updates on several of our servers. All major work should be completed within an hour. Affected services include customer databases, websites, and mail. Customer impact should be minimal.
— Grant & Joe
Sonic.net System Operations
We are presently investigating widespread reports of static or bad voice connections on inbound calls to customers on our voice network. An update will be provided shortly when we have further info.
-Matt and Brian
Update: We have confirmed this issue was caused by an upstream provider. They have reported it is mostly fixed. We await full confirmation of its resolution along with explanation of the cause.
Tonight, Tuesday April 9th, starting at 11:59 PM, we will be performing maintenance on ATM equipment serving legacy DSL, FRATM, and Biz-T customers. We expect downtime to be less than 10 minutes.
– Robbie, Tomoc
Update: The maintenance has been completed successfully.
We are pleased to announce that we’re ready for users to test our new webmail application. It features a modern interface that utilizes ajax and html5 and supports features like drag-and-drop and has a robust plugin system that allows us to extend its functionality. All of our existing webmail applications will be replaced by the new one on April 22nd. We encourage users to start using it now – it really is better in just about every way!
For now it is available at https://webmail-beta.sonic.net
Please let us know if you have any questions or feedback in the forums.
We are performing an emergency reload on equipment serving a subset of legacy DSL, Business-T, and FRATM customers in the Bay Area and Modesto/Stockton area. We will send another update shortly once the maintenance has completed.
-Tim
Update: Maintenance has been completed and all affected customers should be back online.
As of around 11:20PM PST we began experiencing issues with an upstream voice provider for our telecom network. We have confirmed it is an identified issue that they are working to resolve, and have taken steps to reroute our own network to mitigate customer impact. For the duration of this issue, customers may still experience intermittent inbound calling success. Further update will be provided upon outage resolution.
-Matt, Nathan and Tim
Update 1:48AM – Our upstream reports that the issue is resolved, and we’re seeing the same. -Nathan
This morning, at around 9:15AM, a DDoS attack towards Sonic.net owned equipment de-stabilized part of our hosting network resulting in the inability to access to the www.sonic.net website and some others. The issue has been handled and access to the sites restored as of 9:45AM.
– Noc and Soc
At 4:15pm, a small subset of legacy DSL subscriber connections became unstable. We are investigating the cause of this issues and are working to stabilize the affected circuits as fast as possible.
Update: All affected subscribers have been migrated to new equipment. Affected customers may have noticed several service interruptions lasting no more than 3 minutes each. Please note that under some circumstances, rebooting the DSL equipment on site may be required.
-Tomoc and Robbie
Over the past few days we’ve seen a massive increase in both the number and volume of DNS Amplification Attacks using our recursive name servers. This is likely due to the fact that our new name servers provide more verbose answers and are therefore amplify traffic more effectively than our old servers. We unfortunately had to roll back blocking off-net use of our recursive servers and blocking these requests entirely is not currently an option at this time. To mitigate the effects of the attacks both on our systems and their targets, we’ve instituted rate limits on the total number of queries per second any given IP address is able to source to our servers. The rate limits are high enough that they should not interfere with any normal (and acceptable) use. However, it is possible that a customer doing bulk DNS lookups (such as log processing or running a busy mail server) may run into issues and experience intermittent delays resolving host names.
-Kelsey, Augie and Nathan
DNS service was interrupted to our ns1.sonic.net name server cluster at 5pm PST for approximately 10 minutes.
All service has been restored and we are investigating the root cause of the problem.
–Augie
Update: Another brief service interruption today ( 21 March ) at 4:31pm PST for approximately 7 minutes as we were updating our name servers. Service was restored at 4:38pm PST.