Email connectivity for IMAP, POP3, and Webmail is currently down. We believe this to be caused by a power event effecting an isolated set of servers in our datacenter. We will post more information when the systems come back online.
— Joe and the SOC
Update: Our IMAP/POP3 cluster is having a thundering herd problem and we’re working hard to encourage it to come up smoothly. -Kelsey and William
Update: All services have been restored and stabilized. It appears that the root of the problem lies in our central radius authentication servers – while it may be as simple as needing to upgrade the existing cluster of 6 servers with faster CPUs, we suspect we may have hit an internal performance limitation in the radius server software that prevented them from being able answer requests fast enough. We strive for complete redundancy and should not have had any lasting stability problems after a few servers lost power. We’ll investigate this failure over the next few days and resolve as needed to ensure this problem doesn’t occur again. -Kelsey and William
Do you have power once again?
How long do you expect for recovery/restoral?
Thanks; Larry
Yes experiencing no ability to access email. When will this be fixed? Why the outage? This is not good.
Thanks.
Martha
OK – we waited an hour before writing …..
You mean SONIC has no back-up generators to support the system when there is a power failure!
Unbelievable!!! Even in India (where power failure is the norm) ISPs have generators that automatically kick-in and service is restored often w/in minutes.
I have been afraid of just this occurrence. SONIC has been great and as it grows, more and more errors (corporate/technical) proliferate. Is it time for us to find an ISP that is competent to handle power failures.
Yes all email services through Sonic are still down – was going to call support but it appears you are working on the issue – though i have no idea what a thundering herd issue is…!
Oh, lord save us, not a thundering herd problem? I felt a great disturbance in the Force… as if millions of voices suddenly cried out in terror and were suddenly silenced. I fear something terrible has happened.
We are doomed.
Thundering herds are the bane of load balanced cluster systems. You may have systems which are able to handle 20x the nominal load without the slightest issue but cannot stand up under the entire nominal load trying to come up all at once.
Email has been down (in RohnertPark) for several hours but Sonic claims to have tamed their Thundering Herd What’s Up?
Thanks for getting everything back up so quickly. Is it safe to assume that no email was lost? Just delayed?
This only affected peoples ability to send mail, briefly, and to check it via pop or imap. This had no impact at all on inbound mail which continued to flow normally.
Are you certain this is resolved? It’s 10:06pm on the 13th, and in San Jose, I still can’t access my Sonic email (three different addresses).
Unable to connect to mail servers.