Month: November 2013

Intrusive IPv6 Tunnel Maintenance

Tonight, November 25, starting at 11:59pm, we will be performing maintenance on equipment terminating customer IPv6 tunnels. Please note this maintenance will only affect connectivity for customers who have explicitly opted in to our IPv6 tunnel services. Expected downtime for v6 tunnels is 10 minutes.

Update: Maintenance complete. Due to an issue with one of our upstream providers, v6 downtime lasted approximately 50 minutes. All service was restored by 1:05AM.

-Tomoc

Non-intrusive Voice Network Maintenance

11/26 – 12:12A – This maintenance is now complete.

Tonight starting at 11:59PM PST we will be performing a software upgrade on portions of our voice network in Santa Rosa. There is no expected impact to customers, and the maintenance should last no longer than 20 minutes.

-Tim J.

Non-intrusive Voice Network Maintenance

Update – 11:58PM PST – This maintenance is now complete.

Tonight starting at 11:35PM PST, we will be performing a software upgrade of core voice equipment in Santa Rosa. There is no service impact expected during this upgrade, and this is expected to last 15-20 minutes.

Non-Intrusive Network Maintenance

Tonight, Wednesday, November 20th, starting at 11:59pm, we will be performing non-intrusive network maintenance in our Northern California core network.

Update: Due to a configuration error, customers may have experienced issues reaching destinations outside of the Sonic.net network. All reachability issues were resolved within 10 minutes.

-Tomoc

Database Server Maintenace

Tonight, November 15th, at 11pm, Operations will be taking one of our internal database servers offline to clear a hardware error. Work is expected to take no more than 30 minutes.

Update: All work is complete.

— Joe and the SOC

Webmail Outage

Tonight, between approximately 7:50p and 8:20p, webmail was unavailable for half of our customers. Operations has identified a database configuration error as the root cause and is taking steps to ensure the same failure will not happen again.

— Joe and the SOC

Mail Server Upgrades

Tonight, at 12:00AM, we will migrate “mail.sonic.net” to a new cluster.  Users may be unable to send mail through mail.sonic.net during the migration which is expected to take 10-15 minutes to complete.  There’s no new exciting features to tout, just a fresh OS and faster modern hardware.

In other news, our MX server cluster now supports SSL encryption for inbound connections.  This allows the session between the connecting server and our’s to be fully encrypted preventing messages from being available “on the wire” in clear text.  Our outbound servers have supported this kind of encryption when sending mail to destinations that support it for more than a decade.  At this time it looks like approximately 15-20% of inbound email is encrypted in transit.

-Kelsey

Update:  Wed Nov 13 00:48:37 PST 2013, the upgrades are finished and went over as planned.

 

Non-intrusive Voice Network Maintenance

11/5/13 – 12:09AM PST – This maintenance is now complete.

Tonight at 11:59PM PST I will be performing maintenance on a portion of our voice network in Santa Rosa. No customer impact is expected and the maintenance should last between 15 and 30 minutes.

–Tim J.