Author: kelsey.cummings@sonic.com

We’ve been working on several improvements to our recursive DNS cluster configs to improve performance across the board and better support network growth in new regions beyond our existing service foot print in Northern and Southern California and have rolled out several config changes to the DNS proxies that handle ns1 and ns2.sonic.net over the past week.  What we believed was to be the last of those changes was pushed out this afternoon to the entire fleet after having cooked properly on a few systems at 3:15PM.  After that change was pushed, a significant portion of IPv6 DNS requests appeared to be black holed by some of the servers.  The issues continued until about 3:46PM. We are still unclear on the root cause of this but all services are currently stabilized and running as expected at this time.  We will continue to investigate in the hope that we can identify the cause, it seems possible it could be a bug in the dns specific load balancing software itself.

It is worth noting that our expectation was that most clients would have both v6 and v4 servers configured but it is evident that is not the case and it is likely that the majority of v6 enabled clients on our network with no fail over to v4 requests.  If you have static configured name servers, we’d suggest you list both the v6 and v4 address listed below.

2001:5a8::11
2001:5a8::33
50.0.1.1
50.0.2.2

-Kelsey, William and the rest of Systems.

Dane Jasper here, to write a final “message of the day” (MOTD) to honor the retirement of our shell server platform which will be turned off on June 30th when it’s operating system goes end of life.

Since starting Sonic in 1994, the use of Linux has been a key part of our operation, and providing remote text-based shell access to a Linux host has been included in our offerings.  Shell access was a very useful capability in the early days of the internet, when home systems were connected at slow speeds, while a centrally hosted shell server would have very high speed access to the internet.  And shell access provided a centralized gateway to much of the early culture of the internet, via text-based access to the Usenet discussion forums and internet relay chat (IRC) channels, email and more.  Today the internet has largely moved beyond these text-only applications, and Sonic members have too.  Today just one hundred die-hard members continue to log in and access our shell service platform.  As a result of the evolution of the internet away from shell applications and services, plus the workload involved in keeping a secure shell platform online, we have made the decision to finally shut down our Linux shell platform.

As an alternative, we would suggest now that high speed access is widely available at home, consider installing Linux on a small computer in your home!  This was how Scott and I got started with Linux at Sonic, back in the early days of “Sonoma InterConnect” (SonIC, get it?).

There are also lots of remote cloud-based shell solutions available now too.  Here are some starting points if you’d like to establish a shell account or set up a Linux box yourself:
https://www.linuxtrainingacademy.com/get-access-shell-account-learn-linux
https://aruljohn.com/freeshell

Please post questions or comments at https://forums.sonic.net/viewforum.php?f=13

For those needing to continue to maintain procmail recipes you willl need to use ftp://ftp.sonic.net/.

To our Linux shell server platform: So long, and thanks for all the fish.

-Dane, Scott, Kelsey and everyone at Sonic.