One of our central syslog servers had a series of raid failures that lead to filesystem corruption late last night. Normally, this would only be an issue for our backend systems but at approximately 8:30AM this morning local logging on our MX servers, which forward logs to that syslog server, blocked. This caused inbound mail handling to stop along with other services including the redis sentinel cluster that is used to store configuration information. That, in turn, caused the membertools to have issues since they depend upon some of the configuration stored in redis. The MX servers were all back up and running by 9:15AM. We’re continuing to work on restoring the failed syslog server but there is no customer impact at this time. – Sonic Ops
cant retrieve mail today at 4:27 pm
Yes, we had an mail outage from 4:15pm to 4:40pm.