Sun Nov 18 20:54:04 PST 2007 — Incoming mail server outage. Tonight our incoming mail server cluster stopped serving mail for POP3, IMAP, and webmail customers. Our operations staff is currently working on this problem. No other services should be affected. — John F
Update: Sun Nov 18 21:13:04 PST 2007 — One of our four clustered filers that handles customer email experienced a catastrophic disk failure. Catastrophic because it took both the A and B FC-AL loops causing all disks on it’s loop to go unavailable. As a result, all POP and IMAP services went off-line. A hard power reset brought all of the other disks back on-line and the offending disk has been removed. All services have now been restored and no data has been lost. -Kelsey and Nathan