At 7:11PM tonight one of our two Core4 CRAC (Computer Room Air Conditioning) units unexpectedly shut itself down. Nothing instills fear more than receiving pages titled “Sys A Enable Switch Turned Off / Service Now” and “High Discharge Air Temperature” in rapid succession. After the initial panic passed, it was clear that all redundant systems we operating correctly and the second system had responded correctly and ramped up to handle the total cooling load. Once on site, there was no outward indication why System A had shutdown as the system enable switch was correctly in the “On” position and it had supply power. However, upon further investigation, it was apparent that the enable switch was water-logged, oxidized and shorted-out, signaling the system to shutdown. The switch has been removed from service and both systems are 100% operational again.
Although it is disappointing to see a system failure caused by something as simple as an improperly weatherproofed mechanical room and control panel, it is rewarding to see our commitment to and investment in redundancy pay off. And, ultimately, that the prototype Core4 CRAC system behaved as expected.
Special thanks goes out to Jimmy and Kent of Bell Products who interrupted their dinners to come out and verify that all systems were functioning correctly.
-Kelsey, Nathan and Russ