Dr Jon Summers, DCA Advisory Board and Scientific Lead in Data Centres at Research Institutes of Sweden, offers his best practice advice and discusses holistic air-cooling control in depth.
When it comes to data centre operation, the control system and strategy that is adopted to keep all the IT systems humming within their desired environmental envelope is the prime focus. It is therefore not surprising that the providers of cooling equipment for data centres highlight their control systems as one of their most important and unique selling points. Digital control of the cooling systems has grown to become elaborate but can be seen as the differentiator when acquiring a supplier.
Data centres draw power from the electrical grid that is transformed and propagated to ultimately be ‘pumped’ into very many small volumes at up to 200A of current, only to be converted into the thermal energy at the rate of up to 1MW per square metre, where the heat is required to be transferred away from these small volumes and ultimately rejected or used to displace other sources of primary thermal energy generation. Principally, for data centres, heat is transferred by convection – where the thermal energy is picked up by a fluid (liquid or gas) and propagated by a flow. The heat transfer has three parts to it: first, from the small volumes (ok, lets just call them servers) to the edge of the data hall and then secondly through the walls of the hall, again this is done using a fluid to the point where the heat is transferred for the third time into the environment. This is also achieved by the convection of the external air or alternatively the heat can be carried away by convection to another heat transfer point for thermal energy applications.
In the north of Sweden, the three highlighted heat transfer processes can be performed using an approach called direct fresh air cooling, where the outside air – which is usually cool – is carried directly into the data hall via convection, using large cooling fans, where it is pushed towards the entrance of the air-cooled servers. The server fans then draw this air over their internal hot surfaces (where the electrical current stops in its tracks) by convection to the back of the servers, where the (now hot) air is pulled out of the data hall by extract fans with some of the air being drawn back to the cooling units and mixed with the incoming air to warm up outside air that during part of the year is too cold to operate with the servers. Of course, warm outside air can be cooled using water and very low relative humidity outside air can be humidified with water, so the cooling system requires an evaporative function that serves to both humidify and cool. As part of an EU H2020 funded project called BodenType DC (project 768875), a direct fresh air data centre was constructed in Boden, Sweden and operated for one and a half years where cooling control strategies could be developed and tested together with the collection of operation telemetry. Two partners, RISE AB and EcoCooling Ltd, respectively an expert in server operations based in Sweden and a direct fresh air cooler manufacturer based in the UK, managed to create a holistic cooling control strategy simply by taking control of the cooler supply and extract fans synchronously with control of all the server fans.
The holistic cooling control strategy makes use of the affinity laws of fans, half the speed of rotation of the fan uses only one-eighth of the fan power – the so-called cube law. Slowing down the server fans was achieved at the server level using a small PLC that interrupted the fan signal communication with the server and imposed a slower speed to target an imposed constant CPU temperature. All 480 servers in the BodenType DC data hall were operated with a targeted CPU temperature and the same 120-hour server digital workload profile was operated first with native server fan control and then with the targeted CPU temperature of 70°C. The servers reported all fan speeds to a Modbus server along with individual power consumption of each server that was constantly read by the EcoCooling direct fresh air-cooling control system, which in turn tuned the supply and extract fans to match the accumulated air consumption of the 480 servers.
Taking over the control of the fans in servers is perhaps not regarded as mainstream, but modern servers do offer some degree of control over their fans at the BIOS level. However, the holistic cooling control strategy, even without the overridden server fan control, does produce a more responsive cooling system as has been demonstrated by the BodenType DC project and it can be simply achieved by collecting aggregated server data that the cooling control system can use to set their fan cooler supply and extract speeds.
One consequence of targeting a constant CPU temperature is an interesting relationship between the instantaneous Power Usage Effectiveness (iPUE) and the temperature increase between the cold and hot areas, the ∆T, in the data hall.
Finally, holistic cooling control is not only restricted to direct fresh air-cooling systems, but the approach can also be readily applied to Computer Room Air Handling units and it is a vehicle to lower the (annualised) ISO PUE simply by improved fan speed control.
