U.S. patent application number 12/556040 was filed with the patent office on 2011-03-10 for apparatus and method for adjusting coolant flow resistance through liquid-cooled electronics rack(s).
This patent application is currently assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION. Invention is credited to Wayne A. Barringer, David P. Graybill, Madhusudan K. Iyengar, Roger R. Schmidt, James J. Steffes, Gerard V. Weber, JR..
Application Number | 20110056675 12/556040 |
Document ID | / |
Family ID | 43646778 |
Filed Date | 2011-03-10 |
United States Patent
Application |
20110056675 |
Kind Code |
A1 |
Barringer; Wayne A. ; et
al. |
March 10, 2011 |
APPARATUS AND METHOD FOR ADJUSTING COOLANT FLOW RESISTANCE THROUGH
LIQUID-COOLED ELECTRONICS RACK(S)
Abstract
Apparatuses and methods are presented for adjusting coolant flow
resistance through one or more liquid-cooled electronics racks.
Flow restrictors are employed in association with multiple heat
exchange tube sections of a heat exchange assembly, or in
association with a plurality of coolant supply lines or coolant
return lines feeding multiple heat exchange assemblies. Flow
restrictors associated with respective heat exchange tube sections
(or respective heat exchange assemblies) are disposed at the
coolant channel inlet or coolant channel outlet of the tube
sections (or of the heat exchange assemblies). These flow
restrictors tailor coolant flow resistance through the heat
exchange tube sections or through the heat exchange assemblies to
enhance overall heat transfer within the tube sections or across
heat exchange assemblies by tailoring coolant flow. In one
embodiment, the flow restrictors tailor a coolant flow distribution
differential across multiple heat exchange tube sections or across
multiple heat exchange assemblies.
Inventors: |
Barringer; Wayne A.;
(Poughkeepsie, NY) ; Graybill; David P.;
(Poughkeepsie, NY) ; Iyengar; Madhusudan K.;
(Poughkeepsie, NY) ; Schmidt; Roger R.;
(Poughkeepsie, NY) ; Steffes; James J.;
(Poughkeepsie, NY) ; Weber, JR.; Gerard V.;
(Poughkeepsie, NY) |
Assignee: |
INTERNATIONAL BUSINESS MACHINES
CORPORATION
Armonk
NY
|
Family ID: |
43646778 |
Appl. No.: |
12/556040 |
Filed: |
September 9, 2009 |
Current U.S.
Class: |
165/299 ;
165/80.4 |
Current CPC
Class: |
H05K 7/20736 20130101;
H05K 7/20763 20130101; H05K 7/20709 20130101; H05K 7/20272
20130101; H05K 7/20281 20130101; H05K 7/2079 20130101 |
Class at
Publication: |
165/299 ;
165/80.4 |
International
Class: |
G05D 23/00 20060101
G05D023/00; F28F 13/00 20060101 F28F013/00 |
Claims
1. A cooling apparatus for an electronics rack, the cooling
apparatus comprising: an air-to-liquid heat exchange assembly for
an electronics rack, wherein air moves through the electronics rack
from an air inlet side to an air outlet side thereof and the
air-to-liquid heat exchange assembly is configured to couple to the
electronics rack at one of the air inlet side or the air outlet
side thereof, and wherein the air-to-liquid heat exchange assembly
comprises: an air-to-liquid heat exchanger comprising a coolant
inlet plenum, a coolant outlet plenum, and multiple heat exchange
tube sections coupled in parallel between the coolant inlet plenum
and the coolant outlet plenum, each heat exchange tube section
comprising a coolant channel having a coolant channel inlet and a
coolant channel outlet, each coolant channel inlet being in fluid
communication with the coolant inlet plenum and each coolant
channel outlet being in fluid communication with the coolant outlet
plenum, wherein system coolant flows through the multiple heat
exchange tube sections in parallel; and at least one flow
restrictor associated with the multiple heat exchange tube
sections, each flow restrictor of the at least one flow restrictor
being associated with a respective heat exchange tube section of
the multiple heat exchange tube sections and being disposed at one
of the coolant channel inlet or the coolant channel outlet thereof,
the at least one flow restrictor tailoring coolant flow resistance
through the respective heat exchange tube section to enhance heat
transfer within the multiple heat exchange tube sections of the
air-to-liquid heat exchanger.
2. The cooling apparatus of claim 1, wherein the air-to-liquid heat
exchange assembly further comprises multiple flow restrictors
associated with the multiple heat exchange tube sections, each flow
restrictor of the multiple flow restrictors being associated with a
respective heat exchange tube section of the multiple heat exchange
tube sections and being disposed at one of the coolant channel
inlet or the coolant channel outlet thereof, and wherein the
multiple flow restrictors tailor a coolant flow distribution
differential through the multiple heat exchange tube sections, the
coolant flow distribution differential being tailored based on at
least one of an air temperature differential across at least two
heat exchange tube sections of the multiple heat exchange tube
sections or location of the multiple heat exchange tube sections
relative to the electronics rack when the air-to-liquid heat
exchange assembly is coupled thereto.
3. The cooling apparatus of claim 1, wherein the at least one flow
restrictor facilitates defining different coolant flow resistances
through at least two heat exchange tube sections of the multiple
heat exchange tube sections.
4. The cooling apparatus of claim 3, wherein the air-to-liquid heat
exchange assembly further comprises multiple flow restrictors
associated with the multiple heat exchange tube sections, each flow
restrictor of the multiple flow restrictors being associated with a
respective heat exchange tube section of the multiple heat exchange
tube sections and being disposed at one of the coolant channel
inlet or the coolant channel outlet thereof, and wherein each flow
restrictor of the multiple flow restrictors comprises a respective
fixed diameter orifice, and at least two respective fixed diameter
orifices of at least two flow restrictors of the multiple flow
restrictors have different diameters.
5. The cooling apparatus of claim 3, wherein the air-to-liquid heat
exchanger assembly further comprises multiple flow restrictors
associated with the multiple heat exchange tube sections, each flow
restrictor of the multiple flow restrictors being associated with a
respective heat exchange tube section of the multiple heat exchange
tube sections and being disposed at one of the coolant channel
inlet or the coolant channel outlet thereof, and wherein the
multiple flow restrictors tailor coolant flow resistance through
the respective heat exchange tube sections to achieve a uniform
coolant flow across the multiple heat exchange tube sections of the
air-to-liquid heat exchanger.
6. The cooling apparatus of claim 1, wherein the at least one flow
restrictor comprises at least one adjustable flow restrictor, each
adjustable flow restrictor of the at least one adjustable flow
restrictor comprising a dynamically adjustable orifice opening size
for dynamically adjusting system coolant flow resistance through
the respective heat exchange tube section of the multiple heat
exchange tube sections.
7. The cooling apparatus of claim 6, wherein the dynamically
adjustable orifice opening size of the at least one adjustable flow
restrictor is one of passively controlled or actively controlled,
and wherein the passively controlled adjustable flow restrictor
comprises a temperature sensor for sensing airflow across the at
least one respective heat exchange tube section and for adjusting
the dynamically adjustable orifice opening size based on sensed
temperature of the airflow, and the actively controlled adjustable
flow restrictor comprises a pressure and temperature sensor for
sensing pressure and temperature of system coolant at least one of
the coolant channel inlet or the coolant channel outlet of the
respective heat exchange tube section and for adjusting the
dynamically adjustable orifice opening size based, at least in
part, on sensed pressure and temperature of system coolant within
the respective heat exchange tube section at the coolant channel
inlet or the coolant channel outlet thereof.
8. A cooling apparatus for a plurality of electronics racks, each
electronics rack comprising a heat exchange assembly, the cooling
apparatus comprising: a coolant distribution unit for supplying
cooled system coolant to the heat exchange assemblies of the
plurality of electronics racks; a plurality of coolant supply
lines, each coolant supply line being in fluid communication with
the coolant distribution unit and the respective heat exchange
assembly of a respective electronics rack of the plurality of
electronics racks and facilitating supply of system coolant from
the coolant distribution unit to the respective heat exchange
assembly; a plurality of coolant return lines, each coolant return
line coupling in fluid communication the heat exchange assembly of
a respective electronics rack of the plurality of electronics racks
with the coolant distribution unit and facilitating return of
exhausted system coolant from the respective heat exchange assembly
to the coolant distribution unit, wherein system coolant circulates
in a closed loop between the coolant distribution unit and the heat
exchange assemblies via, at least in part, the plurality of coolant
supply lines and the plurality of coolant return lines; and a
plurality of flow restrictors associated with at least one of the
plurality of coolant supply lines or the plurality of coolant
return lines, each flow restrictor being associated with a
respective coolant line of the plurality of coolant supply lines or
the plurality of coolant return lines for tailoring coolant flow
resistance through the respective heat exchange assembly, and
wherein the plurality of flow restrictors tailor coolant flow
resistance through at least one of the plurality of coolant supply
lines or the plurality of coolant return lines to enhance overall
heat transfer through the heat exchange assemblies of the plurality
of electronics racks.
9. The cooling apparatus of claim 8, wherein the plurality of flow
restrictors tailor a coolant flow distribution differential through
the heat exchange assemblies of the plurality of electronics racks,
the coolant flow distribution differential being tailored based on
rack-level power consumption of the plurality of electronics
racks.
10. The cooling apparatus of claim 9, wherein the plurality of flow
restrictors facilitate exhausting system coolant being in a
super-heated, thermodynamic state.
11. The cooling apparatus of claim 8, wherein the plurality of flow
restrictors facilitate defining different coolant flow resistances
through at least two heat exchange assemblies of the plurality of
electronics racks.
12. The cooling apparatus of claim 8, wherein at least one flow
restrictor of the plurality of flow restrictors comprises at least
one adjustable flow restrictor, each adjustable flow restrictor of
the at least one adjustable flow restrictor comprising a
dynamically adjustable orifice opening size for dynamically
adjusting system coolant flow resistance through the respective
heat exchange assembly.
13. The cooling apparatus of claim 12, further comprising coolant
pressure and temperature sensors associated with at least one of
the plurality of coolant supply lines or the plurality of coolant
return lines for sensing pressure and temperature of system coolant
passing therethrough, wherein sensed pressure and temperature of
the system coolant facilitates dynamic adjustment of at least one
adjustable orifice opening size of the at least one adjustable flow
restrictor.
14. The cooling apparatus of claim 8, further comprising a
controller for adjusting pump speed of a coolant pump of the
coolant distribution unit based on total power consumption of the
plurality of electronics racks, wherein the plurality of flow
restrictors facilitate directing a higher system coolant flow to at
least one heat exchange assembly of at least one electronics rack
of the plurality of electronics racks with a higher power
consumption than at least one other electronics rack of the
plurality of electronics racks.
15. The cooling apparatus of claim 8, wherein at least one heat
exchange assembly of the heat exchange assemblies of the plurality
of electronics racks comprises: an air-to-liquid heat exchanger
comprising a coolant inlet plenum, a coolant outlet plenum, and
multiple heat exchange tube sections coupled in parallel between
the coolant inlet plenum and the coolant outlet plenum, each heat
exchange tube section comprising a coolant channel having a coolant
channel inlet and a coolant channel outlet, each coolant channel
inlet being in fluid communication with the coolant inlet plenum
and each coolant channel outlet being in fluid communication with
the coolant outlet plenum, and wherein system coolant flows through
the multiple heat exchange tube sections in parallel; and at least
one flow restrictor associated with the multiple heat exchange tube
sections, each flow restrictor of the at least one flow restrictor
being associated with a respective heat exchange tube section of
the multiple heat exchange tube sections and being disposed at one
of the coolant channel inlet or the coolant channel outlet thereof,
the at least one flow restrictor tailoring coolant flow resistance
through the respective heat exchange tube section to enhance heat
transfer within the multiple heat exchange tube sections of the
air-to-liquid heat exchanger.
16. The cooling apparatus of claim 15, wherein the at least one
air-to-liquid heat exchange assembly further comprises multiple
flow restrictors associated with the multiple heat exchange tube
sections, each flow restrictor of the multiple flow restrictors
being associated with a respective heat exchange tube section of
the multiple heat exchange tube sections and being disposed at one
of the coolant channel inlet or the coolant channel outlet thereof,
and wherein the multiple flow restrictors tailor a coolant flow
distribution differential through the multiple heat exchange tube
sections, the coolant flow distribution differential being tailored
based on at least one of an air temperature differential across at
least two heat exchange tube sections of the multiple heat exchange
tube sections or location of the multiple heat exchange tube
sections relative to the respective electronics rack.
17. The cooling apparatus of claim 15, wherein the at least one
flow restrictor facilitates defining different coolant flow
resistances through at least two heat exchange tube sections of the
multiple heat exchange tube sections.
18. The cooling apparatus of claim 15, wherein the at least one
flow restrictor comprises at least one adjustable flow restrictor,
each adjustable flow restrictor of the at least one adjustable flow
restrictor comprising a dynamically adjustable orifice opening size
for dynamically adjusting system coolant flow resistance through
the respective heat exchange tube section of the multiple heat
exchange tube sections.
19. A method of facilitating cooling of a plurality of electronics
racks, each electronics rack comprising a heat exchange assembly,
the method comprising: providing a plurality of coolant supply
lines and a plurality of coolant return lines coupled in fluid
communication between a coolant distribution unit and the heat
exchange assemblies of the plurality of electronics racks, the
coolant distribution unit supplying cooled system coolant for the
heat exchange assemblies, wherein when operational, system coolant
circulates in a closed loop between the coolant distribution unit
and the heat exchange assemblies via, at least in part, the
plurality of coolant supply lines and the plurality of coolant
return lines; and providing a plurality of flow restrictors
associated with at least one of the plurality of coolant supply
lines or the plurality of coolant return lines, each flow
restrictor being associated with a respective coolant line of the
plurality of coolant supply lines or the plurality of coolant
return lines for tailoring coolant flow resistance through the heat
exchange assembly of the respective electronics rack, and wherein
the plurality of flow restrictors tailor coolant flow resistance
through at least one of the plurality of coolant supply lines or
the plurality of coolant return lines to enhance overall heat
transfer through the heat exchange assemblies of the plurality of
electronics racks.
20. The method of claim 19, further comprising facilitating cooling
of multiple heat exchange tube sections of the heat exchange
assembly of at least one electronics rack of the plurality of
electronics racks, wherein the heat exchange assembly of the at
least one electronics rack comprises: an air-to-liquid heat
exchanger comprising a coolant inlet plenum, a coolant outlet
plenum, and multiple heat exchange tube sections coupled in
parallel between the coolant inlet plenum and the coolant outlet
plenum, each heat exchange tube section comprising a coolant
channel having a coolant channel inlet and a coolant channel
outlet, each coolant channel inlet being in fluid communication
with the coolant inlet plenum and each coolant channel outlet being
in fluid communication with the coolant outlet plenum, wherein
system coolant flows through the multiple heat exchange tube
sections in parallel; and wherein the facilitating cooling
comprises providing at least one flow restrictor associated with
the multiple heat exchange tube sections, each flow restrictor of
the at least one flow restrictor being associated with a respective
heat exchange tube section of the multiple heat exchange tube
sections and being disposed at one of the coolant channel inlet or
the coolant channel outlet thereof, the at least one flow
restrictor tailoring coolant flow resistance through the respective
heat exchange tube section to enhance heat transfer within the
multiple heat exchange tube sections of the at least one heat
exchange assembly.
Description
BACKGROUND
[0001] The present invention relates in general to a method and
apparatus for adjusting coolant flow resistance within one or more
liquid-cooled electronics racks or between multiple electronics
racks.
[0002] The power dissipation of integrated circuit chips, and the
modules containing the chips, continues to increase in order to
achieve increases in processor performance. This trend poses a
cooling challenge at both module and system level. Increased
airflow rates are needed to effectively cool high power modules and
to limit the temperature of the air that is exhausted into the
computer center.
[0003] In many large server applications, processors along with
their associated electronics (e.g., memory, disk drives, power
supplies, etc.) are packaged in removable drawer configurations
stacked within a rack or frame. In other cases, the electronics may
be in fixed locations within the rack or frame. Typically, the
components are cooled by air moving in parallel airflow paths,
usually front-to-back, impelled by one or more air moving devices
(e.g., fans or blowers). In some cases it may be possible to handle
increased power dissipation within a single drawer by providing
greater airflow, through the use of a more powerful air moving
device or by increasing the rotational speed (i.e., RPMs) of an
existing air moving device. However, this approach is becoming
problematic at the rack level in the context of a computer
installation (i.e., data center).
[0004] The sensible heat load carried by the air exiting the rack
is stressing the availability of the room air-conditioning to
effectively handle the load. This is especially true for large
installations with "server farms" or large banks of computer racks
close together. In such installations, liquid cooling (e.g., water
or refrigerant cooling) is an attractive technology to manage the
higher heat fluxes. The liquid absorbs the heat dissipated by the
components/modules in an efficient manner. Typically, the heat is
ultimately transferred from the liquid to an outside environment,
whether air or liquid cooled.
BRIEF SUMMARY
[0005] In one aspect, the shortcomings of the prior art are
overcome and additional advantages are provided through the
provision of a cooling apparatus for an electronics rack. The
cooling apparatus includes an air-to-liquid heat exchange assembly
for the electronics rack, wherein air moves through the electronics
rack from an air inlet side to an air outlet side thereof, and the
air-to-liquid heat exchange assembly is configured to couple to the
electronics rack at one of the air inlet side or the air outlet
side. The air-to-liquid heat exchange assembly includes an
air-to-liquid heat exchanger and at least one flow restrictor. The
air-to-liquid heat exchanger comprises a coolant inlet plenum, a
coolant outlet plenum, and multiple heat exchange tube sections
coupled in parallel between the coolant inlet plenum and the
coolant outlet plenum. Each heat exchange tube section includes a
coolant channel having a coolant channel inlet and a coolant
channel outlet. Each coolant channel inlet is in fluid
communication with the coolant inlet plenum and each coolant
channel outlet is in fluid communication with the coolant outlet
plenum, wherein coolant flows through the multiple heat exchange
tube sections in parallel. The at least one flow restrictor is
associated with the multiple heat exchange tube sections. Each flow
restrictor of the at least one flow restrictor is associated with a
respective heat exchange tube section and is disposed at one of the
coolant channel inlet or the coolant channel outlet thereof. When
operational, the at least one flow restrictor tailors coolant flow
resistance through the respective heat exchange tube section to
enhance heat transfer within the multiple heat exchange tube
sections of the air-to-liquid heat exchanger.
[0006] In another aspect, a cooling apparatus for a plurality of
electronics racks is provided, wherein each electronics rack
includes a heat exchange assembly. The cooling apparatus includes a
coolant distribution unit, a plurality of coolant supply lines, a
plurality of coolant return lines, and a plurality of flow
restrictors. The coolant distribution unit supplies cooled system
coolant to the heat exchange assemblies of the plurality of
electronics racks, and each coolant supply line of the plurality of
coolant supply lines is coupled in fluid communication with the
coolant distribution unit and the heat exchange assembly of a
respective electronics rack for facilitating supply of system
coolant from the coolant distribution unit to the respective heat
exchange assembly. Each coolant return line is coupled in fluid
communication between the heat exchange assembly of a respective
electronics rack and the coolant distribution unit for facilitating
return of exhausted system coolant from the heat exchange assembly
to the coolant distribution unit. In operation, system coolant
circulates in a closed loop between the coolant distribution unit
and the heat exchange assemblies via, at least in part, the
plurality of coolant supply lines and the plurality of coolant
return lines. The plurality of flow restrictors are associated with
at least one of the plurality of coolant supply lines or the
plurality of coolant return lines. Each flow restrictor is
associated with a respective coolant line of the plurality of
coolant supply lines or the plurality of coolant return lines for
tailoring coolant flow resistance through the respective heat
exchange assembly. The plurality of flow restrictors tailor coolant
flow resistance through at least one of the plurality of coolant
supply lines or the plurality of coolant return lines to enhance
overall heat transfer through the heat exchange assemblies of the
plurality of electronics racks.
[0007] In a further aspect, a method of facilitating cooling of a
plurality of electronics racks is provided, wherein each
electronics rack comprises a heat exchange assembly. The method
includes: providing a plurality of coolant supply lines and a
plurality of coolant return lines coupled in fluid communication
between a coolant distribution unit and the heat exchange
assemblies of the plurality of electronics racks, the coolant
distribution unit supplying cooled system coolant for the heat
exchange assemblies, wherein when operational, system coolant
circulates in a closed loop between the coolant distribution unit
and the heat exchange assemblies via, at least in part, the
plurality of coolant supply lines and the plurality of coolant
return lines; and providing a plurality of flow restrictors
associated with at least one of the plurality of coolant supply
lines or the plurality of coolant return lines, each flow
restrictor being associated with a respective coolant line of the
plurality of coolant supply lines or the plurality of coolant
return lines for tailoring coolant flow resistance through the heat
exchange assembly of the respective electronics rack, and wherein
the plurality of flow restrictors tailor coolant flow resistance
through at least one of the plurality of coolant supply lines or
the plurality of coolant return lines to enhance overall heat
transfer through the heat exchange assemblies of the plurality of
electronics racks.
[0008] Further, additional features and advantages are realized
through the techniques of the present invention. Other embodiments
and aspects of the invention are described in detail herein and are
considered a part of the claimed invention.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0009] The subject matter which is regarded as the invention is
particularly pointed out and distinctly claimed in the claims at
the conclusion of the specification. The foregoing and other
objects, features, and advantages of the invention are apparent
from the following detailed description taken in conjunction with
the accompanying drawings in which:
[0010] FIG. 1 depicts one embodiment of a conventional raised floor
layout of an air-cooled data center;
[0011] FIG. 2 depicts one problem addressed by the present
invention, showing recirculation of airflow patterns in one
implementation of a raised floor layout of an air-cooled data
center;
[0012] FIG. 3 is a top plan view of one embodiment of an
electronics rack with an air-to-liquid heat exchanger mounted to an
outlet door thereof, in accordance with one aspect of the present
invention;
[0013] FIG. 4 is a top plan view of one embodiment of a data center
employing cooling apparatuses comprising outlet door air-to-liquid
heat exchangers, in accordance with an aspect of the present
invention;
[0014] FIG. 5 is a schematic of one embodiment of a coolant
distribution unit to be used in the data center of FIG. 4, in
accordance with an aspect of the present invention;
[0015] FIG. 6 is a partial cross-sectional, elevational view of one
embodiment of an electronics rack door and cooling apparatus
mounted thereto, taken along line 6-6 in FIG. 7, in accordance with
an aspect of the present invention;
[0016] FIG. 7 is a cross-sectional, top plan view of the door and
cooling apparatus of FIG. 6, taken along line 7-7 in FIG. 6, in
accordance with an aspect of the present invention;
[0017] FIG. 8 is a schematic of one embodiment of a data center
comprising a cooling apparatus for distributing coolant flow
between electronics racks of the data center, in accordance with an
aspect of the present invention;
[0018] FIG. 9A is a graph of one embodiment of pressure drop
through several heat exchangers at different load versus coolant
flow rates through the heat exchangers, illustrating potential
mal-distribution of coolant flow across the heat exchangers of
multiple electronics racks of a data center, which is addressed in
accordance with an aspect of the present invention;
[0019] FIG. 9B is a graph of pressure drop through several heat
exchangers at different load versus coolant flow rates through the
heat exchangers, wherein multiple flow restrictors are utilized to
tailor coolant flow resistance through the heat exchangers of the
multiple electronics racks to ensure that the high heat load
electronics rack receives a maximum coolant flow, in accordance
with an aspect of the present invention;
[0020] FIG. 10 depicts one embodiment of logic for adjusting
coolant flow resistance to multiple heat exchange assemblies of
multiple electronics racks to be cooled, in accordance with an
aspect of the present invention;
[0021] FIG. 11 is a partial cross-sectional, elevational view of
the electronics rack door and cooling apparatus of FIG. 6, with a
plurality of flow restrictors shown disposed at the coolant channel
inlets and coolant channel outlets of the multiple heat exchange
tube sections, and taken along line 11-11 in FIG. 12, in accordance
with an aspect of the present invention;
[0022] FIG. 12 is a cross-sectional, top plan view of the door and
cooling apparatus of FIG. 11, taken along line 12-12 in FIG. 11, in
accordance with an aspect of the present invention;
[0023] FIG. 13 is a partial cross-sectional, elevational view of
one embodiment of an electronics rack door and cooling apparatus
mounted thereto with multiple flow restrictors at the coolant
channel inlets of the heat exchange tube sections which comprise
different-sized orifice diameters that tailor coolant flow
resistance through the multiple heat exchange tube sections, in
accordance with an aspect of the present invention;
[0024] FIG. 14 is a schematic of one embodiment of a heat exchange
assembly comprising a heat exchange tube section with a first,
fixed diameter flow restrictor at the coolant channel inlet of the
heat exchange tube section and a second, fixed diameter flow
restrictor at the coolant channel outlet of the heat exchange tube
section for adjusting coolant flow resistance through the tube
section, in accordance with an aspect of the present invention;
[0025] FIG. 15A is a schematic of one embodiment of a flow
restrictor with a fixed orifice diameter for tailoring coolant flow
resistance, in accordance with an aspect of the present
invention;
[0026] FIG. 15B depicts an alternate embodiment of a flow
restrictor with a fixed orifice diameter for tailoring coolant flow
resistance, in accordance with an aspect of the present
invention;
[0027] FIG. 16 is a schematic of an alternate embodiment of a heat
exchange assembly comprising a heat exchanger with a passively
controlled, adjustable flow restrictor disposed at the coolant
channel inlet of the heat exchange tube section for dynamically
adjusting coolant flow resistance through the heat exchange tube
section based on sensed airflow temperature across the heat
exchange tube section, in accordance with an aspect of the present
invention;
[0028] FIG. 17 depicts an alternate embodiment of a heat exchange
assembly comprising a heat exchange tube section with an actively
controlled, adjustable flow restrictor at the coolant channel inlet
thereof, and pressure and temperature sensors at the coolant
channel inlet and coolant channel outlet of the heat exchange tube
section for dynamically adjusting the orifice opening size of the
actively controlled, adjustable flow restrictor based on sensed
pressure and temperature of coolant within the heat exchange tube
section, in accordance with an aspect of the present invention;
and
[0029] FIG. 18 depicts one embodiment of logic for controlling
coolant flow resistance through multiple heat exchange tube
sections of a heat exchange assembly associated with an electronics
rack to enhance heat transfer within the multiple heat exchange
tube sections of the heat exchange assembly, in accordance with an
aspect of the present invention.
DETAILED DESCRIPTION
[0030] As used herein, the terms "electronics rack", "rack-mounted
electronic equipment", and "rack unit" are used interchangeably,
and unless otherwise specified include any housing, frame, rack,
compartment, blade server system, etc., having one or more heat
generating components of a computer system or electronics system,
and may be, for example, a stand alone computer processor having
high, mid or low end processing capability. In one embodiment, an
electronics rack may comprise multiple electronics subsystems, each
having one or more heat generating components disposed therein
requiring cooling. "Electronics subsystem" refers to any
sub-housing, blade, book, drawer, node, compartment, etc., having
one or more heat generating electronic components disposed therein.
Each electronics subsystem of an electronics rack may be movable or
fixed relative to the electronics rack, with the rack-mounted
electronics drawers of a multi-drawer rack unit and blades of a
blade center system being two examples of subsystems of an
electronics rack to be cooled.
[0031] "Electronic component" refers to any heat generating
electronic component of, for example, a computer system or other
electronics unit requiring cooling. By way of example, an
electronic component may comprise one or more integrated circuit
dies and/or other electronic devices to be cooled, including one or
more processor dies, memory dies and memory support dies. As a
further example, the electronic component may comprise one or more
bare dies or one or more packaged dies disposed on a common
carrier. As used herein, "primary heat generating component" refers
to a primary heat generating electronic component within an
electronics subsystem, while "secondary heat generating component"
refers to an electronic component of the electronics subsystem
generating less heat than the primary heat generating component to
be cooled. "Primary heat generating die" refers, for example, to a
primary heat generating die or chip within a heat generating
electronic component comprising primary and secondary heat
generating dies (with a processor die being one example).
"Secondary heat generating die" refers to a die of a multi-die
electronic component generating less heat than the primary heat
generating die thereof (with memory dies and memory support dies
being examples of secondary dies to be cooled). As one example, a
heat generating electronic component could comprise multiple
primary heat generating bare dies and multiple secondary heat
generating dies on a common carrier. Further, unless otherwise
specified herein, the term "liquid-cooled cold plate" refers to any
conventional thermally conductive structure having a plurality of
channels or passageways formed therein for flowing of liquid
coolant therethrough. In addition, "metallurgically bonded" refers
generally herein to two components being welded, brazed or soldered
together by any means.
[0032] As used herein, "air-to-liquid heat exchange assembly" means
any heat exchange mechanism characterized as described herein
through which liquid coolant can circulate; and includes, one or
more discrete air-to-liquid heat exchangers coupled either in
series or in parallel. An air-to-liquid heat exchanger may
comprise, for example, one or more coolant flow paths, formed of
thermally conductive tubing (such as copper or other tubing) in
thermal or mechanical contact with a plurality of air-cooled
cooling fins. Size, configuration and construction of the
air-to-liquid heat exchange assembly and/or air-to-liquid heat
exchanger thereof can vary without departing from the scope of the
invention disclosed herein. A "liquid-to-liquid heat exchanger" may
comprise, for example, two or more coolant flow paths, formed of
thermally conductive tubing (such as copper or other tubing) in
thermal communication with each other. Size, configuration and
construction of the liquid-to-liquid heat exchanger can vary
without departing from the scope of the invention disclosed herein.
Further, "data center" refers to a computer installation containing
one or more electronics racks to be cooled. As a specific example,
a data center may include one or more rows of rack-mounted
computing units, such as server units.
[0033] One example of facility coolant and system coolant is water.
However, the concepts disclosed herein are readily adapted to use
with other types of coolant on the facility side and/or on the
system side. For example, one or more of the coolants may comprise
a brine, a fluorocarbon liquid, a liquid metal, or other similar
coolant, or refrigerant, while still maintaining the advantages and
unique features of the present invention. As a specific example,
the concepts described hereinbelow with reference to FIGS. 8-18
employ water as facility coolant and a refrigerant as system
coolant.
[0034] Reference is made below to the drawings (which are not drawn
to scale to facilitate understanding of the invention), wherein the
same reference numbers used throughout different figures designate
the same or similar components.
[0035] As shown in FIG. 1, in a raised floor layout of an air
cooled computer installation or data center 100 typical in the
prior art, multiple electronics racks 110 are disposed in one or
more rows. A computer installation such as depicted in FIG. 1 may
house several hundred, or even several thousand microprocessors. In
the arrangement of FIG. 1, chilled air enters the computer room via
floor vents from a supply air plenum 145 defined between the raised
floor 140 and a base or sub-floor 165 of the room. Cooled air is
taken in through louvered covers at air inlet sides 120 of the
electronics racks and expelled through the back (i.e., air outlet
sides 130) of the electronics racks. Each electronics rack 110 may
have an air moving device (e.g., fan or blower) to provide forced
inlet-to-outlet airflow to cool the electronic components within
the drawer(s) of the rack. The supply air plenum 145 provides
conditioned and cooled air to the air-inlet sides of the
electronics racks via perforated floor tiles 160 disposed in a
"cold" air aisle of the data center. The conditioned and cooled air
is supplied to plenum 145 by one or more conditioned air units 150,
also disposed within the computer installation 100. Room air is
taken into each conditioned air unit 150 near an upper portion
thereof. This room air comprises in part exhausted air from the
"hot" air aisles of the data center defined by opposing air outlet
sides 130 of the electronics racks 110.
[0036] Due to the ever increasing airflow requirements through
electronics racks, and limits of air distribution within the
typical computer room installation, recirculation problems within
the room may occur. This is shown in FIG. 2 for a raised floor
layout, wherein hot air recirculation 200 occurs from the air
outlet sides 130 of the electronics racks back to the cold air
aisle defined by the opposing air inlet sides 120 of the
electronics rack. This recirculation can occur because the
conditioned air supplied through tiles 160 is typically only a
fraction of the airflow rate forced through the electronics racks
by the air moving devices disposed therein. This can be due, for
example, to limitations on the tile sizes (or diffuser flow rates).
The remaining fraction of the supply of inlet side air is often
made up by ambient room air through recirculation 200. This
recirculating flow is often very complex in nature, and can lead to
significantly higher rack unit inlet temperatures than might be
expected.
[0037] The recirculation of hot exhaust air from the hot aisle of
the computer room installation to the cold aisle can be detrimental
to the performance and reliability of the computer system(s) or
electronic system(s) within the racks. Data center equipment is
typically designed to operate with rack air inlet temperatures in
the 18-35.degree. C. range. For a raised floor layout such as
depicted in FIG. 1, however, temperatures can range from
15-20.degree. C. at the lower portion of the rack, close to the
cooled air input floor vents, to as much as 45-50.degree. C. at the
upper portion of the electronics rack, where the hot air can form a
self-sustaining recirculation loop. Since the allowable rack heat
load is limited by the rack inlet air temperature at the "hot"
part, this temperature distribution correlates to an inefficient
utilization of available air conditioning capability. Computer
installation equipment almost always represents a high capital
investment to the customer. Thus, it is of significant importance,
from a product reliability and performance view point, and from a
customer satisfaction and business perspective, to achieve a
substantially uniform temperature across the air inlet side of the
rack unit. The efficient cooling of such computer and electronic
systems, and the amelioration of localized hot air inlet
temperatures to one or more rack units due to recirculation of air
currents, are addressed by the apparatuses and methods disclosed
herein.
[0038] FIG. 3 depicts one embodiment of a cooled electronics
system, generally denoted 300, in accordance with an aspect of the
present invention. In this embodiment, electronics system 300
includes an electronics rack 310 having an inlet door 320 and an
outlet door 330, which respectively have openings to allow for the
ingress and egress of external air, respectively, through the air
inlet side and air outlet side of electronics rack 310. The system
further includes at least one air-moving device 312 for moving
external air across at least one electronics subsystem 314
positioned within the electronics rack. Disposed within outlet door
330 is an air-to-liquid heat exchanger 340 across which the
inlet-to-outlet airflow through the electronics rack passes. A
cooling unit 350 is used to buffer the air-to-liquid heat exchanger
from facility coolant 360, for example, provided via a computer
room water-conditioning unit (not shown). Air-to-liquid heat
exchanger 340 removes heat from the exhausted inlet-to-outlet
airflow through the electronics rack via the system coolant, for
ultimate transfer in cooling unit 350 to facility coolant 360 via
liquid-to-liquid heat exchanger 352 disposed therein. This cooling
apparatus advantageously reduces heat load on existing
air-conditioning units within the data center, and facilitates
cooling of electronics racks by cooling the air egressing from the
electronics rack and thus cooling any air recirculating to the air
inlet side thereof.
[0039] As shown in FIG. 3, a system coolant loop 345 couples
air-to-liquid heat exchanger 340 to cooling unit 350. In one
embodiment, the system coolant employed is water. By way of
example, such a system is described in U.S. Pat. No. 7,385,810 B2,
issued Jun. 10, 2008, and entitled "Apparatus and Method for
Facilitating Cooling of an Electronics Rack Employing a Heat
Exchange Assembly Mounted to an Outlet Door Cover of the
Electronics Rack".
[0040] In this co-pending application, the inlet and outlet plenums
mount within the door and are coupled to supply and return
manifolds disposed beneath a raised floor. Presented hereinbelow
are enhanced variations on such an outlet door heat exchanger.
Specifically, disclosed hereinbelow is an air-to-liquid heat
exchanger which employs a pumped refrigerant as the system coolant.
Connection hoses for the pumped refrigerant system are, in one
embodiment, metal braided hoses, and the system coolant supply and
return headers for the pumped refrigerant system are mounted
overhead relative to the electronics racks within the data center.
Thus, for the pumped refrigerant system described below, system
coolant enters and exits the respective system coolant inlet and
outlet plenums at the top of the door and rack. Further, because
pumped refrigerant is employed, the hose and couplings used in the
pumped refrigerant systems described below are affixed at both
ends, i.e., to the system coolant plenums on one end and to the
overhead supply and return headers on the other end.
[0041] FIG. 4 is a plan view of one embodiment of a data center,
generally denoted 400, employing cooled electronics systems, in
accordance with an aspect of the present invention. Data center 400
includes a plurality of rows of electronics racks 310, each of
which includes an inlet door 320 and a hinged outlet door 330, such
as described above in connection with the embodiment of FIG. 3.
Each outlet door 330 supports an air-to-liquid heat exchanger and
system coolant inlet and outlet plenums as described further
hereinbelow. Multiple cooling units 350, referred to hereinbelow as
pumping units, are disposed within the data center (along with one
or more air-conditioning units (not shown)). In this embodiment,
each pumping unit forms a system coolant distribution subsystem
with one row of a plurality of electronics racks. Each pumping unit
includes a liquid-to-liquid heat exchanger where heat is
transferred from a system coolant loop to a facility coolant loop.
Chilled facility coolant, such as water, is received via facility
coolant supply line 401, and is returned via facility coolant
return line 402. System coolant, such as refrigerant, is provided
via a system coolant supply header 410 extending over the
respective row of electronics racks, and is return via a system
coolant return header 420 also extending over the respective row of
electronics racks. In one embodiment, the system coolant supply and
return headers 410, 420 are hard-plumbed within the data center,
and preconfigured to align over and include branch lines extending
towards electronics racks of a respective row of electronics
racks.
[0042] FIG. 5 depicts one embodiment of a cooling unit 350 for the
data center 400 of FIG. 4. Liquid-to-liquid heat exchanger 352
condenses a vapor-liquid refrigerant mixture passing through the
system coolant loop comprising system coolant supply header 410 and
system coolant return header 420. (In one embodiment, the system
coolant has undergone heating and partial vaporization within the
respective air-to-liquid heat exchangers disposed within the outlet
doors of the electronics racks.) The facility coolant loop of
liquid-to-liquid heat exchanger 352 comprises facility coolant
supply line 401 and facility coolant return line 402, which in one
embodiment, provide chilled facility water to the liquid-to-liquid
heat exchanger. A control valve 501 may be employed in facility
coolant supply line 401 to control facility coolant flow rate
through the liquid-to-liquid heat exchanger 352. After the
vapor-liquid refrigerant mixture condenses within liquid-to-liquid
heat exchanger 352, the condensed refrigerant is collected in a
condensate reservoir 510 for pumping via a redundant pump assembly
520 back to the respective row of electronics racks via system
coolant supply header 410. As shown in FIG. 5, a bypass line 530
with a bypass valve 531 may be employed to control the amount of
system coolant fed back through the system coolant supply header,
and hence, control temperature of system coolant delivered to the
respective air-to-liquid heat exchangers mounted to the doors of
the electronics racks.
[0043] FIGS. 6 & 7 depict one embodiment of outlet door 330
supporting air-to-liquid heat exchanger 340, and system coolant
inlet and outlet plenums 601, 701. Referring to both figures
collectively, outlet door frame 331 supports a rigid flap 600,
which attaches, for example, by brazing or soldering, to a plate
710 secured between the system coolant inlet plenum 601 and system
coolant outlet plenum 701.
[0044] In FIG. 6, right angle bend 610 is shown disposed at the top
of system coolant inlet plenum 601. This right angle bend defines a
horizontal inlet plenum portion, which extends above the top of
door 330. The coolant inlet to system coolant inlet plenum 601 is
coupled to a connect coupling 611 for facilitating connection
thereof to the respective supply hose, as described above. The
air-to-liquid heat exchanger comprises a plurality of
horizontally-oriented heat exchange tube sections 620. These heat
exchange tube sections 620 each comprise a coolant channel having
an inlet and an outlet, with each coolant channel being coupled to
the system coolant inlet plenum 601 and each coolant channel outlet
being coupled to the system coolant outlet plenum 701. A plurality
of fins 630 are attached to horizontally-oriented heat exchange
tube sections 620 for facilitating transfer of heat from air
passing across the air-to-liquid heat exchanger to coolant flowing
through the plurality of heat exchange tube sections 620. In one
embodiment, the plurality of fins are vertically-oriented,
rectangular fins attached to horizontally-oriented heat exchange
tube sections 620.
[0045] Due to the low saturation (boiling) temperature of liquid
refrigerant, removal of a heat load exiting the back of an
electronics rack via the refrigerant will cause the refrigerant to
vaporize within the heat exchange tube sections of the
air-to-liquid heat exchanger, resulting in two-phase flow and
latent heat transfer. Two-phase latent heat transfer is very
effective as a heat removal method; however, problems occur in the
area of refrigerant flow distribution within the air-to-liquid heat
exchanger and across multiple air-to-liquid heat exchangers of the
data center due to vaporization of the refrigerant.
[0046] For example, within an air-to-liquid heat exchanger at the
air outlet side of an electronics rack such as described above,
liquid refrigerant is pumped into a vertical supply plenum, from
which the refrigerant flows through several parallel heat exchange
tube sections spanning the width of the air-to-liquid heat
exchanger, eventually mixing in the vertical return plenum. As a
result of slightly lower refrigerant flow rates in the lower heat
exchange tube sections of the air-to-liquid heat exchanger caused
by pressure drops due to pipe fittings and friction, refrigerant
flowing through these lower sections will have a tendency to
vaporize first upon introduction of a (uniform) heat load to the
air-to-liquid heat exchanger.
[0047] When liquid refrigerant vaporizes in one of the heat
exchange tube sections due to an applied heat load, the pressure
drop experienced across that heat exchange tube section will equal
several times the magnitude of the pressure drop experienced by
single-phase liquid refrigerant flowing through the tube section.
This increased pressure drop creates a "resistance" for the
refrigerant to flow in the lower tube sections where two-phase
latent heat transfer is occurring. As liquid flows through the
coolant inlet plenum, with several parallel paths to choose from,
more liquid will flow through the tube section with the least
resistance, that is, the lowest pressure drop. It has been observed
through testing that latent heat removal affects increase from the
upper sections of the rear door heat exchanger to the lower
sections thereof. The greater the degree of vaporization due to
increased latent heat transfer occurring in the lower heat exchange
tube sections, the larger the pressure drop, which causes a
mal-distribution of refrigerant flow through the heat exchanger
(and higher coolant pumping power consumption). Increased amounts
of liquid bypass the lower sections of the rear door heat
exchanger, where latent heat transfer is occurring, resulting in
increased single-phase liquid flow through the upper heat exchange
tube sections and decreased two-phase flow through the lower heat
exchange tube sections of the air-to-liquid heat exchanger.
Single-phase refrigerant flow does not provide the desired heat
removal effects of latent heat transfer, and thus is to be
avoided.
[0048] It is one goal of the present invention to develop an
effective mechanism for eliminating mal-distribution of refrigerant
flow through multiple electronics racks of a data center, as well
as within a heat exchange assembly between the heat exchange tube
sections thereof to enhance heat transfer and/or minimize coolant
pumping requirements.
[0049] FIGS. 8-10 address coolant flow mal-distribution between
electronics racks of a data center, while FIGS. 11-18 address
mal-distribution of coolant flow between heat exchange tube
sections of a heat exchange assembly coupled to or associated with
an electronics rack.
[0050] Referring first to FIG. 8, a data center 800 is illustrated
comprising a plurality of electronics racks 810 and a coolant
distribution unit 820. Each electronics rack 810 includes a heat
exchange assembly 811, such as described herein. Specifically, heat
exchange assembly 811 includes an air-to-liquid heat exchanger, a
coolant inlet plenum, and a coolant outlet plenum, with multiple
heat exchange tube sections of the air-to-liquid heat exchanger
being coupled in parallel between the coolant inlet plenum and the
coolant outlet plenum. The coolant distribution unit 820 may
comprise, for example, a cooling unit such as described above in
connection with FIG. 5. Within coolant distribution unit 820, a
liquid-to-liquid heat exchanger is employed to facilitate transfer
of heat from system coolant to facility coolant passing through the
coolant distribution unit (via facility coolant supply line 801 and
facility coolant return line 802). As illustrated in FIG. 8, a main
system coolant supply line 805 supplies cooled system coolant to a
coolant supply manifold 821, and a main coolant return line 806
receives exhausted system coolant via a coolant return manifold
822. A plurality of coolant supply lines 823 and a plurality of
coolant return lines 824 facilitate coupling coolant distribution
unit 820 in fluid communication with the plurality of heat exchange
assemblies 811 associated with the electronics racks 810. In one
embodiment, quick connect couplings 812 are employed to connect the
individual coolant supply lines and coolant return lines to the
respective heat exchange assemblies 811. By way of example, these
quick connect couplings may comprise various types of commercially
available couplings, such as those available from Colder Products
Company, of St. Paul, Minn., USA, or Parker Hannifin, of Cleveland,
Ohio, USA.
[0051] FIG. 9A illustrates potential mal-distribution of system
coolant flow through the heat exchange assemblies of the
electronics racks using the closed system coolant loop
configuration of FIG. 8. As illustrated, due to the effects of
two-phase flow on pressure drop, for an overall constant pressure
drop across the heat exchange assemblies, an electronics rack at no
information technology (IT) load experiences the highest coolant
flow rate through its associated heat exchange assembly, while an
electronics rack at full IT load experiences the lowest coolant
flow rate through its heat exchange assembly. These flow
characteristics for the different heat load conditions cause
undesirable refrigerant flow mal-distribution at the data center
level.
[0052] Returning to FIG. 8, a plurality of flow restrictors 825 are
provided in association with the plurality of coolant supply lines
823 (in one embodiment), with each flow restrictor being associated
with a respective coolant supply line 823 for tailoring coolant
flow resistance through that line. In addition, pressure and
temperature sensors 826 are provided (in one embodiment) on each
coolant supply line 823 and each coolant return line 824. Flow
restrictors 825 are, by way of example, adjustable flow
restrictors, each of which comprises a dynamically adjustable
orifice opening size for tailoring coolant flow resistance through
the respective coolant line, and thus through the respective heat
exchange assembly of the associated electronics rack based on its
heat load.
[0053] By dynamically adjusting the orifice opening sizes of the
adjustable flow restrictors, a cooling apparatus is provided which
is able to tailor (or adjust) coolant flow through the respective
heat exchange assemblies, for example, based on the current IT
loads of the associated electronics racks. This is illustrated in
FIG. 9B. As shown, the flow restrictors are controlled so that, for
a constant overall pressure drop across the heat exchange
assemblies associated with the plurality of electronics racks in
the data center, an electronics rack at no IT load receives the
lowest (or no) coolant flow rate, while an electronics rack at full
IT load receives the highest coolant flow rate through its
respective air-to-liquid heat exchanger. This adjusting of the flow
resistance is significant in a system which employs refrigerant and
latent heat transfer, such as proposed herein.
[0054] FIG. 10 illustrates one embodiment of logic for controlling
the adjustable flow restrictors illustrated in FIG. 8, as well as
overall coolant flow. Initially, each adjustable flow restrictor
(i.e., each rack-level adjustable flow restrictor) is set to 50%
open position 1000, and the rack-level IT loads are obtained from
the electronics racks or measured using power measurement devices
1010. This rack-level load (or power consumed) information is
provided to a data center control unit, for example, disposed
within the one or more coolant distribution units (CDUs) of the
data center 1020. Based on the total rack power utilizations, the
control unit estimates the total refrigerant flow required and sets
the pump speed of the CDUs to force the estimated coolant flow
through the data center coolant distribution loop 1030. The
rack-level adjustable flow restrictor for the highest
power-consuming electronics rack is set to full open position 1040,
and coolant pressure and temperature information is collected from
the coolant supply and return measurement locations 1050, for
example, from pressure and temperature sensors associated with the
plurality of coolant supply lines and the plurality of coolant
return lines, such as discussed above in connection with FIG. 8.
The rack-level adjustable flow restrictors for the remaining heat
exchangers are then set to ensure that the exhausting coolant from
each heat exchange assembly is in a super-heated, thermodynamic
state within a specified range of super-heated temperatures, based
on the pressure and temperature data for that rack 1060. Logic
checks the flow through the data center distribution coolant loop
at the coolant distribution unit(s) and adjusts the pump speed to
force the estimated required flow through the loop based on the
total of the rack heat loads 1070, and returns (for example, after
a defined time delay) to collect the current rack-level power
utilizations from the electronics racks 1010, before repeating the
process.
[0055] Note that pressure and temperature sensors 826 are provided
in the plurality of coolant supply lines 823 and the plurality of
coolant return lines 824 in the data center embodiment illustrated
in FIG. 8. These pressure and temperature sensors allow for the
determination of the thermodynamic state of the refrigerant as it
enters and exits the heat exchange assemblies 811 and associated
flow restrictors 825. It is desirable for the coolant exiting the
heat exchanger subassembly 811 to be slightly super-heated, that
is, with no liquid content. Pressure (P)-enthalpy (H) diagrams for
R-134a refrigerant are available in the literature, which indicate
the regions in which such a refrigerant is sub-cooled, saturated,
or super-heated. These diagrams (or functions) utilize variables,
such as pressure and temperature (or enthalpy if the quality of the
two-phase mixture needs to be known). Thus, the thermodynamic state
of the coolant can be determined and controlled using pressure and
temperature data. The pressure and temperature values measured will
be input into a coolant-dependant algorithm (defined by the P-H
diagram/properties of the coolant) that determines if the coolant
is super-heated. This algorithm can be readily ascertained by one
skilled in the art.
[0056] If the coolant is not super-heated (i.e., the coolant is
sub-cooled or in a two-phase saturated condition), the algorithm
will modulate the adjustable flow restrictors 825 associated with
the heat exchange assemblies until the exiting coolant is
super-heated. This ensures that all coolant exiting the heat
exchanger has utilized its latent coolant effects and there is a
100% vapor in the return plenum. The modulation of the adjustable
orifices serves to increase the flow resistance, and thus,
redirects coolant flow to ensure sufficient vaporization and
cooling in all sections of the heat exchange assemblies. If the
heat load of a specific electronics rack that has a low coolant
flow suddenly increases, then the extent of super-heat will be
determined using the same pressure and temperature sensor
information. If the degree of super-heat is too much, then the
controller will open the respective flow restrictor, thereby
reducing the flow resistance through the heat exchange assembly and
thus attracting more coolant flow, thereby reducing the degree of
super-heat. Thus, one skilled in the art will note that the control
algorithm employed can determine the thermodynamic state using
pressure and temperature data, manipulate the flow restrictor to
force a super-heated condition, and also force the degree of
super-heat so as to be within a specific temperature differential
in excess of the saturated condition. For example, if for a
specific design, the saturated temperature of the refrigerant flow
is 18.degree. C., then the flow restrictor may be controlled to
force the exhaust refrigerant vapor to be at 20.degree. C.
[0057] Various actively controlled, adjustable flow restrictors are
available in the art. For example, reference the EX4 or EX6
refrigerant flow control valves offered by Emerson Electric
Company, of St. Louis, Mo., U.S.A.
[0058] FIGS. 11-18 depict a further aspect of the present
invention, wherein one or more flow restrictors are employed within
a rear door, air-to-liquid heat exchanger (such as described above)
for tailoring coolant flow resistance through one or more heat
exchange tube sections of the air-to-liquid heat exchanger to
enhance overall heat transfer across the multiple heat exchange
tube sections. In one example, the one or more flow restrictors
ensure that vaporization occurs within each tube section of the
multiple tube sections of the heat exchanger for a given operating
condition or range of conditions. By achieving this, flow
resistance gradients that might otherwise exist within the rear
door heat exchanger are eliminated, allowing for a more uniform
refrigerant flow and consistent latent heat transfer in the tube
sections. Once latent heat removal occurs roughly equally within
the heat exchange tube sections (for uniform heat loads) of the
rear door heat exchanger, greater heat removal is realized.
[0059] Various installations of flow restrictors within a rear door
heat exchanger are described below. In a system where the rear door
heat exchanger (or multiple rear door heat exchangers) receives
refrigerant pumped from a coolant distribution unit, the
refrigerant should be maintained as a sub-cooled liquid through the
supply lines in communication with the rear door heat exchanger(s).
Once the sub-cooled liquid (refrigerant) reaches its saturation
pressure for a given temperature, the liquid begins to vaporize. To
bring sub-cooled refrigerant into saturation, a flow restrictor
(such as described above in connection with FIG. 8) may be employed
within the supply and/or return lines immediately before and/or
after the rear door heat exchanger, which is designed to create a
pressure drop to bring the refrigerant to saturation before
entering the heat exchanger. This method ensures that the
refrigerant is on the verge of vaporization as delivered to the
coolant inlet plenum of the heat exchanger.
[0060] To further facilitate heat transfer across the heat exchange
tube sections of the rear door heat exchanger, at least one fixed
(or adjustable) flow restrictor is provided for each tube section,
as illustrated in FIGS. 11 & 12. In one embodiment, these fixed
or adjustable flow restrictors are disposed at the coolant channel
inlets (and/or coolant channel outlets) to the respective heat
exchange tube sections of the heat exchanger.
[0061] Referring collectively to FIGS. 11 & 12, and as noted
above, rack outlet door 330 supports air-to-liquid heat exchanger
340, and system coolant inlet and outlet plenums 601, 701. Outlet
door frame 331 supports a rigid flap 600, which attaches, for
example, by brazing or soldering, to a plate 710 secured between
the system coolant inlet plenum 601 and system coolant outlet
plenum 701. In FIG. 11, a right angle bend 610 is shown disposed at
the top of system coolant inlet plenum 601. This right angle bend
defines a horizontal inlet plenum portion which extends above the
top of door 330, which facilitates attaching a supply hose to the
hinged outlet door. The air-to-liquid heat exchanger comprises a
plurality of horizontally-oriented heat exchange tube sections 620.
These heat exchange tube sections 620 each comprise a coolant
channel having an inlet and an outlet, with each coolant channel
inlet being coupled to the system coolant inlet plenum 601 and each
coolant channel outlet being coupled to the system coolant outlet
plenum 701. A plurality of fins 630 are attached to the
horizontally-oriented heat exchange tube sections 620 for
facilitating transfer of heat from air passing across the
air-to-liquid heat exchanger to coolant flowing through the
plurality of heat exchange tube sections 620. In one embodiment,
the plurality of fins are vertically-oriented, rectangular fins
attached to horizontally-oriented heat exchange tube sections
620.
[0062] As illustrated in FIGS. 11 & 12, a plurality of flow
restrictors 1100 are provided at the heat exchange tube sections.
In this embodiment, each heat exchange tube section 620 has a first
flow restrictor at its coolant channel inlet and a second flow
restrictor at its coolant channel outlet (by way of example only).
These flow restrictors may comprise any desired combination of
fixed or adjustable flow restrictors to accomplish the desired
tailoring of coolant flow resistance through the respective heat
exchange tube sections. In this initial example, the flow
restrictors are assumed to comprise fixed orifice diameters, with
at least two fixed orifice diameters of the flow restrictors being
differently sized to define different coolant flow resistances
through at least two heat exchange tube sections of the multiple
heat exchange tube sections of the rear door heat exchanger. By
defining different coolant flow resistances, the multiple flow
restrictors tailor coolant flow to facilitate overall heat transfer
within the multiple heat exchange tube sections of the
air-to-liquid heat exchanger, for example, by facilitating
vaporization of refrigerant within each of the heat exchange tube
sections, or by equalizing flow across the heat exchange tube
sections of the rear door heat exchanger, notwithstanding the
presence of heat transfer gradients across the heat exchange tube
sections. In an alternative embodiment, the flow restrictors may
again comprise fixed orifice diameters (or opening sizes), with
each orifice opening size of the flow restrictors being identical
to ensure a common coolant flow through the multiple heat exchange
tube sections of the rear door heat exchanger. This implementation
might be advantageous where there is uniform heat flux across the
heat exchange tube sections.
[0063] FIG. 13 depicts an alternate embodiment of an apparatus for
facilitating cooling or removal of heat from an electronics rack,
in accordance with an aspect of the present invention. This cooling
apparatus is (in one embodiment) a rear door heat exchanger (such
as described above) wherein only a portion of the heat exchanger is
illustrated, including system coolant inlet plenum 601 and heat
exchange tube sections 620. In this embodiment, multiple flow
restrictors 1300 are illustrated at the coolant channel inlets of
the respective heat exchange tube sections 620. Also, these flow
restrictors 1300 are shown to comprise different orifice diameters
(or opening sizes), wherein the apparatus transitions from a
smaller orifice diameter 1310a to a larger orifice diameter 1310ff,
progressing downwards from the top of the rear door heat exchanger
towards the bottom. Using different orifice diameters within the
flow restrictors produces different flow resistances, and for
example, different magnitudes of flow resistances across the heat
exchange tube sections. These different orifice diameters may be
tailored to ensure equivalent or desired amounts of refrigerant
flow to facilitate vaporization within each heat exchange tube
section. Varying amounts of power consumption (or heat load) may be
applied to the rear door heat exchanger from the associated
electronics rack, and thus, varying the orifice diameters based on
location of the heat exchange tube sections facilitates
accommodating a range of heat load configurations.
[0064] FIG. 14 depicts a partial rear door heat exchange apparatus
for facilitating cooling of exhaust air at the air outlet side of
an electronics rack. The apparatus includes an air-to-liquid heat
exchanger comprising a coolant inlet plenum 601, a coolant outlet
plenum 701 and multiple heat exchange tube sections 620', only one
of which is illustrated in FIG. 14. Each heat exchange tube section
includes a sinusoidal coolant channel formed, in this example, from
a plurality of straight channels with U-shaped bends attached to
the ends thereof, and including a coolant channel inlet 1401 and
coolant channel outlet 1411. Each coolant channel inlet 1401 is in
fluid communication with coolant inlet plenum 601, and each coolant
channel outlet 1411 is in fluid communication with coolant outlet
plenum 701. As illustrated, multiple braze points 1420 are employed
during one manufacturing embodiment of the sinusoidal heat exchange
tube section to attach the straight channel portions to the
U-shaped portions, as well as to attach the coolant channel inlet
and coolant channel outlet to the respective plenums.
[0065] During fabrication of the rear door heat exchanger, a first
flow restrictor 1400 can be placed into the heat exchange tube
section at the coolant channel inlet, and a second flow restrictor
1410 can be placed in the heat exchange tube section at the coolant
channel outlet. As explained further below, these flow restrictors
may be brazed or crimped into position, followed by the normal
brazing 1420 of the straight channel sections used in forming the
desired heat exchange tube section configuration. By way of example
only, first flow restrictor 1400 and second flow restrictor 1410
have fixed diameter orifices selected to adjust the flow resistance
through the respective heat exchange tube section based on testing
of the heat exchange design with two-phase refrigerant heat
transfer. Note that although illustrated in FIG. 14 as including
two flow restrictors, that is, one at the coolant channel inlet and
one at the coolant channel outlet, the cooling apparatus disclosed
herein could employ one flow restrictor of fixed orifice diameter
(or adjustable orifice opening size) at either the coolant channel
inlet or coolant channel outlet of the heat exchange tube section,
or more than two flow restrictors disposed throughout the heat
exchange tube section.
[0066] FIGS. 15A & 15B depict alternate embodiments of a fixed
orifice diameter flow restrictor, which may be used (for example)
at the coolant channel inlet or coolant channel outlet of one or
more heat exchange tube sections of the rear door heat exchanger
described above.
[0067] In FIG. 15A, a flow restrictor disk 1500 is shown positioned
in a heat exchanger tube section 620', for example, by brazing or
other means of attaching the perimeter of the disk to the inner
wall of the heat exchange tube section (which is assumed to be
fabricated of metal). Flow restrictor 1500 has an orifice 1510
extending therethrough with a fixed orifice diameter D.
[0068] In the alternate embodiment of FIG. 15B, a
cylindrical-shaped flow restrictor 1550 is illustrated crimped 1555
in position within a heat exchange tube section 620', for example,
at the coolant channel inlet or coolant channel outlet thereof.
This cylindrical flow restrictor includes a cylindrical-shaped
orifice 1560 extending therethrough. In the embodiment illustrated,
the cylindrical-shaped orifice 1560 has a constant, fixed diameter
D.
[0069] By way of specific example, a cylindrical flow restrictor
such as depicted in FIG. 15B may be bored out to form a shell that
may be inserted directly into an existing heat exchange tube
section during the manufacturing process thereof. In one example,
the heat exchange tube section utilizes 3/8 inch piping, and the
cylindrical flow restrictor may be held in position via crimping to
constrict the piping around the cylindrical flow restrictor. As
with the disk-type flow restrictor, insertion of the cylindrical
flow restrictor into the rear door heat exchanger piping is a
simple manufacturing operation, is inexpensive and is a cost
effective approach to achieving the desired tailoring of flow
resistance through the respective heat exchange tube section. The
fabrication method described herein ensures that all coolant flows
through the orifice, resulting in the target flow restriction.
Cylindrical flow restrictors containing different orifice diameters
may be employed throughout the rear door heat exchanger to achieve
the desired tailoring of coolant flow resistances through the heat
exchange tube sections to enhance overall heat transfer within the
multiple heat exchange tube sections of the air-to-liquid heat
exchanger.
[0070] To satisfy changing cooling requirements across a rear door
heat exchanger or between multiple rear door heat exchangers (as
discussed above in connection with FIGS. 8-10), adjustable flow
restrictors may be employed.
[0071] FIG. 16 illustrates one embodiment of a passively
controlled, adjustable flow restrictor 1600 at the coolant channel
inlet 1401 of heat exchange tube section 620' coupling the tube
section to coolant inlet plenum 601. As illustrated, heat exchange
tube section 620' also couples to coolant outlet plenum 701 via its
coolant channel outlet 1411. A plurality of fins 630 are attached
to the horizontally-oriented heat exchange tube sections 620' for
facilitating transfer of heat from air passing across the
air-to-liquid heat exchanger to coolant flowing through the
plurality of heat exchange tube sections 620' (only one of which is
illustrated in the figure). In this embodiment, passively
controlled adjustable flow restrictor 1600 comprises a thermal
sensor 1610 for sensing temperature or exhaust airflow passing
across the heat exchange tube section 620'. The sensed temperature
1610 is fed back 1601 to the passively controlled adjustable flow
restrictor 1600.
[0072] As a specific example, temperature sensor 1610 might
comprise a thermal sensing bulb and pneumatic/spring-actuated wire
inserted into the air stream and coupling back to the passively
controlled, adjustable flow restrictor 1600 located, for example,
adjacent to the coolant channel inlet of the heat exchange tube
section 620'. As a specific example, the passively controlled,
adjustable flow restrictor might comprise a thermostatic-actuated
valve, such as provided by Metrix Valve Corp. of Glendora, Calif.,
USA. This configuration provides the advantage that each heat
exchange tube section is self-monitoring and adjusts the coolant
flow resistance therethrough as required to cool the heat load
passing across that tube section. No additional power or wiring is
required to achieve the automated control. Additionally, the
passively controlled, adjustable flow restrictor is reverse-acting
in that as temperature of airflow across the tube section drops,
the flow restrictor automatically at least partially closes,
producing a greater pressure drop across, and lower coolant flow
through, the heat exchange tube section.
[0073] FIG. 17 depicts an alternate embodiment of a cooling
apparatus in accordance with an aspect of the present invention. In
this embodiment, the apparatus again comprises an air-to-liquid
heat exchanger having a coolant inlet plenum 601, coolant outlet
plenum 701 and multiple heat exchange tube sections 620' (only one
of which is shown). A plurality of fins 630 are attached to the
horizontally-oriented heat exchange tube sections 620' for
facilitating transfer of heat from air passing across the
air-to-liquid heat exchanger to coolant flowing through the heat
exchange tube sections 620'. In the embodiment illustrated, an
actively controlled, adjustable flow restrictor 1700 is
illustrated, such as described above in connection with the cooling
apparatus of FIGS. 8-10. In this embodiment, the actively
controlled, adjustable flow restrictor 1700 is disposed at the
coolant channel inlet 1401 of heat exchange tube section 620', and
includes an adjustable orifice of varying opening size which is
actively controlled (e.g., by a rack-level control unit (not
shown)). In addition, pressure and temperature sensors 1710 and
1720 are provided at the coolant channel inlet 1401 and coolant
channel outlet 1411, respectively, of heat exchange tube section
620'. The rack-level control unit employs the sensed pressure and
temperature readings for system coolant passing through the
multiple heat exchange tube sections in determining whether to
increase or decrease orifice opening sizes within the actively
controlled, adjustable flow restrictors associated with the
multiple heat exchange tube sections, and thereby tailor coolant
flow resistance through the heat exchange tube sections. This
tailoring of coolant flow is controlled to enhance heat transfer
within the multiple heat exchange tube sections, for example, by
ensuring that latent heat transfer to refrigerant occurs within
each heat exchange tube section.
[0074] FIG. 18 illustrates one embodiment of logic for controlling
adjustable flow restrictors in a heat exchange assembly, such as
depicted in FIG. 17. Initially, each adjustable flow restrictor
(i.e., each tube section level flow restrictor within the heat
exchanger) is set to 50% open position 1800, and the power
consumption of the rack unit subsystems in opposing relation to the
individual heat exchange tube sections is collected from the
corresponding subsystems or is ascertained using power measurement
devices 1810. This sectional rack power information is then
employed by a rack-level control unit 1820 (for example). The logic
sets the adjustable flow restrictor associated with the heat
exchange tube section that is in opposing relation to the highest
power consuming subsystem in the electronics rack to a full open
position 1830. This is the heat exchange tube section which will
experience the highest exhaust airflow temperature from the
electronics rack. Next, the logic sets the adjustable flow
restrictors for the remaining heat exchange tube sections to ensure
that the exhausting system coolant from each heat exchange tube
section is in a super-heated thermodynamic state within a specific
range of super-heat temperatures, based on the sensed pressure and
temperature data for the individual heat exchange tube sections
1840, and then returns to collect updated rack subsection power
consumption information 1810.
[0075] Note that in the embodiment of FIG. 17, pressure and
temperature sensor 1720 is disposed at the coolant channel outlet
1411 of heat exchange tube section 620' to measure the temperature
and pressure of the coolant exiting the heat exchange tube section.
From this data, the thermodynamic state of the coolant can be
determined within each section. It is also desired that the coolant
exiting each of the parallel-coupled heat exchange tube sections be
slightly super-heated (i.e., above saturation temperature with no
liquid content). Measuring temperature and pressure at the outlet
of the heat exchange tube sections provides a mechanism for
determining if the coolant exiting the heat exchange tube sections
is super-heated. Pressure-enthalpy (P-H) diagrams for R-134a
refrigerant are available in the art, which indicate the regions in
which the coolant is saturated, as well as super-heated, by the
variables of pressure, temperature and enthalpy. Pressure and
temperature measurement of the coolant provides sufficient data to
determine if the coolant is saturated or super-heated. The pressure
and temperature values measured will be input to a
coolant-dependant algorithm (defined by the P-H diagram/properties
of the coolant), which determines if the coolant is super-heated.
If the coolant is not super-heated (i.e., coolant is sub-cooled or
saturated), then the adjustable orifice opening size will be
modulated until the exiting coolant is super-heated. This ensures
that all coolant exiting each heat exchange tube section has
utilized its latent cooling effect and is 100% vapor in the return
plenum. The modulation of the adjustable orifice opening size
serves to increase the flow resistance in a single heat exchange
tube section, while redirecting coolant flow to ensure sufficient
vaporization and cooling in all sections of the rear door heat
exchanger. If the heat load across a specific heat exchange tube
section which has a low coolant flow suddenly increases, then the
extent of super-heat will be determined using the same pressure and
temperature sensor information. If the degree of super-heat is too
much, then the control algorithm opens the valve, thereby reducing
the flow resistance of the heat exchange tube section, and thus
attracting more coolant flow and reducing the degree of super-heat.
Therefore, the controller determines the thermodynamic state using
the pressure and temperature data, manipulates the valve position
to force a super-heated condition, and also forces the degree of
super-heat so as to be within a specific temperature differential
in excess of the saturated condition. For example, if for a
specific design, the saturated temperature of the refrigerant flow
is 18.degree. C., then the valve may be controlled to force the
exhaust refrigerant vapor to be at 20.degree. C.
[0076] Further details and variations of liquid-based cooling
apparatuses and methods for cooling electronics systems and/or
electronics racks are disclosed in co-filed U.S. patent application
Ser. No. ______, entitled "Pressure Control Unit and Method
Facilitating Single-Phase Heat Transfer in a Cooling System"
(Attorney Docket No. POU920090027US1), and co-filed U.S. patent
application Ser. No. ______, entitled "Control of System Coolant to
Facilitate Two-Phase Heat Transfer in a Multi-Evaporator Cooling
System", (Attorney Docket No. POU920090068US1), and co-filed U.S.
patent application Ser. No. ______, entitled "System and Method for
Facilitating Parallel Cooling of Liquid-Cooled Electronics Racks",
(Attorney Docket No. POU920090085US1), and co-filed U.S. patent
application Ser. No. ______, entitled "Cooling System and Method
Minimizing Power Consumption in Cooling Liquid-Cooled Electronics
Racks", (Attorney Docket No. POU920090087US1), the entirety of each
of which is hereby incorporated herein by reference.
[0077] As will be appreciated by one skilled in the art, aspects of
the controller described above may be embodied as a system, method
or computer program product. Accordingly, aspects of the controller
may take the form of an entirely hardware embodiment, an entirely
software embodiment (including firmware, resident software,
micro-code, etc.) or an embodiment combining software and hardware
aspects that may all generally be referred to herein as a
"circuit", "module" or "system". Furthermore, aspects of the
controller may take the form of a computer program product embodied
in one or more computer readable medium(s) having computer readable
program code embodied thereon.
[0078] Any combination of one or more computer readable medium(s)
may be utilized. The computer readable medium may be a computer
readable storage medium. A computer readable storage medium may be,
for example, but not limited to, an electronic, magnetic, optical,
or semiconductor system, apparatus, or device, or any suitable
combination of the foregoing. More specific examples (a
non-exhaustive list) of the computer readable storage medium
include the following: an electrical connection having one or more
wires, a portable computer diskette, a hard disk, a random access
memory (RAM), a read-only memory (ROM), an erasable programmable
read-only memory (EPROM or Flash memory), an optical fiber, a
portable compact disc read-only memory (CD-ROM), an optical storage
device, a magnetic storage device, or any suitable combination of
the foregoing. In the context of this document, a computer readable
storage medium may be any tangible medium that can contain or store
a program for use by or in connection with an instruction execution
system, apparatus, or device.
[0079] A computer-readable signal medium may include a propagated
data signal with computer-readable program code embodied therein,
for example, in baseband or as part of a carrier wave. Such a
propagated signal may take any of a variety of forms, including,
but not limited to, electromagnetic, optical, or any suitable
combination thereof. A computer-readable signal medium may be any
computer-readable medium that is not a computer-readable storage
medium and that can communicate, propagate, or transport a program
for use by or in connection with an instruction execution system,
apparatus or device.
[0080] Program code embodied on a computer readable medium may be
transmitted using an appropriate medium, including but not limited
to wireless, wireline, optical fiber cable, RF, etc., or any
suitable combination of the foregoing.
[0081] Computer program code for carrying out operations for
aspects of the present invention may be written in any combination
of one or more programming languages, including an object oriented
programming language, such as Java, Smalltalk, C++ or the like, and
conventional procedural programming languages, such as the "C"
programming language or similar programming languages.
[0082] Aspects of the present invention are described above with
reference to flowchart illustrations and/or block diagrams of
methods, apparatus (systems) and computer program products
according to embodiments of the invention. It will be understood
that each block of the flowchart illustrations and/or block
diagrams, and combinations of blocks in the flowchart illustrations
and/or block diagrams, can be implemented by computer program
instructions. These computer program instructions may be provided
to a processor of a general purpose computer, special purpose
computer, or other programmable data processing apparatus to
produce a machine, such that the instructions, which execute via
the processor of the computer or other programmable data processing
apparatus, create means for implementing the functions/acts
specified in the flowchart and/or block diagram block or
blocks.
[0083] These computer program instructions may also be stored in a
computer readable medium that can direct a computer, other
programmable data processing apparatus, or other devices to
function in a particular manner, such that the instructions stored
in the computer readable medium produce an article of manufacture
including instructions which implement the function/act specified
in the flowchart and/or block diagram block or blocks.
[0084] The computer program instructions may also be loaded onto a
computer, other programmable data processing apparatus, or other
devices to cause a series of operational steps to be performed on
the computer, other programmable apparatus or other devices to
produce a computer implemented process such that the instructions
which execute on the computer or other programmable apparatus
provide processes for implementing the functions/acts specified in
the flowchart and/or block diagram block or blocks.
[0085] The flowchart and block diagrams in the figures illustrate
the architecture, functionality, and operation of possible
implementations of systems, methods and computer program products
according to various embodiments of the present invention. In this
regard, each block in the flowcharts or block diagrams may
represent a module, segment, or portion of code, which comprises
one or more executable instructions for implementing the specified
logical function(s). It should also be noted that, in some
alternative implementations, the functions noted in the blocks may
occur out of the order noted in the figures. For example, two
blocks shown in succession may, in fact, be executed substantially
concurrently, or the blocks may sometimes be executed in the
reverse order, depending upon the functionality involved. It will
also be noted that each block of the block diagrams and/or
flowchart illustration, and combinations of blocks in the block
diagrams and/or flowchart illustration, can be implemented by
special purpose hardware-based systems that perform the specified
functions or acts, or combinations of special purpose hardware and
computer instructions.
[0086] Although embodiments have been depicted and described in
detail herein, it will be apparent to those skilled in the relevant
art that various modifications, additions, substitutions and the
like can be made without departing from the spirit of the invention
and these are therefore considered to be within the scope of the
invention as defined in the following claims.
* * * * *