U.S. patent application number 15/041034 was filed with the patent office on 2016-06-09 for virtual machine packing method using scarcity.
The applicant listed for this patent is Microsoft Technology Licensing, LLC. Invention is credited to Rina Panigrahy, Kunal Talwar, Lincoln K. Uyeda, Ehud Wieder.
Application Number | 20160162309 15/041034 |
Document ID | / |
Family ID | 42935360 |
Filed Date | 2016-06-09 |
United States Patent
Application |
20160162309 |
Kind Code |
A1 |
Uyeda; Lincoln K. ; et
al. |
June 9, 2016 |
VIRTUAL MACHINE PACKING METHOD USING SCARCITY
Abstract
A method for packing virtual machines onto host devices may
calculate scarcity values for several different parameters. A
host's scarcity for a parameter may be determined by multiplying
the host's capacity for a parameter with the overall scarcity of
that parameter. The sum of a host's scarcity for all the parameters
determines the host's overall scarcity. Hosts having the highest
scarcity are attempted to be populated with a group of virtual
machines selected for compatibility with the host. In many cases,
several different scenarios may be evaluated and an optimal
scenario implemented. The method gives a high priority to those
virtual machines that consume scarce resources, with the scarcity
being a function of the available hardware and the virtual machines
that may be placed on them.
Inventors: |
Uyeda; Lincoln K.; (Seattle,
WA) ; Panigrahy; Rina; (Sunnyvale, CA) ;
Wieder; Ehud; (Sunnyvale, CA) ; Talwar; Kunal;
(San Francisco, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Microsoft Technology Licensing, LLC |
Redmond |
WA |
US |
|
|
Family ID: |
42935360 |
Appl. No.: |
15/041034 |
Filed: |
February 11, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13914420 |
Jun 10, 2013 |
8681450 |
|
|
15041034 |
|
|
|
|
12422123 |
Apr 10, 2009 |
8464267 |
|
|
13914420 |
|
|
|
|
Current U.S.
Class: |
718/1 |
Current CPC
Class: |
G06F 9/45533 20130101;
Y02D 10/32 20180101; G06F 9/455 20130101; G06F 9/45558 20130101;
G06F 9/5088 20130101; Y02D 10/00 20180101; Y02D 10/36 20180101;
G06F 9/5077 20130101; G06F 9/5044 20130101; G06F 2009/4557
20130101; Y02D 10/22 20180101 |
International
Class: |
G06F 9/455 20060101
G06F009/455; G06F 9/50 20060101 G06F009/50 |
Claims
1. A system, the system comprising: system memory; one or more
processors; one or more hosts, the one or more hosts offering a
plurality of resources for use by virtual machines; a virtual
machine manager, using the one or more processors, configured to:
determine a scarcity of each of the plurality of resources based on
the demand for each of the plurality of resources by the one or
more virtual machines in view of the supply for each of the
plurality of resources at the one or more hosts; determine a
scarcity score for each of the one or more hosts, the scarcity
score calculated from a local supply of each of the plurality of
resources available at the host in view of the scarcity for each of
the plurality of resources; and assign the one or more virtual
machines to the one or more hosts giving precedence to assigning
virtual machines demanding resources that are more scare to hosts
that have larger supplies of the resources that are more scarce
based on the determined scarcity for each of the plurality of
resources and the determined scarcity score for each of the one or
more hosts.
2. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to assign the one or more
virtual machines to the one or more hosts comprises a virtual
machine manager, using the one or more processors, configured to
place the one or more virtual machines onto the one or more hosts
by looping through the one or more virtual machines according to a
virtual machine order.
3. The system of claim 2, further comprising the virtual machine
manager, using the one or more processors, configured to: calculate
a virtual machine scarcity score for each of the one or more
virtual machines; and determine the virtual machine order for the
one or more virtual machines by ranking each of the one or more
virtual machines relative to one another based on the calculated
virtual machine scarcity scores for each of the one or more virtual
machines.
4. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to determine a scarcity of
each of the plurality of resources comprises a virtual machine
manager, using the one or more processors, configured to determine
a scarcity for a binary resource.
5. The system of claim 1, a virtual machine manager, using the one
or more processors, configured to assign the one or more virtual
machines to the one or more hosts comprises a virtual machine
manager, using the one or more processors, configured to assign at
least one virtual machine with a binary resource to a host.
6. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to determine a scarcity of
each of the plurality of resources comprises a virtual machine
manager, using the one or more processors, configured to determine
a scarcity for at non-consumable resource.
7. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to determine a scarcity of
each of the plurality of resources comprises a virtual machine
manager, using the one or more processors, configured to determine
a scarcity for at consumable resource.
8. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to assign the one or more
virtual machines to the one or more hosts comprises a virtual
machine manager, using the one or more processors, configured to
consider and reject an assignment of a virtual machine to a
host.
9. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to determine a scarcity of
each of the plurality of resources comprises a virtual machine
manager, using the one or more processors, configured to determine
a scarcity for a resource based on the scarcity of a plurality of
sub resources associated with the resource.
10. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to assign the one or more
virtual machines to the one or more hosts comprises a virtual
machine manager, using the one or more processors, configured to
assign the one or more virtual machines to the one or more hosts
based on a power consumption score for each of the one or more
virtual machines.
11. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to assign the one or more
virtual machines to the one or more hosts comprises a virtual
machine manager, using the one or more processors, configured to:
assign a virtual machine to a host, the host selected from among
the one or more host; and assign another virtual machine providing
redundancy for the virtual machine to another host, the other host
selected from among the one or more hosts.
12. The system of claim 1, wherein a virtual machine manager, using
the one or more processors, configured to assign the one or more
virtual machines to the one or more hosts comprises a virtual
machine manager, using the one or more processors, configured to
assign the one or more virtual machines to the one or more hosts
taking into account the placement of a group of virtual machines
already placed on the one or more hosts.
13. The system of claim 1, further comprising: one or more
additional virtual machines assigned to the one or more hosts; and
the virtual machine manager, using the one or more processors,
configured to: determine the demand for each of the plurality of
resources based on the individual demand for each of the plurality
of resources by each of the one or more virtual machines; and
determine the supply for each of a plurality of resources, the
supply calculated based on the amount of the resource available at
each of the one or more hosts in view of the amount of the resource
consumed by the one or more additional virtual machines.
14. A method for use at a computer system, the method for
determining placement of one or more virtual machines onto one or
more hosts, the one or more hosts offering a plurality of resources
for use by virtual machines, the method comprising: determining a
scarcity of each of the plurality of resources based on the demand
for each of the plurality of resources by the one or more virtual
machines in view of the supply for each of the plurality of
resources at the one or more hosts; determining a scarcity score
for each of the one or more hosts, the scarcity score calculated
from a local supply of each of the plurality of resources available
at the host in view of the scarcity for each of the plurality of
resources; and assigning the one or more virtual machines to the
one or more hosts giving precedence to assigning virtual machines
demanding resources that are more scare to hosts that have larger
supplies of the resources that are more scarce based on the
determined scarcity for each of the plurality of resources and the
determined scarcity score for each of the one or more hosts.
15. The method of claim 14, further comprising: calculating a
virtual machine scarcity score for each of the one or more virtual
machines; and determining a virtual machine order for the one or
more virtual machines by ranking each of the one or more virtual
machines relative to one another based on the calculated virtual
machine scarcity scores for each of the one or more virtual
machines; and wherein assigning the one or more virtual machines to
the one or more hosts comprises placing the one or more virtual
machines onto the one or more hosts by looping through the one or
more virtual machines according to the virtual machine order.
16. The method of claim 14, wherein assigning the one or more
virtual machines to the one or more hosts comprises considering and
rejecting an assignment of a virtual machine to a host.
17. The method of claim 14, wherein determining a scarcity of each
of the plurality of resources comprises determining a scarcity for
a resource based on the scarcity of a plurality of sub resources
associated with the resource.
18. The method of claim 14, wherein assigning the one or more
virtual machines to the one or more hosts comprises assigning the
one or more virtual machines to the one or more hosts taking into
account the placement of a group of virtual machines already placed
on the one or more hosts.
19. The method of claim 14, wherein assigning the one or more
virtual machines to the one or more hosts comprises assigning the
one or more virtual machines to the one or more hosts based on a
power consumption score for each of the one or more virtual
machines.
20. A computer program product for use at a computer system, the
computer program product for implementing a method for placing one
or more virtual machines onto one or more hosts, the one or more
hosts offering a plurality of resources for use by virtual
machines, the computer program product comprising one or more
computer storage memories having stored thereon computer-executable
instructions that, when executed at a processor, cause the computer
system to perform the method, including the following: determine a
scarcity of each of the plurality of resources based on the demand
for each of the plurality of resources by the one or more virtual
machines in view of the supply for each of the plurality of
resources at the one or more hosts; determine a scarcity score for
each of the one or more hosts, the scarcity score calculated from a
local supply of each of the plurality of resources available at the
host in view of the scarcity for each of the plurality of
resources; and assign the one or more virtual machines to the one
or more hosts giving precedence to assigning virtual machines
demanding resources that are more scare to hosts that have larger
supplies of the resources that are more scarce based on the
determined scarcity for each of the plurality of resources and the
determined scarcity score for each of the one or more hosts.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of and priority to U.S.
patent application Ser. No. 13/914,240, entitled "Virtual Machine
Packing Method Using Scarcity", filed Jun. 10, 2013 by Lincoln K.
Uyeda et al., the entire contents of which are expressly
incorporated by reference. That application claims the benefit of
and priority to U.S. patent application Ser. No. 12/422,123, now
U.S. Pat. No. 8,464,267, entitled "Virtual Machine Packing Method
Using Scarcity", filed Apr. 10, 2009 by Lincoln K. Uyeda et al.,
the entire contents of which are expressly incorporated by
reference
BACKGROUND
[0002] Virtual machines are computer software implementations of a
computer device, where the virtual machine may execute programs
like a physical computer. Virtual machines are widely used in data
centers where hundreds or thousands of host machines may be
operating. In order to manage the computing resources, many data
centers run server computers as virtual machines because virtual
machines can be moved from one host device to another.
[0003] Many data centers operate on a cyclical basis, where demand
may be higher during certain periods of the day or certain days of
the week. During low demand times, virtual machines may be
consolidated to certain host machines so that other host machines
may be turned off or operated in a reduced power mode.
[0004] In many cases, a datacenter may consume large amounts of
electricity. As hardware is turned off during periods of reduced
computer load, the datacenter may reduce its electricity costs
substantially.
[0005] The process of placing virtual machines onto host devices
may be a complex packing problem.
SUMMARY
[0006] A method for packing virtual machines onto host devices may
calculate scarcity values for several different parameters. A
host's scarcity for a parameter may be determined by multiplying
the host's capacity for a parameter with the overall scarcity of
that parameter. The sum of a host's scarcity for all the parameters
determines the host's overall `size`. Hosts having the largest
`size` are attempted to be populated with a group of virtual
machines selected for compatibility with the host. In many cases,
several different scenarios may be evaluated and an optimal
scenario implemented. The method gives a high priority to those
virtual machines that consume scarce resources, with the scarcity
being a function of the available hardware and the virtual machines
that may be placed on them.
[0007] This Summary is provided to introduce a selection of
concepts in a simplified form that are further described below in
the Detailed Description. This Summary is not intended to identify
key features or essential features of the claimed subject matter,
nor is it intended to be used to limit the scope of the claimed
subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] In the drawings,
[0009] FIG. 1 is a diagram illustration of an embodiment showing a
system for managing virtual machines.
[0010] FIG. 2 is a flowchart illustration of an embodiment showing
a method for managing virtual machines.
[0011] FIG. 3 is a flowchart illustration of an embodiment showing
a method for gathering supply and demand data.
[0012] FIG. 4 is a flowchart illustration of an embodiment showing
a method for performing placement analyses.
DETAILED DESCRIPTION
[0013] Several virtual machines may be placed onto multiple host
devices using a heuristic based on the scarcity of various
monitored dimensions. The scarcity mechanism may identify hosts
that have a resource that is in high demand and may pack the
virtual machines according to the resource.
[0014] The heuristic may be applied in several manners. In one
case, the heuristic may be used to determine an optimized placement
of virtual machines across an entire datacenter. In another case, a
group of virtual machines may be placed onto a set of hosts that
are already running one or more virtual machines.
[0015] The heuristic may be used to analyze multiple combinations
of virtual machine placement and to select an optimized version
from the combinations. In such embodiments, many millions of
options may be simulated and one may be selected based on the
simulations.
[0016] Throughout this specification and claims, the term `host` is
used to describe a physical hardware platform on which a virtual
machine may be executed. In some embodiments, a lightweight
operating system may run on the host and may support one, two, or
many more virtual machines. In some embodiments, many tens or even
hundreds of virtual machines may be executed on one physical host
device.
[0017] The term `host` may be any type of hardware platform on
which a virtual machine may be executed. In some cases, the
hardware platform may be a server computer with specialized
processors, memory, disk storage, and network connections that are
specifically designed to run virtual machines. In other cases, the
host may be a conventional desktop or server computer, or may be a
portable device such as a mobile phone, laptop computer, or other
device. For the purposes of this specification, many examples of
using the heuristic may involve datacenter applications, however,
those skilled in the art will appreciate that many other
implementations may be possible.
[0018] As an example of an application other than the datacenter, a
loosely connected group of personal computers may be used as hosts
to perform background processing by hosting one or more virtual
machines. The background processing may execute virtual machines
during the times that the personal computers are idle. Such an
embodiment may be used for performing massive computational
problems that may not be time sensitive, for example. In such an
embodiment, the heuristic described in this specification may be
applied to place multiple virtual machines across hosts that may
have wide variations in capacities and capabilities.
[0019] Throughout this specification, like reference numbers
signify the same elements throughout the description of the
figures.
[0020] When elements are referred to as being "connected" or
"coupled," the elements can be directly connected or coupled
together or one or more intervening elements may also be present.
In contrast, when elements are referred to as being "directly
connected" or "directly coupled," there are no intervening elements
present.
[0021] The subject matter may be embodied as devices, systems,
methods, and/or computer program products. Accordingly, some or all
of the subject matter may be embodied in hardware and/or in
software (including firmware, resident software, micro-code, state
machines, gate arrays, etc.) Furthermore, the subject matter may
take the form of a computer program product on a computer-usable or
computer-readable storage medium having computer-usable or
computer-readable program code embodied in the medium for use by or
in connection with an instruction execution system. In the context
of this document, a computer-usable or computer-readable medium may
be any medium that can contain, store, communicate, propagate, or
transport the program for use by or in connection with the
instruction execution system, apparatus, or device.
[0022] The computer-usable or computer-readable medium may be, for
example but not limited to, an electronic, magnetic, optical,
electromagnetic, infrared, or semiconductor system, apparatus,
device, or propagation medium. By way of example, and not
limitation, computer readable media may comprise computer storage
media and communication media.
[0023] Computer storage media includes volatile and nonvolatile,
removable and non-removable media implemented in any method or
technology for storage of information such as computer readable
instructions, data structures, program modules or other data.
Computer storage media includes, but is not limited to, RAM, ROM,
EEPROM, flash memory or other memory technology, CD-ROM, digital
versatile disks (DVD) or other optical storage, magnetic cassettes,
magnetic tape, magnetic disk storage or other magnetic storage
devices, or any other medium which can be used to store the desired
information and which can accessed by an instruction execution
system. Note that the computer-usable or computer-readable medium
could be paper or another suitable medium upon which the program is
printed, as the program can be electronically captured, via, for
instance, optical scanning of the paper or other medium, then
compiled, interpreted, of otherwise processed in a suitable manner,
if necessary, and then stored in a computer memory.
[0024] Communication media typically embodies computer readable
instructions, data structures, program modules or other data in a
modulated data signal such as a carrier wave or other transport
mechanism and includes any information delivery media. The term
"modulated data signal" means a signal that has one or more of its
characteristics set or changed in such a manner as to encode
information in the signal. By way of example, and not limitation,
communication media includes wired media such as a wired network or
direct-wired connection, and wireless media such as acoustic, RF,
infrared and other wireless media. Combinations of the any of the
above should also be included within the scope of computer readable
media.
[0025] When the subject matter is embodied in the general context
of computer-executable instructions, the embodiment may comprise
program modules, executed by one or more systems, computers, or
other devices. Generally, program modules include routines,
programs, objects, components, resources, data structures, etc.
that perform particular tasks or implement particular abstract data
types. Typically, the functionality of the program modules may be
combined or distributed as desired in various embodiments.
[0026] FIG. 1 is a diagram of an embodiment 100 showing a system
for managing virtual machines. Embodiment 100 is a simplified
example of a system that may manage virtual machines on several
host devices.
[0027] The diagram of FIG. 1 illustrates functional components of a
system. In some cases, the component may be a hardware component, a
software component, or a combination of hardware and software. Some
of the components may be application level software, while other
components may be operating system level components. In some cases,
the connection of one component to another may be a close
connection where two or more components are operating on a single
hardware platform. In other cases, the connections may be made over
network connections spanning long distances. Each embodiment may
use different hardware, software, and interconnection architectures
to achieve the functions described.
[0028] Embodiment 100 is an example of a datacenter that may
contain many host devices, each of which may execute multiple
virtual machines. Virtual machines may be a software version of a
device. In a large datacenter environment, server computers may be
implemented as virtual machines. As demand shrinks, virtual
machines may be consolidated onto some host devices so that other
hosts may be powered down or operated in a reduced energy state. As
demand rises, hosts may be brought online and virtual machines may
be spread among the available hosts.
[0029] In many cases, hosts may have different capabilities and
different capacities. The resources available on a host may be
categorized into consumable and non-consumable resources.
[0030] A consumable resource is one that is consumed by a virtual
machine and cannot be shared with other virtual machines or a host
operating system. An example of a consumable resource may be random
access memory (RAM). Each virtual machine may have a dedicated
amount of RAM assigned to the virtual machine, and that RAM may not
be used by other virtual machines. Another example may be disk
space or other storage.
[0031] In some cases, a consumable resource may be considered as a
summation of various component resources. For example, a hard disk
resource may be considered the summation of disk read speed, disk
write speed, and capacity.
[0032] A non-consumable resource may be a resource that can be
shared by virtual machines or by a host operating system. An
example may be a number of processors available on a host device.
Some virtual machines may operate using two, four, or more
processors or cores. Even though a virtual machine uses four
processors, other virtual machines or the host operating system may
also use the same processors.
[0033] A binary resource may be a subset of a non-consumable
resource. A binary resource may be a resource that is either
present or not on a host. For example, a host may or may not have a
graphics processor or other feature that may be used by a
particular virtual machine.
[0034] The various resources may be discussed in this specification
as dimensions for consideration when placing virtual machines onto
hosts. The term `dimensions` is intended to include resources or
any other factor that may be considered when matching virtual
machines to a host. In some cases, a dimension may not correspond
to a resource available on a host or used by a virtual machine.
[0035] The virtual machines may be placed on hosts using a concept
of scarcity that may be calculated for the various dimensions.
Hosts having the scarcest resources may be considered first when
attempting to place virtual machines so that the scarce resources
are properly allocated. The scarcest dimensions may drive the
virtual machine placement process by sorting and selecting hosts
and virtual machines based on the scarcity of the dimensions.
[0036] The virtual machine placement may be analyzed by considering
the placement of multiple virtual machines in many different
combinations. By selecting combinations that conform to a
heuristic, a reduced set of virtual machine combinations may be
considered as opposed to considering every combination of virtual
machines and hosts.
[0037] In many datacenters, several thousands or even tens of
thousands of host devices may be present, each of which may execute
several virtual machines. In such an embodiment, considering every
combination of virtual machine and host device may take an
extraordinary amount of processing time. The heuristic may be used
to reduce the number of combinations to consider to a more
reasonable amount, allowing large datacenter applications to be
considered.
[0038] The heuristic involves gathering supply and demand data.
Supplied resources or dimensions may be gathered from the hosts,
and resources or dimensions may be demanded by the virtual
machines. By considering supply minus demand divided by supply, a
dimensionless value may be obtained as a scarcity factor for each
individual dimension. Those dimensions having a high scarcity
factor indicate that the dimensions may be the gating items in
successfully placing virtual machines.
[0039] After determining a scarcity factor for each dimension, a
scarcity score for a host may be computed by multiplying the
scarcity factor by the capacity for the dimension divided by the
maximum value and multiplied by the scarcity factor. The sum of all
dimensions may be used as a scarcity score for the host. The
scarcity score may be considered a `size` for the host. Hosts
having the greatest `size` may be considered first when placing
virtual machines. The hosts with the greatest `size` may have the
largest amount of the scarcest resources.
[0040] The heuristic may involve identifying a set of virtual
machines to consider for placement onto a specific host. In some
cases, a binary or non-consumed resource may eliminate one or more
virtual machines from consideration for particular host. For
example, a virtual machine that runs on four processors may be
eliminated from consideration if a host only has two
processors.
[0041] The set of virtual machines that are considered may also
include removing virtual machines that may be incompatible with
each other. For example, many datacenters may have virtual machines
that provide redundant services so that if a host fails and a
virtual machine is unavailable, another host with a redundant
virtual machine may still be operational. In such an example both
of the redundant virtual machines may not be located on the same
host.
[0042] In order to remove incompatible virtual machines,
relationships between various virtual machines may be identified
and an independent set of virtual machines may be selected.
[0043] The heuristic may be used to evaluate a set of hosts in
different scenarios. In one scenario, an optimized placement of
virtual machines may be considered across an entire datacenter of
host devices that are not hosting any virtual machines. Such a
scenario may be useful when considering a fully optimized
datacenter, regardless of any starting condition.
[0044] In a second scenario, the placement of a group of virtual
machines may be evaluated with a given set of starting conditions.
Such a scenario may involve considering a set of virtual machines
already placed onto hosts and may find a placement for a second
group of virtual machines onto a set of hosts executing a first
group of virtual machines. In such a scenario, the virtual machines
that are placed onto a host may be selected to be compatible with
any existing virtual machines on the host.
[0045] When a set of starting conditions are considered, the
scarcity of a resource may be calculating using the available
amount of a resource. In other words, the available amount of a
resource may be the total amount of the resource provided by a
host, minus the consumed amount of the resource by any virtual
machine already assigned to the host. This mechanism may alter the
scarcity of the dimensions drastically than when considering the
first scenario.
[0046] Embodiment 100 is an example of a system for managing
virtual machines in a datacenter environment. The virtual machine
manager 102 may be a device that organizes a datacenter 104, where
multiple virtual machines 106 may be executed by various host
devices.
[0047] In a datacenter 104, a rack 108 may contain multiple host
devices. A typical datacenter may have racks with blade enclosures
110 that may include multiple blade servers 112. Some racks may
include standalone servers 114, storage devices 116, and power
supplies 118. In many cases, a rack or group of racks may have
cooling systems, power conditioning units, uninterruptible power
supplies, and other components.
[0048] Datacenters may consume large amounts of energy, but may
have data loads that may change during the course of the day, days
of the week, or other seasonal variation. During periods of low
demand, virtual machines may be moved off of certain hosts so that
those hosts may be shut down or operated in a reduced power mode.
In such a case, a group of hosts may be identified and those
virtual machines currently on the hosts may be placed onto the
remaining hosts, while the remaining hosts may continue to operate
other virtual machines.
[0049] When a group of hosts may be identified for reduced power
mode or for power off mode, a datacenter operator may wish to power
down an entire rack of devices or groups of racks so that some of
the overhead services, such as cooling and power supplies may be
also be shut down or operated in a reduced power mode. The virtual
machines operating on those racks may be moved to other hosts.
[0050] In many datacenters, different host may have different
capabilities. Older devices may have slower processing speeds or
may have less memory than newer devices. In some cases, newer
devices may have capabilities that older devices may not have.
[0051] Some virtual machines may have different resource demands
than other virtual machines. For example, some virtual machines may
consume large amounts of processor capacity while other virtual
machines may consume miniscule amounts of the same resource. In
another example, some virtual machines may perform large amounts of
disk operations while other virtual machines may not.
[0052] The virtual machine manager 102 may be an application that
operates on a hardware platform. The virtual machine manager 102
may be comprised of software components 120 that operate on
hardware components 122. The hardware components 122 may include a
processor, random access memory 126, disk or other storage 128, and
a network connection 130. In some embodiments, the hardware
components 122 may be virtualized.
[0053] The virtual machine manager 102 may have software components
120 that include a data gatherer 132, a placement manager 134, and
a placement service 136. The data gatherer 132 may collect status
of various dimensions from host devices and virtual machines. The
placement manager 134 may determine a placement for a set of
virtual machines on the hosts, and the placement service 136 may
move the virtual machines to the designated hosts.
[0054] Embodiment 200 presented later in this specification may
illustrate one method that incorporates the operations of the
software components 120. Embodiment 300 may illustrate one method
that may be performed by a data gatherer 132 and embodiment 400 may
illustrate one method that may be performed by the placement
manager 134.
[0055] The data gatherer 132 may collect data on dimensions or
resources supplied by hosts and demanded by virtual machines. The
supply and demand information may be used to determine scarcity of
the various dimensions, which may in turn be used by the placement
manager 134 to determine various placements for virtual machines
onto hosts.
[0056] The placement manager 134 may evaluate many different
placement configurations. Each placement configuration may be a set
of specific virtual machines that are executed on a specific host
device. In some embodiments, many thousands, millions, or more
placement configurations may be evaluated. For each placement
configuration, a score may be computed for an optimizing parameter.
For example, each configuration may have a power consumption score
from which an optimized configuration may be selected.
[0057] Embodiment 100 is an example of a datacenter. Other
applications may include the distribution of virtual machines onto
personal computers for performing computational problems as
background processes. The personal computers may be located
throughout the world and used to perform large scale computational
problems in one embodiment.
[0058] FIG. 2 is a flowchart illustration of an embodiment 200
showing a method for managing virtual machines. Embodiment 200 is a
simplified example of a method that may be performed by a virtual
machine manager 102 as illustrated in embodiment 100.
[0059] Other embodiments may use different sequencing, additional
or fewer steps, and different nomenclature or terminology to
accomplish similar functions. In some embodiments, various
operations or set of operations may be performed in parallel with
other operations, either in a synchronous or asynchronous manner.
The steps selected here were chosen to illustrate some principles
of operations in a simplified form.
[0060] Embodiment 200 is a general process for determining a
placement of virtual machines onto host devices. In block 202,
supply and demand data may be gathered, and in block 204, many
placement variations may be performed. Each variation in block 206
may be analyzed in block 208 to determine a score. Based on the
score, a variation may be selected in block 210. In block 212, the
virtual machines may be placed onto host devices according to the
selected placement.
[0061] The process of embodiment 200 may be performed at any time
several virtual machines may be placed onto host devices. For
example, if a rack of host devices is to be shut down for power
savings, maintenance, or some other reason, the virtual machines
operating on the rack of host devices may be moved to other host
devices.
[0062] The analysis of different placement options in block 204 may
include using scarcity of the various dimensions to identify those
host devices having a large `size` and by packing virtual machines
onto hosts based on their scarcity. This heuristic identifies those
dimensions that are more likely to be difficult to satisfy when
placing virtual machine.
[0063] FIG. 3 is a flowchart illustration of an embodiment 300
showing a method for gathering supply and demand data. Embodiment
300 is a simplified example of a method that may be performed by a
data gatherer 132 as illustrated in embodiment 100 and by block 202
in embodiment 200.
[0064] Other embodiments may use different sequencing, additional
or fewer steps, and different nomenclature or terminology to
accomplish similar functions. In some embodiments, various
operations or set of operations may be performed in parallel with
other operations, either in a synchronous or asynchronous manner.
The steps selected here were chosen to illustrate some principles
of operations in a simplified form.
[0065] Embodiment 300 is one method by which supply and demand data
may be gathered. Supply data may relate to resources or other
dimensions that are found on host devices, while demand data may
relate to matching resources or dimensions that are used by virtual
machines. After gathering data, scarcity is determined and a
scarcity score may be determined for each host device.
[0066] Embodiment 300 may be performed on a set of host devices
with different initial states. In one scenario, the host devices
may be analyzed as if no virtual machines are currently running. In
such a scenario, an optimized placement of virtual machines over an
entire datacenter may be identified. In another scenario, a group
of virtual machines may be placed on a datacenter or group of hosts
that is already hosting a set of virtual machines.
[0067] In block 302, the initial state for analysis is
determined.
[0068] In blocks 304 through 308, the demand is determined for each
dimension of each virtual machine. In block 304, each virtual
machine is analyzed. For each dimension in block 306, a demand for
the dimension is determined in block 308.
[0069] The dimensions analyzed may vary between embodiments. In
many cases, the dimensions may include consumed resources,
non-consumed resources, and binary resources as well as other
dimensions.
[0070] For each host device in block 310, each dimension is
analyzed in block 312. The supply for the dimension is determined
in block 314.
[0071] For each dimension in block 316, a scarcity is determined in
block 318.
[0072] Scarcity may be determined through the following
formula:
Scarcity dim = VM demand dim ( host supply dim - VM demand dim )
##EQU00001##
[0073] For each dimension, scarcity may be computed by dividing the
sum of all virtual machine demand by the difference between the sum
of all host supply and the sum of all virtual machine demand.
Scarcity is a dimensionless number for each dimension. A high
scarcity value indicates that the dimension is scarce, while a low
scarcity value indicates that the dimension is plentiful.
[0074] In block 320, each host is analyzed. In block 322, a
scarcity score for the host is determined. The scarcity score may
indicate the `size` of a host in terms of the scarcity of its
resources. Those hosts having the largest supply of scarce
resources are considered `large`.
[0075] The scarcity for each host may be determined using the
following formula:
ScarcityScore host = dim ( Capacity host * Scarcity dim MaxCapcity
dim ) ##EQU00002##
[0076] The scarcity score for each host may be the sum of a factor
for each dimension. The factor may be computed by multiplying the
capacity of the dimension for the host by the scarcity of the
dimension, then dividing by the maximum capacity of any host for
the dimension.
[0077] FIG. 4 is a flowchart illustration of an embodiment 400
showing a method for performing a placement analysis. Embodiment
400 is a simplified example of a method that may be performed by a
placement manager 134 as illustrated in embodiment 100.
[0078] Other embodiments may use different sequencing, additional
or fewer steps, and different nomenclature or terminology to
accomplish similar functions. In some embodiments, various
operations or set of operations may be performed in parallel with
other operations, either in a synchronous or asynchronous manner.
The steps selected here were chosen to illustrate some principles
of operations in a simplified form.
[0079] Embodiment 400 is an example of a heuristic that may be used
to identify potential virtual machine placements. In many
embodiments, the heuristic of embodiment 400 may be performed to
generate many different virtual machine placements, each of which
may be scored and one of which may be selected for implementation.
In some such embodiments, different parameters within the method of
embodiment 400 may be altered to select different hosts or
different virtual machines to generate different virtual machine
placements.
[0080] In some embodiments, the method of embodiment 400 may be
performed once to place multiple virtual machines. In such an
embodiment, the computation time used for a lengthy optimization
may be skipped and a reasonable placement may be made.
[0081] The placement analysis may be started in block 402.
[0082] The hosts may be sorted using the scarcity score in block
404. The scarcity score may be used to generate an ordered list of
hosts, from which the first host may be selected in block 406.
[0083] After the host is selected in block 406, the virtual
machines may be attempted to be placed on the host. After analyzing
all of the virtual machines, the next host is selected and the
process is repeated.
[0084] In block 408, the binary resources available on the host are
used to identify those virtual machines that are compatible with
the host. The virtual machines that are not compatible with the
host may be excluded from the analysis.
[0085] Binary resources may be any resource or dimension that is
either present on the host or not. In many cases, the virtual
machine may not consume the binary resource. An example may be a
graphics processor that is used by a virtual machine.
[0086] In block 410, an independent set of virtual machines may be
identified for conflict mitigation. In some embodiments, some
virtual machines may or may not be located on the same hosts for a
variety of reasons. For example, some virtual machines operate as
redundant copies of each other and are operated on separate virtual
machines.
[0087] In selecting the independent set of virtual machines in
block 410, some embodiments may perform the process of embodiment
400 multiple times for each independent set. Each time the process
is performed, a separate virtual machine placement may be generated
and scored. An optimized virtual machine placement may be selected
based on the score.
[0088] For the set of virtual machines being considered, a scarcity
score for the virtual machines may be determined. One formula for
determining the scarcity of the virtual machines is as follows:
VMScore = dim scarcity dim * ( Capacity host * Scarcity dim
MaxCapacity dim ) p ##EQU00003##
[0089] The virtual machine score uses a similar formula for
determining the host score, but raises the capacity times scarcity
divided by max capacity to a power. Different embodiments may use
different values of P. However, suitable results may be obtained
with P=2.
[0090] The power term may enable virtual machines that consume very
scarce resources to be weighted higher than other virtual machines.
Other embodiments may use different formulas for generating a score
for virtual machines.
[0091] In block 414, the virtual machines may be sorted by scarcity
score, and in block 416, each virtual machine may be analyzed
according to the scarcity score order.
[0092] The current virtual machine may be attempted to be placed on
the host in block 418. In some cases, a virtual machine may consume
more resources than are available or may be incompatible for some
other reason, making the attempted placement a failure in block
420. In such a case, the next virtual machine may be analyzed in
block 416.
[0093] If a virtual machine can be successfully placed in block
420, the loop of block 416 may be exited in block 422 and the
process may return to block 412. When the process returns to block
412, the virtual machines may be re-scored using the VMScore
formula above, but using the current capacity of the virtual
machine. As virtual machines are placed on a host, the host's
resources may be used up. As some resources are used up, some may
become more scarce than others. Performing another analysis of the
virtual machines in block 412 allows for the consumption of
resources and may be better able to use the available resources
when re-computing the virtual machine scarcity.
[0094] After each virtual machine is analyzed in block 416, if some
virtual machines still are not placed and additional hosts have not
been analyzed, the process may return to block 406 from block 424.
If the process is done in block 424, the process may end in block
426.
[0095] Embodiment 400 is an example of a heuristic that may be used
for generating one or many virtual machine placements. In some
cases, the process of embodiment 400 may be performed for different
independent sets of virtual machines in block 410. In other cases,
the process of embodiment 400 may be performed by analyzing every
third virtual machine in the sorted list in block 416. Another
example may be to perturb the ordered list of hosts to create
different variations. Other embodiments may alter different
parameters within the process of embodiment 400 to generate
different virtual machine placements.
[0096] When a virtual machine placement is created, the virtual
machine placement may be scored. In one embodiment, the scoring may
be according to power consumption, although other embodiments may
have different scoring and analysis mechanisms.
[0097] The foregoing description of the subject matter has been
presented for purposes of illustration and description. It is not
intended to be exhaustive or to limit the subject matter to the
precise form disclosed, and other modifications and variations may
be possible in light of the above teachings. The embodiment was
chosen and described in order to best explain the principles of the
invention and its practical application to thereby enable others
skilled in the art to best utilize the invention in various
embodiments and various modifications as are suited to the
particular use contemplated. It is intended that the appended
claims be construed to include other alternative embodiments except
insofar as limited by the prior art.
* * * * *