U.S. patent application number 13/899395 was filed with the patent office on 2014-09-18 for method and apparatus for evaluation of skill level progression and matching of participants in a multi-media interactive environment.
This patent application is currently assigned to Ignite Game Technologies, Inc.. The applicant listed for this patent is Ignite Game Technologies, Inc.. Invention is credited to Jonathan HASWELL.
Application Number | 20140274304 13/899395 |
Document ID | / |
Family ID | 51529471 |
Filed Date | 2014-09-18 |
United States Patent
Application |
20140274304 |
Kind Code |
A1 |
HASWELL; Jonathan |
September 18, 2014 |
METHOD AND APPARATUS FOR EVALUATION OF SKILL LEVEL PROGRESSION AND
MATCHING OF PARTICIPANTS IN A MULTI-MEDIA INTERACTIVE
ENVIRONMENT
Abstract
In an online interactive simulation or game environment, in
which an object's motion is controlled by a player, a player's
skill level is quickly quantified as they control the object to
properly place the player in games with those of like skill without
having to compete with others. An optimal motion is established for
the object's travel during a portion of a game. At each increment
of travel, an optimum velocity or time delta is established. The
motion of the object that is being controlled by a player who is
being rated is then tracked relative to the established optimal
motion. Deviations therebetween are calculated on an incremental
basis, and the aggregate score is used to determine the player's
skill level for a set of equivalent conditions.
Inventors: |
HASWELL; Jonathan; (San
Francisco, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Ignite Game Technologies, Inc. |
San Francisco |
CA |
US |
|
|
Assignee: |
Ignite Game Technologies,
Inc.
San Francisco
CA
|
Family ID: |
51529471 |
Appl. No.: |
13/899395 |
Filed: |
May 21, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61779332 |
Mar 13, 2013 |
|
|
|
Current U.S.
Class: |
463/23 |
Current CPC
Class: |
A63F 13/10 20130101;
A63F 13/44 20140902; A63F 13/573 20140902; A63F 13/798
20140902 |
Class at
Publication: |
463/23 |
International
Class: |
A63F 13/40 20060101
A63F013/40 |
Claims
1. A computer implemented method for quantifying and using a
progression of a player's skill with respect to an interactive
computer game simulation, comprising: continually quantifying a
player's skill during one or more events of a game type; based on a
plurality of quantifications of the player's skill, predicting a
skill level of the player for a next event; and matching a player
with other players of similar skill for the next event, based at
least in part on the predicted skill level.
2. The method of claim 1, wherein quantifying the progression of
the player's skill comprises comparing said player's skill level
with a target skill level at one or more points in time during an
event.
3. The method of claim 2, wherein said one or more points in time
coincide with increments of any of time and distance.
4. The method of claim 1, further comprising: creating skill
cohorts comprising groupings of players with like skills; and
during a registration period, admitting a player to an appropriate
skill cohort based on the player's predicted skill level.
5. The method of claim 4, further comprising: after close of the
registration period, identifying players who are likely to cross
between skill cohorts during a next event; and adjusting the skill
cohorts as appropriate based in part on the identifying of players
likely to cross between skill cohorts.
6. The method of claim 5, further comprising: after the close of
the registration period and based on available player liquidity,
adjusting the skill cohorts as appropriate to create matches
optimized for minimal skill variance within a cohort.
7. A method for quantifying and using progressions of skill for a
plurality of players with respect to an interactive computer game
simulation, comprising: continually quantifying a player's skill
during one or more events of a game type; and determining for a
plurality of players over a first time period comprising a first
plurality of events, a first set of players who have progressed
from a first skill level to a second skill level at a first
rate.
8. The method of claim 7, further comprising: determining for the
first set of players over a second time period comprising a second
plurality of events, equipment comprising any of a car type and a
track type that has enabled a second set of players within the
first set of players to advance faster to a third skill level
during the second time period; and encouraging the second set of
players to continue using their current equipment.
9. The method of claim 7, further comprising: determining over a
second time period comprising a second plurality of events and for
the first set of players, equipment comprising any of a car type
and a track type that has enabled a third set of players within the
first set of players to advance more slowly to the third skill
level during the second time period; and encouraging the third set
of players to upgrade their equipment.
10. A method for quantifying and using a progression of a player's
skill with respect to an interactive computer game simulation,
comprising: continually quantifying a first player's skill during
one or more events of a game type; analyzing a plurality of
quantifications of the first player's skill to determine the first
player's skill progression rate of increase or decrease over time;
and determining by a drop in the skill progression rate of increase
that the first player is constrained by equipment, the equipment
comprising any of an input device and characteristics of a game
object model.
11. The method of claim 10, further comprising: comparing the drop
in the skill progression rate of increase for the first player
exhibiting the recorded drop in the skill progression rate of
increase with other players having similar equipment, wherein the
other players exhibited increased rates of skill progression
increase after upgrading their equipment; and determining that the
first player is constrained by his equipment based on similarities
of skill progression rates of increase between the first player and
the other players.
12. The method of claim 11, further comprising: suggesting to the
first player that he should upgrade his equipment.
13. A method for quantifying and using a progression of a player's
skill with respect to an interactive computer game simulation,
comprising: continually quantifying a first player's skill during
one or more events of a game type; analyzing a plurality of
quantifications of the first player's skill to determine the
player's skill progression rate of increase or decrease over time;
and determining by abnormal variations in the skill level of a
player that the player appears to be sandbagging.
14. The method of claim 13, further comprising: warning the player
if he appears to be sandbagging.
15. The method of claim 13, further comprising: penalizing the
player if he appears to be sandbagging.
16. The method of claim 13, wherein the abnormal variations in the
player's skill level comprise negative variations determined during
a qualifying event followed by a skill level determined for the
player measured during a race event that is consistent with a high
skill level previously exhibited by the player.
17. The method of claim 13, wherein determining the abnormal
variations in the player's skill level comprises comparing the
variations in the player's skill level with a global data set of
player skill variations.
18. An apparatus for quantifying and using a progression of a
player's skill with respect to an interactive computer game
simulation, comprising: a processor continually quantifying a
player's skill during one or more events of a game type; based on a
plurality of quantifications of the player's skill, said processor
predicting a skill level of the player for a next event; and said
processor matching a player with other players of similar skill for
the next event, based at least in part on the predicted skill
level.
19. The apparatus of claim 18, wherein quantifying the progression
of the player's skill comprises said processor comparing said
player's skill level with a target skill level at one or more
points in time during an event.
20. The apparatus of claim 19, wherein said one or more points in
time coincide with increments of any of time and distance.
21. The apparatus of claim 18, further comprising: said processor
creating skill cohorts comprising groupings of players with like
skills; and during a registration period, said processor admitting
a player to an appropriate skill cohort based on the player's
predicted skill level.
22. The apparatus of claim 21, further comprising: after close of
the registration period, said processor identifying players who are
likely to cross between skill cohorts during a next event; and said
processor adjusting the skill cohorts as appropriate based in part
on the identifying of players likely to cross between skill
cohorts.
23. The apparatus of claim 22, further comprising: after the close
of the registration period and based on available player liquidity,
said processor adjusting the skill cohorts as appropriate to create
matches optimized for minimal skill variance within a cohort.
24. An apparatus for quantifying and using progressions of skill
for a plurality of players with respect to an interactive computer
game simulation, comprising: a processor continually quantifying a
player's skill during one or more events of a game type; and said
processor determining for a plurality of players over a first time
period comprising a first plurality of events, a first set of
players who have progressed from a first skill level to a second
skill level at a first rate.
25. The apparatus of claim 24, further comprising: said processor
determining for the first set of players over a second time period
comprising a second plurality of events, equipment comprising any
of a car type and a track type that has enabled a second set of
players within the first set of players to advance faster to a
third skill level during the second time period; and said processor
encouraging the second set of players to continue using their
current equipment.
26. The apparatus of claim 24, further comprising: said processor
determining over a second time period comprising a second plurality
of events and for the first set of players, equipment comprising
any of a car type and a track type that has enabled a third set of
players within the first set of players to advance more slowly to
the third skill level during the second time period; and said
processor encouraging the third set of players to upgrade their
equipment.
27. An apparatus for quantifying and using a progression of a
player's skill with respect to an interactive computer game
simulation, comprising: a processor continually quantifying a first
player's skill during one or more events of a game type; said
processor analyzing a plurality of quantifications of the first
player's skill to determine the first player's skill progression
rate of increase or decrease over time; and said processor
determining by a drop in the skill progression rate of increase
that the first player is constrained by equipment, the equipment
comprising any of an input device and characteristics of a game
object model.
28. The apparatus of claim 27, further comprising: said processor
comparing the drop in the skill progression rate of increase for
the first player exhibiting the recorded drop in the skill
progression rate of increase with other players having similar
equipment, wherein the other players exhibited increased rates of
skill progression increase after upgrading their equipment; and
said processor determining that the first player is constrained by
his equipment based on similarities of skill progression rates of
increase between the first player and the other players.
29. The apparatus of claim 28, further comprising: said processor
suggesting to the first player that he should upgrade his
equipment.
30. An apparatus for quantifying and using a progression of a
player's skill with respect to an interactive computer game
simulation, comprising: a processor continually quantifying a first
player's skill during one or more events of a game type; said
processor analyzing a plurality of quantifications of the first
player's skill to determine the player's skill progression rate of
increase or decrease over time; and said processor determining by
abnormal variations in the skill level of a player that the player
appears to be sandbagging.
31. The apparatus of claim 30, further comprising: said processor
warning the player if he appears to be sandbagging.
32. The apparatus of claim 30, further comprising: said processor
penalizing the player if he appears to be sandbagging.
33. The apparatus of claim 30, wherein the abnormal variations in
the player's skill level comprise negative variations determined
during a qualifying event followed by a skill level determined for
the player measured during a race event that is consistent with a
high skill level previously exhibited by the player.
34. The apparatus of claim 30 wherein determining the abnormal
variations in the player's skill level comprises comparing the
variations in the player's skill level with a global data set of
player skill variations.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. provisional patent
application Ser. No. 61/779,332, filed Mar. 13, 2013, which
application is incorporated herein in its entirety by this
reference thereto.
BACKGROUND OF THE INVENTION
[0002] 1. Technical Field
[0003] The invention relates to interactive electronic games and
interactive simulation environments. More particularly, the
invention relates to skill quantification, skill progression
analysis and skill matching with respect to an interactive
simulation environment.
[0004] 2. Description of the Background Art
[0005] Player skill can be evaluated in games for a number of
purposes. For multi-player games, a ranking system is predominately
used both to identify and to track the skills of gamers in a game
to match them into competitive matches better.
[0006] Probably the most popular skill ranking and matching system
used today for multi-player games is Elo. The Elo rating system
calculates the relative skill levels of players in two-player
games, such as chess. It is named after its creator Arpad Elo, a
Hungarian-born American physics professor. The Elo system was
developed as an improved chess rating system, but today it is also
used in many other games. Per the Elo system, each player has a
rating, which is a number. A higher number indicates a better
player, based on results against other rated players. The winner of
a contest between two players gains a certain number of points in
his rating and the losing player loses the same amount. The number
of points won or lost in a contest depends on the difference in the
ratings of the players. Thus, a player gains more points by beating
a higher-rated player than by beating a lower-rated player. Over a
series of games, a player's rating goes up if the player does
better than expected.
[0007] Another popular skill matching system is the TrueSkill.TM.
system developed by Microsoft Research. The TrueSkill ranking
system only uses the final standings of all teams in a game to
update the skill estimates (ranks) of all gamers playing in the
game. One difference from other ranking systems is that the
TrueSkill ranking system characterizes skill by two numbers, i.e.
the average skill of the gamer, and the degree of uncertainty in
the gamer's skill. If the uncertainty is high, the ranking system
does not yet know exactly the skill of the gamer. In contrast, if
the uncertainty is small, the ranking system has a strong belief
that the skill of the gamer is close to their rated average
skill.
[0008] Both the Elo and TrueSkill rating systems, as well as other
known skill rating systems, assume that the skill of a player in a
multi-player competitive game can only be determined as a result of
competition. It would be advantageous if a player's skill could be
determined quickly without competing against others, especially for
games that include online interactive simulation environments, such
as vehicle racing.
SUMMARY OF THE INVENTION
[0009] Embodiments of the invention provide an online interactive
simulation and/or game environment in which an object's motion is
controlled by a player along a path. A player's skill level is
quickly quantified as they control the object traversing the path.
In this way, the player can be properly placed in games with those
of like skill without having to compete with others. An optimal
path is established for the object's travel during a portion of a
game, and at each increment of travel, an optimum velocity or time
delta is established. The path of the object that is being
controlled by a player who is being rated is then tracked over the
same path. At each distance increment the object's position and
velocity or time delta are recorded and compared with an optimum
value. Deviations therebetween are calculated on an incremental
basis, and the aggregate score is used to determine the player's
skill level for a set of equivalent conditions.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is a schematic diagram depicting the path of a
simulated vehicle as it negotiates a section of simulated race
track or course, including a straight portion and a turn, and also
including incremental measurements that are made to measure player
skill relative to target or optimal skill according to the
invention;
[0011] FIG. 2 is a flow chart describing how player skill is
measured for a player who is controlling a virtual object as it
negotiates a path in a virtual environment according to the
invention;
[0012] FIG. 3 is a block schematic diagram that shows a system for
measuring player skill according to the invention;
[0013] FIG. 4 is a table that lists player controllable inputs
according to the invention; and
[0014] FIG. 5 is a block schematic diagram that depicts a machine
in the exemplary form of a computer system within which a set of
instructions for causing the machine to perform any of the herein
disclosed methodologies may be executed.
DETAILED DESCRIPTION OF THE INVENTION
[0015] An apparatus and method is described for operating an online
interactive simulation and/or game environment, in which an
object's motion along a path is controlled by a player. A player's
skill level is quickly quantified while the player controls the
object traversing the path, so that the player can be properly
placed in games with those of like skill. This is accomplished
without having the player compete with others.
[0016] In embodiments of the invention, an optimal path is
established for the object's travel during a portion of a game and,
at each increment of travel, an optimum velocity or time delta is
established. The path of the object that is being controlled by a
player being rated is then tracked over the same path. At each
distance increment the object's position and velocity or time delta
are recorded and compared with the optimum. Deviations therebetween
are calculated on an incremental basis, and the aggregate
determines the player's skill level for a set of equivalent
conditions.
[0017] For simplicity, the embodiments described herein focus on
vehicle racing as an exemplary and non-limiting embodiment of the
invention. The method and apparatus described herein is also
applicable to any interactive simulation environment where a player
controls the movements of an object along a path, and where an
optimum path and velocity profile can be constructed for a
particular set of conditions. The method and apparatus described
herein is also applicable to any skill quantification where an
individual's skill may be measured with respect to an optimum or
target skill level without that player having to compete with other
players.
[0018] A vehicle may race over a closed-loop or open loop course.
For a closed-loop course, each circumvention of the course is
referred to as a lap. In one embodiment of the invention, over the
course of a lap, for example, a vehicle's path is tracked both on
straight sections or straights, as well an in turns. It is in turns
where a vehicle's optimum path and incremental velocity is most
influenced by track conditions. It is also in turns that a player's
skill tends to deviate the most when compared with an optimal level
of skill because a mistake made in a turn can have a major negative
affect on velocity for a straight that follows the turn.
[0019] The fastest path around a race track is dependent on a
number of variables, including but not limited to: [0020] The
vehicle type and capabilities; [0021] The track surface condition,
e.g. dry, wet, oil covered, debris covered, etc.; [0022] One or
more instances of passing another vehicle or being passed; and
[0023] The entry point for a turn;
[0024] The optimal or target path and time/velocity profile for a
player controlled object's travel along that path shall be herein
called tS or Target Skill. For a vehicle racing simulation or game
environment, this normally includes a target time duration for
traversing the path that is the minimum possible under the chosen
conditions. However, for some simulation or game environments the
duration for traversing the path may not be the minimum possible
but, instead, may include a specific profile of velocity along an
optimal physical path. The path and velocity profile for a player
controlled object's travel along a path as guided and influenced by
a specific player shall be herein called pS or Player Skill.
Knowing Skill
[0025] A key premise of Skillquant.TM., an apparatus and method
according to the invention that is also associated with the online
virtual multi-player racing environment called Simraceway.RTM., is
that skill is knowable and precisely quantifiable. Those skilled in
the art will appreciate that the invention is readily applied to
other systems and that references to Skillquant.TM. and
Simraceway.RTM. are solely for purposes of example and explanation,
and not by way of limitation. While one specific Skillquant
implementation cannot necessarily be used for all game types
universally, adaptations of the apparatus and method described
herein enable variations on Skillquant implementations that support
other types of vehicle performance and racing games, and other
genres of games where an object is guided over a path by a player,
and where an optimal profile for velocity over that path can be
determined.
Players
[0026] A player is a participant who is able to influence the
outcome of the game. However, the identification of players may not
always be simple. For instance, in the online interactive racing
game Simraceway.RTM., pit crews and officials are players as well
as drivers because they can influence the outcome of a match or
race. Also, the abstract third party of chance can in some cases be
considered a participant as well. For instance, a pit crew's
performance can be determined by a real player participating in an
event. Alternately, the performance of tasks normally performed by
a pit crew can be performed automatically by the game provider,
e.g. by artificial intelligence (AI) players, with variations on
that performance being applied according to chance. Just as the
best F1 driver occasionally has a bad pit stop through no fault of
the driver, a chance element can accomplish a similar effect.
Target Skill (Hereinafter "tS")
[0027] Target Skill is the optimal skill that a player can in
theory exhibit over the game or simulation at any given point in
the game. This is the element of the gameplay wherein the player
can control the outcome by performing as close as possible to this
optimal level. For games of material skill (S) and material chance
(C), tS is concerned with the player's successful control of S.
Note that for games where the element of chance C is exerted by the
provider, the exercise of a high degree of player skill pS even
approaching the tS level does not necessarily derive an outcome in
the player's favor, if for instance chance C is the predominating
factor.
[0028] Overall, target skill (tS) is the optimal driving line for a
vehicle racing game or simulation. However, a specific target skill
(tS) calculation is different for each game type when the
techniques described herein are applied to other forms of games or
simulations.
[0029] Different methods and/or algorithms are available to
determine the fastest line on a race track. One method is to have a
computer system run a simulation to drive a ghost vehicle around a
track while taking different lines through turns, braking at
different points, and applying the throttle at different times
exiting a turn until, through a process of successive
approximation, a convergence is observed on a line and velocity
profile that provides the fastest lap for a set of conditions.
Player Skill (Hereinafter "pS")
[0030] Player Skill is the skill exhibited by a player relative to
target skill (tS). When a player is for example controlling a
vehicle, pS is determined based on the vehicle's position relative
to the limit line, where the limit line is the optimal physical
path for a given set of conditions. Here, for example, the value 0
means that the vehicle is perfectly aligned with the optimal path
and deviation values above this, measured for example in
centimeters, are less well aligned.
[0031] The track diagram 100 of FIG. 1 shows a portion of a vehicle
racetrack, including a straight 112 leading up to a turn or corner
114. Here, the path representing target skill (tS) 102 is shown as
a solid line, and the path representing an exemplary player skill
(pS) 104 is shown as a dotted line. A vehicle enters this section
of a race track at an entry point 106. According to the tS line
102, the fastest path turns-in from the far side of the track and
subsequently approaches the inner edge 108 at the apex of the turn.
For this example, the inner edge of the turn comprises a shaded
area representing a curbing. After passing near the apex of the
corner, the tS line drifts to the outside of the track as it
reaches the exit 110 of the turn.
[0032] FIG. 1 shows a number of exemplary deviations 116 where the
player controlled physical path (pS) deviates from the optimal tS
path. According to an embodiment of the invention, physical
deviations from the optimal path are calculated at each distance
increment based upon which skill is being monitored. This distance
increment could be any physical scale, including for example
centimeters. While the increment for calculating deviations may
occur at regular distance intervals along a path, the resolution
for some simulations may be limited to the frame-rate for a video
display output. In FIG. 1, two adjacent deviation measurements 118
are shown with an exemplary distance increment 120 separating
them.
[0033] FIG. 2 is a flow diagram 200 that describes a process for
determining pS relative to tS for a player controlled object
traversing a physical path, where the physical path is represented
in a virtual simulation environment.
[0034] In step S210, for a given set of conditions an optimal path
is established for a player controlled virtual object and an
optimal timing or velocity profile is established for traversing
that path. The combination of the optimal physical path and optimal
timing or velocity profile represents target skill (tS).
[0035] In step S220, the motion of the virtual object is monitored
as it traverses the path under player control with respect to time
and position.
[0036] In step S230, at each increment of distance along the path
the system measures and records any distance deviation of the
controlled object's position from the ideal path, and any timing
deviation from the ideal timing profile. The timing deviation may
be represented as a time delta relative to an optimal time for the
controlled object to pass through a specific distance increment
along the path. The timing deviation may also be represented as a
velocity relative to an optimal velocity at a specific distance
increment along the path.
[0037] Subsequently in step S240, each of the distance and timing
deviations at all distance increments along the path are
aggregated.
[0038] In step S250, a score rating for the particular player
controlling the object along this path is created based on the
aggregated result, providing an overall pS value for the player
with respect to that path. As a player participates in more games,
the player's pS ratings can be aggregated to determine a pS skill
rating that benefits from experience over multiple games.
Skillquant Assumptions
[0039] In an embodiment of the invention, the following is assumed:
[0040] The system has perfect information, i.e. all material input
variables are knowable in a virtual game or race. They are not in
real life. [0041] Every vehicle and track combination has been
tested and given a tS limit line or target skill value for each of
a set of conditions. If a condition changes, the tS limit line may
have changed as well. The handling of changed conditions is
described below. [0042] Events for a vehicle racing game can
include for example the following types: [0043] Hotlap--Single
player events in pursuit of fastest lap [0044] Challenge
Races--Multi-lap, multiplayer races of <10 laps [0045] Endurance
Challenges--Multi-lap, multiplayer races of >10 laps and
involving a pit stop [0046] Only Endurance Challenges require
pitting. [0047] By default, pit crews have equal skill when they
are artificially generated. Alternately an element of chance can be
injected at random by occasionally having some driver's pit stops
take longer. [0048] All cars by default have fixed set ups, e.g.
suspension adjustment, brake bias adjustment, wing settings, etc.
If any vehicle parameter may be adjusted by a driver, then that
parameter becomes a "Player Controllable Input." [0049] If it is
desirable to minimize the need to adjust tS dynamically or choose
an alternate tS due to changing track conditions then, by default,
track or environment condition changes either do not occur or have
definable start and end times during an event. This enables their
impact on the tS limit line to be quantified more easily. Note that
more complex solutions that take changing conditions into account
are described below.
[0050] For an exemplary vehicle racing game, a player (p) may
include any of the following: [0051] Online racers--Human or
artificial intelligence (AI) or simulated racers [0052] Pit
Crews--Human or AI [0053] Actual Chance--Network interruptions,
power surges or failures, client or server processing stutters or
delays [0054] Simulated Chance--Programmed random variables
representing the real chance factors exhibited in real motorsport,
for example tire blowouts, mechanical failures, third party player
accidents, etc.
Event Instance and Game Instance
[0055] An event instance is a single complete session of a
simulation. This is different for each event type and is defined
uniquely for each event. In Simraceway terms, this may be, for
example, a Hotlap Competition, Challenge, or Quick Race, where the
number of laps is typically different for each type of event. In
Simraceway for example, a Hotlap allows the player to attempt to
achieve a single perfect lap, whilst a Challenge event may ask
entrants to complete several hundred laps with the added difficulty
of each entrant trying to interfere with each other's line.
However, these can both be calculated in the same way by distilling
the problem to the lap level and summing the equally weighted
variance of the event's constituent laps. In embodiments of the
invention described herein for closed-loop vehicle racing events, a
game may be thought of as a single lap where the Skillquant scores
for each constituent lap are aggregated to determine the
corresponding event's Skillquant score.
[0056] Note that the definitions for game and event instances for
some game and event types is straightforward in some cases but not
for others. For example, a golf game may be viewed not as a game of
18 holes, but rather 18 games of a single hole played in an
arbitrary sum succession. Alternately, a game of golf may be viewed
as a succession of games where each game comprises a single stroke,
with the event still comprising 18 holes. Here, to provide a total
pS rating for an event, the individual pS ratings for all strokes
may be summed over the total of all 18 holes.
Dynamically Adjusting tS for Changing Conditions
[0057] When a player, for example in a vehicle racing game, finds
himself forced off the normal tS line, the player's skill can still
be judged effectively if tS is temporarily adjusted for that
section of the track. This can be accomplished by at least two
methods: [0058] A plurality of alternate tS lines can be predefined
before an event or game such that when a vehicle deviates from the
optimal line, they are thereafter compared with an alternate tS
line, at least for that portion of the track. [0059] The tS line
for that portion of the track can be dynamically altered according
to an algorithm that takes into account the changing conditions.
For instance, if a vehicle moves off the optimal tS line toward the
inside while entering a turn to pass a slower car, the algorithm
can dynamically adjust tS so that the vehicle's position deviation
is essentially zero at that point, and the optimal velocity or time
profile is set to be consistent with the driver's velocity as his
vehicle pulls even with the vehicle being passed.
Computing pS
[0060] In an embodiment, Player Skill (pS) is derived as the
distance and time variance from Target Skill (tS) at each distance
increment along a path on a single point basis, or optionally on a
covered volumetric approach where instead of determining an
object's position as a single point, the system according to the
invention uses a total 2-dimensional volume of the object in
determining the object's position. In the case of a vehicle racing
game, the latter method is slightly more accurate, particularly
where rotation or yaw of a racing vehicle is concerned.
[0061] For a PDC hotlap event (2.3 miles per lap) a lap provides,
for example in raw form, approximately 370 k data points (2.3
miles*160,934 cm per mile) for monitoring pS. Each data point for
pS includes a position or distance deviation, typically measured
for example in cm, and a timing deviation which can be implemented
as either units of .DELTA. time, or units of instantaneous .DELTA.
velocity for a particular distance increment along the path, the
velocity being compared with the instantaneous velocity
corresponding to tS at that point. To compute pS, as shown in FIG.
2, these deviations are recorded as shown in step S230 and then
aggregated per step S240. The aggregation can be performed
according to a number different methods, for example: [0062]
Distance and timing deviations can be recorded separately at each
distance increment, a standard deviation can be calculated for each
parameter for that distance increment, and the results then
aggregated for the entire path. [0063] Distance and timing
deviations can be aggregated at each distance increment, and then a
standard deviation can be calculated that aggregates the combined
deviations over the entire path. [0064] Distance and timing
deviations can be recorded separately at each distance increment
and multiplied together for that distance increment. A standard
deviation can be calculated for the results of the multiplications
for each distance increment, and the results then aggregated for
the entire path.
[0065] Regardless of which method is chosen for aggregation of the
deviations, it may be advantageous to compute the standard
deviation for the aggregated deviations. It may also be
advantageous to convert the aggregated deviations to a logarithmic
form because there can be for some events a significant range
between pS scores of different players. A logarithmic scale enables
metering very high fidelity at one end, for example the
centimeters/millimeters and 10ths of seconds of leaders, and yet
also handle new users who are a lap behind. Scores below a certain
level may be cut off or eliminated because they are essentially too
low to be meaningful.
[0066] After computing a pS score, it is added to the Skillquant
database and then typically rendered to the user as a percentile
specific to a game or event.
Generalized System Configuration
[0067] FIG. 3 is an overview diagram 300 that shows the major
components involved in operating a virtual simulation environment
or game according to the invention, including Mods (virtual models)
for vehicles 310 and tracks 312 for inclusion in an exemplary
vehicle racing game. An optimal tS line and associated timing
profiles are established with respect to at least a portion of a
track 312. A game client 302 resides on a user machine 304 and
communicates via a network with a game server 306 located, for
example, on a host game website 308. Most often the network
connecting client 302 and server 306 is the Internet.
[0068] To install a new model of a vehicle 310 or a track 312, an
appropriate compressed file containing the necessary files and
directory structure are downloaded from a content delivery network
(CDN) 314 or, alternatively, from the game website 308. Software
operating on any of user machine 304, game server 306, or CDN 314
may operate on one or more processors and/or one or more servers,
and use memory and database resources located thereon. Computing
and database resources according to the invention may also operate
in the cloud. The one or more processors and/or one or more servers
may be co-located, spread over different locations, located in the
cloud, or a combination thereof. For purposes of the invention, the
use of servers that are part of a cloud computing platform provides
a capability for easily expanding or contracting computing
resources required. Thus, a system implemented according to the
invention can quickly and efficiently adapt to client needs and it
is only necessary pay for the computing resources needed at any
point in time. A provider of cloud computing resources essentially
provides infrastructure as a Service (IaaS)
Player Controllable Inputs (Hereinafter "PCIs")
[0069] Each game has its own definition of player controllable
inputs. While one embodiment of the invention described herein
evaluates pS for a player in terms of incremental variation of an
object or vehicle's path with respect to a position and velocity
profile when compared with an optimal tS path, another embodiment
for judging player skill compares a player's operation of his game
console or controller device (their PCIs) with those of a virtual
player operating an equivalent set of PCIs, for an equivalent set
of operating conditions. For this alternative embodiment or complex
form of the invention it is assumed when a player operates their
PCIs in a like manner to a virtual player performing optimal
operation of equivalent PCI's and under the same conditions, that
their skill level approaches that of a target skill level (tS).
[0070] A player's operation of PCIs may be measured in different
terms, and these terms should be appropriately adapted to a
particular game and game controller console. For a vehicle racing
game for example, PCIs such as brake and throttle may be measured
in terms of percentages relative to a maximum brake position and a
maximum throttle position, respectively. PCI's such as steering
input can also be measured on a percentage basis, or alternately
measured on a basis measuring degrees of rotation where 360.degree.
represents a full rotation of a rotatable steering input device,
and 540.degree. represents one and a half rotations of a rotatable
steering input device. If a joystick is used instead controlled
steering input, a percentage basis might be more appropriate. When
pS is determined and compared with tS on the basis of PCIs, it is
important that methods implemented at the provider properly take
into account the relative influence of each PCI. This is critical
for a player to be correctly assessed when this embodiment is
used.
[0071] Thus, for a complex form based on PCIs, pS can be
alternately calculated whereby pS variance from tS is measured not
by the object's variance with respect to a tS comprising a physical
path and a timing profile (the simple form), but, instead, by the
variance on PCIs compared with a tS defined for PCIs. For example,
a player may hold a throttle position on a game console (pS) at 72%
vs. an optimal tS position of 76%, at a specific increment (time or
distance) of the path that the controlled object traverses.
[0072] Table 400 of FIG. 4 provides a list of player controllable
inputs. Column 402 shows inputs controlled by beginners and column
404 shows inputs controlled by intermediate players. For the
beginner and intermediate players, a (%) is shown for each input
type and represents the % of control a system provides to those
broad skill class users. This is sometimes described as a skills
appropriate experience for the user. Controllable inputs for
advanced players are shown in column 406, the simulation column 408
refers to a broad label for the top group of Skillquant ranges for
ease of reference. Note that the % values are estimates and vary
with driver and/or player experience level. For example different
aides can be offered to beginners vs. intermediates vs.
experts.
[0073] The complex form with metering at player controlled inputs,
rather than simply using positional output, can require up to eight
separate calculations vs. just one calculation for the simple form.
However, for combining with data taken from real racing events,
telemetry data can be easily integrated because it already meters
these driver and/or player inputs. The complex form is not
necessarily required for everyday use, but has applications in
professional services for driver development. Whilst more complex
to build from the ground up, given that all telemetry can typically
be accessed fairly easily, it may present an engineering
shortcut.
[0074] Key differences of the complex form compared with the simple
form include: [0075] pS derivation becomes a function of actual
player controllable inputs rather than their output, being the
motion of the object, vehicle, or car. This adds eight calculation
steps. [0076] tS itself becomes more complex to calculate,
consistent with a simulation which considers all environmental
dynamic variables rather than assuming they are constant. This adds
seven calculation steps for the following conditions, including but
not limited to: track condition, tire wear, brake fade, fuel load,
track temperature and altitude/fuel mix. [0077] Variance is
calculated as both latitudinal and longitudinal from tS on the
limit line.
Other Benefits of Measuring Skill
[0078] Metering the skill that creates an outcome, e.g. lap time or
position for a vehicle racing simulation, as opposed to simply
measuring the outcome has other benefits to both the game provider
and the players. Some benefits include the following: [0079] Data
Insight: The game and game provider can get an understanding of the
skill behind the outcome, which helps the game and game provider
improve skill matching. Improved skill matching directly influences
engagement and where monetization is included, improved
monetization.
Matching Engine
[0080] For asynchronous events, it is useful to use pS to match
skill among players to: [0081] Determine (dynamically) the initial
skill range of entrants. [0082] Calculate, according to a user
operated ranges, skill cohorts, i.e. groupings of players with like
skill, according to a desired minimum number of players and
according to prize dynamics, the goal being to achieve the tightest
range of skill cohort that achieves the required minimum number of
entrants after which point a new cohort should be created. [0083]
Admit users to their relevant skill cohort. [0084] After event
entry registration closes, true up above the skill cohorts. [0085]
Identify users who may cross skill cohorts.
Player Liquidity
[0086] Player liquidity represents the pool of players that are
available at any one time for any one skill; race class; prize
type; location; etc., for example the real time pool of players
from which an operator and/or provider of a simulation or game (the
"provider") can reasonably match players against each other. The
term tightly manage refers to the variance of skill that must be
applied to draw enough matches. The more liquid the environment,
for example the more players online at once and available to play,
the tighter matches are. A dynamic element in the process of skill
matching refers to widening and/or narrowing the range of skill
variance used by the provider to drive matches according to the
prevailing player liquidity. Quantifying a player's skill enables
the game and game provider to manage player liquidity tightly via
dynamic skill classing.
Other Skill Quantification Methods
[0087] While embodiments of the invention focus primarily on
comparing a player's skill level with respect to guiding a vehicle
along an optimal path that includes an optimal velocity profile,
other methods for establishing a target skill level and rating a
player skill level relative to the target skill level are possible.
These include for a racing game comparing lap times for a player
with an optimal lap time under equivalent conditions and with
equivalent equipment. Another method can include evaluating
instantaneous acceleration of an object being guided by a player at
different points in the game. Skill can also be evaluated such that
consistency of performance is included as a component of evaluating
a player's skill. Regardless of what methods are used to evaluate a
player's skill level, it is also useful to measure the skill
progression of a player over time. Based on evaluations of skill
progression, benefits arise for both a player and a provider.
Skill Progression and Real Time Experience Curves
[0088] Players improve their skill at different rates. Even a
player who eventually has great skill may initially improve slowly.
When first placing a player in a multi-player competition, it is
critical for encouraging and maintaining player participation that
their initial experience is as positive as possible. Therefore, it
is advantageous that a player be matched against other players of
similar skill, even in their very first competition and, therefore,
their skill must be known prior to their first competition. Then,
during the player's first competition they can enjoy a fair and
competitive matching with other players, and simultaneously can be
re-evaluated on their skill as the game progresses so that they are
subsequently optimally matched for their second competition.
[0089] Continually quantifying a player's skill reveals to the game
and game provider a user's likely skill progression through the
game, and provides an opportunity to optimize the content and
events that can be served to the player. Continually quantifying a
player's skill also enables the establishment of real time
experience curves which can be used to improve the comfort level of
a player by having them feel competitive and, where a game includes
any degree of monetization, get a player racing for cash
sooner.
[0090] This is accomplished for example by dynamically matching car
and race types to: [0091] A player's current skill level. [0092]
Use a player's experience curve to understand to what level the
provider can take the player's skill with a given car and/or track
combination.
[0093] For example, a global skill data set may show that players
who have progressed from skill level A to skill level B at rate X
progress to skill point C faster if their next race is with car
type A with track type A, and progress slower if their next race is
with Car type B and Track type B.
[0094] The following example contrasts the methods of tracking
skill described herein with prior art methods based on event
outcomes. Assume a first player is running in 9.sup.th position in
a 12 hr endurance race, however eventually places first because
position's 1 and 2 caused a pile-up (accident) on the last lap that
took-out themselves and the cars in positions 3-8 closely
following. For this scenario, the winning driver is not really the
most skilled in that race. They may have been for that last lap
because it may have taken great skill to navigate the pile up, but
they may also have just been lucky. Their pS line relative to a
dynamically-adjusted tS line for the last lap would determine that.
So, the winner's score here would be 9.sup.th position for the laps
preceding the last and then 1.sup.st for one lap, which is not the
same level of skill as the binary outcome would suggest.
Highlighting Restrained Players
[0095] Continually quantifying a player's skill level enables a
game or game provider to highlight players and/or racers who are
very skilled but are restrained by poor equipment and or vehicles.
For example, a player may have chosen a vehicle simulation model
that is inferior to others they compete against. Alternately, they
may be using an inferior game control console or input device that
provides an additional degree of difficulty that other game
consoles may not impose.
[0096] By tracking the rate of skill progression with a given
controller or input device type, which are tracked by the provider,
any leaps in skill level that occur as a user upgrades to a
different controller device are compared to a global dataset of
such data. The results of such a comparison can reveal where
players are hitting skill ceilings because of their interface
equipment. For example, they may have progressed well with
keyboards but have achieved their maximum available skill level
with a keyboard and need to upgrade to a joypad or steering wheel
input device to increase the skill level they can effectively
exert.
Highlighting Sandbagging Players
[0097] Continually quantifying a player's skill enables a game or
game provider to highlight players and/or racers who are very
skilled but are attempting to sandbag, wherein a player
intentionally makes mistakes to lower their apparent skill level
which eventually enables them to be matched with other players they
can easily beat. In embodiments of the invention, this is
accomplished by looking for unusual, positive skill variance within
events. For example, assume that an event consists of 2 levels,
i.e. qualifying and race. Here, qualifying determines which of
three classes (A to C, C is lowest) of race a player enters, where
all classes may offer the same level of prize. If a user sandbags
or artificially lowers their skill level to qualify for class C,
but is in fact a Class B or Class A skilled player, the provider
can track the skill leap from qualifying to race and trigger a
variance flag. The variance flag indicates that having compared
this particular player's skill progression to a global data set, it
has been determined that no user with their equivalent starting
skill level has ever progressed to that degree from one session to
another. As a result of the variance flag, the provider may either
warn or penalize the player in question, or alternately log the
event and wait until a pattern of sandbagging has been established
for that player before issuing a warning or penalty.
Increased Richness of Game Data
[0098] Continually quantifying a player's skill enables a game or
game provider to accumulate a richer set of performance data for
the player in a game, rather than just scoring the outcome. It
provides skill data on a player's intra game performance, not
simply a game's binary outcome, thereby providing a richer insight
on how the black and white data of results was achieved.
[0099] According to embodiments of the invention, Skill
Quantification ("Skillquant") is a proprietary, modular, skill
quantification and management engine that may be implemented as
part of interactive electronic games and interactive simulation
environments. Its central principle that differentiates from other
skill and ranking systems is that the maximum skill a player can
exert over a game is, in fact, knowable. As such, the skill of a
specific player can be evaluated and measured against the known
maximum level without that player having ever competed head-to-head
with another competitor.
[0100] According to embodiments of the invention, player skill may
be metered at distance or time increments having any chosen
duration, however it is preferred to meter player skill in digital
environments at the highest possible fidelity. This degree of
unique data visibility facilitates the following functions, in a
modular system configuration: [0101] Accurate, real-time, player
skill quantification [0102] Accurate player skill matching with
minimal player history [0103] Optimization of player engagement
through real time experience curves [0104] Player liquidity
management [0105] Compliance monitoring for both operator and
competent authority [0106] Detection of cheating and fraudulent
player behavior patterns
Consistency
[0107] Consistency can be a simple and robust constituent of a
skill evaluation. However, this test must be applied within the
correct skill tolerance. For example, the variance of skill that
equals good or highly skilled for one game may well be different
for another. A skill consistency number, supported by a
statistically robust data set, can be hard to find. This is
particularly the case when trying to make a legal classification
for an entire population. For instance, using the skill tolerance
or skill consistency of PGA golf players is not a good proxy for
the entire user base of golf. Similarly, using the skill tolerance
of F1 drivers is not a good proxy for the entire user base of race
car drivers.
[0108] If a government, for example, wanted to determine whether
racing enthusiasts in general should be allowed to play online
interactive race car games for cash prizes, it is appropriate to
determine what the predominant skill range of racing enthusiasts
is. This should be tested, not simply at the top level of skill,
which is generally where most data lies and so is tempting as a
proxy, and which however represents a skill that is not possible
for the general populace that would be participating online. It is
necessary to determine what the general skill level is as
practically operated by the predominant number of players playing
the game. Digital environments, with perfect information available,
are attractive for this reason.
Skill and Chance
[0109] Virtually every game of every genre, digital or otherwise,
has some element of skill and some element of chance. There are
some exceptions to this rule but fewer than one would expect, and
these are contingent upon the definition of a game. For instance,
one exception is "Naughts and Crosses" (alias tic-tac-toe) which on
the surface seems to be a game where competent players always score
a draw. However, it may actually be a simple form of mental
exercise for developing young minds. Overall, an objective view of
the subject requires one to detach their preconceived notions of
games classification and instead focus entirely on the data.
Skill/Chance Paradox and Legal Classification
[0110] Where a game requires so much skill that to compete
effectively it is practically beyond the reach of a majority of
users, then upon testing it behaves as a game of chance for that
majority of users. This can be relevant for legal classification
purposes where competent authorities are not only concerned with
testing the technical position of the game, but also scrutinizing
its practical operation (predominance, predominance test) which is
a well known legal test. A good example of this is exemplified by
some variants of roulette that can actually be played as a game of
skill by a very small number of people able to calculate the
landing position of the ball based on its trajectory, but whereas
the vast majority cannot. Another example are synthetic CDO's,
which in a context similar to a racing game would be regulated as a
game of chance, not a financial markets instrument. For such
scenarios, if a test was applied to these financial markets to
determine the degree of gambling involved, the predominant number
of users would not be correctly determining and/or predicting their
outcome from the engagement, and so would not able to skillfully
control the outcome of that game. Thus, because there is an entry
cost, i.e. acquisition of the CDO, and prize, i.e. the profit,
these would be gambling. For these financial markets, core winners
would probably be the market making banks who made the margin on
each buy and sell trade, a lot like a casino. Note that financial
products such as these evade gambling testing not by successfully
passing the test of being a game of skill. Being correctly
determined as a game of skill, specifically the thing that gambling
is not, would represent the test that truly should determine how
they should pass. Instead, governments simply carve them out by
un-proven classification. Financial product definitions simply say
that if something is a gambling game, it is not a financial
investment product and vice versa.
[0111] With regard to compliance monitoring with respect to
governmental regulations for online gambling, it is important to be
able to rate the components of skill and chance in an online game
when players wager on the outcome of a competition. As mentioned
above, if the skill level required to compete effectively is
extremely high, a competition takes on more characteristics of a
game of chance for the average player. By matching players by skill
levels and doing so right from the beginning of their
participation, the ability to compete effectively based on skill is
effective from the beginning and the game is may be classified more
as a game of skill.
[0112] To make a game more interesting and enticing for some
players, but still focus predominantly on skill, additional
elements of chance may be injected into a competition. For
instance, in a vehicle racing game, simulated weather conditions
may be suddenly changed during an event, or a vehicle may suddenly
leak simulated oil on the track creating a locally slippery
condition. These sudden chance events may catch some drivers
unaware regardless of their skill level. Also, automated pit stops
may randomly encounter delays that cost even the best drivers
time.
Skill/Chance Compliance Monitoring
[0113] Some governmental authorities monitor online games to ensure
that they qualify as games of skill, especially when wagers are
made on the outcomes. Thus, it is useful to quantify the
composition of a game with respect to skill versus chance. In
simplest terms, if by exercising their control console a player can
control the outcome of a game instance, in this context, their
finishing position after a lap or game, and relative position to
tS, then this is a game of skill. If they cannot, it is a game of
chance. The percentage degree to which they can exert that control
is the percentage of skill contained in the game and the inverse
value is the percentage of chance.
Computer Implementation
[0114] FIG. 5 is a block schematic diagram that depicts a machine
in the exemplary form of a computer system 1600 within which a set
of instructions for causing the machine to perform any of the
herein disclosed methodologies may be executed. In alternative
embodiments, the machine may comprise or include a network router,
a network switch, a network bridge, personal digital assistant, a
cellular telephone, a Web appliance or any machine capable of
executing or transmitting a sequence of instructions that specify
actions to be taken.
[0115] The computer system 1600 includes a processor 1602, a main
memory 1604 and a static memory 1606, which communicate with each
other via a bus 1608. The computer system 1600 may further include
a display unit 1610, for example, a liquid crystal display (LCD) or
a cathode ray tube (CRT). The computer system 1600 also includes an
alphanumeric input device 1612, for example, a keyboard; a cursor
control device 1614, for example, a mouse; a disk drive unit 1616,
a signal generation device 1618, for example, a speaker, and a
network interface device 1628.
[0116] The disk drive unit 1616 includes a machine-readable medium
1624 on which is stored a set of executable instructions, i.e.
software, 1626 embodying any one, or all, of the methodologies
described herein below. The software 1626 is also shown to reside,
completely or at least partially, within the main memory 1604
and/or within the processor 1602. The software 1626 may further be
transmitted or received over a network 1630 by means of a network
interface device 1628.
[0117] In contrast to the system 1600 discussed above, a different
embodiment uses logic circuitry instead of computer-executed
instructions to implement processing entities. Depending upon the
particular requirements of the application in the areas of speed,
expense, tooling costs, and the like, this logic may be implemented
by constructing an application-specific integrated circuit (ASIC)
having thousands of tiny integrated transistors. Such an ASIC may
be implemented with CMOS (complementary metal oxide semiconductor),
TTL (transistor-transistor logic), VLSI (very large systems
integration), or another suitable construction. Other alternatives
include a digital signal processing chip (DSP), discrete circuitry
(such as resistors, capacitors, diodes, inductors, and
transistors), field programmable gate array (FPGA), programmable
logic array (PLA), programmable logic device (PLD), and the
like.
[0118] It is to be understood that embodiments may be used as or to
support software programs or software modules executed upon some
form of processing core (such as the CPU of a computer) or
otherwise implemented or realized upon or within a machine or
computer readable medium. A machine-readable medium includes any
mechanism for storing or transmitting information in a form
readable by a machine, e.g. a computer. For example, a machine
readable medium includes read-only memory (ROM); random access
memory (RAM); magnetic disk storage media; optical storage media;
flash memory devices; electrical, optical, acoustical or other form
of propagated signals, for example, carrier waves, infrared
signals, digital signals, etc.; or any other type of media suitable
for storing or transmitting information.
[0119] Although the invention is described herein with reference to
the preferred embodiment, one skilled in the art will readily
appreciate that other applications may be substituted for those set
forth herein without departing from the spirit and scope of the
present invention. Accordingly, the invention should only be
limited by the Claims included below.
* * * * *