U.S. patent application number 12/037923 was filed with the patent office on 2008-08-28 for artificial player character for massive multi-player on-line game.
Invention is credited to THEODORE BEALE.
Application Number | 20080207331 12/037923 |
Document ID | / |
Family ID | 39716536 |
Filed Date | 2008-08-28 |
United States Patent
Application |
20080207331 |
Kind Code |
A1 |
BEALE; THEODORE |
August 28, 2008 |
ARTIFICIAL PLAYER CHARACTER FOR MASSIVE MULTI-PLAYER ON-LINE
GAME
Abstract
An artificial intelligence system for autonomous artificial
player characters in an on-line game is provided. The artificial
player character (APC) makes use of an adaptive neural net wherein
the function of the artificial player character at any given time
is determined as a composition of other functions defined by one or
more of the following: the game designer, the personality
preference filters of the artificial player character, the current
statistical levels of the artificial player character, the database
of historical interactions of the artificial player character, a
master database of historical interactions by other artificial
player characters, and interactions of the artificial player
character with other characters and/or the game environment.
Inventors: |
BEALE; THEODORE; (Zug,
CH) |
Correspondence
Address: |
DICKE, BILLIG & CZAJA
FIFTH STREET TOWERS, 100 SOUTH FIFTH STREET, SUITE 2250
MINNEAPOLIS
MN
55402
US
|
Family ID: |
39716536 |
Appl. No.: |
12/037923 |
Filed: |
February 26, 2008 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60903381 |
Feb 26, 2007 |
|
|
|
Current U.S.
Class: |
463/42 |
Current CPC
Class: |
G06N 3/004 20130101 |
Class at
Publication: |
463/42 |
International
Class: |
A63F 9/24 20060101
A63F009/24 |
Claims
1. A computer game, comprising: an artificial player character
comprising a set of at least two probability-related values which
determine an activity of the artificial player character, each of
the probability-related values comprising a preference filter
including a filter identification and a filter value for the filter
identification, wherein the filter value is an integer value and
defines how relevant the preference filter is to an encounter of
the artificial player character during the computer game.
2. The computer game of claim 1, further comprising: a player
character; and one or more of an administrative character, a
non-player character, a monster character, a pet character, and an
environmental character.
3. The computer game of claim 1, wherein the artificial player
character further comprises a set of character statistics for the
artificial player character, a set of graphics for the artificial
player character, and a list of possessions of the artificial
player character.
4. The computer game of claim 1, wherein the artificial player
character further comprises an experience database including a
record of historical encounters of the artificial player character
and results of the historical encounters of the artificial player
character.
5. The computer game of claim 1, wherein the filter value of the
preference filter for the artificial player character is assigned
by an individual.
6. The computer game of claim 1, wherein the filter value of the
preference filter for the artificial player character is assigned
by a random process.
7. The computer game of claim 1, wherein the filter value of the
preference filter for the artificial player character is assigned
by a learned process including a record of historical encounters of
the artificial player character and results of the historical
encounters of the artificial player character.
8. The computer game of claim 1, wherein the filter value of the
preference filter for the artificial player character is assigned
by an evolutionary process combining at least two preference
filters.
9. The computer game of claim 1, wherein the preference filter
includes one of a loyalty filter, an ambition filter, an
exploration filter, an adventure filter, a risk-taking filter, an
aggression filter, a conversation filter, an attachment filter, a
persistence filter, a trading filter, an anchorage filter, an
altruism filter, a sanity filter, a greed filter, a social filter,
a romance filter, a negotiation filter, an occupation filter, an
economy filter, and a learning filter.
10. The computer game of claim 1, wherein the computer game is a
massive multi-player on-line game.
11. The computer game of claim 1, wherein the encounter is
presented by one of a player character, a non-player character,
another artificial player character, a monster character, and an
environment of the computer game.
12. The computer game of claim 1, wherein the artificial player
character is introduced into the computer game in place of a player
character when the player character leaves the computer game.
13. The computer game of claim 1, wherein the artificial player
character is introduced into the computer game when less than a
predetermined number of player characters are available to play the
computer game.
14. A method of playing a computer game including an artificial
player character, the method comprising: presenting the artificial
player character with an interaction; determining which of a
plurality of preference filters for the artificial player character
are applicable to the interaction, including comparing a filter
value of each of the preference filters to the interaction and
determining a relevance of each of the preference filters to the
interaction, the filter value defining how relevant a corresponding
preference filter is to the interaction of the artificial player
character during the computer game; calculating a priority of the
preference filters relevant to the interaction, and applying the
relevant preference filters to the interaction; determining an
action of the artificial player character to the interaction based
on the relevant preference filters; and performing the action by
the artificial player character.
15. The method of claim 14, further comprising: identifying a set
of at least two probability-related values which determine an
activity of the artificial player character, including, for each of
the probability-related values, assigning the filter value to the
corresponding preference filter, wherein the filter value is an
integer value and defines how relevant the preference filter is to
the interaction of the artificial player character during the
computer game.
16. The method of claim 15, wherein assigning the filter value to
the corresponding preference filter includes assigning the filter
value by an individual.
17. The method of claim 15, wherein assigning the filter value to
the corresponding preference filter includes assigning the filter
value by a random process.
18. The method of claim 15, wherein assigning the filter value to
the corresponding preference filter includes assigning the filter
value by a learned process including a record of historical
encounters of the artificial player character and results of the
historical encounters of the artificial player character.
19. The method of claim 15, wherein assigning the filter value to
the corresponding preference filter includes assigning the filter
value by an evolutionary process combining at least two preference
filters.
20. The method of claim 14, wherein each of the preference filters
include one of a loyalty filter, an ambition filter, an exploration
filter, an adventure filter, a risk-taking filter, an aggression
filter, a conversation filter, an attachment filter, a persistence
filter, a trading filter, an anchorage filter, an altruism filter,
a sanity filter, a greed filter, a social filter, a romance filter,
a negotiation filter, an occupation filter, an economy filter, and
a learning filter.
21. The method of claim 14, wherein the computer game is a massive
multi-player on-line game.
22. The method of claim 14, wherein the interaction is presented by
one a player character, a non-player character, another artificial
player character, a monster character, and an environment of the
computer game.
23. The method of claim 14, further comprising: introducing the
artificial player character into the computer game in place of a
player character when the player character leaves the computer
game.
24. The method of claim 14, further comprising: introducing the
artificial player character into the computer game when less than a
predetermined number of player characters are available to play the
computer game.
25. A computer-readable medium including instructions for
performing a method of playing a computer game including an
artificial player character, the method comprising: presenting the
artificial player character with an interaction; determining which
of a plurality of preference filters for the artificial player
character are applicable to the interaction, including comparing a
filter value of each of the preference filters to the interaction
and determining a relevance of each of the preference filters to
the interaction, the filter value defining how relevant a
corresponding preference filter is to the interaction of the
artificial player character during the computer game; calculating a
priority of the preference filters relevant to the interaction, and
applying the relevant preference filters to the interaction;
determining an action of the artificial player character to the
interaction based on the relevant preference filters; and
performing the action by the artificial player character.
26. A computer-readable medium including instructions for
performing a method of playing a computer game including an
artificial player character, the method comprising: monitoring a
multiplayer group of the computer game; and introducing the
artificial player character into the computer game if one of (a) a
player character of the multiplayer group of the computer game
leaves the computer game and (b) less than a predetermined number
of player characters are available to form the multiplayer group of
the computer game, wherein introducing the artificial player
character into the computer game includes determining a type and a
level of the artificial player character and assigning the
artificial player character to the multiplayer group of the
computer game.
27. The computer-readable medium including instructions for
performing a method of playing a computer game including an
artificial player character of claim 26, the method further
comprising: after introducing the artificial player character into
the computer game, removing the artificial player character from
the computer game and replacing the artificial player character
with a new player character if the new player character is looking
for a group similar to the multiplayer group of the computer game
including the artificial player character.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority under 35 U.S.C. .sctn.
119(e) to U.S. Provisional Patent Application Ser. No. 60/903,381,
filed on Feb. 26, 2007, and incorporated herein by reference. This
application is related to U.S. Non-Provisional patent application
Ser. No. ______, filed on even date herewith, having attorney
docket number B575.101.102, and incorporated herein by
reference.
FIELD OF THE INVENTION
[0002] The invention relates generally to a form of an autonomous
artificially intelligent software entity for a computer game. More
specifically the invention relates to a form of an autonomous
artificially intelligent player character for a massive
multi-player on-line game.
BACKGROUND OF THE INVENTION
[0003] Massive multi-player on-line (MMO) games are computer games
capable of supporting a number of players (e.g., hundreds or
thousands of players) simultaneously. In one arrangement, players
log onto a central server over the Internet and participate in a
gaming experience shared by numerous remotely connected users. As
such, MMO games are usually played via remote clients accessing one
of a number of central servers. Typically, MMO games are played in
a large persistent virtual environment that enables players to
compete with and against each other on a grand scale and to
interact cooperatively with people around the world.
[0004] In one application, MMO games provide a form of electronic
entertainment which allows numerous individuals to simultaneously
take part in an ongoing virtual world. These games are customarily
set in a fantasy role-playing environment, although they can
theoretically be set in any conceivable fictional genre, historical
setting, or other environment. The usual way in which the player
experiences the MMO game is to create an avatar, or Player
Character, which then travels throughout the virtual world and
interacts with a wide variety of characters which are either
controlled by other players or by the game server.
[0005] The Player Character customarily spends most of its time
improving itself through the collection of experience points, which
are earned, for example, through killing monsters and completing
adventures assigned to the Player Character by server-controlled
Non-Player Characters (NPC) scattered throughout the virtual world
in strategic locations.
[0006] FIG. 1 demonstrates the usual decision-making process for a
human-controlled Player Character presented with the opportunity to
engage in an adventure of an MMO game. The adventure structure and
the basic form of the interaction between Player Characters and
Non-Player Characters is generally a simple form of
offer-and-reward which is contingent, for example, upon the player
killing a certain number of monsters, collecting a certain number
of items, or traveling from one point to another.
[0007] The decision-making utilized in this process naturally
requires human involvement in order to make the Player Character's
decision to embark on the adventure in search of the promised
rewards or to reject it. This is a somewhat more complicated
process than it might look at first, since it requires balancing a
number of variables which include, for example, the perceived
ability of the Player Character to find Monster Y, the amount of
time required to do so, the risks of travel to Monster Y's
location, the ability of the Player Character to kill x number of
Monster Y, the likelihood that the Player Character will survive
the encounter, the trustworthiness of the NPC to make good on its
end of the bargain, and finally, the value of the reward on offer
to the Player Character.
[0008] A similar process is required for a wide variety of
interactions between Player Characters and NPCs, Monsters, and the
environment itself. These include, but are not limited to, travel
from point A to point B, economic production, economic
transactions, character allegiances and alliances, and combat. With
the sole exception of combat, wherein an attack on a non-autonomous
character type will inevitably be responded to by an attack in
self-defense, these processes inevitably require at least one of
the two parties involved in the interaction to be an autonomous
human-controlled character.
[0009] As outlined in the Table of FIG. 2, MMO games often contain
six distinct variants of game characters, which can be divided into
two categories, autonomous and non-autonomous. These variants can
be further differentiated on the basis of their control mechanism
and state.
[0010] 1. Player Characters are human-controlled characters who
take active part in the on-line game. Also known as "avatars" they
are autonomous, travel freely about the game environment subject
only to the limits set by the game designers, and their attributes
evolve over time. Their statistics, appearance, skill sets, and
possessions are dynamic, constantly changing over the course of the
game. Player Characters are only present within the game when the
human player controlling them is on-line.
[0011] 2. Administrative Characters are human-controlled characters
who only enter the on-line game in order to deal with specific
problems, usually those brought to the attention of the game's
managers by the users. Also known as "DMs" or "Game Masters", these
Administrative Characters are autonomous and serve police and
technical support functions for the participating users, however
their attributes are static as they exist primarily for specific
game management purposes. They are very seldom encountered.
[0012] 3. Non-Player Characters are server-controlled characters
who primarily exist in order to provide information, supplies, and
rewards for the Player-Characters. Usually called "NPCs", they are
non-autonomous, seldom move of their own volition outside of a
strictly limited area (if at all), and their attributes do not
evolve over time. Their statistics, appearance and skill sets are
static, their possessions may or may not be dynamic; if their
possessions are variable, the variable alters solely on the basis
of actions on the part of the Player Characters. Their presence is
typically constant within the game although their appearance may
occasionally be made variable by the use of a trigger based on a
Player Character's action.
[0013] 4. Monster Characters are server-controlled characters who
primarily serve as the generic opposition for the Player
Characters. Often described generically as "enemies" they are
non-autonomous, their movement is strictly circumscribed and their
basic attributes do not vary over time although their statistics
and state will invariably alter radically in the course of a combat
encounter with a Player Character or a Pet. Their presence and
location are usually constant and unchanging within the game,
subject only to their temporary elimination by a Player Character
or the failure of a Player Character to trigger their presence.
Their statistics, appearance, and skill sets are essentially
static, and their decision-making capabilities are crude and
essentially limited to a binary option of fight or flight.
[0014] 5. Pets are a special form of Non-Player Character which
occasionally accompany certain types of Player Characters. They are
server-controlled partially-autonomous characters which unlike
other NPCs, possess dynamic statistics and skill sets as well as
the ability to move about the game environment by following around
the Player Character to whom they belong.
[0015] 6. Environmental Characters are server-controlled characters
who primarily serve a decorative purpose within the game. They are
non-autonomous, their movement is typically limited, and they
seldom have significant interaction with the Player Characters.
[0016] In the field of Massive Multi-player On-line games, the
division between the activities of the Player Characters and the
other five character forms is distinct and easily defined. Player
Characters voluntarily engage in goal-oriented adventures, economic
activity, and combat, develop skills, abilities, talents, and
occupations along elective paths, and obtain experience,
reputation, money, and physical possessions. In contrast, none of
the five other character forms are able to do any of these things
as without the direction of a human mind providing purpose and
personal preferences for the character, they simply do not have the
capacity.
[0017] Even the sole other dynamic character type, the Pet, is
wholly dependent upon the ability of the owning Player Character's
human mind to make decisions for it as its movements are based upon
the Player Character to whom they belong. For example, the Pet's
dynamic statistics and skills are chosen by the owning Player
Character, and even its semi-autonomous ability to initiate combat
is completely at the discretion of the Player Character.
[0018] And while MMO games are rightly very popular in the gaming
community, they are already running into a fundamental design
limitation due to their heavy reliance upon Player Characters to
provide the dynamic aspect of the entertainment. Since MMO game
designers have limited control over the decision-making processes
of the participating Player Characters and are forced to leave the
vast majority of these decisions to the discretion of human beings
controlling the Player Character, they are forced to spend an
inordinate amount of time limiting an individual Player Character's
ability to destroy the gaming experience for himself and others.
Every aspect of the design, from the storyline to the physics
model, is forced to take into account the possibility that any
player may, at any time, elect to behave in a manner detrimental to
the designer's purpose for the game.
[0019] This discretion is often exercised to the disadvantage of
other Player Characters and to the detriment of the game
experience. For example, it is not uncommon for a Player Character
to find himself suddenly abandoned in the midst of a horde of
enemies because the other Player Characters with whom he was
cooperating have departed the scene, either because it was
necessary for them to go off-line or simply because they had
already achieved their goals and were uninterested in continuing to
help the Player Character. This occurs frequently in current MMO
games and severely detracts from the game's entertainment
value.
[0020] Furthermore, the wide variety in maturity levels and
sophistication among the players of an MMO game means that Player
Characters are often interacting with each other under the handicap
of possessing mutually incompatible entertainment goals.
[0021] For these and other reasons, there is a need for providing
MMO game designers with a mechanism for greater control over Player
Characters in a massive multi-player on-line game.
SUMMARY
[0022] One aspect of the present invention provides an artificial
player character for a computer game as illustrated and described
herein.
[0023] One aspect of the present invention provides a method of
defining an artificial player character for a computer game as
illustrated and described herein.
[0024] One aspect of the present invention provides a
computer-readable medium including computer-executable instructions
for performing a method of defining an artificial player character
for a computer game as illustrated and described herein.
[0025] One aspect of the present invention provides a computer game
including an artificial player character as illustrated and
described herein.
[0026] One aspect of the present invention provides a method of
playing a computer game including an artificial player character as
illustrated and described herein.
[0027] One aspect of the present invention provides a
computer-readable medium including computer-executable instructions
for performing a method of playing a computer game including an
artificial player character as illustrated and described
herein.
BRIEF DESCRIPTION OF THE DRAWINGS
[0028] FIG. 1 is a process diagram illustrating an existing
decision-making process for a human-controlled Player Character in
an MMO game.
[0029] FIG. 2 is a table outlining character variants for existing
game characters in an MMO game.
[0030] FIG. 3 is a diagram illustrating one embodiment of a neural
net illustrating one embodiment of AI-based decision-making for an
Artificial Player Character.
[0031] FIG. 4 is a diagram illustrating one embodiment of a model
of an Artificial Player Character.
[0032] FIG. 5 is a diagram illustrating another embodiment of a
model of an Artificial Player Character.
[0033] FIG. 6 is a process diagram illustrating one embodiment of
decision making for interaction of an Artificial Player
Character.
[0034] FIG. 7 is a process diagram illustrating one embodiment of
determining which Preference Filters of an Artificial Player
Character are relevant and how they are subsequently applied.
[0035] FIG. 8 is a diagram illustrating one embodiment of
determining which Preference Filters of an Artificial Player
Character have priority.
[0036] FIG. 9 is a table illustrating one embodiment of Preference
Filters for an Artificial Player Character.
[0037] FIG. 10 is a table illustrating one embodiment of a portion
of a database of exemplary encounters of an Artificial Player
Character.
[0038] FIG. 11 is a process diagram illustrating one embodiment of
updating a database of encounters of an Artificial Player
Character.
[0039] FIG. 12 is a process diagram illustrating one embodiment of
an exemplary encounter by an exemplary Artificial Player
Character.
[0040] FIG. 13 is a process diagram illustrating one embodiment of
decision-making for actions of an Artificial Player Character.
[0041] FIG. 14 is a process diagram illustrating one embodiment of
triggering actions for a character in an MMO game.
[0042] FIG. 15 is a process diagram illustrating one embodiment of
incorporating an Artificial Player Character in an MMO game.
[0043] FIG. 16 is a process diagram illustrating another embodiment
of incorporating an Artificial Player Character in an MMO game.
DETAILED DESCRIPTION
[0044] An Artificial Player Character (APC) is a client-side or
server-side software entity which possesses a specific identity
within a Massive Multi-player On-line (MMO) game and is capable of
fully autonomous action within the virtual gaming world. In one
embodiment, the Artificial Player Character is an artificially
intelligent (AI) player character capable of simulating learned
behavior and of propagating itself through the creation of distinct
entities which share aspects of its core elements.
[0045] When a plurality of players play a Massive Multi-player
On-line game, they encounter two types of characters, player
characters and non-player characters. Player characters are
controlled by humans and therefore possess a wide variety of
motivations and objectives, while non-player characters possess
only response-based motivations and static objectives assigned to
each individual non-player character by the programmers of the MMO
game. A process for creating proactive motivations and dynamic
objectives for non-player characters allows non-player characters
to independently determine their immediate and long-term objectives
based on unique preferences derived from their virtual personality,
then act proactively on the basis of those objectives for the
duration of their existence within the MMO game environment.
[0046] FIG. 3 is a diagram of a neural net illustrating one method
of AI-based decision-making for an action of an APC. For example,
from inputs "X" and "Y", filters F1, F2, and F3 produce an APC
Action 10. In a game environment, an APC frequently faces multiple
simultaneous inputs, wherein, for example, X represents the
opportunity of the APC to have a conversation with a nearby
blacksmith NPC, Y represents the proximity of an auction house, and
Z represents the state of the player-character's armor. The filters
are values standing for the APC's unique preferences, wherein, for
example, F1 represents the APC's aggressiveness, F2 represents its
spending preference, and F3 represents its racial tolerance. Facing
the same inputs, in one embodiment, an APC with an F1 higher than
F2 and F3 will elect to talk to the blacksmith to get its armor
repaired, whereas an APC with a high F3 will not talk to the
blacksmith, if the blacksmith is of a different racial type than
the APC.
[0047] In one embodiment, an APC is comprised of a set of game or
character statistics for the APC, a set of graphical images for the
APC and a dynamic list of "possessions" of the APC which may be
utilized within the game, and a set of two or more
probability-related values used for the purpose of determining a
personality trait and the corresponding behavior or activity of the
APC. The APC may also possess a database, such as an experience
database, containing the record of historical encounters and the
results of those encounters (see, e.g., FIG. 10).
[0048] The set of two or more probability-related values used for
the purpose of determining the activity of the APC are referred to
as Preference Filters. A Preference Filter is defined as the
APC-specific probability-based preferences related to a specific
group of potential game interactions or encounters. One example of
a Preference Filter, such as Preference Filters 20 of FIGS. 4 and
5, includes a relevance or Filter ID 22 paired with a variable
integer or Filter Value 24 and related to a specified behavioral
trait. In this example, the relevance or Filter ID 22 and the
corresponding variable integer or Filter Value 24 may be used by
the game designer to determine how relevant the Preference Filter
20 is to any potential game interaction or encounter of the APC.
The variable integer can be of any value, and in one exemplary
embodiment will range between one and 99. Accordingly, in an MMO
game making use of twenty Preference Filters, a range of variable
integers from one to 99 would allow for a near infinite (20 to the
99th power, specifically) variety of APCs with uniquely individual
personality traits and corresponding behaviors or activities.
Examples of Preference Filters for an APC are described below with
reference to FIG. 9.
[0049] FIGS. 4 and 5 illustrate two possible models of an
Artificial Player Character. The Artificial Player Character model
of FIG. 4 provides one embodiment of a model of an Artificial
Player Character 30 without a learning capacity, and the Artificial
Player Character model of FIG. 5 provides one embodiment of a model
of an Artificial Player Character 30' with a learning capacity.
[0050] As illustrated in FIG. 4, one embodiment of a model of an
Artificial Player Character 30 contains the basic elements of
Preference Filters 20, Character Statistics 40, and Graphics and
Possessions 50. In one embodiment, Preference Filters 20 include
Filter IDs 22 and corresponding Filter Values 24, Character
Statistics 40 include statistics 42 and corresponding values 44,
and Graphics and Possessions 50 include items 52, and corresponding
attributes 54 and values 56. As illustrated in FIG. 5, in addition
to the basic elements of Preference Filters 20, Character
Statistics 40, and Graphics and Possessions 50, another embodiment
of a model of an Artificial Player Character 30' contains the
additional element of an Experience Database 60. In one embodiment,
Experience Database 60 includes Experience IDs 62, and
corresponding descriptions 64 of the experiences and average values
66 of the experiences. The Experience Database, as described below,
gives the APC a capacity for learned behavior based on past
encounters (i.e., a simulated learning capacity).
[0051] Character Statistics, such as Character Statistics 40 of
FIGS. 4 and 5, describe the APC's capabilities within the game
environment, and may include attributes such as height, weight,
experience level, strength, speed, intelligence, charisma, stamina,
constitution, attack power, defense rating, hit points, experience
points, magical mana and so forth of the APC. This allows for a
wide range of unique capabilities from one APC to the next as well
as a measure to compare the relative abilities of one APC to
another.
[0052] While Character Statistics describe what an APC can do,
Graphics and Possessions, such as Graphics and Possessions 50 of
FIGS. 4 and 5, indicate what an APC looks like and what an APC
owns. Graphics include a visual representation of the APC as it is
viewed within the game environment, while Possessions may include
money as well as the various weapons, armor, trinkets, consumables,
quest materials, and other items that may be of use or economic
value to the APC.
[0053] In one embodiment, Possessions contain both Attributes and
dynamic Values, such as Attributes 54 and Values 56 of FIGS. 4 and
5. A specific possession, such as a shield, for example, may
possess several Attributes such as increasing various Character
Statistics or conveying other benefits, and may also possess both a
monetary value as well as a use value to the specific APC. For
example, a shield possessed by an APC that is of the wrong class to
make use of it may possess zero use value, but may not be valueless
to the APC due to its monetary value.
[0054] In one embodiment, both Character Statistics and Possessions
are individually tracked and monitored, and taken in combination
with various environmental and inter-player encounters, represent
potential Inputs for the APC in certain situations. For example, an
encounter with an armor vendor may represent a potential Input for
an APC if the vendor is (a) selling the type of armor worn by the
APC, (b) is selling armor that would increase the APC's Character
Statistics (in this case, Defense Rating), and (c) selling the
armor for less money than the APC possesses.
[0055] FIG. 6 illustrates one way in which the APC makes a decision
regarding an encounter or interaction presented by a Player
Character, an NPC, another APC, a Monster Character or the
environment of the game world. For example, after the presented
encounter or interaction, as illustrated at 102, applicable
preference filters are determined by relevance, as illustrated at
104. After passing through the relevant Preference Filters (e.g.,
F2, F3, F4), Preference Filter priorities are calculated and
applied, as illustrated at 106. As such, from the Preference Filter
having priority (e.g., F2), an APC action is determined, as
illustrated at 108.
[0056] FIG. 7 illustrates one way in which the APC can determine
which of its Preference Filters are relevant to the decision-making
process and how they are subsequently applied, and FIG. 8
illustrates one way in which the APC can determine which of the
Preference Filters that are applicable to the current decision have
priority in the decision-making process.
[0057] As illustrated in FIG. 7, in one embodiment, when an APC
receives an alert, which can be based on the APCs proximity to
another character or environmental object, a time-based alarm, or a
statistically-based warning, as illustrated at 202, the specific
type of encounter is identified by its Encounter ID Tag, as
illustrated at 204. In one embodiment, checking this Encounter ID
tag against the Experience Database allows the APC to save
computational cycles by overriding the Filter system and
automatically determining an action if an override is relevant, as
illustrated at 206 and 208. If not, the Encounter ID tag will
dictate the specific Filters which are relevant to the situation,
as illustrated at 210 and 212. In one embodiment, for simple
encounters and more predictable action, the relevant filter with
the largest value is selected to determine the APC action, as
illustrated at 214 and 216. In one embodiment, for more complex
encounters, the values of the relevant filters are calculated as
relative probabilities, as illustrated at 218, any relevant modes
from the Experience Database are applied, as illustrated at 220,
and a random number is generated, as illustrated at 222, to select
between the various probabilities to determine the APC action, as
illustrated at 224.
[0058] As an example of the decision-making process of FIG. 7, due
to a fall from a height, for example, an APCs health might drop
below 20 when the APC is not under attack, thus triggering a
warning that the APC is about to die. The Encounter ID tag
indicates that the Experience Database should check for overrides,
and the override indicates that the APC should drink a health
potion if it has one, thus dictating the APC's action. In a more
complicated encounter, if the APC's health drops below 20 due to an
unexpected attack from a powerful enemy nearby, for example, the
Encounter ID's lack of an override may indicate that the relevant
Filters in this Encounter are Risk-Taking, Aggression, and Sanity.
High values for the first two filters cause the APC to recklessly
attack despite the low probability of survival, whereas low values
or a high value for Sanity, except in the case of an unlikely
random probability determination, cause the APC to take action to
increase its chance of survival by running away.
[0059] FIG. 8 illustrates one way in which the APC can determine
which of the Preference Filters that are applicable to the current
decision have priority in the decision-making process. When an
Encounter takes place, as illustrated at 302, the appropriate
Encounter ID tag provides the relevant Filters to the APC, as
illustrated at 304 and 306. The process then ignores all other
Filters and returns the numerical values only for the relevant
Filters in order to determine the APC's action in the situation, as
illustrated at 308 and 310. The result may be computed either by a
straightforward comparison or by converting the values to relative
probabilities and selected by a random probability determination,
which may be further altered by applicable modifiers from the
Experience Database.
[0060] FIG. 9 illustrates one embodiment of a potential set of
Preference Filters 20, their function 26, their relevance or Filter
IDs 22, and their Filter values 24 for an exemplary APC. In one
embodiment, the potential set of Preference Filters 20 include a
loyalty filter, an ambition filter, an exploration filter, an
adventure filter, a risk-taking filter, an aggression filter, a
conversation filter, an attachment filter, a persistence filter, a
trading filter, an anchorage filter, an altruism filter, a sanity
filter, a greed filter, a social filter, a romance filter, a
negotiation filter, an occupation filter, an economy filter, and a
learning filter. This structure of Preference Filters allows for a
vast range of divergent behaviors. For example, an APC with high
values assigned to Preference Filters for Aggression, Attachment,
and Altruism is likely to immediately launch an unprovoked attack
on anything that might be interpreted as posing a threat to an
individual in the group with which it is identified, whereas an APC
with low values assigned to Preference Filters such as Risk-Taking,
Attachment, and Social would not be inclined to join a group in the
first place, nor risk its own safety in defending other members of
it. The disclosed set of Preference Filters is one example of one
potential set of Preference Filters, and may include or encompass
other filter criteria.
[0061] The Filter values 24 for an APC's Preference Filters 20 may
be assigned in various ways. In one embodiment, static values for
Preference Filters of an APC are determined by the game designer or
other individual. In another embodiment, values for Preference
Filters of an APC are assigned by a random process. Assigning
values by a random process would allow for the widest variability
of APC behavior, although some of the behaviors may prove to be
undesirable for the gaming experience.
[0062] In one embodiment, values for Preference Filters of an APC
are assigned by a "learned" process. In the "learned" process, the
APC's Preference Filters are permitted to evolve by means of access
to a database containing the record of historical encounters and
the results of those encounters. In one embodiment, additional
weight is given to Preference Filters whose use is "effective" as
defined by the game designer, and less weight is given to
Preference Filters whose use proves to be "ineffective" as defined
by the game designer.
[0063] In one embodiment, assigning APC Preference Filter Values
for an APC is by an "evolutionary" method wherein two or more APC
databases and Preference Filter sets are combined to produce a
"child" APC with its own distinct Preference Filter values. This
method can produce Artificial Player Characters that evolve into a
more interesting and effective opposition over time instead of
simply respawning as perfect clones of Artificial Player Characters
that were eliminated from the game on a previous occasion. In
addition, this method can produce Artificial Player Characters as
Non-Player Characters that will interact with the Player-Characters
in a more complex and entertaining manner.
[0064] The Preference Filters provide the APC with a simulated and
unique set of motivations that can be applied to every game
encounter, thus providing a wide range of potential variability
within the context of the encounters. Therefore, the game designer
may want to define every variant of encounter in terms of the
Preference Filters to which it is relevant. For example, an
encounter between a Race X class 1 (Dwarf Warrior) and a traveling
Race X class 6 (Dwarf Merchant) may be tagged with specific
relevance IDs for Loyalty, Attachment, Conversation, and Trading,
whereas a subsequent encounter by the same Race X class 1 with a
Race Y class 1 who specializes in warrior training may be tagged
with specific relevance IDs for Ambition and Aggression.
[0065] In one embodiment, when the game queries the APC regarding
its prevailing attitude towards the imminent encounter, (which will
usually be triggered via a basic proximity setting), a first step
is to determine which of the relevance IDs are relevant to the
situation at hand (see, e.g., FIG. 6). In the case of the encounter
described above, for example, which might be titled "Proximity of a
traveling Dwarf Merchant," the specific IDs mentioned above (i.e.,
Loyalty, Attachment, Conversation, and Trading) would be selected.
In addition, the general IDs deemed relevant to all APC-NPC
encounters, in this case perhaps Aggression, Sanity, and Social,
may also be selected.
[0066] In one embodiment, once the relevant Preference Filters have
been identified, the game will then compare the relative strength
of the competing interests based on their assigned values and
calculate a decision regarding the APC's action (see, e.g., FIG.
6). This calculation may include anything from a simple comparison
of the values followed by application of whichever value happens to
be the largest to a complex fuzzy logic application which
introduces a random element into the process and combines the
various values of the Preference Filters to produce a variable
result.
[0067] Using the comparison method for the encounter described
above, and positing, for example, a value of 99 for Conversation
and a value of 33 for all the other relevant Preference Filters,
the game will ascertain that the APC has a propensity to speak with
the traveling Dwarf Merchant and will attempt to strike up a
conversation with him. This APC action may lead to a new encounter,
"Conversation with a Dwarf Merchant," and inspire the game to
engage in a second round of relevance ID checks. In one embodiment,
this second round of relevance ID checks may include an inquiry
into Negotiation as well as an inquiry into the APC's present
material requirements as indicated by the state of its current
possessions.
[0068] FIG. 10 is an example of a database, such as Experience
Database 60, which records the results of an APC encounter,
catalogues the APC encounter according to an encounter ID, and
averages the results for use in evolved APC behavior, and FIG. 11
illustrates one embodiment of updating the database of encounters
for the APC.
[0069] As illustrated in FIG. 10, one embodiment of Experience
Database 60 includes Experience IDs 62, corresponding descriptions
64 of the experiences, a number 63 of such experiences, a previous
result 65 of such experiences, and an average value 66 of such
experiences, and an over/under value 67 for the experience, and an
under action 69 of the APC for the experience. As such, Experience
Database 60 records the results of an APC encounter, catalogues the
APC encounter according to an encounter ID, and averages the
results for use in future APC encounters to simulate learning
behavior, with each encounter type having a specific ID as well as
a coded description.
[0070] In the embodiment of FIG. 10, CILO indicates Combat against
1 opponent of the same level, whereas 13L6 indicates an Invitation
to join a group that contains characters an average of 6 levels
higher than the APC. The # sign indicates the number of such
encounters the APC has already experienced, Previous Result rates
the previous encounter on a positive scale from 0 to 5, and AVG is
the average result for all of the previous encounters of that type.
In the embodiment of FIG. 10, the 425 combat encounters with a
single character (4.2) have turned out much better than the two
previous invitations to join a group with higher level characters
(1.0). O/U represents the Over/Under for the average of previous
encounters that trigger the override when the next encounter takes
place, and U indicates the override-dictated action. Because the
APC does not have complete control over whether an encounter type
will take place or not, the overrides will not prevent additional
information being added to the Experience Database.
[0071] FIG. 11 illustrates one embodiment of the way in which the
Experience Database of the APC's encounters is updated. In one
embodiment, when the encounter is complete, as illustrated at 402,
a scored result for the encounter is assigned a numerical value
according to a predefined metric for the encounter type, as
illustrated at 404. This result is recorded and averaged with the
previous results, as illustrated at 406, and the database is
updated to reflect these modified results, as illustrated at 408.
For example, if an APC encountered a Dragon that was 10 levels
higher, but was unable to flee even though the override indicated
that he should and wound up killing the Dragon, the 5 result might
be enough to change the average encounter score above the O/U
value. The next time the APC encounters a monster that is 10 levels
higher, instead of the Experience Database override telling him to
flee, the APC would consult its preference filters. This thus
allows for dynamic "learned" behavior that varies depending upon
past results.
[0072] The database containing the record of historical encounters
and the results of those encounters is referred to as the APC's
Experience Database. In one embodiment, an experience database,
such as Experience Database 60 (FIG. 10), is maintained for only
those APCs which exhibit "learned" behavior that modifies the
particular application of the Preference Filters over time
according to the results of past applications. The results of each
encounter by the APC are stored in its Experience Database for
future reference by the game in modifying the application of the
Preference Filters or modifying the Preference Filters themselves
in the case of an evolutionary "learned behavior" approach, for the
individual APC or for use in creating more effective future APCs.
In one embodiment, an override trigger capable of overriding the
action dictated by the Preference Filters may be provided.
[0073] In one embodiment, the Experience Database functions by
assigning a value expressing the results of the encounter and
storing it according to the appropriate encounter type. For
example, a combat encounter between a level 12 APC and three level
15 Orcs in which the APC killed two Orcs before being forced to run
away from the third will be given an encounter ID, summarized in
the database as C3L3, and may be assigned a result value of 2
(defeat, hit/damage ratio<0.5) on a hypothetical scale of 0-5.
This results scale, of course, may be set to any level of detail,
for example, from binary to near infinite.
[0074] Accordingly, with this example, each time the APC engages in
a similar encounter, regardless of whether it is a level 2
character fighting three level 5 Farm Pigs or a level 50 character
fighting three level 53 Iron Hellknights, the results would be
recorded and averaged with the results from previous encounters of
the same type. The greater the number of encounters stored in the
Experience Database, the more weight it would be provided as a
modifier applied to the Preference Filters.
[0075] In one embodiment, based on the encounters stored in the
Experience Database, the Preference Filter value itself may be
modified. For example, an APC which regularly records results of 0
(death, hit/damage ratio<0.25) for a specific encounter ID will
eventually "learn" to exert a powerful modifier to avoid engaging
in that encounter type regardless of how high its Preference Filter
value for Risk-taking or Aggression might be. The extent to which
this modifier influences the Preference Filters and, therefore, the
APC's actions may be defined by the game designer.
[0076] FIG. 12 illustrates an example of a specific encounter by an
exemplary APC. If an APC dwarf happened to be engaged in mining
gold and was encountered by a normally hostile orc of a similar
level that did not attack, as illustrated at 502, the APC would be
free to consider its next action. Since the mere presence of a
hostile character would not be enough to trigger an override, the
Preference Filters are queried and consulted, as illustrated at
504, and those determined to be most relevant to the Encounter type
based on the ID tag (i.e., Aggression, Conversation and
Persistence) are returned, as illustrated at 506 and 508. The
competing Filter Values are then calculated, as illustrated at 510.
In one embodiment, the APC's level of Conversation (90), is more
than three times higher than either Aggression or Persistence, thus
leading to a 15 percent probability of a response dictated by
Aggression, a 67 percent probability of a response dictated by
Conversation, and an 18 percent probability of a response dictated
by Persistence, as illustrated at 512. At this point, any
situational mods are applied, as illustrated at 514, and a random
probability determination produces a result of 87, as illustrated
at 516. As this result falls into the top 18 percent, the action of
Persistence is applied such that the APC ignores the orc and
continues mining, as illustrated at 518. The largely positive
results of the encounter are then computed and stored in the APC's
Experience Database, as illustrated at 520.
[0077] FIG. 13 is a process diagram demonstrating one way in which
an APC can make decisions about its actions regarding changes to
its dynamic statistics using a combination of pre-set triggers and
Preference Filters. A character statistic can change for a variety
of positive and negative reasons, from simple time passing to
receiving damage in battle, collecting items of economic value, and
successfully accomplishing a task. When a character statistic
changes, as illustrated at 602, the APC will check to see if there
is a pre-set override attached to that statistical change, as
illustrated at 604. Statistics may change for combat or non-combat
reasons. For example, an APC with an alchemy skill improves that
skill by successfully brewing a potion. If, for example, the
threshold required to learn a new potion deemed sufficiently
important by the game's designers has been crossed, the APC will
take action to go to the nearest Alchemy trainer and learn that new
potion. If the change causes the level to fall below the pre-set
trigger, as illustrated at 606, the process continues, as
illustrated at 608 and described below with reference to FIG. 14,
for example, based on whether the change is for combat or
non-combat reasons. If there is no pre-set threshold, the APC may
or may not take action depending on its unique Preference Filters,
as illustrated at 610, 612, 614, 616, 618, and 620, similar to that
described above with reference to FIG. 12.
[0078] FIG. 14 is a process diagram demonstrating one way to
determine if a change causes the level to fall below a pre-set
trigger. In one embodiment, following a character statistics
change, as illustrated at 702, the process is divided, for example,
into combat and non-combat responses, as illustrated at 704 and
706, because combat situations present unique danger to the APC and
may require a different response. In one embodiment, if an APC is
damaged by an enemy attack to such an extent that its hit points
fall below a pre-set level, as illustrated at 708, the APC will
check to see if it possesses any means of restoring that statistic,
as illustrated at 710, such as a health potion or a restorative
spell. If the APC possesses the means, it will perform the action
that will restore the statistic to the highest possible level, as
illustrated at 712. For example, if the APC has a minor health
potion and a major health potion, it will drink the major one, and
then respond as determined by its Preference Filters. If, on the
other hand, the APC possesses no means to restore the statistic,
the APC, for example, will flee from the attacker, as illustrated
at 714. The process for statistical changes that are
non-combat-related, as illustrated at 706 and 716, is similar,
except that the APC will not flee if it has no means of improving
the statistic, instead it will simply continue with what it was
doing, as illustrated at 718.
[0079] FIG. 15 is a process diagram illustrating one embodiment of
monitoring the formation of multiplayer groups and utilizing APCs
as player-character substitutes in order to form functional
multiplayer groups in the absence of sufficient player characters.
In one embodiment, an APC Manager monitors the formation of
multiplayer groups, as illustrated at 802. In one embodiment, a
player announces formation of a multiplayer group, as illustrated
at 804. If other players join the group within the specified time,
as illustrated at 806, the group embarks on and completes the
multiplayer quest, as illustrated at 808. If, after a
pre-determined amount of time passes, an insufficient number of
players required to complete the group quest join the group, as
illustrated at 810, the APC Manager determines the appropriate APC
types and levels required to complete the group, as illustrated at
812, and assigns the correct number of APCs of a suitable level to
the group, as illustrated at 814, in order to allow one or more
player-characters to engage in group quests regardless of the
number of interested and available human players. Thereafter, the
group, including the APC(s), embarks on and completes the
multiplayer quest, as illustrated at 816.
[0080] FIG. 16 is a process diagram illustrating the current result
of multiplayer group breakdown and comparing this with one
embodiment of monitoring multiplayer group rosters and allowing
multiplayer groups to avoid breakdown due to the loss of one or
more members by substituting Artificial Player Characters when
player characters exit the group, and removing Artificial Player
Characters when player characters subsequently become available. In
one embodiment, an APC Manager monitors the multiplayer groups, as
illustrated at 902. In one embodiment, a group of human players
embark on a multiplayer quest, as illustrated at 904.
Traditionally, if one or more player characters drop out in
mid-quest, as illustrated at 906, the quest cannot be completed and
is abandoned, as illustrated at 908. By monitoring the multiplayer
groups, however, the APC Manager determines the appropriate APC
types and levels required to complete the group, as illustrated at
910, and assigns the APCs to replace the drop-outs, as illustrated
at 912, if one or more player characters drop out in mid-quest.
Accordingly, the quest can be completed with the APC(s) taking the
place of the player character(s) that dropped out.
[0081] In one embodiment, as the quest continues, the APC Manager
monitors PCs looking for a group, as illustrated at 914. If a
player is looking for the type of group already existing and
including one or more APCs, the APC Manager will offer the player
character the choice to join the existing group in mid-quest, as
illustrated at 916. If the player character wishes to join the
existing group, the APC Manager will remove the APC most similar to
the new player character, and replace the APC with the player
character, as illustrated at 918. Accordingly, the quest can be
completed, as illustrated at 920, with the player character taking
the place of the previously assigned APC.
[0082] In one embodiment, the APC Manager allows multiplayer groups
to complete multiplayer quests that would otherwise be abandoned
when members drop out in mid-quest. This is accomplished by
substituting appropriate APCs for the missing player characters
whenever a group member exits the game or the group, thus allowing
the remaining player characters to continue with their quest in the
company of the APC(s). Because the APC Manager actively monitors
PCs looking for groups, it is aware when an individual player
character is in search of a similar group or quest, at which point
the APC Manager will offer the player character the choice to join
the multiplayer group in mid-quest. In one embodiment, if the
player character wishes to join the group, the APC Manager will
remove the APC most similar to the new player character, replacing
it with the player character.
[0083] In addition to simulating intelligent decision-making by the
APC regarding its actions when presented with encounters within the
game, the present invention also provides a means of simulating
more intelligent tactical decision-making with regards to the APC's
use of various items and possessions within the context of those
encounters. For example, in a conventional MMO game, a wounded
monster will usually attempt to heal itself to the full extent of
its capacity regardless of how much time it possesses. With the
more sophisticated possibilities of the present invention, an APC
would have the capability of determining the optimal balance of
health and time required to give it the greatest possibility of
surviving the encounter.
[0084] The present invention enhances the ability of the game
designer to exert influence over the game events, encounters, and
interactions experienced by the participating users. It also
provides the game designer with an effective means of implementing
the continuous evolution of artificial intelligence within a
massive multi-player on-line game and, in one embodiment, offers
the prospect of eliminating the human element entirely.
[0085] The present invention provides a system for the introduction
of autonomous, dynamic artificial player characters which do not
require human direction and are not subject to human control to the
field of massive multi-player on-line games. These Artificial
Player Characters serve as a substitute for human-controlled Player
Characters which are less subject to the negative aspects of human
behavior and decision-making that detract from the game's ethos,
and provide the MMO game designer with an enhanced ability to
create an interesting and enjoyable game experience for the human
players as they encounter the virtual world.
[0086] The autonomous Artificial Player Character of the present
invention is more complex than the automated combat scripts
("bots") which are used in many 3D shooters and is applicable to a
much broader range of human/AI and AI/AI interactions than mere
combat. It is also substantially different from the utilities
illegally used in some MMO games which automate tasks for a Player
Character, because all of the activities performed by the utilities
require human decision-making and initiation.
[0087] The present invention also makes technologically feasible
the possibility of a massive multi-player on-line game which
requires neither multiple players nor on-line access. For example,
with application of the APC to an MMO game, it becomes
theoretically possible to create a dynamic massive multi-player
on-line game without requiring either human players or on-line
access. This has interesting utility in non-gaming applications
wherein it may be considered desirable to have access to an
evolutionary process from inside the process itself.
[0088] One embodiment of the present invention provides a system
for the implementation of an artificial player character in a
massive multi-player on-line game. The system may include a
software entity with a unique, artificially intelligent capacity
for decision-making regarding every form of encounter it
experiences in the virtual game world, including but not limited
to, adventuring, combat, conversation, exploration, economic
activity, questing, and trading. As such, the entity may possess
complete autonomy within the virtual game world. In one embodiment,
this autonomy is subject to the discretion of the game designer or
a human player to whom ownership of the artificial player character
has been granted. In addition, the entity may possess a status
equivalent to human-controlled Player Characters within the virtual
game world.
[0089] One embodiment of the present invention provides a system
for the implementation of learned behavior by artificial player
characters, non-player characters, and monsters in a massive
multi-player on-line game. The system may include a set of values
that dictate the preferred behaviors of an individual artificial
player character, non-player character, or monster, and a database
containing a record of past encounters and the results of those
encounters. The results of the actions dictated by the
implementation of the set of values are stored in the database such
that the interpretation of the historical results are implemented
to modify the set of values for future encounters. In one
embodiment, the database may contain provisions for overriding the
set of values based upon historical results.
[0090] One embodiment of the present invention provides an
automated system for creating artificial player characters,
non-player characters, and monsters for a massive multi-player
on-line game. The system may include the combination of two or more
sets of values that dictate the preferred behaviors of the
specified artificial player character, non-player character, or
monster into a single set of values dictating the preferred
behaviors of a new entity.
[0091] One embodiment of the present invention provides a system
for creating a zero-player off-line version of a massive
multi-player on-line game. The system may include a
three-dimensional graphical virtual environment entirely populated
with artificial player characters which may be visited by
non-participatory human-controlled avatars or a limited number of
participatory human-controlled player characters.
[0092] One embodiment of the present invention provides a process
for monitoring the requirements of individual player characters
attempting to form multiplayer groups and providing Artificial
Player Characters to substitute for player characters when
insufficient player characters are available for group
formation.
[0093] One embodiment of the present invention provides a process
for monitoring the requirements of multiplayer groups and
substituting Artificial Player Characters for player characters as
needed in order to prevent premature dissolution of those groups.
This embodiment also encompasses a system of removing APCs from the
group and replacing them with human-controlled player characters
when interested player characters become available.
[0094] Although the invention herein has been described with
reference to particular embodiments, it is to be understood that
these embodiments are merely illustrative of the principles and
applications of the present invention. It is therefore to be
understood that numerous modifications may be made to the
illustrative embodiments and that other arrangements may be devised
without departing from the spirit and scope of the present
invention as defined by the appended claims.
* * * * *