U.S. patent application number 16/003000 was filed with the patent office on 2019-05-09 for systems and methods for user-based arbitration and peer review for online multiuser systems.
The applicant listed for this patent is RIOT GAMES, INC.. Invention is credited to Christopher Burdick, Tom Cadwell, Steve Mescon.
Application Number | 20190134502 16/003000 |
Document ID | / |
Family ID | 49212319 |
Filed Date | 2019-05-09 |
View All Diagrams
United States Patent
Application |
20190134502 |
Kind Code |
A1 |
Mescon; Steve ; et
al. |
May 9, 2019 |
SYSTEMS AND METHODS FOR USER-BASED ARBITRATION AND PEER REVIEW FOR
ONLINE MULTIUSER SYSTEMS
Abstract
The field of the invention relates to systems and methods for
managing user behavior in online multiuser systems, and more
particularly to systems and methods that provide user-based
arbitration for online multiuser systems. In one embodiment, a
multi-user online system enables a first user to report a second
user's behavior while online. The system receives the report and
merges the report with the second user's recorded interactions with
the online system related to the report to create an arbitration
file. The arbitration file is then made available to a third user
to review the arbitration file and determine whether action is
warranted.
Inventors: |
Mescon; Steve; (Los Angeles,
CA) ; Burdick; Christopher; (Los Angeles, CA)
; Cadwell; Tom; (Los Angeles, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
RIOT GAMES, INC. |
Los Angeles |
CA |
US |
|
|
Family ID: |
49212319 |
Appl. No.: |
16/003000 |
Filed: |
June 7, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14808394 |
Jul 24, 2015 |
10016675 |
|
|
16003000 |
|
|
|
|
13424670 |
Mar 20, 2012 |
9120019 |
|
|
14808394 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A63F 13/798 20140902;
A63F 13/35 20140902; A63F 13/12 20130101; A63F 13/75 20140902; A63F
13/352 20140902; A63F 13/87 20140902; A63F 13/533 20140902 |
International
Class: |
A63F 13/352 20140101
A63F013/352; A63F 13/75 20140101 A63F013/75; A63F 13/533 20140101
A63F013/533; A63F 13/798 20140101 A63F013/798; A63F 13/87 20140101
A63F013/87; A63F 13/35 20140101 A63F013/35 |
Claims
1. An online multiuser game system, comprising: an online game
session server system coupled to a public network for access by a
plurality of users, wherein the online game server includes: a
messaging system; a recording system that records each user's
interaction with the online game server; and a game file database
that stores the recorded interactions, including messaging data, by
one or more game identifications; a statistics server system
communicatively coupled to the online game session server, wherein
the statistics server system includes a statistics server database
and is configured to derive from the game file database messaging
and game interaction data from the recorded interactions to store
in said statistics server database; a report server system
communicatively coupled to the online game server system, wherein
the report server system includes a report server database and a
report engine configured to enable a first user to submit one or
more reports over the public network for one or more users during a
particular game, and further wherein the report engine is further
configured to store said reports in said report server database;
and an arbitration server system communicatively coupled to the
report server system and the statistics server system and is
accessible by the plurality of users via said public network,
wherein the arbitration server system is configured to query the
reporting server database and the statistics server database to
create one or more arbitration files for the one or more reported
users that includes the first user's one or more reports and the
messaging and game interaction data associated with said particular
game.
2. The system of claim 1, wherein the one or more reports indicate
that the one or more users exhibited undesirable behavior during
the one or more users' interaction with the game system during said
particular game according to the first user.
3. The system of claim 2, wherein the arbitration server system is
further configured to enable a plurality of third party users to
submit a vote for the one or more arbitration files indicating
whether to punish or exonerate the one or more users.
4. The system of claim 2, wherein the arbitration server system is
further configured to disable the one or more users' access to the
game system if a punishment vote threshold has been reached for the
one or more users.
5. The system of claim 4, wherein the vote threshold comprises
number of votes total received for the arbitration file and
percentage of votes for punishment.
6. The system of claim 1, wherein the one or more reports indicate
that the one or more users exhibited desirable behavior or skill
during the one or more users' interaction with the game system
during said particular game according to the first user.
7. The system of claim 6, wherein the arbitration server system is
further configured to enable a plurality of third party users to
submit a vote for the one or more arbitration files indicating
whether to reward the one or more users.
8. The system of claim 1, wherein the one or more reports includes
the first user's reporting history, including data that indicates a
ratio of how many reports the first user has submitted compared to
how many games the first user has played.
9. The system of claim 1, wherein the messaging system includes
text messaging.
10. The system of claim 1, wherein the messaging system includes
online chat.
11. The system of claim 1, wherein the arbitration server is
further configured to receive multiple reports for a user where
each report is submitted by a different user for a particular
game.
12. A method for managing user behavior for a multi-user online
game system comprising: recording each user's interaction with the
online game system, including messages exchanged between users;
storing said recordings in a game file database; enabling a first
user to submit at least one report for at least one second user
that displayed certain behavior during the at least one second
user's interaction with the online game server during a particular
game; presenting the first user's at least one report with the
recordings for the particular game to an arbitrating user; and
enabling a plurality of arbitrating users to review the
presentation of the first user's at least one report and to submit
a vote related to confirming the first user's at least one
report.
13. The method of claim 12, wherein the at least one report alleges
that the at least one second user exhibited undesirable behavior
during the at least one second user's interaction with the game
system during said particular game.
14. The method of claim 13, further comprising the step of enabling
the plurality of arbitrating users to submit a vote indicating
whether to punish or exonerate the at least one second user.
15. The method of claim 13, further comprising the step of
disabling the at least one second user's access to the game system
if a punishment vote threshold has been reached for the one or more
users.
16. The method of claim 15, wherein the vote threshold comprises
number of votes total received for the presentation and percentage
of votes for punishment.
17. The method of claim 12, wherein the at least one report alleges
that the at least one second user exhibited desirable behavior or
skill during the at least one second user's interaction with the
game system during said particular game.
18. The method of claim 17, further comprising the step of enabling
the plurality of arbitrating users to submit a vote for the
presentation indicating whether to reward the at least one second
user.
19. The method of claim 12, wherein the at least one report
includes the first user's reporting history, including data that
indicates a ratio of how many reports the first user has submitted
compared to how many games the first user has played.
20. The method of claim 12, wherein the at least second user's vote
count is reduced if no other reports are submitted for that at
least one second user during a predetermined period of time after
the first user's report is submitted.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. patent
application Ser. No. 14/808,394, filed Jul. 24, 2015, which is a
continuation of U.S. patent application Ser. No. 13/424,670, filed
Mar. 20, 2012, now U.S. Pat. No. 9,120,019, both of which are
hereby incorporated by reference in their entireties herein for all
purposes.
FIELD OF THE INVENTION
[0002] The field of the invention relates to systems and methods
for managing user behavior in online multiuser systems, and more
particularly to systems and methods that provide user-based
arbitration to enable peer review for online multiuser systems.
BACKGROUND OF THE INVENTION
[0003] Controlling inappropriate behavior in public online
multiuser systems is challenging, particularly when these systems
have millions of user accounts, such as popular multiplayer online
games, e.g., League of Legends from Riot Games
(www.leagueoflegends.com). These games are generally low cost to
participate in and users can register anonymously. Moreover, these
games can be very competitive, thereby enabling aggressive and
undesirable behavior, such as cheating or inappropriate
communications or any unacceptable behavior as determined by a
player's peers. For example, during a game, an aggressive user may
send text messages to other users with profanity, insults, racial
slurs, and/or threats of violence. In a system with millions of
users, this behavior may by exhibited by hundreds or thousands of
participants if unchecked, and having the game administrator police
every user and/or incident is highly burdensome and expensive.
Moreover, it may be desirable to reward positive behavior or
exceptional skill in such systems. Accordingly, an improved system
for encouraging and discouraging certain behavior and game play
would be desirable.
SUMMARY OF THE INVENTION
[0004] The field of the invention relates to systems and methods
for managing user behavior in online multiuser systems, and more
particularly to systems and methods that provide user-based peer
review for game play in online multiuser systems.
[0005] In one embodiment, a multi-user online system enables a
first user to report one or more users' behavior, whether
undesirable or exceptional, while interacting with the online
system. The system receives the report and creates one or more
arbitration files that merge the one or more reports with the one
or more users' recorded interactions with the online system. The
arbitration file is then made available to arbitrating users to
review the one or more arbitration files and determine whether
punishment and/or reward is warranted, thereby enabling
crowd-sourced peer review.
[0006] Other systems, methods, features and advantages of the
invention will be or will become apparent to one with skill in the
art upon examination of the following figures and detailed
description. It is intended that all such additional systems,
methods, features and advantages be included within this
description, be within the scope of the invention, and be protected
by the accompanying claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] In order to better appreciate how the above-recited and
other advantages and objects of the inventions are obtained, a more
particular description of the embodiments briefly described above
will be rendered by reference to specific embodiments thereof,
which are illustrated in the accompanying drawings. It should be
noted that the components in the figures are not necessarily to
scale, emphasis instead being placed upon illustrating the
principles of the invention. Moreover, in the figures, like
reference numerals designate corresponding parts throughout the
different views. However, like parts do not always have like
reference numerals. Moreover, all illustrations are intended to
convey concepts, where relative sizes, shapes and other detailed
attributes may be illustrated schematically rather than literally
or precisely.
[0008] FIG. 1 is an exemplary diagram of a multiuser online game
system in accordance with a preferred embodiment of the present
invention;
[0009] FIG. 2a is an exemplary diagram of a game session server
system in accordance with a preferred embodiment of the present
invention;
[0010] FIG. 2b is an exemplary diagram of a game client user
interface in accordance with a preferred embodiment of the present
invention;
[0011] FIG. 3 is an exemplary diagram of a statistics server system
in accordance with a preferred embodiment of the present
invention;
[0012] FIG. 4a is an exemplary diagram of a report server system in
accordance with a preferred embodiment of the present
invention;
[0013] FIG. 4b is an exemplary diagram of another game client user
interface related to the report server system in accordance with a
preferred embodiment of the present invention;
[0014] FIG. 4c is an exemplary diagram of another game client user
interface related to the report server system in accordance with a
preferred embodiment of the present invention;
[0015] FIG. 5a is an exemplary diagram of an arbitration server
system in accordance with a preferred embodiment of the present
invention;
[0016] FIG. 5b is an exemplary diagram of an arbitration client
user interface related to the arbitration server system in
accordance with a preferred embodiment of the present
invention;
[0017] FIG. 5c is an exemplary diagram of another arbitration
client user interface related to the arbitration server system in
accordance with a preferred embodiment of the present invention;
and
[0018] FIG. 6 is an exemplary process in accordance with a
preferred embodiment of the present invention.
[0019] FIG. 7 is another exemplary process in accordance with a
preferred embodiment of the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
Preferred Systems
[0020] Turning to FIG. 1, a large multiuser online game system 1000
over a public network 1050, such as the Internet, is shown. An
example of such a game system 1000 is League of Legends
(www.leagueoflegends.com). League of Legends is a session-based,
multiplayer online battle-arena game where rival teams compete
against one another for victory on highly stylized battlefields and
landscapes. Users can install a League of Legends game client on
their personal computing device 1100 to establish a game session
over the public network 1050 with the game system's 1000 datacenter
1300, which provides the online game interaction with the plurality
of users 1100. The datacenter 1300 includes a plurality of server
systems operating on a plurality of server machines communicatively
coupled to each other via the public network 1050 and/or a secure
virtual private network (not shown). The server machines each
include a processor, memory, an operating system, an input/output
interface and network interface all known in the art. In accordance
with a preferred embodiment, the datacenter 1300 includes a game
session server system 1500, a local server manager system 1600, a
statistics server system 1700, a reporting server system 1800, and
an arbitration server system 1800. Each of these server systems
will be described in more detail below
[0021] Game Session Server System 1500
[0022] Turning to FIG. 2a, a more detailed diagram of a game
session server system 1500 is shown. The game session server system
1500 provides the game interaction with the users' game client 1100
via the game client interface 1530, which is generally an
application interface known in the art accessible over the public
network 1050 by the game client 1100, e.g., in a traditional client
server model. A game engine 1520 coupled to the game client
interface 1530 is included to manage the interaction between the
plurality of users 1100 and the game system 1000. The game session
server system 1500 further includes a chat engine 1540 known in the
art that enables the various users 1100 participating in a
particular game session to communicate with each other via text
messages. Audio, pictures, and multimedia may also be exchanged
with the chat engine 1540. Both the game engine 1520 interactions
as well as the chat messages exchanged can be recorded and stored
in a game files database 1510. This enables, among other things,
replay and history analysis by not only the users but also the
administrator and other systems as will be described below. The
game session server system 1500 further includes a local server
manager interface 1550 for communications with the local server
manager 1600.
[0023] Turning to FIG. 2b, an example game client 1100 user
interface is shown. In online games such as League of Legends, each
user is generally represented by a personalized graphical avatar in
the user interface, (shown as "X" in this example), and the game
client 1100 user interface may show the logical position of one
user's avatar, X User 1, relative to another, X User 2. The game
client 1100 user interface may also include a chat interface ("Chat
Room") that enables participating users to communicate with one
another, via chat engine 1540, beyond interactions with the avatars
(Xs).
[0024] A game session server system 1500 generally includes
physical servers distributed geographically in the same areas as
its users. For example, users in the Southern California region may
connect to a game session server system 1600 located in Los Angeles
to initiate game sessions to ensure optimum performance.
[0025] Local Server Manager System 1600
[0026] The local server manager system 1600, also shown in FIG. 2a,
also includes a plurality of distributed physical servers and
controls the interactions with users' game clients 1100 before and
after a game session. For example, users connect to a local server
manager system 1600 to initiate game sessions via the game session
server 1500. Upon the completion of a session-based game between
multiple users, the game session server 1500 directs the users to
their respective local server manager system 1600. Further, the
local server manager system 1600 will query the data associated
with that game from the game session server's game file database
1510 via the game session server's local server manager interface
1550.
[0027] Statistics Server System 1700
[0028] Turning to FIG. 3, a more detailed diagram of the statistics
server system 1700 is shown. The statistics server system 1700
generally receives data from the game files database 1510 delivered
by a local server manager 1600 via a local server manager interface
1740. The statistics server system 1700 includes a statistics
server engine 1730 that receives the game files data from the local
server manager 1600 and generates in-game statistics and metrics
for storage in its own database 1710. The in-game statistics may
include statistics for a particular player and/or a particular
game, and may indicate a particular behavior pattern or skill
level. For example, in League of Legends, where part of the game
playing involves defeating other user avatars or computer
controlled minions, statistics may include number of defeats over
avatars, number of defeats over computer controlled minions, number
of game play objectives completed, etc. . . . Such data can be
generated and filtered by the statistics server engine 1730 and the
results are stored in the statistics server database 1710. The
statistics server engine 1730 further collects from the game files
database 1510 the chat logs, which are also stored in the
statistics server database 1710. The statistics server system 1700
further includes an arbitration server interface 1720, which is
also generally an application interface known in the art for access
by the arbitration server 1800.
[0029] Reporting Server System 1800
[0030] Turning to FIG. 4a, a more detailed diagram of a report
server system 1800 is shown. Because a large number of users can
access and participate on the system 1000 anonymously, undesirable
behavior may occur including any unacceptable behavior as
determined by a player's peers. For example, one or more offending
users in a game session may cheat or may send offensive chat
messages, using profanity and/or racial slurs. On the other hand,
certain users may exhibit exceptional skill and/or extraordinary
behavior as determined by a player's peers. For example, a user may
demonstrate exceptional game play or selfless-team oriented game
play with encouraging and friendly communications. The game system
1000 provides a report server system 1800 that enables a second
user to memorialize and report such behavior, whether desirable or
undesirable, or skill to a third party, such as an administrator or
other users.
[0031] In the case where a second user wishes to report undesirable
behavior, the second user can use a reporting user interface
included in the game client 1100, an example of which is shown in
FIG. 4b. Upon completion of a game where the one or more offending
users displayed undesirable behavior, the second user, with the
user interface shown in FIG. 4b, may identify the one or more
offending users, specify the undesirable behavior displayed, and
add additional comments. The report server system 1800 receives the
second user's one or more reports via the game client interface
1830, which is also generally an application interface such as that
described above.
[0032] In the case where a second user wishes to report desirable
and/or exceptional behavior or skill, the second user can use a
reporting user interface 1150 included in the game client 1100, an
example of which is shown in FIG. 4c. Upon completion of a game
where the one or more exceptional users displayed desirable
behavior or skill, the second user, with the user interface shown
in FIG. 4c, may identify the one or more exceptional users, specify
the behavior or skill displayed, add a rating, for example, a score
from 1-5, and add additional comments. The report server system
1800 receives the second user's one or more reports via the game
client interface 1830, which is also generally an application
interface such as that described above.
[0033] The report server system 1800 includes a reporting engine
1820 component that is configured to receive reports submitted by
users, via the game client interface 1830, and store them in a
report archive database 1810, which can be a relational-type
database, such as MySQL. Preferably, a coherence cache is used,
which is a cache coupled to the database. The cache is in-memory,
which enables faster querying of the database in real time, thereby
improving scalability. From this report archive database 1810, not
only can a specific report be derived, but also the identification
of particular users that have been most frequently reported (e.g.,
top accused offenders or top rated players).
[0034] Moreover, in the case of undesirable behavior, one or more
"harassment scores" may be derived based on the frequency and type
of report received. Further, the timing of the reports submitted
may also be stored. This can indicate whether reports have stopped
for a particular accused offender, thereby indicating improved
behavior. This can be a factor in an algorithm that assesses the
"harassment score" for a particular user. Also provided is the
identification of users that have most frequently submitted reports
(i.e., "snitch score"). If a user has been reported by another user
having a high snitch score, such data may be used to affect the
harassment score as will be described in further detail below. In
the case of exceptional behavior, a positive player rating may be
established based on the frequency of positive reports. Moreover,
if a positive report is from a user that indiscriminately submits
positive reports for a large percentage of the game, such data may
be used to affect player rating.
[0035] The report server system 1800 further includes an interface
1850 with the arbitration server system 1800, which is also
generally an application interface known in the art.
[0036] Arbitration Server System 1900
[0037] Turning to FIG. 5a, a more detailed diagram of an
arbitration server system 1900 is shown. As mentioned above, it is
a challenge to manage behavior in large multiuser online game
systems. To address this issue, the game system 1000 in accordance
with a preferred embodiment of the present invention generally
provides for certain users of the game system 1000 to provide peer
review of the merits of other users' reports submitted to the
reporting server system 1800. To that end, the game system 1000
includes an arbitration server system 1900 having an arbitration
engine 1920 that generates case files to be presented to a user
that wishes to arbitrate a particular report. The arbitration
engine 1920 queries a report from the reporting server system 1800,
via a report server interface 1960 (also an application interface
known in the art), and generates a case file that merges the
queried report with data from the particular game at issue. This
data is retrieved from the statistics server system 1700 via the
statistics server interface 1950 using the identification of the
particular game at issue. This case file is presented to the
arbitrating client's 1200 user interface, such as that shown in
FIGS. 5b and/or 5c.
[0038] In the case of arbitrating undesirable behavior, as shown in
FIG. 5b, the arbitrating user's client 1200 is presented with "game
details," e.g., the results of the game, such as damage dealt and
damage taken. In the case of League of Legends, also included is
performance score, "gold earned," and "healing done" as examples.
The user interface 1200 also identifies "report details," such as
the player(s) who submitted reports related to the game-at-issue,
the alleged reason for the report submission(s), such as verbal
harassment, and other related statistics relevant to the
allegations. If the reporting user also submits comments, such
comments will also be displayed as well as the actual
communications between players, which is typically where many of
the complaints result from. Moreover, a "credibility grade" or
score, such as the snitch score described above, may be associated
and displayed with the reporting user. For example, if the
reporting user submits a report for a high percentage of the games
played, e.g., over 50%, then a low credibility grade (or high
snitch score) is provided, e.g., a "C." This may be stored and
displayed in the reporting details for an arbitrating user to
consider when voting. Moreover, the system 1000 may incorporate
this statistic to reduce the "harassment score" for a particular
offending user and/or punishment rendered.
[0039] From all of this data, the arbitrating user then has the
option to either vote for punishment, vote for exoneration ("Don't
Punish"), skip vote, or end arbitration ("Done for now"). Other
options include more "soft" votes, e.g., the arbitrating user can
also vote as to whether such behavior is simply unacceptable
without rendering a punishment vote. Moreover, the arbitrating user
can also vote as to type of punishment, e.g., whether to simply
issue a warning, disable certain features, issue a temporary ban,
or issue a permanent ban from the game system 1000. To prevent
users from abusing the arbitration system, the arbitrating server
1900 can also specify the amount of time allotted in between case
files for review ("Time before next report"). Also included are the
voting statistics ("46"). After a user submits a vote, the results
are stored in a case file database 1920, which can be a relational
database such as MySQL. These results can then enable an
administrator and system 1000 to verify the "harassment score" in
the report server system 1800 and issue certain punishments for the
particularly high offenders, such as issuing a warning or
temporarily (or permanently) disabling offending users' access to
the system 1000. In yet another alternative, for games that allow
users to gain powers, weapons, and other features that can enable
users to gain significant advantages over other users in the game,
such powers, weapons, and other features can be disabled or
reduced. The process for determining what punishment to issue can
optionally include relevant voting statistics described above, if
available, for example vote as to type of punishment.
[0040] In the case of reviewing exceptional behavior or skill, as
shown in FIG. 5c, like with undesirable behavior, the arbitrating
user's interface 1250 is presented with "game details," e.g., the
results of the game, such as damage dealt and damage taken. In the
case of League of Legends, also included is performance score,
"gold earned," and "healing done" as examples. The user interface
1250 also identifies "report details," such as the player(s) who
submitted reports related to the game-at-issue, the alleged reason
for the report submission(s), such as selfless play and exceptional
performance and other related statistics relevant to the
allegations. If the reporting user also submits comments, such
comments will also be displayed as well as the actual
communications between players. Moreover, a "credibility grade" or
score, such as the snitch score described above, may be associated
and displayed with the reporting user. For example, if the
reporting user indiscriminately submits a report for a high
percentage of the games played, then a low credibility grade is
provided, e.g., a "C." This may be stored and displayed in the
reporting details for an arbitrating user to consider when voting.
Moreover, the system 1000 may incorporate this statistic to reduce
the "performance rating" for a particular user and/or reward
issued.
[0041] From all of this data, the arbitrating user then has the
option to either vote for issuing a reward, vote for no reward,
skip vote, or end arbitration ("Done for now"). Other options
include more "soft" votes, e.g., the arbitrating user can also vote
as to whether such behavior or skill is indeed exceptional without
rendering a reward vote. Moreover, the arbitrating user can also
vote as to type of reward, e.g., whether to simply issue a
complement, enable certain features, or issue grants of game play
free of charge. To prevent users from abusing the arbitration
system, the arbitrating server 1900 can also specify the amount of
time allotted in between case files for review ("Time before next
report"). Also included are the voting statistics ("46"). After a
user submits a vote, the results are stored in a case file database
1920, which can be a relational database such as MySQL. These
results can then enable an administrator and system 1000 to verify
the "performance rating" in the report server system 1800 and issue
certain rewards for the particularly exceptional players, such as
issuing a certificate or adding special game play features
exclusively for the players who exhibit exceptional skill or
behavior The process for determining what reward to issue can
optionally include relevant voting statistics described above, if
available, for example vote as to type of reward.
[0042] Arbitrating users are generally more experienced users
within the system 1000. The system 1000 can establish a
qualification threshold (e.g., an objective skill level or game
level) that a user would be required to achieve in order to connect
to the arbitration server 1900 (in the case of League of Legends,
an arbitrating user must be a "level 30 summoner"). Further, they
can be rewarded for participating in the arbitration process, with
increased levels, powers or features such as those described above,
particularly if they consistently vote with the majority. However,
preferably, there are parameters that are set to avoid abuse and
potential conflicts of interest. For example, an arbitrating user
would be precluded from obtaining a case file that they initiated
via the report server system 1800 or that they are the subject of
in such a report. Also, because different geographical regions and
cultures may have different standards for what is considered to
offensive (or exceptional) behavior, such arbitrating users may be
organized accordingly. Moreover, if an arbitrating user
consistently fails to vote with the majority, the credibility of
the arbitrator's vote may be questioned. In such a case, such an
arbitrating user's vote can be given less weight in calculating the
harassment score for a particular offending user or performance
ratings for a particular exceptional player.
[0043] In a preferred system 1000, these parameters are stored in
the arbitration server database 1910 and can be modified by the
administrator via the administrator client interface 2000. These
parameters as well as others to assess punishment may include:
[0044] disablement/enablement of the arbitration server system
1900; [0045] number of votes required for a case to be flagged for
an administrator or system 1000 take action, such as punishment or
rewards; [0046] the threshold (e.g, percent-based) for a case to be
considered for punishment/rewards (e.g., 51% of votes for
punishment/rewards would be required for actual punishment/rewards
consideration); [0047] maximum "pool size" of cases that will be
loaded into the queue evaluation by arbitrating users; [0048]
threshold at which arbitration engine 1920 will fill in the queue
case files for arbitrating users to review; [0049] interval at
which the arbitration engine 1920 process will run (e.g., 5 mins.);
[0050] interval at which arbitrating user must wait to vote when
presented with a case file (e.g., 60 seconds) to ensure thorough
review; [0051] identification of certain cases for the
administrator to manually enter into the queue for arbitrating
users to review; [0052] identification of cases that have reached
verdict and require action such as punishment or exoneration or
rewards; [0053] list of current cases awaiting verdict and voting
statistics; [0054] voting and behavior history of users and
arbitrating users; [0055] the number of allowed votes per day per
user; [0056] region allocation for arbitrating users; [0057]
qualifications for arbitrating users, e.g., certain level or number
of games played; [0058] archived cases [0059] anti-collusion
features that prevents arbitrating users from using the arbitration
server 1900 to inappropriately gain awards.
[0060] Other data that can be stored in the database 1910 include:
[0061] rewards issued to users; [0062] history of rewards program;
[0063] punishment history of particular users; [0064] last
punishment and decay (e.g., if a user has not been punished for a
long time, that may indicate improved behavior; [0065] voting
history of arbitrating users, including accuracy; [0066] voting
history for particular cases, such as total votes, date voting
started and date voting ended and verdict.
Preferred Processes
[0067] In the case of undesirable behavior, turning to FIG. 6, a
reporting and arbitration process 3000 is shown in accordance with
a preferred embodiment of the present invention to provide peer
review. After the completion of a game, game data including
recordings of game interactions and chat log are stored in the game
files database 1510 of the game session server 1500 (Action Block
3010). If a second user witnesses undesirable behavior by one or
more offending users, the second user can submit one or more
reports to the report server system 1800 via its client interface
1100, such as that shown in FIG. 4b over the public network 1050
(Action Block 3020). If a third user wishes to arbitrate case files
(Decision Block 3030), then the third user ("arbitrating user") can
make a request via an arbitration client 1200 over the public
network 1050 to the arbitration server system 1900. In response,
the arbitration engine 1920 will create a case or arbitration file
by querying a report from the report server system 1800 (Action
Block 3040). Determining which report will be queried may depend on
a variety of factors and/or algorithms. For example, the report
queried may be random or prioritized based on type of undesirable
behavior reported and/or harassment score of a particular user
(e.g., top offender).
[0068] Once a report is returned to the arbitration server system
1900, the arbitration engine 1920 will identify the game and user
associated with that report ("game ID" and "user ID",
respectively). The arbitration engine 1920 will use the game ID to
query the game data associated with that game from the statistics
server system 1700 database 1710 (Action Block 3050), including
game details and chat data. This data will be merged with the
report and transmitted to the third user's arbitration client 1200
as a case file (Action Block 3060). The User ID may further be used
to obtain game data associated with that allegedly offending user
from other games within the statistics server system 1700 database
1710. If the arbitrating user submits a vote (Decision Block 3070),
then the arbitration system 1900 assesses whether a vote threshold
has been met (Decision Block 3080). For instance, an assessment is
made as to whether a certain minimum number of votes have been cast
and whether the majority ruled in favor of punishment (or
exoneration). For example, if a certain percentage of votes (e.g.,
51%) for punishment are received for a particular case file that
received a certain number of total votes (e.g., 100), then
appropriate action may be taken (Action Block 3090), e.g.,
automatic punishment by the arbitration server 1900. Generally,
punishment can be issued manually by the system's 1000
administrator or automatically by the system 1000 itself, e.g., via
the arbitration server 1900. For example, the arbitration server
1900 can issue a warning or notification (e.g., via email) or
disable temporary or permanent access to the system 1000 or disable
or reduce power levels and features. Moreover, the arbitration
server 1900 can notify the offending user of the reports submitted,
vote status, and ultimate punishment if any. Subsequently, the vote
data is then stored with the report in the report server database
1810 and/or arbitration server database 1910 (Action Block 3100)
for review and action by the administrator and/or system 1000.
[0069] For exceptional and desirable behavior or skill, turning to
FIG. 7, a reporting and arbitration process 4000 to enable peer
review is shown in accordance with a preferred embodiment of the
present invention. After the completion of a game, game data
including recordings of game interactions and chat log are stored
in the game files database 1510 of the game session server 1500
(Action Block 4010). If a second user witnesses exceptional skill
or positive behavior by one or more users, the second user can
submit a report to the report server system 1800 via its client
interface 1150, such as that shown in FIG. 4c over the public
network 1050 to provide positive feedback (Action Block 4020). If a
third user wishes to review a positive behavior or skill case files
(Decision Block 4030), then the third user ("arbitrating user") can
make a request via an arbitration client 1250 over the public
network 1050 to the arbitration server system 1900. In response,
the arbitration engine 1920 will create a case or arbitration file
by querying a report from the report server system 1800 (Action
Block 4040). Determining which report will be queried may depend on
a variety of factors and/or algorithms. For example, the report
queried may be random or prioritized based on type of desirable
and/or positive behavior or skill reported and/or performance score
of a particular user (e.g., top performer).
[0070] Once a report is returned to the arbitration server system
1900, the arbitration engine 1920 will identify the game and user
associated with that report ("game ID" and "user ID",
respectively). The arbitration engine 1920 will use the game ID to
query the game data associated with that game from the statistics
server system 1700 database 1710 (Action Block 4050), including
game details and chat data. This data will be merged with the
report and transmitted to the third user's arbitration client 1200
as a case file (Action Block 4060). The User ID may further be used
to obtain game data associated with that exceptional user from
other games within the statistics server system 1700 database 1710.
If the arbitrating user submits a vote (Decision Block 4070), then
the arbitration system 1900 assesses whether a vote threshold has
been met (Decision Block 4080). For instance, an assessment is made
as to whether a certain minimum number of votes have been cast and
whether the majority ruled in favor of reward. For example, if a
certain percentage of votes (e.g., 51%) for rewards are received
for a particular case file that received a certain number of total
votes (e.g., 100), then appropriate action may be taken (Action
Block 4090), e.g., automatic issuance of rewards by the arbitration
server 1900. Generally, rewards can be issued manually by the
system's 1000 administrator or automatically by the system 1000
itself, e.g., via the arbitration server 1900. For example, the
arbitration server 1900 can issue a notification (e.g., via email)
or increase power levels and features. Moreover, the arbitration
server 1900 can notify the exceptional user of the reports
submitted, vote status, and ultimate reward if any to provide that
positive feedback that encourages continued positive behavior and
experience. Subsequently, the vote data is then stored with the
report in the report server database 1810 and/or arbitration server
database 1910 (Action Block 4100) for review and action by the
administrator and/or system 1000. These processes effectively
enable the game system 1000 to leverage participation by the
system's 1000 users to discourage undesirable behavior and
encourage exceptional behavior and game play to ensure a desirable
experience for the rest of the users.
[0071] In the foregoing specification, the invention has been
described with reference to specific embodiments thereof. It will,
however, be evident that various modifications and changes may be
made thereto without departing from the broader spirit and scope of
the invention. For example, the reader is to understand that the
specific ordering and combination of process actions described
herein is merely illustrative, and the invention may appropriately
be performed using different or additional process actions, or a
different combination or ordering of process actions. For example,
this invention is particularly suited for interpersonal
relationships; however, the invention can be used for any
relationship in general. Additionally and obviously, features may
be added or subtracted as desired. Accordingly, the invention is
not to be restricted except in light of the attached claims and
their equivalents.
* * * * *