U.S. patent application number 17/348562 was filed with the patent office on 2021-10-07 for systems and methods for establishing direct communication between a server system and a video game controller.
The applicant listed for this patent is Sony Interactive Entertainment LLC. Invention is credited to Roelof Roderick Colenbrander.
Application Number | 20210308561 17/348562 |
Document ID | / |
Family ID | 1000005655244 |
Filed Date | 2021-10-07 |
United States Patent
Application |
20210308561 |
Kind Code |
A1 |
Colenbrander; Roelof
Roderick |
October 7, 2021 |
Systems and Methods for Establishing Direct Communication Between a
Server System and a Video Game Controller
Abstract
Systems and methods for establishing direct communication
between a server system and a video game controller are described.
The systems and methods include the video game controller and a
computing device. An application is executed by the server system
when a session is established by the computing device with the
server system. Once the application is executed, the video game
controller is used to send an identifier to the server system. The
server system verifies the identifier to pair the session with the
video game controller. When the video game controller is paired
with the session, the video game controller can be used to change a
state of a virtual scene that is displayed on the computing device
or on a display screen.
Inventors: |
Colenbrander; Roelof Roderick;
(San Mateo, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Sony Interactive Entertainment LLC |
San Mateo |
CA |
US |
|
|
Family ID: |
1000005655244 |
Appl. No.: |
17/348562 |
Filed: |
June 15, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
17016001 |
Sep 9, 2020 |
11045722 |
|
|
17348562 |
|
|
|
|
16181211 |
Nov 5, 2018 |
10773156 |
|
|
17016001 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A63F 13/77 20140902;
H04L 65/1069 20130101; A63F 13/79 20140902; A63F 13/23
20140902 |
International
Class: |
A63F 13/23 20060101
A63F013/23; H04L 29/06 20060101 H04L029/06; A63F 13/79 20060101
A63F013/79; A63F 13/77 20060101 A63F013/77 |
Claims
1. A method for pairing a hand-held controller (HHC) with a
session, comprising: establishing, by a display device, the session
with a server system; after the session is established, receiving,
by the display device from the server system, information
associated the session and a first instruction to provide
information identifying the HHC; sending, from the display device
to the HHC, the information associated the session and the first
instruction to provide the information identifying the HHC; and in
response to receiving the first instruction, providing, by the HHC,
the information identifying the HHC to the server system to pair
with the session.
2. The method of claim 1, further comprising: receiving login
information from the display device; determining whether the login
information is authentic, wherein said establishing the session
includes enabling an execution, by the server system, of an
application upon determining that the login information is
authentic.
3. The method of claim 2, wherein the information associated with
the session includes information identifying the application and a
destination address of the server system.
4. The method of claim 3, further comprising generating a second
instruction to provide the information identifying the HHC to the
destination address of the server system, wherein said providing
the information identifying the HHC includes: applying a wireless
communication protocol to the second instruction and the
information identifying the HHC; and after applying the wireless
communication protocol, sending the second instruction and the
information identifying the HHC via a router to a modem for
transferring the information identifying the HHC via a computer
network to the server system.
5. The method of claim 3, further comprising: after receiving the
destination address of the server system from the display device,
generating, by a processor of the HHC, a second instruction to send
the information identifying the HHC to the server system; in
response to receiving the second instruction, applying, by a
broadband communication circuit of the HHC, a cellular
communication protocol to the information identifying the HHC to
generate one or more transfer units; sending the one or more
transfer units via a cellular network and a computer network to the
server system.
6. The method of claim 1, further comprising allowing the HHC to
control an image displayed on the display device after the HHC is
paired with the session, wherein the image is displayed on the
display device based on an execution of the application.
7. The method of claim 1, wherein the information identifying the
HHC includes a device identification of the HHC.
8. The method of claim 1, wherein said sending the information
associated with the session and the first instruction to the HHC
from the display device is performed using a short distance
wireless communication protocol.
9. The method of claim 1, wherein the session is a game session, or
a video session, or a virtual image session, or an augmented
reality image session, or a virtual reality image session.
10. The method of claim 1, wherein the display device is a smart
television.
11. A system for pairing a hand-held controller (HHC) with a
session, comprising: the HHC; and a display device coupled to the
HHC, wherein the display device is configured to establish the
session with a server system, wherein after the session is
established, the display device is configured to receive from the
server system, information associated the session and a first
instruction to provide information identifying the HHC, wherein the
display device is configured to send to the HHC, the information
associated the session and the first instruction to provide the
information identifying the HHC, wherein in response to receiving
the first instruction, the HHC is configured to provide the
information identifying the HHC to the server system to pair with
the session.
12. The system of claim 11, wherein the display device is
configured to send login information to the server system, wherein
the session is established between the display device and the
server system when the login information is authenticated, wherein
the display device is configured to display a representation of an
application after the login information is authenticated.
13. The system of claim 12, wherein the information associated with
the session includes information identifying the application and a
destination address of the server system.
14. The system of claim 13, wherein the HHC includes: a processor;
and a wireless communication circuit coupled to the processor,
wherein the processor is configured to generate a second
instruction to provide the information identifying the HHC to the
destination address of the server system, wherein to provide the
information identifying the HHC, the wireless communication circuit
is configured to: apply a wireless communication protocol to the
second instruction and the information identifying the HHC; and
after the wireless communication protocol is applied, send the
second instruction and the information identifying the HHC via a
router to a modem for transferring the information identifying the
HHC via a computer network to the server system.
15. The system of claim 13, wherein the HHC includes: a processor;
and a broadband communication circuit coupled to the processor,
wherein when the destination address of the server system, is
received, the processor is configured to generate a second
instruction to send the information identifying the HHC to the
server system, wherein the processor is configured to send the
second instruction to the broadband communication circuit, wherein
when the second instruction is received, the broadband
communication circuit is configured to apply a cellular
communication protocol to the information identifying the HHC to
generate one or more transfer units, wherein the broadband
communication circuit is configured to send the one or more
transfer units via a cellular network and a computer network to the
server system.
16. The system of claim 11, wherein the HHC is configured to
control an image displayed on the display device after the HHC is
paired with the session, wherein the image is generated based on an
execution of the application.
17. A hand-held controller (HHC) for pairing with a session,
comprising: a first communication circuit configured to receive
information associated the session and a first instruction to
provide information identifying the HHC, wherein the information
associated the session is received from a display device after a
session is established between the display device and a server
system; a second communication circuit; and a processor coupled to
the first communication circuit and the second communication
circuit, wherein the processor is configured to receive the
information associated with the session and the first instruction
from the first communication circuit, wherein when the first
instruction is received by the processor, the processor is
configured to access the information identifying the HHC and
provide the information identifying the HHC to the second
communication circuit, wherein the second communication circuit is
configured to send the information identifying the HHC to the
server system to pair with the session,
18. The HHC of claim 17, wherein the information associated with
the session includes information identifying an application and a
destination address of the server system.
19. The HHC of claim 18, wherein the processor is configured to
generate a second instruction to provide the information
identifying the HHC to the destination address of the server
system, wherein to provide the information identifying the HHC, the
first communication circuit is configured to apply a wireless
communication protocol to the second instruction and the
information identifying the HHC, wherein after the wireless
communication protocol is applied, the first communication circuit
is configured to send the second instruction and the information
identifying the HHC via a router to a modem for transferring the
information identifying the HHC via a computer network to the
server system.
20. The HHC of claim 18, wherein upon receiving the destination
address of the server system, the processor is configured to
generate a second instruction to send the information identifying
the HHC to the server system, wherein the processor is configured
to send the second instruction and the information identifying the
HHC to the second communication circuit, wherein in response to
receiving the second instruction, the second communication circuit
is configured to apply a cellular communication protocol to the
information identifying the HHC to generate one or more transfer
units, wherein the second communication circuit is configured to
send the one or more transfer units via a cellular network and a
computer network to the server system.
Description
CLAIM OF PRIORITY
[0001] This application is a continuation of and claims the benefit
of and priority, under 35 U.S.C. .sctn. 120, to U.S. application
Ser. No. 17/016,001 filed on Sep. 9, 2020, titled "Systems and
Methods for Establishing Direct Communication Between a Server
System and a Video Game Controller", which is a continuation of
U.S. application Ser. No. 16/181,211, filed on Nov. 5, 2018, titled
"Systems and Methods for Establishing Direct Communication Between
a Server System and a Video Game Controller", and now issued as
U.S. Pat. No. 10,773,156, both of which are incorporated by
reference herein in their entirety for all purposes.
FIELD
[0002] The present disclosure relates to systems and methods for
establishing direct communication between a server system and a
video game controller.
BACKGROUND
[0003] Generally, a game is accessed for play via a game console.
The game console is coupled to a game controller. A gamer connects
the game console to a television. The game or an update for the
game is downloaded from a server to the game console and the gamer
plays the game using the game controller.
[0004] It is in this context that embodiments described in the
present disclosure arise.
SUMMARY
[0005] Embodiments described in the present disclosure provide
systems and methods for establishing direct communication between a
server system and a video game controller.
[0006] In some embodiments, the systems and methods described
herein provide a manner in which the video game controller can be
used with any display device that is coupled to a computer network.
There is no need to use a game console to access an application
that is executed on the server system. For example, the video game
controller can be used with a computing device, such as a cell
phone, or a tablet, or a television. There is no need to use the
game console. A session of execution of the application is accessed
using the computing device. Once the session is accessed, an
identifier that is associated with the video game controller is
sent to the server system. The server system verifies the
identifier. Upon verifying the identifier, the server system
determines to pair the video game controller with the session. Once
the video game controller is paired with the session, the video
game controller can be used by a user to change a state of a
virtual scene that is displayed on the computing device.
[0007] Some advantages of the herein described systems and methods
for establishing direct communication between the server system and
the video game controller include that there is no need to use the
game console between the video game controller and the server
system. The user accesses the session by providing his/her login
information. Once the session between the computing device and the
server system is established, the identifier associated with the
video game controller is sent from the video game controller to the
server system. When the identifier is verified by the server
system, the server system pairs the video game controller with the
session and the user is allowed to change the state of the virtual
scene that is displayed on the computing device or another display
device. Hence, there is no need for use of the game console to use
the video game controller. The video game controller can be used
regardless of whether the game console is used to access the
virtual scene generated by executed of the application stored on
the server. The non-use of the game console reduces a number of
hops between the video game controller and the server system. Each
hop is used to receive the data, analyze the data to determine its
destination address, and send the data to the destination address.
By removing the game console, a hop associated with the game
console is reduced. The reduction of a number of hops between the
video game controller and the server system decreases latency time
between the video game controller and the server system and
provides for faster game play.
[0008] Other aspects described will become apparent from the
following detailed description, taken in conjunction with the
accompanying drawings, illustrating by way of example the
principles of embodiments described in the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Various embodiments described in the present disclosure may
best be understood by reference to the following description taken
in conjunction with the accompanying drawings in which:
[0010] FIG. 1A is a diagram of an embodiment of a system to
illustrate pairing of a controller with a session.
[0011] FIG. 1B is a diagram of an embodiment of the controller to
illustrate wireless communication of the controller with a
router.
[0012] FIG. 1C is a diagram of an embodiment of a system to
illustrate use of a cellular connection channel by a
controller.
[0013] FIG. 1D is a diagram of an embodiment of the controller of
the system of FIG. 1C to illustrate use of a broadband
communication integrated circuit (IC) chip to transfer
identification information from the controller of FIG. 1C to the
server system.
[0014] FIG. 1E is a diagram of an embodiment of the server system
to illustrate pairing of the session with a controller.
[0015] FIG. 1F-1 is a diagram of an embodiment of a pairing-request
notification that is displayed on a display screen of a computing
device.
[0016] FIG. 1F-2 is a diagram of an embodiment of a controller to
illustrate a display of a pairing-request notification on a display
screen of a controller.
[0017] FIG. 1G is a diagram of an embodiment of a controller to
illustrate a haptic feedback device, an audio device, and a display
device of the controller.
[0018] FIG. 2 is a diagram of an embodiment of a system to
illustrate a mirroring effect of the computing device.
[0019] FIG. 3 is a diagram of an embodiment of a system to
illustrate an execution of a discovery program for discovering a
device ID of the controller of FIG. 1E.
[0020] FIG. 4A is a diagram of an embodiment of a controller to
illustrate use of a calibration processor within a controller.
[0021] FIG. 4B is a diagram of an embodiment of a controller to
illustrate use of the calibration processor within the
controller.
[0022] FIG. 5 is a diagram of an embodiment of a server system to
illustrate calibration of the sensor values by the calibration
processor of the server system.
[0023] FIG. 6 illustrates a perspective view of an embodiment of a
video game controller for interfacing with an interactive
program.
[0024] FIG. 7 is a flowchart of an embodiment of a method to
illustrate pairing of a controller with a session.
[0025] FIG. 8 is a flowchart of an embodiment of a method to
illustrate pairing of a controller with a session.
DETAILED DESCRIPTION
[0026] Systems and methods for establishing direct communication
between a server system and a video game controller are described
herein.
[0027] It should be noted that various embodiments described in the
present disclosure may be practiced without some or all of these
specific details. In other instances, well known process operations
have not been described in detail in order not to unnecessarily
obscure various embodiments described in the present
disclosure.
[0028] FIG. 1A is a diagram of an embodiment of a system 100 to
illustrate pairing of a controller 102 with a session 106. Examples
of the controller 102 include a hand-held controller, which can be
a Sony.TM. DualShock.TM. controller, a gun-shaped controller, a
PlayStation.TM. Move.TM. controller, a stick-shaped controller, a
cellular phone, a mobile device, a tablet, a video game controller,
a joystick, a glove-shaped controller, or a steering wheel-shaped
controller, etc. Examples of the session 106 include a game
session, a video session, a virtual image session, an augmented
reality image session, a virtual reality image session, etc. As an
illustration, the session 106 is an instance of execution of an
application 1, which can be a game execution application that
allows a play of a game application or a video conferencing
application that allows images of a real-world environment to be
transferred via the computer network 122 between two clients.
Examples of the real-world environment include a room or a confined
location or an enclosed environment or an environment that is
enclosed by walls or a cubicle. The session 106 starts when a user
A logs into a user account 1 and ends when the user A logs out of
the user account 1. For example, an instance of execution of the
session 106 starts when the user A logs into his/her user account 1
and ends for the user account 1 when the user A logs out of the
user account 1. The logging out by the user A disables game play by
the user A and the logging in by the user A into the user account 1
is performed to enable the game play. The user A logs out of the
user account 1 by selecting a button on the controller 102 or
another controller, described herein.
[0029] The system 100 includes the controller 102, a computing
device 114, a computer network 122, a server system 104, a router
118, and a modem 120. As an example, the router 118 and the modem
120 are locally situated with respect to the controller 102. For
example, the router 118 and the modem 120 are situated in the same
real-world environment in which the controller 102 is situated.
[0030] The computing device 114 is a machine for performing
calculations. Examples of the computing device 114 include a smart
television, a tablet, a smart phone, a head-mounted display (HMD),
an electronic computer, an information processing system, a desktop
computer, and a laptop computer. Another example of the computing
device 114 includes a combination of a display device and a game
console. The display device is coupled to the game console. The
computer network 122 is a group of computing hardware devices that
are linked together through communication channels to facilitate
communication and resource sharing among users. The computer
network 122 can be a wide area network or a local area network or a
combination of the wide area network and the local area network.
Internet is an example of the wide area network and Intranet is an
example of the local area network.
[0031] The server system 104 includes a single or multiple servers
that execute the application 1 and other applications. For example,
each server includes a server operating system (OS) that is
designed to run on the server. Each server is a specialized
computer that serves requests from a client, such as a computing
device or a controller or a display device, described herein. The
specialized computer includes a processor and a memory device. The
processor is coupled to the memory device. As used herein, terms,
such as a processor, an application specific integrated circuit
(ASIC), a programmable logic device (PLD), and a central processing
unit (CPU), a server, a microprocessor, are used herein
interchangeably. Examples of a memory device include a read-only
memory and a random access memory. To illustrate, the memory device
is a flash memory, a hard disk drive, or a redundant array of
independent disks (RAID).
[0032] The modem 120 is a device that performs modulation and
demodulation of data to allow the client to connect to the computer
network 122. As an example, the modem 120 applies a network
communication protocol, such as Transmission Control Protocol
(TCP/IP) to extract data from transfer packets. The modem 120
applies the network communication protocol to embed data into
transfer packets. An example of the modem 120 is a broadband modem
that allows the client to access the Internet via a cable or a
digital subscriber line (DSL). The modem 120 routes a data signal
received from the router 118 to the server system 104 via the
computer network 122. In addition, the modem 120 routes a data
signal received from the server system 104 via the computer network
122 to the router 118. The modem 120 is coupled at one end to a
cable or the DSL, which is coupled to the computer network 122. In
one embodiment, the terms data and information are used herein
interchangeably.
[0033] The router 118 directs a data signal that is received from
the server system 104 via the computer network 112 to the computing
device 114 or the controller 102 based on a destination address
received by the router 118 with the data signal. For example, the
router 118 determines that the data signal has a destination
address of the computing device 114 and directs the data signal to
the computing device 114 assigned the destination address. An
example of a destination address is an Internet Protocol (IP)
address or a media access layer (MAC) address or a combination
thereof. The destination address of the computing device 114 is
different from the destination address of the controller 102. For
example, at least one alphanumeric character of the destination
address of the controller 102 is different from at least one
alphanumeric character of the destination address of the computing
device 114. The router 118 determines that another data signal has
a destination address of the controller 102 and directs the other
data signal to the controller 102. An example of the router 118 is
a broadband router. The computing device 114 has a display screen
116, such as a liquid crystal display (LCD) screen, a light
emitting diode (LED) display screen, or a plasma display screen,
which is a display part of a monitor or a display device.
[0034] The server system 104 stores multiple user accounts 1
through N. Each user account is assigned to a user. For example,
the user account 1 is assigned to the user A and another user
account 2 is assigned to another user B.
[0035] The controller 102 is coupled to the router 118 via a
connection channel CC1. An example of the connection channel CC1 is
a wireless connection, such as a wireless local area network (LAN)
or a Wi-Fi connection or a Bluetooth.TM. connection. Wi-Fi is a
wireless networking technology that uses radio waves or
electromagnetic waves to provide a wireless high-speed connection
to the computer network 122. Similarly, the computing device 114 is
connected to the router 118 via a connection channel CC2, which is
also a wireless connection. The connection channel CC1 or CC2 can
be a radio frequency and has a specific data rate, measured in bits
per second, and a bandwidth, measured in hertz. An example of each
of the connection channel CC1 and CC2 is a wireless connection
channel that applies a wireless communication protocol, such as
Wi-Fi.TM. or Bluetooth.TM.. The connection channel CC2 of the
computing device 114 with the router 118 is different from the
connection channel CC1 of the controller 102 with the router 118.
For example, the wireless connection of the computing device 114
with the router 118 has a different data transfer rate between the
computing device 114 and the router 118 compared to a data transfer
rate between the controller 102 and the router 118.
[0036] The router 118 is coupled to the modem 120 via an Ethernet
cable EC. The modem 120 is coupled to the computer network 122 via
a connection C1 and the server system 104 is coupled to the
computer network 122 via another connection C2. The connection C1
includes a group of cables or digital subscriber lines and the
connection C2 includes a gateway device and a group of cables or
digital subscriber lines. A combination of the connection channel
CC1, the router 118, the cable EC, the modem 120, and the
connection C1 is sometimes referred to herein as a connection
channel 111.
[0037] The user A powers on the computing device 114 and accesses
the user account 1. For example, the user A accesses a login
website from the server system 104 via the computer network 122.
The user A uses an input device, such as a keyboard or a pad or a
mouse, of the computing device 114 to provide his/her login
information, which can include a username and a password, assigned
to the user A. The login information is sent from the computing
device 114 via the connection channel CC2 to the router 118. The
router 118 sends the login information to the modem 120 via the
cable EC. The modem 120 generates one or more transfer packets
embedding the login information and sends the transfer packets via
the connection C1, the computer network 122, and the connection C2
to the server system 104.
[0038] A server of the server system 104 determines that the login
information is authentic and provides access to the user account 1
and establishes the session 106. To establish the session 106, the
server system 104 executes an instance of the application 1 and
sends one or more transfer packets that include information
associated with the session 106 to the computing device 11 via the
connection C2, the computer network 122, the connection C1, the
modem 120, the cable EC, the router 118, and the connection channel
CC2. An example of the information associated with the session 106
includes information identifying the application 1, such as a name
of the application 1 that is allowed access upon providing access
to the user account 1, a title of the application 1, or virtual
image data that identifies the application 1, or audio data that
identifies the application 1, or a combination of two or more
thereof. To illustrate, the information associated with the session
106 has image data for displaying an introductory image of a video
game at the computing device 114 and audio data that is
synchronized to the display of the introductory image. As another
illustration, the information associated with the session 105
includes a uniform resource locator (URL) of a gaming website
accessed to view features of and to play the video game. Another
example of the information associated with the session 105 includes
a notice that the user A provides his/her biometric information via
the controller 102 by using a biometric information scanner BIS of
the controller 102. To illustrate, the information associated with
the session 106 includes the notice that the user A press his/her
finger against a fingerprint reader on the controller 102 or that
the user A speak into a microphone of the controller 102 or that
the user A scan his/her eye into a biometric eye scanner of the
controller 102. Examples of a biometric information scanner include
the fingerprint reader, the microphone, and the biometric eye
scanner. Another example of the information associated with the
session 105 includes the information identifying the application 1
and the notice that the user A provides his/her biometric
information via the controller 102. The biometric information is
unique to the user A and distinguishes the user A from other users.
Yet another example of the information associated with the session
106 includes the information identifying the application 1 and an
instruction to be sent to the controller 102 to provide its device
identification (ID). Examples of the device ID include a serial
number of the controller 102, a MAC address of the controller 102,
and a combination thereof. The device ID of the controller 102 is
unique to the controller 102 and distinguishes the controller 102
from other controllers, which can be the same in structure and
function as the controller 102. Still another example of the
information associated with the session 106 includes the
information identifying the application 1 and an instruction to be
sent to the controller 106 to provide a network identification
(ID). Examples of the network ID include a network address of the
router 118. The network ID stored within the controller 102 is
unique to the router 118 and distinguishes the router 118 from
other routers.
[0039] One or more transfer packets having the information
associated with the session 106 and sent from the server system 104
also include a destination address of the router 118 and one or
more additional destination addresses, such as the destination
address of the computing device 114 and the destination address of
the controller 102. Upon receiving the transfer packets including
the information associated with the session 106, the modem 120
applies the network communication protocol to depacketize the
transfer packets to identify the destination address of the router
118, the destination address of the computing device 114, the
destination address of the controller 102, and the information
associated with the session 106. It should be noted that the
destination address of the controller 102 is pre-stored in a memory
device of the server system 104. For example, when the user A makes
an in-person purchase of the controller 102, the user A provides
his/her user account information to an entity that sells the
controller 102 to the user A. The entity makes a selection via an
input device of a client device to register the controller 102 with
the user account 1. Upon receiving a signal generated based on the
selection from the client device, one or more processors, described
herein, of the server system 104 register a device ID of the
controller 102 with the user account 1 to register the controller
102 with the user account 1. To illustrate, the one or more
processors of the server system 104 stores a link between the user
account 1 and a device ID of the controller 102 within a mapping
database, described below. As another example, when the user A
makes an online purchase of the controller 102, the user A provides
his/her user account information to a website, such as a retailer
website or a manufacturer website, accessed by the user A for the
purchase. The user account information is sent from the website to
the server system 104 for the one or more processors to register
the controller 102 with the user account 1. Examples of the user
account information include an email address used by accessing the
user account 1 of the user A, a user name assigned to the user
account 1, a residence address associated with the user account 1,
or a phone number associated with the user account 1, or a
combination thereof. The modem 120 sends the destination address of
the computing device 114, the destination address of the controller
102, and the information associated with the session 106 via the
cable EC to the router 118.
[0040] Upon receiving the destination address of the computing
device 114 and the information associated with the session 106, the
router 118 determines that the information associated with the
session 106 is to be sent to computing device 114 and sends the
information associated with the session 106 to the computing device
114 via the connection channel CC2. A combination of the connection
channel CC2, the router 118, the cable EC, the modem 120, the
connection C1, the computer network 122, the connection C2 is
referred to herein as a communication channel 108, which is
illustrated using long and short dashed lines "".
[0041] Similarly, upon receiving the destination address of the
controller 102 and the information associated with the session 106,
the router 118 determines that the information associated with the
session 106 is to be sent to controller 102 and sends the
information associated with the session 106 to the controller 102
via the connection channel CC1. A combination of the connection
channel CC1, the router 118, the cable EC, the modem 120, the
connection C1, the computer network 122, the connection C2 is
referred to herein as a communication channel 110, which is
illustrated using "o"s.
[0042] Upon receiving the information associated with the session
106, the computing device 114 displays the information on the
display screen 116 or outputs the audio data within the information
associated with the session 106 via one or more speakers of the
computing device 114 or both displays the information and outputs
the audio data. Upon reading the information associated with the
session 106 displayed on the display screen 116 or listening to the
audio data, the user A provides his/her biometric ID, such as voice
or fingerprint, or retinal information, to the biometric
information scanner BIS. For example, the user A says "Hey
Controller" to provide his/her voice to the biometric information
scanner BIS.
[0043] In case the information associated with the session 106
includes the instruction for the controller 102 to provide its
device ID, the controller 102 accesses the device ID from a memory
device of the controller 102. Also, in case the information
associated with the session 106 includes the instruction for the
controller 102 to provide the network ID, the controller 102
accesses the network ID from a memory device of the controller 102.
The biometric ID or the device ID or the network ID a combination
of two or more thereof is referred to herein as identification
information.
[0044] The identification information and an instruction to provide
the identification information to a destination address of the
server system 104 is sent from the controller 102 via the
communication channel CC1 to the router 118. As an example, the URL
of the gaming website is stored in firmware of a read-only memory
or in another memory device of the controller 102. The URL includes
the destination address of the server system 104. As another
example, the computing device 114 provides the URL of the server
system 104 via a wireless connection 115 to the controller 102. The
computing device 114 or another device, such as the game console,
is bypassed or not used in sending the identification information.
For example, the identification information is not sent from the
controller 102 to the computing device 114 or the game console. As
another example, the identification information is not addressed by
the controller 102 to be sent to the computing device 114 or to the
game console. The identification information is addressed by the
controller 102 to be sent to the server system 104. The computing
device 114 is coupled to the controller 102 via a wireless
connection 115, such as a Bluetooth.TM. connection or a
Bluetooth.TM. connection or another short-range connection.
[0045] The router 118 receives the identification information and
the instruction via the communication channel CC1, identifies the
destination address of the server system 104 within the
instruction, and routes the identification information to the modem
120 via the cable EC. The modem 120 receives the identification
information and the instruction, identifies the destination address
of the server system 104 within the instruction, applies the
network communication protocol to the identification information to
generate transfer packets and sends the transfer packets via the
connection C1, the computer network 122, and the connection C2 to
the server system 104. The server system 104 receives the
identification information and pairs the controller 102 with the
session 106 upon verifying that the identification information.
[0046] The server system 104 generates and sends a pairing
notification in response to pairing the controller 102 with the
session 106. For example, the pairing notification includes image
frames to be displayed on the computing device 114, or audio frames
to be output as sound by the computing device 114, or a combination
thereof. The image frames include information for displaying the
pairing notification, such as color and intensity and texture of
the pairing notification. As another example, the pairing
notification includes an instruction to a rendering program that is
executed by a graphical processing unit (GPU) of the computing
device 114, or an instruction to an audio processor of the
computing device 114 for processing audio data, or both the
instructions. The server system 104 sends the pairing notification
via the connection C2, the computer network 122, the connection C1,
the modem 120, the cable EC, the router 108, and the connection
channel CC2 to the computing device 114. The modem 120 receives one
or more transfer packets including the pairing information and a
destination address of the computing device 114 from the server
system 104 via the connection C2, the computer network 122, and the
connection C1. The modem 120 applies the network communication
protocol to the transfer packets to extract the destination address
of the computing device 114 and the pairing notification, and sends
the pairing notification and the destination address to the router
118 via the cable EC. The router 118 determines from the
destination address of the computing device 114 that the pairing
notification is to be sent to the computing device 114, and sends
the pairing notification to the computing device 114 via the
connection channel CC2. For example, the router 118 identifies from
an IP address of the computing device 114 that the pairing
notification is to be sent to the computing device 114. Upon
receiving the pairing notification that includes the image frames
or audio frames or a combination thereof, the pairing information
is output as one or more images on the display screen 116 of the
computing device 114 or as sound via one or more speakers of the
computing device 114 or a combination thereof. Upon receiving the
pairing information that includes the instruction to the rendering
program. the GPU of the computing device 114 executes the rendering
program to display the pairing notification on the display screen
116. Also, when the pairing notification includes the instruction
to the audio processor, the audio processor and one or more
speakers of the computing device 114 execute the instruction to
process the audio data to output the audio data as sound.
Similarly, when the pairing notification includes both the
instructions, both the GPU and the audio processor synchronize with
each other to output the pairing notification as images and
sound.
[0047] Once the session 106 is paired, such as linked or associated
with, with the controller 102, the controller 102 can be used by
the user A to interact with the session 106 to establish a direct
communication between the server system 104 and the controller 102.
For example, the controller 102 can be used by the user A to play
the game generated by executing the application 1. To illustrate,
the controller 102 can be used by the user A to provide input data,
such as controller movement data, controller button presses, etc.,
to the server system 104 via the communication channel 110 to
change a state of a virtual scene being displayed on the computing
device 114.
[0048] Before the session 106 is paired with the controller 102,
the controller 102 cannot be used by the user 102 to interact with
the session 106 of execution of the application 1 to change the
state of the virtual scene displayed on the computing device 114.
For example, the user A selects or moves an input device of the
controller 102. The selection or movement of the input device
generates input information, which is not processed by or
disallowed the server system 102 to change the state of the virtual
scene. The input information transferred from the controller 102 to
the server system 102 via the communication channel 110 for
changing the state of the virtual scene generated by execution of
the application 1 is not processed by the server system 104 for
changing the state of the virtual scene until the controller 102 is
paired with the session 106.
[0049] In one embodiment, in a multi-player game, such as a
multi-player race car game or a soccer game, the logging out by the
user A from the user account 1 does not affect other instances of
execution of the session 106 for other users also playing the game.
The instances of the session 106 for the other users continue until
they log out of their corresponding user accounts.
[0050] In an embodiment, the user A physically transfers the
controller 102 or another controller, described herein, to another
user. If the other user uses the user account 1 assigned to the
user A, the instance of the session 106 associated with the user
account 1 continues. However, if the other user users his/her user
account 2 after logging out of the user account 1, another instance
of the session 106 initiates for the user account 2.
[0051] In one embodiment, the session 106 ends when the computing
device 114 on which the virtual scene is being displayed is
disconnected from the server system 104 or when the computing
device 114 is powered off, or a communication signal between the
computing device 114 and the server system 104 is weak, e.g., has
an amount of power that is less than a threshold, or a
communication device of the computer network 122 is not functional
or is malfunctioning, or a communication device of the computing
device 114 is not functional or is malfunctioning.
[0052] In an embodiment, both the router 118 and the modem 120 are
integrated into a single hardware device.
[0053] In one embodiment, the connection channel CC2 is a wired
connection channel, such as a coax cable. Similarly, the connection
channel CC1 is a wired connection.
[0054] In an embodiment, functions described herein as being
performed by the server system 104 are performed by one or more
processors of the server system 104. Similarly, in an embodiment,
functions described herein as being performed by the modem 120 are
performed by one or more processors of the modem 120. Also, in an
embodiment, functions described herein as being performed by the
router 118 are performed by one or more processors of the router
118. In one embodiment, functions described herein as being
performed by the controller 102 are performed by one or more
processors of the controller 102.
[0055] In an embodiment, a pairing notification, described herein,
is not sent from the server system 104 to the computing device
114.
[0056] In one embodiment, the computing device 114 is not coupled
to the controller 102 via the wireless connection 115.
[0057] In one embodiment, the controller 102 sends a request to the
computing device 114 to obtain the destination address of the
server system 104 before sending the identification information to
the server system 104. For example, The computing device 114
requests that the wireless connection 115 occur between the
computing device 114 and the controller 102. Once the wireless
connection 115 is established, the controller 102 sends a request
to the server system 104 via the wireless connection 115 for
obtaining the destination address. Upon receiving the request, the
computing device 114 provides the destination address of the server
system 104 to the controller 102 via the wireless connection
115.
[0058] In an embodiment, the session 106 does not end when another
user uses the controller 102. The other user receives the
controller 102 from the user A while the session 106 is active
after being initiated. The other user provides his/her login
information to the server system 104 via the computing device 114
to log into his/her user account to initiate another instance of
the session 106 for the user account of the other user. The session
106 remains active and does not end when the other user switches to
his/her account from the user account 1.
[0059] FIG. 1B is a diagram of an embodiment of the controller 102
to illustrate wireless communication of the controller 102 with the
router 118. The controller 102 includes a motion sensor system 152,
a wireless communication integrated circuit (IC) 156, an input
device 150, a computer network ID chip 151, a wireless
communication IC 157, the biometric identification sensor BIS, a
processor 194, and a device ID chip 158. The processor 194 is
coupled to the motion sensor system 152, wireless communication
integrated circuit (IC) 156, the input device 150, the computer
network ID chip 151, the wireless communication IC 157, the
biometric identification sensor BIS, and the device ID chip 158.
The input device 150 is coupled to the motion sensor system 152.
Examples of the motion sensor system 152 include one or more
gyroscopes, one or more accelerometers, and one or more
magnetometers to facilitate a determination of a position and
orientation of the controller 102 and to facilitate a determination
of changes to the position and orientation. Examples of the input
device 150 include a button, a switch, a touch screen, a stylus, a
joystick, a microphone, a gun trigger, etc. Examples of the
wireless communication IC 157 include a Bluetooth.TM. device or a
Wi-Fi.TM. device that enables communication between the controller
102 and the computing device 114 (FIG. 1A) via the wireless
connection 115 (FIG. 1A). To illustrate, a Bluetooth.TM. device
includes a processor that facilitates a transfer of data over short
distances using short-wavelength ultra high frequency (UHF) radio
waves in an industrial, scientific, and medical (ISM) band from 2.4
to 2.485 GHz between a controller and another device, such as a
computing device, building personal area networks (PANs).
[0060] The device ID chip 158 is a memory chip that stores the
device ID, such as a MAC address or a unique identification number,
of the controller 102. For example, the device ID chip 158 stores a
sequence of alphanumeric characters, which are unique to the
controller 102 and distinguishes the controller 102 from other
controllers. An example of the wireless communication IC 156
includes a Wi-Fi.TM. device that allows Wi-Fi.TM. communication
between the controller 102 and the router 118 via the connection
channel CC1 (FIG. 1A). To illustrate, the wireless communication IC
156 is a network interface card (NIC) that couples a controller, as
described herein, via the connection channel CC1 and the router 118
to the computer network 122.
[0061] The computer network ID chip 151 is another memory chip that
stores the network ID, such as an Internet Protocol (IP) address,
of the router 118 of FIG. 1A. The router 118 is identified within
the computer network 122 using the network ID stored in the
computer network ID chip 151. For example, the computer network ID
chip 151 stores a sequence of alphanumeric characters, which are
unique to the router 118 and distinguishes the router 118 from
other routers within the computer network 122.
[0062] The device ID of the controller 102 and the other
corresponding device IDs of the other controllers are
pre-registered with the server system 104 before the session 106
(FIG. 1A) starts. For example, the device ID unique to the
controller 102 is hardwired into the controller 102 and the server
system 104 stores the device ID. The device ID of the controller
102 and additional device IDs of the other controllers are stored
in a device ID database of the server system 104. The device ID
database is stored in one or more memory devices of the server
system 104.
[0063] Also, the network ID identifying the router 118 is
pre-registered with the server system 104 before the controller 102
sends the identification information to the server system 104 to
pair the controller 102 with the session 106. For example, during
establishment of the session 106, the server system 104 sends an
instruction to the computing device 114 via the communication
channel 108 to provide the network ID. Upon receiving the
instruction, the computing device 114 accesses the network ID
stored in a memory device of the computing device 114, and sends
the network ID via the communication channel 108 (FIG. 1A) to the
server system 104. The server system 104 receives the network ID
and stores the network ID in a network ID database of the server
system 104. The network ID database is stored in one or more memory
devices of the server system 104. Also, the server system 104
associates, such as establishes a one-to-one correspondence or a
link, between the network ID and the user account 1 after the
session 106 is established.
[0064] Moreover, the biometric ID identifying the user A is
pre-registered with the server system 104 before the controller 102
sends the identification information to the server system 104 to
pair the controller 102 with the session 106. For example, the
computing device 114 has a biometric information scanner. After the
session 106 is established or while the session 106 is being
established, the server system 104 sends an instruction to the
computing device 114 via the communication channel 108 to request
the biometric ID from the user A. When the computing device 114
receives the instruction, the computing device 114 displays a
message or output a sound to request the biometric ID from the user
A. Upon receiving the biometric ID via the biometric information
scanner of the computing device 114 from the user A, the computing
device 114 sends the biometric ID via the communication channel 108
to the server system 104. The server system 104 receives the
biometric ID and stores the biometric ID in a biometric ID database
of the server system 104. The biometric ID database is stored in
one or more memory devices of the server system 104. Also, the
server system 104 associates, such as establishes a one-to-one
correspondence or a link, between the biometric ID and the user
account 1.
[0065] After the session 106 is established between the computing
device 114 and the server system 104, the user A provides his/her
biometric ID to the biometric identification scanner BIS of the
controller 102 to pair the controller 102 with the session 106. For
example, after the session 106 starts, upon viewing the notice that
the user A press his/her finger against a fingerprint reader on the
controller 102 or that the user A speak into the microphone of the
controller 102 or that the user 102 scan his/her eye into the
biometric eye scanner of the controller 102, the user A provides
his/her biometric information to the biometric information scanner
BIS of the controller 102. Upon receiving the biometric ID, the
biometric identification scanner BIS of the controller 102
generates a biometric identification signal and sends the biometric
identification signal to the processor 194. The processor 194
receives the biometric identification signal and generates an
instruction to send the biometric identification signal to the
server system 104. For example, the instruction includes a
destination address of the server system 104. The processor 194
sends the instruction and the biometric identification signal to
the wireless communication IC 156. The wireless communication IC
156 applies the wireless communication protocol to the biometric
identification signal and the instruction and sends the biometric
identification signal and the instruction via the connection
channel CC1 (FIG. 1A) to the router 118. The router 118 determines
from the destination address within the instruction that the
biometric identification signal is to be sent to the server system
104 and sends the instruction and the biometric identification
signal to the modem 120. The modem 120 applies the network
communication protocol to the biometric identification signal to
generate one or more transfer packets and sends the transfer
packets via the connection C1, the computer network 122, and the
connection C2 to the server system 104. The transfer packets having
the biometric identification signal is an example of a connection
request.
[0066] As an alternative or in addition to sending the biometric
identification signal after the session 106 is established between
the computing device 114 and the server system 104, the processor
194 accesses the device ID from the device ID chip 158 and
generates an instruction to send the device ID to the server system
104 to pair the controller 102 with the session 102. The processor
194 sends the instruction and the device ID to the wireless
communication IC 156. The wireless communication IC 156 applies the
wireless communication protocol to the device ID and the
instruction and sends the device ID and the instruction via the
connection channel CC1 (FIG. 1A) to the router 118. The router 118
determines from the instruction that the device ID is to be sent to
the server system 104 and sends the instruction and the device ID
to the modem 120. The modem 120 generates one or more transfer
packets having the device ID and sends the transfer packets via the
connection C1, the computer network 122, and the connection C2 to
the server system 104. The transfer packets having the device ID is
another example of the connection request.
[0067] Also, as an alternative or in addition to sending the
biometric identification signal after the session 106 starts, or as
an alternative or in addition to sending the device ID after the
session 106 starts, the processor 194 accesses the network ID from
the computer network ID chip 151 and generates an instruction to
send the network ID to the server system 104 to pair the controller
102 with the session 106. The processor 194 sends the instruction
and the network ID to the wireless communication IC 156. The
wireless communication IC 156 applies the wireless communication
protocol to the network ID and the instruction and sends the
network ID and the instruction via the connection channel CC1 to
the router 118. The router 118 determines from the instruction that
the network ID is to be sent to the server system 104 and sends the
instruction and the network ID to the modem 120. The modem 120
generates one or more transfer packets from the network ID and
sends the transfer packets via the connection C1, the computer
network 122, and the connection C2 to the server system 104. The
transfer packets having the network ID is yet another example of
the connection request.
[0068] It should be noted that the connection request is not sent
via the game console or the computing device 114. There is no use
of the game console or the computing device 114 in transferring the
connection request from the controller 102 to the server system
104.
[0069] Once the device ID, or the biometric ID, or the network ID,
or a combination of two or more thereof is used by the server
system 104 to pair the controller 102 with the session 106, the
controller 102 can be used by the user A to generate and provide
inputs for changing the state of the virtual scene that is
displayed upon execution of application 1. For example, once the
controller 102 is paired with the server system 104, the user A can
use the controller 102 to change the state of the virtual scene.
The user A uses the controller 102 by selecting or moving the input
device 150 or moving the controller 102 to provide a selection. The
input device 150 generates an input signal upon receiving the
selection or movement of the input device 150 and/or the motion
sensor system 152 generates an input signal when the controller 102
moves. The input signal generated by the motion sensor 152 includes
data for determining the position or orientation of the controller
102 and/or data for determining a position and an orientation of
the input device 150. Examples of the data for calculating the
position or orientation of the controller 102 includes an
acceleration of the controller 102, an angular velocity of the
controller 102, and an orientation of the controller 102. The
position and orientation of the controller 102 and the position and
orientation of the input device 150 are measured with reference to
a reference co-ordinate system within the controller 102. Examples
of the data for calculating the position and orientation of the
input device 150 include an acceleration of the input device 150,
an angular velocity of the input device 150, and an orientation of
the input device 150. The processor 194 receives one or more of the
input signals.
[0070] The processor 194 receives the one or more input signals to
output input information. As an example, the input information
includes the data for calculating the position or orientation or a
combination thereof of the controller 102 or includes the data for
calculating the position or orientation or a combination thereof of
the input device 150 or the selection made by the user A via the
input device 150. To illustrate, the input information includes
which of a plurality of buttons of the input device 150 are
selected by the user A or which of a plurality of joysticks of the
input device 150 are moved and in which direction. In addition to
the input information, the processor 194 also generates an
instruction to send the input information to a destination address
of the server system 104.
[0071] The processor 194 sends the input information via the
communication channel 110 to the server system 104. For example,
the processor 194 sends the input information and the instruction
to the wireless communication IC 156. The wireless communication IC
156 applies the wireless communication protocol to the input
information and the instruction and sends the input information and
the instruction via the connection channel CC1 (FIG. 1A) to the
router 118. The router 118 determines from the instruction that the
input information is to be sent to the server system 104 and sends
the instruction and the input information to the modem 120. The
modem 120 generates one or more transfer packets from the input
information and sends the transfer packets via the connection C1,
the computer network 122, and the connection C2 to the server
system 104.
[0072] In one embodiment, the wireless communication IC 156 and the
wireless communication IC 157 are integrated into a single
integrated circuit chip. In an embodiment, the wireless
communication IC 156 is integrated into one integrated circuit chip
and the wireless communication IC 157 is integrated into another
integrated circuit chip.
[0073] FIG. 1C is a diagram of an embodiment of a system 159 to
illustrate use of a communication channel 155, which includes a
cellular connection channel 151 and a cellular connection channel
153. The communication channel 155 is illustrated as small dashes
"" in FIG. 1C. Each of the cellular connection channels 151 and 152
is a wireless connection channel, such as a radio frequency
connection channel The communication channel 155 includes a
cellular network 154 or a mobile network, which includes multiple
cell towers, such as a tower TW. Examples of the cellular network
154 include a mobile broadband network, a fourth-generation (4G)
mobile network, a 4G long term evolution (4G LTE), and a
fifth-generation (5G) mobile network. The cellular network 154
transfers data using a cellular communication protocol, such as
mobile broadband protocol, the 4G protocol, the 4G LTE protocol, or
the 5G protocol. The system 159 includes a controller 160, which is
the same in structure and function as the controller 102 (FIG. 1B),
except that the controller 160 couples to the computer network 122
via the cellular network 154 instead of via the router 118. For
example, the destination address of the server system 104 is stored
in firmware of the controller 160 so that the controller 160 can
send the identification information to the server system 104. As
another example, the controller 160 is coupled to the computing
device 114 via the wireless connection 115 (FIG. 1A) to send a
request for the destination address of the server system 114. Upon
receiving the request, the computing device 114 sends the
destination address to the controller 160 via the wireless
connection 115.
[0074] Each of the cell towers includes a transceiver to transfer
data received from the controller 160 to the computer network 122
and received from the computer network 122 to the controller 160. A
combination of the cellular connection channel 151, the multiple
towers, the cellular connection channel 153, the computer network
122, and the connection C2 is the communication channel 155.
[0075] The system 159 is similar, in structure and function, to the
system 100 of FIG. 1A except that in the system 159 the controller
160 is used. The controller 160 is similar, in structure and
function, to the controller 102 except that the controller 160
communicates with the server system 104 via the computer network
122 and one or more cell towers. Instead of communicating with the
computer network 122 via the connection channel CC1, the router
118, the cable EC, the modem 120, and the connection C1, the
controller 160 communicates with the computer network 122 via the
cellular connection channel 151, the tower TW, and the cellular
connection channel 152.
[0076] When the session 106 (FIG. 1A) of the application 1 is
established or starts, the controller 160, such as a subscriber
identification module (SIM) card of the controller 160, applies the
cellular communication protocol to the identification information
and an instruction to send the identification information to
generate one or more transfer units to transfer to the server
system 104. The controller 160 sends the transfer units including
the identification information and the instruction via the cellular
connection channel 151 to the tower TW. The computing device 114 or
another device, such as the game console, is bypassed in sending
the identification information. For example, the identification
information is not sent from the controller 160 to the computing
device 114 or the game console. As another example, the
identification information is not addressed by the controller 160
to be sent to the computing device 114 or to the game console. The
identification information is addressed by the controller 160 to be
sent to the server system 104.
[0077] A transceiver of the tower TW receives the transfer units
having the identification information and the instruction and the
transceiver forwards the transfer units to the computer network
122. A gateway, such as a modem, of the computer network 122
demodulates the transfer units by applying the cellular
communication protocol to extract the instruction and the
identification information from the transfer units, and applies the
network communication protocol to the identification information to
generate transfer packets, and sends the transfer packets having
the identification information to the server system 104 via the
connection C2. A transfer of the input information from the
controller 160 to the server system 102 via the communication
channel 155 for changing the state of the virtual scene generated
by execution of the application 1 is disabled or not enabled by the
controller 160 until the controller 160 is paired with the session
106.
[0078] FIG. 1D is a diagram of an embodiment of the controller 160
to illustrate use of a broadband communication IC 162 to transfer
the identification information from the controller 160 to the
server system 104 (FIG. 1C). The controller 160 includes the input
device 150, the motion sensor system 152, the wireless
communication IC 157, the biometric identification scanner BIS, the
processor 194, the broadband communication IC 162, and the device
ID chip 158. An example of the broadband communication IC 162 is
the SIM card. The processor 194 is coupled to the input device 150,
the motion sensor system 152, the wireless communication IC 157,
the biometric identification scanner BIS, the broadband
communication IC 162, and the device ID chip 158.
[0079] It should be noted that components that are common between
the controller 102 of FIG. 1A and the controller 160 operate in the
same manner as that described above with respect to the controller
102 except that the components operate in the manner described
above with respect to the controller 160 instead of the controller
102. For example, the motion sensor system 152 measures data for
calculating a position and orientation of the controller 160
instead of the controller 102.
[0080] A device ID of the controller 160 is pre-registered with the
server system 104 before the session 106 (FIG. 1C) starts. For
example, the device ID unique to the controller 160 is hardwired
into the controller 160 and the server system 104 stores the device
ID. The device ID of the controller 160 is stored in the device ID
database of the server system 104.
[0081] After the session 106 is established between the computing
device 114 (FIG. 1C) and the server system 104 (FIG. 1C), the user
A provides his/her biometric ID to the biometric identification
scanner BIS of the controller 160 to pair the controller 160 with
the session 106. For example, after the session 106 starts, upon
viewing the notice on the computing device 114 that the user A
press his/her finger against a fingerprint reader on the controller
160 or that the user A speak into a microphone of the controller
160 or that the user 102 scan his/her eye into a biometric eye
scanner of the controller 160, the user A provides his/her
biometric information to the biometric information scanner BIS of
the controller 160. Upon receiving the biometric ID, the biometric
identification scanner BIS of the controller 160 generates a
biometric identification signal and sends the biometric
identification signal to the processor 194. The processor 194
receives the biometric identification signal and generates an
instruction to send the biometric identification signal to the
server system 104. For example, the instruction includes a
destination address of the server system 104. The processor 194
sends the instruction and the biometric identification signal to
the broadband communication IC 162. The broadband communication IC
162 applies the cellular communication protocol to the biometric
identification signal and the instruction to generate one or more
transfer units, and sends the transfer units via the cellular
connection channel 151 (FIG. 1C) to the tower TW, which forwards
the transfer units to the computer network 122. The computer
network 122 applies the cellular communication protocol to the
transfer units to extract the biometric ID and the instruction from
the transfer units, further applies the network communication
protocol to the biometric ID and the instruction to generate one or
more transfer packets, and sends the transfer packets via the
connection C2 to the server system 104.
[0082] As an alternative or in addition to sending the biometric
identification signal after the session 106 starts, the processor
194 of the controller 160 accesses the device ID of the controller
160 from the device ID chip 158 and generates an instruction to
send the device ID to the server system 104 to pair the controller
160 with the session 106. The processor 194 sends the instruction
and the device ID to the broadband communication IC 162. The
broadband communication IC 162 applies the cellular communication
protocol to the device ID and the instruction to generate one or
more transfer units and sends the transfer units via the cellular
connection channel 151 (FIG. 1C) to the tower TW, which forwards
the transfer units to the computer network 122. The computer
network 122 applies the cellular communication protocol to the
transfer units to extract the device ID and the instruction from
the transfer units, further applies the network communication
protocol to the device ID and the instruction to generate one or
more transfer packets, and sends the transfer packets via the
connection C2 to the server system 104.
[0083] Similarly, when the controller 160 is paired with the
session 106 to establish a direct communication between the
controller 160 and the server system 104, the processor 194 sends
the input information via the communication channel 155 to the
server system 104. For example, the processor 194 sends the input
information and an instruction to send the input information to the
wireless communication IC 156. The wireless communication IC 156
applies the wireless communication protocol to the input
information and the instruction in order to generate one or more
transfer units and sends the transfer units having the input
information and the instruction via the cellular connection channel
151 to the tower TW. The tower TW determines from the instruction
that the transfer units having the instruction and the input
information are to be sent to the server system 104 and sends the
transfer units via other cellular towers to the computer network
122. The computer network 122 applies the cellular communication
protocol to the transfer units to extract the input information and
the instruction from the transfer units, further applies the
network communication protocol to the input information and the
instruction to generate multiple transfer packets, and sends the
transfer packets via the connection C2 to the server system
104.
[0084] In one embodiment, the broadband communication IC 162 and
the wireless communication IC 157 are integrated into a single
integrated circuit chip. In an embodiment, the broadband
communication IC 162 is integrated into one integrated circuit chip
and the wireless communication IC 157 is integrated into another
integrated circuit chip.
[0085] In an embodiment, one controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. In this embodiment, the processor 194 of the controller
determines whether a handover from using the connection channel 111
(FIG. 1A) to using the cellular network 154 (FIG. 1C) or vice versa
is to be performed. For example, the processor 194 determines that
the cellular network 154 has better quality of service (QoS) than
the connection channel 111 or that there is a lower latency in
transferring data via the cellular network 154 compared to the
connection channel 111. To illustrate, the processor 194 determines
that the cellular network 154 has a lower amount of data loss or a
shorter ping time compared to an amount of loss of data sent via
the connection channel 111. Upon determining that the cellular
network 154 has better QoS, the processor 194 determines to use the
broadband communication IC 162 instead of the wireless
communication IC 156 for transferring data, described herein, such
as the identification information, from the controller to the
server system 104. To illustrate, when the user A uses the
controller in his/her car, the cellular network 154 allow for
better QoS and lower latency compared to when the user A uses the
controller in his/her home. When the user A uses the controller in
his/her home, the connection channel 111 allows for better QoS and
lower latency compared to the cellular network 154.
[0086] It should be noted that in the embodiment, the processor 194
determines the amount of data loss by sending a pre-determined
number of transfer units to provide to the server system 104 via
the communication channel 155 (FIG. 1C) and requesting one or more
processors, described herein, of the server system 104 for a count
of the transfer units received by the server system 104. Similarly,
the processor 194 determines the amount of transfer packets lost by
sending a pre-determined number of transfer units to provide to the
server system 104 via the communication channel 110 (FIG. 1A) and
requesting one or more processors, described herein, of the server
system 104 for a count of the transfer packets received by the
server system 104 via the communication channel 110. The processor
194 compares the amount of transfer units lost with the amount of
transfer packets lost to determine the lower amount of loss of
data. Also, the processor 194 determines the latency by sending a
pre-determined number of transfer units to provide to the server
system 104 via the communication channel 155 and requesting one or
more processors, described herein, of the server system 104 to send
the transfer units back to the processor 194 via the communication
channel 155. By determining an amount of time for sending and
receiving the transfer units via the communication channel 155, the
latency is determined. Similarly, the processor 194 determines the
latency by sending a pre-determined number of transfer packets to
provide to the server system 104 via the communication channel 110
and requesting one or more processors, described herein, of the
server system 104 to send the transfer packets back to the
processor 194 via the communication channel 110. By determining an
amount of time for sending and receiving the transfer packets via
the communication channel 110, the latency is determined. Both the
latencies are compared by the processor 194 to determine the lower
latency.
[0087] In an embodiment, one controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. The processor 194 of the controller determines power cost
associated with using the cellular network 154 compared to using
the connection channel 111. For example, the processor 194 queries
a battery of the controller after a pre-determined time period for
which the wireless communication IC 156 (FIG. 1B) is used and the
broadband communication IC 162 (FIG. 1D) is not used. The battery
is coupled to the processor 194 and provides power to all
components, described herein, of the controller. Also, the
processor 194 queries the battery after the same amount of the
pre-determined time period for which the broadband communication IC
162 is used and the wireless communication IC 156 (FIG. 1B) is not
used. The processor 194 determines whether power of the battery is
used more in the pre-determined time period for which the wireless
communication IC 156 is used and the broadband communication IC 162
is not used or in the pre-determined time period for which the
broadband communication IC 162 is used and the wireless
communication IC 156 is not used. Upon determining that the battery
is used more in the pre-determined time period for which the
wireless communication IC 156 is used and the broadband
communication IC 162 is not used, the processor 194 determines that
the power cost associated using the connection channel 111 is
greater than that associated with using the cellular network 154.
On the other hand, upon determining that the battery is used more
in the pre-determined time period for which the broadband
communication IC 162 is used and the wireless communication IC 156
is not used, the processor 194 determines that the power cost
associated using the cellular network 154 is greater than that
associated with using the connection channel 111. The processor 194
uses either the wireless communication IC 156 or the broadband
communication IC 162 for which the power cost is lower.
[0088] In one embodiment, one controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. The processor 194 displays a message to the user A via a
display screen of the controller or provides the message in the
form of audio data to be output as sound from one or more speakers
of the controller to receive a selection from the user A whether
the cellular network 154 or the connection channel 111 is to be
used. The user A may decide to use either the cellular network 154
or the connection channel 111 to obtain better QoS to positively
impact his/her playtime. Upon receiving the selection from the user
A via the input device 150 indicating that the cellular network 154
be used, the processor 194 determines to use the broadband
communication IC 162 for transferring data, described herein, to
the server system 104. On the other hand, upon receiving the
selection from the user A via the input device 150 indicating that
the connection channel 111 be used, the processor 194 determines to
use the wireless communication IC 156 for transferring data,
described herein, to the server system 104.
[0089] In an embodiment, the controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. In this embodiment, there is no handover, described above.
Rather, the processor 194 determines to use both the wireless
communication IC 156 and the broadband communication IC 162 to
connect to the server system 104. The processor 194 sends the same
data, described herein, such as the identification information, via
the wireless communication IC 156 and the broadband communication
IC 162 to the server system 104. The server system 104 processes
the same data received first, either from the wireless
communication IC 156 or the broadband communication IC 162. For
example, a communication interface of the server system 104
receives one or more transfer packets having a packet ID or a
timestamp from the wireless communication IC 156 and receives one
or more transfer packets having the same packet ID or the same
timestamp from the broadband communication IC 162 and processes the
one or more transfer packets received first.
[0090] FIG. 1E is a diagram of an embodiment of the server system
104 to illustrate pairing of the session 106 (FIG. 1A) with a
controller 164. The controller 102 of FIG. 1A and the controller
160 of FIG. 1C are examples of the controller 164. The server
system 104 includes an authentication processor 176, a session
linking processor 178, the device ID database, the network ID
database, a user account database, the mapping database, a session
ID database, the biometric ID database, a pairing notification
processor 180, a communication interface 170, a session ID
processor 183, and an execution server ES1.
[0091] The communication interface 170 is coupled to the session ID
processor 183, the pairing notification processor 180, the
authentication processor 176 and the execution server ES1. The
pairing notification processor 180 is coupled to the authentication
processor 176 and the session linking processor 178. The
authentication processor 176 is coupled to the session linking
processor 178 and to the mapping database, which is stored in one
or more memory devices of the server system 104. The mapping
database is coupled to the device ID database, the user account
database, the biometric ID database, and the network ID database.
The session linking processor 178 is coupled to the session ID
database. The session ID processor 183 is coupled to the execution
server ES1, the session ID database, and to the session linking
processor 178.
[0092] The server system 104 is coupled via a communication network
177 to the controller 164. Examples of the communication network
177 include the communication channel 110 of FIG. 1A and the
communication channel 155 of FIG. 1C. Additional examples of the
communication network 177 include the communication channel 108 of
FIGS. 1A and 1C. To illustrate, the communication network 177
includes the communication channels 110 and 108 or the
communication channels 155 and 108.
[0093] The server system 104 is coupled to a display device 166,
such as an LED display device, an LCD display device, or a plasma
display device. The display device 166 can be an input device, such
as a touchscreen display, that receives a selection from an
administrative user. An example of the display device 166 is the
computing device 114 of FIGS. 1A and 1C.
[0094] The execution server ES1 has an input channel 184, such as a
computer port, which is coupled via the session ID processor 183 to
the communication interface 170 for receiving data from the
communication interface 170. The execution server ES1 further has
an execution engine 186 that executes the application 1, such as a
video conferencing application or a game, examples of which include
a video game, a virtual reality game, and an augmented reality
game. Examples of an engine, as described herein, include a
processor and a computer software module. An application is
executed, for example, to change the state of the virtual scene
displayed on the computing device 114 or on another display screen,
described herein. Examples of the state of the virtual scene
include a position of a virtual object in the virtual scene to be
displayed on the client, an orientation of the virtual object, a
color of the object, a shade of the virtual object, a texture of
the virtual object, a position of a background in the virtual
scene, an orientation of the background, a color of the background,
a shade of the background, a texture of the background, or a
combination of two or more thereof. One or more video frames, or
one or more audio frames, or a combination thereof, are generated
by the execution engine ES1 and the one or more frames include the
state or a change to the state. The one or more frames that are
generated are sometimes referred to herein as a video output.
Examples of the video frames include I-frames, P-frames, and
B-frames.
[0095] The communication interface 170 can be a network interface,
such as a network interface controller or a network interface card,
that applies the network communication protocol to transfer packets
that are received via the communication network 177 from the
controller 164 to extract data from the transfer packets, and
applies the network communication protocol to data to generate one
or more transfer packets that are to be transferred via the
communication network 177 to the controller 164. An example of the
communication interface 170 includes a processor that is coupled to
the network interface.
[0096] Examples of the encoder 188 include a processor or a
computer software module that performs compression of frames. For
example, the encoder 188 receives frames from the execution engine
186 and performs intraframe or interframe compression by applying a
compression protocol, such as H.264 or another frame compression
standard.
[0097] In an operation 1, the display device 166 sends one or more
transfer packets having the login information via the communication
network 177 to the communication interface 170. The communication
interface 170 applies the network communication protocol to the
transfer packets to extract the login information from the transfer
packets. A processor of the communication interface 170 identifies
information extracted from the transfer packs as being the login
information and upon identifying so, provides the login information
to the session ID processor 183. For example, the processor of the
communication interface 170 determines that a structure, such as a
series of alphanumeric characters or a structure of a username or a
structure of a password, of the information received from the
transfer packets matches that of the login information and
determines that the information is login information.
[0098] The session ID processor 183 determines whether the login
information is authentic and establishes the session 106 of
execution of the application 1 upon determining that the login
information is authentic. Also, the session ID processor 183 allows
access to the user account 1 upon determining that the login
information is authentic. An example of the session 1 is a
temporary and interactive interchange or communication of data or
information between the display device 166 and the server system
102. The session 106 ends when the user A logs out of the user
account 1 or there is a loss of connection between the server
system 102 and the display device 166. The loss of connection
between the server system 102 and the display device 166 can be due
to a malfunction of the communication channel 110 of FIG. 1A or the
communication channel 155 of FIG. 1C. On the other hand, the
session ID processor 183 does not establish the session 106 upon
determining that the login information is not authentic.
[0099] The session ID processor 183 assigns a session ID1, such as
an alphanumeric number, to the session 106, links the session ID
with the user account 1 assigned to the user A, and stores the
session ID1 in the session ID database. The session ID1 is linked
to the user account 1 by establishing a one-to-one relationship
between the session ID1 and the user account 1 for the session and
the link is stored by the session ID processor 183 in the mapping
database.
[0100] The input channel 184 receives a determination from the
session ID processor 183 that the login information is authentic
and provides the determination to the game execution engine 186.
Upon receiving the determination that the login information is
authentic, the game execution engine 186 generates one or more
frames, such as audio frames, or video frames, or a combination
thereof, of the information associated with the session 106. To
illustrate, the game execution engine 186 generates the frames that
include a name of the application 1 or a title of the application 1
or a virtual image of the application 1. The frames are encoded by
the encoder 188 and sent to the communication interface 170 in an
operation 2 to output encoded frames. The communication interface
170 applies the network communication protocol to the encoded
frames to generate a stream having one or more transfer packets and
sends the stream via the communication network 177 to the display
device 166. Examples of the stream sent from the communication
interface 170 include a video stream or an audio stream or a
combination thereof.
[0101] After receiving the information associated with the session,
in an operation 3, the controller 164 sends the identification
information, such as a biometric ID1 or a device ID1 or a network
ID1 or a combination of two or more thereof, via the communication
network 177 to the communication interface 170 to pair the
controller 164 with the session 106. The communication interface
170 applies the network communication protocol to one or more
transfer packets that include the identification information to
extract the identification information from the transfer packets,
sends the identification information to the authentication
processor 176.
[0102] The processor of the communication interface 170 identifies
information extracted from the transfer packets as being the
identification information and upon identifying so, provides the
identification information to the authentication processor 176. For
example, the processor of the communication interface 170
determines that a structure, such as a series of phonemes or a
fingerprint or an IP address or a device ID, of the information
received from the transfer packets matching that of the
identification information and determines that the information is
the identification information.
[0103] Upon receiving the biometric ID1 from the communication
interface 170, the authentication processor 176 retrieves a
pre-registered biometric ID1 from the biometric ID database and
processes the received biometric ID1 to determine whether the
received biometric ID1 is authentic. For example, the
authentication processor 176 determines whether there is a match
between the received biometric ID1 and the pre-registered biometric
ID1. The pre-registered biometric ID1 is pre-registered with the
user account 1 assigned to the user A by being stored in the
biometric ID database and is linked to the user account 1. Upon
determining that the match occurs, the authentication processor 176
determines that the received biometric ID1 is authentic to verify
the received biometric ID1. On the other hand, upon determining
that the match does not occur, the authentication processor 176
determines that the received biometric ID1 not authentic. In the
same manner, upon receiving the device ID1 from the communication
interface 170, the authentication processor 176 retrieves a
pre-registered device ID1 from the device ID database and processes
the received device ID1 to determine whether the received device
ID1 is authentic. Moreover, in the same manner, upon receiving the
network ID1 from the communication interface 170, the
authentication processor 176 retrieves a pre-registered network ID1
from the network ID database and processes the received network ID1
to determine whether the received network ID1 is authentic.
[0104] It should be noted that the mapping database pre-stores an
association, such as a one-to-one correspondence or a link or a
mapping, between the pre-registered biometric ID1 and the user
account 1. For example, before the biometric ID1 is received from
the controller 164 to pair the controller 164 with the session 106,
the session ID processor 183 receives the pre-registered biometric
ID1 from the computing device 114 via the communication channel 108
of FIG. 1A or 1C, and stores the pre-registered biometric ID1 in
the biometric ID database. In addition, the session ID processor
183 establishes the association between the pre-registered
biometric ID1 and the user account 1, stores the association within
the mapping database, and stores the pre-registered biometric ID1
within the biometric ID database. In a similar manner, before the
network ID1 and the device ID1 are received from the controller 164
to pair the controller 164 with the session 106, the session ID
processor 183 stores an association between the pre-registered
device ID1 and/or the user account 1 within the mapping database
and an association between the pre-registered network ID1 and the
user account 1 within the mapping database, stores the
pre-registered network ID1 within the network ID database, and
stores the pre-registered device ID1 within the device ID database.
The user account 1 and the other user accounts 2 through N are
stored within the user account database.
[0105] The authentication processor 176 provides the determination
that the received biometric ID1 is authentic to the session linking
processor 178. The session linking processor 178 establishes an
association between the session ID1 and the received biometric ID1
and stores the association within the mapping database to pair the
session 116 with the controller 164. For example, the session
linking processor 178 generates a one-to-one link or a mapping or a
correspondence between the session ID1 of the session 106 and the
received biometric ID1. In the same manner, the session linking
processor 178 establishes an association between the session ID1
and the received device ID1 and stores the association within the
mapping database. As an example, the session linking processor 178
generates a one-to-one link or a mapping or a correspondence
between the session ID1 of the session 106 and the received device
ID1. Also, in the same manner, the session linking processor 178
establishes an association between the session ID1 and the received
network ID1 and stores the association within the mapping database.
For example, the session linking processor 178 generates a
one-to-one link or a mapping or a correspondence between the
session ID1 of the session 106 and the received network ID1.
[0106] Upon establishing the association between the identification
information that is received from the controller 164 and the
session 106, in an operation 4, the session linking processor 178
provides a determination of the association to the pairing
notification processor 180. In response to receiving the
determination of the association, the pair notification processor
180 generates the pairing notification and sends the pairing
notification to the communication interface 170. The pairing
notification processor 180 also generates an instruction to send
the pairing notification to the display device 166 or an
instruction to send the pairing notification to the controller 164
or to both the display device 166 and the controller 164. The
communication interface 170 receives the instruction from the
pairing notification processor 180, applies the network
communication protocol to the pairing notification and to a
destination address of the display device 166 or to a destination
address of the controller 164 or to both the destination addresses
to generate one or more transfer packets, and sends the transfer
packets via the communication network 177 to the display device 166
or to the controller 164 or to both the controller 164 and the
display device 166.
[0107] When the authentication processor 176 determines that the
received biometric ID1 or the received device ID1 or the received
network ID1 or a combination of two or more thereof is not
authentic, the authentication processor 176 informs the pairing
notification processor 180 of the determination. Upon receiving the
determination, the pairing notification processor 180 generates a
not-paired notification and an instruction to send the not-paired
notification to the display device 166 or the controller 164 or
both the controller 164 and the display device 166. Upon receiving
the instruction and the not-paired notification, the communication
interface 170 applies the network communication protocol to a
destination address of the display device 166 or a destination
address of the controller 164 or both the destination addresses and
to the not-paired notification to generate one or more transfer
packets and sends the transfer packets via the communication
network 177 to the display device 166 or to the controller 164 or
to both the controller 164 and the display device 166.
[0108] The communication interface 170 sends the transfer packets
including a pairing-related notification, such as the not-paired
notification or the pairing notification, via the connection C2
(FIG. 1A), the computer network 122 (FIG. 1A), the connection C1
(FIG. 1A), the modem 120 (FIG. 1A), the cable EC (FIG. 1A), the
router 108 (FIG. 1A), and the connection channel CC2 (FIG. 1A) to
the display device 166. The modem 120 of FIGS. 1A and 1C receives
the transfer packets including the pairing-related notification and
the destination address of the display device 166. The modem 120
applies the network communication protocol to the transfer packets
to extract the destination address of the display device 166 and
the pairing-related notification, and sends the pairing-related
notification and the destination address to the router 118 of FIGS.
1A and 1C. The router 118 determines from the destination address
of the display device 166 that the pairing-related notification is
to be sent to the display device 166, and sends the pairing-related
notification to the display device 166. For example, the router 118
identifies from an IP address of the display device 166 that the
pairing-related notification is to be sent to the display device
166. Upon receiving the pairing-related notification, a GPU of the
display device 166 displays the pairing-related notification on a
display screen of the display device 166. In case the
pairing-related notification is audio data, an audio processor of
the display device 166 outputs the pairing-related notification as
sound. Similarly, when the pairing-related notification includes
both image and audio data, both the GPU and the audio processor of
the display device 166 synchronize with each other to output the
pairing-related notification as images and sound.
[0109] In one embodiment, the communication interface 170 sends the
transfer packets including the pairing-related notification via the
connection C2, the computer network 122, the connection C1, the
modem 120, the cable EC, the router 108, and the connection channel
CC1 to the controller 102 of FIG. 1A. The modem 120 of FIGS. 1A and
1C receives the transfer packets including the pairing-related
notification and the destination address of the display device 166.
The modem 120 applies the network communication protocol to the
transfer packets to extract the destination address of the
controller 102 and the pairing-related notification, and sends the
pairing-related notification and the destination address to the
router 118 of FIGS. 1A and 1C. The router 118 determines from the
destination address of the controller 102 that the pairing-related
notification is to be sent to the controller 102, and sends the
pairing-related notification to the controller 102 via the
connection channel CC1. For example, the router 118 identifies from
an IP address of the controller 102 that the pairing-related
notification is to be sent to the controller 102. Upon receiving
the pairing-related notification, a GPU of the controller 102
displays the pairing-related notification on a display screen of a
display device of the controller 102. In case the pairing-related
notification is audio data, an audio processor of the display
device of the controller 102 outputs the pairing-related
notification as sound. Similarly, when the pairing-related
notification includes both image and audio data, both the GPU and
the audio processor of the controller 102 synchronize with each
other to output the pairing-related notification as images and
sound.
[0110] In an embodiment, the communication interface 170 sends the
transfer packets including the pairing-related notification via the
connection C2, the computer network 122, the connection C1, and the
cellular network 154 (FIG. 1C) to the controller 160 of FIG. 1C.
The computer network 122 of FIG. 1C receives the transfer packets
including the pairing-related notification and the destination
address of the controller 160. The computer network 122 applies the
network communication protocol to the transfer packets to extract
the destination address of the controller 160 and the
pairing-related notification, further applies the cellular
communication protocol to the destination address and the
pairing-related notification to generate one or more transfer
units, and sends the transfer units to the tower TW of the cellular
network 154 via the cellular connection channel 153. The tower TW
forwards the transfer units to the controller 160 of FIG. 1C via
the cellular connection channel 151 to the controller 160 according
to the destination address of the controller 160. The controller
160 applies the cellular communication protocol to obtain the
pairing-related notification. Upon receiving the pairing-related
notification, a GPU of the controller 160 displays the
pairing-related notification on a display screen of a display
device of the controller 160. In case the pairing-related
notification is audio data, an audio processor of the display
device of the controller 160 outputs the pairing-related
notification as sound. Similarly, when the pairing-related
notification includes both image and audio data, both the GPU and
the audio processor of the controller 160 synchronize with each
other to output the pairing-related notification as images and
sound.
[0111] In an operation 5, when the controller 164 is paired, such
as linked, with the session 106, the controller 164 can be used by
the user A to generate one or more transfer packets having the
input information and the session ID1 to change the state of the
virtual scene. For example, a selection of a button on the
controller 164 by the user A enables a virtual weapon to fire or a
virtual user to jump in a video game. As another example, a
selection of a joystick on the controller 164 by the user A enables
a change a number of areas on a display screen on the display
device 166 in which live streams showing images of different users
are displayed. To illustrate, instead of two live streams showing
two users from two different locations on the display screen of the
display device 166, a live stream of one of the two users is
received by the display device 166 for display on the display
screen.
[0112] The communication interface 170 receives the transfer
packets that include the input information and the session ID1 from
the controller 164 via the communication network 177 and applies
the network communication protocol to the transfer packets to
extract the input information and the session ID1. The processor of
the communication interface 170 determines that information within
the transfer packets have a structure, such as a series of button
presses or a series of joystick movements, that matches a structure
of the input information and upon determining so, sends the input
information to the session ID processor 183. The session ID
processor 183 provides the input information via the input channel
184 to the execution engine 186. The session ID processor 183 can
authenticate the session ID1 before sending the input information
to the execution engine 186.
[0113] The execution engine 186 is interacted with by the
controller 164 to determine to change the state of the virtual
scene according to the input information and generates multiple
frames, such as audio frames or video frames or a combination
thereof, having the state of the virtual scene. For example, the
execution engine 186 determines to change a position, or in
orientation, or an intensity, or a color, or a shape, or a
combination therefore of the virtual object based on the input
information to generate the frames. Examples of the frames includes
I frames, P frames, and B frames. The execution engine 186 provides
the frames to the encoder 188, which compresses the frames to
output the encoded frames. For example, the frames are compressed
using the H.264 standard. In addition, the execution engine 186
generates an instruction to send the frames to the display device
166.
[0114] The encoder 180 provides the encoded frames to the
communication interface 170. The communication interface 170
applies the network communication protocol to the encoded frames
and the instruction to generate one or more transfer packets and
sends the transfer packets via the connection C2 of FIGS. 1A and
1C, the computer network 122, and the connection C1 of FIGS. 1A and
1C to the modem 120 of FIGS. 1A and 1C. The modem 120 applies the
network communication protocol to the transfer packets to extract
the instruction and the encoded frames from the transfer packets
and sends the instruction and the encoded frames to the router 118
of FIGS. 1A and 1C via the Ethernet connection EC. The router 118
determines from the destination address within the instruction that
the encoded frames are to be sent to the display device 166, and
sends the encoded frames to the display device 166 via the
connection channel CC2 of FIGS. 1A and 1C. The display device 166
decodes the encoded frames to output frames to further display
images of the virtual scene, having the virtual object and the
virtual background, on the display screen of the display device 166
or output audio data via speakers of the display device 166 or a
combination thereof. For example, the display device 166 displays
the virtual object that has changed its position or its orientation
or its color or its intensity or its texture. As another example,
the display device 166 outputs a sound made by the virtual object.
When the display device 166 displays the images or outputs a sound
associated with the images according to the encoded frames received
from the server system 104 in response to the input information
received from the controller 164, the server system 104 or the
application 1 and the controller 164 interact with each other.
[0115] It should be noted that in an embodiment, instead of each
processor of the server system 104, a software module, such as a
part of a computer software program, is used and the software
modules are executed by one or more processors of the server system
104. Each software module can contain one or several routines. In
one embodiment, functions of some processors of the server system
104 are executed as modules and the remaining processors of the
server system 104 are hardware components, such as integrated
circuits.
[0116] In an embodiment, functions, described herein, as being
performed by the authentication processor 176, the session ID
processor 183, the processor of the communication interface 170,
the pairing notification processor 180, the encoder 188, and the
execution engine 186, are instead performed by another other number
of processors or servers of the server system 104.
[0117] In one embodiment, the user A decides whether to pair the
controller 164 with the session 164. Instead of sending the
determination that the biometric ID1 or the received device ID1 or
received network ID1 or a combination of two or more thereof is
authentic to the session linking processor 178, the authentication
processor 176 sends the determination regarding the authentication
to the pairing notification server 180. Upon receiving the
determination from the authentication processor 176, the pairing
notification processor 180 generates a pairing-request notification
and an instruction to send the pairing-request notification to the
display device 166. The pairing-request notification includes a
request to pair the controller 164 with the session 106. The
pairing notification processor 180 sends the pairing-request
notification and the instruction to the communication interface
170.
[0118] In this embodiment, upon receiving the instruction and the
pairing-request notification, the communication interface 170
applies the network communication protocol to a destination address
of the display device 166 and to the pairing-request notification
to generate one or more transfer packets and sends the transfer
packets via the communication channel 108 of FIGS. 1A and 1C to the
display device 166.
[0119] Continuing with the embodiment, the communication interface
170 sends the transfer packets via the connection C2, the computer
network 122, the connection C1, the modem 120, the cable EC, the
router 108, and the connection channel CC2 to the display device
166. The modem 120 of FIGS. 1A and 1C receives transfer packets
including the pairing-request notification and a destination
address of the display device 166. The modem 120 applies the
network communication protocol to the transfer packets to extract
the destination address of the display device 166 and the
pairing-request notification, and sends the pairing-request
notification and the destination address to the router 118 of FIGS.
1A and 1C. The router 118 determines from the destination address
of the display device 166 that the pairing-request notification is
to be sent to the display device 166, and sends the pairing-request
notification to the display device 166. For example, the router 118
identifies from an IP address of the display device 166 that the
pairing-request notification is to be sent to the display device
166. Upon receiving the pairing-request notification, the GPU of
the display device 166 displays the pairing-request notification on
a display screen of the display device 166. In case the
pairing-request notification is audio data, the audio processor of
the display device 166 outputs the pairing-request notification as
sound. Similarly, when the pairing-request notification includes
both image and audio data, both the GPU and the audio processor of
the display device 166 synchronize with each other to output the
pairing-request notification as images and sound.
[0120] Moreover, in the embodiment, upon listening or viewing or
both viewing and listening to the pairing-request notification, the
user A makes a selection on the controller 164 to indicate whether
to pair the controller 164 with the session 106. For example, the
user A selects a first button on the controller 164 to provide a
positive response to indicate that the user A wishes to pair with
the session 106. The user A selects a second button on the
controller 164 to provide a negative response to indicate that the
user A does not wish to pair with the session 106.
[0121] In the embodiment, the controller 164 generates input data
indicating the selection made by the user A regarding the pairing.
The wireless communication IC 157 (FIG. 1B or 1D) of the controller
164 sends the input data via the wireless connection 115 between
the controller 164 and the display device 166 to the display device
166. The GPU of the display device 166 receives the input data and
displays the positive or the negative response on the display
device 166.
[0122] In the embodiment, the controller 164 also sends the input
data via the communication channel 110 of FIG. 1A or the
communication channel 155 of FIG. 1C to the server system 104. For
example, the input data is sent from the controller 164 via the
connection channel CC1 of FIG. 1A to the router 108 of FIG. 1A,
which routes the input data via the cable EC to the modem 120 of
FIG. 1A. The modem 120 generates one or more transfer packets by
applying the network communication protocol to the input data and
sends the transfer packets via the connection C1 of FIG. 1A, the
computer network 122 of FIG. 1A, and the connection C2 of FIG. 1A
to the server system 104.
[0123] Furthermore, in the embodiment, the communication interface
170 receives the transfer packets having the input data and applies
the network communication protocol to extract the input data from
the transfer packets. The processor of the communication interface
170 identifies information within the transfer packets as having a
structure of the input data, and upon identifying so, sends the
input data to the session linking processor 178. The session
linking processor 178 determines whether the response within the
input signal is positive or negative. In response to receiving the
input data having the positive response, the session linking
processor 178 pairs the controller 164 with the session 106. On the
other hand, in response to receiving the input data having the
negative response, the session linking processor 178 does not pair
the controller 164 with the session 106. In this manner, the user A
makes a decision whether to pair the controller 164 with the
session 106.
[0124] In one embodiment, the controller 164 includes components of
both the controllers 102 and 160. For example, the processor 194 is
coupled to both the broadband communication IC 162 of FIG. 1D and
the wireless communication IC 156 of FIG. 1B. In this example, the
processor 194 of the controller 164 sends the input data via the
wireless communication IC 156 and the communication channel 110 of
FIG. 1A and via the broadband communication IC 162 and the
communication channel 155 of FIG. 1C to the server system 104. In
an embodiment, the pairing-request notification is sent from the
server system 104 to the controller 164 via the communication
network 177 for display on a display screen of the controller 164
or for output via one or more speakers of the controller 164.
[0125] It should be noted that in one embodiment, the
identification information is different from the login information.
For example, the login information is a series of alphanumeric
characters, such as a username or a password, that are provided or
entered by the user A via an input device of a controller to the
server system 104. The identification information is not the series
of alphanumeric characters that are provided or entered by the user
A via the input device of the controller to the server system
104.
[0126] In an embodiment, until the controller 102 is paired with
the server system 104, the user A cannot use the controller 102 to
change the state of the virtual scene. For example, the user A uses
the controller 102 by selecting or moving the input device 150 to
provide a selection to further generate the input information to
change the state of the virtual scene and one or more transfer
packets having the input information are generated and sent via the
communication network 177 to the communication interface 170. The
communication interface 170 applies the network communication
protocol to extract the input information for changing the state of
the virtual scene and provides the input information to the session
ID processor 183. The processor of the communication interface 170
determines that information within the transfer packets have a
structure, such as a series of button presses or a series of
joystick movements, that matches a structure of the input
information for changing the state of the virtual scene and upon
determining so, sends the input information to the session ID
processor 183.
[0127] In this embodiment, upon receiving the input information,
the session ID processor 183 sends a request to the session linking
processor 178 to determine whether the controller 164 from which
the input information is received is paired with the session 106.
Upon receiving a determination from the session linking processor
178 that the controller 164 is not paired with the session 106, the
session ID processor 183 does not provide the input information via
the input channel 184 to the execution engine 186 and the execution
engine 186 cannot process the input information to change the state
of the virtual scene.
[0128] In an embodiment, one controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. In this embodiment, the communication interface 170 determines
whether a handover from using the connection channel 111 (FIG. 1B)
to using the cellular network 154 (FIG. 1C) or vice versa is to be
performed. For example, the processor of the communication
interface 170 determines that the cellular network 154 has better
QoS than the connection channel 111 or that there is a lower
latency in transferring data via the cellular network 154 compared
to the connection channel 111. To illustrate, the processor of the
communication interface 170 determines that the cellular network
154 has a lower amount of packet loss or a shorter ping time
compared to the connection channel 111. Upon determining that the
cellular network 154 has better QoS, the processor of the
communication interface 170 determines to use the broadband
communication IC 162 instead of the wireless communication IC 156
for transferring data, described herein, such as the pairing
information or a pairing-request notification, from the server
system 104 to the controller.
[0129] In one embodiment, the controller includes both the wireless
communication IC 156 of FIG. 1B and the broadband communication IC
162 in addition to remaining components illustrated in FIG. 1B or
1D. In this embodiment, there is no handover, described above.
Rather, the processor of the communication interface 170 determines
to use both the wireless communication IC 156 and the broadband
communication IC 162 to connect to the controller. The processor of
the communication interface 170 sends the same data, described
herein, such as the pairing information or a pairing-request
notification, via the wireless communication IC 156 and the
broadband communication IC 162 to the controller. The processor 194
processes the data received first, either by the wireless
communication IC 156 or the broadband communication IC 162, from
the server system 104. For example, the wireless communication IC
156 receives one or more transfer packets having a packet ID or a
timestamp from the server system 104 and the broadband
communication IC 162 receives one or more transfer units having the
same packet ID or the same timestamp from the server system 104.
When the wireless communication IC 156 receives the one or more
transfer packets first compared to the reception of the transfer
units by the broadband communication IC 162, the wireless
communication IC 156 extracts information from the one or more
transfer packets and sends the information to the processor 194 for
processing.
[0130] In an embodiment, the computing device 114 (FIG. 1A) can be
activated from a sleep mode when a selection of one or more of the
input devices 150 (FIG. 1B) is made by the user A on the controller
164. The server system 104 wakes up the computing device 114 when
the user A turns selects a PS.TM. button the controller 164 or
selects the fingerprint pad on the controller 164. In the sleep
mode, the computing device 114 operates in a low power state in
which it still can receive messages from the server system 104.
[0131] FIG. 1F-1 is a diagram of an embodiment of a pairing-request
notification 190 that is displayed on the display screen 116 of the
computing device 114. The pairing-request notification 190 is an
example of the pairing-request notification that is generated by
the pairing notification processor 180 of FIG. 1E. The
pairing-request notification 190 includes that the biometric
information of the user A is authenticated. Moreover, the
pairing-request notification 190 includes a question regarding
whether the user A wishes to pair with the controller 164 of FIG.
1E. The user A selects one or more buttons and one or more
joysticks on the controller 164 to indicate whether the user A
wishes to pair the session 106 and the controller 164. The
controller 164 generates the input data indicating the selection
made by the user A and.
[0132] FIG. 1F-2 is a diagram of an embodiment of a controller 192
to illustrate a display of the pairing-request notification 190 on
a display screen 195 of the controller 192. Examples of a display
screen, as described herein, include a liquid crystal display
screen or a light emitting diode screen or a plasma display screen.
The controller 192 is an example of the controller 102 of FIG. 1A,
of the controller 160 of FIG. 1C, and of the controller 164 of FIG.
1E. The controller 192 has the display screen 195 on which the
pairing-request notification 190 is displayed. The controller 192
includes a GPU that is coupled to the processor 194 of FIGS. 1B and
1D and the GPU is coupled to the display screen 195.
[0133] The pairing notification processor 180 of FIG. 1E generates
the pairing-request notification 190 and an instruction to send the
pairing-request notification 190 to the controller 192. The pairing
notification processor 180 sends the pairing-request notification
and the instruction to the communication interface 170 of FIG.
1E.
[0134] Upon receiving the instruction and the pairing-request
notification 190, the communication interface 170 applies the
network communication protocol to a destination address of the
controller 192 and to the pairing-request notification 190 to
generate one or more transfer packets and sends the transfer
packets via the communication channel 110 of FIG. 1A to the
controller 192. With reference to FIG. 1A, the communication
interface 170 sends the transfer packets via the connection C2, the
computer network 122, the connection C1, the modem 120, the cable
EC, the router 108, and the connection channel CC1 to the
controller 192. The modem 120 receives the transfer packets
including the pairing-request notification 190 and a destination
address of the controller 192. The modem 120 applies the network
communication protocol to the transfer packets to extract the
destination address of the controller 192 and the pairing-request
notification 190, and sends the pairing-request notification 190
and the destination address to the router 118. The router 118
determines from the destination address of the controller 192 that
the pairing-request notification 190 is to be sent to the
controller 192, and sends the pairing-request notification to the
controller 192 via the connection channel CC1. For example, the
router 118 identifies from an IP address of the controller 192 that
the pairing-request notification 190 is to be sent to the
controller 192. Upon receiving the pairing-request notification
190, the GPU of the controller 192 displays the pairing-request
notification 190 on the display screen 195. In case the
pairing-request notification is audio data, an audio processor and
one or more speakers of the controller 192 outputs the
pairing-request notification as sound. Similarly, when the
pairing-request notification includes both image and audio data,
both the GPU and the audio processor of the controller 192
synchronize with each other to output the pairing-request
notification as images and sound.
[0135] In one embodiment in which the cellular communication
protocol of FIG. 1C is used, upon receiving the pairing-request
notification 190 and the instruction to send the pairing-request
notification 190 to the controller 160 from the pairing
notification processor 180 of the server system 104, the
communication interface 170 applies the network communication
protocol to a destination address of the controller 160 and to the
pairing-request notification 190 to generate one or more transfer
packets and sends the transfer packets via the communication
channel 155 of FIG. 1C to the controller 160. With reference to
FIG. 1C, the communication interface 170 sends the transfer packets
via the connection C2, the computer network 122, the cellular
connection channel 153, the tower TW, and the cellular connection
channel 151 to the controller 160. The gateway of the computer
network 122 applies the network communication protocol to extract
the destination address and the pairing-request notification 190
from the transfer packets, and applies the cellular communication
protocol to the destination address and pairing-request
notification 190 to generate one or more transfer units, and sends
the transfer units via the cellular communication channel 155 to
the tower TW. The transceiver of the cell tower TW forwards the
transfer units to the controller 160 via the cellular communication
channel 151 according the destination address of the controller
160. The broadband communication IC 162 (FIG. 1D) of the controller
160 applies the cellular communication protocol to the transfer
units to extract pairing-request notification 190 and provides the
pairing-request notification 190 to the GPU of the controller 160
for display of the pairing-request notification 190 on the display
screen of the controller 160. In case the pairing-request
notification 190 is to be output as sound, the broadband
communication IC 162 provides the pairing-request notification 190
to the audio processor of the controller 160 to output the
pairing-request notification 190 as sound. In case the
pairing-request notification 190 includes both image and audio
data, the broadband communication IC 162 provides the image data of
the pairing-request notification 190 to the GPU of the controller
160 and the audio data of the pairing-request notification 190 to
the audio processor of the controller 160 to synchronize output of
the audio data as sound with a display of the video data.
[0136] FIG. 1G is a diagram of an embodiment of the controller 192
to illustrate a haptic feedback device 196, an audio device 101,
and a display device 107 of the controller 192. The controller 192
has the components of the controller 102 of FIG. 1B and has
additional components, such as the haptic feedback device 196, the
audio device 101, and the display device 107.
[0137] The display device 107, the haptic feedback device 196, and
the audio device 101 are coupled to the processor 194. The display
device 107 includes a processor 109, such as a GPU, and the display
screen 195. The display screen 195 is coupled to the processor
109.
[0138] The audio device 101 includes an audio processor 105 and a
speaker 103. The speaker 103 is coupled to the audio processor 105.
The haptic feedback device 196 includes a processor 198C, a driver
198B, and a mechanical component 198A. The processor 198C is
coupled to the driver 198B, which is coupled to the mechanical
component 198A. An example of a driver includes one or more
transistors that are coupled to each other. An example of the
mechanical component 198A includes a tactile sensor, such as a
motor or a transducer.
[0139] The processor 194 receives a notification, such as the
pairing-related notification or the pairing-request notification
190, from the wireless communication IC 156 and provides the
notification to the processor 109. The processor 109 applies a
rendering operation to the notification to display the video data
on the display screen 195.
[0140] Moreover, the processor 194 receives audio data of the
notification, such as the pairing-related notification or the
pairing-request notification 190, from the wireless communication
IC 156 and provides the audio data to the audio processor 105. The
audio data of the notification has the same information that is
included within one or more images of the notification or has
additional information that is to be output as sound with the
images of the notification. The audio processor 105 processes, such
as filters or amplifies or converts from a digital to an analog
form, the audio data and provides the processed audio data to the
speaker 103. The speaker 103 outputs the processed audio data as
sound.
[0141] Also, the processor 198C receives haptic feedback data to be
output with the notification, such as the pairing-related
notification or the pairing-request notification 190, or generated
upon execution of the application 1 (FIG. 1A) from the wireless
communication IC 156 and provides the haptic feedback data to the
processor 198C. Upon receiving the haptic feedback data, processor
198C sends a signal to the driver 198B and the driver 198B outputs
a current upon receiving the signal. The mechanical component 198A
vibrates according to the current to provide haptic feedback or
tactile feedback to the user A, who is holding the controller
192.
[0142] FIG. 2 is a diagram of an embodiment of a system 200 to
illustrate a mirroring effect of the computing device 114. The
system 200 includes the components of the system 100 of FIG. 1A or
the system 159 of FIG. 1D except that the system 200 includes a
display device 202. An example of the display device 202 is a
television or a smart television or a monitor. The display device
202 has a display screen 204, such as a light emitting diode screen
or a liquid crystal display screen or a plasma display screen.
[0143] The computing device 114 communicates with the display
device 202 via a wireless or a wired connection 206. For example, a
dongle 252 is plugged into a high definition multimedia interface
(HDMI) port of the display device 202 to allow the computing device
114 to stream media, such as video and audio frames, wirelessly
from the computing device 114 to the display device 202. The video
frames of the media are displayed on the display screen 204 and the
audio frames on the media are output via one or more speakers of
the display device 202. An example of the wired connection 206
includes an HDMI cable.
[0144] In one embodiment, in addition to outputting the media on
the display device 202, the media is output by the computing device
114. For example, the video frames of the media are displayed on
the display screens 116 and 204 and the audio frames are output via
one or more speakers of the computing device 114 and one or more
speakers of the display device 202.
[0145] When the session 106 is established between the computing
device 114 and the server system 104 and before pairing of the
session 106 with the controller 164, the information associated
with the session 106, such as an image of a title of the game, is
displayed first on the display screen 116 of the computing device
114. A processor of the computing device 114 streams or sends the
media having the information associated with the session 106 to the
display device 202 via the wired or wireless connection 206. A GPU
of the display device 202 displays images of the information
associated with the session 106 on the display screen 204 and the
one or more speakers of the display device 202 output sounds having
the information associated with the session 106 or output sounds
having other information in synchronization with the information
associated with the session 106.
[0146] Moreover, when the notification, such as the pairing-related
notification or the pairing-request notification, is displayed on
the computing device 114, the notification is streamed or sent from
the processor of the computing device 114 to the display device 202
via the wired or wireless connection 206. The GPU of the display
device 202 displays the notification on the display screen 204.
Moreover, an audio processor of the display device 202 outputs
sounds of the notification via the one or more speakers of the
display device 202.
[0147] FIG. 3 is a diagram of an embodiment of a system 300 to
illustrate an execution of a discovery program 306 for discovering
the device ID of the controller 164. The computing device 114
includes a processor 302, a memory device 304, a wireless
communication IC 308, and a wireless communication IC 310. The
wireless communication IC 310 applies the wireless communication
protocol to one or more communication packets received from the
computing device 114 via the connection channel CC2 to extract data
from the communication packets and to data that is to be
transferred from the processor 302 via the connection channel CC2
to the computing device 114 to generate one or more communication
packets. Examples of the wireless communication IC 308 include a
Bluetooth.TM. device that enables communication between the
controller 164 and the computing device 114 (FIG. 1A). The
processor 302 is coupled to the memory device 304 and to the
wireless communication IC 308. The discovery program 306 is stored
within the memory device 304. The processor 302 is coupled to the
memory device 304, the wireless communication IC 310, and the
wireless communication IC 308.
[0148] When the session 106 of execution of the application 1 is
established between the computing device 114 and the server system
104 and before the session 106 is paired with the controller 164,
the processor 302 executes the discovery program 306 to request the
device ID of the controller 164 from the controller 164. The
processor 194 (FIG. 1B or 1D) of the controller 164 retrieves the
device ID from the device ID chip 158 and provides the device ID to
the wireless communication IC 157 (FIG. 1B or 1D). The wireless
communication IC 157 applies a wireless communication protocol,
such as a Bluetooth.TM. protocol, to the device ID, to generate one
or more transfer datagrams and sends the transfer datagrams via the
wireless connection 115 to the wireless communication IC 308 of the
computing device 114.
[0149] The wireless communication IC 308 applies the wireless
communication protocol to the one or more transfer datagrams
received from the controller 164 to extract the device ID from the
transfer datagrams and provides the device ID to the processor 302.
The processor 302 generates an instruction to send the device ID to
the server system 104. The processor 302 sends the instruction and
the device ID to the wireless communication IC 310, which applies
the wireless communication protocol to the device ID and the
instruction to generate one or more communication packets and sends
the communication packets via the connection channel CC2 to the
router 118.
[0150] The router 118 applies the wireless communication protocol
to the communication packets to obtain the device ID and the
instruction, directs the device ID and the instruction to the modem
120. The modem 120 applies the network communication protocol to
the device ID and the instruction to generate one or more transfer
packets and sends the transfer packets via the connection C1, the
computer network 122, and the connection C2 to the server system
104.
[0151] FIG. 4A is a diagram of an embodiment of a controller 408 to
illustrate use of a calibration processor 404 within the controller
408. The controller 408 is an example of the controller 102 of FIG.
1A or the controller 164 of FIG. 1E or the controller 192 of FIG.
1F-2. To illustrate, the controller 408 includes the components of
the controller 102. The controller 408 includes the motion sensor
system 152, the calibration processor 404, the processor 194, and
the wireless communication IC 156. The calibration processor 404 is
coupled to the motion sensor system 152 and to the processor
194.
[0152] The motion sensor system 152 measures sensor values, such as
data for calculating a position and orientation of the controller
408, or data for calculating the position and orientation of the
input device 150 of the controller 408, or a combination thereof
and provides the sensor values to the calibration processor 404.
For example, the motion sensor system 152 measures an acceleration
of the controller 408 and an orientation of the controller 408 with
respect to a reference co-ordinate system of the controller 408. As
another example, the motion sensor system 152 measures an
acceleration of the input device 150 and an orientation of the
input device 150 with respect to the reference co-ordinate system
of the controller 408.
[0153] The calibration processor 404 receives the sensor values
from the motion sensor system 152 and calibrates one or more of the
sensor values. For example, the calibration processor 404 scales,
such as multiplies, adds, or subtracts, a factor to the sensor
values to output calibrated sensor values, which are an example of
the input information described above with reference to FIG. 1B. An
example of the factor includes a real number. To illustrate, when
the controller 408 moves from a position P1 to a position P2, which
are 2 inches apart, the calibration processor 404 scales a distance
between the positions P1 and P2 to be less than 2 inches apart or
greater than 2 inches apart. As another illustration, when the
input device 150 moves from a position PO1 to a position PO2, which
are 1 centimeter apart, the calibration processor 404 scales a
distance between the positions PO1 and PO2 to be less than 1
centimeter apart or greater than 1 centimeter apart.
[0154] The calibrated sensor values are provided by the calibration
processor 404 to the processor 194, which generates an instruction
to send the calibrated sensor values to the server system 104. The
processor 194 sends the instruction and the calibrated sensor
values to the wireless communication IC 156. The wireless
communication IC 156 generates one or more transfer units
incorporating the instruction and the calibrated sensor values by
applying the network communication protocol and sends the transfer
units to the router 118. The router 118 applies the wireless
communication protocol to obtain the instruction and the calibrated
sensor values, and sends the instruction and the calibrated sensor
values to the modem 120 (FIG. 1A). The modem 120 applies the
network communication protocol to the calibrated sensor values to
generate one or more transfer packets, and sends the transfer
packets via the communication channel 108 (FIG. 1A) to the server
system 104 (FIG. 1A). The calibration of the sensor values at the
controller 408 reduces a workload of the server system 104 of FIG.
1A.
[0155] FIG. 4B is a diagram of an embodiment of a controller 450 to
illustrate use of the calibration process 404 within the controller
450. The controller 450 is an example of the controller 160 of FIG.
1C or the controller 164 of FIG. 1E or to controller 192 of FIG.
1F-2. The controller 450 includes the motion sensor system 152, the
calibration processor 404, the processor 194, and the broadband
communication IC 162.
[0156] The calibrated sensor values are provided by the calibration
processor 404 to the processor 194, which generates an instruction
to send the calibrated sensor values to the server system 104. The
processor 194 sends the instruction and the calibrated sensor
values to the broadband communication IC 162. The broadband
communication IC 162 generates one or more transfer units
incorporating the calibrated sensor values and the instruction by
applying the cellular communication protocol and sends the one or
more transfer units via the cellular network 154 (FIG. 1C) to the
server system 104 (FIG. 1C). The calibration of the sensor values
at the controller 450 reduces a workload of the server system 104
of FIG. 1C.
[0157] FIG. 5 is a diagram of an embodiment of a server system 500
to illustrate calibration of the sensor values by the calibration
processor 404 of the server system 500. Instead of the calibration
being done by a controller, the calibration is performed by the
server system 500. The server system 500 is an example of the
server system 104 of FIG. 1A or 1C. The server system 500 includes
the components of the server system 104. The server system 500
further includes the calibration processor 404, which is coupled to
the input channel 184 and to the communication interface 170.
[0158] The communication interface 170 of the server system 500
receives one or more transfer packets having the input information
that further includes the sensor values and applies the network
communication protocol to the transfer packets to extract the
sensor values. The processor of the communication interface 170
determines that information within the transfer packets matches a
structure of the sensor values and sends the sensor values to the
calibration processor 404. The calibration processor 404 receives
the sensor values and calibrates the sensor values to output the
calibrated sensor values. The calibration processor 404 provides
the calibrated sensor values via the input channel 184 to the
execution engine 186. The execution engine 186 applies the
calibrated sensor values during execution of the application 1 to
change the state of the virtual scene to generate one or more
frames for encoding.
[0159] FIG. 6 illustrates a perspective view of a video game
controller 600 for interfacing with an interactive program, such as
the application 1, in accordance with an embodiment of described in
the present disclosure. The video game controller 600 is an example
of the controller 102 of FIG. 1A or the controller 160 of FIG. 1C
or the controller 164 of FIG. 1E. The video game controller 600
includes a main body 602 and extensions 604A and 604B that extend
from the main body 602. The extension 604A and 604B are configured
to be held by the user A's left and right hands, respectively, and
thus function as handles to enable secure gripping of the video
game controller 600 by the user A. On a top surface of the main
body 602, there are included various input devices, such as buttons
606A, 606B, 606C, and 606D, joysticks 608A and 608B, and multiple
directional pads 610A, 610B, 610C, and 610D. Also shown is the top
portion of a three-dimensional (3D) control bar 612 which extends
through the main body 602 of the video game controller 600 from top
to bottom. A speaker 614 is provided for playing sounds which
provide feedback to the user A.
[0160] Additionally, the video game controller 600 includes a touch
panel 616 defined on a top side of the main body 602 which faces
the head of the user A when the extensions 604A and 604B are held
by the user A in the left and right hands, respectively. The touch
panel 616 is oriented in a substantially horizontal fashion and
situated between groups A and B so that the user A holding the
extensions 604A and 604B can readily use the touch panel 616 with
the thumb of either hand of the user A. The group A includes the
buttons 606A, 606B, 606C, and 606D and the group B includes the
directional pads 610A, 610B, 610C, and 610D. The touch panel 616
utilizes a touch-sensitive technology (e.g. resistive, capacitive,
etc.) to detect touch gestures made by the user A. In the
illustrated embodiment, the touch panel 616 provides a tactile
sensation by virtue of its shape that enables the user A to readily
determine an approximate vertical position of his/her thumb on the
touch panel 616 based on feeling alone.
[0161] The video game controller 600 includes a fingerprint pad 618
that receives a touch of one or more fingers of the user A to
output the biometric ID, described above. The fingerprint pad 618
is an example of the fingerprint reader, described above. The
fingerprint pad 618 is located between the joysticks 608A and 608B
and below the touch panel 616.
[0162] In one embodiment, the fingerprint pad 618 is located on any
other location on the controller 600. For example, the fingerprint
pad 618 is integrated within the touch panel 616. As another
example, a fingerprint pad is integrated within a surface of the
extension 604A or a surface of the extension 604B or within both
the surfaces. As yet another example, the touch panel 616 is a part
of a display screen, such as a liquid crystal display or a light
emitting diode display or a plasma display, and the fingerprint pad
618 is integrated within the display screen. As still another
example, a fingerprint pad is integrated within a top surface of
either the joystick 608A or the joystick 608B or multiple
fingerprint pads are integrated on surfaces of the joysticks 608A
and 608B. As yet another example, the fingerprint pad 618 is
located on a back side of the controller 600.
[0163] In one embodiment, the video game controller 600 includes
one or more microphones for capturing sound from the real-world
environment. As an example, the microphones are arranged as an
array of microphones. The arrangement is a linear array of
microphones. When three or more microphones are included in the
array of microphones, it is possible to determine a location of a
sound source relative to the microphone array based on analysis of
audio data captured by the microphone array. More specifically, the
sound source can be localized relative to the microphone array
based on a relative timing of its sound as captured by each of the
microphones of the microphone array. Taken in combination with the
a location and an orientation of the video game controller 600
(e.g. as determined based on the motion sensor system 152 of FIGS.
1B and 1D and tracking methods as defined elsewhere herein), and by
extension the location and orientation of the microphone array, a
location of the sound source within the interactive environment can
be determined. Furthermore, the captured sound can be processed to
exclude sounds which do not emanate from a certain region of the
real-world environment. The sound of the user A is captured by the
microphones to output the biometric information, described
above.
[0164] FIG. 7 is a flowchart to illustrate an embodiment of a
method 700 for establishing direct communication between a server
system and a video game controller. The method includes an
operation 702 of establishing a session for executing an
application for a user account over a first communication channel
The execution of the application at the server system produces a
video output that is encoded to produce a video stream that is
communicated to a display device associated with the user account
over the first communication channel. The method 700 further
includes an operation 704 of receiving, by the server system, a
unique identifier from the video game controller over a second
communication channel The method 700 also includes an operation 706
of processing, by the server system, the unique identifier to
authenticate the unique identifier. The unique identifier is
authenticated to enable a linking of the video game controller to
the session. The method 700 includes an operation 708 of using,
during the session, an input from the video game controller via the
second communication channel to control interactivity of the
application that provides the video stream over the first
communication channel to the display device.
[0165] FIG. 8 is a flowchart 800 to illustrate an embodiment of a
method 800 for establishing direct communication with a server
system. The method includes an operation 802 of sending, by a video
game controller, a unique identifier for a user account via a first
communication channel to the server system. The operation 802 of
sending the unique identifier occurs after a session for executing
an application is established over a second communication channel
between the server system and a display device. The method 800 also
includes an operation 804 of receiving an indication via the second
communication channel that the video game controller is paired with
the session when the unique identifier is authenticated. The method
800 includes an operation 806 of sending an input from the video
game controller via the first communication channel to the server
system to change a state associate with the application. The
operation 806 of sending the input occurs after the operation 804
of receiving the indication.
[0166] Embodiments described in the present disclosure may be
practiced with various computer system configurations including
hand-held devices, microprocessor systems, microprocessor-based or
programmable consumer electronics, minicomputers, mainframe
computers and the like. Several embodiments described in the
present disclosure can also be practiced in distributed computing
environments where tasks are performed by remote processing devices
that are linked through a wire-based or wireless network.
[0167] With the above embodiments in mind, it should be understood
that a number of embodiments described in the present disclosure
can employ various computer-implemented operations involving data
stored in computer systems. These operations are those requiring
physical manipulation of physical quantities. Any of the operations
described herein that form part of various embodiments described in
the present disclosure are useful machine operations. Several
embodiments described in the present disclosure also relates to a
device or an apparatus for performing these operations. The
apparatus can be specially constructed for the required purpose, or
the apparatus can be a computer selectively activated or configured
by a computer program stored in the computer. In particular,
various machines can be used with computer programs written in
accordance with the teachings herein, or it may be more convenient
to construct a more specialized apparatus to perform the required
operations.
[0168] Various embodiments described in the present disclosure can
also be embodied as computer-readable code on a non-transitory
computer-readable medium. The non-transitory computer-readable
medium is any data storage device that can store data, e.g., a RAM,
a ROM, a flash memory, a disk, etc., which can be thereafter be
read by a computer system. Examples of the computer-readable medium
include hard drives, network attached storage (NAS), ROM, RAM,
compact disc-ROMs (CD-ROMs), CD-recordables (CD-Rs), CD-rewritables
(RWs), magnetic tapes and other optical and non-optical data
storage devices. The non-transitory computer-readable medium can
include computer-readable tangible medium distributed over a
network-coupled computer system so that the computer-readable code
is stored and executed in a distributed fashion.
[0169] Although the method operations were described in a specific
order, it should be understood that other housekeeping operations
may be performed in between operations, or operations may be
adjusted so that they occur at slightly different times, or may be
distributed in a system which allows the occurrence of the
processing operations at various intervals associated with the
processing, or operations may be performed in a different order, as
long as the processing of the overlay operations are performed in
the desired way.
[0170] It should further be noted that in an embodiment, one or
more features from any embodiment described above are combined with
one or more features of any other embodiment without departing from
a scope described in various embodiments described in the present
disclosure.
[0171] Although the foregoing embodiments have been described in
some detail for purposes of clarity of understanding, it will be
apparent that certain changes and modifications can be practiced
within the scope of the appended claims. Accordingly, the present
embodiments are to be considered as illustrative and not
restrictive, and the various embodiments described in the present
disclosure is not to be limited to the details given herein, but
may be modified within the scope and equivalents of the appended
claims.
* * * * *