U.S. patent application number 12/774221 was filed with the patent office on 2011-03-17 for system and method for generating television screen pointing information using an external receiver.
Invention is credited to Jeyhan Karaoguz, Nambirajan Seshadri.
Application Number | 20110063522 12/774221 |
Document ID | / |
Family ID | 43730008 |
Filed Date | 2011-03-17 |
United States Patent
Application |
20110063522 |
Kind Code |
A1 |
Karaoguz; Jeyhan ; et
al. |
March 17, 2011 |
SYSTEM AND METHOD FOR GENERATING TELEVISION SCREEN POINTING
INFORMATION USING AN EXTERNAL RECEIVER
Abstract
A system and method, in a television receiver, for generating
screen pointing information, substantially as shown in and/or
described in connection with at least one of the figures, as set
forth more completely in the claims.
Inventors: |
Karaoguz; Jeyhan; (Irvine,
CA) ; Seshadri; Nambirajan; (Irvine, CA) |
Family ID: |
43730008 |
Appl. No.: |
12/774221 |
Filed: |
May 5, 2010 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61242234 |
Sep 14, 2009 |
|
|
|
Current U.S.
Class: |
348/734 ;
348/E5.096 |
Current CPC
Class: |
G06F 3/0304 20130101;
H04N 21/4826 20130101; H04N 21/4828 20130101; G06F 3/0308 20130101;
H04N 21/858 20130101; H04N 21/42222 20130101; H04N 21/4725
20130101; H04N 21/4728 20130101; H04N 21/4524 20130101; H04N
21/8173 20130101; H04N 21/4622 20130101; H04N 21/4782 20130101;
H04N 21/25841 20130101; G06F 3/0386 20130101; G06F 3/0346 20130101;
H04N 21/812 20130101; H04N 21/23892 20130101; H04N 5/445 20130101;
H04N 21/234318 20130101; H04N 9/8205 20130101; H04N 21/4325
20130101; H04N 21/8126 20130101; H04N 5/76 20130101; H04N 21/2668
20130101; H04N 21/438 20130101; H04N 21/472 20130101; H04N 21/4722
20130101; H04N 21/2408 20130101; H04N 21/42204 20130101; H04N
21/47805 20130101; H04N 21/44008 20130101; G06F 3/0412 20130101;
H04N 21/47815 20130101; H04N 21/42206 20130101; H04N 21/8133
20130101; H04N 21/4334 20130101; H04N 21/41265 20200801; G06F
3/0428 20130101; G06F 3/0325 20130101; H04N 21/436 20130101; H04N
21/845 20130101; H04N 21/8545 20130101; H04N 21/482 20130101; H04N
21/42209 20130101 |
Class at
Publication: |
348/734 ;
348/E05.096 |
International
Class: |
H04N 5/44 20060101
H04N005/44 |
Claims
1. A method, in a television receiver housed separately from a
television having a screen, for generating screen pointing
information, the method comprising: in the television receiver:
receiving sensor information related to a location on the screen to
which a user is pointing; processing said received sensor
information to determine said location on the screen to which the
user is pointing; and generating information indicative of said
determined location on the screen.
2. The method of claim 1, comprising, in the television receiver,
outputting a signal to the television that, when processed by the
television, causes the television to display on the screen a visual
indication of said determined location on the screen.
3. The method of claim 2, comprising, in the television receiver,
overlaying the visual indication on a television program being
output to the television for presentation on the screen.
4. The method of claim 1, wherein receiving sensor information
comprises receiving sensor information from the television.
5. The method of claim 4, wherein receiving sensor information from
the television comprises receiving sensor information associated
with one or more sensors integrated in the television screen.
6. The method of claim 4, wherein receiving sensor information from
the television comprises receiving sensor information associated
with one or more sensors integrated off-screen in a body of the
television.
7. The method of claim 1, wherein receiving sensor information
comprises receiving sensor information from one or more sensors
integrated in the television receiver.
8. The method of claim 1, wherein receiving sensor information
comprises receiving sensor information from one or more sensors
separate from both of the television and the television
receiver.
9. The method of claim 1, comprising, in the television receiver,
determining location of the user.
10. The method of claim 9, comprising, in the television receiver,
processing said received sensor information and said determined
location of the user to determine said location on the screen to
which the user is pointing.
11. The method of claim 1, comprising, in the television receiver,
performing a calibration procedure with the user to develop a
manner of processing the received sensor information to determine
said location on the screen to which the user is pointing.
12. The method of claim 1, wherein processing said received sensor
information to determine said location on the screen to which the
user is pointing comprises selecting a sensor corresponding to a
strongest sensor signal.
13. The method of claim 1, wherein processing said received sensor
information to determine said location on the screen to which the
user is pointing comprises interpolating between a plurality of
sensor signal strengths.
14. A television receiver, housed separately from a television
having a screen, that generates screen pointing information, the
television receiver comprising: at least one module operable to, at
least: receive sensor information related to a location on the
screen to which a user is pointing; process said received sensor
information to determine said location on the screen to which the
user is pointing; and generate information indicative of said
determined location on the screen.
15. The television receiver of claim 14, wherein the at least one
module comprises: a user interface module; and a processor
module.
16. The television receiver of claim 14, wherein the at least one
module is operable to output a signal to the television that, when
processed by the television, causes the television to display on
the screen a visual indication of said determined location on the
screen.
17. The television receiver of claim 16, wherein the at least one
module is operable to overlay the visual indication on a television
program being output to the television for presentation on the
screen.
18. The television receiver of claim 14, wherein the at least one
module is operable to receive sensor information by, at least in
part, operating to receive sensor information from the
television.
19. The television receiver of claim 18, wherein the at least one
module is operable to receive sensor information from the
television by, at least in part, operating to receive sensor
information associated with one or more sensors integrated in the
television screen.
20. The television receiver of claim 18, wherein the at least one
module is operable to receive sensor information from the
television by, at least in part, operating to receive sensor
information associated with one or more sensors integrated
off-screen in a body of the television.
21. The television receiver of claim 14, wherein the at least one
module is operable to receive sensor information by, at least in
part, operating to receive sensor information from one or more
sensors integrated in the television receiver.
22. The television receiver of claim 14, wherein the at least one
module is operable to receive sensor information by, at least in
part, operating to receive sensor information from one or more
sensors separate from both of the television and the television
receiver.
23. The television receiver of claim 14, wherein the at least one
module is operable to determine location of the user.
24. The television receiver of claim 23, wherein the at least one
module is operable to process said received sensor information and
said determined location of the user to determine said location on
the screen to which the user is pointing.
25. The television receiver of claim 14, wherein the at least one
module is operable to perform a calibration procedure with the user
to develop a manner of processing the received sensor information
to determine said location on the screen to which the user is
pointing.
26. The television receiver of claim 14, wherein the at least one
module is operable to process said received sensor information to
determine said location on the screen to which the user is pointing
by, at least in part, operating to select a sensor corresponding to
a strongest sensor signal.
27. The television receiver of claim 14, wherein the at least one
module is operable to process said received sensor information to
determine said location on the screen to which the user is pointing
by, at least in part, operating to interpolate between a plurality
of sensor signal strengths.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY
REFERENCE
[0001] This patent application is related to and claims priority
from provisional patent application Ser. No. 61/242,234 filed Sep.
14, 2009, and titled "TELEVISION SYSTEM," the contents of which are
hereby incorporated herein by reference in their entirety. This
patent application is also related to U.S. patent application Ser.
No. ______, filed concurrently with, titled "SYSTEM AND METHOD FOR
GENERATING SCREEN POINTING INFORMATION IN A TELEVISION", Attorney
Docket No. 21034US02; and U.S. patent application Ser. No. ______,
filed concurrently herewith, titled "SYSTEM AND METHOD FOR
GENERATING SCREEN POINTING INFORMATION IN A TELEVISION CONTROL
DEVICE", Attorney Docket No. 21036US02. The contents of each of the
above-mentioned applications are hereby incorporated herein by
reference in their entirety.
FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
[0002] [Not Applicable]
SEQUENCE LISTING
[0003] [Not Applicable]
MICROFICHE/COPYRIGHT REFERENCE
[0004] [Not Applicable]
BACKGROUND OF THE INVENTION
[0005] Present television receivers are incapable of providing
pointing information to television program viewers. Further
limitations and disadvantages of conventional and traditional
approaches will become apparent to one of skill in the art, through
comparison of such systems with the present invention as set forth
in the remainder of the present application with reference to the
drawings.
BRIEF SUMMARY OF THE INVENTION
[0006] Various aspects of the present invention provide a system
and method, in a television receiver, for generating screen
pointing information, substantially as shown in and/or described in
connection with at least one of the figures, as set forth more
completely in the claims. These and other advantages, aspects and
novel features of the present invention, as well as details of
illustrative aspects thereof, will be more fully understood from
the following description and drawings.
BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS
[0007] FIG. 1 is a diagram illustrating an exemplary television
system in accordance with various aspects of the present
invention.
[0008] FIG. 2 is a diagram illustrating an exemplary television
receiver in accordance with various aspects of the present
invention.
[0009] FIG. 3 is a diagram illustrating an exemplary television
system with on-screen television sensors in accordance with various
aspects of the present invention.
[0010] FIG. 4 is a diagram illustrating an exemplary television
system with off-screen television sensors in accordance with
various aspects of the present invention.
[0011] FIG. 5 is a diagram illustrating an exemplary television
system with off-television sensors in accordance with various
aspects of the present invention.
[0012] FIG. 6 is a diagram illustrating an exemplary television
system with television receiver sensors in accordance with various
aspects of the present invention.
[0013] FIG. 7 is a diagram illustrating an exemplary television
system with television controller sensors in accordance with
various aspects of the present invention.
[0014] FIG. 8 is a diagram illustrating an exemplary television
receiver in accordance with various aspects of the present
invention.
[0015] FIG. 9 is a flow diagram illustrating the generation of
on-screen pointing information in accordance with various aspects
of the present invention.
[0016] FIG. 10 is a flow diagram illustrating the generation of
on-screen pointing information in accordance with various aspects
of the present invention.
DETAILED DESCRIPTION OF VARIOUS ASPECTS OF THE INVENTION
[0017] The following discussion will refer to various communication
modules, components or circuits. Such modules, components or
circuits may generally comprise hardware and/or a combination of
hardware and software (e.g., including firmware). Such modules may
also, for example, comprise a computer readable medium (e.g., a
non-transitory medium) comprising instructions (e.g., software
instructions) that, when executed by a processor, cause the
processor to perform various functional aspects of the present
invention. Accordingly, the scope of various aspects of the present
invention should not be limited by characteristics of particular
hardware and/or software implementations of a module, component or
circuit unless explicitly claimed as such. For example and without
limitation, various aspects of the present invention may be
implemented by one or more processors (e.g., a microprocessor,
digital signal processor, baseband processor, microcontroller,
etc.) executing software instructions (e.g., stored in volatile
and/or non-volatile memory). Also for example, various aspects of
the present invention may be implemented by an application-specific
integrated circuit ("ASIC") and/or other hardware components.
[0018] Additionally, the following discussion will refer to various
television system modules (e.g., television receiver modules). It
should be noted that the following discussion of such various
modules is segmented into such modules for the sake of illustrative
clarity. However, in actual implementation, the boundaries between
various modules may be blurred. For example, any or all of the
functional modules discussed herein may share various hardware
and/or software components. For example, any or all of the
functional modules discussed herein may be implemented wholly or
in-part by a shared processor executing software instructions.
Additionally, various software sub-modules that may be executed by
one or more processors may be shared between various software
modules. Accordingly, the scope of various aspects of the present
invention should not be limited by arbitrary boundaries between
various hardware and/or software components, unless explicitly
claimed.
[0019] The following discussion may also refer to communication
networks and various aspects thereof. For the following discussion,
a communication network is generally the communication
infrastructure through which a communication device (e.g., a
portable communication device, television, television controller,
television provider, television programming provider, television
receiver, video recording device, etc.) may communicate with other
systems. For example and without limitation, a communication
network may comprise a cable and/or satellite television
communication network, a cellular communication network, a wireless
metropolitan area network (WMAN), a wireless local area network
(WLAN), a wireless personal area network (WPAN), any home or
premises communication network, etc. A particular communication
network may, for example, generally have a corresponding
communication protocol according to which a communication device
may communicate with the communication network. Unless so claimed,
the scope of various aspects of the present invention should not be
limited by characteristics of a particular type of communication
network.
[0020] The following discussion will at times refer to an on-screen
pointing location. Such a pointing location refers to a location on
the television screen to which a user (either directly or with a
pointing device) is pointing. Such a pointing location is to be
distinguished from other types of on-screen location
identification, such as, for example, using arrow keys and/or a
mouse to move a cursor or to traverse blocks (e.g., on an on-screen
program guide) without pointing.
[0021] Additionally, the following discussion will at times refer
to television programming. Such television programming generally
includes various types of television programming (e.g., television
programs, news programs, sports programs, music television, movies,
television series programs and/or associated advertisements,
educational programs, live or recorded,
broadcast/multicast/unicast, etc.). Such television programming
video content is to be distinguished from other non-programming
video content that may be displayed on a television screen (e.g.,
an electronic program guide, user interface menu, a television
set-up menu, a typical web page, a document, a graphical video
game, etc.). Various aspects of the present invention may, for
example, comprise determining an on-screen pointing location during
the presentation of television programming on the screen of the
television.
[0022] Turning first to FIG. 1, such figure is a diagram
illustrating a non-limiting exemplary television system 100 in
accordance with various aspects of the present invention. The
exemplary system 100 includes a television provider 110. The
television provider 110 may, for example, comprise a television
network company, a cable company, a movie-providing company, a news
company, an educational institution, etc. The television provider
110 may, for example, be an original source of television
programming (or related information). Also for example, the
television provider 110 may be a communication company that
provides programming distribution services (e.g., a cable
television company, a satellite television company, a
telecommunication company, a data network provider, etc.). The
television provider 110 may, for example, provide programming and
non-programming information and/or video content. The television
provider 110 may, for example, provide information related to a
television program (e.g., information describing or otherwise
related to selectable objects in programming, etc.).
[0023] The exemplary television system 100 may also include a third
party program information provider 120. Such a provider may, for
example, provide information related to a television program. Such
information may, for example, comprise information describing
selectable objects in programming, program guide information,
etc.
[0024] The exemplary television system 100 may include one or more
communication networks (e.g., the communication network(s) 130).
The exemplary communication network 130 may comprise
characteristics of any of a variety of types of communication
networks over which video content and/or information related to
video content may be communicated. For example and without
limitation, the communication network 130 may compare
characteristics of a cable television network, a satellite
television network, a telecommunication network, the Internet, a
local area network (LAN), a personal area network (PAN), a
metropolitan area network (MAN), any of a variety of different
types of home networks, etc.
[0025] The exemplary television system 100 may include a first
television 140. Such a first television 140 may, for example,
comprise networking capability enabling such television 140 to
communicate directly with the communication network 130. For
example, the first television 140 may comprise one or more embedded
television receivers or transceivers (e.g., a cable television
receiver, satellite television transceiver, Internet modem, etc.).
Also for example, the first television 140 may comprise one or more
recording devices (e.g., for recording and/or playing back video
content, television programming, etc.).
[0026] The exemplary television system 100 may include a first
television controller 160. Such a first television controller 160
may, for example, operate to control operation of the first
television 140. The first television controller 160 may comprise
characteristics of any of a variety of television controlling
devices. For example and without limitation, the first television
controller 160 may comprise characteristics of a dedicated
television control device, a universal remote control, a cellular
telephone or personal computing device with television control
capability, etc.
[0027] The first television controller 160 may, for example,
transmit signals directly to the first television 140 to control
operation of the first television 140. The first television
controller 160 may also, for example, operate to transmit signals
(e.g., via the communication network 130) to the television
provider 110 to control video content being provided to the first
television 140, or to conduct other transactions (e.g., business
transactions, etc.).
[0028] As will be discussed in more detail later, various aspects
of the present invention include a user pointing to a location on a
television screen (e.g., pointing to an object or person presented
in television programming). In such a scenario, the user may
perform such pointing in any of a variety of manners. One of such
exemplary manners includes pointing with a television control
device. The first television controller 160 provides a non-limiting
example of a device that a user may utilize to point to an
on-screen location.
[0029] The exemplary television system 100 may also include a
television receiver 150. The television receiver may, for example,
operate to provide a communication link between a television and/or
television controller and a communication network and/or
information provider. For example, the television receiver 150 may
operate to provide a communication link between the second
television 141 and the communication network 130, or between the
second television 141 and the television provider 110 (and/or third
party program information provider 120) via the communication
network 130.
[0030] The television receiver 150 may comprise characteristics of
any of a variety of types of television receivers. For example and
without limitation, the television receiver 150 may comprise
characteristics of a cable television receiver, a satellite
television receiver, etc. Also for example, the television receiver
150 may comprise a data communication network modem for data
network communications (e.g., with the Internet, a LAN, PAN, MAN,
telecommunication network, etc.). The television receiver 150 may
also, for example, comprise recording capability (e.g., programming
recording and playback, etc.). The following discussion of FIGS.
2-10 will present various non-limiting illustrative aspects of such
a television receiver 150.
[0031] The exemplary television system 100 may include a second
television controller 161. Such a second television controller 161
may, for example, operate to control operation of the second
television 141 and the television receiver 150. The second
television controller 161 may comprise characteristics of any of a
variety of television controlling devices. For example and without
limitation, the second television controller 161 may comprise
characteristics of a dedicated television control device, a
dedicated television receiver control device, a universal remote
control, a cellular telephone or personal computing device with
television control capability, etc.
[0032] The second television controller 161 may, for example,
transmit signals directly to the second television 141 to control
operation of the second television 141. The second television
controller 161 may, for example, transmit signals directly to the
television receiver 150 to control operation of the television
receiver 150. The second television controller 161 may
additionally, for example, operate to transmit signals (e.g., via
the television receiver 150 and the communication network 130) to
the television provider 110 to control video content being provided
to the television receiver 150, or to conduct other transactions
(e.g., business transactions, etc.).
[0033] As will be discussed in more detail later, various aspects
of the present invention include a user pointing to a location on a
television screen (e.g., pointing to an object or person presented
in television programming). In such a scenario, the user may
perform such pointing in any of a variety of manners. One of such
exemplary manners includes pointing with a television control
device. The second television controller 161 provides one
non-limiting example of a device that a user may utilize to point
to an on-screen location.
[0034] The exemplary television system 100 was provided to provide
a non-limiting illustrative foundation for discussion of various
aspects of the present invention. Thus, the scope of various
aspects of the present invention should not be limited by any
characteristics of the exemplary television system 100 unless
explicitly claimed.
[0035] Turning next to FIG. 2, such figure is a diagram
illustrating an exemplary television receiver 200 in accordance
with various aspects of the present invention. The exemplary
television receiver 200 may, for example, share any or all
characteristics with the exemplary television receiver 150
illustrated in FIG. 1 and discussed previously and/or with any of
the exemplary television receivers discussed herein.
[0036] The exemplary television receiver 200 includes a first
communication interface module 210. The first communication
interface module 210 may, for example, operate to communicate over
any of a variety of communication media and utilizing any of a
variety of communication protocols. For example, though the first
communication interface module 210 is illustrated coupled to a
wireless RF antenna via a wireless port 212, the wireless medium is
merely illustrative and non-limiting. The first communication
interface module 210 may, for example, operate to communicate with
one or more communication networks (e.g., cable television
networks, satellite television networks, telecommunication
networks, the Internet, local area networks, personal area
networks, metropolitan area networks, etc.) via which television
video content (e.g., television programming) and/or other data is
communicated. Also for example, the first communication module 210
may operate to communicate with local sources of television video
content (e.g., video recorders, receivers, gaming devices, etc.).
Additionally, for example, the first communication module 210 may
operate to communicate with a television controller (e.g., directly
or via one or more intermediate communication networks). Further
for example, the first communication module 210 may operate to
communicate with a television utilizing any of a variety of
television communication connections and/or protocols (e.g.,
composite video, component video, HDMI, etc.).
[0037] The exemplary television receiver 200 includes a second
communication interface module 220. The second communication
interface module 220 may, for example, operate to communicate over
any of a variety of communication media and utilizing any of a
variety of communication protocols. For example, the second
communication interface module 220 may communicate via a wireless
RF communication port 222 and antenna, or may communicate via a
non-tethered optical communication port 224 (e.g., utilizing laser
diodes, photodiodes, etc.). Also for example, the second
communication interface module 220 may communicate via a tethered
optical communication port 226 (e.g., utilizing a fiber optic
cable), or may communicate via a wired communication port 228
(e.g., utilizing coaxial cable, twisted pair, HDMI cable, Ethernet
cable, any of a variety of wired component and/or composite video
connections, etc.). The second communication interface module 220
may, for example, operate to communicate with one or more
communication networks (e.g., cable television networks, satellite
television networks, telecommunication networks, the Internet,
local area networks, personal area networks, metropolitan area
networks, etc.) via which television video content and/or other
data is communicated. Also for example, the second communication
module 220 may operate to communicate with local sources of
television video content (e.g., video recorders, other receivers,
gaming devices, etc.). Additionally, for example, the second
communication module 220 may operate to communicate with a
television controller (e.g., directly or via one or more
intervening communication networks). Further for example, the
second communication module 220 may operate to communicate with a
television utilizing any of a variety of television communication
connections and/or protocols (e.g., composite video, component
video, HDMI, etc.).
[0038] The exemplary television receiver 200 may also comprise
additional communication interface modules, which are not
illustrated. Such additional communication interface modules may,
for example, share any or all aspects with the first 210 and second
220 communication interface modules discussed above.
[0039] The exemplary television receiver 200 may also comprise a
communication module 230. The communication module 230 may, for
example, operate to control and/or coordinate operation of the
first communication interface module 210 and the second
communication interface module 220 (and/or additional communication
interface modules as needed). The communication module 230 may, for
example, provide a convenient communication interface by which
other components of the television receiver 200 may utilize the
first 210 and second 220 communication interface modules.
Additionally, for example, in an exemplary scenario where a
plurality of communication interface modules are sharing a medium
and/or network, the communication module 230 may coordinate
communications to reduce collisions and/or other interference
between the communication interface modules 210, 220.
[0040] The exemplary television receiver 200 may comprise one or
more television interface modules 235. The television interface
module 235 may, for example, operate to manage communications
between the television receiver 200 and one or more televisions
that are communicatively coupled thereto (e.g., via the first 210
and/or second 220 communication interface modules). For example,
the television interface module 235 may operate to communicate
general television programming video information to a television
(e.g., while the television receiver 200 is operating to determine
an on-screen pointing location).
[0041] Also, for example, as will be discussed in more detail
later, the television interface module 235 may output a signal to
the television or television controller or other device with a
display, where such signal comprises characteristics adapted to
cause the television (or other device) to output a visual
indication of on-screen pointing location. Such an indication may,
for example, be communicated with (e.g., as a part of) television
programming being communicated to the television (or other device),
or such an indication may be communicated to the television (or
other device) independent of television programming.
[0042] The exemplary television receiver 200 may additionally
comprise one or more user interface modules 240. The user interface
module 240 may generally operate to provide user interface
functionality to a user of the television receiver 200. For
example, and without limitation, the user interface module 240 may
operate to provide for user control of any or all standard
television receiver commands (e.g., channel control, on/off,
television output settings, input selection, etc.). The user
interface module 240 may, for example, operate and/or respond to
user commands utilizing user interface features disposed on the
television receiver (e.g., buttons, touch screen, microphone, etc.)
and may also utilize the communication module 230 (and/or first 210
and second 220 communication interface modules) to communicate with
a television controller (e.g., a dedicated television remote
control, a universal remote control, a cellular telephone, personal
computing device, gaming controller, etc.) or a television. For
example, various user interface features of the television receiver
200 may comprise utilization of the television (e.g., utilizing the
television screen for menu-driven or other GUI associated with
television receiver operation).
[0043] The user interface module 240 may also operate to interface
with and/or control operation of any of a variety of sensors that
may be utilized to ascertain an on-screen pointing location.
Non-limiting examples of such sensors will be provided later (e.g.,
in the discussion of FIGS. 3-7 and elsewhere herein). For example
and without limitation, the user interface module 240 may operate
to receive signals associated with respective sensors (e.g., raw or
processed signals directly from the sensors, through intermediate
devices (e.g., a television, television control, surround sound
system, etc.), via the communication interface modules 210, 220,
etc.). Also for example, in scenarios in which such sensors are
active sensors (as opposed to purely passive sensors), the user
interface module 240 may operate to control the transmission of
signals (e.g., RF signals, optical signals, acoustic signals, etc.)
from such sensors.
[0044] The exemplary television receiver 200 may comprise one or
more processors 250. The processor 250 may, for example, comprise a
general purpose processor, digital signal processor,
application-specific processor, microcontroller, microprocessor,
etc. For example, the processor 250 may operate in accordance with
software (or firmware) instructions. As mentioned previously, any
or all functionality discussed herein may be performed by a
processor executing instructions. For example, though various
modules are illustrated as separate blocks or modules in FIG. 2 for
illustrative clarity, such illustrative modules, or a portion
thereof, may be implemented by the processor 250.
[0045] The exemplary television receiver 200 may comprise one or
more memories 260. As discussed above, various aspects may be
performed by one or more processors executing instructions. Such
instructions may, for example, be stored in the one or more
memories 260. Such memory 260 may, for example, comprise
characteristics of any of a variety of types of memory. For example
and without limitation, such memory 260 may comprise one or more
memory chips (e.g., ROM, RAM, EPROM, EEPROM, flash memory,
one-time-programmable OTP memory, etc.), hard drive memory, CD
memory, DVD memory, etc.
[0046] The exemplary television receiver 200 may also comprise one
or more calibration modules 251 that operate to perform various
calibration activities. Examples of such calibration activities
will be provided later in this discussion. Briefly, such
calibration activities may, for example, comprise interacting with
a user and/or user pointing device to determine sensor signals
under known circumstances (e.g., determine sensor signals in
response to known screen pointing circumstances), and processing
such sensor signals to develop algorithms (e.g., transformation
matrices, static positional equations, etc.) to determine screen
pointing location based on sensor signals received during normal
operation. As will also be discussed later, such calibration may
also be utilized to establish signal gain (or energy) patterns
utilized in determining pointing location.
[0047] The exemplary television receiver 200 may comprise one or
more location-determining modules 252. For example, as will be
discussed later, various on-screen pointing location determinations
may comprise processing location information. As a non-limiting
example, knowing the location of a user (e.g., including the
location of a pointing device being utilized by the user) may
simplify the solution of various pointing direction determinations.
For example, knowing exactly where a pointing device is located
(e.g., in three-dimensional space) or where a pointing device is
located along a line (e.g., knowing device location in
two-dimensional space or land surface coordinates) relative to the
television screen (and/or relative to the television receiver) may
remove a number of unknown variables from applicable positional
equations. Note that such positional information may, in various
exemplary scenarios, also comprise orientation information for a
pointing device (e.g., yaw, pitch and/or roll). Such orientation
information may be determined in various manners (e.g., through
gyroscopic means, sensor alignment with known references,
etc.).
[0048] The location-determining module 252 may operate to determine
user (or pointing device) location in any of a variety of manners.
For example and without limitation, the location-determining module
252 may operate to receive location information from the pointing
device (e.g., via one of the communication interface modules 210,
220). For example, such a pointing device may comprise positioning
system capability (e.g., global positioning system, assisted GPS,
cellular or other triangulation systems, etc.) and communicate
information describing the position of the pointing device to the
television receiver 200.
[0049] Also for example, the location-determining module 252 may
(e.g., via the user interface modules 240) utilize sensor signals
to determine the position (which may include orientation) of the
pointing device (or user thereof). For example, a signal from a
pointing device may arrive at different sensors at different times
(or at different phases). Such temporal or phase differences may be
processed to determine the location of the pointing device relative
to the known location of such sensors. Further for example, the
location-determining module 252 may operate to communicate pointing
device location information with an external system that operates
to determine the location of the pointing device. Such an external
system may, for example, comprise a cellular telephony
triangulation system, a home or premises-based triangulation
system, a global positioning system, an assisted global positioning
system, etc.
[0050] The exemplary television receiver 200 may also comprise one
or more sensor processing module(s) 253. As will be explained
below, the sensor processing module 253 may operate to receive
sensor information (e.g., from the user interface module(s) 240,
from the television interface module 235, etc.) and process such
received sensor information to determine a location on the
television screen to which a user is pointing. Various examples of
such processing will be provided below. Briefly, such processing
may, for example, comprise selecting a sensor with the strongest
signal, interpolating between a plurality of sensors, interpolating
between a plurality of sensors having strongest signals,
determining gain (or energy) pattern intersections, etc. Various
aspects of the present invention comprise, for example, determining
on-screen pointing location during presentation of television
programming (e.g., programming received from a television
broadcaster, video recording device, etc.).
[0051] Various aspects of the present invention will now be
illustrated by way of non-limiting example. Throughout the
following discussion, reference will continue to be made to the
various modules of the television receiver 200 illustrated in FIG.
2. It should be noted that the following non-limiting examples
provide specific examples of various aspects, and as such, the
scope of various aspects of the present invention should not be
limited by characteristics of any of the specific examples, unless
specifically claimed.
[0052] FIG. 3 is a diagram illustrating an exemplary television
system 300 with on-screen television sensors in accordance with
various aspects of the present invention. The television system 300
includes a television 301 comprising a television screen 303. The
television system 300 also includes a television controller 320 (or
other pointing device) pointing to an on-screen pointing location
330 along a line 325 between the television controller 320 and the
on-screen pointing location 330.
[0053] The television system 300 also comprises a television
receiver 350 that is communicatively coupled to the television 301
via a communication link 351 (e.g., a two-way communication link
providing video information to the television 301 and/or receiving
sensor information from the television 301). The television
receiver 350 may share any or all aspects with the exemplary
receivers 150, 200 discussed previously and all other receivers
discussed herein. Accordingly, various aspects of the television
receiver 350 will be explained herein with reference to various
components of the exemplary television receiver 200 illustrated in
FIG. 2. The exemplary television receiver 350 is also
communicatively coupled to the television controller 320 via a
communication link 352.
[0054] The exemplary television screen 303 comprises an array of
sensors integrated into the television screen 303. One of such
sensors is labeled sensor 310. Any of a variety of sensor types may
be utilized, non-limiting examples of which include light sensors
or photo detectors (e.g., photo diodes) and RF sensors (e.g.,
antenna elements or loops).
[0055] The array of sensors may be integrated in the television
screen 303 in any of a variety of manners, non-limiting examples of
which will now be provided. For example, the television screen 303
may comprise an array of liquid crystal display (LCD) pixels for
presenting video media to a user. An array of photo diodes and/or
antenna elements may be integrated between or behind LCD pixels.
For example, every LCD pixel may be associated with a corresponding
photo diode and/or antenna element, or every N.times.M block of LCD
pixels may be associated with a corresponding photo diode or
antenna element.
[0056] As a non-limiting example, an array of photo diodes and/or
RF antenna elements may be formed into a substrate beneath or
behind transparent LCD substrates. As another example, a photo
diode array and/or antenna element array may be interposed between
or behind an array of LCD thin film transistors. Also for example,
an array of photo diodes and/or RF antenna elements (or other
sensors) may be incorporated into a transparent screen overlay.
Note that is such an implementation, such transparent screen
overlay may be installed after-market. For example, a user that has
a television receiver 350 with the capability to determine
on-screen pointing location may install the transparent screen
overlay. In such an exemplary scenario, there may be one or more
communication links established between the television receiver 350
and the sensors in the overlay, where such communication links may
be independent of a communication link over which non-sensor
information (e.g., video and/or control information) is
communicated between the television 301 and the television receiver
350. Such communication link may, for example, be adapted to
communicate information from each sensor to the television receiver
350 serially (e.g., in a time-multiplexed manner) and/or in
parallel.
[0057] In a photo detector implementation, passive photo detectors
may receive varying amounts of respective light energy depending on
the pointing direction of a light source aimed at the screen. Also
for example, received signals (e.g., pulsed signals) may arrive at
different sensors at different respective times/phases (e.g., being
indicative of relative position and/or pointing direction, which
may also be utilized in a pointing determination). In such a photo
detector implementation (e.g., utilizing photo diodes), photo
detectors may, for example, be tuned to react to particular light
frequencies to reduce interference from output pixel light and/or
associated reflections, ambient light, etc. As a non-limiting
example, photo diodes may be tuned to detect light that is not
visible to the human eye, visible light frequencies that are
relatively rare, light patterns that are unlikely to occur in a
television program (e.g., particular pulse codes), etc.
[0058] In an antenna element implementation, an array of antenna
elements may be formed on a substrate and placed behind light
producing and/or filtering elements in the LCD screen (e.g., so as
to avoid interfering with emitted light) or may be formed on a
transparent substrate within or in front of the lighted region of
the LCD display (e.g., utilizing microscopic antenna elements that
are too small to significantly interfere with light emitted from
the display). As discussed above, such an implementation may be
integrated with the television screen 303, but may also be added as
an overlay (e.g., as a production option or an after-market user or
technician installation).
[0059] In an RF antenna implementation, passive antennas (or
elements of an overall antenna matrix) may receive varying
respective amounts of RF energy depending on the pointing direction
of a directional RF source aimed at the screen. Also for example,
received signals (e.g., pulsed signals) may arrive at different
antennas at different respective times/phases (e.g., being
indicative of relative position and/or pointing direction, which
may also be utilized in a pointing determination)
[0060] In an exemplary scenario, a user may point a pointing device
(e.g., a remote controller, a laser pointer, directional RF
transmitter, specifically designed eyewear, a mobile computing
device, a mobile communication device, a gesture tracking device or
glove, etc.) at the television screen 303, where the pointing
device directs transmitted energy (e.g., light energy, RF energy,
acoustic energy, etc.) at a particular location on the television
screen 303 to which the pointing device is being pointed. Note that
such transmitted energy will likely be transmitted directionally
and be associated with an intensity or gain pattern with the
highest intensity likely at the center of the pattern (i.e., along
the pointing line 325) and decreasing as a function of angle from
the center of the pattern (or distance on the screen from the
on-screen pointing location).
[0061] In such an exemplary scenario, each sensor of the array of
sensors integrated into the screen 303 will likely receive some
respective amount of energy. For example, the sensor nearest the
screen pointing location 330 (i.e., along the pointing line 325)
will likely receive the highest amount of energy, sensors adjacent
to the screen pointing location 330 will likely receive a next
highest range of energy, and sensors away from the pointing
location 330 will likely receive progressively less amounts of
energy from the pointing device 320, as a function of distance from
the pointing location 330, until such energy is lost in the noise
floor.
[0062] In such an exemplary scenario, the television receiver 350
(e.g., the user interface module 240 of the television receiver 200
illustrated in FIG. 2) may receive signals indicative of the energy
received by the sensors of the sensor array. The television
receiver 350 may receive such signals in various manners, depending
on the degree of integration of such sensors into the television
301. For example, in an exemplary scenario where the sensors are
fully integrated into the television screen 303 and operationally
integrated into the television 301, the television receiver 350 may
receive such signals via a communication interface between the
television receiver 350 and the television 301. Also for example,
in another exemplary scenario where the sensors are overlaid on the
television screen 303, and where operation of such sensors is
independent of the television 301, the television receiver 350 may
receive such signals via a communication link directly between the
television receiver 350 and the sensors, where such a communication
link may be independent of other communication links between the
television receiver 350 and the television 301. Such communication
link may, for example, be adapted to communicate information from
each sensor to the television receiver 350 serially (e.g., in a
time-multiplexed manner) and/or in parallel.
[0063] The user interface module 240 may then, for example, provide
information of such received sensor signals to the sensor
processing module 253 for processing. The sensor processing module
253 may then, for example, operate to process such information to
determine the screen pointing location. The sensor processing
module 253 may perform such processing in any of a variety of
manners, non-limiting examples of which will be provided below.
[0064] For example, the sensor processing module 253 may operate to
select the sensor with the highest received energy and determine
that the location of such selected sensor is the on-screen pointing
location. For example, in an exemplary scenario where the spatial
resolution of screen-integrated sensors is relatively fine, such
operation may reliably yield a desired level of accuracy without
undue processing overhead.
[0065] In another example, the sensor processing module 253 may
operate to select the sensor with the highest received energy and a
plurality of sensors adjacent to such sensor. Then, for example,
the sensor processing module 253 may interpolate between the
locations of such sensors (e.g., based, at least in part, on
weighting). For example, in a first dimension in which a sensor to
the right of the highest energy sensor has a higher received energy
than a sensor to the left of the highest energy sensor, the sensor
processing module 253 may determine that the pointing location is
to the right of the highest energy sensor. How much distance to the
right may, for example, be determined as a function of the ratio
between respective energies received by the right and left sensors.
Such calculation may, for example, be a linear or non-linear
calculation. Such calculation may also, for example, consider the
expected energy pattern of a transmitting pointing device (e.g., in
a scenario where energy fall-off is logarithmic as opposed to
linear).
[0066] In an additional example, the sensor processing module 253
may operate to select all sensors receiving a threshold amount of
energy (e.g., an absolute threshold level, a threshold level
relative to the highest energy sensor, etc.). Then, for example,
the sensor processing module 253 may interpolate between the
locations of such sensors (e.g., based, at least in part, on
respective energy weighting). For example, the sensor processing
module 253 may perform non-linear splining between sensors in a
horizontal direction with sensor location on a first axis and
sensor energy on a second axis. The sensor processing module 253
may then operate to select the point on the sensor location axis
corresponding to the peak sensor energy on the vertical axis. Such
splining and selecting may then be repeated in the vertical
direction. Alternatively for example, the sensor processing module
253 may operate to perform multi-dimensional splining to create a
surface based on sensor energy and select the highest point on such
surface and the corresponding screen coordinates of such
surface.
[0067] In a further example, the sensor processing module 253 may
operate to select a first sensor (e.g., the sensor with the highest
received energy). Then, for example, the sensor processing module
253 may utilize information of the relative distance between the
selected sensor and the pointing device, information of the gain
pattern for the signal transmitted from the pointing device to the
selected sensor, and calibration information to determine where the
pointing device may be pointed in order for the sensor to receive
such energy. For example, this may result in a first closed figure
(e.g., a circle, cloverleaf, etc.) drawn around the sensor on the
screen plane. Then the sensor processing module 253 may repeat the
procedure for a second sensor (e.g., a sensor with the second
highest received energy), resulting in a second closed figure. The
sensor processing module 253 may then, for example, determine the
point(s) of intersection between the first and second figures. If
only one point of intersection lies within the border of the
screen, then such point of intersection might be utilized as an
estimate of the pointing location. If, however, there are two
potentially significant points of intersection (or more depending
on the figures), then the sensor processing module 253 may repeat
the procedure for a third sensor (e.g., the sensor with the third
highest energy, a sensor generally along the line perpendicular to
a line segment between the first and second sensors, etc.) and
determine a point nearest the intersection of the first, second and
third closed figures. Such a point of intersection may then be
utilized as an estimate of the pointing location.
[0068] The above-mentioned examples of screen-integrated sensors
and related pointing location determinations were presented as
exemplary illustrations. Though the above-mentioned examples
generally discuss light and/or RF energy sensors, other types of
sensors may also be integrated into a television screen or overlaid
thereon. For example and without limitation, the sensors may
comprise acoustic sensors that operate to sense acoustic energy
(e.g., directed acoustic energy directed to a pointing location on
the screen). For example, such directed acoustic energy may be
formed at frequencies beyond the range of human hearing (e.g., and
at frequencies beyond the range of pet hearing as well).
[0069] Also note that various energy radiation patterns may be
used, and/or a plurality of energy radiation patterns may be used.
For example, though (e.g., for illustrative clarity) the discussion
herein generally discusses a single energy emission from the
pointing device, a plurality of energy emissions may be utilized.
For example and without limitation, a pointing device may transmit
a plurality of different directed energy emissions (e.g., light,
RF, etc.) toward the pointing direction. Also for example, a
pointing device may transmit one or more energy emissions that move
relative to the pointing direction (e.g., in a raster pattern or
any other pattern).
[0070] After determining on-screen pointing location, the
television receiver 350 may communicate information of such
determined location in various manners. For example and without
limitation, the sensor processing module 253 of the television
receiver 200 may utilize the television interface module 235 to
communicate information of such on-screen pointing location to the
television 301 for presentation to the user. Also for example, the
sensor processing module 253 may utilize the user interface module
240 to communicate information of such on-screen pointing location
to the television controller 320 for presentation to the user. Such
communication will also be addressed in the discussions of FIGS.
9-10.
[0071] In addition to various television configurations in which
sensors are integrated into the television screen, sensors may be
incorporated into the television off-screen. Such sensors may, for
example, be incorporated in a border around the screen (or overlaid
thereon). For example and without limitation, FIG. 4 is a diagram
illustrating an exemplary television system 400 with off-screen
television sensors in accordance with various aspects of the
present invention. The television system 400 includes a television
401 comprising a television screen 403. The television system 400
also includes a television controller 420 (or other pointing
device) pointing to an on-screen pointing location 430 along a
pointing line 425 between the television controller 420 and the
on-screen pointing location 430.
[0072] The television system 400 also comprises a television
receiver 450 that is communicatively coupled to the television 401
via a communication link 451 (e.g., a two-way communication link
providing video information to the television 401 and/or receiving
sensor information from the television 401). The television
receiver 450 may share any or all aspects with the exemplary
receivers (150, 200 and 350) discussed previously and all other
receivers discussed herein. Accordingly, various aspects of the
television receiver 450 will be explained herein with reference to
various components of the exemplary television receiver 200
illustrated in FIG. 2. The exemplary television receiver 450 is
also communicatively coupled to the television controller 420 via a
communication link 452.
[0073] The exemplary television 401 comprises an array of sensors
integrated into the television 401 around the border of the screen
403. Three of such sensors are labeled 410, 411 and 412. As
discussed above, any of a variety of sensor types may be utilized,
non-limiting examples of which include light sensors or photo
detectors (e.g., photo diodes), RF sensors (e.g., antenna
elements), acoustic sensors (e.g., microphones), etc.
[0074] The array of sensors may be integrated around the television
screen 403 in any of a variety of manners. For example, such
sensors may be integrated in a border of the television screen 403
that is not used for outputting video content. Such a configuration
may, for example, avoid sensor interference with video content
being displayed on the screen. Also for example, as illustrated in
FIG. 4, such sensors may be mounted to a border material of the
television 401.
[0075] For example, an array of photo detectors (e.g., photo
diodes) and/or antenna elements (e.g., individual antennas or
elements of an antenna array, for example, a phased array) may be
incorporated into a border of the television 401 around the screen
403. For example, every screen pixel row and/or column may be
associated with a pair of corresponding photo diodes and/or antenna
elements, or every N.times.M block of screen pixels may be
associated with one or more corresponding photo diodes or antenna
elements (e.g., a row and column sensor, two row and two column
elements, etc.).
[0076] In a photo detector implementation, passive photo detectors
may receive varying amounts of respective light energy depending on
the pointing direction of a light source pointed at the screen.
Also for example, received signals (e.g., pulsed signals) may
arrive at different sensors at different respective times/phases
(e.g., being indicative of relative position and/or pointing
direction, which may also be utilized in a pointing determination).
In such a photo detector implementation (e.g., utilizing photo
diodes), photo detectors may, for example, be tuned to react to
particular light frequencies to reduce interference from output
pixel light and/or associated reflections. As a non-limiting
example, photo diodes may be tuned to detect light that is not
visible to the human eye, visible light frequencies that are
relatively rare, light patterns that are unlikely to occur in a
television program (e.g., particular pulse codes), etc. In one
example, the photo detectors integrated with the television body
off-screen may comprise photo diodes that operate to detect energy
from a laser pointer or directed infrared energy from a television
controller or other pointing device. Note that analogously to the
on-screen sensors discussed previously, various aspects may
comprise mounting (e.g., adhering) sensors to the television body
off-screen. Such sensor installation may, for example, occur at the
factory or after-market by a technician or user.
[0077] In an antenna element implementation, an array of antenna
elements may be positioned around the border of the screen 403. In
an RF antenna implementation, passive antennas (or elements of an
overall antenna matrix) may receive varying amounts of respective
RF energy depending on the pointing direction of a directional RF
source aimed at the screen. Also for example, received signals
(e.g., pulsed signals) may arrive at different antennas at
different respective times/phases (e.g., being indicative of
relative position and/or pointing direction, which may also be
utilized in a pointing determination). Note that analogously to the
on-screen sensors discussed previously, various aspects may
comprise mounting (e.g., adhering) sensors to the television body
off-screen. Such sensor installation may, for example, occur at the
factory or after-market by a technician or user.
[0078] In an exemplary scenario, a user may point a pointing device
(e.g., a remote controller, a laser pointer, directional RF
transmitter, specifically designed eyewear, a mobile computing
device, a mobile communication device, a gesture tracking device or
glove, etc.) at the television screen 403, where the pointing
device directs transmitted energy (e.g., light and/or RF energy
and/or acoustic energy) at a particular location on the television
screen 403 to which the device is being pointed. Note that such
transmitted energy will likely be transmitted directionally and be
associated with an intensity or gain pattern with the highest
intensity likely at the center of the pattern (i.e., along the
pointing line 425) and decreasing as a function of angle from the
center of the pattern. Such a gain pattern is generally represented
in FIG. 4 by the concentric circles around the on-screen pointing
location 430. Note, however, that in practice such a gain pattern
is likely to be more complex than the illustrated pattern (e.g.,
including lobes with respective peaks and nulls).
[0079] In such an exemplary scenario, each sensor of the sensors
integrated into the television around the border of the screen 403
will likely receive some respective amount of energy. For example,
along a particular axis the sensor nearest the screen pointing
location 430 (i.e., along the pointing line 425) will likely
receive the highest amount of energy, sensors along the particular
axis adjacent to the screen pointing location 430 will likely
receive a next highest range of energy, and sensors away from the
pointing location 430 will likely receive progressively less
amounts of energy from the pointing device 420, as a function of
distance from the pointing location 430, until such energy is lost
in the noise floor.
[0080] For example, along the horizontal axis, sensor 410 is
closest to the pointing location 430 and will likely receive the
highest energy, with sensors adjacent to the left and right of
sensor 410 receiving the next highest amounts of energy, and so on.
Also, along the vertical axis, sensors 411 and 412 will likely
receive close to the highest amount of energy, with sensors above
and below such sensors 411, 412 receiving the next highest amounts
of energy and so on.
[0081] In such an exemplary scenario, the television receiver 450
(e.g., the user interface module 240 of the television receiver 200
illustrated in FIG. 2) may receive signals indicative of the energy
received by the sensors of the television. The television receiver
450 may receive such signals in various manners, depending on the
degree of integration of such sensors into the television 401. For
example, in an exemplary scenario where the sensors are fully
integrated into the television 401 (e.g., into a border around the
screen 403) and operationally integrated into the television 401,
the television receiver 450 may receive such signals via a
communication interface between the television receiver 450 and the
television 401. Also for example, in another exemplary scenario
where the sensors are overlaid on (e.g., adhered to) the television
screen 401, and where operation of such sensors is independent of
the television 401, the television receiver 450 may receive such
signals via a communication link directly between the television
receiver 450 and the sensors, where such a communication link may
be independent of other communication links between the television
receiver 450 and the television 401. Such communication link may,
for example, be adapted to communicate information from each sensor
to the television receiver 450 serially (e.g., in a
time-multiplexed manner) and/or in parallel.
[0082] The user interface module 240 may then, for example, provide
information of such received sensor signals to the sensor
processing module 253 for processing. The sensor processing module
253 may then, for example, operate to process such information to
determine the screen pointing location. The sensor processing
module 253 may perform such processing in any of a variety of
manners, non-limiting examples of which will be provided below.
[0083] For example, the sensor processing module 253 may operate to
select the sensor with the highest received energy along each of
the horizontal and vertical axes and determine that the respective
locations of such selected sensors correspond to the horizontal and
vertical coordinates of the on-screen pointing location. For
example, in an exemplary scenario where the spatial resolution of
screen border sensors is relatively fine, such operation may
reliably yield a desired level of accuracy without undue processing
overhead. For example, the sensor processing module 253 may
determine that sensors 410 and 411 have the highest received energy
for the horizontal and vertical axes, respectively, and thus
determine that the on-screen pointing location is represented in
the horizontal axis by the horizontal location of the sensor 410
and represented in the vertical axis by the vertical location of
the sensor 411. Note that in scenarios where two sensors have
relatively similar energy levels (e.g., as might occur at sensors
411 and 412, the sensor processing module 253 may select a vertical
midpoint between such sensors.
[0084] In another example, the sensor processing module 253 may
operate to select, for each screen axis, the sensor with the
highest received energy and a plurality of sensors adjacent to such
sensor. Then, for example, the sensor processing module 253 may
interpolate between the locations of such sensors (e.g., based, at
least in part, on weighting). For example, in the horizontal
dimension in which a sensor to the right of the highest energy
sensor 410 has a higher received energy than a sensor to the left
of the highest energy sensor 410, the sensor processing module 253
may determine that the pointing location along the horizontal axis
is to the right of the highest energy sensor 410. How much distance
to the right may, for example, be determined as a function of the
ratio between respective energies received by the right and left
sensors. Such calculation may, for example, be a linear or
non-linear calculation. Such calculation may also, for example,
consider the expected energy pattern of a transmitting pointing
device (e.g., in a scenario where energy fall-off is logarithmic as
opposed to linear). The sensor processing module 253 may then, for
example, repeat such operation in the vertical direction.
[0085] In another example, the sensor processing module 253 may
operate to select all sensors in each of the axes receiving a
threshold amount of energy (e.g., an absolute threshold level, a
threshold level relative to the highest energy sensor, etc.). Then,
for example, the sensor processing module 253 may interpolate
between the locations of such sensors (e.g., based, at least in
part, on respective energy weighting). For example, the sensor
processing module 253 may perform non-linear splining between
sensors in a horizontal direction with sensor location on a first
axis and sensor energy on a second axis. The sensor processing
module 253 may then operate to select the point on the sensor
location axis corresponding to the peak sensor energy on the
vertical axis. Such splining and selecting may then be repeated in
the vertical screen direction. Alternatively for example, the
sensor processing module 253 may operate to perform
multi-dimensional splining to create a surface based on sensor
energy and select the highest point on such surface and the
corresponding screen coordinates of such surface.
[0086] After determining on-screen pointing location, the
television receiver 450 may communicate information of such
determined location in various manners. For example and without
limitation, the sensor processing module 253 of the television
receiver 200 may utilize the television interface module 235 to
communicate information of such on-screen pointing location to the
television 401 for presentation to the user. Also for example, the
sensor processing module 253 may utilize the user interface module
240 to communicate information of such on-screen pointing location
to the television controller 420 for presentation to the user. Such
communication will also be addressed in the discussions of FIGS.
9-10.
[0087] In addition to various television configurations in which
sensors are integrated into the television off-screen or off the
video presentation portion of the screen, sensors may be
incorporated into the television system off-television. Such
sensors may, for example, be incorporated in other components of a
television system besides the television. For example and without
limitation, FIG. 5 is a diagram illustrating an exemplary
television system 500 with off-television sensors in accordance
with various aspects of the present invention. The television
system 500 includes a television 501 comprising a television screen
503. The television system 500 also includes a television
controller 520 (or other pointing device) pointing to an on-screen
pointing location 530 along a pointing line 525 between the
television controller 520 and the on-screen pointing location
530.
[0088] The television system 500 also comprises a television
receiver 550 that is communicatively coupled to the television 501
via a communication link 561 (e.g., a two-way communication link
providing video information to the television 501 and/or receiving
sensor information from the television 501). The television
receiver 550 is also illustrated with one or more communication
links 563 to the various sensors 551-556 independent of the
communication link 561. Note that in various exemplary scenarios,
the television receiver 550 (e.g., a user interface module 240) may
receive sensor information via the television communication link
561 and/or via the independent communication link(s) 563. The
exemplary television receiver 550 is also communicatively coupled
to the television controller 520.
[0089] The television receiver 550 may share any or all aspects
with the exemplary receivers (150, 200, 350 and 450) discussed
previously and all other receivers discussed herein. Accordingly,
various aspects of the television receiver 550 will be explained
herein with reference to various components of the exemplary
television receiver 200 illustrated in FIG. 2.
[0090] The exemplary television system 500 comprises an array of
sensors integrated into audio speaker components (e.g., surround
sound speakers) positioned around the television 501. For example,
the television system 500 comprises a left speaker 531 comprising a
top sensor 552 and a bottom sensor 551. Also for example, the
television system 500 comprises a right speaker 533 comprising a
top sensor 556 and a bottom sensor 555. Additionally for example,
the television system comprises a center speaker 532 comprising a
left sensor 553 and a right sensor 554. As discussed above, any of
a variety of sensor types may be utilized, non-limiting examples of
which include light sensors or photo detectors (e.g., photo
diodes), RF sensors (e.g., antenna elements), acoustic sensors
(e.g., microphones), etc. Note that the audio speaker component
example discussed herein is merely illustrative and that such
sensors may be installed in any of a variety of locations (e.g.,
dedicated sensor boxes, attached to furniture, etc.).
[0091] The array of sensors may be positioned around the television
501 in any of a variety of manners. For example, such sensors may
be positioned around the television 501 generally in the same plane
as the television screen 503. In such an exemplary scenario,
on-screen pointing location may be determined in a manner similar
to the interpolation and/or gain pattern intersection discussed
above with regard to off-screen and/or on-screen sensors. Note that
since the locations of the sensors are likely to be inconsistent
between various television system configurations, a calibration
procedure may be implemented (e.g., by the calibration module 251).
Such calibration will be discussed in more detail below.
[0092] In an exemplary configuration, one or more photo detectors
(e.g., photo diodes) and/or antenna elements (e.g., individual
antennas or elements of an antenna array) may be incorporated into
a plurality of respective surround sound speakers positioned around
the television 501.
[0093] For example, in a photo detector implementation, passive
photo detectors may receive varying amounts of respective light
energy depending on the pointing direction of a light source aimed
at the screen. As discussed previously, directed energy (e.g.,
light, RF, acoustic, etc.) may be transmitted in a pattern (or
envelope), so even if a pointing device is pointed to a location on
the television screen 530 along pointing line 525, sensors
off-screen (or even off-television) may still receive energy from
the transmission (albeit likely not with the same intensity at
which energy is delivered along the pointing line 525). Also for
example, received signals (e.g., pulsed signals) may arrive at
different sensors at different respective times/phases (e.g., being
indicative of relative position and/or pointing direction, which
may also be utilized in a pointing determination).
[0094] In a photo detector implementation (e.g., utilizing photo
diodes), photo diodes may, for example, be tuned to react to
particular light frequencies to reduce interference from output
pixel light and/or associated reflections, ambient light, room
lighting, etc. As a non-limiting example, photo diodes may be tuned
to detect light that is not visible to the human eye, visible light
frequencies that are relatively rare, light patterns that are
unlikely to occur in a television program (e.g., particular pulse
codes), etc. In one example, the photo detectors integrated with
off-television components may comprise photo diodes that operate to
detect energy from a laser pointer or directed infrared energy from
a television controller (or other pointing device). Note that
analogously to the on-screen sensors discussed previously, various
aspects may comprise mounting (e.g., adhering) sensors to various
off-television components. Such sensor installation may, for
example, occur at the factory or after-market by a technician or
user.
[0095] In an antenna element implementation, an array of antenna
elements may be positioned around off-television components (e.g.,
in surround sound components). In an RF antenna implementation,
passive antennas (or elements of an overall antenna matrix) may
receive varying amounts of respective RF energy depending on the
pointing direction of a directional RF source pointed at a location
on the screen. Also for example, received signals (e.g., pulsed
signals) may arrive at different antennas at different respective
times/phases (e.g., being indicative of relative position and/or
pointing direction, which may also be utilized in a pointing
determination). Note that analogously to the on-screen sensors
discussed previously, various aspects may comprise mounting (e.g.,
adhering) sensors to the off-television components. Such sensor
installation may, for example, occur at the factory or after-market
by a technician or user.
[0096] In an exemplary scenario, a user may point a pointing device
(e.g., a remote controller, a laser pointer, directional RF
transmitter, specifically designed eyewear, a mobile computing
device, a mobile communication device, a gesture tracking device or
glove, etc.) at the television screen 503, where the pointing
device directs transmitted energy (e.g., light and/or RF energy
and/or acoustic energy) at a particular location on the television
screen 503 to which the user is pointing with the pointing device.
Note that such transmitted energy will likely be transmitted
directionally and be associated with an intensity or gain pattern
with the highest intensity at the center of the pattern (i.e.,
along the pointing line 525) and decreasing as a function of angle
from the center of the pattern. Such a gain pattern was discussed
previously in the discussion of FIG. 4.
[0097] In such an exemplary scenario, each sensor of the sensors
integrated into the television system 500 off-television will
likely receive some respective amount of energy. For example, along
a particular axis, the sensor nearest to the screen pointing
location 530 (i.e., along the pointing line 525) will likely
receive the highest amount of energy, a sensor next nearest to the
screen pointing location 530 will likely receive a next highest
range of energy, and sensors away from the pointing location 530
will likely receive progressively less amounts of energy from the
pointing device 520, as a function of distance from the pointing
location 530 and/or angle off the pointing line 525 (e.g., until
such energy is lost in the noise floor). For example, sensor 553 is
nearest to the pointing location 530 and will likely receive the
highest energy, sensor 552 is next nearest to the pointing location
530 and will likely receive the next highest energy, and so on.
[0098] Note that in the implementation illustrated in FIG. 5, in
particular since there are a relatively low number of sensors,
signals from a same sensor may be utilized in determining multiple
axes of pointing location. As mentioned previously, a calibration
procedure may be performed when the system 500 is configured to
assist in such pointing determination.
[0099] In an exemplary scenario, the television receiver 550 (e.g.,
the user interface module 240 of the television receiver 200
illustrated in FIG. 2) may receive signals indicative of the energy
received by the sensors of the television system 500. The
television receiver 550 may receive such signals in various
manners, depending on the degree of integration of such sensors
into the television 501. For example, in an exemplary scenario
where the sensors are fully integrated into the television system
500 components (e.g., surround sound speaker components 531-533)
and operationally integrated into such components, the television
receiver 550 may receive such signals via a communication interface
between the television receiver 550 and the respective
off-television components (e.g., via a communication link 563
between the television receiver 550 and the surround sound speaker
components 531-533). Also for example, in another exemplary
scenario where the sensors are overlaid on (e.g., adhered to) the
off-television components, and where operation of such sensors is
independent of the television 501, the television receiver 550 may
receive such signals via a communication link directly between the
television receiver 550 and the individual sensors, where such a
communication link may be independent of other communication links
between the television receiver 550 and the television 501 and/or
independent of other communication links between the television
receiver 550 and other television system 500 components (e.g.,
surround sound speaker components 531-533).
[0100] The user interface module 240 may then, for example, provide
information of such received sensor signals to the sensor
processing module 253 for processing. The sensor processing module
253 may then, for example, operate to process such information to
determine the screen pointing location. The sensor processing
module 253 may perform such processing in any of a variety of
manners, non-limiting examples of which will be provided below.
[0101] In an exemplary scenario, the sensor processing module 253
may operate to estimate a position between sensor positions based
on relative sensor energy. For example, in the horizontal
dimension, sensor 552 may correspond to a relatively high amount of
energy, and sensor 556 may correspond to a relatively low amount of
received energy. The sensor processing module 253 may, for example,
estimate a horizontal position relatively closer to sensor 552 by
an amount proportional to the relative difference between
respective amounts of energy. The sensor processing module 253 may
perform a similar estimation utilizing sensors 551 and 555. Various
horizontal position estimations may then be averaged. Alternatively
for example, respective energies for the left speaker 531 sensors
may be averaged, respective energies for the right speaker 533
sensors may be averaged, and such left and right speaker average
energies may then be utilized to determine a horizontal pointing
location. The sensor processing module 253 may then, for example,
perform a similar pointing direction estimate in the vertical
direction.
[0102] In another exemplary scenario, a calibration procedure may
be performed to determine an expected sensor energy level (e.g.,
absolute or relative) when the user is pointing at the sensor. In
such a scenario, combined with a gain pattern and user (or pointing
device) location relative to the television 501, a first line
(e.g., a circle or arc) may be drawn around a first sensor 552.
Similarly, a second line (e.g., a circle or arc) may be drawn
around a second sensor 553, and the intersection of the first and
second lines utilized as an estimate of pointing location.
Additional lines associated with other sensors may also be
utilized. Such additional lines may, for example, be utilized when
selecting between multiple line intersections and/or for greater
accuracy or resolution. Note that such line intersection solution
may be applied to any of the previously discussed scenarios (e.g.,
as illustrated in FIGS. 3-4). A non-limiting example of this was
presented in the discussion of FIG. 3, and another example will be
provided in the following discussion of FIG. 7.
[0103] After determining on-screen pointing location, the
television receiver 550 may communicate information of such
determined location in various manners. For example and without
limitation, the sensor processing module 253 of the television
receiver 200 may utilize the television interface module 235 to
communicate information of such on-screen pointing location to the
television 501 for presentation to the user. Also for example, the
sensor processing module 253 may utilize the user interface module
240 to communicate information of such on-screen pointing location
to the television controller 520 for presentation to the user. Such
communication will also be addressed in the discussions of FIGS.
9-10.
[0104] As discussed above, pointing sensors may be incorporated
into the television system off-television (i.e., placed separately
in stand-alone housings, integrated with other apparatus, attached
to other apparatus, etc.). Another example of such off-television
sensor placement is presented in FIG. 6. In particular, the screen
pointing sensors may be integrated into the television receiver.
FIG. 6 is a diagram illustrating an exemplary television system 600
with television receiver sensors in accordance with various aspects
of the present invention.
[0105] The television system 600 includes a television 601
comprising a television screen 603. The television system 600 also
includes a television controller 620 (or other pointing device)
pointing to an on-screen pointing location 630 along a pointing
line 625 between the television controller 620 and the on-screen
pointing location 630.
[0106] The television system 600 also comprises a television
receiver 650 that is communicatively coupled to the television 601
via a communication link 651 (e.g., a two-way communication link
providing video information to the television 601 and/or
communicating sensor information and/or screen pointing information
with the television 601). The television receiver 650 comprises an
array of screen pointing sensors. A portion of the sensors are
labeled (661-665) for discussion purposes. Note that such sensors
may be arranged in any of a variety of configurations (e.g., matrix
configuration, border configuration, placed only at the front
corners, etc.). The pointing sensors may, for example, be
integrated into the television receiver 650 and/or attached to the
television receiver 650 in any of a variety of manners (e.g., in
any manner similar to those discussed previously with regard to the
televisions and/or television system components discussed
previously).
[0107] Note that in various exemplary scenarios, the television
receiver 650 (e.g., a user interface module 240) may receive
additional sensor information from other sensors via the television
communication line 651 and/or other communication links. The
exemplary television receiver 650 is also communicatively coupled
to the television controller 620 via a communication link 652.
[0108] The television receiver 650 may share any or all aspects
with the exemplary receivers (150, 200, 350, 450 and 550) discussed
previously and all other receivers discussed herein. Accordingly,
various aspects of the television receiver 650 will be explained
herein with reference to various components of the exemplary
television receiver 200 illustrated in FIG. 2.
[0109] The exemplary television receiver 650 comprises an array of
sensors integrated into the television receiver 650. For example,
the television receiver 650 comprises a lower left sensor 661,
upper left sensor 662, upper right sensor 663, lower right sensor
664 and center sensor 665. As discussed above, any of a variety of
sensor types may be utilized, non-limiting examples of which
include light sensors or photo detectors (e.g., photo diodes), RF
sensors (e.g., antenna elements), acoustic sensors (e.g.,
microphones), etc.
[0110] The exemplary television receiver 650 may be positioned
around the television 601 in any of a variety of manners. For
example, the television receiver 650 (and thus the sensors) may be
positioned around the television 601 in an orientation such that
the front face of the television receiver 650 (and thus the
sensors) is generally in the same plane as the television screen
603. Such placement is not necessary, but may be advantageous from
an accuracy perspective. In such an exemplary scenario, on-screen
pointing location may be determined in a manner similar to the
interpolation and/or gain pattern intersection discussed above with
regard to off-screen and/or on-screen sensors. Note that since the
locations of the sensors are likely to be inconsistent between
various television system configurations (i.e., it is unlikely that
every user will place/position the television receiver 650 in the
same manner), a calibration procedure may be implemented (e.g., by
the calibration module 251). Such calibration was discussed
previously and will also be revisited below.
[0111] In an exemplary configuration, one or more photo detectors
(e.g., photo diodes) and/or antenna elements (e.g., individual
antennas or elements of an antenna array) may be incorporated into
the faceplate of the television receiver 650. Note that additional
sensors positioned away from the television receiver 650 may also
be utilized (e.g., any of the previously discussed sensor
placements).
[0112] For example, in a photo detector implementation, passive
photo detectors may receive varying amounts of respective light
energy depending on the pointing direction of a light source aimed
at the screen. As discussed previously, directed energy (e.g.,
light, RF, acoustic, etc.) may be transmitted in a pattern (or
envelope), so even if a pointing device is pointed to a location on
the television screen 630 along pointing line 625, sensors
off-screen (e.g., sensors integrated into the television receiver
650) may still receive energy from the transmission (albeit likely
not with the same intensity at which energy is delivered along the
pointing line 625). Also for example, received signals (e.g.,
pulsed signals) may arrive at different sensors at different
respective times/phases (e.g., being indicative of relative
position and/or pointing direction, which may also be utilized in a
pointing determination).
[0113] In a photo detector implementation (e.g., utilizing photo
diodes), photo diodes may, for example, be tuned to react to
particular light frequencies to reduce interference from output
pixel light and/or associated reflections, ambient light, room
lighting, etc. As a non-limiting example, photo diodes may be tuned
to detect light that is not visible to the human eye, visible light
frequencies that are relatively rare, light patterns that are
unlikely to occur in a television program (e.g., particular pulse
codes), etc. In one example, the photo detectors integrated with
the television receiver 650 may comprise photo diodes that operate
to detect energy from a laser pointer or directed infrared energy
from a controller (or other pointing device). Note that analogously
to the on-screen sensors discussed previously, various aspects may
comprise mounting (e.g., adhering) sensors to various television
receiver 650 locations and/or to various off-receiver components.
Such sensor installation may, for example, occur at the factory or
after-market by a technician or user.
[0114] In an antenna element implementation, an array of antenna
elements may be positioned at locations on the television receiver
650 (e.g., only on the television receiver 650 and/or at locations
around the television receiver 650). In an RF antenna
implementation, passive antennas (or elements of an overall antenna
matrix) may receive varying amounts of respective RF energy
depending on the pointing direction of a directional RF source
pointed at a location on the screen. Also for example, received
signals (e.g., pulsed signals) may arrive at different antennas at
different respective times/phases (e.g., being indicative of
relative position and/or pointing direction, which may also be
utilized in a pointing determination). Note that analogously to the
on-screen sensors discussed previously, various aspects may
comprise mounting (e.g., adhering) sensors to the television
receiver 650. Such sensor installation may, for example, occur at
the factory or after-market by a technician or user.
[0115] In an exemplary scenario, a user may point a pointing device
(e.g., a remote controller, a laser pointer, directional RF
transmitter, specifically designed eyewear, a mobile computing
device, a mobile communication device, a gesture tracking device or
glove, etc.) at the television screen 603, where the pointing
device directs transmitted energy (e.g., light and/or RF energy
and/or acoustic energy) at a particular location on the television
screen 603 to which the user is pointing with the pointing device.
Note that such transmitted energy will likely be transmitted
directionally and be associated with an intensity or gain (or
energy) pattern with the highest intensity at the center of the
pattern (i.e., along the pointing line 625) and decreasing as a
function of angle from the center of the pattern. Such a gain
pattern was discussed previously in the discussion of FIG. 4.
[0116] In such an exemplary scenario, each sensor of the sensors
integrated into the television receiver 650 off-television will
likely receive some respective amount of energy. For example, along
a particular axis, the sensor nearest to the screen pointing
location 630 (i.e., along the pointing line 625) will likely
receive the highest amount of energy, a sensor next nearest to the
screen pointing location 630 will likely receive a next highest
range of energy, and sensors away from the pointing location 630
will likely receive progressively less amounts of energy from the
pointing device 620, as a function of distance from the pointing
location 630 and/or angle off the pointing line 625 (e.g., until
such energy is lost in the noise floor). For example, sensor 662 is
nearest to the pointing location 630 and will likely receive the
highest energy, sensors 661 and 663 are further from the pointing
location 630, etc., and so on.
[0117] Note that in the implementation illustrated in FIG. 6, in
particular since there are a relatively low number of sensors,
signals from a same sensor may be utilized in determining multiple
axes of pointing location. As mentioned previously, a calibration
procedure may be performed when the system 600 is configured to
assist in such pointing determination.
[0118] In an exemplary scenario, the television receiver 650 (e.g.,
the user interface module 240 of the television receiver 200
illustrated in FIG. 2) may receive signals indicative of the energy
received by the sensors of the television receiver 650. The
television receiver 650 may receive such signals in various
manners, depending on the degree of integration of such sensors
into the television receiver 650 and/or various components of the
television system 600. For example, in an exemplary scenario where
the sensors are fully integrated into the television receiver 650,
the television receiver 550 may receive such signals via direct
internal link with such sensors. Also for example, in a scenario
where various sensors are off the television receiver 650, the
television receiver 650 may receive information from such sensors
via direct communication link or via communication link with the
various components with which such sensors are integrated.
[0119] The user interface module 240 may then, for example, provide
information of such received sensor signals to the sensor
processing module 253 for processing. The sensor processing module
253 may then, for example, operate to process such information to
determine the screen pointing location. The sensor processing
module 253 may perform such processing in any of a variety of
manners, non-limiting examples of which will be provided below.
[0120] In an exemplary scenario, the sensor processing module 253
may operate to estimate a position between sensor positions based
on relative sensor energy. For example, in the horizontal
dimension, sensor 662 may correspond to a relatively high amount of
energy, and sensor 663 may correspond to a relatively low amount of
received energy. The sensor processing module 253 may, for example,
estimate a horizontal position relatively closer to sensor 662 by
an amount proportional to the relative difference between
respective amounts of energy. The sensor processing module 253 may
perform a similar estimation utilizing sensors 661 and 664. Various
horizontal position estimations may then be averaged. Alternatively
for example, respective energies for the left side sensors 661, 662
may be averaged, respective energies for the right side sensors
663, 664 sensors may be averaged, and such left and right speaker
average energies may then be utilized (e.g., in conjunction with
energy pattern characteristics) to determine a horizontal pointing
location. The sensor processing module 253 may then, for example,
perform a similar pointing direction estimate in the vertical
direction.
[0121] In another exemplary scenario, a calibration procedure may
be performed to determine an expected sensor energy level (e.g.,
absolute or relative) when the user is pointing at the sensor
(and/or other known locations). In such a scenario, combined with a
gain pattern and user (or pointing device) location relative to the
television 601, a first line (e.g., a circle or arc) may be drawn
around a first sensor 662. Similarly, a second line (e.g., a circle
or arc) may be drawn around a second sensor 663, and the
intersection of the first and second lines utilized as an estimate
of pointing location. Additional lines associated with other
sensors may also be utilized. Such additional lines may, for
example, be utilized when selecting between multiple line
intersections or to increase accuracy and/or resolution of the
pointing determination. Note that such line intersection solution
may be applied to any of the previously discussed scenarios (e.g.,
as illustrated in FIGS. 3-5) or other scenarios discussed herein. A
non-limiting example of this was presented in the discussion of
FIG. 3, and another example will be provided in the following
discussion of FIG. 7.
[0122] After determining on-screen pointing location, the
television receiver 650 may communicate information of such
determined location in various manners. For example and without
limitation, the sensor processing module 253 of the television
receiver 200 may utilize the television interface module 235 to
communicate information of such on-screen pointing location to the
television 601 for presentation to the user on the television
screen 603. Also for example, the sensor processing module 253 may
utilize the user interface module 240 to communicate information of
such on-screen pointing location to the television controller 620
for presentation to the user (e.g., on a display of the television
controller 620). Such communication will also be addressed in the
discussions of FIGS. 9-10.
[0123] Various aspects of the present invention may also, for
example, include one or more sensors incorporated into the pointing
device. FIG. 7 is a diagram illustrating an exemplary television
system 700 utilizing pointing device sensors in accordance with
various aspects of the present invention.
[0124] The exemplary television system 700 includes a television
701 having a television screen 703. The television system 700 also
comprises a television receiver 750 that is communicatively coupled
to the television 701 via a communication link 751 (e.g., a two-way
communication link providing video information to the television
701 and/or receiving sensor information from the television 701).
The exemplary television receiver 750 is also communicatively
coupled to the television controller 720 via a communication link
752.
[0125] The television receiver 750 may share any or all aspects
with the exemplary receivers (150, 200, 350, 450, 550 and 650)
discussed previously and all other receivers discussed herein.
Accordingly, various aspects of the television receiver 750 will be
explained herein with reference to various components of the
exemplary television receiver 200 illustrated in FIG. 2.
[0126] The television system 700 includes a television controller
720 (e.g., a pointing device) that comprises one or more sensors
(e.g., a plurality of antenna array elements, a plurality of photo
detectors, etc.). In such a configuration, sensor information may
be communicated to the television receiver 750 (e.g., to the user
interface module 240 via the first 210 or second 220 communication
interface modules). Such sensor information may, for example, be
communicated to the television receiver 750 directly (e.g., via
communication link 752) or indirectly (e.g., via the television 701
and communication link 751). Such information may then be
communicated to the sensor processing module 253 for the
determination of an on-screen pointing location.
[0127] In the exemplary configuration, the television 701 includes
eight emitters (e.g., light emitters, RF transmitters, etc.)
located around the border of the television screen 703. Note that
such emitters may be positioned anywhere proximate the television
system 700. For example, the television 701 includes a first
emitter 711, second emitter 712, third emitter 713, fourth emitter
714, fifth emitter 715, sixth emitter 716, seventh emitter 717 and
eighth emitter 718. Such emitters may each emit a signal that may
be received at sensors on-board the controller 720. Such sensors
may, for example, make up a directional receiver. In such a
configuration, the controller 720 (or other pointing device) may be
pointed to a location 730 on the screen 703 along a pointing line
725. With such an orientation and a directional signal reception
pattern, the sensors on-board the controller 720 will receive the
emitted signals at respective signal levels. Such sensor signals
may then be processed in a manner similar to the manners discussed
above to determine the on-screen pointing direction for the
pointing device 720.
[0128] For example, through a calibration procedure, it may be
known that the pointing device at a particular location should
receive a particular amount of energy from each of the emitters
711-718 when pointed directly at such emitters (or at some other
known location). In such a scenario, the pointing device may
measure respective signal energies received from each of the
emitters (e.g., each distinguishable by frequency, coding, timing
and/or timeslotting, etc.) and communicate such information to the
television receiver 750. The pointing device 720 may also, for
example, communicate pointing device position (and/or orientation)
information to the television receiver 750. The television receiver
750 may receive such sensor and/or position information via at
least one of the communication interface modules 210, 220 and/or
the user interface module 240 and process such sensor information
with the sensor processing module 253.
[0129] The sensor processing module 253 may, for example, select a
first emitter 712 (e.g., the emitter corresponding to the highest
energy received at the pointing device). The sensor processing
module 253 may then process the location of the pointing device,
the receive gain pattern for the pointing device, and the energy
received from the first emitter 712 to determine a first figure
(e.g., an arc 752) along which the pointing device, if pointed,
would be expected to receive the measured energy. Similarly, the
sensor processing module 253 may perform such a procedure for a
second emitter 711 resulting in a second figure (e.g., an arc 751).
The intersection of such arcs may be utilized as an estimate of
on-screen pointing location. Additionally, for accuracy or for
selecting between multiple intersection points, should they occur,
the sensor processing module 253 may perform such a procedure for a
third emitter 714 resulting in a third figure (e.g., an arc 754),
and so on. The intersection of the three arcs 752, 751, 754 may
then be utilized as an estimate of on-screen pointing location.
[0130] Alternatively, the solution need not be based on a known
position (location) of the pointing device, nor on absolute
received energy levels. In such a scenario, differences in received
energy from the various emitters may be processed with or without
position information of the on-screen pointing device. For example,
the pointing device 720 may have six degrees of freedom (e.g.,
three positional degrees of freedom and three orientational degrees
of freedom). In such a scenario, if the position and orientation of
the television 701 are known, the unknown six degrees of freedom
for the pointing device 720 may be ascertained by processing six
known values related to such six degrees of freedom (e.g., related
by a known signal energy pattern). In such a scenario, measurements
associated with six emitters on the television (and potentially
more) may be utilized to solve for the six degrees of freedom of
the pointing device 720.
[0131] The above-mentioned exemplary scenarios were presented to
illustrate numerous manners in which the television receiver 750
(e.g., sensor processing module 253) may operate to determine
on-screen pointing location. Such examples are merely exemplary and
thus the scope of various aspects of the present invention should
not be limited by any particular characteristics of such examples
unless explicitly claimed.
[0132] As discussed above, the calibration module 251 of the
television receiver 200 may operate to perform calibration
operations. Such calibrating may be performed in any of a variety
of manners. For example and without limitation, calibration may be
utilized to determine expected received energy when transmitters
and receivers are located and oriented in a particular manner. For
example, a non-limiting example of a calibration procedure may
comprise presenting an on-screen target at various locations and
measuring respective sensor signals received when the pointing
device is being pointed at such targets. Also for example, a
calibration procedure may comprise directing a user (e.g., using
the user interface module 240) to point to each of a plurality of
sensors to determine an expected amount of received energy when the
user is pointing directly at such sensors.
[0133] As mentioned previously, signal energy (or gain) pattern may
be utilized in various on-screen pointing determinations. Such an
energy (or gain) pattern may be predefined for a particular
pointing device (e.g., at the factory), but may also be measured by
the television receiver 200. In a non-limiting example, the
calibration module 251 may direct the user to utilize a pointing
device to point to a location on the screen and process information
received from multiple sensors (e.g., embedded in the screen,
embedded in the television around the border of the screen, located
in off-television devices, located on the television receiver 750,
located in the pointing device, etc.) to develop a custom gain
pattern for the particular pointing device. For example, such
calibration may determine the shape of the gain pattern, the signal
energy falloff characteristics, etc.
[0134] Various aspects discussed above included the processing of
position information. In such exemplary cases, the television
receiver 200 may comprise one or more location modules 252 that
operate to determine relevant position information. The location
module 252 may operate to perform such location determining (e.g.,
of the user or pointing device and/or the television) in any of a
variety of manners. For example, the location module 252 may
utilize a communication interface module 210, 220 to receive
position information (e.g., of the television receiver 200 or of
the pointing device) from an external source of such information
(e.g., global positioning system, cellular triangulation system,
home triangulation system, etc.).
[0135] Also for example, the location module 252 may receive
position information directly from the pointing device (e.g., where
such pointing device has position-determining capability). For
example, in a non-limiting exemplary scenario, where the pointing
device is a handheld computer, such computer may comprise GPS (or
A-GPS) capability to determine its position. In such a scenario,
the pointing device may wirelessly communicate information of its
position to the television receiver 200, and ultimately to the
location module 252 via a communication interface module 210,
220.
[0136] Additionally for example, the location module 232 may
operate to process sensor information to determine location of the
pointing device (e.g., location in relation to the television
screen). For example, as mentioned previously, a signal (e.g., a
pulse) transmitted from a pointing device to the television will
arrive at different sensors at different points in time depending
on the respective distance from the pointing device to each sensor.
The location module 232 may process such time-of-arrival
information at various sensors to determine the position of the
pointing device relative to the television. Similarly, in a
scenario including signal emitters associated with the television
and sensors on the pointing device, simultaneously transmitted
signals from different emitters will arrive at the pointing device
at different respective times depending on the position of the
pointing device relative to such emitters. Alternatively, the
location module 232 may also operate to process phase difference
information (in addition to timing information or instead of such
information) to determine pointing device location.
[0137] Once the television receiver 200 (e.g., the sensor
processing module 253) determines an on-screen pointing location,
the television receiver 200 may utilize such information in any of
a variety of manners. For example and without limitation, the
sensor processing module 253 may operate to generate information of
the determined on-screen pointing location, and one or more modules
of the television receiver 200 may operate to communicate a signal
(e.g., to a television, television controller, other display
device, etc.) that comprises characteristics that cause
presentation of a visual indication (e.g., on the television
screen, controller screen, other display, etc.) to indicate to the
user the on-screen location to which the television receiver 200
has determined the user is pointing. Such a visual indication may,
for example, comprise characteristics of a cursor or other
graphical construct, bright spot, highlighting, color variation,
brightness variation, etc. For example, the television receiver 200
may operate to overlay such indication on video content (e.g.,
television programming) being presented to the user (e.g.,
presented on the television screen, presented on a screen of the
television controller, etc.).
[0138] Additionally for example, the sensor processing module 253
may provide information of the determined on-screen pointing
location to one or more other modules of the television receiver
200 (e.g., the processing module 250 and/or other modules thereof)
to identify an object in video content (e.g., television
programming) to which a user is pointing. In such an exemplary
scenario, one or more modules of the television receiver 200 may
operate to communicate signals (e.g., to a television, television
controller having a screen, other display device, etc.) that cause
highlighting of an object to which the user is pointing and/or
provide information regarding such object.
[0139] Further for example, various modules of the television
receiver 200 (e.g., the processor module 250) may operate to
communicate on-screen pointing location information to television
system components separate from the television (e.g., to a
different television receiver, video recorder, remote programming
source, communication network infrastructure, advertising company,
provider of goods and/or services, etc.). Also for example, various
modules of the television receiver 200 may operate to communicate
information of the determined on-screen pointing location to the
pointing device of the user (e.g., for providing pointing feedback
to the user at a remote controller, etc.).
[0140] FIG. 2 provided a diagram illustrating an exemplary
television receiver 200 in accordance with various aspects of the
present invention. FIG. 8 provides another diagram illustrating an
exemplary television receiver 800 in accordance with various
aspects of the present invention. The exemplary television receiver
800 may share any or all aspects with any of the television
receivers discussed herein and illustrated in FIGS. 1-7. For
example, the exemplary television receiver 800 (or various modules
thereof) may operate to perform any or all functionality discussed
herein. As with the exemplary television receiver 200, the
components of the exemplary television receiver 800 may be
co-located a single housing.
[0141] For example, the television receiver 800 comprises a
processor 830. Such a processor 830 may, for example, share any or
all characteristics with the processor 250 discussed with regard to
FIG. 2. Also for example, the television receiver 800 comprises a
memory 840. Such memory 840 may, for example, share any or all
characteristics with the memory 260 discussed with regard to FIG.
2.
[0142] Also for example, the television receiver 800 may comprise
any of a variety of user interface module(s) 850. Such user
interface module(s) 850 may, for example, share any or all
characteristics with the user interface module(s) 240 discussed
previously with regard to FIG. 2. For example and without
limitation, the user interface module(s) 850 may comprise: a
display device, a camera (for still or moving picture acquisition),
a speaker, an earphone (e.g., wired or wireless), a microphone, a
video screen (e.g., a touch screen display), a vibrating mechanism,
a keypad, a remote control interface, and/or any of a variety of
other user interface devices (e.g., a mouse, a trackball, a touch
pad, touch screen, light pen, game controlling device, etc.).
[0143] The exemplary television receiver 800 may also, for example,
comprise any of a variety of communication modules (805, 806, and
810). Such communication module(s) may, for example, share any or
all characteristics with the communication interface module(s) 210,
220 discussed previously with regard to FIG. 2. For example and
without limitation, the communication interface module(s) 810 may
comprise: a Bluetooth interface module; an IEEE 802.11, 802.15,
802.16 and/or 802.20 module; any of a variety of cellular
telecommunication interface modules (e.g., GSM/GPRS/EDGE,
CDMA/CDMA2000/1x-EV-DO, WCDMA/HSDPA/HSUPA, TDMA/PDC, WiMAX, etc.);
any of a variety of position-related communication interface
modules (e.g., GPS, A-GPS, etc.); any of a variety of
wired/tethered communication interface modules (e.g., USB, Fire
Wire, RS-232, HDMI, component and/or composite video, Ethernet,
wireline and/or cable modem, etc.); any of a variety of
communication interface modules related to communicating with
external memory devices; etc. The exemplary television receiver 800
is also illustrated as comprising various wired 806 and/or wireless
805 front-end modules that may, for example, be included in the
communication interface modules and/or utilized thereby.
[0144] The exemplary television receiver 800 may also comprise any
of a variety of signal processing module(s) 890. Such signal
processing module(s) 890 may, for example, be utilized to assist in
processing various types of information discussed previously (e.g.,
with regard to sensor processing, position determination, video
processing, image processing, audio processing, general user
interface information data processing, etc.). For example and
without limitation, the signal processing module(s) 890 may
comprise: video/graphics processing modules (e.g. MPEG-2, MPEG-4,
H.263, H.264, JPEG, TIFF, 3-D, 2-D, MDDI, etc.); audio processing
modules (e.g., MP3, AAC, MIDI, QCELP, AMR, CMX, etc.); and/or
tactile processing modules (e.g., Keypad I/O, touch screen
processing, motor control, etc.).
[0145] Various aspects of the present invention were previously
exemplified by non-limiting illustrations and described in terms of
operations performed by various modules of the television. Various
aspects of the present invention will now be illustrated in the
form of method flow diagrams.
[0146] FIG. 9 is a flow diagram 900 illustrating the generation of
on-screen pointing information (e.g., in a television receiver) in
accordance with various aspects of the present invention. The
exemplary method 900 may, for example, share any or all
characteristics with the television receiver operation discussed
previously.
[0147] The exemplary method 900 may begin executing at step 905.
The exemplary method 900 may begin executing in response to any of
a variety of causes and/or conditions. For example and without
limitation, the method 900 may begin executing in response to a
user command to begin, detected user interaction with a pointing
device (e.g., a television controller), detected user presence in
the vicinity, detected user interaction with a television
implementing the method 900, etc. Also for example, the method 900
may begin executing in response to a television presenting
programming or other video content for which on-screen pointing is
enabled and/or relevant.
[0148] The exemplary method 900 may, for example at step 910,
comprise receiving pointing sensor information. For example and
without limitation, step 910 may comprise any or all sensor
information receiving characteristics described previously with
regard the various modules of the exemplary television receivers
illustrated in FIGS. 1-8 and discussed previously. For example,
step 910 may share any or all sensor information receiving
characteristics discussed previously with regard to at least the
user interface module 240, television interface module 235,
processor module 250, communication interface modules 210, 220,
sensor processing module 253, location module 252 and calibration
module 251.
[0149] Step 910 may, for example, comprise receiving sensor
information from (or associated with) sensors integrated in the
television receiver. Also for example, step 910 may comprise
receiving sensor information from (or associated with) off-receiver
sensors (e.g., integrated with or attached to a television,
off-television sensors, sensors integrated with a pointing device
(e.g., a television controller), etc. As discussed previously, such
sensors may comprise any of a variety of characteristics, including
without limitation, characteristics of light sensors, RF sensors,
acoustic sensors, active and/or passive sensors, etc.
[0150] In general, step 910 may comprise receiving pointing sensor
information. Accordingly, the scope of various aspects of the
present invention should not be limited by characteristics of any
particular manner of receiving pointing sensor information unless
explicitly claimed.
[0151] The exemplary method 900 may, at step 920, comprise
processing received sensor information (e.g., as received at step
910) to determine a location on a screen of the television to which
a user is pointing (e.g., pointing with a pointing device). For
example and without limitation, step 920 may comprise any or all
pointing location processing characteristics described previously
with regard the various modules of the exemplary television
receivers illustrated in FIGS. 1-8 and discussed previously. For
example, step 920 may share any or all pointing location
determining characteristics discussed previously with regard to at
least the processor module 250, sensor processing module 253,
location module 252 and calibration module 251.
[0152] Step 920 may, for example, comprise determining on-screen
pointing location in any of a variety of manners. For example, step
920 may comprise determining on-screen pointing location based on a
location of a selected sensor, based on interpolation between
sensor locations (e.g., linear and/or non-linear interpolation),
based on determining energy pattern intersection(s), etc. Many
examples of such determining were provided previously.
[0153] In general, step 920 may comprise processing received sensor
information (e.g., independently and/or in conjunction with other
information) to determine a location on a screen of the television
to which a user is pointing. Accordingly, the scope of various
aspects of the present invention should not be limited by
characteristics of any particular manner of performing such
processing unless explicitly claimed.
[0154] The exemplary method 900 may, at step 930, comprise
generating information indicative of a determined on-screen
pointing location (e.g., as determined at step 920). For example
and without limitation, step 930 may comprise any or all pointing
location information generation characteristics described
previously with regard the various modules of the exemplary
television receivers illustrated in FIGS. 1-8 and discussed
previously. For example, step 930 may share any or all information
generation characteristics discussed previously with regard to at
least the processor module 250, sensor processing module 253,
location module 252, calibration module 251, television interface
module 235, user interface module 240 and/or communication
interface modules 210, 220.
[0155] Step 930 may, for example, comprise generating such
information in any of a variety of manners. For example, step 930
may comprise generating on-screen pointing location data to
communicate to internal modules of the television receiver, to
equipment external to the television receiver (e.g., to the
television and/or television controller), to television network
components, to a television programming source, etc. Such
information may, for example, be communicated to various system
components and may also be presented to the user (e.g., utilizing
visual feedback displayed on a screen of a television, television
controller, etc.). Such information may, for example, be generated
in the form of screen coordinates, identification of a video
content object (e.g., a programming object or person) to which an
on-screen pointing location corresponds, generation of an on-screen
cursor or highlight or other graphical feature, etc.
[0156] In general, step 930 may comprise generating information
indicative of a determined on-screen pointing location.
Accordingly, the scope of various aspects of the present invention
should not be limited by characteristics of any particular manner
of generating such information unless explicitly claimed.
[0157] The exemplary method 900 may, at step 995, comprise
performing continued processing. Such continued processing may
comprise characteristics of any of a variety of types of continued
processing, various examples of which were presented previously.
For example and without limitation, step 995 may comprise looping
execution flow back up to any earlier step (e.g., step 910). Also
for example, step 995 may comprise presenting (or causing the
presentation of) visual feedback indicia of the on-screen pointing
location for a user. Additionally for example, step 995 may
comprise communicating information of the on-screen pointing
location to system components external to the television receiver
implementing the method 900 (e.g., to a television, television
controller, etc.). Further for example, step 995 may comprise
utilizing the on-screen pointing information to identify a video
content object (e.g., an object presented in television
programming) to which a user is pointing, etc.
[0158] In general, step 995 may comprise performing continued
processing. Accordingly, the scope of various aspects of the
present invention should not be limited by characteristics of any
particular manner of performing continued processing unless
explicitly claimed.
[0159] Turning next to FIG. 10, such figure is a flow diagram 1000
illustrating the generation of on-screen pointing information
(e.g., in a television receiver) in accordance with various aspects
of the present invention. The exemplary method 1000 may, for
example, share any or all characteristics with the television
receiver operation discussed previously (e.g., in reference to
FIGS. 1-9).
[0160] The exemplary method 1000 may begin executing at step 1005.
Step 1005 may, for example, share any or all characteristics with
step 905 of the exemplary method 900 illustrated in FIG. 9 and
discussed previously.
[0161] The exemplary method 1000 may, for example at step 1008,
comprise performing a calibration procedure with the user. Such a
calibration procedure may, for example, be performed to develop a
manner of processing received sensor information to determine
on-screen pointing location. Step 1008 may, for example, comprise
any or all calibration aspects discussed previously (e.g., with
reference to the calibration module 251).
[0162] The exemplary method 1000 may, for example at step 1010,
comprise receiving pointing sensor information. For example and
without limitation, step 1010 may comprise any or all sensor
information receiving characteristics described previously with
regard the various modules of the exemplary television receivers
illustrated in FIGS. 1-8 and FIG. 9 (e.g., step 910) and discussed
previously.
[0163] The exemplary method 1000 may, for example at step 1015,
comprise determining user position (e.g., determining position of a
user pointing device). For example and without limitation, step
1015 may comprise any or all position determining characteristics
discussed previously with regard to FIGS. 1-9. Note that position
may also, for example, include orientation.
[0164] For example, step 1015 may share any or all position
determining characteristics discussed previously with regard to at
least the processor module 250, sensor processing module 253,
location module 252 and calibration module 251. For example, step
1015 may comprise determining user position based, at least in
part, on received sensor signals. Also for example, step 1015 may
comprise determining user position based, at least in part, on
position information received from one or more systems external to
the television receiver implementing the method 1000.
[0165] In general, step 1015 may comprise determining user position
(e.g., pointing device position). Accordingly, the scope of various
aspects of the present invention should not be limited by
characteristics of any particular manner of determining user
position unless explicitly claimed.
[0166] The exemplary method 1000 may, for example, at step 1020,
comprise processing received sensor information (e.g., as received
at step 1010) and/or user position information (e.g., as determined
at step 1015) to determine a location on a screen of the television
to which a user is pointing (e.g., pointing with a pointing
device). For example and without limitation, step 1020 may comprise
any or all pointing location determination characteristics
described previously with regard the various modules of the
exemplary television receivers illustrated in FIGS. 1-8 and FIG. 9
(e.g., step 920) and discussed previously. For example, step 1020
may share any or all pointing location determining characteristics
discussed previously with regard to at least the processor module
250, sensor processing module 253, location module 252 and
calibration module 251.
[0167] Step 1020 may, for example, comprise determining on-screen
pointing location in any of a variety of manners. For example, step
1020 may comprise determining on-screen pointing location based on
a location of a selected sensor, based on location of the pointing
device, based on interpolation between sensor locations (e.g.,
linear and/or non-linear interpolation), based on energy pattern
intersection points, etc. Many examples of such determining were
provided previously.
[0168] In general, step 1020 may comprise processing received
sensor information and/or user position information to determine a
location on a screen of the television to which a user is pointing
(e.g., pointing with a pointing device). Accordingly, the scope of
various aspects of the present invention should not be limited by
characteristics of any particular manner of performing such
processing unless explicitly claimed.
[0169] The exemplary method 1000 may, at step 1030, comprise
generating information indicative of a determined on-screen
pointing location (e.g., as determined at step 1020). For example
and without limitation, step 1030 may comprise any or all
information generation characteristics described previously with
regard the various modules of the exemplary television receivers
illustrated in FIGS. 1-8 and FIG. 9 (e.g., step 930) and discussed
previously. For example, step 1030 may share any or all information
generation characteristics discussed previously with regard to at
least the processor module 250, sensor processing module 253,
location module 252, calibration module 251, television interface
module 235, user interface module 240 and/or communication
interface modules 210, 220.
[0170] The exemplary method 1000 may, at step 1095, comprise
performing continued processing. Such continued processing may
comprise characteristics of any of a variety of types of continued
processing, various examples of which were presented previously.
For example and without limitation, step 1095 may comprise looping
execution flow back up to any earlier step (e.g., step 1008). Also
for example, step 1095 may comprise presenting (and/or causing the
presentation of) visual feedback indicia of the on-screen pointing
location for a user. Additionally for example, step 1095 may
comprise communicating information of the on-screen pointing
location to system components external to the television receiver
implementing the method 1000. Further for example, step 1095 may
comprise utilizing the on-screen pointing information to identify a
video content object (e.g., an object presented in television
programming) to which a user is pointing, etc.
[0171] In general, step 1095 may comprise performing continued
processing. Accordingly, the scope of various aspects of the
present invention should not be limited by characteristics of any
particular manner of performing continued processing unless
explicitly claimed.
[0172] In summary, various aspects of the present invention provide
a system and method in a television receiver for generating screen
pointing information. While the invention has been described with
reference to certain aspects and embodiments, it will be understood
by those skilled in the art that various changes may be made and
equivalents may be substituted without departing from the scope of
the invention. In addition, many modifications may be made to adapt
a particular situation or material to the teachings of the
invention without departing from its scope. Therefore, it is
intended that the invention not be limited to the particular
embodiment disclosed, but that the invention will include all
embodiments falling within the scope of the appended claims.
* * * * *