U.S. patent application number 13/184522 was filed with the patent office on 2012-10-11 for user interface for audio video display device such as tv.
This patent application is currently assigned to SONY CORPORATION. Invention is credited to Steven Friedlander, Sabrina Tai-Chen Yeh, DAVID YOUNG.
Application Number | 20120260171 13/184522 |
Document ID | / |
Family ID | 46965840 |
Filed Date | 2012-10-11 |
United States Patent
Application |
20120260171 |
Kind Code |
A1 |
YOUNG; DAVID ; et
al. |
October 11, 2012 |
USER INTERFACE FOR AUDIO VIDEO DISPLAY DEVICE SUCH AS TV
Abstract
For a graphical user interface (GUI) for an audio video display
device (AVDD) showing plural content information panels arranged in
a grid, each content panel belongs to a respective category, with
content panels of different categories being mixed on the screen. A
category icon with an associated color and shape is presented next
to each content panel to make it easy for a viewer to understand
which category of item the content panels represents. Examples of
some categories are favorites, recorded, recommended, frequently
viewed, and now playing. Next to each icon, a numeral pertaining to
the grid position of the content may also appear to facilitate
voice selection of a content panel by number. The numerals are
enlarged when voice command input is enabled.
Inventors: |
YOUNG; DAVID; (San Diego,
CA) ; Yeh; Sabrina Tai-Chen; (Laguna Beach, CA)
; Friedlander; Steven; (San Diego, CA) |
Assignee: |
SONY CORPORATION
|
Family ID: |
46965840 |
Appl. No.: |
13/184522 |
Filed: |
July 16, 2011 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61473098 |
Apr 7, 2011 |
|
|
|
Current U.S.
Class: |
715/719 |
Current CPC
Class: |
G06F 3/04842 20130101;
G06F 3/017 20130101; G06F 3/0485 20130101; H04N 21/4314 20130101;
H04N 21/482 20130101; H04N 21/42204 20130101 |
Class at
Publication: |
715/719 |
International
Class: |
G06F 3/048 20060101
G06F003/048 |
Claims
1. Audio video display device (AVDD) comprising: processor; video
display presenting demanded images; computer readable storage
medium bearing instructions executable by the processor to: present
on the display a first graphical user interface (GUI) including
plural content panels, each content panel representing a respective
video content; and present on the display next to each respective
content panel a respective numeral representing a position in the
GUI of the respective content panel, the numerals having ah
appearance when voice command input is enabled that is not present
on the display when voice command input is not enabled.
2. The AVDD of claim 1, wherein the GUI includes eight content
panels and a current video panel together arranged in a three by
three grid, the eight content panels being changed responsive to
viewer input requiring a change to the content panels, the current
video panel not being changed responsive to viewer input requiring
a change to the content panels.
3. The AVDD of claim 1, wherein the processor, responsive to a
determination that voice command input mode is enabled, indicates
that voice command input mode is enabled using a voice command
input mode enable icon on the display.
4. The AVDD of claim 3, wherein the voice command input mode enable
icon is not presented responsive to a determination that voice
command input mode is not enabled, the processor causing the voice
command input mode enable icon to appear on the display responsive
to a determination that voice command input mode is enabled.
5. The AVDD of claim 3, wherein the voice command input mode enable
icon is presented responsive to a determination that voice command
input mode is not enabled with a first appearance, the processor
causing the voice command input mode enable icon to appear on the
display with a second appearance different from the first
appearance responsive to a determination that voice command input
mode is enabled.
6. The AVDD of claim 1, wherein the numerals do not appear on the
display when voice command input is not enabled.
7. The AVDD of claim 1, wherein the numerals appear on the display
with a first appearance when voice command input is not enabled and
with a second appearance when voice command input is enabled.
8. The AVDD of claim 7, wherein the second appearance is
characterized by being larger than the first appearance.
9. Method, comprising: presenting a graphical user interface (GUI)
on an audio video display device (AVDD) showing plural content
information panels arranged in a grid; and presenting a respective
numeral next to but not superimposed on each panel, the numerals
indicating a grid position of the respective panel.
10. The method of claim 9, wherein the numerals have an appearance
when voice command input is enabled that is not present on the
display when voice command input is not enabled.
11. The method of claim 9, wherein the GUI includes eight content
panels and a current video panel together arranged in a three by
three grid, the eight content panels being changed responsive to
viewer input requiring a change to the content panels, the current
video panel not being changed responsive to viewer input requiring
a change to the content panels.
12. The method of claim 10, comprising, responsive to a
determination that voice command input mode is enabled, indicating
that voice command input mode is enabled using a voice command
input mode enable icon on the GUI.
13. The method of claim 12, wherein the voice command input mode
enable icon is not presented responsive to a determination that
voice command input mode is not enabled, the method including
causing the voice command input mode enable icon to appear on the
display responsive to a determination that voice command input mode
is enabled.
14. The method of claim 12, wherein the voice command input mode
enable icon is presented responsive to a determination that voice
command input mode is not enabled with a first appearance, the
method including causing the voice command input mode enable icon
to appear on the display with a second appearance different from
the first appearance responsive to a determination that voice
command input mode is enabled.
15. The method of claim 9, wherein the numerals do not appear on
the display when voice command input is not enabled.
16. The method of claim 9, wherein the numerals appear on the
display with a first appearance when voice command input is not
enabled and with a second appearance when voice command input is
enabled.
17. The method of claim 16, wherein the second appearance is
characterized by being larger than the first appearance.
18. Audio video display device (AVDD) comprising: processor; video
display presenting demanded images; computer readable storage
medium bearing instructions executable by the processor to: present
on the display a first graphical user interface (GUI) including
plural content panels, each content panel representing a respective
video content; present on the display next to each respective
content panel a respective numeral, the numerals having an
appearance when voice command input is enabled that is not present
on the display when voice command input is not enabled; and present
on the display one and only one voice command input mode enable
icon indicating that voice command input is enabled.
19. The AVDD of claim 18, wherein the GUI includes eight content
panels and a current video panel together arranged in a three by
three grid the eight content panels being changed responsive to
viewer input requiring a change to the content panels, the current
video panel not being changed responsive to viewer input requiring
a change to the content panels.
20. The AVDD of claim 19, wherein the processor, responsive to a
determination that voice command input mode is enabled, causes the
voice command input mode enable icon to appear on the display if
not already appearing prior to the determination that voice command
input mode is enabled, or to change its appearance if already
appearing prior to the determination that voice command input mode
is enabled.
Description
[0001] This application claims priority to U.S. provisional patent
application Ser. No. 61/473,098, filed Apr. 7, 2011.
I. FIELD OF THE INVENTION
[0002] The present application relates generally to user interfaces
(UI) for audio video display devices (AVDD) such as televisions
(TVs).
II. BACKGROUND OF THE INVENTION
[0003] Graphical user interfaces (GUI) for audio video display
devices (AVDD) such as TVs have been provided for enabling viewers
to browse and select video streams for play on the TV. An
electronic program guide (EPG) is an example.
[0004] As understood herein, when a grid-based GUI is used it can
be helpful to label the grid items in a way that is easy to
understand and that can facilitate selection of a grid item by
numeral.
SUMMARY OF THE INVENTION
[0005] According to principles set forth further below, an audio
video display device (AVDD) has a processor, a video display
presenting demanded images, and a computer readable storage medium
bearing instructions executable by the processor. The processor
presents on the display a first graphical user interface (GUI)
including plural content panels, each content, panel representing a
respective, video content. The instructions executed by the
processor also direct presentation on the display next to each
respective content panel a respective numeral representing a
position in the GUI of the respective content panel. The numerals
have an appearance when voice command input is enabled that is not
present on the display when voice command input is not enabled.
[0006] The GUI may include eight content panels and a current video
panel together arranged in a three by three grid. The eight content
panels can be changed, responsive to viewer input requiring a
change to the content panels, but the current video panel may not
being changed as a result of the same viewer input.
[0007] In response to a determination that voice command input mode
is enabled, the processor can indicate the enabled state of voice
command mode using a voice command input mode enable icon on the
display. Presentation of the voice command input mode enable icon
is not executed by the processor when there is a determination that
voice command input mode is not enabled.
[0008] The voice command input mode enable icon can be presented
responsive to a determination that voice command input mode is not
enabled with a first appearance. The processor can cause the voice
command input mode enable icon to appear on the display with a
second appearance different from the first appearance in response
to a determination that voice command input mode is enabled. In one
example, the second appearance may be characterized by being larger
than the first appearance.
[0009] The numerals of the respective content panels may not appear
on the display when voice command input is not enabled. In other
examples, the numerals can appear on the display with a first
appearance when voice command input is not enabled and with a
second appearance when voice command input is enabled.
[0010] In another aspect, a method includes presenting a graphical
user interface (GUI) on an audio video display device (AVDD)
showing plural content information panels arranged in a grid. The
method also includes presenting a respective numeral next to but
hot superimposed on each panel, the numerals indicating a grid
position of the respective panel.
[0011] In another aspect, the AVDD includes a processor, a video
display presenting demanded images, a computer readable storage
medium bearing instructions executable by the processor. The
processor can present on the display a first graphical user
interface (GUI) including plural content panels, each content panel
representing a respective video content. The processor may present
on the display next to each respective content panel a respective
numeral, the numerals having an appearance when voice command input
is enabled that is not present on the display when voice command
input is not enabled. Thirdly, the processor can also present on
the display one and only one voice command input mode enable icon
indicating that voice command input is enabled.
[0012] The details of the present invention, both as to its
structure and operation, can best be understood in reference to the
accompanying drawings, in which like reference numerals refer to
like parts, and in which:
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] FIG. 1 is a block diagram of a non-limiting example system
in accordance with present principles;
[0014] FIG. 2 is a plan view of an example remote control (RC) that
may be used to control the graphical user interfaces (GUI)
described herein, schematically showing the processor and wireless
transceiver;
[0015] FIG. 3 is a screen shot of a nine panel GUI in a coarse mode
suitable for gesture control;
[0016] FIG. 4 is a screen shot of a nine panel GUI in a fine mode
suitable for RC control;
[0017] FIG. 5 is a screen shot of a first example of the GUI when
voice command input is not enabled, in which the voice command
input mode enable icon and the grid numerals do not appear;
[0018] FIG. 6 is a screen shot of a second example of the GUI when
voice command input is not enabled, in which the voice command
input mode enable icon and the grid numerals appear but less
prominently, either by being smaller and/or by being lowlighted,
compared to FIG. 7; and
[0019] FIG. 7 is a screen shot of an example of the GUI when voice
command input is, enabled, in which the voice command input mode
enable icon and the grid numerals appear in a prominent format.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
[0020] Referring initially to the non-limiting example embodiment
shown in FIG. 1, a system 10 includes an audio video display device
(AVDD) 12 such as a TV including a TV tuner 16 communicating with a
TV processor 18 accessing a tangible computer readable storage
medium 20 such as disk-based or solid state storage. The AVDD 12
can output audio on one or more speakers 22. The AVDD 12 can
receive streaming video from the Internet using a built-in wired or
wireless modem 24 communicating with the processor 18 which may
execute a software-implemented browser 26. Video is presented under
control of the TV processor 18 on a TV display 28 such as but not
limited to a high definition TV (HDTV) flat panel display, and may
be a touch screen display. User commands to the processor 18 may be
wirelessly received from a remote control (RC) 30 using, e.g., rf
or infrared. Audio-video display devices other, than a TV may be
used, e.g., smart phones, game consoles, personal digital,
organizers, notebook computers and other types of computers,
etc.
[0021] TV programming from one or more terrestrial TV broadcast
sources 32 as received by a terrestrial, broadcast antenna 34
which, communicates with the AVDD 12 may be presented on the
display 28 and speakers 22. The terrestrial broadcast programming
may conform to digital ATSC standards and may carry within it a
terrestrial broadcast EPG, although the terrestrial, broadcast EPG
may be received from alternate sources, e.g., the Internet via
Ethernet, or cable communication link, or satellite communication
link.
[0022] TV programming from a cable TV head end 36 may also be
received at the TV for presentation of TV signals on the display 28
and speakers 22. When basic cable only is desired, the cable from
the wall typically carries TV signals in QAM or NTSC format and is
plugged directly into the "F-type connector" 38 on the TV chassis
in the U.S., although the connector used for this purpose in other
countries may vary. In contrast, when the user has an extended
cable subscription for instance, the signals from the head end 36
are typically sent through a STB 40 which may be separate from or
integrated within the TV chassis but in any case which sends HDMI
baseband signals to the TV. Other types of connections may be used,
e.g., MOCA, USB, 1394 protocols, DLNA.
[0023] Similarly, HDMI baseband signals transmitted from a
satellite source 42 of TV broadcast signals received by an
integrated receiver/decoder (IRD) 44 associated with a home
satellite dish may be input to the AVDD 12 for presentation on the
display 28 and speakers 22. Also, streaming video may be received
from the Internet 46 for presentation on the display 28 and
speakers 22. The streaming video may be received at the computer
modem 24 of it may be received at an in-home modem 48 that is
external to the AVDD 12 and conveyed to the AVDD 12 over a wired or
wireless Ethernet link and received at an RJ45 or 802.11x antenna
on the TV chassis.
[0024] Also, in some embodiments a video camera 50, which may be
integrated in the chassis if desired or mounted separately and
electrically connected thereto, may be connected to the processor
18 to provide to the processor 18 video images of viewers looking
at the display 28. Furthermore, a microphone 52 may be provided on
the chassis or separate therefrom and can be electrically connected
to the processor 18 to provide viewer-generated, voice commands to
the processor 18.
[0025] FIG. 2 shows that an example RC 30 may include a power on
key 60 that can be toggled to energize and deenergize the AVDD 12.
A touch-sensitive pad 62 may be provided against which a user can
move his finger to correspondingly move a screen cursor on the
display 28. Tapping the pad 62 can generate a "select" signal, it
being understood that point-and-click devices other than the touch
sensitive pad 62 may be used.
[0026] Also, a back key 64 may be provided to cause the display 28
to go back a screen shot, i.e., to present the screen shot
immediately prior to the one being displayed when the back signal
is generated, so as to enable a user to navigate through the
various GUIs, shown herein. A home key 66 may be provided to cause
the below-described "home" GUI to be presented on the display 28,
and an options key 68 may be provided to allow a user to cause a
list of GUI options to be presented in the display 28. Volume
output by the speakers 22 of the AVDD 12 can be controlled using a
volume up/down rocker key 70. Manipulation of a key or the pad 62
on the RC 30 causes a respective signal to be sent to an RC
processor 72 in the RC 30, which transmits the corresponding
command to the AVDD 12 using a wireless transceiver 74 such as but
not limited to an infrared transceiver. In addition, if desired
four arrow keys may be arranged on the RC 30 to move a cursor up,
down, left, and right on the display 28. Also, channel up/down keys
may be provided as well as a microphone for voice input. A full
QWERTY keyboard of keypad may be provided if desired. The touch pad
62 may be provided with a buzzer or other device to generate
tactile feedback to the user.
[0027] FIGS. 3 and 4 show an example home GUI that may be presented
on the display 28 of the AVDD 12. In the embodiment shown in FIG.
3, nine information panels arranged as shown in three rows of three
panels each present visual content. The panels shown in FIG. 3 are
of equal size with each other and are rectangular as shown.
According to the embodiment shown in FIG. 3, the top left panel 76
of the GUI always shows the currently selected video content,
typically either televised, content, from a selected TV channel or
streaming video from a selected website. Information pertaining to
available video content is presented in the other eight panels.
This information typically includes a photograph of other image, a
content source logo indicating the source of the content
represented by the panel, and text typically describing the title
of the content and other information.
[0028] As shown at 78, a numeral may be provided next to each
panel, so that the panels may be numbered onscreen from one to nine
as shown. This facilitates a user selecting to play video content
from a panel by saying the numeral associated with the panel. Also,
as shown at 80 a genre or category indicator such as an icon may be
presented next to each panel. The genre indicator may be a dark
blue or purple arrow 80a in the case of the first panel 76
indicating that the first panel 76 is presenting the currently
selected video. The genre indicator may be a light blue heart 80b
(as shown in FIG. 3 for panel 2) indicating the underlying video
content has been selected by the user as a "favorite". Other genre
indicators (such as a red circle 80c) may be presented respectively
representing "history", meaning that the content associated with
the panel, had been presented previously, a yellow star 80d
representing a "recommended" category, meaning that the content
associated with; the panel is recommended by a recommendation
engine executed by the AVDD processor 18, and so on, e.g., a
vertical column of spaced green boxes 80e representing a
"frequently viewed" category, an icon representing "promotional
content", etc. A user can select to present panels associated with
content of only a single genre but as shown in FIG. 3 mixed
category content panels mat be presented on a single screen.
[0029] Accordingly, it may be appreciated that by using a standard
category icon placed to the side of content panel, the viewer can
more easily identify the underlying video's category by both shape
and color. The numeral is also provided to facilitate selecting a
content panel using a voice command by voicing the number. Note
further that the category icons 80 are placed to the left of the
content panels, to which they pertain. This uniform location makes
it easy to find this marking identifier.
[0030] The above icon principles may be used in any grid of
thumbnails displayed by, e.g., personal computers, cameras, tablet
computers, wireless telephones, etc. Note that the icons 80 in FIG.
3 appear, in the borders between content panels so as not to
conceal the panels.
[0031] Additionally, if desired the GUI shown in FIG. 3 may present
gesture hints 82, describing in text and/or animated hand motion a
gesture and its associated command that the user may make, which
can be captured by the camera 50 and correlated to the respective
command by the processor 18 executing image recognition software.
Thus, while viewer input may be afforded by making the display 28 a
touch-sensitive display that a viewer can touch and thereby enter
commands, present principles envision using viewer gestures in free
space, i.e., gestures in which the viewer is distanced from the
display 28 as would typically be the case for a TV viewer, with the
viewer not touching the display but making gestures that are
captured by the camera 50 and correlated to commands by the
processor 18 executing image recognition software.
[0032] Further, a menu of selections may be presented as shown
along the bottom of the GUI, including a "just for you" selector 84
to cause the GUI to contain panels, with content personalized to a
recognized user. A search selector 86 can be selected to cause a
search for a user-input term to be executed. Also, a bookmark
selector 88 can be selected to bookmark the currently playing video
in panel 76 or to bookmark content in a panel over which the screen
cursor is positioned.
[0033] Still referring to FIG. 3, a settings selector element 90 if
selected causes a setting menus to be presented to control settings
of the GUI. A queue selector element 92 may be presented to cause a
list of enqueued programs to be presented on the display 28, and an
inputs selector element 94 may be presented, selection of which
causes a list of available input sources, e.g., "disk player,
cable, satellite" to be presented on the display 28. A
recommendations selector element 96 may be presented and if
selected causes a list of recommended programming to be presented
on the display 28. The list may be generated by a content
recommendation engine such as the "Navi" engine made by Sony Corp.
A "what's on" selector element 98 may be provided and if selected
causes a list of current and future programs such as an electronic
program: guide (EPG) to be presented on the display 28. Indeed, an
EPG may be presented in one of the panels shown in FIG. 3.
[0034] In any case, as mentioned above in the example shown the
currently selected video content is always presented in the upper
left panel 76 of the GUI. Should the user select another panel by
using the RC 30 or by speaking the number of the panel or by the
appropriate gesture captured by the camera 50, video content from
the source associated with the selected panel automatically
replaces the video content which was presented in the upper left
panel 76 prior to receiving the new select signal. Indeed, should
the user scroll the panels left of right to view additional content
the upper left panel 76 remains unmoving as the other panels move
on and off the display 28 as they are replaced by previously hidden
content panels, with the upper left panel 76 always showing the
currently selected video program.
[0035] The panel layout shown in FIG. 3 is somewhat coarse in that
a visible border space 100 of many pixels width exists between
adjacent panels as shown. Such a coarse representation facilitates
control using gestures, although navigation by voice and by use of
the RC 30 is always preferably enabled regardless of whether the
GUI is in the coarse or fine view. FIG. 4 shows that if the user
selects the fine view, the border space 100 disappears between
adjacent panels, a view more suitable for RC control than gesture
control although as stated above all three modes of command input
(RC, voice, and, gesture) remain enabled simultaneously with each
other if desired.
[0036] Transition between the two views of FIGS. 3 and 4 may be
effected by an appropriate hand gesture in free space (e.g., a
vertical motion of the hand distanced from the display 28 and
detected by the camera 50) or by an appropriate voice command
(e.g., "fine" or "coarse"), or by selecting, using the RC 30,
"fine" or "coarse" from a settings menu accessed by selecting the
settings selector element 90 or by pressing a key on the RC 30 or
by simply grasping the RC, which is detected by the camera 50 and
inferred by the processor 18 to correlate to a command to move to
the "fine" screen, of FIG. 4. The larger panels in the fine mode of
FIG. 4 can be used to present more information than is presented in
the panels of FIG. 3.
[0037] The GUI shown in FIGS. 3 and 4 may be removed from the
screen and the currently playing video presented on the full
display 28 by, e.g., toggling the home key on the RC 30. Voice
command input may be enabled by voicing a phrase such as "hello
TV`, which is detected by the microphone 52 and using voice
recognition software correlated by the processor 18 to a command to
enable voice commands. Voice input may also be enabled using the RC
30 to select the "search" selector element 86. To indicate that
voice commands are enabled, a voice command input mode enable icon
such as an image of a microphone 102 can be presented on the
display 28. Gesture command input may be enabled by waving the
hand, which is detected by the camera 50 and using image
recognition software correlated by the processor 18 to a command to
enable gesture commands. To indicate that gesture commands are
enabled, an icon such as an image of a hand 104 can be presented on
the display 28.
[0038] Non-limiting examples of corresponding RC, voice, and
gesture commands that accomplish the same tasks are:
TABLE-US-00001 RC COMMAND VOICE COMMAND GESTURE Channel up "Channel
up" upward motion of hand Channel down "Channel down" downward
motion of hand Volume decrease "Quieter" thumb down Volume increase
"louder" thumb up
[0039] Additional gesture commands can be: pointed finger--attach,
cursor; move pointed finger of speak "move cursor [left] [right]"
--move cursor per finger movement/voice command; clenched fist or
spoken "select" --select, panel under cursor for further info; pump
clenched fist or voice command "play" --select panel under cursor
to be new current video for presentation in pinned panel 76; two
clenched fists or voice command "more info" --show expanded
metadata window for cursor video, etc.
[0040] When the voice input is not enabled, the processor 18 may
direct the display 28 to present the not enabled state of the voice
command input in one of two forms. FIG. 5 illustrates the display
28 with current video pinned panel 76 and the first example of
voice command input mode not being enabled, as conveyed by a lack
of voice command input mode enable icon 102.
[0041] The absence of icon 102 informs the viewer that voice
command input mode is hot enabled, a state of mode that may also be
conveyed by a presence of an icon 102 in its first form on the
display 28 under the direction of the processor 18, as in FIG. 6.
This second example of the voice command input mode not being
enabled may be presented to the viewer by the first form of the
icon 102, which should be distinguishable from a second form of the
icon 102. In this embodiment, the first form of the icon 102 is the
image of an outline of a microphone.
[0042] Moving to FIG. 7, the voice command input mode enable icon
102 is presented on the display 28 by the processor 18 in its
second form, a solid image of a microphone. The solid icon 102
informs the viewer that the voice Command input mode is enabled.
This second form (solid) of an image, here, a microphone, is
distinguishable from its first form (hollow).
[0043] As mentioned above, a voice command input mode enable icon
102 may appear or have its appearance altered to indicate that
voice command input mode is enabled. Likewise, the grid numerals
may appear if not already on display or alter their appearance if
on display responsive to a determination that voice command input
mode is enabled. FIGS. 5-7 illustrate. Specifically, when voice
command input is not enabled, the grid numerals either may not
appear (FIG. 5) or appear in dashed lines or lowlighted of with
relatively small size (FIG. 6), whereas when voice command is
enabled the grid numeral appear relatively more prominently, e.g.,
of larger size or brighter appearance than when voice command is
not enabled (FIG. 7). In this way, when voice command is enabled,
the viewer is better aided in visualizing the grid numbers of the
content panels so as to better articulate a desired voice command,
e.g., "more metadata for Number Three", etc.
[0044] While the particular USER INTERFACE FOR AUDIO VIDEO DISPLAY
DEVICE SUCH AS TV is herein shown and described in detail, it is to
be understood that the subject matter which is encompassed by the
present invention is limited only by the claims.
* * * * *