U.S. patent application number 15/001514 was filed with the patent office on 2016-05-12 for system and method for distributing auxiliary data embedded in video data.
This patent application is currently assigned to SONY CORPORATION. The applicant listed for this patent is SONY CORPORATION. Invention is credited to Mark Eyer.
Application Number | 20160134909 15/001514 |
Document ID | / |
Family ID | 43855868 |
Filed Date | 2016-05-12 |
United States Patent
Application |
20160134909 |
Kind Code |
A1 |
Eyer; Mark |
May 12, 2016 |
SYSTEM AND METHOD FOR DISTRIBUTING AUXILIARY DATA EMBEDDED IN VIDEO
DATA
Abstract
A system and method for distributing auxiliary data embedded in
video data includes a content source that embeds the auxiliary data
into the video data. The content source then encodes the video data
together with the auxiliary data to create a distribution multiplex
including compressed video data. A decoder receives and
decompresses the distribution multiplex to reproduce the video data
with the auxiliary data embedded. A television or other device then
detects and extracts the auxiliary data from the video data. The
television or other device processes the auxiliary data to support
a variety of possible interactive applications including displaying
a synchronized widget on a display of the television.
Inventors: |
Eyer; Mark; (Park Ridge,
NJ) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
SONY CORPORATION |
TOKYO |
|
JP |
|
|
Assignee: |
SONY CORPORATION
TOKYO
JP
|
Family ID: |
43855868 |
Appl. No.: |
15/001514 |
Filed: |
January 20, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12798118 |
Mar 30, 2010 |
9277183 |
|
|
15001514 |
|
|
|
|
61251099 |
Oct 13, 2009 |
|
|
|
Current U.S.
Class: |
725/146 ;
725/151 |
Current CPC
Class: |
H04N 21/44008 20130101;
H04N 21/23614 20130101; H04N 21/435 20130101; H04N 21/2665
20130101; H04N 7/173 20130101; H04N 21/8133 20130101; H04N 21/236
20130101; H04N 21/434 20130101; H04N 21/4348 20130101; H04N 5/44508
20130101; H04N 21/235 20130101; H04N 21/47205 20130101; H04N
21/2353 20130101; H04N 21/8173 20130101 |
International
Class: |
H04N 21/236 20060101
H04N021/236; H04N 21/435 20060101 H04N021/435; H04N 21/235 20060101
H04N021/235; H04N 21/434 20060101 H04N021/434 |
Claims
1. (canceled)
2. A method for receiving video data, the method comprising:
decoding a distribution multiplex including the video data embedded
with auxiliary data, the auxiliary data including content
identification data that identifies specific content; and
displaying the video data and the embedded auxiliary data, wherein
the auxiliary data is displayed by modulating pixel luminance or
chrominance.
3. The method according to claim 2, wherein the method further
comprises: detecting and extracting the auxiliary data from the
distribution multiplex; and executing an application related to the
video data based on the detected and extracted auxiliary data.
4. The method according to claim 3, wherein the detecting and
extracting the auxiliary data includes detecting a predefined
pattern in the decoded distribution multiplex, the predefined
pattern corresponding to a location of the auxiliary data embedded
in the video data.
5. The method according to claim 2, wherein the displayed auxiliary
data has rectangular shape.
6. The method according to claim 2, wherein same auxiliary data is
continued to be displayed during display of plural frames of the
video data.
7. A receiving device comprising: circuitry configured to decode a
distribution multiplex including the video data embedded with
auxiliary data, the auxiliary data including content identification
data that identifies specific content; and display the video data
and the embedded auxiliary data, wherein the auxiliary data is
displayed by modulating pixel luminance or chrominance.
8. The receiving device according to claim 7, wherein the circuitry
is further configured to: detect and extract the auxiliary data
from the distribution multiplex; and execute an application related
to the video data based on the detected and extracted auxiliary
data.
9. The receiving device according to claim 8, wherein the detecting
and extracting the auxiliary data includes detecting a predefined
pattern in the decoded distribution multiplex, the predefined
pattern corresponding to a location of the auxiliary data embedded
in the video data.
10. The receiving device according to claim 7, wherein the
displayed auxiliary data has rectangular shape.
11. The receiving device according to claim 7, wherein same
auxiliary data is continued to be displayed during display of
plural frames of the video data.
12. A method for distributing video data, the method comprising:
embedding auxiliary data into video data; encoding the video data
embedded with the auxiliary data to create a distribution
multiplex, the auxiliary data includes content identification data
that identifies specific content and is encoded with pixel
luminance or chrominance; distributing the distribution multiplex
to at least one receiving device through a distribution
network.
13. The method according to claim 12, wherein the auxiliary data is
used to execute an application related to the video data by the at
least one receiving device.
14. The method according to claim 12, wherein the auxiliary data
includes a predefined pattern, the predefined pattern corresponding
to a location of the auxiliary data embedded in the video data.
15. The method according to claim 12, wherein the auxiliary data is
embedded in the video data such that the auxiliary data displayed
by the at least one receiving device has rectangular shape.
16. The method according to claim 12, wherein the same auxiliary
data is embedded into plural frames of the video data.
17. A distribution device comprising: circuitry configured to embed
auxiliary data into video data; encode the video data embedded with
the auxiliary data to create a distribution multiplex, the
auxiliary data includes content identification data that identifies
specific content and is encoded with pixel luminance or
chrominance; distribute the distribution multiplex to at least one
receiving device through a distribution network.
18. The distribution device according to claim 17, wherein the
auxiliary data is used to execute an application related to the
video data by the at least one receiving device.
19. The distribution device according to claim 17, wherein the
auxiliary data includes a predefined pattern, the predefined
pattern corresponding to a location of the auxiliary data embedded
in the video data.
20. The distribution device according to claim 17, wherein the
auxiliary data is embedded in the video data such that the
auxiliary data displayed by the at least one receiving device has
rectangular shape.
21. The distribution device according to claim 17, wherein the same
auxiliary data is embedded into plural frames of the video data.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority in, U.S. Provisional Patent
Application No. 61/251,099 entitled "Video-Embedded Metadata" that
was filed on Oct. 13, 2009. The foregoing related Application is
commonly assigned, and is hereby incorporated by reference.
BACKGROUND SECTION
[0002] 1. Field of the Invention
[0003] This invention relates generally to techniques for
implementing television systems, and relates more particularly to a
system and method for distributing auxiliary data embedded in video
data.
[0004] 2. Description of the Background Art
[0005] Implementing effective methods for distribution of metadata
within digital television systems is a significant consideration
for designers and manufacturers of contemporary electronic
entertainment systems. However, effectively implementing such
systems may create substantial challenges for system designers. For
example, enhanced demands for increased system functionality and
performance may require more capabilities and require additional
hardware and software resources. Impediments to the effective
delivery of metadata in advanced systems may result in a
corresponding detrimental economic impact due to operational
inefficiencies, lost revenue opportunities, and reduced
functionality.
[0006] Furthermore, enhanced system capability to perform various
advanced operations can offer additional benefits to the end user,
but may also place increased demands on the control and management
of various system components. For example, an enhanced electronic
system that effectively supports synchronized television widget
functionality may benefit from methods providing flexible carriage
of the data stream supporting this functionality.
[0007] Due to growing demands on system resources and substantially
increasing data magnitudes, it is apparent that developing new
techniques for implementing and utilizing data distribution through
digital television systems is a matter of concern for related
electronic technologies. Therefore, for all the foregoing reasons,
developing effective systems for implementing and utilizing data
distribution through digital television systems remains a
significant consideration for designers, manufacturers, and users
of contemporary electronic entertainment systems.
SUMMARY
[0008] In accordance with the present invention, a system and
method are disclosed for distributing auxiliary data embedded in
video data. In accordance with one embodiment of the present
invention, a content source or other appropriate entity initially
produces content data that typically includes video data and audio
data. The content source or other appropriate entity then creates
auxiliary data to support advanced interactive features such as
"synchronized widgets" or Internet-enhanced interactivity on a
television device. The content source or other appropriate entity
inserts or embeds the auxiliary data into the video data.
[0009] The content source or other appropriate entity then encodes
the audio data and the video data (including the embedded auxiliary
data) to create a compressed distribution multiplex. The content
source or other appropriate entity distributes the distribution
multiplex in any effective manner to a television in an electronic
network. In certain embodiments, a decoder device of a set-top box
or other appropriate entity receives and decodes the distribution
multiplex distributed by the content source to reproduce
uncompressed audio data and uncompressed video data (including the
embedded auxiliary data).
[0010] A detection module of the television scans the video data to
locate the embedded auxiliary data by utilizing any effective
techniques. Next, an extraction module of the television extracts
the located auxiliary data from the video data. Finally, an
auxiliary data module of the television processes the extracted
auxiliary data to successfully display and support one or more
interactive applications residing in the television. For all of the
foregoing reasons, the present invention thus provides an improved
system and method for distributing auxiliary data embedded in video
data.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG. 1A is a block diagram of an electronic system, in
accordance with one embodiment of the present invention;
[0012] FIG. 1B is a diagram of a display from the television of
FIG. 1A, in accordance with one embodiment of the present
invention;
[0013] FIG. 2 is a block diagram for one embodiment of the content
source of FIG. 1A, in accordance with the present invention;
[0014] FIG. 3 is a block diagram for one embodiment of the source
memory of FIG. 2, in accordance with the present invention;
[0015] FIG. 4 is a block diagram for one embodiment of the
auxiliary data of FIG. 3, in accordance with the present
invention;
[0016] FIG. 5 is a block diagram for one embodiment of the
television from FIG. 1A, in accordance with the present
invention;
[0017] FIG. 6 is a block diagram for one embodiment of the TV
memory from FIG. 5, in accordance with the present invention;
[0018] FIGS. 7A and 7B are diagrams of auxiliary data embedded in
video data, in accordance with two different embodiments of the
present invention; and
[0019] FIGS. 8A-8C are a flowchart of method steps for distributing
auxiliary data, in accordance with one embodiment of the present
invention.
DETAILED DESCRIPTION
[0020] The present invention relates to an improvement in
television systems. The following description is presented to
enable one of ordinary skill in the art to make and use the
invention, and is provided in the context of a patent application
and its requirements. Various modifications to the disclosed
embodiments will be readily apparent to those skilled in the art,
and the generic principles herein may be applied to other
embodiments. Thus, the present invention is not intended to be
limited to the embodiments shown, but is to be accorded the widest
scope consistent with the principles and features described
herein.
[0021] The present invention is described herein as a system and
method for distributing auxiliary data embedded in video data, and
includes a content source that embeds the auxiliary data into the
video data. The content source then encodes the video data together
with the auxiliary data to create a distribution multiplex
including compressed video data. A decoder receives and
decompresses the distribution multiplex to reproduce the video data
with the auxiliary data embedded. A television or other device then
detects and extracts the auxiliary data from the video data. The
television or other device processes the auxiliary data to support
a variety of possible interactive applications including displaying
a synchronized widget on a display of the television.
[0022] Referring now to FIG. 1A, a block diagram of an electronic
system 110 is shown, in accordance with one embodiment of the
present invention. In the FIG. 1A embodiment, electronic system 110
may include, but is not limited to, a content source 114, a set-top
box 118, an interface 126, a television 122, an optional network
134, and an optional server 130. In alternate embodiments,
electronic system 110 may be implemented using components and
configurations in addition to, or instead of, certain of those
components and configurations discussed in conjunction with the
FIG. 1A embodiment. For example, any number of televisions 122 may
be similarly deployed in electronic system 110. In addition,
network 134 and server 130 may not be included in all embodiments
of the present invention.
[0023] In the FIG. 1A embodiment, content source 114 may be
implemented as one or more electronic devices or other entities
that prepare and distribute content data, including video data and
audio data, for reproduction by television 122. In the FIG. 1A
embodiment, content source 114 may be implemented as any
appropriate entity. For example, content source 114 may include a
television broadcasting facility, a satellite television
distribution facility, or an Internet server entity. Additional
details regarding the implementation and utilization of content
source 114 are further discussed below in conjunction with FIGS.
2-4.
[0024] In the FIG. 1A embodiment, content source 114 creates an
encoded distribution multiplex containing the content data in a
compressed format, and then distributes the distribution multiplex
through a distribution network via path 116 to a decoder device. In
the FIG. 1A embodiment, the decoder device is implemented in a
set-top box 118. However, in other embodiments, the decoder device
may be implemented as any appropriate entity, either external to,
or integral with, television 122. In certain embodiments,
additional devices or entities may be interposed between content
source 114 and set-top box 118. Examples of such entities may
include, but are not limited to, a broadcast network affiliate and
a service provider (such as a satellite or cable head-end).
[0025] In the FIG. 1A embodiment, set-top box 118 decodes the
encoded distribution multiplex to generate uncompressed A/V data
(video data and audio data) that is provided to television 122 via
an appropriate interface 126. In the FIG. 1A embodiment, interface
126 may be implemented in any effective manner. For example,
interface 126 may be implemented according to a High Definition
Multimedia Interface (HDMI) standard that provides a high-speed
parallel interface to deliver uncompressed video data and audio
data, and control/timing signals to television 122. Television 122
may then responsively receive and reproduce the video data and
audio data for utilization by a system user. Additional details
regarding the implementation and utilization of television 122 are
further discussed below in conjunction with FIGS. 5-6.
[0026] In the FIG. 1A embodiment, electronic system 110 supports
the creation and display of various types of "widgets" on
television 122 in addition to the main content data. Widgets may
include discrete areas that are displayed on television 122 to
provide any desired type of information. Additional details
regarding widgets are further provided below in conjunction with
FIG. 1B. In the FIG. 1A embodiment, electronic system 110
advantageously supports synchronized widgets that provide
information that is related to (synchronized) to the main content
data that is currently being displayed on television 122. In order
to successfully support synchronized widgets, electronic system 110
also provides certain types of auxiliary data to television
122.
[0027] In various embodiments, television 122 may obtain auxiliary
data from any appropriate source including, but not limited to,
content source 114 or server 130. In the FIG. 1A embodiment,
television 122 may communicate with server 130 via any effective
network 134 including, but not limited to, the Internet. Additional
details regarding the creation, distribution, and utilization of
auxiliary data are further discussed below in conjunction with
FIGS. 4, 7, and 8.
[0028] The present invention generally involves embedding auxiliary
data in a video signal so that the auxiliary data may be quickly
and easily recovered by receiving devices like television 122. In
certain embodiments, content source 114 inserts auxiliary data
within a distributed video signal so that the auxiliary data
travels through the distribution chain, comes into a consumer's
home via a compressed interface (from a cable, satellite, or IPTV
service provider), is de-compressed in set-top box 118, and then
travels to television 122 in an uncompressed format, where
television 122 retrieves and utilizes the embedded auxiliary data
to support synchronized widgets. The foregoing techniques are
intended to circumvent service providers or other entities from
blocking the consumer's access to auxiliary data that is required
to provide enhanced functionality to television 122.
[0029] Certain cable, satellite, and IPTV entities typically
provide system users with set-top boxes that are interfaced to
digital televisions via HDMI uncompressed video interfaces or other
appropriate means. If a content owner wishes to include auxiliary
data (such as a URL, applet, etc.) with the content data, and if
that auxiliary data travels with the content data as a separate
digital stream (or as metadata within the compressed bit stream),
the auxiliary data will be blocked at the set-top box 118.
[0030] Conventionally, a set-top box 114 does not pass ancillary
data streams in the distribution multiplex, because the set-top box
decodes only audio data and video data, and then passes only the
uncompressed video data and audio data across to the television.
Ancillary data streams are therefore unavailable to the television.
If service providers (those offering the set-top boxes) perceive
that providing access to any ancillary data is competitive to their
business model, they may not be inclined to help the consumer
electronics industry by providing such access.
[0031] By embedding auxiliary data within the video data, the
auxiliary data survives compression/decompression and is able to
arrive intact at television 122. In other words, the present
invention advantageously embeds auxiliary data within the video
signal (encoded within the video image, not as a separate ancillary
data stream). The present invention therefore successfully
overcomes the architectural roadblock discussed above. The
implementation and utilization of the FIG. 1A electronic system 110
is further discussed below in conjunction with FIGS. 1B-8B.
[0032] Referring now to FIG. 1B, a diagram of a display 138 from
the television 122 of FIG. 1A is shown, in accordance with one
embodiment of the present invention. The FIG. 1B embodiment is
presented for purposes of illustration, and in alternate
embodiments, display 138 may be implemented using components and
configurations in addition to, or instead of, certain of those
components and configurations discussed in conjunction with the
FIG. 1B embodiment.
[0033] In the FIG. 1B embodiment, display 138 includes a main
screen region that typically displays video data provided by a
content source 114 (FIG. 1A). In the FIG. 1B embodiment, display
138 also includes a widget 144 that resides in a discrete area
displayed on display 138 to provide any desired type of additional
information. In various different embodiments, widget 144 may be
implemented in any desired shape or size, and may be displayed in
any appropriate location. Furthermore, any desired number of
different widgets are equally contemplated, including the
possibility of multiple widgets on the display at any given
time.
[0034] In the FIG. 1B embodiment, display 138 supports synchronized
widgets that function to provide information that is related
(synchronized) to the video data that is currently being displayed
on display 138. For example, widget 144 may be utilized to display
financial information of specific relevance to the viewer (e.g.,
his/her investment portfolio) during a television program regarding
economic news or investment topics. In another example, widget 144
may be utilized during a televised automobile race to display
relevant information or statistics regarding specific race car
drivers, racecars, or automobile racing in general. Additional
details regarding the implementation and utilization of
synchronized widgets 144 is further discussed below in conjunction
with FIGS. 2-8B.
[0035] Referring now to FIG. 2, a block diagram for one embodiment
of the FIG. 1A content source 114 is shown, in accordance with the
present invention. In the FIG. 2 embodiment, content source 114 may
include, but is not limited to, a central processing unit (CPU)
212, a source memory 220, and input/output interfaces (I/O
interfaces) 224. In alternate embodiments, content source 114 may
be implemented using components and configurations in addition to,
or instead of, those components and configurations discussed in
conjunction with the FIG. 2 embodiment. In addition, content source
114 may alternately be implemented as any other desired type of
electronic device or entity.
[0036] In the FIG. 2 embodiment, CPU 212 may be implemented to
include any appropriate and compatible microprocessor device(s)
that preferably execute software instructions to thereby control
and manage the operation of content source 114. In the FIG. 2
embodiment, source memory 220 may be implemented to include any
combination of desired storage devices, including, but not limited
to, read-only memory (ROM), random-access memory (RAM), and various
types of non-volatile memory, such as floppy disks or hard disks.
The contents and functionality of source memory 220 are further
discussed below in conjunction with FIGS. 3 and 4.
[0037] In the FIG. 2 embodiment, I/O interfaces 224 may include one
or more input and/or output interfaces to receive and/or transmit
any required types of information for content source 114. For
example, in the FIG. 2 embodiment, content source 114 may utilize
I/O interfaces 224 to communicate with other entities in electronic
system 110 (FIG. 1A). Furthermore, a system user may utilize I/O
interfaces 224 to communicate with content source 114 by utilizing
any appropriate and effective techniques. Additional details
regarding content source 114 are further discussed below in
conjunction with FIGS. 3-4.
[0038] Referring now to FIG. 3, a block diagram for one embodiment
of the FIG. 2 source memory 220 is shown, in accordance with the
present invention. In the FIG. 3 embodiment, source memory 220
includes, but is not limited to, one or more source applications
312, video data 316, audio data 318, an encoder 320, auxiliary
(aux) data 322, an auxiliary (aux) data manager 324, and
miscellaneous information 328. In alternate embodiments, source
memory 220 may include components in addition to, or instead of,
those components discussed in conjunction with the FIG. 3
embodiment.
[0039] In the FIG. 3 embodiment, source application(s) 312 may
include program instructions that are preferably executed by CPU
212 (FIG. 2) to perform various functions and operations for
content source 114. The particular nature and functionality of
source application(s) 312 preferably varies depending upon factors
such as the specific type and particular functionality of the
corresponding content source 114. In the FIG. 3 embodiment, video
data 316 may include any appropriate information or data for
display on, or for processing within, television 122 (FIG. 1A).
Similarly, audio data 318 may include any appropriate information
or data for reproduction by television 122 (FIG. 1A).
[0040] In the FIG. 3 embodiment, encoder 320 may include any
appropriate means for converting video data 316 and audio data 318
into a compressed distribution multiplex for distribution to
television 122. In the FIG. 3 embodiment, aux data manager 324
coordinates and manages various functions for creating aux data
322, and embedding aux data 322 as an integral part of video data
316, in accordance with the present invention. Miscellaneous 328
may include any additional information for utilization by content
source 114.
[0041] In the FIG. 3 embodiment, the present invention is disclosed
and discussed as being implemented primarily as software. However,
in alternate embodiments, some or all of the functions of the
present invention may be performed by appropriate electronic
hardware circuits that are configured for performing various
functions that are equivalent to those functions of the software
modules discussed herein. Additional details regarding the
functionality of aux data manager 324 and aux data 322 are further
discussed below in conjunction with FIGS. 4, 7, and 8.
[0042] Referring now to FIG. 4, a block diagram of the FIG. 3 aux
data 322 is shown, in accordance with one embodiment of the present
invention. In the FIG. 4 embodiment, aux data 322 may include, but
is not limited to, trigger data 412, widget content 416,
synchronization (sync) data 418, content identification (ID) data
420, pointer data 422, and miscellaneous information 424. In
alternate embodiments, aux data 322 may be implemented using
various components and functionalities in addition to, or instead
of, those components and functionalities discussed in conjunction
with the FIG. 4 embodiment.
[0043] In the FIG. 4 embodiment, trigger data 412 may include any
type of information that defines various characteristics of a
widget 144 (FIG. 1B). For example, trigger data 412 may include,
but is not limited to, data that defines a widget 144 with respect
to widget visual appearance and behavior, information presented by
a widget (such as readout values), widget graphical states (such as
colors, levels, or settings), and optimal widget location, shape,
size, and display times.
[0044] In the FIG. 4 embodiment, widget content 416 may include any
content data for display in a widget 144. In certain embodiments,
widget content 416 may alternately be obtained from sources or
entities other than aux data 322. In the FIG. 4 embodiment,
synchronization (sync) data 418 may include any appropriate means
for allowing television 122 to detect aux data 322 while it is
embedded in video data 316. For example, in certain embodiments,
sync data 418 may include a pre-defined identification pattern that
indicates the specific location of aux data 322 within video data
316.
[0045] In the FIG. 4 embodiment, content ID data 420 may include
any appropriate information for identifying the specific content of
a given corresponding program. For example, in certain embodiments,
content ID data 420 may include an International Standard
Audio-Visual Number (ISAN) number as an identifier. In the FIG. 4
embodiment, pointer data 422 may include any type of required
information that television 122 utilizes to locate and obtain
additional information (such as widget content or trigger data) for
using in producing synchronized widgets 144.
[0046] For example, pointer data 422 may include, but is not
limited to, a URL that identifies an Internet location where more
information pertaining to the currently-displayed video data 316
may be found. The URL could represent a website on server 130 (FIG.
1A) or elsewhere providing more information about a product being
advertised, a URL of a home page of an episode or series, a website
where a viewer could sign up for a service or vote on a program,
etc. In the FIG. 4 embodiment, miscellaneous 424 may include any
additional information for utilization by television 122. For
example, in certain embodiments, miscellaneous 424 may include one
or more scripts or executable programs. Additional details
regarding the creation, distribution, and utilization of aux data
322 are further discussed below in conjunction with FIGS. 7 and
8.
[0047] Referring now to FIG. 5, a block diagram for one embodiment
of the FIG. 1A television (TV) 122 is shown, in accordance with the
present invention. In the FIG. 2 embodiment, TV 122 may include,
but is not limited to, a central processing unit (CPU) 512, a
display 138, a TV memory 520, and input/output interfaces (I/O
interfaces) 524. In alternate embodiments, TV 122 may be
implemented using components and configurations in addition to, or
instead of, those components and configurations discussed in
conjunction with the FIG. 5 embodiment. In addition, TV 122 may
alternately be implemented as any other desired type of electronic
device or entity.
[0048] In the FIG. 5 embodiment, CPU 512 may be implemented to
include any appropriate and compatible microprocessor device(s)
that preferably execute software instructions to thereby control
and manage the operation of TV 122. The FIG. 5 display 138 may
include any effective type of display technology including a
cathode-ray-tube monitor or a liquid-crystal display device with an
appropriate screen for displaying various information to a device
user. In the FIG. 5 embodiment, TV memory 520 may be implemented to
include any combination of desired storage devices, including, but
not limited to, read-only memory (ROM), random-access memory (RAM),
and various types of non-volatile memory, such as floppy disks or
hard disks. The contents and functionality of TV memory 520 are
further discussed below in conjunction with FIG. 6.
[0049] In the FIG. 5 embodiment, I/O interfaces 524 may include one
or more input and/or output interfaces to receive and/or transmit
any required types of information for TV 122. For example, in the
FIG. 5 embodiment, TV 122 may utilize I/O interfaces 524 to
communicate with other entities in electronic system 110 (FIG. 1A).
Furthermore, a system user may utilize I/O interfaces 524 to
communicate with TV 122 by utilizing any appropriate and effective
techniques. Additional details regarding TV 122 are further
discussed below in conjunction with FIGS. 6-8B.
[0050] Referring now to FIG. 6, a block diagram for one embodiment
of the FIG. 5 TV memory 520 is shown, in accordance with the
present invention. In the FIG. 6 embodiment, TV memory 520
includes, but is not limited to, one or more TV applications 612,
video data 316, audio data 318, a detection module 620, and
extraction module 622, an auxiliary (aux) data module 624,
auxiliary (aux) data 322, and miscellaneous information 628. In
alternate embodiments, TV memory 520 may include components in
addition to, or instead of, those components discussed in
conjunction with the FIG. 6 embodiment.
[0051] In the FIG. 6 embodiment, TV application(s) 312 may include
program instructions that are preferably executed by CPU 512 (FIG.
5) to perform various functions and operations for TV 122. The
particular nature and functionality of TV application(s) 612
preferably varies depending upon factors such as the specific type
and particular functionality of the corresponding TV 122. In the
FIG. 6 embodiment, video data 316 may include any appropriate
information or data for display on television 122 (FIG. 1A).
Similarly, audio data 318 may include any appropriate information
or data for reproduction by television 122 (FIG. 1A).
[0052] In the FIG. 6 embodiment, detection module 620 may be
utilized by TV 122 to detect and locate aux data 322 that has been
embedded in video data 316, as discussed above. In the FIG. 6
embodiment, extraction module 620 may be utilized by TV 122 to
remove the detected aux data 322 from the video data 316. In the
FIG. 3 embodiment, aux data module 624 coordinates and manages
various functions for processing the extracted aux data 322 to
effectively support synchronized widgets 144 (FIG. 1B) or other TV
applications, in accordance with the present invention.
Miscellaneous 628 may include any additional information for
utilization by TV 122.
[0053] In the FIG. 6 embodiment, the present invention is disclosed
and discussed as being implemented primarily as software. However,
in alternate embodiments, some or all of the functions of the
present invention may be performed by appropriate electronic
hardware circuits that are configured for performing various
functions that are equivalent to those functions of the software
modules discussed herein. Additional details regarding the
functionality of aux data module 324 and aux data 322 are further
discussed below in conjunction with FIGS. 7 and 8.
[0054] Referring now to FIGS. 7A and 7B, diagrams of auxiliary
(aux) data 322 embedded in video data 316 are shown, in accordance
with two different embodiments of the present invention. FIGS. 7A
and 7B present a frontal view of a display 138 from TV 122 (FIG.
1A). The embodiments of FIGS. 7A and 7B are presented for purposes
of illustration, and in alternate embodiments, aux data 322 may be
embedded using techniques and configurations in addition to, or
instead of, certain of those techniques and configurations
discussed in conjunction with the embodiments of FIGS. 7A and
7B.
[0055] In the FIG. 7A embodiment, display 138 includes a main
screen region that typically displays video data 316 provided by a
content source 114 (FIG. 1A). In the FIG. 7A embodiment, the
displayed video data 316 on display 138 also includes embedded aux
data 322 that is preferably located in an unobtrusive area of
display 138. In various, different embodiments, aux data 322 may be
implemented in any desired shape or size, and may be displayed in
any appropriate location(s) on display 138. For purposes of
illustration, the aux data 322 in FIG. 7A is depicted as a small
cross-hatched rectangle. However, any effective configuration or
appearance is equally contemplated for implementing aux data
322.
[0056] In the FIG. 7A embodiment, aux data 322 may be encoded to
represent any required information (see FIG. 4) in any effective
manner. For example, in certain embodiments, aux data 322 may be
formatted by utilizing conventional or enhanced bar code
technologies. In other words, aux data 322 could be effectively
formatted as a video two-dimensional bar code that is embedded in a
corner or at the edge of the displayed video data 316. In addition,
the bar code or other formatting of aux data 322 could be displayed
as a part of a small graphical logo icon known as a "bug."
Furthermore, in various other embodiments, aux data 322 may encoded
or displayed by utilizing any other effective techniques. For
example, aux data 322 could be implemented by modulating various
attributes of luminance or chrominance information. In addition,
aux data 322 may be implemented by utilizing multiple display
lines.
[0057] Such an encoding of aux data 322 could represent a
substantial amount of information, and could be quite small and
dense, as aux data 322 would be read by the TV 122 processing video
data 316 in video memory. Where printed barcodes are optimized for
readout by laser scanners, the type of video barcode used for aux
data 322 is embedded in a digital video signal, which is processed
directly by the TV 122 (as pixel luminance or chrominance
samples).
[0058] In certain embodiments, quantization errors in the video
compression could possibly obliterate a video barcode (so a bar
code occurring within a fast-moving, hard-to-compress video
sequence might not survive). However, if the bar code is left
on-screen for some amount of time (a few seconds), that concern is
mitigated. The resulting barcode image may not need to be shown
with high contrast (black lines on white background), since TV 122
will be able to extract the information via a filtering mechanism.
The bar code could thus be encoded with various shades of gray (as
long as there is enough contrast for reliable extraction).
[0059] As discussed above, aux data 322 could be displayed in
conjunction with a graphical logo icon ("bug"), as a caption or
border, or it could be placed at one more of the extreme edges of
the image (because these are usually cropped before display, and
are less obtrusive in any case). The bits of aux data 322 could be
spread out spatially over the area of the video frame if the
pattern of their location was known to the TV 122 beforehand. Even
a small amount of aux data 322, such as the content ID data 420 or
the pointer data 422 of FIG. 4, can be of great help in enhancing
the user experience, as this information can be expanded via an
interaction with a web server 130 (see FIG. 1A) to obtain
additional required information including, but not limited to, aux
data 322 or content data.
[0060] In the FIG. 7B embodiment, display 138 includes a main
screen region that typically displays video data 316 provided by a
content source 114 (FIG. 1A). In the FIG. 7B embodiment, the
displayed video data 316 on display 138 also includes embedded aux
data 322 that is preferably located in an unobtrusive area of
display 138. In various different embodiments, aux data 322 may be
implemented in any desired shape or size, and may be displayed in
any appropriate location(s) on display 138. For purposes of
illustration, the aux data 322 in FIG. 7B is depicted as a thin
cross-hatched line. However, any effective configuration or
appearance is equally contemplated for implementing aux data
322.
[0061] In the FIG. 7B embodiment, aux data 322 may be encoded to
represent the required information (see FIG. 4) in any effective
manner. For example, in certain embodiments, aux data 322 may be
formatted as one or more horizontal lines of digital video
information positioned in or near the region of the video signal's
vertical blanking interval (VBI). Because digital television is
often encoded with 1280 to 1920 horizontal pixels per scan line,
the FIG. 7B VBI configuration for aux data 322 may provide a
substantial amount of digital information to TV 122.
[0062] The present invention thus supports a method of camouflaging
aux data 322 as video data 316 so that a portion of active video
(potentially visible to the viewer) is used to convey the aux data
322. In addition, the present invention includes standardizing an
encoding format for video aux data 322 to survive video compression
and decompression. The present invention further supports embedding
aux data 322 in the video image so that the aux data 322 can be
recovered (detected, extracted, and processed by TV 122) in a
standardized way, without excessive CPU overhead. The
implementation and utilization of aux data 322 are further
discussed below in conjunction with FIGS. 8A-8C.
[0063] Referring now to FIGS. 8A-8C, a flowchart of method steps
for distributing auxiliary data 322 embedded in video data 316 is
shown, in accordance with one embodiment of the present invention.
The FIG. 8 example is presented for purposes of illustration, and
in alternate embodiments, the present invention may utilize steps
and sequences other than certain of those steps and sequences
discussed in conjunction with the FIG. 8 embodiment.
[0064] In the FIG. 8A embodiment, in step 812, a content source 114
or other appropriate entity initially produces A/V content data
that typically includes video data 316 and audio data 318. In step
814, the content source 114 or other appropriate entity then
creates auxiliary (aux) data 322 to support various advanced
interactive features on a television device 122, such as displaying
one or more synchronized widgets 144. In step 816, the content
source 114 or other appropriate entity inserts the aux data 322
into the video data 316.
[0065] In step 818, the content source 114 or other appropriate
entity compresses the audio data 318 and the video data 316
(including the embedded aux data 322) to create a compressed
distribution multiplex. The FIG. 8A process then advances to step
822 of FIG. 8B through connecting letter "A."
[0066] In step 822 of FIG. 8B, a set-top box 118 or other
appropriate entity receives and demultiplexes the distribution
multiplex distributed by content source 114 to produce compressed
audio data and video data. In step 824, a decoder device of set-top
box 118 or other appropriate entity then uncompresses the
compressed audio data and video data to produce uncompressed audio
data 318 and uncompressed video data 316 (including the embedded
aux data 322). In step 826, the set-top box 118 or other
appropriate entity formats the audio data 318 and the video data
316 for delivery to a display 138 of the television 122. The FIG.
8B process then advances to step 828 of FIG. 8C through connecting
letter "B."
[0067] In step 828 of FIG. 8C, television 122 or other appropriate
entity receives the uncompressed audio data 318 and uncompressed
video data 316 (including the embedded aux data 322). In step 830,
a detection module 620 of the television 122 scans the video data
316 to detect the embedded aux data 322 by utilizing any effective
techniques. In step 832, an extraction module 622 of television 122
extracts the located aux data 322 from the video data 316. Finally,
in step 834, an aux data module 624 processes the extracted aux
data 322 to successfully support appropriate advanced interactive
features, such as displaying one or more synchronized widgets 144
on a display 138 of television 122. The FIG. 8C process may then
terminate.
[0068] In certain alternate embodiments, aux data 322 may similarly
be created and inserted into the video data 316 by any other
appropriate entity at any point along the distribution path. In
certain of these alternate embodiments, aux data 322 may be
inserted without completely decompressing video data 316. For
example, individual macro-blocks of compressed video data 316
(without any aux data 322) could be replaced by corresponding
compressed macro-blocks that contain the aux data 322 already
embedded. For all of the foregoing reasons, the present invention
thus provides an improved system and method for distributing
auxiliary data embedded in video data.
[0069] The invention has been explained above with reference to
certain embodiments. Other embodiments will be apparent to those
skilled in the art in light of this disclosure. For example, the
present invention may readily be implemented using configurations
and techniques other than those described in the embodiments above.
Additionally, the present invention may effectively be used in
conjunction with systems other than those described above.
Therefore, these and other variations upon the discussed
embodiments are intended to be covered by the present invention,
which is limited only by the appended claims.
* * * * *