U.S. patent application number 11/292256 was filed with the patent office on 2006-06-15 for system and method for the creation, synchronization and delivery of alternate content.
This patent application is currently assigned to Sony Electronics Inc.. Invention is credited to Brant L. Candelore, Leo M. JR. Pedlow.
Application Number | 20060130121 11/292256 |
Document ID | / |
Family ID | 36612709 |
Filed Date | 2006-06-15 |
United States Patent
Application |
20060130121 |
Kind Code |
A1 |
Candelore; Brant L. ; et
al. |
June 15, 2006 |
System and method for the creation, synchronization and delivery of
alternate content
Abstract
According to one embodiment, A nonlinear editing system
comprises a video encoder, an audio compressor and an engine unit.
The engine unit is adapted to synchronize the video with the first
alternate video and to synchronize the audio with the first
alternate audio so that, where the scene of the program includes
objectionable content, either (i) the video is substituted with the
first alternate video, (ii) the audio is substituted with the first
alternate audio, or (iii) both the video and the audio are
substituted with the first alternate video and the first alternate
audio.
Inventors: |
Candelore; Brant L.; (San
Diego, CA) ; Pedlow; Leo M. JR.; (San Diego,
CA) |
Correspondence
Address: |
BLAKELY SOKOLOFF TAYLOR & ZAFMAN
12400 WILSHIRE BOULEVARD
SEVENTH FLOOR
LOS ANGELES
CA
90025-1030
US
|
Assignee: |
Sony Electronics Inc.
|
Family ID: |
36612709 |
Appl. No.: |
11/292256 |
Filed: |
December 1, 2005 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60636504 |
Dec 15, 2004 |
|
|
|
60637305 |
Dec 16, 2004 |
|
|
|
Current U.S.
Class: |
725/145 ;
348/E7.061; 725/115; 725/116; 725/146; G9B/27.012; G9B/27.051 |
Current CPC
Class: |
G11B 2220/2562 20130101;
H04N 21/458 20130101; H04N 21/8106 20130101; H04N 21/8456 20130101;
G11B 27/34 20130101; H04N 21/84 20130101; H04N 21/23439 20130101;
H04N 21/2365 20130101; H04N 7/163 20130101; H04N 21/4532 20130101;
H04N 21/4542 20130101; H04N 21/8541 20130101; G11B 20/00159
20130101; G11B 27/034 20130101 |
Class at
Publication: |
725/145 ;
725/116; 725/115; 725/146 |
International
Class: |
H04N 7/173 20060101
H04N007/173; H04N 7/16 20060101 H04N007/16 |
Claims
1. A nonlinear editing system comprising: a video encoder to
receive a first video stream including video originally intended as
video for a scene of a program and a second video stream including
a first alternate video for replacing the video; an audio
compressor to receive a first audio stream including audio
originally intended as audio for the scene of the program and a
second audio stream including a first alternate audio for replacing
the audio; and an engine unit to synchronize the video with the
first alternate video and to synchronize the audio with the first
alternate audio so that, where the scene of the program includes
objectionable content, either (i) the video is substituted with the
first alternate video, (ii) the audio is substituted with the first
alternate audio, or (iii) both the video and the audio are
substituted with the first alternate video and the first alternate
audio.
2. The nonlinear editing system of claim 1, wherein the video
encoder includes a first video encoder to process the first video
stream and a second video encoder to process the second video
stream.
3. The nonlinear editing system of claim 1, wherein the second
video stream includes a blank screen video between the first
alternate video and a second alternate video.
4. The nonlinear editing system of claim 3, wherein the second
audio stream includes muted audio between the first alternate audio
and a second alternate audio.
5. The nonlinear editing system of claim 1, wherein the engine unit
uses synchronization marks inserted by the video encoder and the
audio compressor.
6. The nonlinear editing system of claim 4, wherein the engine unit
trims the blank screen video from the second video stream and the
muted audio from the second audio stream.
7. The nonlinear editing system of claim 6, wherein the engine unit
insert a reminder of the second video stream after trimming of the
blank screen video to be multiplexed into a primary transport
stream including the video and the audio.
8. The nonlinear editing system of claim 7, wherein the engine unit
insert a reminder of the second audio stream after trimming of the
muted audio to be multiplexed into the primary transport
stream.
9. A nonlinear editing system comprising: a video encoder to
receive a first video stream including video originally intended as
video for a scene of a program and a second video stream including
a first alternate video for replacing at least a portion of the
video; an audio compressor to receive a first audio stream
including audio originally intended as audio for the scene of the
program and a second audio stream including a first alternate audio
for replacing at least a portion of the audio; and an engine unit
to determine if the scene of the program includes objectionable
content and to substitute main content, being at least one of the
video and the audio, with replacement content if the scene of the
program includes objectionable content and the replacement does not
feature the objectionable content.
10. The nonlinear editing system of claim 9, wherein the
replacement content includes the alternate video.
11. The nonlinear editing system of claim 9, wherein the
replacement content includes the alternate audio.
12. The nonlinear editing system of claim 9, wherein the
replacement content is one of an ad banner and distorted display
images.
13. The nonlinear editing system of claim 9, wherein the engine
unit determining if the scene of the program features objectionable
content by accessing information provided with main content
including the video and the audio and comparing the information
with prior established parental control limits.
14. The nonlinear editing system of claim 9, wherein the engine
unit accessing information provided with the main content and
comparing the information with prior established parental control
limits by determining whether an Age-based rating assigned to the
scene of the program associated with the main content exceeds an
Age-based parental control limit being the prior established
parental control limit and to substitute the main content with the
replacement content if (i) the Age-based rating of the scene
associated with the main content exceeds the Age-based parental
control limit and (ii) an Age-based rating of the replacement
content is less than or equal to the Age-based parental control
limit.
15. The nonlinear editing system of claim 9, wherein the engine
unit comparing of the information by analyzing tag information
provided with the main content for the scene, determining by
analysis of the tag information if the main content includes
content identified as objectionable, and substituting the main
content with the replacement content if the main content includes
objectionable content as identified by the tag information and the
replacement content does not include the objectionable content.
16. A machine readable medium to store software that, if executed
by an engine unit, will cause a non-linear editing system to
perform the following operations: receiving a first video stream
including video originally intended for a scene of a program and a
second video stream including a first alternate video for replacing
the video; receiving a first audio stream including audio
originally intended for the scene of the program and a second audio
stream including a first alternate audio for replacing the audio;
and synchronizing the video with the first alternate video and to
synchronize the audio with the first alternate audio so that, where
the scene of the program includes objectionable content, either (i)
the video is substituted with the first alternate video, (ii) the
audio is substituted with the first alternate audio, or (iii) both
the video and the audio are substituted with the first alternate
video and the first alternate audio.
17. The machine readable medium of claim 16, wherein the
synchronizing performed by the software includes trimming blank
screen video from the second video stream and muted audio from the
second audio stream.
18. The machine readable medium of claim 17, wherein the
synchronizing performed by the software further includes inserting
a reminder of the second video stream after trimming of the blank
screen video to be multiplexed into a primary transport stream
including the video and the audio.
19. The machine readable medium of claim 18, wherein the
synchronizing performed by the software further includes inserting
a reminder of the second audio stream after trimming of the muted
audio to be multiplexed into the primary transport stream.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of priority on U.S.
Provisional Patent Application No. 60/636,504 filed Dec. 15, 2004
and U.S. Provisional Patent Application No. 60/637,305 filed Dec.
16, 2004, the contents of which are incorporated by reference.
BACKGROUND
[0002] 1. Field
[0003] Embodiments of the invention relate to the field of digital
content transmissions and display. More specifically, one
embodiment of the invention relates to a system and method for
creation, synchronization and delivery of alternate content for
dynamic substitution of unwanted video or audio on a scene-by-scene
basis.
[0004] 2. General Background
[0005] Television is used to deliver content, such as entertainment
and educational programs, to viewers. A growing number of parents
are now watching and listening to content perceived by their
children, in an effort to better shield their young children from
exposure to objectionable content, namely violence, sexual
situations, indecent or suggestive language and the like.
[0006] In television programming, for example, the
removal/restoration of potentially objectionable content is
implemented during content authoring/editing using a nonlinear
editing system.
[0007] As illustrated in FIG. 1, a conventional nonlinear editing
system 100 is used to select desired portions of audio and video
sequences (scenes) taken from one or more libraries 110, which
contain all of the raw content footage (e.g., video, audio, etc.)
under consideration, and then to establish their temporal
relationships. The temporal relationships involve the arrangement
of adjacent sequences of the same content type.
[0008] For instance, as shown in FIG. 2, video sequences 202, 204,
206 and 208, namely uncompressed digital video 200, are arranged in
a temporal relationship to each other. Audio sequences 212, 214,
216 and 218, namely audio 210, are similarly arranged with respect
to each other. Thereafter, each video sequence 202, 204, 206 and
208 is synchronized with one or more audio sequences which may
represent different forms of audio such as dialog tracks 212 and
214, music track 216, sound effects 218, or the like. Normally, the
audio is in the form of Pulse Code Modulation (PCM) audio samples.
Even though the end product consists of one continuous video
sequence with a single synchronized audio track (containing a
composition of multiple forms of audio accomplished by a digital
mixdown unit 120 of FIG. 1), all digital components 200 and 210
comprising the content remain distinct and separate while in
nonlinear editing system 100.
[0009] Referring back to FIG. 1, after the final edited version of
the content is completed, it is then assembled into a compilation
including the selected video and audio sequences. This involves the
selected video and audio sequences being compressed, perhaps using
MPEG-2 video encoder 130 and AC-3 audio compressor 140 for video
and audio respectively, and synchronized with each other to produce
a continuous transport stream 150 of a reduced size. This size
reduction is what makes the transmission, broadcast and/or storage
of digital video content practical, allowing it to fit on
inexpensive media like DVDs or to be carried in standard six
megahertz (6 MHz) Radio Frequency (RF) spectral channel concurrent
with eight or more other A/V services.
[0010] After compression and placement into transport stream 150,
the video and audio cannot be edited. Transport stream 150 features
video running at a constant frame per second rate and compressed
audio synchronized with the video. The audio is continuous as well,
where silence is represented by specific data values. Conventional
nonlinear editing system 100 offers no mechanism to produce any
transport streams with alternate content that can be substituted
for selected video and audio.
[0011] At the destination, with respect to the rendering of video
programs, televisions have been implemented with technologies that
allow parents to preclude the playback of programs that exceed a
selected age-based rating. A black screen is shown for any program
that exceeds the selected age-based rating in order to prevent such
viewing. This program blocking can be disabled with a parental
password.
[0012] However, there are a number of major shortcomings of these
technologies. For instance, the filtering only occurs at the
program level, namely a viewer gets to view the entire program or
nothing at all. In other words, there is no selective blocking
capability, and clearly no ability to substitute alternate content
provided with the transport stream.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Embodiments of the invention are illustrated by way of
example and not by way of limitation in the figures of the
accompanying drawings, in which like references indicate similar
elements and in which:
[0014] FIG. 1 is a block diagram of the operations of a
conventional nonlinear editing system.
[0015] FIG. 2 is a block diagram of a display of content processed
by the conventional nonlinear editing system.
[0016] FIG. 3 is a block diagram of a display of content processed
by a dynamic nonlinear editing system.
[0017] FIG. 4 is a block diagram of the operations of the dynamic
nonlinear editing system developed in accordance with one
embodiment of the invention.
[0018] FIG. 5 is an exemplary block diagram of a content delivery
network consistent with certain embodiments of the invention.
[0019] FIG. 6 is an exemplary block diagram of a content delivery
back-end consistent with certain embodiments of the invention.
[0020] FIG. 7 is an exemplary diagram of a content filter unit
implemented within a digital device of the content delivery system
of FIG. 5.
[0021] FIGS. 8A and 8B are exemplary diagrams of an Adaptation
field of an I-frame, being a portion of a MPEG packet of the
digital data stream.
[0022] FIG. 9 is an exemplary diagram of a first embodiment of a
content replacement process consistent with certain embodiments of
the invention.
[0023] FIG. 10 is an exemplary diagram of a second embodiment of a
content replacement process consistent with certain embodiments of
the invention.
[0024] FIG. 11 is an illustrative flowchart of a process for
determining whether a segment of content associated with a program
should be prevented from being displayed.
[0025] FIG. 12 is an illustrative flowchart of a process for
substituting replacement segments of content based on parental
control parameters.
[0026] FIG. 13 is an exemplary embodiment of a Age-based Rating
scheme for modification or substitution of scenes including
objectionable content.
[0027] FIG. 14 is an exemplary embodiment of a Content-based Rating
scheme for modification or substitution of scenes including
objectionable content.
[0028] FIGS. 15A and 15B are exemplary embodiments of a Tag Rating
scheme for modification or substitution of scenes including
objectionable content.
DETAILED DESCRIPTION
[0029] Various embodiments of the invention relate to a system and
method for creation, synchronization and delivery of alternate
content for dynamic substitution of unwanted video or audio on a
scene-by-scene basis. According to one embodiment of the invention,
a nonlinear editing system supporting dynamic content replacement
(hereinafter referred to as "dynamic nonlinear editing system") is
implemented within the content delivery network. At a digital
device, a content filter unit is implemented within the digital
device and is configured to provide parental control by performing
scene-by-scene evaluation of content associated with the incoming
transport stream (e.g., video, images, audio or any combination
thereof) and subsequent modification or elimination of
objectionable content. Of course, the content filter unit may be
implemented at the front-end in lieu of at the back-end that is
described below in detail.
[0030] In short, the combination of the dynamic nonlinear editing
system and the content filter unit provides scene-by-scene
filtration, and even to the granularity of frame-by-frame, to block
or replace individual scenes or words. Such blocking may be
accomplished by the content filter unit residing with the customer
through screen blocking or obscuring pixels for a particular image
or muting audio. Alternatively, content within the scene can be
replaced in order to reduce the rating of the program (referred to
as "down-rating"), or may even be replaced to reinsert content that
was previously obscured/removed in order for the programming to
achieve the higher rating (referred to as "up-rating").
[0031] According to one embodiment of the invention, the
replacement content is transmitted in the same digital transport
stream as the primary content, and is synchronized with the primary
content that it is replacing for swapping on-the-fly. Thus, the
rendering of higher or lower rated content is accomplished, where
programs can be created and broadcast with multiple versions having
different content and ratings. Also, all content, including
advertisements, news, sporting events and the like may be filtered
as well in lieu of just movies and television shows.
[0032] For purposes of the present description, the term "digital
device" may refer to consumer electronics that is adapted to tune,
receive, decrypt, descramble and/or decode transmissions from any
content provider. The "consumer electronics" may include, but is
not limited or restricted to a television, a set-top box, a
personal digital assistant (PDA), a computer, a cellular telephone,
a video game console, a portable music player, a stereo system, a
personal video recorder, or the like. Examples of "content
providers" may include a terrestrial broadcaster, a cable or
satellite television distribution system, or a company providing
content for download over the Internet or other Internet Protocol
(IP) based networks like an Internet service provider.
[0033] In the following description, certain terminology is used to
describe features of the invention. For example, in certain
situations, the terms "component," "unit" and "logic" are
representative of hardware and/or software configured to perform
one or more functions. For instance, examples of "hardware"
include, but are not limited or restricted to an integrated circuit
such as a processor (e.g., a digital signal processor,
microprocessor, application specific integrated circuit, a
micro-controller, etc.). Of course, the hardware may be
alternatively implemented as a finite state machine or even
combinatorial logic.
[0034] An example of "software" includes executable code in the
form of an application, an applet, a routine or even a series of
instructions. The software may be stored in any type of machine
readable medium such as a programmable electronic circuit, a
semiconductor memory device such as volatile memory (e.g., random
access memory, etc.) and/or non-volatile memory (e.g., any type of
read-only memory "ROM", flash memory, etc.), a floppy diskette, an
optical disk (e.g., compact disk or digital video disc "DVD"), a
hard drive disk, a tape, or the like.
[0035] In addition, the term "program" generally represents a
stream of digital content that is configured for transmission to
one or more digital devices for viewing and/or listening. For
instance, the program may be configured as a transport stream.
According to one embodiment, the program may contain multiple
identifiers such as Packet Identifiers (PIDs) when the program is
MPEG (Moving Pictures Expert Group) compliant compressed video.
These multiple PIDs provide for replacement of one scene of content
for another. The "scene" is defined as one or more frames of
content, namely video, images, audio or any combination thereof.
For instance, a scene may be one or more frames of video and/or
audio, or even a packet of music downloaded into a portable music
player.
[0036] More specifically, multiple PIDs may be associated with a
single program in order to provide content replacement for this
program. Content replacement provides an enhanced level of
viewing/listening customization. For example, content replacement
can be used to provide targeted advertising to an audience by the
substitution of one advertisement determined to be "objectionable"
for another. Content replacement can also be used to provide
multiple scenes, endings or other segments for a program, or
perhaps to provide multiple views in a sportscast or other program.
Other applications for the content replacement of the invention
fall within the spirit and scope of the invention.
[0037] Certain types of content, such as movies, have an associated
content rating established by a supervisory body to inform
potential viewers of the type of material that may form a part of
the content. General ratings of G, PG, PG-13, R and NC-17 are
commonly established for most movies produced in the United States
by the Classification and Ratings Administration (CARA) of the
Motion Picture Association (MPA) or the Motion Picture Association
of America (MPAA). Television programming is similarly rated by the
TV Parental Guidelines Monitoring Board, which provides ratings of
TV-Y, TV-Y7, TV-Y7-FV, TV-G, TV-14 and TV-MA.
[0038] For purposes of this document, content having a rating
provided under such rating systems will be referred to as having a
"higher" rating if it is intended for more mature audiences, and a
"lower" rating if it is intended for a more general or less mature
audience. Hence, the previously described rating systems are listed
from lowest to highest. Therefore, within this terminology, an R
rating would be considered a "higher" rating than a G rating. Such
nomenclature is defined herein for simplicity of description of the
above rating system as well as equivalent systems, without intent
for the nomenclature to pass any judgment on the quality of such
content.
[0039] Additionally, the term "rating" is intended to embrace any
suitable content rating system, domestic or foreign, public,
private or even governmental that is formed to serve a similar
function. Such rating information may be provided as data embedded
within program specific information such as data inserted into an
Adaptation field of a MPEG packet for example when the content is
video.
[0040] While this invention is susceptible of embodiment in many
different forms, there is shown in the drawings and will herein be
described in detail specific embodiments, with the understanding
that the present disclosure is to be considered as an example of
the principles of the invention and not intended to limit the
invention to the specific embodiments shown and described.
[0041] I. General Architecture of Content Delivery Front-End
[0042] In order to support dynamic content substitution, the
authoring process associated with the dynamic nonlinear editing
system is modified to allow concurrent editing of one or more
alternate video tracks and additional audio tracks. Each of the
alternate video tracks is subordinate to the primary video track
and is temporally synchronized with it. After post-processing,
these video tracks will be carried in the same transport stream,
namely as part of the same program. The dynamic nonlinear editing
system is extended to accommodate the additional tracks for
alternate video and audio and is complementary to the editing
paradigm established for this tool.
[0043] As described earlier, the "final cut" is assembled, mixed
(audio) and streamed to compression equipment (encoders). A video
encoder can only accept a single, continuous video stream. The
primary video meets the criterion. The alternate video, however, is
normally a non-continuous sequence of video that is transmitted
concurrently with the primary video and, in some situations, used
to replace the primary video or portions thereof.
[0044] Referring to FIG. 3, a block diagram of a display of content
processed by a dynamic nonlinear editing system being part of a
content delivery front-end is shown. The dynamic nonlinear editing
system 300 is modified to support synchronization and delivery of
replacement content, such as alternate video 315 and/or audio
tracks 360 for example.
[0045] As shown, alternate video 320 is provided during a first
time period 312 (from T2 and T3) of a master timeline 310, which
corresponds to the display duration of video scene 330. Also,
alternate video 340 is provided during a second time period 314
(from T6 and T7) of master timeline 310, corresponding to the
duration of video scene 350. The duration of first and second time
periods 312 and 314 may constitute the display time for one or more
frames of video (e.g. 15 frames of video constituting one-half
second of display time, or perhaps more frames).
[0046] As further shown, alternate audio 360 includes multiple
audio segments 362-366. A first alternate audio segment 362 occurs
at the start of time T0 and ends at time T0.sub.A while a second
alternate audio segment 363 occurs at time T2.sub.A and ends at
time T3. Between time periods T4 and T5, a third and fourth
alternate audio segments 364-365 are placed, namely between
T4.sub.A-T4.sub.B and T4.sub.C-T4.sub.D. Similarly, a fifth
alternate audio segment 366 occurs at time T6.sub.A and ends at
time T6.sub.B. For this embodiment, first, fourth and fifth
alternate audio segments 362, 365 and 366 constitute alternate
dialog. Second audio segment 363 constitutes an alternate music
track and third audio segment 364 constitutes an alternate special
effects track.
[0047] Referring to FIG. 4, a block diagram of the operations of
dynamic nonlinear editing system 300 developed in accordance with
one embodiment of the invention is shown. Dynamic nonlinear editing
system 300 receives information from content storage units 410 and
412, which are configured to store video and audio content. Dynamic
nonlinear editing system 300 is adapted to sequence video and audio
retrieved from content storage units 410 and 412, and then to
establish their temporal relationships. The temporal relationships
involve the arrangement of adjacent sequences of the same content
type.
[0048] In order to remain compatible with video encoder 420,
dynamic nonlinear editing system 300 fills the time periods before,
after and between alternate video 320 and 340 with synchronized
black screen in order to create a continuous video stream for
processing by video encoder 420. It is contemplated that the
encoding of the primary and alternate video will occur on the same
encoder, and therefore, the two processes will occur serially. As a
result, dynamic nonlinear editing system 300 will need to
communicate with video encoder 420 so that synchronization
information may be inserted by video encoder 420 into both
resultant compressed video streams 430 and 440 for post-encoding
reprocessing where video streams 430 and 440 are combined with
proper synchronization.
[0049] Similarly, dynamic nonlinear editing system 300 fills the
time periods before, after and between alternate audio segments
362-366 with muted audio in order to create a continuous audio
stream, for processing by audio compressor 450. It is contemplated
that the encoding of the primary and alternate audio may occur
serially on the same audio compressor 450, or on multiple audio
compressors. When using the same audio compressor 450 as shown,
dynamic nonlinear editing system 300 will need to communicate with
audio compressor 450 so that synchronization information may be
inserted by audio compressor 450 into both resultant compressed
audio streams 460 and 470 for post-encoding reprocessing.
[0050] A transport processing engine unit 480 receives video
streams 430 and 440 and audio streams 460 and 470. Transport
processing engine unit 480 conducts a trimming operation of
alternate content as well as performs content synchronization,
content re-multiplexing and meta data insertion. These operations
are complementary with the functionality of device 510 of FIG. 5,
and perhaps substantially similar where content replacement is
conducted in the front end.
[0051] First, the alternate content contains blank video (black
screen) and muted audio between alternate content segments. This is
a byproduct of preparing the content for compression. Transport
processing engine unit 480 trims all black screen content and muted
audio purposefully added to create the continuous audio. This
allows the alternate content to be multiplexed into the primary
transport stream.
[0052] As an illustrative example, referring to FIG. 3, video scene
330 and alternate video 320 need to be processed to ensure that
these video segments are temporally equivalent within the transport
stream. In other words, these video segments need to be virtually
identical in duration and aligned temporally where such editing
occurs after being encoded/compressed and during placement within
the transport stream. The same temporal alignment is required for
video scene 350 and its alternate video 340.
[0053] Secondly, transport processing engine unit 480 uses
synchronization marks inserted by video encoder 420 and audio
compressor 450 to ensure that the digital device will display
either the primary or alternate video at the same time. As an
example, for MPEG transmissions, three synchronization elements are
used for video: PCR (primary clock reference)--clocking at
approximately 27 megahertz (MHz); DTS (decoding time stamp)--placed
on video itself to start conversion from MPEG to analog video
format; and PTS (presentation time stamp)--when picture should be
displayed. Both primary video 330 and alternate video 320 should
have the same synchronization marks.
[0054] Similar synchronization marks may be used for audio, and
thus, the primary and alternative audio will have the same
synchronization marks. This allows alternate content to be
correctly located temporally within the transport stream so that
primary/alternate video as well as the primary/secondary audio can
be contextually located adjacent to each other.
[0055] Third, transport processing engine unit 480 merges the
alternate content into the transport stream and inserts meta data
as needed to enable suitably equipped playback devices, such as
digital device as described below, to dynamically select any
combination or primary/alternate content during broadcast or
playback of the resultant composite content. As part of the merging
process, the re-multiplexer corrects PCR synchronization marks and
other tasks normally encountered and associated with the
re-multiplexing process.
[0056] According to one embodiment, the meta data is inserted into
an Adaptation field of one or more MPEG-based packets forming the
transport stream. The meta data may provide Age-based rating
information, content tag information or the like as described
below.
[0057] The composite, homogeneous output 490 from transport
processing engine unit 480 is then returned to the normal content
process flow, where it is stored or forwarded to a distribution
phase by a content provider as describe below.
[0058] II. General Architecture of Content Delivery Back-End
[0059] Referring to FIG. 5, an exemplary block diagram of a content
delivery back-end 500 consistent with certain embodiments of the
invention is shown. Content delivery back-end 500 comprises a
digital device 510 that receives digital content such as a program
from one or more content providers 520. The program may be
propagated as a digital data stream for example in compliance with
any data compression scheme. Examples of a data compression scheme
include, but are not limited or restricted MPEG standards.
[0060] According to one embodiment of the invention, digital device
510 decodes an incoming program, and evaluates the content of the
incoming program on a scene-by-scene basis to determine whether
such scenes include any objectionable content. Such evaluation may
involve a review of the rating assigned to each scene or perhaps
access data within tags assigned to each scene to determine whether
the scene includes any objectionable content. These evaluation
schemes are described below.
[0061] Since digital device 510 may be adapted to increase or
decrease the rating of the program, the term "objectionable
content" has two meanings. One meaning is that it is content
featuring certain subject matter that may be inappropriate for the
viewing or listening audience at this time. Another meaning is that
it is low-rated content that can be substituted for higher rated
content due to the mature nature of the viewing or listening
audience and their request for up-rating of content.
[0062] Content provider 520 provides the digital content to digital
device 510 through transmission medium 530, which operates as a
communication pathway for the program within the content delivery
network. The transmission medium 530 may include, but is not
limited to electrical wires, optical fiber, cable, a wireless link
established by wireless signaling circuitry, or the like.
[0063] Content provider 520 may be adapted with a satellite antenna
600 to receive a multiplexed stream of content from a satellite
transmission as shown in FIG. 6. The stream of content is received,
demodulated and decrypted at a satellite receiver 605 before being
routed to PID Mapping logic 610. The stream of content may include
replacement content associated with secondary PIDs. Alternatively,
the replacement content may be retrieved from a local content
database 615 or other sources of content.
[0064] PID mapping logic 610 maps the incoming content from
whatever source to a set of primary PIDs and a set of secondary
PIDs. The incoming content includes (i) content originally intended
for rendering (referred to as the "primary content") and (ii)
substitution content for various scenes (referred to as
"replacement content"). The primary content is associated with the
primary PIDs and the replacement content is associated with the
secondary PIDs. According to this embodiment, PID mapping logic 610
may also be adapted to insert flags (or markers) into the content
in order to identify the location where replacement content is to
be inserted.
[0065] The mapped content then is routed to PSI/PMT insertion logic
620 that inserts Program Specific Information (PSI) and Program Map
Tables (PMT) into the content for use by the decoding side in
decoding the programming. If the content is to be encrypted, it may
optionally pass through encryption logic 625 prior to modulation at
a modulator (e.g., a QAM modulator) 630. The modulated stream of
content is then transmitted via transmission medium 530 to content
delivery back-end 500, which features one or more digital
device(s). For instance, the modulated stream of content may be
routed to via a cable plant 635 to content delivery back-end 500.
The operation performed by content provider 520, including but not
limited to the PID mapping for content replacement, is carried out
under control of a control computer 640.
[0066] In general, a system can be used to form a content
substitution encoder consistent with certain embodiments of the
invention in which input data inclusive of primary content and
replacement content. PID mapping logic 610 assigns a primary PID to
the primary content and assigns a secondary PID to the replacement
content. Private data to identify the primary content by the
primary PID and replacement content by the secondary PID, along
with the primary content mapped to the primary PID and the
replacement content mapped to the secondary PID, are assembled into
a data stream.
[0067] Of course, it is contemplated that PID mapping logic 610 may
be instructed to remap the PIDs to select only the desired content
based on parental control parameters routed to content provider 520
from various digital devices. Hence, if utilized, secondary PIDs
may be associated with previously substituted segments of primary
content.
[0068] Referring now to FIG. 7, an exemplary diagram of digital
device 110 of the content delivery system 100 is shown. Digital
device 110 comprises a power supply unit 700, a tuner/demodulation
unit 710 and data processing unit 720. Power supply unit 700 is
adapted to provide powered to tuner/demodulation unit 710 and data
processing unit 720. Power supply unit 700 may provide regulated
power based on input power received from a wall socket (e.g., 110
volts alternating current "VAC", 220 VAC, etc.). Tuner/demodulation
unit 710 tunes to a channel to receive the program and demodulates
an incoming program to recover the content. Then, the content is
routed as a digital bit stream to data processing unit 720.
[0069] According to one embodiment of the invention, data
processing unit 720 comprises a processor 730 (e.g., central
processing unit) and/or a decoder 740. These components may be
implemented as separate integrated circuits or as a single
integrated circuit. As described herein, data processing unit 720
features logic operating to decode and process incoming information
as well as a content filter unit 750 that is adapted to
alternatively select replacement content during playback of video
and/or audio.
[0070] In general, according to one embodiment of the invention,
content filter unit 750 is designed to select alternate PID(s) from
the digital bit stream based on findings by data processing unit
720, which accesses meta data in Adaptation fields within MPEG
packets to obtain information for use in evaluating whether a scene
has objectionable content.
[0071] According to one embodiment, data processing unit 720 is
adapted to process the closed captioning and emergency alert system
(EAS) data as well as guide data. In addition, data processing unit
720 may be adapted to access the meta data in Adaptation fields of
MPEG packets in order to provide signaling to content filter unit
750 regarding the subject matter within each scene that is
scheduled to be played back in the near future. This allows content
filter unit 750 to determine whether the subject matter constitutes
objectionable content.
[0072] Data processing unit 720 (or perhaps a separate graphics
component) is adapted for rendering on-screen menus or guide data.
For instance, data processing unit 720 may be adapted to collect
guide data and for storage of such guide data in local memory (not
shown). When such data is requested, it is displayed and may even
overlay video concurrently being displayed. Also, data processing
unit 720 may be adapted to distort or blank out displayed
images.
[0073] Referring to FIGS. 8A and 8B, exemplary diagrams of an
Adaptation field 810 of an I-frame 800, being a portion of a MPEG
packet 805 of the digital data stream is shown. Adaptation field
810 features control information 815 (also referred to as "private
data") to initiate a filtering operation is shown. According to one
embodiment of the invention, Adaptation field 810 includes control
information 815 used by data processing unit 720 of FIG. 7 to (i)
determine if content replacement is enabled or disabled and (ii)
evaluate whether replacement content should be used. Such
evaluation is conducted ahead of any scheduled rendering (playback
for visual or audible perception) of scenes of digital content.
[0074] Adaptation field 810 includes control information 815 that
features a plurality of sub-fields, including but not limited or
restricted to one or more of the following: (1) PID sub-field 820,
(2) Conversion type sub-field 825; (3) Content type sub-field 830;
(4) Replacement type sub-field 835; (5) Length sub-field 840; (6)
Rating sub-field 845; (7) Tag sub-field 850; (8) Intensity level
sub-field 855; (9) Filter blocking summary sub-field 860 and (10)
Flag deletion sub-field 865.
[0075] As illustrated in FIG. 5B, the PID sub-field 820 is adapted
to identify what PIDs are provided for a scene started by I-frame
800. These PIDs may include a single (Primary) PID where no
replacement content is available, and/or one or more secondary PIDs
(e.g., PID-2, PID-3, etc.).
[0076] Conversion type sub-field 825 identifies whether the
replacement content associated with a secondary PID is an up-rating
(e.g., increasing the rating level and making the program for more
mature audiences) or a down-rating (e.g., decreasing the rating
level and making the displayed program for more general
audiences).
[0077] Content type sub-field 830 identifies the type of content
associated with I-frame 800 and the scene associated with I-frame
800. For instance, content type sub-field 830 may identify the
content as a movie, a television series, a news clip, a live
sporting event, a live entertainment event such as a concert, etc.
This information is used to allow content filter unit 750 of FIG. 7
to filter a variety of types of content, especially those types of
content that have not been subject to a ratings scheme.
[0078] Replacement type sub-field 835 identifies the type of
replacement scheme available. For instance, the replacement scheme
may involve full scene replacement or inter-scene data replacement
(e.g., pixel obfuscation, audio muting, etc.) to reduce
transmission bandwidth requirements. Alternatively, the replacement
scheme may simply involve the insertion of content markers that are
used by personal video recorders (PVRs) or other types of digital
devices to adjust content playback during post-recording
solutions.
[0079] Length sub-field 840 identifies the number of frames (I, B,
P) associated with the length of the digital data stream forming
the content.
[0080] Rating sub-field 845 provides an age or content rating
associated with the scene that are used to determine if any
modification, replacement or preclusion involving the rendering of
a scene is required.
[0081] Tag sub-field 850 provided expanded filtering capabilities
that enable filtering to be conducted based on specific types of
content found in each scene and not the overall rating of the
scene.
[0082] Intensity level sub-field 855 provides an alphanumeric value
of the intensity of certain subject matter, such as levels of
violence, sex or language for example, to provide greater
granularity for the filtering operation. This granularity may avoid
false-positives where some content may feature content that is in
an objectionable category, but such inclusion is minimal or not
graphic.
[0083] Filter blocking summary sub-field 860 provides for a
contextual (tag) overrides described below.
[0084] Flag deletion sub-field 865 is a flag (or marker) used by
PVR and perhaps other recording digital devices to remove blocking
content.
[0085] III. Content Replacement Techniques
[0086] Referring to FIG. 9, an exemplary diagram of a first
embodiment of a content replacement process consistent with certain
embodiments of the invention is shown. For this embodiment, no
replacement content is provided in the digital data stream, but
control information is placed within the MPEG packets such as
within the Adaptation field of the I-frame as described above. In
response to detecting objectionable content, accomplished by
comparing preset parental control parameters with information
contained in rating, tag and/or intensity sub-fields 845-855 of
FIG. 8B, the content filter unit causes a black frame 900 to be
displayed in lieu of the objectionable content. Alternatively, in
lieu of a black screen, a pre-stored banner 910, namely selected
advertisements paid by various companies having a rating no greater
than the limits (e.g., ratings, absence of certain subject matter,
etc.) established for the parental control parameter(s)
(hereinafter referred to as "parental control limits"), may be
displayed if the blocking is for a longer duration.
[0087] Referring now to FIG. 10, an exemplary diagram of a second
embodiment of a content replacement process consistent with certain
embodiments of the invention is shown. For this embodiment, primary
content 1000 formed by a plurality of primary content segments
1010.sub.1-1010.sub.N (where N.gtoreq.1), which are associated with
corresponding primary identifiers (ID-1), such as a Packet
Identifier (PID), and loaded into content filter unit 750 of FIG.
7. Replacement content 1020, namely replacement content segments
1030.sub.2, 1030.sub.5-1030.sub.7 & 1035.sub.2 associated with
certain secondary identifiers (e.g., ID-2, ID-3), is also provided
to the content filter unit. Replacement content segments
1030.sub.2, 1030.sub.5-1030.sub.7 & 1035.sub.2 may be lower
rated or higher rated content than corresponding primary content
segments 1010.sub.2 & 1010.sub.5-1010.sub.7.
[0088] In response to detecting objectionable content in primary
content segments 1010.sub.2 and 1010.sub.7, replacement content
segments 1035.sub.2 and 1030.sub.7 are substituted for content
segment 1010.sub.2 and 1010.sub.7, respectively. The collection of
primary content segments and replacement content segments are
output for viewing and/or listening by the end user. Upon
evaluation of the primary content segment 1010.sub.3, it is
determined that this also includes objectionable content, but no
replacement content segment is provided. As a result, screen
blanking or audio muting is conducted as described in FIG. 7 as
denoted by "X".
[0089] Referring now to FIG. 11, an illustrative flowchart of a
process for determining whether a segment of content associated
with a program should be prevented from being displayed is shown.
As illustrated, initially programmed parental control parameters
are examined to determine what constitutes "objectionable content"
for this content filter unit (block 1100). The parental control
parameters are preset as shown in FIGS. 13, 14 and 15A.
[0090] Next, a determination is made whether the content filter
unit has been enabled (block 1110). If not, the V-chip is used as
an alternative source for regulating the content to be viewed
(arrow 1115). If the content filter unit has been enabled, a first
evaluation is whether the program being transmitted can be altered
and maintained within the parental control limits set for the
content filter unit (block 1120). This can be accomplished by
analysis of a meta data regarding the content that summarizes the
ratings (age or content based), content labels or content tags on a
per scene or per program basis. As a result, replacement content
having a lower rating (referred to as "down-rating") or having a
higher rating may be substituted for the primary (chosen) content,
provided such content still falls within the parental control
limits (also referred to as "up-rating").
[0091] For each scene, a portion of the incoming content segment,
such as the Adaptation field for example, is accessed to retrieve
information therefrom (block 1130). A determination is made whether
the received content segment includes control information to enable
filtering of the content (block 1140). If not, the received content
segment is rendered without filtering (block 1150). If the received
content segment includes control information, a determination is
made whether the primary content should be substituted for
replacement content (blocks 1160 and 1170). The replacement scheme
is set forth in FIG. 12.
[0092] FIG. 12 provides an illustrative flowchart of a process for
substituting replacement content based on parental control
parameters. Initially, as shown in block 1200, parental control
parameters associated with the scene are examined. Next, a
determination is made whether the display of the scene is
controlled based on an Age Rating (block 1210). If so, Age-based
rating processing is conducted where the Age Rating of the scene is
compared to Aged-based parental control limits programmed by the
end user and monitored by the content filter unit (block 1215). In
the event that the Age Rating of the scene exceeds the Aged-based
parental control limits, the scene is not rendered, but instead may
be down-rated through substitution of replacement content or
blocked. This substitution may be regulated through use of the
synchronization marks described above.
[0093] As an option, the content filter unit can be programmed to
substitute replacement content having a higher rating than the
primary content. The higher rated content would be displayed if the
following conditions are met: (1) replacement content has a higher
rating than the primary content, and (3) the Age Rating of the
replacement content does not exceed the Aged-based parental control
limits.
[0094] For instance, as shown in FIG. 13, an exemplary ratings
table 1300 for objectionable content is shown. Herein, the content
filter unit is programmed by setting an Age-based parental control
limit within a first parental control parameter 1310. As an
example, the Age Rating is established to cover age-based rating
levels G, PG, PG-13, R and NC-17 ratings used in movies as well as
rating levels TV-Y, TV-Y7, TV-Y7-FV, TV-G, TV-14 and TV-MA used in
television broadcasts. For instance, TV-14 may have the same Age
rating level as PG-13 (V,S), where "V" denotes violence and "S"
denotes sexual content.
[0095] As an illustrative example, a first scene 1320 of a program
may feature PG-13 content as identified by a primary identifier
(ID-1), and thus, no substitution is conducted if the Aged-based
parental control limit is set to "PG-13". However, if the
Aged-based parental control limit is set to allow only "G" (or
lesser rated) movies or related television broadcasting (e.g.
TV-G), replacement content associated with a first alternative
identifier (ID-2) would be rendered for the first scene. However,
if no age threshold was set, the "R" version of the scene would be
rendered by substitution of the "PG-13" content associated with
ID-1 with "R" scene content associated with a second alternative
identifier (ID-3). The same operations would occur for each
scene.
[0096] As a result, for "PG-13" Aged-based parental control limits
set within first parental control parameter 1310, content segments
for scenes 1-8 associated with ID-1 would be rendered. For "G"
Aged-based limits set within first parental control parameter 1310,
content segments associated with ID-1 (scenes 4 and 8) and ID-2
(scenes 1-3 & 5-7) would be shown. Where no Aged-based limits
are established and the content filter unit is programmed to
increase the rating of the content where possible, content
associated with ID-1 (scene 3), ID-2 (scene 8) and ID-3 (scenes 1-2
& 4-7) would be shown.
[0097] Referring back to FIG. 12, another determination is whether
the display of the scene is controlled based on content label
scheme (blocks 1220 and 1225). With respect to content label
evaluation scheme, a determination is made if any particular
category of content is present. This is similar to Content Rating
scheme describe below where the "Intensity" parental control limits
are set to "0", which also indicates that if there is any content
with this subject matter, replacement content without such subject
matter (and within the limits set for other types of subject
matter) is selected. If no replacement content is available, the
content is blocked or muted.
[0098] Yet another determination is whether the display of the
scene is controlled based on content rating scheme (block 1230). If
so, an Intensity rating is applied to a variety of subject matter
categories that parents may find objectionable for their children
(block 1235). For instance, these categories may be broadly defined
such as violence, sex, language, and the like. For those content
segments that include this subject matter, "Intensity" values are
assigned for each scene based on the maturity level of the subject
matter.
[0099] As an illustrative example, as shown in FIG. 14, if a first
scene 1400 for primary content 1410 of a program includes a fist
fight along with a curse word, the violence rating may be set to a
first prescribed value, which will likely be fairly low (e.g., 2
out of 15). Depending on the curse word spoken and the audible
nature of the curse word, the language rating may be set to a
second prescribed value (e.g., 6 out of 15). Replacement content
1412 is provided and associated with ID-2, it may include the
elimination of the fist flight and merely a shouting match without
any curse words. This replacement content may be substituted if the
violence parental control parameters for violence and language are
less than "2" and "6", respectively. Otherwise, the primary content
is shown since these Intensity values are less than or equal to the
parental control limits of "6", "7" and "3" for violence, language
and sex parental control parameters 1420, 1422 and 1424.
[0100] Of course, as an alternative, there may be replacement
content 1414 where the fist fight is substituted with a knife fight
as originally intended by the director for this program. This
replacement content would have a higher violence rating (e.g., 6
out of 15). Since the parental control limits for violence were set
at "6", replacement content associated with ID-3 would be shown if
up-rating of the content is desired.
[0101] For these parent control limits, if no up-rating is desired,
scenes 1, 5-6 and 8 would render content associated with ID-1
(primary ID) and scenes 2-4 and 7 would render content associated
with ID-2. With respect to scenes 5 and 7, since no up-rating is
desired, the primary content is normally retained as a default,
providing scenes 5 and 7 fall within the parental control limits
selected. However, it is contemplated that the default setting can
be configured to select the content segment having a rating closest
to the parental control limits.
[0102] In the event that up-rating of content has been selected,
scenes 2 and 8 would render content associated with ID-1 (primary
ID), scenes 3-6 would render content associated with ID-2 (first
alternative ID) and scenes 1 and 7 would render content associated
with PID-3 (second alternative ID).
[0103] Referring back to FIG. 12, another determination is whether
the display of the scene is controlled based on a tag rating scheme
(blocks 1240 and 1245). As shown in FIGS. 15A and 15B, the "tag
rating" scheme provides a substantial granularity as to what
specific content may be excluded. According to one embodiment, each
category 1500-1540 and sub-category is uniquely assigned a unique
bit pattern so that a multi-byte tag value associated with each
scene may be used to identify multiple categories where potential
objectionable content may be found. Hence, specific renderings of
violence, language, sex, etc. may be prevented.
[0104] As an optional configuration, each category may be assigned
a particular bit and sub-categories are identified by bit
combinations inclusive of the bit assigned to the category. As an
example, as shown in FIG. 15B, a tag table may be created where a
fourth least significant bit (bbb00001xxx) of a double word tag
(4-bytes, where "b" denotes byte) is set when the scene involves
violence. For the violence categories, a number of subcategories
may be used to denote blood and gore (bbb00001001); sexual assault
(bbb00001010), disturbing images (bbb00001011), etc.
[0105] Hence, for each scene, the tag is extracted and processed,
such as conducting Exclusive OR'ed with the parent control
parameter tags set by the user, in order to determine if any
objectionable content is found in the scene. If not, the original
content may be displayed. Otherwise, if objectionable content is
found, a determination is made whether the replacement content
features the objectionable content. If not, the replacement content
is displayed. Otherwise, if down-rating is needed, no override is
indicated, and no replacement content is available, the primary
content is blocked or muted.
[0106] Upon determining that the scene includes objectionable
content, as an optional feature, a determination may be conducted
to understand the context that the objectionable content is
featured.
[0107] For instance, as an illustrative example, certain nudity may
be prevalent for a medical procedure featured in the program.
Therefore, if the program is a documentary on medical procedures,
various bits reserved in the parental control parameters or
separate context identifiers coupled be selected as illustrated in
FIG. 15B. Hence, certain sub-categories of sex tags may be ignored.
Similarly, a gun fight may be prevalent in a documentary regarding
the Wild West so any violence tags below a predetermined rating
(e.g., rating "6") would be ignored. Hence, the context parameter
can be used to override the content segment replacement operations
described above.
[0108] In the foregoing description, the invention is described
with reference to specific exemplary embodiments thereof. It will,
however, be evident that various modifications and changes may be
made thereto without departing from the broader spirit and scope of
the present invention as set forth in the appended claims. The
specification and drawings are accordingly to be regarded in an
illustrative rather than in a restrictive sense.
* * * * *