U.S. patent application number 09/201529 was filed with the patent office on 2002-02-07 for method and apparatus for simultaneously producing compressed play and trick play bitstreams from a video frame sequence.
Invention is credited to GORDON, DONALD F., LUDVIG, EDWARD A..
Application Number | 20020015576 09/201529 |
Document ID | / |
Family ID | 22746204 |
Filed Date | 2002-02-07 |
United States Patent
Application |
20020015576 |
Kind Code |
A1 |
GORDON, DONALD F. ; et
al. |
February 7, 2002 |
METHOD AND APPARATUS FOR SIMULTANEOUSLY PRODUCING COMPRESSED PLAY
AND TRICK PLAY BITSTREAMS FROM A VIDEO FRAME SEQUENCE
Abstract
An "all-in-one" encoder that simultaneously produces an MPEG-2
compliant fast forward, fast reverse and play bitstreams from a
sequence of video frames, e.g., 601-format video. The encoder
subsamples the video sequence, extracting a plurality of frames
from the video sequence and buffering the subsampled frames.
Simultaneous with the subsampling and buffering, the source frames
are encoded within a real-time encoder, e.g., an MPEG-2 encoder.
The buffered frames are recalled from the buffer and coupled to a
second real-time encoder. The second encoder forms both the fast
forward and fast reverse bitstreams using a time multiplexing
technique wherein a group of pictures (GOP) for the fast forward
stream can be formed, followed by the compression of the same GOP
having the frames organized in reverse order. As such, the
compressed GOPs are represented by the fast forward and fast
reverse bitstreams. The play, fast forward, and fast reverse
bitstreams for each GOP are organized into a file and stored on the
mass storage device (e.g., disk drive array) of the file
server.
Inventors: |
GORDON, DONALD F.; (LOS
ALTOS, CA) ; LUDVIG, EDWARD A.; (REDWOOD CITY,
CA) |
Correspondence
Address: |
MOSER, PATTERSON & SHERIDAN L.L.P.
595 SHREWSBURY AVE
FIRST FLOOR
SHREWSBURY
NJ
07702
US
|
Family ID: |
22746204 |
Appl. No.: |
09/201529 |
Filed: |
November 30, 1998 |
Current U.S.
Class: |
386/314 ;
348/E5.008; 375/E7.017; 375/E7.022; 375/E7.151; 375/E7.179;
375/E7.211; 375/E7.22; 375/E7.253; 386/E5.052 |
Current CPC
Class: |
H04N 21/23608 20130101;
H04N 21/2387 20130101; H04N 21/4331 20130101; H04N 19/177 20141101;
H04N 21/643 20130101; H04N 5/783 20130101; H04N 21/4344 20130101;
H04N 21/4147 20130101; H04N 21/234381 20130101; H04N 5/78263
20130101; H04N 19/61 20141101; H04N 21/434 20130101; H04N 19/587
20141101; H04N 21/8455 20130101; H04N 21/231 20130101; H04N 9/8042
20130101; H04N 21/47202 20130101; H04N 19/114 20141101; H04N 9/8227
20130101; H04N 21/236 20130101 |
Class at
Publication: |
386/68 ;
386/111 |
International
Class: |
H04N 005/783 |
Claims
What is claimed is:
1. Apparatus for producing a plurality of encoded video bitstreams
comprising: a first encoder for producing a first bitstream that
contains information that, when decoded, produces a standard
forward play video frame sequence; a frame subsampler; a buffer
that stores subsampled frames of the video sequence; a second
encoder for producing a second bitstream that contains information
that, when decoded, produces a first trick play frame sequence; a
third encoder for producing a third bitstream that contains
information that, when decoded, produces a second trick play video
frame sequence; and a controller that selects subsampled frames
from the buffer and couples to selected frames to the second and
third encoders.
2. The apparatus of claim 1 wherein said first encoder is an MPEG
encoder that encodes N frames of the video sequence.
3. The apparatus of claim 1 wherein said second and third encoders
are MPEG encoders that encodes R/N frames of the video sequence
having N frames.
4. The apparatus of claim 1 wherein said first trick play video
frame sequence is a fast forward video frame sequence.
5. The apparatus of claim 1 wherein said second trick play video
frame sequence is a fast reverse video frame sequence.
6. The apparatus of claim 1 wherein the controller multiplexes
selection of the frames from the buffer to apply plurality of
subsampled frames to said second encoder to form said second
bitstream and then apply a plurality of subsampled frames to said
third encoder to form said third bitstream.
7. A method of encoding frames of a video sequence to form a
plurality of bitstreams comprising the steps of: encoding said
frames to form a first bitstream; subsampling said frames;
buffering said subsampled frames; recalling said buffered frames in
a forward time sequence order; encoding said recalled buffered
frames to form a second bitstream; recalling said buffered frames
in a reverse time sequence order; encoding said recalled buffered
frames to form a third bitstream.
8. The method of claim 7 wherein said first bitstream when decoded
forms a standard play frame sequence.
9. The method of claim 7 wherein said second bistream, when
decoded, forms a fast forward frame sequence.
10. The method of claim 5 wherein said third bitstream, when
decoded, forms a fast reverse frame sequence.
11. A method of encoding frames of a video sequence to form a
plurality of bitstreams comprising the steps of: encoding said
frames to form a first bitstream; while encoding said frames to
form said first bitstream, selecting a subsampled set of frames
from said frames of said video sequence; encoding the subsample set
of frames to form a second bitstream.
12. The method of claim 11 further comprising the steps of:
buffering said subsampled set of frames; recalling said buffered
frames in a forward time sequence order; and encoding said recalled
buffered frames to form said second bitstream.
13. The method of claim 12 wherein said second bitstream, when
decoded, forms a fast forward video frame sequence.
14. The method of claim 11 further comprising the steps of:
buffering said subsampled set of frames; recalling said buffered
frames in a reverse time sequence order; and encoding said recalled
buffered frames to form said second bitstream.
15. The method of claim 14 wherein said second bitstream, when
decoded, forms a fast reverse video frame sequence.
16. The method of claim 11 further comprising the steps of:
buffering said subsampled frames of said frames in said video
sequence; recalling said buffered frames in a forward time sequence
order; encoding said recalled buffered frames to form a second
bitstream; recalling said buffered frames in a reverse time
sequence order; encoding said recalled buffered frames to form a
third bitstream.
17. The method of claim 16 wherein said first bitstream when
decoded forms a standard play frame sequence.
18. The method of claim 16 wherein said second bistream, when
decoded, forms a fast forward frame sequence.
19. The method of claim 16 wherein said third bitstream, when
decoded, forms a fast reverse frame sequence.
Description
BACKGROUND OF THE DISCLOSURE
[0001] 1. Field of the Invention
[0002] The invention relates to video-on-demand systems and, more
particularly, the invention relates to a method and apparatus for
simultaneously generating compressed play and trick play bitstreams
from a video frame sequence.
[0003] 2. Description of the Background Art
[0004] In a video-on-demand system such as the OnSet.TM. system
manufactured by DIVA Systems Corporation, a file server is used for
streaming video information to users (subscribers) of the system.
The OnSet.TM. system is described in U.S. patent application Ser.
No. 08/984,710, filed Dec. 3, 1997, and the file server is
described in U.S. Pat. Nos. 5,671,377 and 5,581,778. The
disclosures of this application and these patents are incorporated
herein by reference. The OnSet.TM. system contains service provider
equipment coupled through an information distribution network to
subscriber equipment. This system provides subscribers VCR-like
controls to enable a subscriber to select information content, for
example, a movie, then play, fast forward, rewind, pause, or stop
the selected movie. The subscriber enters control commands through
the subscriber equipment and the service provider equipment
executes the commands to fulfill the purpose of the command, e.g.,
play, fast forward, rewind, stop or pause the movie.
[0005] A file server forms a portion of the service provider
equipment and stores, for a given movie, a standard play stream
(i.e., an MPEG-2 compressed video bitstream), a fast forward stream
and a fast reverse play (rewind) stream. The fast forward and fast
reverse streams are also MPEG-2 compressed video, where the fast
forward stream is a compressed signal containing only every Nth
frame of the uncompressed play video and the fast reverse stream is
a compressed signal containing every Nth frame of the uncompressed
play video played backwards. To store the video information, each
stream is divided into portions (e.g., N-byte segments of
compressed data) and striped onto a disk array. The disk array
stores all the information that the file server can directly
access.
[0006] The process for generating the fast forward and fast reverse
streams is time consuming and expensive. The process requires a
frame by frame analysis of the video stream to enable the frames to
be encoded in an MPEG-2 compliant form, yet provide a fast forward
and fast reverse effect when decoded. As such, the video sequence
is processed to extract every Rth frame (i.e., one of every R
frames, where R is an integer greater than 1) to form a fast
forward sequence and then the fast forward sequence is encoded
(compressed). For a fast reverse stream, a sequence of frames in
reverse order is created and encoded. Such encoding can cost $17 to
$20 per minute of video. This encoding process can not be used for
producing fast forward and fast reverse streams in real time such
that a real time program can be encoded and stored for almost
immediate use of VCR-like functions.
[0007] Therefore, there is a need in the art for an improved
encoding system for a video-on-demand system to ensure real-time
availability of fast forward and fast reverse functions.
SUMMARY OF THE INVENTION
[0008] The disadvantages associated with the prior art are overcome
by the invention of an "all-in-one" encoder that simultaneously
produces an MPEG-2 compliant fast forward, fast reverse and play
bitstreams from a sequence of video frames, e.g., 601-format video.
The encoder of the present invention subsamples the video sequence,
extracting a plurality of frames from the video sequence and
buffering the subsampled frames. Simultaneous with the subsampling
and buffering, the source frames are encoded within a real-time
encoder, e.g., an MPEG-2 encoder. The buffered frames are recalled
from the buffer and coupled to a second real-time encoder. The
second encoder forms both the fast forward and fast reverse
bitstreams using a time multiplexing technique wherein a group of
pictures (GOP) for the fast forward stream can be formed, followed
by the compression of the same GOP having the frames organized in
reverse order. As such, the compressed GOPs are represented by the
fast forward and fast reverse bitstreams. The play, fast forward,
and fast reverse bitstreams for each GOP are organized into a file
and stored on the mass storage device (e.g., disk drive array) of
the file server.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] The teachings of the present invention can be readily
understood by considering the following detailed description in
conjunction with the accompanying drawings, in which:
[0010] FIG. 1 depicts a block diagram of the all-in-one encoder of
the present invention;
[0011] FIG. 2 illustrates the method of the present invention used
to produce a fast forward bitstream; and
[0012] FIG. 3 illustrates the method of the present invention used
to produce a fast reverse bitstream.
[0013] To facilitate understanding, identical reference numerals
have been used, where possible, to designate identical elements
that are common to the figures.
DETAILED DESCRIPTION
[0014] FIG. 1 depicts a block diagram of an all-in-one encoder 100
comprising a first encoder 102, frame subsampler 104, a frame
buffer 106, a second encoder 108 and a controller 110. The first
encoder 102 encodes a source video sequence in a conventional
manner, i.e., compressing the source video sequence in real-time as
the frames are input to the encoder. The second encoder 108
operating in conjunction with the subsampler 104 and the buffer 106
encodes a subsampled version of the source video sequence to form a
fast forward and fast reverse bitstreams (collectively referred to
herein as trick play bitstreams or trick play streams). The first
encoder (the play stream encoder 102) contains a real-time MPEG-2
encoder that produces an MPEG-2 compliant, compressed video
bitstream (a play stream) from a sequence of 601-format video
frames. The second encoder (the trick play stream encoder 104) is
also an MPEG-2 real-time encoder 112.
[0015] For the following discussion, frame numbering is used to
describe the temporal order in which frames occur in source
material where 1 is the first frame in the source material frame
sequence and 2 is the second frame and so on. For the following
description, R is an integer and defines a play-back speed
multiplier which has significance in the trick play processes. The
speed multiplier R is a variable that is established by the
controller 110. The controller 110 is a microprocessor or
microcontroller as well as support circuits such as memory, clocks,
power supplies, etc. that is programmed to perform the control
functions of the all in one encoder 100.
[0016] Throughout this disclosure, the exemplary trick play streams
are fast forward and fast reverse. Of course other forms of trick
play streams may be generated using the all-in-one encoder 100 of
the present invention. If the multiplier R equals 2, playback
(decoding) of a fast forward stream is twice normal single speed.
If R equals 3, playback is three times normal play speed and so on.
Generally, the trick play streams include a nine times fast forward
stream and a nine times fast reverse stream. Alternatively, 32
times normal fast forward and fast reverse streams are also
available. The term source or source video is used to describe the
uncompressed video material such as 601-format video from which the
trick play bitstreams are generated.
[0017] Using the all-in-one encoder 100, a fast forward MPEG video
bitstream is generated from an uncompressed video source such that
when the stream is played back linearly through a standard MPEG
compliant video decoder, the resulting imagery contains every Rth
frame of the original video sequence. As such, the display of the
sequence has a fast forward effect.
[0018] A fast reverse stream is simultaneously produced by the
all-in-one encoder such that when linearly played back through a an
MPEG compliant video decoder, the decoded stream produces a
sequence of frames that play in reverse relative to the original
frame sequence.
[0019] The play bitstream is formed using a conventional
MPEG-compliant encoder 102 that compresses a plurality of frames (N
frames that form a group-of-pictures (GOP)). To form the trick play
streams having N frame GOPs, the subsampler 104 extracts one out of
every R consecutive frames (arranged in increasing time code order)
from the source sequence. The selected N frames are buffered in the
frame buffer 106. For example, if R is two and the buffer stores N
frames, the buffer stores frames 1, 3, 5, 7, 9, and so on up to
frame 2N+1. The buffer 106 stores N successive frames that define a
GOP for the trick play bitstreams.
[0020] To produce a fast forward stream, the N frames from the
buffer 106 are recalled under the control of the controller 110 and
coupled sequentially into the real-time encoder 108. The real-time
encoders 102 and 108 used herein may be any form of encoder that
compresses video information. One illustrative MPEG encoder is the
E4 chip set available from C-Cubed of Springfield, Va. The output
is a compressed bitstream representing a fast forward GOP.
[0021] To produce a fast reverse stream, the N frames from the
buffer 106 are recalled under the control of the controller 110 and
are coupled in reverse time order into the encoder 108. Although
two encoders could be used to produce each of the trick play
streams independently, as long as R equals two or more, a single
encoder can be used to produce both trick play tracks using time
multiplexing technique. To perform the multiplexed encoding, the
fast forward GOP is encoded first, then the fast reverse GOP. After
the buffer is filled with another N frames, the GOP pair is again
encoded and so on. The recall order of the frames from the buffer
and the multiplexing process are controlled by controller 110 by
addressing the buffer in forward order for the fast forward GOP and
in reverse order for the fast reverse GOP.
[0022] The forward and reverse GOPs contain sequence start and
sequence end codes such that, when stored in a mass storage device,
the bitstreams are clearly delimited.
[0023] The forward and reverse GOP bitstreams are stored with the
play stream in a storage medium such as a disk drive array or
magneto-optical disk. The fast forward stream is written
sequentially, one GOP bitstream at a time, from the lowest address
to the highest address. The reverse bitstream is stored in reverse
order from highest address to lowest address. The last byte in the
reverse GOP is placed in the highest address space of the reverse
target file. As such, the reverse GOP precedes, in address space,
the first frame of the immediately previously written reverse
GOP.
[0024] The entire source material frame sequence is encoded in this
manner to simultaneously produce a play, a fast forward, and fast
reverse streams in real-time.
[0025] FIG. 2 illustrates the process by which a fast forward
stream is produced. Sequence 200 represents the RN frames (frames 1
to RN+1) that have been subsampled from a video frame sequence at a
one of R rate from a video frame sequence and stored in the buffer
(e.g., R=2, then 2N frames are subsampled to form a trick play
GOP). These buffered frames are recalled from memory and encoded
and then associated with a normal play stream having a GOP of N
frames. The encoder (at step 202) repetitively produces a sequence
of GOPs 204 that are stored in increasing addresses in memory as
illustrated in sequence 206. The first GOP (GOP 0) contains frames
1 to RN+1, the second GOP (GOP 1) contains frames R(N+1)+1 through
2RN+1, the third GOP (GOP 2) contains frames 2R(N+1)+1 through
3RN+1, and so on. To generalize, a GOP contains frames GR(N+1)+1
through (G+1)RN+1, where G is the GOP number (e.g., 0, 1, 2, 3, . .
. ), R is the subsampling rate, and N is the number of frames in a
standard play GOP.
[0026] FIG. 3 illustrates the process by which a fast reverse
stream is produced. Sequence 300 represents the RN frames that have
been buffered after subsampling at one of R rate (e.g., R=2 in FIG.
3). At step 302, these frames are selected from the buffer in
reverse order as illustrated at 304. The reverse order frames are
encoded, at step 306, to produce a sequence of GOPs 308 that are
stored in reverse time order in memory as illustrated in sequence
310.
[0027] Although various embodiments which incorporate the teachings
of the present invention have been shown and described in detail
herein, those skilled in the art can readily devise many other
varied embodiments that still incorporate these teachings.
* * * * *