U.S. patent application number 13/948268 was filed with the patent office on 2014-01-23 for online video distribution.
This patent application is currently assigned to WireWax Limited. The applicant listed for this patent is WireWax Limited. Invention is credited to Steven James Callanan.
Application Number | 20140026175 13/948268 |
Document ID | / |
Family ID | 46881780 |
Filed Date | 2014-01-23 |
United States Patent
Application |
20140026175 |
Kind Code |
A1 |
Callanan; Steven James |
January 23, 2014 |
ONLINE VIDEO DISTRIBUTION
Abstract
Online video can now be supplied with hyperlinks that are
associated with and move with objects in the video. However, online
video players tend not to report an accurate frame number, and this
results in poor synchronisation of the movement of the hyperlink
and the object. We propose that a subset of the pixels in the frame
be used to encode a numeric frame identifier.
Inventors: |
Callanan; Steven James;
(London, GB) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
WireWax Limited |
London |
|
GB |
|
|
Assignee: |
WireWax Limited
London
GB
|
Family ID: |
46881780 |
Appl. No.: |
13/948268 |
Filed: |
July 23, 2013 |
Current U.S.
Class: |
725/112 |
Current CPC
Class: |
G11B 27/036 20130101;
H04N 21/4725 20130101; H04N 21/8586 20130101; G06F 16/748 20190101;
G11B 27/34 20130101; G11B 27/3072 20130101; H04N 21/4341 20130101;
H04N 7/025 20130101; H04N 21/8583 20130101; H04N 7/08 20130101 |
Class at
Publication: |
725/112 |
International
Class: |
H04N 21/858 20060101
H04N021/858 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 23, 2012 |
GB |
GB1213015.9 |
Claims
1. A video data file comprising; a viewable video sequence
comprising a plurality of video frames, associated hyperlink data
comprising at least i. link address information, and ii. position
data defining a plurality of positions within a plurality of
respective video frames, wherein at least a plurality of the video
frames include a unique frame number indicia encoded as pixel data
within the video frame.
2. A video data file according to claim 1, wherein the hyperlink
data and the viewable video sequence are delivered as separate
streams and the hyperlink data includes a frame number associated
with each of the defined positions.
3. A video data file according to claim 1, wherein the unique frame
number indicia is encoded in a plurality of pixels in the video
frame.
4. A video data file according to claim 3, wherein the plurality of
pixels comprise a group of adjacent pixels defining a rectangular
area.
5. A video data file according to claim 4, wherein the rectangular
area is located at a corner of the video frame.
6. A video data file according to claim 3, wherein the plurality of
pixels comprise a group of adjacent pixels arranged in a linear
array.
7. A video data file according to claim 6, wherein the linear array
is located at an edge of the video frame.
8. A video data file according to claim 1, wherein the position
data includes a location relative to the video frame.
9. A video data file according to claim 8, wherein the position
data includes size information relating to the hyperlink data.
10. A video data file according to claim 1, wherein the unique
frame number indicia is included in substantially all the
frames.
11. A video data file according to claim 1, wherein the plurality
of pixels are located in a non-viewable area of the video
frame.
12. A video player for online video, including: a video decoder for
interpreting and displaying a received viewable video stream
comprising a sequence of video frames, a hyperlink decoder
configured to receive and display over the displayed video stream
hyperlink data comprising link address information and position
data defining a plurality of positions within a plurality of
respective video frames, wherein the player is further arranged to
inspect a subset of the pixels of a video frame to be displayed
time to determine a unique frame number indicia encoded in the
subset and use the thus determined frame number to select a
hyperlink for display over the frame.
13. A system for encoding online video arranged to: receive a
viewable video stream comprising a sequence of video frames,
receive hyperlink data comprising link address information and
position data defining a plurality of positions within a plurality
of respective video frames, encode a unique frame number indicia
into at least the plurality of respective video frames by writing
non-image data to a subset of the pixels of the video frame, and
combine the encoded video stream and the hyperlink data for
download.
14. A system for encoding online video according to claim 13,
wherein the encoded video stream and the hyperlink data are
combined to form a single downloadable stream.
15. A system for encoding online video according to claim 13,
wherein the encoded video stream and the hyperlink data are
combined to form at least two correlated downloadable streams.
16. A video data format comprising a data stream including;
viewable video comprising a sequence of video frames, associated
hyperlink data comprising at least i. link address information, and
ii. position data defining a plurality of positions within a
plurality of respective video frames, wherein at least a plurality
of the video frames include a unique frame number indicia encoded
as pixel data within the video frame.
Description
[0001] This application claims priority from GB Patent Application
1213015.9, filed Jul. 23, 2012, which is incorporated herein by
reference.
FIELD OF THE INVENTION
[0002] The present invention relates to online video
distribution.
BACKGROUND ART
[0003] Online video is now an established communication tool, being
used for social purposes and for commercial marketing and other
purposes. Indeed, the success of online video is such that
content-enabled video services are being developed, allowing
functionality to be co-delivered with the video such as clickable
hyperlinks within the video frame, such as external links that lead
the viewer to a specific website or other internet address, or
popouts to display text or images, or play video or audio alongside
or instead of the video previously being played.
[0004] U.S. Pat. No. 7,620,914 discloses the incorporation of
clickable hyperlinks into a viewable video, sending the hyperlink
data in a separate stream alongside the video data. The two are
then re-united in the video player and the hyperlinks are displayed
over the video data, allowing a user to click the hyperlink as and
when desired.
[0005] U.S. Pat. No. 7,817,822 discloses a motion tracking system
for such hyperlinks. This allows a user to associate the hyperlink
with a specific feature in a frame of the video, following which
the motion tracking system detects movement of that feature in
subsequent frames and adjusts the position of the hyperlink
accordingly. As a result, the hyperlink appears to "float" over the
feature and follow it as it moves in the video. The hyperlink can
therefore be associated with the feature, such as a link offering
more information on or an opportunity to purchase the item forming
that feature. Multiple hyperlinks can then be safely added to a
video or video segment, with the meaning of each hyperlink
remaining clear to a viewer at all times.
SUMMARY OF THE INVENTION
[0006] As U.S. Pat. No. 7,620,914 sends the hyperlink and the video
data in separate streams, when the two streams are united a
correlation needs to be made between the two so that they can be
properly synchronised. If this cannot be done then a system based
on U.S. Pat. No. 7,620,914 will not be able to display the video in
correct synchronisation.
[0007] Most systems use the frame number of the video as a time
code to identify current playback point of the video and hence
identify the hyperlinks and other objects that should be displayed.
However, we have found that most online video players do not in
fact provide an accurate record of the frame number and thus the
returned frame number information is likewise inaccurate. This
inaccuracy is typically of the order of a few frames, such as up to
about 5 frames, and is insignificant for most online purposes. It
is significant in this context, however, and can lead to a distinct
"lag" in the movement of hyperlinks over the displayed video.
[0008] We therefore propose a video data format comprising viewable
video, associated hyperlink data comprising both link data and
position data (which may include a location within or relative to
the frame, and a size of any bounding box or other object) defining
a position within the video frame that may change from frame to
frame, and frame count information encoded as pixel data within the
video stream. This enables the player to determine precisely where
to position the hyperlink or tag, to ensure that the display of
hyperlinks is correlated correctly with the video playback.
[0009] Thus, in a first aspect the present invention provides a
video data file comprising a viewable video sequence comprising a
plurality of frames, associated hyperlink data comprising at least
link address information, and position data defining a plurality of
positions within a plurality of respective video frames, wherein at
least a plurality of the frames include a unique frame number
indicia encoded as pixel data within the frame.
[0010] The hyperlink data and the viewable video can be delivered
as separate streams, in which case the hyperlink data can include a
frame number associated with each of the defined position data
sets.
[0011] We prefer that the unique frame number is encoded in a
plurality of pixels in the video frame. These can be a group of
adjacent pixels, such as a rectangular area which can be located at
a corner of the video frame, for example. Alternatively, a group of
adjacent pixels arranged in a linear array can be used, such as a
group located at an edge of the video frame.
[0012] The unique frame number indicia is ideally included in
substantially all the frames of the viewable video. They can, if
desired, be located in a non-viewable area of the video frame such
as an area concealed by a menu item or icon.
[0013] In second aspect, the present invention provides a video
player for online video, including a video decoder for interpreting
and displaying a received viewable video stream comprising a
sequence of frames, a hyperlink decoder able to receive and display
over the displayed video stream hyperlink data comprising link
address information and position data defining a plurality of
positions within a plurality of respective video frames, wherein
the player is further arranged to inspect a subset of the pixels of
a frame to be displayed time to determine a unique frame number
indicia encoded in the subset and use the thus determined frame
number to select a hyperlink for display over the frame.
[0014] In a third aspect, the present invention provides a system
for encoding online video arranged to receive a viewable video
stream comprising a sequence of frames, receive hyperlink data
comprising link address information and position data defining a
plurality of positions within a plurality of respective video
frames, encode a unique frame number indicia into at least the
plurality of respective video frames by writing non-image data to a
subset of the pixels of the frame, and combine the thus encoded
video stream and the hyperlink data for download.
[0015] The thus encoded video stream and the hyperlink data can be
combined to form a single downloadable stream, or sent as two
correlated downloadable streams.
[0016] Finally, in its fourth aspect the present invention provides
a video data format comprising a data stream including viewable
video comprising a sequence of frames, associated hyperlink data
comprising at least link address information, and position data
defining a plurality of positions within a plurality of respective
video frames, wherein at least a plurality of the frames include a
unique frame number indicia encoded as pixel data within the
frame.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] An embodiment of the present invention will now be described
by way of example, with reference to the accompanying figures in
which;
[0018] FIG. 1 shows an example of an online player window according
to the present invention;
[0019] FIG. 2 shows a group of pixels encoding a frame number;
and
[0020] FIG. 3 shows the manner in which the frame number is encoded
into the pixels of FIG. 2.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0021] Referring to FIG. 1, a video player window 10 comprises a
video display 12 above a timeline 14 showing the progress of the
current playback position through the length of the video. Within
the video display, a video is currently running and depicts a
vehicle 16 on a beach 18.
[0022] A highlight area 20 is shown over the vehicle. This denotes
a clickable hyperlink, which (as discussed above) may result in any
of a variety of actions taking place when a user clicks within the
highlight area. For example, there could be a pop-up box or window
giving further details of the type of car and further information
as to its provenance, or further photographs of the car, or one or
more links to sites where cars of that type can be purchased.
[0023] The video display 12 also includes a number of operating
icons. For example, a "full screen" icon 22 is shown adjacent the
timeline 14 to allow a user to expand the video display to occupy
the entirety of the screen of whatever device is being used for
playback. After a preset period of inactivity, the timeline is made
to disappear automatically, returning if the mouse pointer or other
GUI input becomes active again. FIG. 1 also shows operating icons
24 which help the viewer navigate the tags. In addition, there is a
flyout icon 26 in the upper right corner of the video display 12
which allows a user access to a wider range of control functions;
clicking the flyout icon 26 causes additional icons for these
control functions to be displayed. If present permanently, these
icons could be distracting and would obscure part of the video, so
they are normally hidden and only the flyout icon 26 is displayed
permanently. This does obscure a part of the video, but only a
small part and only in one corner; it is unusual for a corner
location to contain crucial information.
[0024] The part of the video stream that lies underneath this
flyout icon 26 is therefore never seen. According to the present
invention, the pixels beneath this icon are employed to store a
time code instead of video data. An image of a typical time code is
shown in FIG. 2, and is in the form of a 6.times.5 rectangular
block 28 of pixels 30. Of this block, a 5.times.5 square group of
pixels 32 are allowed to adopt an on (white) or off (black) state,
and can thus encode a binary number according to the pattern shown
in FIG. 3. This shows a least-significant-bit (LSB) on the bottom
right hand corner of the rectangle with the bit values increasing
to the left and upwards. This yields a single number encoded by the
pixel values and which can be unique to each frame of a video.
Thus, it can be used to indicate a time associated with the frame,
or a frame number. Thus uniquely identifies the frame concerned
without ambiguity or approximation, and can therefore be used as a
reference by which the correct hyperlink placement is
identified.
[0025] In addition, a vertical strip of pixels 34 to the left of
the 5.times.5 group 32 appear white, black, white, black, white
from top to bottom. These pixels retain this pattern throughout the
video and indicate to the player that the pixel time code exists.
The chances of a chequered column of that arrangement appearing as
part of the actual video frame is very unlikely, so the player
first looks for this pattern in order to confirm that a pixel time
code is present. Of course, a different pattern could be adopted,
for example using a different pattern of light and dark pixels
and/or a different size or shape; all that necessary for such a
marker pattern is that it be known in advance and unlikely to
appear in the actual video by chance.
[0026] If desired, the individual squares 32 which form the bit
values of the time code could each be defined by a plurality of
pixels in the video itself, rather than as individual pixels. In
other words, the 6.times.5 and/or the 5.times.5 groups of squares
could cover substantially more than a 6.times.5 block of pixels in
the raw video data. This and the use of on/off black/white encoding
could help the frame data to be resilient to typical video
compression algorithms; while these algorithms may distort the
squares slightly, they are unlikely to turn a square completely
black (or white) when it was previously white (or black). FIG. 2
shows some squares such as square 32 which are slightly paler than
others, but which are clearly intended to be black and are above a
mid-grey threshold and can therefore be interpreted as black by a
decoding algorithm.
[0027] A 5.times.5 block of pixels allows for 25 bits of data. This
allows a unique frame number of up to about 3.times.10.sup.7 which
is more than adequate for even the longest video at the highest
frame rate, corresponding to about 186 hours of video at 50 frames
per second. If desired, a smaller block of squares could be used,
which would take up less space and could be concealed by a smaller
icon. Alternatively, a non-rectangular block could be used, such as
a linear strip of squares which could be aligned along an edge of
the video such as the top, bottom, or sides of the video. These
could be cropped prior to display and hence remain invisible to the
viewer.
[0028] Alternatively, a larger block of squares could be defined,
providing more than 5.times.5 squares and hence allowing more data
to be stored. This will probably be unnecessary for frame
identification but would allow other data to be encoded such as
(perhaps) the hyperlink data. This latter step might require the
bit depth to be increased by allowing colour variation of the
squares, but care would need to be taken to ensure this data
survived video compression techniques.
[0029] This frame number reference can be incorporated into the
video file during compression or other processing. The algorithm
performing the processing can maintain a simple frame count and can
be programmed to convert this into the appropriate pattern of
squares. These can then be written onto the frame over whatever
pixel values were there originally to create the necessary video
data file which can then be stored together with (or in combination
with) the hyperlink data.
[0030] When the video data is received for display, the code can be
retrieved by the video player by interpreting the pixel data
appropriately, returning a reliable frame number and allowing
proper synchronisation of the hyperlink information.
[0031] It will of course be understood that many variations may be
made to the above-described embodiment without departing from the
scope of the present invention.
* * * * *