U.S. patent application number 12/093453 was filed with the patent office on 2008-10-23 for method and apparatus for automatic detection and identification of unidentified broadcast audio or video signals.
This patent application is currently assigned to MEDIAGUIDE, INC.. Invention is credited to Kwan Cheung.
Application Number | 20080263041 12/093453 |
Document ID | / |
Family ID | 38040386 |
Filed Date | 2008-10-23 |
United States Patent
Application |
20080263041 |
Kind Code |
A1 |
Cheung; Kwan |
October 23, 2008 |
Method and Apparatus for Automatic Detection and Identification of
Unidentified Broadcast Audio or Video Signals
Abstract
A system and method of detecting unidentified broadcast
electronic media content using a self-similarity technique is
presented. The process and system catalogues repeated instances of
content that has not be positively identified, but are sufficiently
similar as to infer repetitive broadcasts. These catalogued
instances may be further processed on the basis of different
broadcast channels, sources, geographic locations of broadcasts or
format to further assist the identification thereof.
Inventors: |
Cheung; Kwan; (Bryn Mawr,
PA) |
Correspondence
Address: |
TED SABETY, c/o Sabety +associates, PLLC
1130 Bedford Rd.
PLEASANTVILLE
NY
10570
US
|
Assignee: |
MEDIAGUIDE, INC.
Berwyn
PA
|
Family ID: |
38040386 |
Appl. No.: |
12/093453 |
Filed: |
November 14, 2006 |
PCT Filed: |
November 14, 2006 |
PCT NO: |
PCT/US06/60891 |
371 Date: |
May 13, 2008 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60736348 |
Nov 14, 2005 |
|
|
|
Current U.S.
Class: |
1/1 ;
707/999.006; 707/E17.014; 707/E17.027; 707/E17.101 |
Current CPC
Class: |
H04H 20/14 20130101;
G06F 16/683 20190101; G06K 9/0055 20130101; G06F 16/634 20190101;
H04H 60/58 20130101 |
Class at
Publication: |
707/6 ;
707/E17.014; 707/E17.027; 707/E17.101 |
International
Class: |
G06F 17/30 20060101
G06F017/30 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 30, 2005 |
US |
11322706 |
Claims
1. A method executed by a digital signal processing system for
detecting at least two substantially similar instances of the same
piece of unregistered program in one or more broadcast signals
comprising: periodically detecting whether two or more unregistered
portions of said one or more broadcast signals is sufficiently
self-similar.
2. The method of claim 1 further comprising detecting one or more
sufficiently similar sets of pattern vectors, wherein each set
corresponds to one of the one or more unregistered portions.
3. The method of claim 2 further comprising assigning an
identification number to one or more of the sets of pattern vectors
detected.
4. The method of claim 1 further comprising determining whether the
piece of unregistered program is substantially the same as a
registered piece of programming and then changing an indicia of
identity associated with the unregistered program to an indicia of
identity associated with the registered program.
5. The method of claim 4 further comprising converting an
identification number assigned to the piece of unregistered program
to an identification number corresponding to the registered piece
of programming.
6. The method of claim 1 further comprising: identifying a piece of
unregistered programming in which sufficient similarities have been
detected between a piece of registered programming and said piece
of unregistered programming; and converting the status of said
identified piece of unregistered programming from unregistered to
registered.
7. The method of claim 1, wherein the two or more unregistered
portions of one of said one or more broadcast signals are
substantially non-overlapping in time.
8. The method of claim 1, wherein each of said unregistered
portions of said one or more broadcast signals comprises one or
more frames in duration.
9. The method of claim 8, wherein the duration is approximately 10
seconds.
10. The method of claim 8, wherein the duration is less than
approximately 10 seconds.
11. The method of claim 2 further comprising storing in computer
memory at least one of said sets of pattern vectors.
12. The method of claim 2 further comprising comparing the audio
quality of said sets of pattern vectors.
13. The method of claim 12 further comprising storing in computer
memory a set of pattern vectors determined to have relatively
better audio quality than other sufficiently similar sets of
pattern vectors.
14. The method of claim 1 further comprising storing in computer
memory an indicia of identity associated with the unregistered
program.
15. The method of claim 1 further comprising storing in computer
memory one of: an approximate start time, an approximate end time,
and an approximate duration time of said two or more of the
self-similar unregistered portions.
16. The method of claim 1 further comprising storing in computer
memory the broadcast source information of the one or more
broadcast signals over which said two or more unregistered portions
are broadcast.
17. The method of claim 1 further comprising adjusting the duration
of said unregistered portions for decreasing the detection time of
self-similarities in the one or more broadcast signals.
18. A method executed by a digital signal processing system for
detecting self-similar repetition in a broadcast signal comprising:
detecting a thread comprising a sequence of one or more portions of
a first unregistered piece of programming, where each consecutive
portion of the first unregistered piece is determined to be
sufficiently similar to a corresponding substantially consecutive
portion of a second unregistered piece of unregistered
programming.
19. The method of claim 18 further comprising: storing in computer
memory as a single piece of programming the consecutive sequence of
sufficiently similar portions where for each portion, the audio
quality is determined to be better than the other corresponding
sufficiently similar consecutive portion.
20. The method of claim 18, wherein the step of determining to be
sufficiently self-similar comprises detecting sufficiently similar
sets of pattern vectors associated with each of the consecutive
portions.
21. The method of claim 18 further comprising assigning a unique
identification number to one or more sets of pattern vectors that
comprise the portions.
22. The method of claim 18 further comprising: determining the end
of the thread by detecting one or more insufficiently similar
portions of the first piece as compared to the corresponding
portions of the second piece.
23. The method of claim 18 where the first piece and the second
piece do not substantially overlap.
24. The method of claim 22, wherein the number of detections of
insufficient similarity is a number greater than a predetermined
threshold.
25. The method of claim 22 further comprising determining one of an
approximate start time, approximate end time and approximate
duration time corresponding to the thread.
26. The method of claim 18 further comprising: identifying a piece
of unregistered programming in which sufficient similarities have
been detected between a piece of registered programming and said
piece of unregistered programming; and converting the status of
said identified piece of unregistered programming from unregistered
to registered.
27. The method of claim 18 further comprising determining whether
the piece of unregistered program is substantially the same as a
registered piece of programming and then changing an indicia of
identity associated with the unregistered program to an indicia of
identity associated with the registered program.
28. The method of claim 18, wherein each of said portions comprises
one or more frames in duration.
29. The method of claim 18 where each of said portions is equal to
or less than approximately 10 seconds.
30. A method executed by a digital signal processing system for
detecting self-similar repetition in a broadcast signal comprising:
detecting insufficient self-similarities between two
non-overlapping pieces of unregistered programming of a broadcast
signal by determining the number of pairs of insufficiently
self-similar corresponding portions from each of the two pieces of
unregistered programming, wherein said pairs of insufficiently
self-similar corresponding portions substantially succeed a
substantial number of substantially consecutive pairs of
sufficiently self-similar corresponding portions.
31. The method of claim 30 further comprising detecting
insufficiently similar sets of pattern vectors associated with each
of the two unregistered corresponding portions of the broadcast
signal.
32. The method of claim 30 further comprising assigning a unique
identification number to one or more sets of pattern vectors that
comprise each of the corresponding portions.
33. The method of claim 30 further comprising: accumulating a
thread of portions determined to be sufficiently self-similar,
wherein each accumulated portion is uniquely selected from each of
said pairs of sufficient self-similar corresponding portions; and
accumulating a thread of said selected portions, wherein the
portions are ordered sequentially in time according to the relative
order of the pair from which the portion was selected.
34. The method of claim 30 further comprising determining one of an
approximate start time, approximate end time and approximate
duration time corresponding to the thread.
36. The method of claim 33, wherein the portion selected is
selected for having at least relatively better quality.
37. The method of claim 30, wherein said sufficient number
comprises a number greater than a predetermined threshold.
38. The method of claim 30 further comprising: identifying a piece
of unregistered programming in which sufficient similarities have
been detected between a piece of registered programming and said
piece of unregistered programming; and converting the status of
said identified piece of unregistered programming from unregistered
to registered.
39. The method of claim 30 further comprising determining whether
the piece of unregistered program is substantially the same as a
registered piece of programming and then changing an indicia of
identity associated with the unregistered program to an indicia of
identity associated with the registered program.
40. The method of claim 30, wherein each of said portions comprises
one or more frames in duration.
41. A digital signal processing system for detecting at least two
substantially similar instances of the same piece of unregistered
program in one or more broadcast signals comprising: a detection
means adapted for periodically detecting whether two or more
unregistered portions of said one or more broadcast signals is
sufficiently self-similar.
42. The system of claim 42, wherein said detection means is further
adapted for detecting one or more sufficiently similar sets of
pattern vectors, wherein each set corresponds to one of the one or
more unregistered portions.
43. The system of claim 42 further comprising: a processor adapted
for assigning an identification number to one or more of the sets
of pattern vectors detected.
44. The system of claim 41 further comprising: a processor adapted
for determining whether the piece of unregistered program is
substantially the same as a registered piece of programming and
then changing an indicia of identity associated with the
unregistered program to an indicia of identity associated with the
registered program.
45. The system of claim 44, wherein said processor is further
adapted for converting an identification number assigned to the
piece of unregistered program to an identification number
corresponding to the registered piece of programming.
46. The system of claim 41 further comprising: a processor adapted
for identifying a piece of unregistered programming in which
sufficient similarities have been detected between a piece of
registered programming and said piece of unregistered programming;
and converting the status of said identified piece of unregistered
programming from unregistered to registered.
47. The system of claim 41, wherein the two or more unregistered
portions of one of said one or more broadcast signals are
substantially non-overlapping in time.
48. The system of claim 41, wherein each of said unregistered
portions of said one or more broadcast signals comprises one or
more frames in duration.
49. The system of claim 48, wherein the duration is approximately
10 seconds.
50. The system of claim 48, wherein the duration is less than
approximately 10 seconds.
51. The system of claim 42 further comprising a computer memory
adapted for storing at least one of said sets of pattern
vectors.
52. A digital signal processing system for detecting self-similar
repetition in a broadcast signal comprising: a detection means
adapted for detecting a thread comprising a sequence of one or more
portions of a first unregistered piece of programming, where each
consecutive portion of the first unregistered piece is determined
to be sufficiently similar to a corresponding substantially
consecutive portion in a second unregistered piece of unregistered
programming.
53. The system of claim 52 further comprising: a processor adapted
for identifying a piece of unregistered programming in which
sufficient similarities have been detected between a piece of
registered programming and said piece of unregistered programming;
and converting the status of said identified piece of unregistered
programming from unregistered to registered.
54. The system of claim 52 further comprising: a processor adapted
for determining whether the piece of unregistered program is
substantially the same as a registered piece of programming and
then changing an indicia of identity associated with the
unregistered program to an indicia of identity associated with the
registered program.
55. A digital signal processing system for detecting self-similar
repetition in a broadcast signal comprising: detection means
adapted for detecting insufficient self-similarities between two
non-overlapping pieces of unregistered programming of a broadcast
signal by determining the number of pairs of insufficiently
self-similar corresponding portions from each of the two pieces of
unregistered programming, wherein said pairs of insufficiently
self-similar corresponding portions substantially succeed a
substantial number of substantially consecutive pairs of
sufficiently self-similar corresponding portions.
56. The system of claim 55, wherein the detection means is further
adapted for detecting insufficiently similar sets of pattern
vectors associated with each of the two unregistered portions of
the broadcast signal.
57. The system of claim 55 further comprising a processor adapted
for assigning a unique identification number to one or more sets of
pattern vectors that comprise the portions.
58. The system of claim 55 further comprising a processor adapted
for selecting a portion from each of said pairs of sufficient
self-similar portions; and accumulating a thread of said selected
portions, wherein the portions are ordered sequentially in time
according to the relative order of the pair from which the portion
was selected.
59. The system of claim 55, wherein the processor is further
adapted for determining one of an approximate start time,
approximate end time and approximate duration time corresponding to
the thread.
60. The method of claim 1 further comprising rendering to a human
listener the one or more self-similar unregistered portions.
Description
[0001] This application claims priority to PCT/US05/04802, filed on
Feb. 16, 2004, and 60/736,348, filed on Nov. 15, 2005, both of
which are incorporated herein by reference.
BACKGROUND AND SUMMARY OF THE INVENTION
[0002] The present invention relates to a method of detecting and
tracking unknown broadcast content items that are periodically
encountered by automatic detection and tracking systems. It is
known in the art that detection of broadcast content, for example,
music broadcast over radio, includes the sampling the of the
identified content to compute numerical representations of features
of the content, sometimes referred to in the art as a fingerprint,
or in the related patent application PCT/US05/04802, filed on Feb.
16, 2004, which is incorporated herein by reference, a pattern
vector. These known pattern vectors are stored in a database and
while the broadcast signals are received, the same computation is
applied to the incoming signal. Then, the detection process entails
searching for matches between the incoming computed pattern vectors
and the vast database of pre-created pattern vectors associated
with the identity of known content.
[0003] This system runs into problems when content that has not
been registered in the database yet and is being broadcast anyway.
In the prior art, these unknown or unmatched programming items
would be ignored. This invention is directed to address this
shortcoming by determining when a likely piece of programming
content has been detected, tracking such detections, and then
submitting the piece for human identification in order that proper
publishing or other indicia of identity be associated with the
content. The system automatically determines which portions of the
broadcast signal are previously un-registered content.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 is a schematic of a Radio Monitoring System.
[0005] FIG. 2 is an Illustration of a repetition of the same
program along the time-axis.
[0006] FIG. 3 is a schematic of how the invention is operated
[0007] FIG. 4 is the workflow of exercising the first grouping and
the second grouping
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
1. Introduction
[0008] Short Coming of a Radio Monitoring System The core of a
modern radio monitoring system (see FIG. 1) is a detection system
consisting of a detection algorithm and a database. The database,
referred to as the detection database, is populated with
identification information of the programs to be detected, e.g.
songs and commercial ads. The identification information includes
the fingerprint of the program (in this document, the term
"pattern" or "pattern vector" will be used instead of
"fingerprint"). Signals received from an electronic media
broadcast, for example, a radio broadcast are processed to extract
patterns at regular intervals in time. Patterns from broadcast are
compared with patterns registered in the detection database. A
detection is made if a program registered in the database matched
with that of the broadcast. Matching includes more than an
identical match, but also close matches and a series of matches
over time that are determined to be sufficiently consistent with a
specific program. The short coming of the system is that detection
result is determined by what is being stored in the database.
Clearly, if a particular song is not registered in the detection
database, the song will never be detected.
[0009] A Semi-Automatic Solution. It is a well-known fact that
certain programs, particularly songs and commercial ads, are
broadcast repeatedly. This repetition provides the opportunity for
a detection algorithm to detect programs that have been repeated,
even if they are not registered in the detection database. An
important principle being employed to design the detection
algorithm on repetitive programs is the self-similarity principle.
If a program is broadcast at time T1 and is repeated at T2,
T2>T1. Programs located along the time-axis at T1 and T2 are
said identical.
[0010] To detect if a program, which was located at T1, is repeated
elsewhere, we can cut out the content located at T1, and run an
identification exercise to identify along the time-axis when this
piece of content is repeated. We will then identify that program
located at T1 if it is repeated at T2. The invention is directed to
exercise the self-similarity principle in order to detect
unregistered programs that have been repeated. The invention is
designed to bring in all programs that have at least one repetition
within a pre-determined period of time. The invention can generate
results autonomously while other matching and identification
processes are running. However, the actual identity of the content
of each of the results is not known until it has been listen by
human operators. Clearly, listening process is a manual process.
Alternatively, the harvested results can be converted to registered
content if it turns out that a piece of content is registered whose
pattern vectors sufficiently match those associated with content
only identified as a unique piece of content as a result of the
operation of the invention. In other words, as content is
registered in the database, it can be checked to see if it has
already been detected by the invention, and the database of
detections updated accordingly.
[0011] If the invention outputs a identifiable piece of content,
referred to herein as a "harvest", such as a new release of a song
not actually registered in the database, the pattern vectors of
this song that have been recovered by the invention can be
extracted from one of the corresponding broadcast signal clips and
registered into the detection database of the media monitoring
system. By definition, each of the harvests has at least two copies
because at a minimum, self-similarity between to instances in time
is necessary to determine the repetition that indicates the
presence of an individually identifiable piece of content.
Selecting which repetition should be registered into the detection
database is determined by the audio quality. The invention also
includes a step called Audio-Selector, which selects from all the
repetitions of the harvested content, the one that has the best
audio quality. The pattern vectors computed from this repetition is
registered in the database, typically with a unique identification
number that is used in lieu of an actual title, because the title
is not known yet.
[0012] When coupling the invention with an existing media
monitoring system, which has detected and identified certain
programs along the time, the invention can be set to identify
similar programs only during time periods where the monitoring
system has not otherwise detected any registered programming. In
this mode, the invention is said to be "harvesting on undetected
time. Another other mode is to harvest on "all time", including
during time periods where registered content has been identified.
This approach is seldom used as it is redundant.
[0013] The invention can also be exercised to two other ways, one
called "batch-mode" and the other "time-progressive" mode. The
latter is employed by the preferred embodiment due to more
economical use of computer memory and CPU processing time. The
detail of time-progressive mode is further described below. In the
batch mode, the invention is exercised periodically on a large
memory of stored pattern vectors computed from incoming broadcast
signals. Every time when it is exercised, the invention will update
the harvest with new harvests generated since the last time the
invention was operated. In the time-progressive mode, the invention
can be exercised at any time at will. The invention is operated to
continually seek self-similar repetition among unidentified periods
of time in a signal. As harvests are created, the harvested content
is then not considered unidentified for these purposes the next
time the content is encountered. In addition, the time progressive
mode can be used so that the harvests from every media source can
be compared to identify content that is self-similar as compared to
instances from other geographic regions or distinct media broadcast
sources.
[0014] Operational Principle of Harvester. The starting point of
the invention is to exercise self-similarity detection on
individual channels of the incoming broadcast signal. An undetected
program is harvested if it has been repeated at least once over a
prescribed period TP, characterized with the start time, TS, and
the end time, TE. Within TP, the invention will detect programs
that are similar to each other. Each harvest is a unique program
that occurred at least twice in TP. Each repetition is given an
index with an "instance id". Each harvest, indexed with a "family
id", has a collection of all the corresponding instance id's.
[0015] Self-Similar Detection. The self-similar detection algorithm
is used on each individual channel to detect programs that have
been repeated more than once within TP. The example presented here
is for audio from a radio station, but the invention would work
equally well on audio from internet, satellite or any other
broadcast medium, or for other media types, including audio-visual
works like television programming. Given a piece of audio recording
of a radio station, the recording was started at time TS and ended
at time TE. A clip referred to as a "bait" is selected from the
recording and used as a reference to be matched against with the
entire, or a selected portion of the recording. Each clip that has
a match to the bait (subject to certain matching criteria) is
referred to as a "catch". The timing information of both the bait
and each of the catches will be registered as an instance. The
timing location information of every (bait, catch) pair is given an
instance id.
[0016] The self-similarity detection is exercised iteratively: The
first bait clip is selected right from the start of the recording.
If there is a catch, the second bait clip is selected right after
the end of the previous bait clip. Otherwise, the next bait clip is
started with a small time offset .tau.>0 from start of the
previous one. The iteration is run until the end of the recording
is reached. The detection is based on the similarity of pattern
vectors.
[0017] Calculation of pattern vectors is describe below, and in
further detail in the sister patent application PCT/US05/04802,
filed on Feb. 16, 2004, incorporated herein by reference. [0018] In
the preferred embodiment, the audio sampling rate is set at 8,000
Hz. These samples are organized into time frames of about 16,384
samples (a frame has a duration of about 2 seconds). Below is the
procedure to generate a pattern vector. [0019] Given a frame of
16,384 signal samples:
[0019] =[x[0]x[1].LAMBDA.x[16384]] [0020] Take the
Fast-Fourier-Transform and obtain 16,384 complex FFT
coefficients:
[0020] =[X[0]X[1].LAMBDA.X[16384]]
[0021] Partition the 16,384 FFT coefficients into 25 subbands,
These 25 subbands are a subset of the original 31 subbands
originally used for the monitoring system described in the sister
patent application PCT/US05/04802. The following are the indices of
FFT's in each subband: Subband #1: 130 to 182; Subband #2: 183 to
255; subband #3: 256 to 357; subband #4: 358 to 501; subband #5:
502 to 702; subband #6: 703 to 984; subband #7: 985 to 1,378;
subband #8: 1,379 to 1,930; subband #9: 1,931 to 2,702; subband
#10: 2,703 to 3,784; subband #11: 3,785 to 5,298; subband #12: 130
to 255; subband #13: 256 to 501; subband #14: 157 to 219; subband
#15: 220 to 306; subband #16: 307 to 429; subband #17: 430 to 602;
subband #18: 603 to 843; subband #19: 844 to 1,181; subband
#20:1,182 to 1,654; subband #21: 1,655 to 2,316; subband #22: 2,317
to 3,243; subband #23: 3,244 to 4,541; subband #24: 157 to 306;
subband #25: 307 to 602.
[0022] The 25 sub-bands cover the frequencies from 63 Hz to 2,587
Hz.
[0023] Let Nk=number of elements in the k-th subband, k=1 to 25.
Also let Mk={mk[1], mk[2], . . . , mk[Nk]} be the set containing
the corresponding indices for the k-th band.
[0024] The first order moment, or "centroid", of each subband is
then computed. Each centroid lies in the open interval (0,1). Below
is the formula:
centroid of the k - th subband = c k = n = 1 Nk n * X [ mk [ n ] ]
n = 1 Nk X [ mk [ n ] ] ; k = 1 to 25 ##EQU00001##
[0025] The pattern vector for each frame is then==[c.sub.1 c.sub.2
.LAMBDA. c.sub.25].
Extracting Pattern Vectors Using 4 Times (4.times.) Pattern Vector
Sampling
[0026] Let the period TP, characterized by the start time T.sub.S
and end time T.sub.E of the period, contains N samples: x[1] to
x[N]. A pattern is extracted from each frame of signal: The first
frame is started from the first sample of TP: x[1] to x[16384].
[0027] Frame to frame distance is set at 4,000 samples (0.5
second). [0028] The second frame is from x[4001] to x[20384].
[0029] The third frame is from x[8001] to x[24384]. [0030] And so
on for the remaining frames.
[0031] The first frame yields the first pattern vector, denoted
as:
.sub.1=[c.sub.1,1c.sub.1,2.LAMBDA.c.sub.1,25] [0032] The last frame
yields
[0032] .sub.M=[c.sub.M,1c.sub.M,2.LAMBDA.c.sub.M,25]
where
M=lower integer of (N-16384)/4000+1.
[0033] The start time of each frame is taken as the time location
of the corresponding pattern vector. Thus, the time location for
.sub.1 is at T.sub.S. And for, the time location for .sub.2 is at
T.sub.S+4000/8000 sec=T.sub.S+0.5 sec, and so on.
Self Similarity Detection using the one-times pattern vector as the
Query Starting from C.sub.--1, take every fourth pattern vector:
K.
[0034] And assign them to query vectors D as:
The result is a total of Q query vectors: to where Q=Quotient of
M/4. The time location of is that of And the time location of is
that of These 1.times. Q vectors will be used one by one to query
the 4.times. pattern vectors of to
Self-Similarity Detection
[0035] To facilitate the disclosure purposes, designate C and D as
the sets that hold the 4.times. and the 1.times. pattern vectors
respectively:
C = [ C .rho. 1 C .rho. 2 .LAMBDA. C .rho. M ] D = [ D .rho. 1 D
.rho. 2 .LAMBDA. D .rho. Q ] ##EQU00002##
[0036] Store every 4.times. pattern vector into a database. We will
refer this database as the self-similar-detection database,
abbreviated as SSD. The index for each of these pattern vectors is
the time-stamp of each frame.
TABLE-US-00001 TABLE 1 A Self-Similarity Detection Database Index
1.sup.st Subband 2.sup.nd Subband 3.sup.rd Subband . . . 25.sup.th
Subband 1 D.sub.1,1 D.sub.1,2 D.sub.1,3 . . . D.sub.1,25 2
D.sub.2,1 D.sub.2,2 D.sub.2,3 . . . D.sub.2,25 . . . Q D.sub.Q,1
D.sub.Q,2 D.sub.Q,3 . . . D.sub.Q,25
[0037] The method of the invention begins with the Phase I
Detection: Generating Catch Threads. The process is described as
below using pseudo code. The BlockSize is a parameter which can be
freely set. In the preferred embodiment, it is set to 5,
corresponding to a time duration of approximately 10 seconds.
[0038] Set BlockSize=5 [0039] Setp=1. [0040] Loop: While
p<=Q-(BlockSize-1): [0041] Read pattern vectors: to
.sub.p+BlockSize-1 Start a new bait by using these five pattern
vectors (corresponding to approximately 10 seconds audio) as the
first five 1.times. pattern vectors of the bait. [0042] Let the
query be r=p top+BlockSize-1: [0043] Query each of the five pattern
vectors to the SSD. [0044] Check whether the returning pattern
vectors satisfy the gap requirement and the error-bound
requirement.
[0045] Practitioners of ordinary skill will recognize that matching
is not necessarily an exact match but a sufficient match. Hence,
matching is determined if appropriate conditions are met between
the pattern vector in the query and a pattern vector in the
database. The first requirement is called the Gap Requirement.
[0046] Gap Requirement: A pattern, .epsilon.C, is said satisfied
with the gap-requirement if the absolute point error between to is
within some prescribed bound:
|c.sub.k,q-d.sub.r,q|.ltoreq.g.sub.q; q=1 to 25; g.sub.q is the gap
set for the q-th subband.
[0047] Note that the gap can be set individually for each subband.
In the preferred embodiment, g.sub.q is set to about 0.1 uniformly
on all subbands. However, these parameters can be adjusted to
balance false positive identifications against false negatives,
processing times and the like. For the purposes of this disclosure,
let RG be the set of pattern vectors in C that satisfy the gap
requirement with respect to query set
[0048] Error-bound requirement: A pattern vector, .epsilon.C, is
satisfies the error-bound requirement if the Norm-1 error between
to is less than some prescribed bound B:
n = 1 25 C s , n - d r , n .ltoreq. B e ##EQU00003##
[0049] In the preferred embodiment, B.sub.e is set at about
0.8.
[0050] Let RE be the set of pattern vectors in C that satisfy the
error bound requirement with respect to query set
[0051] Let RF=RG.andgate.RE. Then RF is the set of all pattern
vectors in C that satisfy both the gap and the error-bound
requirements. Also, let E1 be the set of corresponding Norm-1
errors. The five query patterns: to , will have respectively
RF.sub.p to RF.sub.p+4 as the pattern sets that satisfy both
requirements, and E1.sub.p to E1.sub.p+4 as the sets holding the
corresponding Norm-1 error values.
[0052] Let RF.sub.0 to RF.sub.4 be the five sets of pattern vectors
with respect to the querying of to This step is about building up
"qualified threads". In this step, a thread Th is a sequence of
five frames:
Th: .fwdarw..fwdarw..fwdarw..fwdarw.
where is selected from RF.sub.p, is selected from RF.sub.p+1, . . .
, and is selected from RF.sub.p+4. The offset of the index of two
subsequent frame, e.g. and in the thread has to satisfy the
following Sequencing-Rule:
[0053] Sequencing-Rule: The index F1 has a offset between 2 to 7
relative to F0. This offset is based on the 4.times. sampling of
pattern vectors in C and 1.times. sampling of pattern vectors in D.
Thus, for every advancement of one frame in D, we expect a four
frame advancement in C. Thus, the offset of index between F0 and F1
should be equal to 4. In the preferred embodiment, a range of 2
(4-2) to 7 (4+3) is allowed for such offset is due to the
robustness consideration. [0054] Steps for Sequencing Rule Test:
[0055] Set the number of threads equal to the number of elements in
RF.sub.0=N.sub.T. [0056] Each of the elements of RF.sub.0 is the
first frame of each of the N.sub.T threads. [0057] Let Th.sub.p,q
be the q-th element, q=0 to 4, of the p-th thread, p=1 to N.sub.T.
[0058] Loop: For n=1 to N.sub.T: (n is the thread index) [0059]
Loop: for q=1 to BlockSize-1: (q=the element of the p-th thread)
[0060] Select from the set RF.sub.q members that satisfy the
sequencing rule to the element Th.sub.n,q-1. [0061] If there exists
member(s) in RF.sub.q that satisfied with the Sequencing-Rule,
[0062] Select the one(s) that has the smallest error (the error
values are stored in E1.sub.q) as the new thread element T.sub.n,q.
[0063] Else [0064] The n-th thread is disqualified from further
threading.
[0065] Collect all qualified threads: Each qualifying thread is
required to pass the Sequencing-Rule between all the BlockSize
frames, as described above. As a result all threads that fail to
pass the Sequencing-Rule between any pair of subsequent are
disqualified. If the number of qualified threads is greater than
zero, then the process applies a time restraint test to the
thread.
[0066] In particular, the process examines the time location of
each qualified thread: [0067] Remove threads with time location
"earlier" than +t.sub.m (t.sub.m.gtoreq.0) minutes from the time
location of (This step is used to restrict time locations of all
catch threads be at least tm minutes "after" the time location of
the lead-thread). In the preferred embodiment, t.sub.m is set to 5
minutes. (Setting tm=5 minutes excludes all catches with time
locations within 5 minute to the bait.) [0068] Remove
time-overlapped threads: [0069] If two threads are overlapped in
time, remove the one that has the larger accumulated error. [0070]
Analyze the quality of each remaining thread with the
duration-ratio test: [0071] Given the thread Th.sub.p that is still
qualified, its duration-ratio is calculated with the following
formula:
[0071] duration
ratio=(16384+(Th.sub.p,4-Th.sub.p,0)*4000)/(16384+4*16000) [0072]
If the duration ratio is outside the interval (min_ratio,
max_ratio), the thread Th.sub.p is removed.
[0073] In the preferred embodiment, min_ratio is set to 0.90, and
max_ratio is set to 1.10--allowing +/-10% duration variance. If the
number of remaining qualified threads is greater than zero, these
remaining threads are subject to the next stage of the process.
[0074] Phase II Detection: Tracking on Qualified Threads. The phase
II tracking step involves tracking the remaining qualified threads.
This step of the analysis process shall be presented as
pseudo-code. [0075] Let the number of remaining qualified
threads=N.sub.u. [0076] Denote the k-th qualified thread as Thk
[0077] Set the MaxStep (an integer)=1. [0078] Set q=BlockSize-1.
[0079] Set Steps[n]=0, n=1 to N.sub.u. [0080] Set EndRegister[n]=1,
n=1 to N.sub.u. [0081] Set MinStep=1. [0082] Loop: While
MinStep<MaxStep [0083] Set q=q+1. [0084] Read the next query: is
sent to the to the SSD to obtain the corresponding RF.sub.q and
E1.sub.q. [0085] For loop: for n=1 to N.sub.u [0086] Select from
the set RF.sub.q members that satisfy the sequencing rule relative
to the element T.sub.n,q-1. (Note that the Sequencing-Rule requires
two adjacent frames in the n-th thread to have an offset between 2
to 7. In this loop, the Sequencing-Rule is generalized to be as
offset between two adjacent frames is between Steps[n]*4+2 to
Steps[n]*4+7.) [0087] If there exists at least one member in
RF.sub.q that meets the test, select the one that has the least
error (the error values are stored in E1.sub.q) as the new thread
element T.sub.n,q.
[0087] Set Steps[n]=1.
Else
Set T.sub.n,q=T.sub.n,q-1. [0088] Set Steps[n]=Steps[n]+1. (Note in
this step, the register in Steps[n] is used as a `skipped` counter.
In particular, a thread may have a skip in the q-th query, i.e.
There exists no frame in RFq that is satisfied with the
Sequencing-Rule relative to Tn,q-1. Every time where there is a
skip, the register Steps[n] is incremented. However, as long as
there is no skip, the register Steps[n] is reset to 1.) [0089] End
For loop. [0090] For loop: For n=1 to N.sub.u [0091] If
Steps[n]>MaxStep, [0092] EndRegister[n]=q-1. (Once the n-th
thread has reached the maximum skips denoted by MaxStep, the
location where the maximum skips reached is marked and registered
in EndRegister[n].) [0093] End the For loop. [0094] Compute
MinStep=min (Steps[n]). (The variable MinStep is the smallest
number of skips across all qualified threads. The threading will
continue until MinStep is larger than MaxStep, which is specified
to limit the maximum number of skips.) [0095] End the While
loop.
[0096] Now the phase II process proceeds to measure the duration of
each thread, the duration information of each n-th thread is
registered in EndRegister[n]. [0097] Loop: For n=1 to N.sub.u
[0098] Enter the four parameters to characterize each of the n-th
thread Th.sub.n: [0099] start_time=p [0100]
end_time=p+EndRegister[n]. [0101] First_frame_id=T.sub.n,0 [0102]
Last_frame_id=T.sub.n,EndRegister[n] [0103] End the For Loop.
[0104] Pair up the bait and every catch into an instance: Here,
every catch will be paired up with the bait. The start-time and the
end-time of the bait and the catch, as well as the channel id (such
as the channel's call-letter) will be logged. An instance id is
generated to every bait-catch pair. An instance consists of:
{instance id; channel id; start-time/end-time of the bait;
start-time/end-time of the catch}.
[0105] Once this is complete, the entire iteration of the loop
beginning at Phase I is complete. In pseudo code: [0106] Set
p=p+q+1. (Start the next bait right after the end of the previous
bait.) [0107] Else
[0107] p=p+1.
Else
p=p+1. [0108] End the While loop initiated at the beginning of
Phase I.
[0109] Self-Similarity Detection on Undetected Clips Only. As
mentioned above, the invention can be operated in two operational
modes: harvesting at "all-time", and harvesting in
"undetected-time". In the "all-time" mode, all programs within TP
that have been repeated more than once within TP are detected and
harvested. With a radio monitoring system in place, certain spots
within TP have already been detected and identified by the
monitoring system. The invention can be exercised more effectively
on clips where the radio monitoring system has not made detection.
In this mode, the invention is said to be harvesting just the
undetected time.
[0110] Running the invention in the undetected time requires some
modification to what was presented as the process presented above.
Departures from the all-time mode is presented as follows: The
Self-Similar Database (SSD) is registered with clips where no
detection were made by the monitor. The audio quality of each clip
is measured. There are a total of three quality scores per clip:
The RMS power of the clip represented here as a vector of h
samples: =[z.sub.1 z.sub.2 .LAMBDA. z.sub.h], is equal to
P z = 10 * log ( 1 h n = 1 h z n 2 ) B ##EQU00004##
[0111] If P.sub.z is below -30 dB, the audio power of the clip is
too low and will not be considered. The second is the mean 1st
order auto-correlation of the patterns of the clip: The 4.times.
patterns of the clip is obtained: {, k=1, 2, K, r}, and then
extracted are the 1.times. patterns: = n=1, 2, K, R=quotient(r/4).
Also calculated are the mean 1st order auto
correlation = 1 R - 1 n = 1 R - 1 ( m = 1 25 D n , m z * D n + 1 ,
m z ) . ##EQU00005##
If the mean value is higher than 0.99, the clip is believed to be
contaminated with too much static. On the other hand, if the mean
value is lower than 0.80, the clip is believed to be merely
containing channel noise. In both cases the clip is unusable. These
threshold numbers may be adjusted up or down to adjust for
false-positive versus false negative rates of identification as
required in the specific application of the invention. Also
calculated is the vigilance of the clip: The vigilance is a measure
of the clarity, characterized by norm-1 difference between adjacent
1.times. pattern vectors.
vigilance = 1 R - 1 n = 1 R - 1 ( m = 1 25 D n , m + 1 z - D n , m
z ) ##EQU00006##
[0112] The vigilance value is required to surpass a minimum score
of 0.95. This threshold is used in the preferred embodiment, but
may be adjusted based on the application of the invention.
[0113] The SSD is structured with the additional parametric data as
follows:
TABLE-US-00002 TABLE 2 A Self-Similarity Detection Database
Appended with Indices on Undetected Clips Mean 1st- Undetected RMS
order Pattern 1.sup.st 2.sup.nd 25.sup.th Clips power correlation
Vigilance index Subband Subband . . . subband Clip #1 RMS Mean 1st-
Vigilance (1, 1) D.sub.1,1.sup.1 D.sub.1,1.sup.1 D.sub.1,25.sup.1
Power order For Clip (1, 2) D.sub.2,1.sup.1 D.sub.2,2.sup.1
D.sub.2,25.sup.1 of correlation #1 Clip For Clip (1, M.sub.1)
D.sub.M.sub.1.sub.,1.sup.1 D.sub.M.sub.1.sub.,2.sup.1
D.sub.M.sub.1.sub.,25.sup.1 #1 #1 Clip #2 RMS Mean 1st- Vigilance
(2, 1) D.sub.1,1.sup.2 D.sub.1,1.sup.2 D.sub.1,25.sup.2 Power order
For Clip (2, 2) D.sub.2,1.sup.2 D.sub.2,2.sup.2 D.sub.2,25.sup.2 of
correlation #2 Clip For Clip (2, M.sub.2)
D.sub.M.sub.1.sub.,1.sup.2 D.sub.M.sub.1.sub.,2.sup.2
D.sub.M.sub.1.sub.,25.sup.2 #2 #2 Clip #S RMS Mean 1st- Vigilance
(S, 1) D.sub.1,1.sup.S D.sub.1,1.sup.S D.sub.1,25.sup.S Power order
For Clip (S, 2) D.sub.2,1.sup.S D.sub.2,2.sup.S D.sub.2,25.sup.S of
correlation #S Clip For Clip (S, M.sub.S)
D.sub.M.sub.1.sub.,1.sup.S D.sub.M.sub.1.sub.,2.sup.S
D.sub.M.sub.1.sub.,25.sup.S #S #S
[0114] The clips registered into the SSD can be arranged in the
order of the time location of each clip. Thus, Clip #1 preceded
Clip #2, which preceded Clip #3, and so on. The exercising of the
self-similarity detection algorithm is to take one clip at a time
as the bait to match with all the later clips. Clips that failed
the quality requirement will be excluded from the self-similarity
detection exercise. The matched instances are registered onto a
database:
TABLE-US-00003 TABLE 3 Self-Similarity Detection Information
between Undetected Clips are logged onto a Database. Clip Clip Clip
#1 Clip#2 Clip#3 . . . Clip#S Clip #1 Instance_id + Instance_id + .
. . Instance_id + similar information similar information similar
information between #1 and #2 between #1 and #3 between #1 and #M
Clip #2 .Instance_id + similar . . . Instance_id + similar
information information between #2 and #3 between #2 and #M . . . .
. . . . . . . . . . . . . . Clip #S - 1 Instance_id + similar
information between #M - 1 and #M Clip #S . . .
Harvesting--Mining Harvests from Similar Instances
[0115] The instances collected during the self-similarity detection
exercise will be processed to identify harvests from the instances.
Consider the following scenario where a commercial advertisement
with duration of 30 seconds has been repeated in five different
time locations within the harvesting period TP. [0116] Five
undetected clips, namely, A, B, C, D and E, where A preceded B, B
preceded C, C preceded D, and D preceded E, are checked into the
SSD. Each clip contains the said commercial advertisement. The time
duration of each clip is arbitrary but all longer than 30 seconds.
[0117] The self-similarity detection algorithm will then use A and
the bait and result with four instances [0118] B.sup.0/A.sup.0,
C.sup.0/A.sup.1, D.sup.0/A.sup.2 and E.sup.0/A.sup.3.
[0119] Here, A.sup.0, a segment in A, is the clip found to be
similar to B.sup.0, a segment in B. Likewise, A.sup.1, a segment in
A, is the clip found to be similar to C.sup.0, a segment in C.
[0120] Thus, the self-similarity detection algorithm will also
yield the following instances: With B as the bait: C.sup.1/B.sup.1,
D.sup.1/B.sup.2, E.sup.1/B.sup.3, With C as the bait:
D.sup.2/C.sup.2, E.sup.2/C.sup.3, With D as the bait:
E.sup.3/D.sup.3.
[0121] The 5 instances can be eloquently represented into a table
form:
TABLE-US-00004 The Instance Table Clip Clip A B C D E A
B.sup.0/A.sup.0 C.sup.0/A.sup.1 D.sup.0/A.sup.2 E.sup.0/A.sup.3 B
C.sup.1/B.sup.1 D.sup.1/B.sup.2 E.sup.1/B.sup.3 C D.sup.2/C.sup.2
E.sup.2/C.sup.3 D E.sup.3/D.sup.3 E
[0122] There are a total of ten instances, containing the five
repetitions of the advertisements. [0123] Also note that there are
overlaps among themselves. For example, A.sup.0, A.sup.1, A.sup.2
and A.sup.3 are the audio clips that contain the first occurrence
of the advertisement, except that they are differed slightly in
time-offsets and time-durations.
[0124] Two processes are used to mine harvests from all the
instances. The first process is called Identification. The major
purpose of this process is identify the best representation from
the multiple catches of the same clip. For example, the 4 clips
A.sup.0, A.sup.1, A.sup.2 and A.sup.3 are multiple catches
containing the same content--all of them contain the first
occurrence of the advertisement. The Identification process selects
the most representable out of these four catches. The second
process is the Grouping exercise where all similar clips, in this
case the 5 occurrences of the same advertisements, are being
grouped into the same group.
Identification Process.
[0125] There are two steps in the Identification Process: [0126]
The first step is to run overlapping detection exercises to
identify clips that have time overlaps. [0127] The second step is
to select the most representable from all overlapped clips
identified in the first step.
[0128] The same example is used to illustrate the two steps taken
in the Identification Process: [0129] Collect all the clips that
have been detected by the self-similarity detection algorithm
within the time period TP. In the same example, we have a total of
twenty clips: [0130] A.sup.0, A.sup.1, A.sup.2, A.sup.3, [0131]
B.sup.0, B.sup.1, B.sup.2, B.sup.3, [0132] C.sup.0, C.sup.1,
C.sup.2, C.sup.3, [0133] D.sup.0, D.sup.1, D.sup.2, D.sup.3, [0134]
E.sup.0, E.sup.1, E.sup.2, E.sup.3.
[0135] Note that each of the above clips is either a bait or a
catch. All the clips are then compared with each other to identify
if the two has time overlap.
[0136] Determination of time-overlap of two clips are given below:
[0137] Given two clips, say X and Y. [0138] If X and Y have time
overlap AND the overlap duration is within +/-6 seconds of the
shorter of the two clips. Then X and Y are said overlapped. [0139]
If X and Y is determined to have overlapped, then the shorter of
the two clips is replaced by the longer one. This is being done so
due to the notion that the longer one can contain more information
and is therefore more representable.
[0140] Thus, given the four clips that are overlapped: A.sup.0,
A.sup.1, A.sup.2, A.sup.3, only one clip will survive the
identification process. The following results are obtained after
exercising the Identification process on the twenty clips:
A.sup.0, A.sup.1, A.sup.2, A.sup.3-------->A.sup.2 is the
winner. B.sup.0, B.sup.1, B.sup.2, B.sup.3-------->B.sup.0 is
the winner. C.sup.0, C.sup.1, C.sup.2, C.sup.3-------->C.sup.3
is the winner. D.sup.0, D.sup.1, D.sup.2,
D.sup.3-------->D.sup.0 is the winner. E.sup.0, E.sup.1,
E.sup.2, E.sup.3-------->E.sup.2 is the winner.
[0141] Thus, only five clips are identified from 10 instances.
Grouping Process
[0142] The example five clips identified in the Identification
process above are determined if they can be grouped into a single
family. A family is a collection of clips which passed the
Sufficient-Similarity test.
Sufficient-Similarity Test
[0143] Two clips, X and Y. are said sufficient-similar if both
satisfy the "85% Rule": The duration of the similar segment is no
less than 85% of the durations of either clips. This percentage is
a configuration parameter. Clearly, the higher the percentage, the
tighter the similarity requirement. The 85% number used by the
preferred embodiment can be adjusted higher or lower depending on
the application of the invention. To determine if X and Y are
sufficient similar: [0144] The similarity segment across X and Y is
first identified. [0145] The duration of the segment on X is
measured. [0146] Compute the similarity ratio, R.sub.1, of this
duration to the duration of X. [0147] The duration of the segment
on Y is measured. [0148] Compute the similarity ratio, R.sub.2, of
this duration to the duration of Y. [0149] X and Y are said
sufficient-similar if min(R.sub.1,R.sub.2).gtoreq.85%. [0150]
Otherwise, the two clips are not sufficient-similar. [0151] Group
all sufficient-similar clips into a single family. [0152] A family
id is generated. [0153] The family contains all the information of
all the family members, including the channel id, time location,
and audio quality measures.
[0154] The Grouping process can be represented by a transition of
the Instance Table: Start with the Instance Table after the
self-similarity detection exercise
TABLE-US-00005 The Instance Table Clip Clip A B C D E A
B.sup.0/A.sup.0 C.sup.0/A.sup.1 D.sup.0/A.sup.2 E.sup.0/A.sup.3 B
C.sup.1/B.sup.1 D.sup.1/B.sup.2 E.sup.1/B.sup.3 C D.sup.2/C.sup.2
E.sup.2/C.sup.3 D E.sup.3/D.sup.3 E
[0155] Replace clips with their representative clips within the
Instant Table
TABLE-US-00006 [0155] The Instants Table after Identification
Process Clip Clip A.sup.2 B.sup.0 C.sup.3 D.sup.0 E.sup.2 A.sup.2
B.sup.0/A.sup.2 C.sup.3/A.sup.2 D.sup.0/A.sup.2 E.sup.2/A.sup.2
B.sup.0 C.sup.3/B.sup.0 D.sup.0/B.sup.0 E.sup.2/B.sup.0 C.sup.3
D.sup.0/C.sup.3 E.sup.2/C.sup.3 D.sup.0 E.sup.2/D.sup.0 E.sup.2
TABLE-US-00007 The Cross-Similarity Table Clip Clip A B C D E A
R.sub.1, R.sub.2 R.sub.1, R.sub.2 R.sub.1, R.sub.2 R.sub.1, R.sub.2
B R.sub.1, R.sub.2 R.sub.1, R.sub.2 R.sub.1, R.sub.2 C R.sub.1,
R.sub.2 R.sub.1, R.sub.2 D R.sub.1, R.sub.2 E
[0156] Compute the similarity ratio across clips [0157] Check if
the two ratios in each entry are above 85%. If so, the clips of
that entry are said sufficient-similar. [0158] Collect
sufficient-similar clips into a single family. [0159] Besides
identifying family member from the Cross-Similarity Table, the
following rule is also used to collect family members: [0160] If X
and Y are sufficient-similar, and Y and Z are also sufficient
similar, [0161] then X and Z are said sufficiently similar.
Selecting Family Representative Member
[0162] A family has at least two members. Again, based on the
belief that longer clip is more informative than shorter clip, the
family member that has the longest duration is selected as the
"Lead Member" of the family.
Quality Control with Family Lead's Member
[0163] After a lead member has been selected, the duration-ratio of
the overlap between the representative member to each family
members will be re-measured. Those that fail the 85% Rule will be
discarded from the family.
Effectiveness of the Grouping Process on Back-to-Back
Advertisements
[0164] Within the harvesting period NT, a particular advertisement
were repeated a number of times. And in some of these spins, the
advertisement was purposely paired up back-to-back with another
advertisement. For example, a MacDonalds special combo
advertisement may, from time to time, be purposely paired up with a
Coca-cola advertisement. It is often desirable to separate the
these catches, though they are similar, into two different
families. The Sufficient-Similarity Test is an effective mean for
separating these instances.
[0165] With our running examples with five clips: A, B, C, D, and
E. Here, A, C and E contain a MacDonald special combo
advertisement; while B and D are the MacDonald Special Combo
advertisement followed by a Cocacola advertisement.
[0166] The resulting Cross-Similarity Table is expected to be
similar to:
TABLE-US-00008 Cross-Similarity Table Clip Clip A B C D E A 0.95,
0.48 0.96, 093 0.91, 0.50 0.98, 0.96 B 0.47, 0.90 0.91, 0.93 0.48,
0.91 C 0.93, 0.47 0.89, 0.92 D 0.46, 0.88 E
[0167] Clearly, both B and D will not be combined with A, C and E
into the same family. Instead, A, B and C will be combined into one
family; and B and D will be combined into another family.
Second Membership in Grouping Process.
[0168] The term "membership" originated from classical set theory.
For example, the number .pi. is a member of the set of all real
numbers, but not a member of the set of integers. Referring above
where the rule--referred to as the "85% Rule"--is used to determine
if two clips are sufficiently similar to be grouped into the same
family. Those that pass the condition will be grouped together as
family members. The membership on the family level is referred to
as the "First Membership". The "Second Membership" is referred as
the grouping of families--Similarity of representative members of
two families are measured. The two will be grouped if the
similarity surpasses some prescribed threshold value. Same grouping
rule on the First Membership is used on the Second Membership. The
threshold value for the second membership is set at 50%, or the 50%
Rule is used for this second grouping exercise. The preferred
embodiment uses the 50% threshold for the second membership test,
but this value can be adjusted up or down depending on the
application of the invention. The purpose of creating the second
grouping process is to provide additional information to human
operators to speedup identifying harvests, that is, having
operators determine the actual identity of content that is
harvested but has not been identified with title and publisher
information.
[0169] This is illustrated in the following example:
[0170] Assume five clips: A, B, C, D and E containing two similar
60-second advertisements. All the five advertisements are identical
in the first 40 seconds, carrying the dedicated channel message.
The last 20 seconds are different, carrying the advertised product
information. The clips A, C and E contain the first advertisement.
And D and E contain the second advertisement. The first grouping
exercise will result with two distinct families:
Family #1 containing A, C and E. Family #2 containing B and D.
[0171] The results of the first grouping exercise do not convey the
information that Family #1 and Family #2 are similar. The second
grouping exercise will group both families into a common group,
referred to as a "Community" and a community_id will be assigned,
conveying that the two families are similar with respect to the 50%
Rule. From the data presentation point of view, the second grouping
exercise results with "trunks", where each trunk carries a number
of families, which are branches connected to the same trunk. Human
operators may first run a coarse analysis on a trunk, finding out
what is the common message within the entire trunk (e.g. a
Discovery-Channel advertisement), then pay attention to what are
the specifics to each family (e.g. different video programs).
Time Progressive Harvesting
[0172] The basic units of the Harvesting exercise are the
bait-catch instants being detected in the self-similarity detection
exercise. Exercising of the self-similarity detection algorithm can
be in the batch-mode, where the self-similarity exercise is working
on a particular time period NT. If NT is a very long period, the
self-similarity detection can be exercised in the time-progressive
mode. In this mode, the period NT is being divided into smaller
intervals. For example, the period NT is a 12 hour period starting
at 00:00 to 12:00. One can divided the period into two 6-hour
period: [0173] NT.sub.--1: 00:00 to 06:00 [0174] NT.sub.--2: 06:00
to 12:00
[0175] Exercise the self-similarity detection on the first
interval: [0176] Build the Instance Table.
[0177] Example: In NT.sub.--1, there are 5 undetected clips, A, B,
C, D, and E.
[0178] Thus, the Instant Table is a 5.times.5 table:
TABLE-US-00009 Clip Clip A B C D E A Instance_id + Instance_id +
Instance_id + Instance_id + Similarity information Similarity
information Similarity information Similarity information across A
and B across A and C across A and D across A and E B Instance_id +
Instance_id + Instance_id + Similarity information Similarity
information Similarity information across B and C across CBand D
across B and E C Instance_id + Instance_id + Similarity information
Similarity information across C and D across C and E D Instance_id
+ Similarity information across D and E E
[0179] Assume there are five undetected clips: F, G, H, I, and J,
in NT.sub.--2.
[0180] Append the five new clips from NT.sub.--2 onto the Instant
Table:
TABLE-US-00010 TABLE 4 ##STR00001## Instance Table Updating in the
Time-Progressive Mode. The non-shaded entries are the similarity
information on NT_1. The shaded entries are the similarity
information appended after the self similarity detection exercise
on NT_2.
are the similarity information on NT.sub.--1. The shaded entries
are the similarity information appended after the self similarity
detection exercise on NT.sub.--2.
[0181] The new entries are generated by running the self similarity
detection process described above on both the old and the new
undetected clips. In the time progressive mode, the size of SSD is
limited. In this example, the size of the SSD on NT.sub.--2 is half
to the SSD in the batch mode. Also note that in harvesting
NT.sub.--2, all the undetected clips in NT.sub.--1 and NT.sub.--2
will be used as queries to the SSD. New results in each partition
will be appended to the Instance Table. The partition size of NT
can be arbitrarily fine, as determined by the application. Also,
partitions are not required to be uniform, that is, a partition can
be set for each clip. After the Instance Table has been appended
with new results, both the Identification and the Grouping
processes can be exercised to identify append new members to
existing families, or to identify new families.
Cross Channel Harvesting
[0182] The harvester exercise on a single channel can be easily
applied to harvest similar clips across different channels. In
other words, it might be desirable to find self-similarity of
content clips not just across time, but across other broadcast
sources, where it is assumed that sufficiently similar clips on two
distinct broadcast sources is likely an identifiable piece of
content. The process is performed as follows:
[0183] First, select families from channels to be compared. To save
computational efforts, certain criteria are set on what kind of
families are to be selected for the exercise. Below are three
criteria that can be used:
Clips with similar durations. Clips from stations of similar
formats. Clips that appear most recently, e.g. within 24 hours.
[0184] Recall that every family has a lead member clip. The process
registers each lead clip from the selected families, of selected
channels, into the SSD. Then the self-similarity detection exercise
is run on the SSD. The resulting similarity information is entered
into an Instance Table.
TABLE-US-00011 TABLE 5 Instance Table in a Cross-Channel Exercise
Clip Lead Clip from Lead Clip from Lead Clip from Lead Clip from
Lead Clip from family#1 of family#2 of family#1 of family#2 of
family#1 of Clip channel #1 channel #1 channel #2 channel #2
channel #3 Lead Clip from similarity similarity similarity
similarity family#1 of information information information
information channel #1 Lead Clip from similarity similarity
similarity family#2 of information information information channel
#1 Lead Clip from similarity similarity family#1 of information
information channel #2 Lead Clip from similarity family#2 of
information channel #2 Lead Clip from family#1 of channel #3
[0185] The similarity information will be processed via the
Grouping Process with the 85% Rule to identify similar families
across different channels. All similar families will be combined
into a combined family. A combined-family identification number
will be generated for the combined family. Each combined family
consists of families of different channels. These families are
combined due to high degree of similarity among their lead members.
The combined family thus contains all informations of every family:
channel-id, family-id, time locations and audio-quality index of
every clip of every family. A lead combined-family member is
selected. Again, the clip that has the longest duration is selected
as the lead combined-family member.
Cross Channel Harvesting in the Same Market
[0186] Channels that are within the same market, e.g. all radio
stations in the New York market, will be selected into the
cross-channel harvesting exercise. First, determine what channel
selection criteria are to be applied. Select channels based on the
criteria into the harvesting process. Harvests identified on this
level are referred to "Market-Level" harvests. The Instance-Table
holds similarity information across channels in the same
market.
TABLE-US-00012 TABLE 6 Instance Table in a Market-Level Harvesting
Exercise. Each entry may contain similarity information of multiple
clips. Channel Channel Channel #1 Channel #2 Channel #3 . . .
Channel #M Channel #1 similarity similarity . . . similarity
information information information across across across channels
#1 channels #1 channels #1 and #2 and #3 and #M Channel #2
similarity . . . similarity information information across across
channels #2 channels #2 and #3 and #M Channel #3 . . . similarity
information across channels #3 and #M . . . . . . . . . . . . . . .
. . . Channel #M - 1 similarity information across channels #M - 1
and #M
Cross Channel Harvesting in Different Markets
[0187] Harvests from different markets obtained above are combined
into the cross-channel harvesting exercise. Determine what market
selection criteria are. One may have a presumption that certain
market combination is likely to yield meaningful harvests. Harvests
identified on this level is referred to "National-Level" harvests.
The Instance-Table holds similarity information across channels of
different markets.
TABLE-US-00013 TABLE 7 Instance Table in a National-Level
Harvesting Exercise. Each entry may contain similarity information
of multiple clips of two markets. Market Market Market #1 Market #2
Market #3 . . . Market #N Market #1 similarity similarity . . .
similarity information information information across markets #1
across markets across markets and #2 #1 and #3 #1 and #M Market #2
similarity . . . similarity information information across markets
across markets #2 and #3 #2 and #M Market #3 . . . similarity
information across markets #3 and #M . . . . . . . . . . . . . . .
. . . Market #N - 1 similarity information across markets #N - 1
and #N
Harvest Processing
[0188] The Harvester exercise yields similar programs on three
levels: [0189] Channel level: Similar programs of the same channel.
[0190] Market level: Similar programs of different channels in the
same market. [0191] National level: Similar programs of different
channels of different markets.
[0192] Although here there may be substantial results or harvests
of identifiable but unregistered content as determined on several
levels, they are still remain as unknown programs until after human
listening. The harvest processing stage consists of a number of
steps: starting from the harvests collected on all the three
levels, human listening, identification, to the end where certain
clips out of the harvests are decided to be promoted to the
monitoring system, that is, fully identified and registered in the
monitoring system.
Human Listening--Identifying Harvests
[0193] The families of all the three levels will be presented to
human operators for identification. The system will automatically
select the clip that has the highest audio quality within the
family and present to the human operator. The operator will
identify the clip, and input the meta-data of the clip, including,
for example, the song title, publisher, record label. If it is a
song, the identification of the clip will be made by a format
specialist, which generates the title and artist information.
[0194] Although the present invention has been described and
illustrated in detail, it is to be clearly understood that the same
is by way of illustration and example only, and is not to be taken
by way of limitation. It is appreciated that various features of
the invention which are, for clarity, described in the context of
separate embodiments may also be provided in combination in a
single embodiment. Conversely, various features of the invention
which are, for brevity, described in the context of a single
embodiment may also be provided separately or in any suitable
combination. It is appreciated that the particular embodiment
described in the Appendices is intended only to provide an
extremely detailed disclosure of the present invention and is not
intended to be limiting. It is appreciated that any of the software
components of the present invention may, if desired, be implemented
in ROM (read-only memory) form. The software components may,
generally, be implemented in hardware, if desired, using
conventional techniques. [0195] The spirit and scope of the present
invention are to be limited only by the terms of the appended
claims.
* * * * *