U.S. patent application number 10/866164 was filed with the patent office on 2006-01-05 for method, apparatus and program products for retrieving moving image.
This patent application is currently assigned to TELECOMMUNICATIONS ADVANCEMENT. Invention is credited to Katsunao Takahashi, Nobuyoshi Terashima, Hideyoshi Tominaga.
Application Number | 20060002463 10/866164 |
Document ID | / |
Family ID | 18115384 |
Filed Date | 2006-01-05 |
United States Patent
Application |
20060002463 |
Kind Code |
A1 |
Takahashi; Katsunao ; et
al. |
January 5, 2006 |
Method, apparatus and program products for retrieving moving
image
Abstract
An apparatus for retrieving moving image according to the
present invention has an extracting unit for extracting the
information of the feature quantity of the moving image from the
fixed moving image data, an extracting unit for extracting the
information of the unique identification of the moving image to
identify the moving image according to the information of the
feature quantity of the moving image, and a retrieving unit for
retrieving and identifying the fixed moving image from the object
moving image for retrieving by comparing the information of the
unique identification of the moving image with the information of
the unique identification of the moving image beforehand extracted
from the object moving image. Therefore, the information of the
unique identification of the moving image for retrieving the moving
image depends on the content of the moving image can be extracted,
consequently, the identical moving image can be retrieved and
identified from the moving image by using the information of the
unique identification of the moving image at high speed.
Inventors: |
Takahashi; Katsunao; (Tokyo,
JP) ; Terashima; Nobuyoshi; (Kawasaki, JP) ;
Tominaga; Hideyoshi; (Tokyo, JP) |
Correspondence
Address: |
MORRISON & FOERSTER LLP
1650 TYSONS BOULEVARD
SUITE 300
MCLEAN
VA
22102
US
|
Assignee: |
TELECOMMUNICATIONS
ADVANCEMENT
Tokyo
JP
Matsushita Electric Industrial Co., Ltd.
Osaka
JP
|
Family ID: |
18115384 |
Appl. No.: |
10/866164 |
Filed: |
August 6, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09421264 |
Oct 20, 1999 |
6792043 |
|
|
10866164 |
Aug 6, 2004 |
|
|
|
Current U.S.
Class: |
375/240.01 ;
382/190; 707/E17.028 |
Current CPC
Class: |
G06F 16/785 20190101;
G06F 16/786 20190101; G06F 16/7864 20190101 |
Class at
Publication: |
375/240.01 ;
382/190 |
International
Class: |
H04N 11/02 20060101
H04N011/02; G06K 9/46 20060101 G06K009/46 |
Foreign Application Data
Date |
Code |
Application Number |
Oct 23, 1998 |
JP |
HEI. 10-319890 |
Claims
1-57. (canceled)
58. An apparatus for retrieving a moving image to retrieve a
pre-determined moving image from an object moving image,
comprising: first extracting means for extracting information of
the feature quantity of each frame from the pre-determined moving
image; second extracting means for extracting information of a
unique identification created by arranging one or more physical
quantities of the information of the feature quantity on a time
axis; and retrieving means for retrieving a position of the
pre-determined moving image on the time axis from the object moving
image by comparing the arrangement of the information of the unique
identification of the pre-determined moving image, with that of the
information of the unique identification prepared in accordance
with the object moving image.
59. The apparatus for retrieving the moving image of claim 58,
further comprising: detecting means for detecting cut points of the
moving image from the pre-determined moving image and dividing the
pre-determined moving image into a plurality of shots at each of
the cut points, wherein: the second extracting means extracts the
information of the unique identification created by arranging one
or more physical quantities of the information of the feature
quantity at each of the shots divided by the detecting means; and
the retrieving means retrieves the pre-determined moving image from
the object moving image by comparing the arrangement of the
information of the unique identification of the pre-determined
moving image, with that of the information of the unique
identification prepared in accordance with the object moving
image.
60. The apparatus for retrieving the moving image of claim 58,
wherein the second extracting means extracts the information of the
unique identification created by arranging one or more expanded or
reduced physical quantities of the information of the physical
quantity on a time axis.
61. The apparatus for retrieving the moving image of claim 58,
wherein the first extracting means extracts the information as a
block unit of M.times.N pixels, where M and N are natural numbers,
from the pre-determined moving image data, as the information of
the feature quantity.
62. The apparatus for retrieving the moving image of claim 58,
wherein one or more physical quantities of the information of the
feature quantity are arranged on the time axis making a horizontal
line of a graph represent time.
63. The apparatus for retrieving the moving image of claim 58,
wherein the information of the unique identification prepared in
accordance with the object moving image is a certain time
duration.
64. The apparatus for retrieving the moving image of claim 58,
wherein the retrieving means compares an expanded or reduced value
of the information of the unique identification which is extracted
from the pre-determined moving image, with an expanded or reduced
value of the information of the unique identification which is
prepared in accordance with the object moving image, on the time
axis.
65. The apparatus for retrieving the moving image of claim 58,
wherein the second extracting means extracts a value as the
information of the unique identification of pre-determined moving
image by arranging all information of the feature quantity, or some
information of the feature quantity extracted by uniform or
irregular intervals, on the time axis, from the pre-determined
moving image.
66. The apparatus for retrieving the moving image of claim 58,
wherein the second extracting means extracts a value as the
information of the unique identification of object moving image
prepared in accordance with the object image by arranging all
information of the feature quantity, or some information of the
feature quantity extracted by uniform or irregular intervals, on
the time axis, from the object moving image.
67. The apparatus for retrieving the moving image of claim 58,
wherein the information of the feature quantity of each frame
extracted by the first extracting means is physical information of
the feature quantity obtained from information of luminance and
color difference (Y, Cb, Cr) or information of color (R, G, B) of
each frame.
68. The apparatus for retrieving the moving image of claim 67,
wherein the information of the feature quantity of each frame
extracted by the first extracting means is one or more transformed
values obtained from the information of luminance and color
difference (Y, Cb, Cr) or the information of color (R, G, B) of
each frame.
69. The apparatus for retrieving the moving image of claim 68,
wherein the transformed value obtained from the information of
luminance and color difference (Y, Cb, Cr) of each frame is one or
more of a value, a mean value, a total value, and a difference
value of the physical information of the feature quantity obtained
from the information of luminance and color difference (Y, Cb, Cr)
of each frame; and the transformed value obtained from the
information of color (R, G, B) of each frame is one or more of a
value, a mean value, a total value, and a difference value of the
physical information of the feature quantity obtained from the
information of color (R, G, B) of each frame.
70. The apparatus for retrieving the moving image of claim 68,
wherein the transformed value obtained from the information of
luminance and color difference (Y, Cb, Cr) of each frame is one or
more of a value, a mean value, a total value, and a difference
value of weighted physical information of the feature quantity
obtained from the information of luminance and color difference (Y,
Cb, Cr) of each frame; and the transformed value obtained from the
information of color (R, G, B) of each frame is one or more of the
value, the mean value, the total value, and the difference value of
the weighted physical information of the feature quantity obtained
from the information of color (R, G, B) of each frame.
71. The apparatus for retrieving the moving image of claim 69,
wherein the transformed value obtained from the information of
luminance and color difference (Y, Cb, Cr) of each frame is a
quantified value of the physical information of the feature
quantity obtained from the information of luminance and color
difference (Y, Cb, Cr) of each frame, or a quantified value
obtained by transforming the physical information of the feature
quantity from the information of luminance and color difference (Y,
Cb, Cr) of each frame; and the transformed value obtained from the
information of the information of color (R, G, B) of each frame is
a quantified value of the physical information of the feature
quantity obtained from the information of the information of color
(R, G, B) of each frame, or a quantified value obtained by
transforming the physical information of the feature quantity from
the information of the information of color (R, G, B) of each
frame.
72. The apparatus for retrieving the moving image of claim 58,
wherein the first extracting means extracts information of a
feature quantity of a compressed moving image of each frame by
executing a Discrete Cosine Transform operation or from a coded
stream.
73. The apparatus for retrieving the moving image of claim 72,
wherein the first extracting means extracts Discrete Cosine
Transform coefficients of each frame or information of motion
compensation of each frame, as the information of the feature
quantity of the compressed moving image.
74. The apparatus for retrieving the moving image of claim 73,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by the first extracting means
is one of a value, a mean value, a total value, and a difference
value of physical information of the feature quantity of the
compressed moving image obtained from the Discrete Cosine Transform
coefficients of each frame, and a vector or a value of the vector
which is at least one of a motion vector, a mean motion vector
between frames before and after, a total motion vector, a
difference motion vector, and a motion vector on a whole frame
obtained from the information of the motion compensation.
75. The apparatus for retrieving the moving image of claim 73,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by the first extracting means
is one of a value, a mean value, a total value, and a difference
value of weighted physical information of the feature quantity of
the compressed moving image obtained from the Discrete Cosine
Transform coefficients of each frame, and a vector or a value of
the vector which is at least one of a motion vector, a mean motion
vector between frames before and after, a total motion vector, a
difference motion vector, and a motion vector on a whole frame
obtained from weighted information of the motion compensation.
76. The apparatus for retrieving the moving image of claim 72,
wherein the first extracting means extracts only a direct current
element of the Discrete Cosine Transform coefficients of each
frame, as the information of the feature quantity of the compressed
moving image.
77. The apparatus for retrieving the moving image of claim 73,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by the first extracting means
is a quantified value of one of the Discrete Cosine Transform
coefficients, a value of the motion compensation, and a value
obtained from the value of the motion compensation.
78. The apparatus for retrieving the moving image of claim 58,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by the first extracting means
is the combination of physical information of the feature quantity
of each frame which is obtained from information of luminance and
color difference (Y, Cb, Cr) or information of color (R, G, B) of
each frame, and information of a feature quantity of a compressed
moving image which is obtained by executing a Discrete Cosine
Transform operation or by extracting from a coded stream.
79. An apparatus for monitoring television commercials to retrieve
a pre-determined moving image such as a television commercial from
an object moving image such as the television broadcast and a
stored moving image, comprising: first extracting means for
extracting information of a feature quantity of each frame from the
pre-determined moving image; second extracting means for extracting
information of a unique identification created by arranging one or
more physical quantities of the information of the feature quantity
on a time axis; retrieving means for retrieving a position of the
pre-determined moving image on the time axis from the object moving
image by comparing the arrangement of the information of the unique
identification of the pre-determined moving image, with that of the
information of the unique identification prepared in accordance
with the object moving image; and displaying means for displaying a
result of retrieving by the retrieving means.
80. The apparatus for monitoring television commercials of claim
79, further comprising storing means for storing the result of
retrieving by the retrieving means, wherein the displaying means
displays the stored result stored by the storing means.
81. A method for retrieving a moving image to retrieve a
pre-determined moving image from an object moving image,
comprising: (A) extracting information of a feature quantity of
each frame from the pre-determined moving image; (B) extracting
information of a unique identification created by arranging one or
more physical quantities of the information of the feature quantity
on a time axis; and (C) retrieving a position of the pre-determined
moving image on the time axis from the object moving image by
comparing the arrangement of the information of the unique
identification of the pre-determined moving image, with that of the
information of the unique identification prepared in accordance
with the object moving image.
82. The method for retrieving the moving image of claim 81, further
comprising: (D) detecting cut points of the moving image from the
pre-determined moving image and dividing the pre-determined moving
image into a plurality of shots at each of the cut points, wherein:
the step (B) extracts the information of the unique identification
created by arranging one or more physical quantities of the
information of the feature quantity at each of the shots divided by
the step (D); and the step (C) retrieves the pre-determined moving
image from the object moving image by comparing the arrangement of
the information of the unique identification of the pre-determined
moving image, with that of the information of the unique
identification prepared in accordance with the object moving
image.
83. The method for retrieving the moving image of claim 81, wherein
the step (B) extracts the information of the unique identification
created by arranging one or more expanded or reduced physical
quantities of the information of the physical quantity on a time
axis.
84. The method for retrieving the moving image of claim 81, wherein
the step (A) extracts the information as a block unit of M.times.N
pixels, where M and N are natural numbers, from the pre-determined
moving image data, as the information of the feature quantity.
85. The method for retrieving the moving image of claim 81, wherein
one or more physical quantities of the information of the feature
quantity are arranged on the time axis making a horizontal line of
a graph represent time.
86. The method for retrieving the moving image of claim 81, wherein
the information of the unique identification prepared in accordance
with the object moving image is a certain time duration.
87. The method for retrieving the moving image of claim 81, wherein
the step (C) compares an expanded or reduced value of the
information of the unique identification which is extracted from
the pre-determined moving image, with an expanded or reduced value
of the information of the unique identification which is prepared
in accordance with the object moving image, on the time axis.
88. The method for retrieving the moving image of claim 81, wherein
the step (B) extracts a value as the information of the unique
identification of pre-determined moving image by arranging all
information of the feature quantity, or some information of the
feature quantity extracted by uniform or irregular intervals, on
the time axis, from the pre-determined moving image.
89. The method for retrieving the moving image of claim 81, wherein
the step (B) extracts a value as the information of the unique
identification of the object moving image prepared in accordance
with the object image by arranging all information of the feature
quantity, or some information of the feature quantity extracted by
uniform or irregular intervals, on the time axis, from the object
moving image.
90. The method for retrieving the moving image of claim 81, wherein
the information of the feature quantity of each frame extracted by
the step (A) is physical information of the feature quantity
obtained from information of luminance and color difference (Y, Cb,
Cr) or the information of color (R, G, B) of each frame.
91. The method for retrieving the moving image of claim 90, wherein
the information of the feature quantity of each frame extracted by
the step (A) is one or more transformed values obtained from the
information of luminance and color difference (Y, Cb, Cr) or the
information of color (R, G, B) of each frame.
92. The method for retrieving the moving image of claim 91, wherein
the transformed value obtained from the information of luminance
and color difference (Y, Cb, Cr) of each frame is one or more of a
value, a mean value, a total value, and a difference value of the
physical information of the feature quantity obtained from the
information of luminance and color difference (Y, Cb, Cr) of each
frame; and the transformed value obtained from the information of
color (R, G, B) of each frame is one or more of a value, a mean
value, a total value, and a difference value of the physical
information of the feature quantity obtained from the information
of color (R, G, B) of each frame.
93. The method for retrieving the moving image of claim 91, wherein
the transformed value obtained from the information of luminance
and color difference (Y, Cb, Cr) of each frame is one or more of a
value, a mean value, a total value, and a difference value of
weighted physical information of the feature quantity obtained from
the information of luminance and color difference (Y, Cb, Cr) of
each frame; and the transformed value obtained from the information
of color (R, G, B) of each frame is one or more of a value, a mean
value, a total value, and a difference value of weighted physical
information of the feature quantity obtained from the information
of color (R, G, B) of each frame.
94. The method for retrieving the moving image of claim 92, wherein
the transformed value obtained from the information of luminance
and color difference (Y, Cb, Cr) of each frame is a quantified
value of the physical information of the feature quantity obtained
from the information of luminance and color difference (Y, Cb, Cr)
of each frame, or a quantified value obtained by transforming the
physical information of the feature quantity from the information
of luminance and color difference (Y, Cb, Cr) of each frame; and
the transformed value obtained from the information of the
information of color (R, G, B) of each frame is a quantified value
of the physical information of the feature quantity obtained from
the information of color (R, G, B) of each frame, or a quantified
value obtained by transforming the physical information of the
feature quantity from the information of the information of color
(R, G, B) of each frame.
95. The method for retrieving the moving image of claim 81, wherein
step (A) extracts the information of the feature quantity of the
compressed moving image of each frame by executing a discrete
Cosine Transform operation or extracting from a coded stream.
96. The method for retrieving the moving image of claim 95, wherein
step (A) extracts the Discrete Cosine Transform coefficients of
each frame or the information of motion compensation of each frame,
as the information of the feature quantity of compressed moving
image.
97. The method for retrieving the moving image of claim 96, wherein
the information of the feature quantity of compressed moving image
of each frame extracted by the step (A) is one of a value, a mean
value, a total value, and a difference value of the physical
information of the feature quantity of the compressed moving image
obtained from the Discrete Cosine Transform coefficients of each
frame, and a vector or a value of the vector which is at least one
of a motion vector, a mean motion vector between frames before and
after, a total motion vector, a difference motion vector, and a
motion vector on a whole frame obtained from the information of the
motion compensation.
98. The method for retrieving the moving image of claim 96, wherein
the information of the feature quantity of compressed moving image
of each frame extracted by step (A) is one of a value, a mean
value, a total value, and a difference value of weighted physical
information of the feature quantity of the compressed moving image
obtained from Discrete Cosine Transform coefficients of each frame,
and a vector or a value of the vector which is at least one of a
motion vector, a mean motion vector between frames before and
after, a total motion vector, a difference motion vector, and a
motion vector on a whole frame obtained from the weighted
information of the motion compensation.
99. The method for retrieving the moving image of claim 95, wherein
step (A) extracts only a Direct Current element of the Discrete
Cosine Transform coefficients of each frame, as the information of
the feature quantity of the compressed moving image.
100. The method for retrieving the moving image of claim 96,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by step (A) is a quantified
value of one of the Discrete Cosine Transform coefficients, a value
of the motion compensation, and a value obtained from the value of
the motion compensation.
101. The method for retrieving the moving image of claim 81,
wherein the information of the feature quantity of the compressed
moving image of each frame extracted by the step (A) is the
combination of the physical information of the feature quantity of
each frame which is obtained from information of luminance and
color difference (Y, Cb, Cr) or information of color (R, G, B) of
each frame, and information of a feature quantity of a compressed
moving image which is obtained by executing a Discrete Cosine
Transform operation or by extracting from a coded stream.
102. A method for monitoring television commercials to retrieve a
pre-determined moving image such as a television commercial from an
object moving image such as a television broadcast and stored
moving image, comprising: (A) extracting information of a feature
quantity of each frame from the pre-determined moving image; (B)
extracting information of a unique identification created by
arranging one or more physical quantities of the information of the
feature quantity on a time axis; (C) retrieving a position of the
pre-determined moving image on the time axis from the object moving
image by comparing the arrangement of the information of the unique
identification of the pre-determined moving image, with that of the
information of the unique identification prepared in accordance
with the object moving image; and (E) displaying the result of
retrieving by step (C).
103. The method for monitoring television commercials of claim 102,
further comprising a step (F) of storing the result of retrieving
by step (C), wherein step(E) displays the stored result stored by
step (F).
104. Program products for retrieving a moving image to retrieve a
pre-determined moving image from an object moving image, which is
executed by a computer system, comprising: (A) extracting
information of a feature quantity of each frame from the
pre-determined moving image; (B) extracting information of a unique
identification created by arranging one or more physical quantities
of the information of the feature quantity on a time axis; and (C)
retrieving a position of the pre-determined moving image on the
time axis from the object moving image by comparing the arrangement
of the information of the unique identification of the
pre-determined moving image with that of the information of the
unique identification prepared in accordance with the object moving
image.
105. Program products for monitoring television commercials to
retrieve a pre-determined moving image such as a television
commercial from an object moving image such as the television
broadcast and a stored moving image, which is executed by computer
system, comprising: (A) extracting information of a feature
quantity of each frame from the pre-determined moving image; (B)
extracting information of a unique identification created by
arranging one or more physical quantities of the information of the
feature quantity on a time axis; (C) retrieving a position of the
pre-determined moving image on the time axis from the object moving
image by comparing the arrangement of the information of the unique
identification of the pre-determined moving image, with that of the
information of the unique identification prepared in accordance
with the object moving image; and (E) displaying the result of
retrieving by step (C).
Description
FIELD OF THE INVENTION
[0001] This invention relates to a method and an apparatus for
retrieving moving image, which retrieve and identify fixed moving
image from some moving image. And more particularly to, a method
and an apparatus for retrieving moving image, in which the
information of the unique identification of moving image which
depends on the content of the moving image is extracted and
registered, the registered moving image is retrieved and identified
earlier than video rate from moving image such as moving image
archives sent in broadcasting and the like.
BACKGROUND OF THE INVENTION
[0002] It is well known that there are various technologies on the
international standard encoding system such as MPEG (Motion Picture
Experts Group) as the information compressing technology for
storing and transmitting moving image. It becomes possible that
moving image is sufficiently handled even in the PC (Personal
Computer), if the encoded moving image compressed by this
international standard encoding system is used. The technology for
monitoring the specific moving image by using the PC, for example
monitoring a television commercial (identifying moving image), is
ever studied and is developed.
[0003] As the conventional method for retrieving moving image,
there are several methods for retrieving moving image, for example
a method using the keyword retrieval, a method using the visual
discrimination by the visual observation of the user, and a method
using the number of continuous frames on the shot of moving image
disclosed in Japanese Patent Publication No. JP7-retrieving moving
image by detecting the cut point of the moving image. In this
apparatus for retrieving the cut point of moving image disclosed in
Japanese Patent Publication No. JP7-236153, the time variation of
luminance difference between frames of the reduction image which is
made from prepared reduction image and the mean value component
data of the information of compressed and encoded moving image and
the time variation of color difference histogram correlation value
are used in order to obtain a cut point at high speed from moving
image.
[0004] And, there is an apparatus for processing moving image
disclosed in Japanese Patent Publication No. JP7-152779 as an
apparatus for detecting the characteristic frame of moving image.
In this apparatus for processing moving image disclosed in Japanese
Patent Publication No. JP7-152779, the low frequency part of DCT
(Discrete Cosine Transform) component is used in order to detect
characteristic frame that is used for as index information
(retrieving information), for example a cut point frame, a
motionless frame, a character frame, a specific color frame, etc.,
from compressed moving image data including the DCT operation.
[0005] In the conventional methods and apparatuses for retrieving
moving image, however, there are disadvantages as follows.
[0006] That is to say, in the conventional method for retrieving
moving image by the keyword retrieval, there is a disadvantage that
the considerable cost for registering keyword index information is
taken, because to beforehand register the keyword index information
is necessary. And more, there is a disadvantage with the case in
which keyword index information differs even in the case of
retrieving of identical moving image by the user, because the
feeling of moving image of each user who retrieves moving image and
the way of understanding are different. Moreover, there is a
disadvantage that the time for choosing the keyword index of the
desire is taken, even the system for inputting keyword index in
which the user who retrieves moving image from keyword index
information presented by a list type is made to choose the keyword
index of the desire.
[0007] And, in the conventional method for retrieving moving image
using the visual discrimination by the visual observation of the
user, there is a disadvantage that hands are taken and the cost is
high, because it is the method in which the user who retrieves
moving image memorizes moving image that the user intends to
retrieve and identify, and the user retrieves and identifies the
moving image of the desire by observing all moving images.
[0008] And, in the method for retrieving moving image using the
number of continuous frames on the shot which constitutes moving
image disclosed in Japanese Patent Publication JP7-46517, there is
a disadvantage that it might be misunderstood that moving image is
identical with moving image for retrieving, if the number of
continuous frames of the moving image agrees even if the content of
the moving image differs from the moving image for retrieving,
because the number of continuous frames is not index information
reflecting the content of the meaning of moving image.
[0009] And, in the conventional real time discrimination method of
image scene based on the time series coding of cutting, there is a
disadvantage that the time for the extracting process of a cut
point frame is taken and the cost also rises, because it is
necessary to accurately extract a cut point frame in order to
obtain a right collation result.
[0010] And, in the apparatus for retrieving the cut point of moving
image disclosed in Japanese Patent Publication No. JP7-236153,
there is a disadvantage that it is impossible to retrieve and
identify moving image of the desire automatically, because the
detection of a cut point in moving image is a purpose, and it is
only to detect a cut point by regarding the time variation of
luminance difference and the time variation of color difference
histogram correlation value.
[0011] In the apparatus for processing moving image disclosed in
Japanese Patent Publication No. JP7-152779, there is a disadvantage
that time is taken due to the complicated process and the cost is
high. Because the interim result which corresponds to the result of
DCT operation by the process for reconstructing image data from
compressed image data is made to be signals of the information of
the feature quantity of moving image index, at least a kind of
moving image index is detected from this signals of the information
of the feature quantity of moving image index, and the sum of
difference value with the moving image index of examined image is
processed at the threshold, and then moving image is detected as
index image by the result of this threshold processing.
SUMMARY OF THE INVENTION
[0012] Accordingly, it is an object of the invention to provide a
method and an apparatus for retrieving moving image, in which the
information of the unique identification of moving image used for
retrieving in accordance with the content of the moving image can
be automatically extracted even in the computer with small
resources such as PC, and in which moving image can be retrieved
and identified automatically by using the information of the unique
identification of moving image.
[0013] According to the first feature of the invention, an
apparatus for retrieving moving image to retrieve pre-determined
moving image from object moving image, comprises first extracting
means for extracting the information of the feature quantity of
each frame from the pre-determined moving image, second extracting
means for extracting the information of the unique identification
of moving image to identify the moving image in accordance with the
information of the feature quantity, extracted by the first
extracting means, and retrieving means for retrieving the
pre-determined moving image from the object moving image by
comparing the information of the unique identification, extracted
by the second extracting means, with the information of the unique
identification prepared in accordance with the object moving
image.
[0014] According to the second feature of the invention, a method
for retrieving moving image to retrieve pre-determined moving image
from object moving image, comprises the steps of (A) extracting the
information of the feature quantity of each frame from the
pre-determined moving image, (B) extracting the information of the
unique identification of moving image to identify the moving image
in accordance with the information of the feature quantity,
extracted by said step (A), and (C) retrieving the pre-determined
moving image from the object moving image by comparing the
information of the unique identification, extracted by the step
(B), with the information of the unique identification prepared in
accordance with the object moving image.
[0015] According to the third feature of the invention, a program
for retrieving moving image to retrieve predetermined moving image
from object moving image, which is executed by computer system,
comprises the steps of (A) extracting the information of the
feature quantity of each frame from the pre-determined moving
image, (B) extracting the information of the unique identification
of moving image to identify the moving image in accordance with the
information of the feature quantity, extracted by the step (A), and
(C) retrieving the pre-determined moving image from the object
moving image by comparing the information of the unique
identification, extracted by the step (B), with the information of
the unique identification prepared in accordance with the object
moving image.
BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The invention will be described in more detail in
conjunction with the appended drawings, wherein:
[0017] FIG. 1 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0018] FIG. 2A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0019] FIG. 2B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0020] FIG. 3 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0021] FIG. 4A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0022] FIG. 4B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0023] FIG. 5 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0024] FIG. 6A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0025] FIG. 6B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0026] FIG. 7 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0027] FIG. 8A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0028] FIG. 8B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0029] FIG. 9 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0030] FIG. 10A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0031] FIG. 10B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0032] FIG. 11 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0033] FIG. 12A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0034] FIG. 12B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0035] FIG. 13 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0036] FIG. 14 is a conceptual view showing the process of DCT
(discrete cosine transform) operation in MPEG;
[0037] FIG. 15A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0038] FIG. 15B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0039] FIG. 16 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0040] FIG. 17A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0041] FIG. 17B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0042] FIG. 18 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0043] FIG. 19A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0044] FIG. 19B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0045] FIG. 20 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0046] FIG. 21A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0047] FIG. 21B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0048] FIG. 22 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0049] FIG. 23A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0050] FIG. 23B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0051] FIG. 24 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0052] FIG. 25A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0053] FIG. 25B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0054] FIG. 26 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0055] FIG. 27A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0056] FIG. 27B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0057] FIG. 28 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0058] FIG. 29A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0059] FIG. 29B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0060] FIG. 30 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0061] FIG. 31A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0062] FIG. 31B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0063] FIG. 32 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0064] FIG. 33A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0065] FIG. 33B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0066] FIG. 34 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0067] FIG. 35A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0068] FIG. 35B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0069] FIG. 36 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0070] FIG. 37A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0071] FIG. 37B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0072] FIG. 38 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0073] FIG. 39A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0074] FIG. 39B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0075] FIG. 40 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0076] FIG. 41A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0077] FIG. 41B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0078] FIG. 42 is a block diagram showing the apparatus for
retrieving moving image according to the present invention;
[0079] FIG. 43A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0080] FIG. 43B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0081] FIG. 44 is a block diagram showing the apparatus for
retrieving moving image according to the present invention; and
[0082] FIG. 45A is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention.
[0083] FIG. 45B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
[0084] FIG. 46 is a block diagram showing the apparatus for
monitoring television commercials according to the present
invention; and
[0085] FIG. 47A is a flowchart of the process in the apparatus for
monitoring television commercials according to the present
invention.
[0086] FIG. 47B is a flowchart of the process in the apparatus for
retrieving moving image according to the present invention;
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0087] In the following, a method and an apparatus for retrieving
moving image, and a recording medium according to the invention are
explained in details referring to the drawings.
[0088] An apparatus and a method for retrieving moving image in the
first preferred embodiment according to the invention will be
explained in FIGS. 1, 2A and 2B.
[0089] FIG. 1 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. In FIG.
1, the apparatus for retrieving moving image comprises an
extracting unit 102 for extracting information 103 of the feature
quantity of moving image, which includes physical information of
the feature quantity of moving image such as information of
luminance and color difference (Y, Cb, Cr) of each frame and/or
information of color (R, G, B) of each frame, from moving image
data 101, an extracting unit 104 for arranging at least one
physical information included in the information 103 (such as the
information of the luminance and the color difference (Y, Cb, Cr)
of each frame and/or the information of the color (R, G, B) of each
frame) on the time axis and for extracting information 105 of the
unique identification of moving image to identify the moving image
from the arrangement of the physical information on the time axis,
a storing unit 106 for storing the information 105 of the unique
identification of the moving image extracted by the extracting unit
104, a retrieving unit 109 for retrieving identical information
with the information 105 of the unique identification of the moving
image stored in the storing unit 106 from information 108 of the
unique identification of moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 109, and an
output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110. On the apparatus for
retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0090] FIGS. 2A and 2B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 1. In FIGS. 1, 2A and 2B, at first the
extracting unit 102 for extracting the information of the feature
quantity of the moving image fetches the moving image data 101 to
be a retrieval key for identifying the moving image (in the step
201). Next, the extracting unit 102 extracts the information 103 of
the feature quantity of the moving image, which includes the
physical information of the feature quantity of the moving image
such as the information of the luminance and the color difference
(Y, Cb, Cr) of each frame and/or the information of the color (R,
G, B) of each frame, from the moving image data 101 (in the step
202).
[0091] Next, the extracting unit 104 for extracting the information
of the unique identification of the moving image arranges at least
one value from values of the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103, mean value of the values, total value of the
values and/or difference value of the values on the time axis.
Then, the extracting unit 104 picks out all values arranged on the
time axis, or some values from the values arranged on the time axis
at uniform intervals or irregular intervals. And then, the
extracting unit 104 extracts the picked out values as the
information 105 of the unique identification of the moving image to
be used for identifying the moving image (in the step 203).
[0092] The storing unit 106 for storing the information of the
unique identification of the moving image stores the information
105 of the unique identification of the moving image extracted by
the extracting unit 104 (in the step 204).
[0093] On the other hand, the information 108 of the unique
identification of the moving image is extracted from other moving
image data (the object moving image data for retrieving) (in the
step 205). This extracting process can be executed by the
processing equal to the foregoing steps 201 to 204.
[0094] Then, the retrieving unit 109 for retrieving the information
of the unique identification of the moving image compares the
information 105 of the unique identification of the moving image
stored in the storing unit 106 with the information 108 of the
unique identification of the moving image extracted from the other
moving image data (in the step 206).
[0095] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0096] The process of the above mentioned steps 206 to 209 is
executed for all of the information 108 of the unique
identification of the moving image extracted from the other moving
image data (in the step 210).
[0097] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 104 arranges at least one value from
values of the physical information of the feature quantity of the
moving image (such as the information of the luminance and the
color difference (Y, Cb, Cr) of each frame and the information of
the color (R, G, B) of each frame) based on the information 103 of
the feature quantity of the moving image, mean value of the values,
total value of the values and/or difference value of the values on
the time axis, and picks out all values arranged on the time axis,
or some values from the values arranged on the time axis at uniform
intervals or irregular intervals, and then extracts the picked out
values as the information 105 of the unique identification of the
moving image to be used for identifying the moving image. However,
the above mentioned extracting unit 104 may arrange at least one
value from values of a part or multiple parts of the physical
information of the feature quantity of the moving image (such as
the information of the luminance and the color difference (Y, Cb,
Cr) of each frame and the information of the color (R, G, B) of
each frame) based on the information 103 of the feature quantity of
the moving image, mean value of the values, total value of the
values and/or difference value of the values on the time axis, and
pick out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals, and then extract the picked out values as the
information 105 of the unique identification of the moving image to
be used for identifying the moving image. That is to say, values of
information of feature quantity of moving image in each block unit
may be calculated by dividing a frame into several blocks, and the
information 105 of the unique identification of the moving image
may be extracted by using all of the above mentioned values of the
information of the feature quantity of the moving image. And also,
the information 105 of the unique identification of the moving
image may be extracted by using some values of the information of
the feature quantity of the moving image. As such embodiment, for
example, values of the information of the feature quantity of the
moving image, which are calculated from blocks that are located on
the part except for the part of time display and the part of around
frame (the part with unevenness), are used.
[0098] And more, the extracting unit 104 may calculate values after
executing the weighting on the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103, then may apply the values, mean value of the
values, total value of the values and/or difference value of the
values.
[0099] Moreover, the extracting unit 104 may calculate values of
the physical information of the feature quantity of the moving
image, mean value of the values, total value of the values and/or
difference value of the values, and then may quantize the
calculated values.
[0100] As mentioned above, according to the first embodiment of the
invention, information of the unique identification of moving image
for retrieving moving image depends on the content of the moving
image can be extracted, consequently, identical moving image can be
retrieved and identified from the moving image by using the
information of the unique identification of the moving image at
high speed.
[0101] An apparatus and a method for retrieving moving image in the
second preferred embodiment according to the invention will be
explained in FIGS. 3, 4A and 4B.
[0102] FIG. 3 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 3 with FIG. 1. In FIG. 3, the apparatus for
retrieving moving image comprises an extracting unit 302 for
extracting information 303 of the feature quantity of compressed
moving image, which includes DCT (Discrete Cosine Transform)
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, an extracting unit 304 for arranging at least one information
of feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are
included in the information 303, on the time axis and for
extracting information 305 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis, a storing unit 306 for storing the information 305
of the unique identification of the compressed moving image
extracted by the extracting unit 304, a retrieving unit 309 for
retrieving identical information with the information 305 of the
unique identification of the compressed moving image stored in the
storing unit 306 from information 308 of the unique identification
of compressed moving image beforehand extracted from other moving
image data, a storing unit 110 for storing the retrieving result
provided by the retrieving unit 309, and an output unit 111 for
displaying and/or printing the retrieving result stored in the
storing unit 110. On the apparatus for retrieving the moving image
according to the present invention constituted like the above, an
operation will be explained in the following.
[0103] FIGS. 4A and 4B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 3. Still, an identical step number is
appended to the step having the identical process in FIGS. 4A and
4B with FIGS. 2A and 2B. In FIGS. 3, 4A and 4B, at first the
extracting unit 302 for extracting the information of the feature
quantity of the compressed moving image fetches the compressed
moving image data 301 to be a retrieval key for identifying the
compressed moving image (in the step 401). Next, the extracting
unit 302 extracts the information 303 of the feature quantity of
the compressed moving image, which includes the DCT coefficients of
each frame and the information of the motion compensation of each
frame, from the compressed moving image data 301 (in the step
402).
[0104] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 304 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 304 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 304 extracts the
picked out values as the information 305 of the unique
identification of the compressed moving image to be used for
identifying the compressed moving image (in the step 403).
[0105] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0106] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 401 to 404.
[0107] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0108] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0109] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0110] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 304 calculates values of the physical
information of the feature quantity of the compressed moving image
from the DCT coefficients of each frame, which is included in the
information 303, mean value of the values, total value of the
values and/or difference value of the values, and values such as a
motion vector, a mean motion vector between frames before and
behind, a total motion vector, a difference vector and/or a motion
vector on the whole frame from the information of the motion
compensation. And the above mentioned extracting unit 304 arranges
at least one value from the above mentioned values and then picks
out all values arranged on the time axis, or some values from the
values arranged on the time axis at uniform intervals or irregular
intervals. And then, the above mentioned extracting unit 304
extracts the picked out values as the information 305 of the unique
identification of the compressed moving image to be used for
identifying the compressed moving image. However, the above
mentioned extracting unit 304 may calculate values of a part or
multiple parts of the physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values, and values such as a motion vector, a mean motion vector
between frames before and behind, a total motion vector, a
difference vector and/or a motion vector on the whole frame from a
part or multiple parts of the information of the motion
compensation. That is to say, values of information of feature
quantity of moving image in each block unit may be calculated by
dividing a frame into several blocks, and the information 305 of
the unique identification of the moving image may be extracted by
using all of the above mentioned values of the information of the
feature quantity of the moving image. And also, the information 305
of the unique identification of the moving image may be extracted
by using some values of the information of the feature quantity of
the moving image. As such embodiment, for example, values of the
information of the feature quantity of the moving image, which are
calculated from blocks that are located on the part except for the
part of time display and the part of around frame (the part with
unevenness), are used.
[0111] And more, the extracting unit 304 may execute the weighting
on the physical information of the feature quantity of the
compressed moving image from the DCT coefficients of each frame,
which is included in the information 303, then may calculate values
from the weighted information of the feature quantity of the
compressed moving image, mean value of the values, total value of
the values and/or difference value of the values. On the other
hand, the extracting unit 304 may execute the weighting on the
information of the motion compensation and may calculate a motion
vector, a mean motion vector between frames before and behind, a
total motion vector, a difference vector and/or a motion vector on
a whole frame from the weighted information of the motion
compensation.
[0112] Moreover, in the extracting unit 304, values provided by
quantizing the above mentioned values of the physical information
of the feature quantity of the compressed moving image and the
like, and values provided by quantizing the above mentioned motion
vectors, and the like may be applied.
[0113] As mentioned above, according to the second embodiment of
the invention, information of the unique identification of moving
image for retrieving moving image depends on the content of the
compressed moving image provided by the standard compression
encoding process for moving image such as MPEG can be extracted,
consequently, identical moving image can be retrieved and
identified from the moving image by using the information of the
unique identification of the moving image at high speed.
[0114] An apparatus and a method for retrieving moving image in the
third preferred embodiment according to the invention will be
explained in FIGS. 5, 6A and 6B.
[0115] FIG. 5 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 5 with FIG. 1. In FIG. 5, the apparatus for
retrieving moving image comprises an extracting unit 102 for
extracting information 103 of the feature quantity of moving image,
which includes physical information of the feature quantity of
moving image such as information of luminance and color difference
(Y, Cb, Cr) of each frame and/or information of color (R, G, B) of
each frame, from moving image data 101, a detecting unit 501 for
detecting a cut point of the moving image data 101 as information
502 of a cut point, an extracting unit 503 for arranging at least
one physical information included in the information 103 (such as
the information of the luminance and the color difference (Y, Cb,
Cr) of each frame and/or the information of the color (R, G, B) of
each frame) on the time axis and for extracting information 505 of
the unique identification of moving image to identify the moving
image in each shot unit of the moving image data 101 divided by
using the information 502 of the cut point from the arrangement of
the physical information on the time axis, a storing unit 106 for
storing the information 505 of the unique identification of the
moving image extracted by the extracting unit 503, a retrieving
unit 109 for retrieving identical information with the information
505 of the unique identification of the moving image stored in the
storing unit 106 from information 508 of the unique identification
of moving image beforehand extracted from other moving image data,
a storing unit 110 for storing the retrieving result provided by
the retrieving unit 109, and an output unit 111 for displaying
and/or printing the retrieving result stored in the storing unit
110. On the apparatus for retrieving the moving image according to
the present invention constituted like the above, an operation will
be explained in the following.
[0116] FIGS. 6A and 6B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 5. Still, an identical step number is
appended to the step having the identical process in FIGS. 6A and
6B with FIGS. 2A and 2B. In FIGS. 5, 6A and 6B, at first the
extracting unit 102 for extracting the information of the feature
quantity of the moving image and the detecting unit 501 for
detecting the cut point fetch the moving image data 101 to be a
retrieval key for identifying the moving image (in the step
601).
[0117] Next, the extracting unit 102 extracts the information 103
of the feature quantity of the moving image, which includes the
physical information of the feature quantity of the moving image
such as the information of the luminance and the color difference
(Y, Cb, Cr) of each frame and/or the information of the color (R,
G, B) of each frame, from the moving image data 101 (in the step
202).
[0118] And, the detecting unit 501 detects the cut point of the
moving image data 101 as the information 502 of the cut point (in
the step 602). This cut point of the moving image data 101 shows a
point where the scene of the moving image data 101 changes or a
point where the camera for photographing the moving image data 101
switches.
[0119] Next, the extracting unit 503 for extracting the information
of the unique identification of the moving image arranges at least
one value from values of the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103, mean value of the values, total value of the
values and/or difference value of the values on the time axis.
Then, the extracting unit 503 picks out all values arranged on the
time axis, or some values from the values arranged on the time axis
at uniform intervals or irregular intervals. And then, the
extracting unit 503 extracts the picked out values in each shot
unit of the moving image data 101 divided by the information 502 of
the cut point as the information 505 of the unique identification
of the moving image to be used for identifying the moving image (in
the step 603).
[0120] The storing unit 106 for storing the information of the
unique identification of the moving image stores the information
505 of the unique identification of the moving image extracted by
the extracting unit 503 (in the step 604).
[0121] On the other hand, the information 508 of the unique
identification of the moving image is extracted from other moving
image data (the object moving image data for retrieving) (in the
step 205). This extracting process can be executed by the
processing equal to the foregoing steps 601 to 604.
[0122] Then, the retrieving unit 109 for retrieving the information
of the unique identification of the moving image compares the
information 505 of the unique identification of the moving image
stored in the storing unit 106 with the information 508 of the
unique identification of the moving image extracted from the other
moving image data (in the step 206).
[0123] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0124] The process of the above mentioned steps 206 to 209 is
executed for all of the information 508 of the unique
identification of the moving image extracted from the other moving
image data (in the step 210).
[0125] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 503 for extracting the information of the
unique identification of the moving image arranges at least one
value from values of the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103 of the feature quantity of the moving image, mean
value of the values, total value of the values and/or difference
value of the values on the time axis. However, the extracting unit
503 may arranges at least one value from values of a part or
multiple parts of the physical information of the feature quantity
of the moving image (such as the information of the luminance and
the color difference (Y, Cb, Cr) of each frame and/or the
information of the color (R, G, B) of each frame) based on the
information 103 of the feature quantity of the moving image, mean
value of the values, total value of the values and/or difference
value of the values on the time axis. That is to say, values of
information of feature quantity of moving image in each block unit
may be calculated by dividing a frame into several blocks, and the
information 505 of the unique identification of the moving image
may be extracted by using all of the above mentioned values of the
information of the feature quantity of the moving image. And also,
the information 505 of the unique identification of the moving
image may be extracted by using some values of the information of
the feature quantity of the moving image. As such embodiment, for
example, values of the information of the feature quantity of the
moving image, which are calculated from blocks that are located on
the part except for the part of time display and the part of around
frame (the part with unevenness), are used.
[0126] And more, the extracting unit 503 may calculate values after
executing the weighting on the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103 of the feature quantity of the moving image, then
may apply the values, mean value of the values, total value of the
values and/or difference value of the values.
[0127] Moreover, the extracting unit 503 may calculate values of
the physical information of the feature quantity of the moving
image, mean value of the values, total value of the values and/or
difference value of the values, and then may quantize the
calculated values.
[0128] As mentioned above, according to the third embodiment of the
invention, the information 505 of the unique identification of the
moving image for retrieving the moving image depends on the content
of the moving image can be extracted by the less information
quantity with utilizing the time concept of a shot according to the
information 502 of the cut point. And, identical moving image can
be retrieved and identified from the moving image by using the
information 505 of the unique identification of the moving image at
the higher speed.
[0129] An apparatus and a method for retrieving moving image in the
fourth preferred embodiment according to the invention will be
explained in FIGS. 7, 8A and 8B.
[0130] FIG. 7 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 7 with FIG. 3. In FIG. 7, the apparatus for
retrieving moving image comprises an extracting unit 302 for
extracting information 303 of the feature quantity of compressed
moving image, which includes DCT (Discrete Cosine Transform)
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, a detecting unit 701 for detecting a cut point of the
compressed moving image data 301 as information 702 of a cut point,
an extracting unit 703 for arranging at least one information of
feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are
included in the information 303, on the time axis and for
extracting information 705 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis in each shot unit of the compressed moving image data
301 divided by using the information 702 of the cut point, a
storing unit 306 for storing the information 705 of the unique
identification of the compressed moving image extracted by the
extracting unit 703, a retrieving unit 309 for retrieving identical
information with the information 705 of the unique identification
of the compressed moving image stored in the storing unit 306 from
information 708 of the unique identification of compressed moving
image beforehand extracted from other moving image data, a storing
unit 110 for storing the retrieving result provided by the
retrieving unit 309, and an output unit 111 for displaying and/or
printing the retrieving result stored in the storing unit 110. On
the apparatus for retrieving the moving image according to the
present invention constituted like the above, an operation will be
explained in the following.
[0131] FIGS. 8A and 8B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 7. Still, an identical step number is
appended to the step having the identical process in FIGS. 8A and
8B with FIGS. 4A and 4B. In FIGS. 7, 8A and 8B, at first the
extracting unit 302 for extracting the information of the feature
quantity of the compressed moving image and the detecting unit 701
for detecting the cut point fetch the compressed moving image data
301 to be a retrieval key for identifying the compressed moving
image (in the step 801). Next, the extracting unit 302 extracts the
information 303 of the feature quantity of the compressed moving
image, which includes the DCT coefficients of each frame and the
information of the motion compensation of each frame, from the
compressed moving image data 301 (in the step 402). And, the
detecting unit 701 detects the cut point of the compressed moving
image data 301 as the information 702 of the cut point (in the step
802).
[0132] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 703 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 703 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 703 extracts the
picked out values in each shot unit of the compressed moving image
data 301 divided by using the information 702 of the cut point as
the information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 803).
[0133] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0134] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 801 to 804.
[0135] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0136] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0137] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0138] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 703 calculates values of the physical
information of the feature quantity of the compressed moving image
from the DCT coefficients of each frame, which is included in the
information 303, mean value of the values, total value of the
values and/or difference value of the values, and values such as a
motion vector, a mean motion vector between frames before and
behind, a total motion vector, a difference vector and/or a motion
vector on a whole frame from the information of the motion
compensation. However, the extracting unit 703 may calculate values
of a part or multiple parts of the physical information of the
feature quantity of the compressed moving image from the DCT
coefficients of each frame, which is included in the information
303, mean value of the values, total value of the values and/or
difference value of the values, and values/such as motion vector, a
mean motion vector between frames before and behind, a total motion
vector, a difference vector and/or a motion vector on the whole
frame from a part or multiple parts of the information of the
motion compensation. That is to say, values of information of
feature quantity of moving image in each block unit may be
calculated by dividing a frame into several blocks, and the
information 705 of the unique identification of the moving image
may be extracted by using all of the above mentioned values of the
information of the feature quantity of the moving image. And also,
the information 705 of the unique identification of the moving
image may be extracted by using some values of the information of
the feature quantity of the moving image. As such embodiment, for
example, values of the information of the feature quantity of the
moving image, which are calculated from blocks that are located on
the part except for the part of time display and the part of around
frame (the part with unevenness), are used.
[0139] And more, the extracting unit 703 may execute the weighting
on the physical information of the feature quantity of the
compressed moving image from the DCT coefficients of each frame,
which is included in the information 303, then may calculate the
above mentioned values from the weighted information of the feature
quantity of the compressed moving image. On the other hand, the
extracting unit 703 may execute the weighting on the information of
the motion compensation and may calculate the above mentioned
motion vectors from the weighted information of the motion
compensation.
[0140] Moreover, in the extracting unit 703, values provided by
quantizing the above mentioned values of the physical information
of the feature quantity of the compressed moving image and values
provided by quantizing the above mentioned motion vectors, and the
like may be applied.
[0141] As mentioned above, according to the fourth embodiment of
the invention, the information 705 of the unique identification of
the compressed moving image for retrieving moving image depends on
the content of the compressed moving image provided by the standard
compression encoding process for moving image such as MPEG can be
extracted by the less information quantity with utilizing the time
concept of the shot according to the information 702 of the cut
point. And, identical moving image can be retrieved and identified
from the moving image by using the information 705 of the unique
identification of the compressed moving image at the higher
speed.
[0142] An apparatus and a method for retrieving moving image in the
fifth preferred embodiment according to the invention will be
explained in FIGS. 9, 10A and 10B.
[0143] FIG. 9 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 9 with FIG. 1 and FIG. 3. In FIG. 9, the
apparatus for retrieving moving image comprises an extracting unit
901 for extracting information of motion compensation in the
M.times.N (M and N are natural numbers) pixels unit of moving image
data 101, a DCT (Discrete Cosine Transform) operation unit 902 for
obtaining a DCT coefficient by executing a DCT operation in the
M'.times.N' (M' and N' are natural numbers) pixels unit of the
moving image data 101, an extracting unit 302 for extracting
information 303 of the feature quantity of compressed moving image,
which includes DCT coefficients of each frame and information of
motion compensation of each frame, from compressed moving image
data 301 provided by the standard compression encoding process for
moving image such as MPEG, an extracting unit 304 for arranging at
least one information of feature quantity from the DCT coefficients
of each frame and the information of the motion compensation of
each frame, which are the information 303 extracted by the
extracting unit 302, or by the extracting unit 901 and the DCT
operation unit 902, on the time axis and for extracting information
305 of the unique identification of compressed moving image to
identify the compressed moving image from the arrangement of the
information of feature quantity on the time axis, a storing unit
306 for storing the information 305 of the unique identification of
the compressed moving image extracted by the extracting unit 304, a
retrieving unit 309 for retrieving identical information with the
information 305 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 308 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a storing unit 110 for
storing the retrieving result provided by the retrieving unit 309,
and an output unit 111 for displaying and/or printing the
retrieving result stored in the storing unit 110. On the apparatus
for retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0144] FIGS. 10A and 10B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 9. Still, an identical step number is
appended to the step having the identical process in FIGS. 10A and
10B with FIGS. 4A and 4B. In FIGS. 9, 10A and 10B, at first whether
inputted moving image data is compressed is judged. (in the step
1001).
[0145] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 901 for extracting the
information of the motion compensation and the DCT operation unit
902 fetch the moving image data 101 to be a retrieval key for
identifying the moving image (in the step 1002).
[0146] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the DCT operation unit 902 divides each frame of
the moving image data 101 into the M'.times.N' (M' and N' are
natural numbers) pixels units, then calculates the DCT coefficient
by executing the DCT operation in the M'.times.N' pixels unit (in
the step 1004).
[0147] The information of the motion compensation in the M.times.N
pixels unit provided by the extracting unit 901 and the DCT
coefficient in the M'.times.N' pixels unit provided by the DCT
operation unit 902 are sent to the extracting unit 304 for
extracting the information of the unique identification of the
compressed moving image as the information 303 of the feature
quantity of the compressed moving image.
[0148] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 302 for
extracting the information of the feature quantity of the
compressed moving image fetches the compressed moving image data
301 to be a retrieval key for identifying the compressed moving
image (in the step 401). And the extracting unit 302 extracts the
information 303 of the feature quantity of the compressed moving
image, which includes the DCT coefficients of each frame and the
information of the motion compensation of each frame, from the
compressed moving image data 301 (in the step 402).
[0149] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303 extracted by
extracting unit 302, or the extracting unit 901 and the DCT
operation unit 902, mean value of the values, total value of the
values and/or difference value of the values. On the other hand,
the extracting unit 304 calculates values such as a motion vector,
a mean motion vector between frames before and behind, a total
motion vector, a difference vector and/or a motion vector on a
whole frame from the information of the motion compensation. And
the extracting unit 304 arranges at least one value from the above
mentioned values on the time axis and then picks out all values
arranged on the time axis, or some values from the values arranged
on the time axis at uniform intervals or irregular intervals. And
then, the extracting unit 304 extracts the picked out values as the
information 305 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 403).
[0150] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0151] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404.
[0152] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0153] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0154] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0155] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit and the M'.times.N' pixels unit may be made to be a
value equal to the pixel unit of the standard compression encoding
system such as MPEG.
[0156] As mentioned above, according to the fifth embodiment of the
invention, by executing the preprocessing (the steps 1003 and 1004)
on the uncompressed moving image data 101, the information 305 of
the unique identification of the compressed moving image for
retrieving moving image depends on the content of the moving image
101 and the compressed moving image 301 can be extracted by the
equal processing (the step 403). And identical moving image can be
retrieved and identified from the moving image by using the
information 305 of the unique identification of the compressed
moving image at the higher speed.
[0157] An apparatus and a method for retrieving moving image in the
sixth preferred embodiment according to the invention will be
explained in FIGS. 11, 12A and 12B.
[0158] FIG. 11 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 11 with FIG. 5, FIG. 7 and FIG. 9. In FIG. 11,
the apparatus for retrieving moving image comprises an extracting
unit 901 for extracting information of motion compensation in the M
x N (M and N are natural numbers) pixels unit of moving image data
101, a DCT (Discrete Cosine Transform) operation unit 902 for
obtaining a DCT coefficient by executing a DCT operation in the
M'.times.N' (M' and N' are natural numbers) pixels unit of the
moving image data 101, a detecting unit 501 for detecting a cut
point of the moving image data 101 as information 502 of a cut
point, an extracting unit 302 for extracting information 303 of the
feature quantity of compressed moving image, which includes DCT
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, a detecting unit 701 for detecting a cut point of the
compressed moving image data 301 as information 702 of a cut point,
an extracting unit 703 for arranging at least one information of
feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are
included in the information 303 extracted by the extracting unit
901 and the DCT operation unit 902, or the extracting unit 302, on
the time axis and for extracting information 705 of the unique
identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis in each shot unit of the
compressed moving image data 301 divided by using the information
702 of the cut point, a storing unit 306 for storing the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703, a retrieving
unit 309 for retrieving identical information with the information
705 of the unique identification of the compressed moving image
stored in the storing unit 306 from information 708 of the unique
identification of compressed moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 309, and an
output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110. On the apparatus for
retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0159] FIGS. 12A and 12B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 11. Still, an identical step number is
appended to the step having the identical process in FIGS. 12A and
12B with FIGS. 6A, 6B, 8A, 8B, 10A and 10B. In FIGS. 11, 12A and
12B, at first whether inputted moving image data is compressed is
judged (in the step 1001).
[0160] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 901 for extracting the
information of the motion compensation, the DCT operation unit 902
and the detecting unit 501 for detecting the cut point fetch the
moving image data 101 to be a retrieval key for identifying the
moving image (in the step 1201).
[0161] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the DCT operation unit 902 divides each frame of
the moving image data 101 into the M'.times.N' (M' and N' are
natural numbers) pixels units, then calculates the DCT coefficient
by executing the DCT operation in the M'.times.N' pixels unit (in
the step 1004).
[0162] And, the detecting unit 501 detects the cut point of the
moving image data 101 as the information 502 of the cut point (in
the step 602).
[0163] The information of the motion compensation in the M.times.N
pixels unit provided by the extracting unit 901 and the DCT
coefficient in the M'.times.N' pixels unit provided by the DCT
operation unit 902 are sent to the extracting unit 703 for
extracting the information of the unique identification of the
compressed moving image with the information 502 of the cut point
detected by the detecting unit 501 as the information 303 of the
feature quantity of the compressed moving image.
[0164] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 302 for
extracting the information of the feature quantity of the
compressed moving image and the detecting unit 701 for detecting
the cut point fetch the compressed moving image data 301 to be a
retrieval key for identifying the compressed moving image (in the
step 801). Next, the extracting unit 302 extracts the information
303 of the feature quantity of the compressed moving image, which
includes the DCT coefficients of each frame and the information of
the motion compensation of each frame, from the compressed moving
image data 301 (in the step, 402). And, the detecting unit 701
detects the cut point of the compressed moving image data 301 as
the information 702 of the cut point (in the step 802).
[0165] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 703 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 703 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 703 extracts the
picked out values in each shot unit of the moving image data 101
divided by using the information 502 or in each shot unit of the
compressed moving image data 301 divided by using the information
702 as the information 705 of the unique identification of the
compressed moving image to be used for identifying the compressed
moving image (in the step 803).
[0166] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0167] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804.
[0168] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0169] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0170] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0171] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit and the M'.times.N' pixels unit may be made to be a
value equal to the pixel unit of the standard compression encoding
system such as MPEG.
[0172] As mentioned above, according to the sixth embodiment of the
invention, by executing the preprocessing (the steps 1003 and 1004)
on the uncompressed moving image data 101, the information 705 of
the unique identification of the compressed moving image for
retrieving moving image depends on the content of the moving image
101 and the compressed moving image 301 can be extracted by the
equal processing (the step 803). And, the information 705 of the
unique identification of the compressed moving image for retrieving
moving image can be extracted by the less information quantity with
utilizing the time concept of the shot according to the information
502 and the information 702. Identical moving image can be
retrieved and identified from the moving image by using the
information 705 of the unique identification of the compressed
moving image at the higher speed.
[0173] An apparatus and a method for retrieving moving image in the
seventh preferred embodiment according to the invention will be
explained in FIGS. 13 to 15B.
[0174] FIG. 13 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 13 with FIG. 3 and FIG. 9. In FIG. 13, the
apparatus for retrieving moving image comprises an extracting unit
901 for extracting information of motion compensation in the
M.times.N (M and N are natural numbers) pixels unit of moving image
data 101, an extracting unit 1301 for calculating an average pixel
value, such as information of average color (R, G, B) or
information of average luminance and average color difference (Y,
Cb, Cr), in the M'.times.N' (M' and N' are natural numbers) pixels
unit, an extracting unit 1302 for extracting information 1303 of
the feature quantity of compressed moving image, which includes DC
(Direct Current) components of DCT (Discrete Cosine Transform)
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, an extracting unit 1304 for arranging at least one
information of feature quantity from the DC components of the DCT
coefficients of each frame and the information of the motion
compensation of each frame or the average pixel value, which are
included in the information 1303 extracted by the extracting unit
901 and the extracting unit 1301, or the extracting unit 1302, on
the time axis and for extracting information 305 of the unique
identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis, a storing unit 306 for
storing the information 305 of the unique identification of the
compressed moving image extracted by the extracting unit 1304, a
retrieving unit 309 for retrieving identical information with the
information 305 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 308 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a storing unit 110 for
storing the retrieving result provided by the retrieving unit 309,
and an output unit 111 for displaying and/or printing the
retrieving result stored in the storing unit 110.
[0175] FIG. 14 is a conceptual scheme showing the relationship
between the DCT (Discrete Cosine Transform) operation and the IDCT
(Inverse Discrete Cosine Transform) operation, which are used in
the international standard encoding system such as MPEG, and frame
image.
[0176] The expression of the DCT operation is shown in the
following EXPRESSION 1. F .function. ( u , v ) = { 2 .times. C
.function. ( u ) .times. C .function. ( v ) / N } .times. x = 0 N -
1 .times. y = 0 N - 1 .times. cos .times. { ( 2 .times. x + 1 )
.times. u .times. .times. .pi. / 2 .times. N } .times. cos .times.
{ ( 2 .times. y + 1 ) .times. v .times. .times. .pi. / 2 .times. N
} [ EXPRESSION .times. .times. 1 ] ##EQU1##
[0177] And, the expression of the IDCT operation is shown in the
following EXPRESSION 2. f .function. ( x , y ) = { 2 / N } .times.
u = 0 N - 1 .times. v = 0 N - 1 .times. C .function. ( u ) .times.
C .function. ( v ) .times. cos .times. { ( 2 .times. x + 1 )
.times. u .times. .times. .pi. / 2 .times. N } .times. cos .times.
{ ( 2 .times. y + 1 ) .times. v .times. .times. .pi. / 2 .times. N
} [ EXPRESSION .times. .times. 2 ] ##EQU2##
[0178] However, in the above mentioned EXPRESSIONs 1 and 2,
[0179] "N" is showing a block size of intra-code;
[0180] "x" and "y" are showing a coordinate in a block;
[0181] "f (x,y)" is showing a pixel value of the coordinate (x, y)
in a block;
[0182] "u" and "v" are showing a coordinate after the DCT
conversion;
[0183] "F (u, v)" is showing a DCT coefficient value of the
coordinate (u, v) after the DCT conversion;
[0184] "C (u)" is made 1/ 2, if u=0; [0185] 1, if u.noteq.0;
and
[0186] "C (v)" is made 1/ 2, if v=0; [0187] 1, if v.noteq.0.
[0188] In FIG. 14, for example, in case the block size N of one
frame is 8 and the coordinate (u, v) after the DCT conversion is
(0, 0), F (u, v) in the EXPRESSION 1 is shown in the following
EXPRESSION 3. F .function. ( 0 , 0 ) = ( 1 / 8 ) .times. x = 0 7
.times. y = 0 7 .times. f .function. ( x , y ) [ EXPRESSION .times.
.times. 3 ] ##EQU3##
[0189] And more, an average pixel value in the block is shown in
the following EXPRESSION 4. Average .times. .times. Pixel Value
.times. .times. in .times. .times. the .times. .times. block =
.times. x = 0 7 .times. y = 0 7 .times. f .function. ( x , y ) / 8
.times. 8 .times. .times. = .times. F .function. ( 0 , 0 ) / 8 [
EXPRESSION .times. .times. 4 ] ##EQU4##
[0190] As shown in the above mentioned EXPRESSION 3 and 4, the DC
component F (u, v) of the DCT coefficient becomes a value provided
by multiplying the average pixel value in the block by the
quantization number (N) of the block. That is to say, the DC
components F (u, v) of the DCT coefficients extracted by the
extracting unit 1302 for extracting the information of the feature
quantity of the compressed moving image and the average pixel value
provided by the extracting unit 1301 for extracting the average
value of the block unit may be handled as identical information
data considering the quantization number of the block.
Specifically, the information 1303 extracted by the extracting unit
1302, and the information 1303 depends on the information of the
motion compensation extracted by the extracting unit 901 and the
average pixel value extracted by the extracting unit 1301 may be
considered information of data having an identical content
composition.
[0191] On the apparatus for retrieving the moving image according
to the present invention constituted like the above, an operation
will be explained in the following.
[0192] FIGS. 15A and 15B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 13. Still, an identical step number is
appended to the step having the identical process in FIGS. 15A and
15B with FIGS. 4A, 4B, 10A and 10B. In FIGS. 13, 15A and 15B, at
first whether inputted moving image data is compressed is judged.
(in the step 1001).
[0193] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 901 for extracting the
information of the motion compensation and the extracting unit 1301
for extracting the average value of the block unit fetch the moving
image data 101 to be a retrieval key for identifying the moving
image (in the step 1501).
[0194] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the extracting unit 1301 divides each frame of the
moving image data 101 into the M'.times.N' (M' and N' are natural
numbers) pixels units, then calculates the average pixel value in
the M'.times.N' pixels unit (in the step 1502).
[0195] The information of the motion compensation in the M.times.N
pixels unit provided by the extracting unit 901 and the average
pixel value in the M'.times.N' pixels unit provided by the
extracting unit 1301 are sent to the extracting unit 1304 for
extracting the information of the unique identification of the
compressed moving image as the information 1303 of the feature
quantity of the compressed moving image.
[0196] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 1302 for
extracting the information of the feature quantity of the
compressed moving image fetches the compressed moving image data
301 to be a retrieval key for identifying the compressed moving
image (in the step 401). And the extracting unit 1302 extracts the
information 1303 of the feature quantity of the compressed moving
image, which includes the DC components of the DCT coefficients of
each frame and the information of the motion compensation of each
frame, from the compressed moving image data 301 (in the step
1503).
[0197] Next, the extracting unit 1304 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or values of the average pixel
value of each frame, which are included in the information 1303,
mean value of the values, total value of the values and/or
difference value of the values. On the other hand, the extracting
unit 1304 calculates values such as a motion vector, a mean motion
vector between frames before and behind, a total motion vector, a
difference vector and/or a motion vector on a whole frame from the
information of the motion compensation. And the extracting unit
1304 arranges at least one value from the above mentioned values on
the time axis and then picks out all values arranged on the time
axis, or some values from the values arranged on the time axis at
uniform intervals or irregular intervals. And then, the extracting
unit 1304 extracts the picked out values as the information 305 of
the unique identification of the compressed moving image to be used
for identifying the compressed moving image (in the step 1504).
[0198] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 1304 (in the step
404).
[0199] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404.
[0200] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0201] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0202] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0203] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 1304 calculates values of physical
information of the feature quantity of the compressed moving image
from the DC components of the DCT coefficients of each frame and/or
values of the average pixel value of each frame, which are included
in the information 1303, mean value of the values, total value of
the values and/or difference value of the values. And the
extracting unit 1304 calculates values such as a motion vector, a
mean motion vector between frames before and behind, a total motion
vector, a difference vector and/or a motion vector on a whole frame
from the information of the motion compensation. However, the
extracting unit 1304 may calculate the above mentioned values by
using a part or multiple parts of the physical information of the
feature quantity of the compressed moving image from the DC
components of the DCT coefficients of each frame and/or the average
pixel value of each frame, which are included in the information
1303, and the above mentioned vectors by using a part or multiple
parts of the information of the motion compensation.
[0204] And more, the extracting unit 1304 may execute the weighting
on the physical information of the feature quantity of the
compressed moving image from the DC components of the DCT
coefficients of each frame and/or on the average pixel value of
each frame, then may calculate the above mentioned values from the
weighted information of the feature quantity of the compressed
moving image and/or the weighted average pixel value. On the other
hand, the extracting unit 1304 may execute the weighting on the
information of the motion compensation and may calculate the above
mentioned vectors from the weighted information of the motion
compensation.
[0205] Moreover, in the extracting unit 1304, values provided by
quantizing the above mentioned values of the physical information
or of the average pixel value or the like, and values provided by
quantizing the above mentioned motion vectors and the like may be
applied.
[0206] Further more, the M.times.N pixels unit and the M'.times.N'
pixels unit may be made to be a value equal to the pixel unit of
the standard compression encoding system such as MPEG.
[0207] As mentioned above, according to the seventh embodiment of
the invention, by executing the preprocessing on the moving image,
the information 305 of the unique identification of the compressed
moving image for retrieving the moving image depends on the content
of the moving image data 101 and the compressed moving image data
301 can be extracted by the less information quantity and by
executing the equal process for the moving image data 101 and the
compressed moving image data 301. And, identical moving image can
be retrieved and identified from the moving image by using the
information 305 of the unique identification of the compressed
moving image at the higher speed.
[0208] An apparatus and a method for retrieving moving image in the
eighth preferred embodiment according to the invention will be
explained in FIGS. 16, 17A and 17B.
[0209] FIG. 16 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 16 with FIG. 5, FIG. 7 and FIG. 13. In FIG. 16,
the apparatus for retrieving moving image comprises an extracting
unit 901 for extracting information of motion compensation in the M
x N (M and N are natural numbers) pixels unit of moving image data
101, an extracting unit 1301 for calculating an average pixel
value, such as information of average color (R, G, B) or
information of average luminance and average color difference (Y,
Cb, Cr), in the M'.times.N' (M' and N' are natural numbers) pixels
unit, a detecting unit 501 for detecting a cut point of the moving
image data 101 as information 502 of a cut point, an extracting
unit 1302 for extracting information 1303 of the feature quantity
of compressed moving image, which includes DC (Direct Current)
components of DCT (Discrete Cosine Transform) coefficients of each
frame and information of motion compensation of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, a
detecting unit 701 for detecting a cut point of the compressed
moving image data 301 as information 702 of a cut point, an
extracting unit 1603 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame and the information of the motion compensation of each
frame or the average pixel value, which are included in the
information 1303 extracted by the extracting unit 901 and the
extracting unit 1301, or the extracting unit 1302, on the time axis
and for extracting information 705 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis in each shot unit of the moving image data 101
divided by using the information 502 of the cut point or in each
shot unit of the compressed moving image data 301 divided by using
the information 702 of the cut point, a storing unit 306 for
storing the information 705 of the unique identification of the
compressed moving image extracted by the extracting unit 1603, a
retrieving unit 309 for retrieving identical information with the
information 705 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 708 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a storing unit 110 for
storing the retrieving result provided by the retrieving unit 309,
and an output unit 111 for displaying and/or printing the
retrieving result stored in the storing unit 110. On the apparatus
for retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0210] FIGS. 17A and 17B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 16. Still, an identical step number is
appended to the step having the identical process in FIGS. 17A and
17B with FIGS. 6A, 6B, 8A, 8B, 15A and 15B. In FIGS. 16, 17A and
17B, at first whether inputted moving image data is compressed is
judged (in the step 1001).
[0211] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 901 for extracting the
information of the motion compensation, the extracting unit 1301
for extracting the average value of the block unit and the
detecting unit 501 for detecting the cut point fetch the moving
image data 101 to be a retrieval key for identifying the moving
image (in the step 1701).
[0212] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). And,
the extracting unit 1301 divides each frame of the moving image
data 101 into the M'.times.N' (M' and N' are natural numbers)
pixels units, then calculates the average pixel value in the
M'.times.N' pixels unit (in the step 1502). And more, the detecting
unit 501 detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0213] The information of the motion compensation in the M.times.N
pixels unit provided by the extracting unit 901 and the average
pixel value in the M'.times.N' pixels unit provided by the
extracting unit 1301 are sent to the extracting unit 1603 for
extracting the information of the unique identification of the
compressed moving image with the information 502 of the cut point
detected by the detecting unit 501 as the information 1303 of the
feature quantity of the compressed moving image.
[0214] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 1302 for
extracting the information of the feature quantity of the
compressed moving image and the detecting unit 701 for detecting
the cut point fetch the compressed moving image data 301 to be a
retrieval key for identifying the compressed moving image (in the
step 801). And the extracting unit 1302 extracts the information
1303 of the feature quantity of the compressed moving image, which
includes the DC components of the DCT coefficients of each frame
and the information of the motion compensation of each frame, from
the compressed moving image data 301 (in the step 1503). And then,
the detecting unit 701 detects the cut point of the compressed
moving image data 301 as the information of the cut point 702 (in
the step 802).
[0215] Next, the extracting unit 1603 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of the compressed moving image from the DC components of
the DCT coefficients of each frame and/or values of the average
pixel value of each frame, which are included in the information
1303, mean value of the values, total value of the values and/or
difference value of the values. On the other hand, the extracting
unit 1603 calculates values such as a motion vector, a mean motion
vector between frames before and behind, a total motion vector, a
difference vector and/or a motion vector on a whole frame from the
information of the motion compensation. And the extracting unit
1603 arranges at least one value from the above mentioned values on
the time axis and then picks out all values arranged on the time
axis, or some values from the values arranged on the time axis at
uniform intervals or irregular intervals. And then, the extracting
unit 1603 extracts the picked out values in each shot unit of the
moving image data 101 divided by using the information 502 of the
cut point or in each shot unit of the compressed moving image data
301 divided by using the information 702 of the cut point as the
information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 1702).
[0216] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 1603 (in the step
804).
[0217] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804.
[0218] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0219] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0220] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0221] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit and the M'.times.N' pixels unit may be made to be a
value equal to the pixel unit of the standard compression encoding
system such as MPEG.
[0222] As mentioned above, according to the eighth embodiment of
the invention, by utilizing the time concept of the shot according
to the information of the cut point and by executing the
preprocessing on the moving image, information 1303 of the unique
identification of the compressed moving image for retrieving moving
image depends on the content of the moving image data 101 and the
compressed moving image data 301 can be extracted by the less
information quantity and by executing the equal process for the
moving image data 101 and the compressed moving image data 301.
And, identical moving image can be retrieved and identified from
the moving image by using the information 1303 of the unique
identification of the compressed moving image at the higher
speed.
[0223] An apparatus and a method for retrieving moving image in the
ninth preferred embodiment according to the invention will be
explained in FIGS. 18, 19A and 19B.
[0224] FIG. 18 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 18 with FIG. 3 and FIG. 9. In FIG. 18, the
apparatus for retrieving moving image comprises a DCT (Discrete
Cosine Transform) operation unit 902 for obtaining a DCT
coefficient by executing a DCT operation in the M.times.N (M and N
are natural numbers) pixels unit of moving image data 101 as
information 303 of the feature quantity of compressed moving image,
an extracting unit 302 for extracting information 303 of the
feature quantity of compressed moving image, which includes the DCT
coefficients of each frame, from compressed moving image data 301
provided by the standard compression encoding process for moving
image such as MPEG, an extracting unit 304 for arranging at least
one information of feature quantity from the DCT coefficients of
each frame, which is included in the information 303 extracted by
the DCT operation unit 902 or by the extracting unit 302, on the
time axis and for extracting information 305 of the unique
identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
feature quantity on the time axis, a storing unit 306 for storing
the information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304, a retrieving
unit 309 for retrieving identical information with the information
305 of the unique identification of the compressed moving image
stored in the storing unit 306 from information 308 of the unique
identification of compressed moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 309, and an
output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110. On the apparatus for
retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0225] FIGS. 19A and 19B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 18. Still, an identical step number is
appended to the step having the identical process in FIGS. 19A and
19B with FIGS. 10A and 10B. In FIGS. 18, 19A and 19B, at first
whether inputted moving image data is compressed is judged. (in the
step 1001).
[0226] In case the inputted moving image data is the uncompressed
moving image data 101, the DCT operation unit 902 fetches the
moving image data 101 to be a retrieval key for identifying the
moving image (in the step 1901).
[0227] The DCT operation unit 902 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the DCT coefficient by executing the
DCT operation in the M.times.N pixels unit as the information 303
of the feature quantity of the compressed moving image (in the step
1902).
[0228] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 302 for
extracting the information of the feature quantity of the
compressed moving image fetches the compressed moving image data
301 to be a retrieval key for identifying the compressed moving
image (in the step 401). And the extracting unit 302 extracts the
information 303 of the feature quantity of the compressed moving
image, which includes the DCT coefficients of each frame, from the
compressed moving image data 301 as (in the step 1903).
[0229] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303 extracted by the
extracting unit 902, mean value of the values, total value of the
values and/or difference value of the values. And the extracting
unit 304 arranges at least one value from the above mentioned
values on the time axis and then picks out all values arranged on
the time axis, or some values from the values arranged on the time
axis at uniform intervals or irregular intervals. And then, the
extracting unit 304 extracts the picked out values as the
information 305 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 1904).
[0230] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0231] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404.
[0232] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0233] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0234] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0235] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit may be made to be a value equal to the pixel unit of
the standard compression encoding system such as MPEG.
[0236] As mentioned above, according to the ninth embodiment of the
invention, by executing the preprocessing on the moving image 101,
the information 305 of the unique identification of the compressed
moving image for retrieving moving image depends on the content of
the moving image data 101 and the compressed moving image data 301
can be extracted by executing the equal process. And, identical
moving image can be retrieved and identified from the moving image
by using the information 305 of the unique identification of the
compressed moving image at the higher speed.
[0237] An apparatus and a method for retrieving moving image in the
tenth preferred embodiment according to the invention will be
explained in FIGS. 20, 21A and 21B.
[0238] FIG. 20 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 20 with FIG. 11. In FIG. 20, the apparatus for
retrieving moving image comprises a DCT (Discrete Cosine Transform)
operation unit 902 for obtaining a DCT coefficient as information
303 of the feature quantity of compressed moving image by executing
a DCT operation in the M.times.N (M and N are natural numbers)
pixels unit of moving image data 101, a detecting unit 501 for
detecting a cut point of the moving image data 101 as information
502 of a cut point, an extracting unit 302 for extracting
information 303 of the feature quantity of compressed moving image,
which includes DCT coefficients of each frame, from compressed
moving image data 301 provided by the standard compression encoding
process for moving image such as MPEG, a detecting unit 701 for
detecting a cut point of the compressed moving image data 301 as
information 702 of a cut point, an extracting unit 703 for
arranging at least one information of feature quantity from the DCT
coefficients of each frame, which is included in the information
303 extracted by the DCT operation unit 902 or the extracting unit
302, on the time axis and for extracting information 705 of the
unique identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis in each shot unit of the
compressed moving image data 301 divided by using the information
502 of the cut point and/or the information 702 of the cut point, a
storing unit 306 for storing the information 705 of the unique
identification of the compressed moving image extracted by the
extracting unit 703, a retrieving unit 309 for retrieving identical
information with the information 705 of the unique identification
of the compressed moving image stored in the storing unit 306 from
information 708 of the unique identification of compressed moving
image beforehand extracted from other moving image data, a storing
unit 110 for storing the retrieving result provided by the
retrieving unit 309, and an output unit 111 for displaying and/or
printing the retrieving result stored in the storing unit 110. On
the apparatus for retrieving the moving image according to the
present invention constituted like the above, an operation will be
explained in the following.
[0239] FIGS. 21A and 21B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 20. Still, an identical step number is
appended to the step having the identical process in FIGS. 21A and
21B with FIGS. 12A, 12B, 19A and 19B. In FIGS. 20, 21A and 21B, at
first whether inputted moving image data is compressed is judged.
(in the step 1001).
[0240] In case the inputted moving image data is the uncompressed
moving image data 101, the DCT operation unit 902 and the detecting
unit 501 for detecting the cut point fetch the moving image data
101 to be a retrieval key for identifying the moving image (in the
step 2101).
[0241] The DCT operation unit 902 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the DCT coefficient by executing the
DCT operation in the M.times.N pixels unit as the information 303
of the feature quantity of the compressed moving image (in the step
1902). On the other hand, the detecting unit 501 for detecting the
cut point detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0242] The DCT coefficient in the M.times.N pixels unit provides by
the DCT operation unit 902 is sent to the extracting unit 703 for
extracting the information of the unique identification of the
compressed moving image with the information 502 detected by the
detecting unit 501 as the information 303 of the feature quantity
of the compressed moving image.
[0243] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 302 for
extracting the information of the feature quantity of the
compressed moving image and the detecting unit 701 for detecting
the cut point fetch the compressed moving image data 301 to be a
retrieval key for identifying the compressed moving image (in the
step 801). Next, the extracting unit 302 extracts the information
303 of the feature quantity of the compressed moving image, which
includes the DCT coefficients of each frame, from the compressed
moving image data 301 (in the step 1903). And, the detecting unit
701 detects the cut point of the compressed moving image data 301
as the information 702 of the cut point (in the step 802).
[0244] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. And the extracting unit 703 arranges at least one value
from the above mentioned values on the time axis and then picks out
all values arranged on the time axis, or some values from the
values arranged on the time axis at uniform intervals or irregular
intervals. And then, the extracting unit 703 extracts the picked
out values in each shot unit of the moving image data 101 divided
by using the information 502 or in each shot unit of the compressed
moving image data 301 divided by using the information 702 as the
information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 2102).
[0245] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0246] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804.
[0247] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0248] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0249] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0250] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit may be made to be a value equal to the pixel unit of
the standard compression encoding system such as MPEG.
[0251] As mentioned above, according to the tenth embodiment of the
invention, by executing the preprocessing on the uncompressed
moving image data 101, the information 705 of the unique
identification of the compressed moving image for retrieving moving
image depends on the content of the moving image 101 and the
compressed moving image 301 can be extracted by the equal
processing. And, the information 705 for retrieving moving image
can be extracted by the less information quantity with utilizing
the time concept of the shot according to the information 502 and
the information 702. Identical moving image can be retrieved and
identified from the moving image by using the information 705 of
the unique identification of the compressed moving image at the
higher speed.
[0252] An apparatus and a method for retrieving moving image in the
eleventh preferred embodiment according to the invention will be
explained in FIGS. 22, 23A and 23B.
[0253] FIG. 22 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 22 with FIG. 13. In FIG. 22, the apparatus for
retrieving moving image comprises an extracting unit 1301 for
calculating an average pixel value, such as information of average
color (R, G, B) or information of average luminance and average
color difference (Y, Cb, Cr), in the M.times.N (M and N are natural
numbers) pixels unit, an extracting unit 1302 for extracting
information 1303 of the feature quantity of compressed moving
image, which includes DC (Direct Current) components of DCT
(Discrete Cosine Transform) coefficients of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, an
extracting unit 1304 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame or the average pixel value, which is included in the
information 1303 extracted by the extracting unit 1301 and/or the
extracting unit 1302, on the time axis and for extracting
information 305 of the unique identification of compressed moving
image to identify compressed moving image from the arrangement of
the information of the feature quantity on the time axis, a storing
unit 306 for storing the information 305 of the unique
identification of the compressed moving image extracted by the
extracting unit 1304, a retrieving unit 309 for retrieving
identical information with the information 305 of the unique
identification of the compressed moving image stored in the storing
unit 306 from information 308 of the unique identification of
compressed moving image beforehand extracted from other moving
image data, a storing unit 110 for storing the retrieving result
provided by the retrieving unit 309, and an output unit 111 for
displaying and/or printing the retrieving result stored in the
storing unit 110. On the apparatus for retrieving the moving image
according to the present invention constituted like the above, an
operation will be explained in the following.
[0254] FIGS. 23A and 23B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 22. Still, an identical step number is
appended to the step having the identical process in FIGS. 23A and
23B with FIGS. 15A and 15B. In FIGS. 22, 23A and 23B, at first
whether inputted moving image data is compressed is judged. (in the
step 1001).
[0255] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 1301 for extracting the
average value of the block unit fetches the moving image data 101
to be a retrieval key for identifying the moving image (in the step
2301).
[0256] The extracting unit 1301 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the average pixel value in the
M.times.N pixels unit as the information 1303 of the feature
quantity of the compressed moving image (in the step 1502).
[0257] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 1302 for
extracting the information of the feature quantity of the
compressed moving image fetches the compressed moving image data
301 to be a retrieval key for identifying the compressed moving
image (in the step 401). Then, the extracting unit 1302 extracts
the information 1303 of the feature quantity of the compressed
moving image, which includes the DC components of the DCT
coefficients of each frame, from the compressed moving image data
301 (in the step 1503).
[0258] Next, the extracting unit 1304 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or values of the average pixel
value of each frame, which are included in the information 1303,
mean value of the values, total value of the values and i or
difference value of the values. And the extracting unit 1304
arranges at least one value from the above mentioned values on the
time axis and then picks out all values arranged on the time axis,
or some values from the values arranged on the time axis at uniform
intervals or irregular intervals. And then, the extracting unit
1304 extracts the picked out values as the information 305 of the
unique identification of the compressed moving image to be used for
identifying the compressed moving image (in the step 1504).
[0259] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 1304 (in the step
404).
[0260] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404.
[0261] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0262] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or sprinting the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0263] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0264] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The above
mentioned extracting unit 1304 calculates values of the physical
information of the feature quantity of the compressed moving image
from the DC components of the DCT coefficients of each frame or
values of the average pixel value of each frame, which are included
in the information 1303, mean value of the values, total value of
the values and/ or difference value of the values. However, the
extracting unit 1304 may calculate the above mentioned values by
using a part or multiple parts of the physical information of the
feature quantity of the compressed moving image from the DC
components of the DCT coefficients of each frame or the average
pixel value of each frame, which are included in the information
1303.
[0265] And more, the extracting unit 1304 may execute the weighting
on the physical information of the feature quantity of the
compressed moving image from the DC components of the DCT
coefficients of each frame or the average pixel value of each
frame, which are included in the information 1303, then may
calculate the above mentioned values by the weighted information of
the feature quantity of the compressed moving image or the weighted
average pixel value.
[0266] Moreover, in the extracting unit 1304, values provided by
quantizing the above mentioned physical information or average
pixel value or the like may be applied.
[0267] Further more, the M.times.N pixels unit may be made to be a
value equal to the pixel unit of the standard compression encoding
system such as MPEG.
[0268] As mentioned above, according to the eleventh embodiment of
the invention, by executing the preprocessing on the moving image,
the information 305 of the unique identification of the compressed
moving image for retrieving the moving image depends on the content
of the moving image data 101 and the compressed moving image data
301 can be extracted by the less information quantity and by
executing the equal process for the moving image data 101 and the
compressed moving image data 301. And, identical moving image can
be retrieved and identified from the moving image by using the
information 305 of the unique identification of the compressed
moving image at the higher speed.
[0269] An apparatus and a method for retrieving moving image in the
twelfth preferred embodiment according to the invention will be
explained in FIGS. 24, 25A and 25B.
[0270] FIG. 24 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 24 with FIG. 16. In FIG. 24, the apparatus for
retrieving moving image comprises an extracting unit 1301 for
calculating an average pixel value, such as information of average
color (R, G, B) or information of average luminance and average
color difference (Y, Cb, Cr), in the M.times.N (M and N are natural
numbers) pixels unit of moving image data 101 as information 1303
of the feature quantity of compressed moving image, a detecting
unit 501 for detecting a cut point of the moving image data 101 as
information 502 of a cut point, an extracting unit 1302 for
extracting information 1303 of the feature quantity of compressed
moving image, which includes DC (Direct Current) components of DCT
(Discrete Cosine Transform) coefficients of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, a
detecting unit 701 for detecting a cut point of the compressed
moving image data 301 as information 702 of a cut point, an
extracting unit 1603 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame or the average pixel value, which is included in the
information 1303 extracted by the extracting unit 1301 and/or the
extracting unit 1302, on the time axis and for extracting
information 705 of the unique identification of compressed moving
image to identify the compressed moving image from the arrangement
of the information of the feature quantity on the time axis in each
shot unit of the moving image data 101 divided by using the
information 502 of the cut point or in each shot unit of the
compressed moving image data 301 divided by using the information
702 of the cut point, a storing unit 306 for storing the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 1603, a retrieving
unit 309 for retrieving identical information with the information
705 of the unique identification of the compressed moving image
stored in the storing unit 306 from information 708 of the unique
identification of compressed moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 309, and an
output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110. On the apparatus for
retrieving the moving image according to the present invention
constituted like the above, an operation will be explained in the
following.
[0271] FIGS. 25A and 25B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 24. Still, an identical step number is
appended to the step having the identical process in FIGS. 25A and
25B with FIGS. 17A and 17B. In FIGS. 24, 25A and 25B, at first
whether inputted moving image data is compressed is judged (in the
step 1001).
[0272] In case the inputted moving image data is the uncompressed
moving image data 101, the extracting unit 1301 for extracting the
average value of the block unit and the detecting unit 501 for
detecting the cut point fetch the moving image data 101 to be a
retrieval key for identifying the moving image (in the step
2501).
[0273] The extracting unit 1301 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the average pixel value in the
M.times.N pixels unit (in the step 1502). And, the detecting unit
501 detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0274] On the other hand, in case the inputted moving image data is
the compressed moving image data 301, the extracting unit 1302 for
extracting the information of the feature quantity of the
compressed moving image and the detecting unit 701 for detecting
the cut point fetch the compressed moving image data 301 to be a
retrieval key for identifying the compressed moving image (in the
step 801). Then, the extracting unit 1302 extracts the information
1303.of the feature quantity of the compressed moving image, which
includes the DC components of the DCT coefficients of each frame,
from the compressed moving image data 301 (in the step 1503). And,
the detecting unit 701 detests the cut point of the compressed
moving image data 301 as the information 702 of the cut point (in
the step 802).
[0275] Next, the extracting unit 1603 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or the average pixel value of
each frame, which are included in the information 1303, mean value
of the values, total value of the values and/or difference value of
the values. And the extracting unit 1603 arranges at least one
value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 1603 extracts
the picked out values in each shot unit of the moving image data
101 divided by using the information 502 of the cut point or in
each shot unit of the compressed moving image data 301 divided by
using the information 702 of the cut point as the information 705
of the unique identification of the compressed moving image to be
used for identifying the compressed moving image (in the step
1702).
[0276] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 1603 (in the step
804).
[0277] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804.
[0278] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0279] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0280] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0281] The apparatus and the method for retrieving the moving image
according to the invention are explained the above. The M.times.N
pixels unit may be made to be a value equal to the pixel unit of
the standard compression encoding system such as MPEG.
[0282] As mentioned above, according to the twelfth embodiment of
the invention, by utilizing the time concept of the shot according
to the information of the cut point and executing the simple
preprocessing on the moving image, the information 1303 of the
unique identification of the compressed moving image for retrieving
the moving image depends on the content of the moving image data
101 and the compressed moving image data 301 can be extracted by
the less information quantity and by executing the equal process
for the moving image data 101 and the compressed moving image data
301. And, identical moving image can be retrieved and identified
from the moving image by using the information 1303 of the unique
identification of the compressed moving image at the higher
speed.
[0283] An apparatus and a method for retrieving moving image in the
thirteenth preferred embodiment according to the invention will be
explained in FIGS. 26, 27A and 27B.
[0284] FIG. 26 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 26 with FIG. 1 and FIG. 3. In FIG. 26, the
apparatus for retrieving moving image comprises an extracting unit
102 for extracting information 103 of the feature quantity of
moving image, which includes physical information of the feature
quantity of moving image such as information of luminance and color
difference (Y, Cb, Cr) of each frame and/or information of color
(R, G, B) of each frame, from moving image data 101, an extracting
unit 104 for arranging at least one physical information included
in the information 103 (such as the information of the luminance
and the color difference (Y, Cb, Cr) of each frame and/or the
information of the color (R, G, B) of each frame) on the time axis
and for extracting information 105 of the unique identification of
moving image to identify the moving image from the arrangement of
the physical information on the time axis, a storing unit 106 for
storing the information 105 of the unique identification of the
moving image extracted by the extracting unit 104, a retrieving
unit 109 for retrieving identical information with the information
105 of the unique identification of the moving image stored in the
storing unit 106 from information 108 of the unique identification
of moving image beforehand extracted from other moving image data,
an extracting unit 302 for extracting information 303 of the
feature quantity of compressed moving image, which includes DCT
(Discrete Cosine Transform) coefficients of each frame and
information of motion compensation of each frame, from compressed
moving image data 301 provided by the standard compression encoding
process for moving image such as MPEG, an extracting unit 304 for
arranging at least one information of feature quantity from the DCT
coefficients of each frame and the information of the motion
compensation of each frame, which are included in the information
303, on the time axis and for extracting information 305 of the
unique identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis, a storing unit 306 for
storing the information 305 of the unique identification of the
compressed moving image extracted by the extracting unit 304, a
retrieving unit 309 for retrieving identical information with the
information 305 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 308 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a judging unit 2602 for
judging whether inputted digital moving image data 2601 is the
moving image data 101 or the compressed moving image data 301 and
for outputting the moving image data 101 to the extracting unit 102
and the compressed moving image data 301 to the extracting unit
302, a storing unit 110 for storing a retrieving result provided by
the retrieving unit 109 and/or by the retrieving unit 309, and an
output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110.
[0285] FIGS. 27A and 27B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 26. Still, an identical step number is
appended to the step having the identical process in FIGS. 27A and
27B with FIGS. 2A, 2B, 4A and 4B. In FIGS. 26, 27A and 27B, at
first the compressed/uncompressed judging unit 2602 acquires the
digital moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0286] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the following process (the
steps 202 to 210) is executed.
[0287] The extracting unit 102 extracts the information 103 of the
feature quantity of the moving image, which includes the physical
information of the feature quantity of the moving image such as the
information of the luminance and the color difference (Y, Cb, Cr)
of each frame and/or the information of the color (R, G, B) of each
frame, from the moving image data 101 (in the step 202).
[0288] Next, the extracting unit 104 for extracting the information
of the unique identification of the moving image arranges at least
one value from values of the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103, mean value of the values, total value of the
values and/or difference value of the values on the time axis.
Then, the extracting unit 104 picks out all values arranged on the
time axis, or some values from the values arranged on the time axis
at uniform intervals or irregular intervals. And then, the
extracting unit 104 extracts the picked out values as the
information 105 of the unique identification of the moving image to
be used for identifying the moving image (in the step 203).
[0289] The storing unit 106 for storing the information of the
unique identification of the moving image stores the information
105 of the unique identification of the moving image extracted by
the extracting unit 104 (in the step 204).
[0290] On the other hand, the information 108 of the unique
identification of the moving image is extracted from other moving
image data (the object moving image data for retrieving) (in the
step 205). This extracting process can be executed by the
processing equal to the foregoing steps 201 to 204 (in FIGS. 2A and
2B).
[0291] Then, the retrieving unit 109 for retrieving the information
of the unique identification of the moving image compares the
information 105 of the unique identification of the moving image
stored in the storing unit 106 with the information 108 of the
unique identification of the moving image extracted from the other
moving image data (in the step 206).
[0292] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0293] The process of the above mentioned steps 206 to 209 is
executed for all of the information 108 of the unique
identification of the moving image extracted from the other moving
image data (in the step 210).
[0294] In case the digital moving image data 2601 is the compressed
moving image data 301, the following process (the steps 402 to 410)
is executed.
[0295] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame and the information of the motion
compensation of each frame, from the compressed moving image data
301 (in the step 402).
[0296] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 304 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 304 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 304 extracts the
picked out values as the information 305 of the unique
identification of the compressed moving image to be used for
identifying the compressed moving image (in the step 403).
[0297] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0298] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 401 to 404 (in FIGS. 4A and 4B).
[0299] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0300] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0301] The process of the above mentioned steps 406 to 209.is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0302] As mentioned above, according to the thirteenth embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of image data to be retrieved.
[0303] An apparatus and a method for retrieving moving image in the
fourteenth preferred embodiment according to the invention will be
explained in FIGS. 28, 29A and 29B.
[0304] FIG. 28 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 28 with FIG. 5 and FIG. 7. In FIG. 28, the
apparatus for retrieving moving image comprises an extracting unit
102 for extracting information 103 of the feature quantity of
moving image, which includes physical information of the feature
quantity of moving image such as information of luminance and color
difference (Y, Cb, Cr) of each frame and/or information of color
(R, G, B) of each frame, from moving image data 101, a detecting
unit 501 for detecting a cut point of the moving image data 101 as
information 502 of a cut point, an extracting unit 503 for
arranging at least one physical information included in the
information 103 (such as the information of the luminance and the
color difference (Y, Cb, Cr) of each frame and/or the information
of the color (R, G, B) of each frame) on the time axis and for
extracting information 505 of the unique identification of moving
image to identify the moving image in each shot unit of the moving
image data 101 divided by using the information 502 of the cut
point from the arrangement of the physical information on the time
axis, a storing unit 106 for storing the information 505 of the
unique identification of the moving image extracted by the
extracting unit 503, a retrieving unit 109 for retrieving identical
information with the information 505 of the unique identification
of the moving image stored in the storing unit 106 from information
508 of the unique identification of moving image beforehand
extracted from other moving image data, an extracting unit 302 for
extracting information 303 of the feature quantity of compressed
moving image, which includes DCT (Discrete Cosine Transform)
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, a detecting unit 701 for detecting a cut point of the
compressed moving image data 301 as information 702 of a cut point,
an extracting unit 703 for arranging at least one information of
feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are
included in the information 303, on the time axis and for
extracting information 705 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis in each shot unit of the compressed moving image data
301 divided by using the information 702 of the cut point, a
storing unit 306 for storing the information 705 of the unique
identification of the compressed moving image extracted by the
extracting unit 703, a retrieving unit 309 for retrieving identical
information with the information 705 of the unique identification
of the compressed moving image stored in the storing unit 306 from
information 708 of the unique identification of compressed moving
image beforehand extracted from other moving image data, a judging
unit 2602 for judging whether inputted digital moving image data
2601 is the moving image data 101 or the compressed moving image
data 301 and for outputting the moving image data 101 to the
extracting unit 102 and the compressed moving image data 301 to the
extracting unit 302, a storing unit 110 for storing the retrieving
result provided by the retrieving unit 109 and/or the retrieving
unit 309, and an output unit 111 for displaying and/or printing the
retrieving result stored in the storing unit 110.
[0305] FIGS. 29A and 29B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 28. Still, an identical step number is
appended to the step having the identical process in FIGS. 29A and
29B with FIGS. 6A, 6B, 8A and 8B. In FIGS. 28, 29A and 29B, at
first the compressed/uncompressed judging unit 2602 acquires the
digital moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0306] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the following process (the
steps 202 to 210) is executed.
[0307] The extracting unit 102 extracts the information 103 of the
feature quantity of the moving image, which includes the physical
information of the feature quantity of the moving image such as the
information of the luminance and the color difference (Y, Cb, Cr)
of each frame and/or the information of the color (R, G, B) of each
frame, from the moving image data 101 (in the step 202).
[0308] And, the detecting unit 501 detects the cut point of the
moving image data 101 as the information 502 of the cut point (in
the step 602). This cut point of the moving image data 101 shows a
point where the scene of the moving image data 101 changes or a
point where the camera for photographing the moving image data 101
switches.
[0309] Next, the extracting unit 503 for extracting the information
of the unique identification of the moving image arranges at least
one value from values of the physical information of the feature
quantity of the moving image (such as the information of the
luminance and the color difference (Y, Cb, Cr) of each frame and/or
the information of the color (R, G, B) of each frame) based on the
information 103, mean value of the values, total value of the
values and/or difference value of the values on the time axis.
Then, the extracting unit 503 picks out all values arranged on the
time axis, or some values from the values arranged on the time axis
at uniform intervals or irregular intervals. And then, the
extracting unit 503 extracts the picked out values in each shot
unit of the moving image data 101 divided by the information 502 of
the cut point as the information 505 of the unique identification
of the moving image to be used for identifying the moving image (in
the step 603).
[0310] The storing unit 106 for storing the information of the
unique identification of the moving image stores the information
505 of the unique identification of the moving image extracted by
the extracting unit 503 (in the step 604).
[0311] On the other hand, the information 508 of the unique
identification of the moving image is extracted from other moving
image data (the object moving image data for retrieving) (in the
step 205). This extracting process can be executed by the
processing equal to the foregoing steps 601 to 604 (in FIGS. 6A and
6B).
[0312] Then, the retrieving unit 109 for retrieving the information
of the unique identification of the moving image compares the
information 505 of the unique identification of the moving image
stored in the storing unit 106 with the information 508 of the
unique identification of the moving image extracted from the other
moving image data (in the step 206).
[0313] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or~printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0314] The process of the above mentioned steps 206 to 209 is
executed for all of the information 508 of the unique
identification of the moving image extracted from the other moving
image data (in the step 210).
[0315] In case the digital moving image data 2601 is the compressed
moving image data 301, the following process (the steps 402 to 410)
is executed.
[0316] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame and the information of the motion
compensation of each frame, from the compressed moving image data
301 (in the step 402). And, the detecting unit 701 detects the cut
point of the compressed moving image data 301 as the information
702 of the cut point (in the step 802).
[0317] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 703 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 703 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 703 extracts the
picked out values in each shot unit of the compressed moving image
data 301 divided by using the information 702 of the cut point as
the information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 803).
[0318] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0319] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 801 to 804 (in FIGS. 8A and 8B).
[0320] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0321] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0322] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0323] As mentioned above, according to the fourteenth embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0324] An apparatus and a method for retrieving moving image in the
fifteenth preferred embodiment according to the invention will be
explained in FIGS. 30, 31A and 31B.
[0325] FIG. 30 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 30 with FIG. 9. In FIG. 30, the apparatus for
retrieving moving image comprises an extracting unit 901 for
extracting information of motion compensation in the M.times.N (M
and N are natural numbers) pixels unit of moving image data 101, a
DCT (Discrete Cosine Transform) operation unit 902 for obtaining a
DCT coefficient by executing a DCT operation in the M'.times.N' (M'
and N' are natural numbers) pixels unit of the moving image data
101, an extracting unit 302 for extracting information 303 of the
feature quantity of compressed moving image, which includes DCT
coefficients of each frame and information of motion compensation
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, an extracting unit 304 for arranging at least one information
of feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are the
information 303 extracted by the extracting unit 302, or by the
extracting unit 901 and the DCT operation unit 902, on the time
axis and for extracting information 305 of the unique
identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
feature quantity on the time axis, a storing unit 306 for storing
the information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304, a retrieving
unit 309 for retrieving identical information with the information
305 of the unique identification of the compressed moving image
stored in the storing unit 306 from information 308 of the unique
identification of compressed moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 309, an output
unit 111 for displaying and/or printing the retrieving result
stored in the storing unit 110, and a judging unit 2602 for judging
whether inputted digital moving image data 2601 is the moving image
data 101 or the compressed moving image data 301 and for outputting
the moving image data 101 to the extracting unit 102 and the
compressed moving image data 301 to the extracting unit.
[0326] FIGS. 31A and 31B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 30. Still, an identical step number is
appended to the step having the identical process in FIGS. 31A and
31B with FIGS. 10A and 10B. In FIGS. 30, 31A and 31B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0327] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 901 for extracting the information of the motion
compensation and the DCT operation unit 902. And, the following
process (the steps 1003 to 410) is executed.
[0328] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the DCT operation unit 902 divides each frame of
the moving image data 101 into the M'.times.N' (M' and N' are
natural numbers) pixels units, then calculates the DCT coefficient
by executing the DCT operation in the M'.times.N' pixels unit (in
the step 1004).
[0329] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 302 for extracting the
information of the feature quantity of the compressed moving image.
And, the following process (the steps 402 to 410) is executed.
[0330] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame and the information of the motion
compensation of each frame, from the compressed moving image data
301 (in the step 402).
[0331] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303 extracted by
extracting unit 302, or the extracting unit 901 and the DCT
operation unit 902, mean value of the values, total value of the
values and/or difference value of the values. On the other hand,
the extracting unit 304 calculates values such as a motion vector,
a mean motion vector between frames before and behind, a total
motion vector, a difference vector and/or a motion vector on a
whole frame from the information of the motion compensation. And
the extracting unit 304 arranges at least one value from the above
mentioned values on the time axis and then picks out all values
arranged on the time axis, or some values from the values arranged
on the time axis at uniform intervals or irregular intervals. And
then, the extracting unit 304 extracts the picked out values as the
information 305 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 403).
[0332] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0333] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404 (in FIGS. 10A and 10B).
[0334] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0335] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0336] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0337] As mentioned above, according to the fifteenth embodiment of
the invention, whether the inputted digital moving image data 2601
is the uncompressed moving image data 101 or the compressed moving
image data 301 can be judged automatically. Consequently, identical
moving image can be retrieved and identified from the moving image
at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0338] An apparatus and a method for retrieving moving image in the
sixteenth preferred embodiment according to the invention will be
explained in FIGS. 32, 33A and 33B.
[0339] FIG. 32 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 32 with FIG. 11. In FIG. 32, the apparatus for
retrieving moving image comprises an extracting unit 901 for
extracting information of motion compensation in the M.times.N (M
and N are natural numbers) pixels unit of moving image data 101, a
DCT (Discrete Cosine Transform) operation unit 902 for obtaining a
DCT coefficient by executing a DCT operation in the M'.times.N' (M'
and N' are natural numbers) pixels unit of the moving image data
101, a detecting unit 501 for detecting a cut point of the moving
image data 101 as information 502 of a cut point, an extracting
unit 302 for extracting information 303 of the feature quantity of
compressed moving image, which includes DCT coefficients of each
frame and information of motion compensation of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, a
detecting unit 701 for detecting a cut point of the compressed
moving image data 301 as information 702 of a cut point, an
extracting unit 703 for arranging at least one information of
feature quantity from the DCT coefficients of each frame and the
information of the motion compensation of each frame, which are
included in the information 303 extracted by the extracting unit
901 and the DCT operation unit 902, or the extracting unit 302, on
the time axis and for extracting information 705 of the unique
identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis in each shot unit of the
compressed moving image data 301 divided by using the information
702 of the cut point, a storing unit 306 for storing the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703, a retrieving
unit 309 for retrieving identical information with the information
705 of the unique identification of the compressed moving image
stored in the storing unit 306 from information 708 of the unique
identification of compressed moving image beforehand extracted from
other moving image data, a storing unit 110 for storing the
retrieving result provided by the retrieving unit 309, an output
unit 111 for displaying and/or printing the retrieving result
stored in the storing unit 110, and a judging unit 2602 for judging
whether inputted digital moving image data 2601 is the moving image
data 101 or the compressed moving image data 301 and for outputting
the moving image data 101 to the extracting unit, the DCT (Discrete
Cosine Transform) operation unit 902 and the detecting unit, and
the compressed moving image data 301 to the extracting unit 302 and
the detecting unit 701.
[0340] FIGS. 33A and 33B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 32. Still, an identical step number is
appended to the step having the identical process in FIGS. 33A and
33B with FIGS. 12A and 12B. In FIGS. 32, 33A and 33B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0341] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 901 for extracting the information of the motion
compensation, the DCT operation unit 902 and the detecting unit 501
for detecting the cut point. And, the following process (the steps
1003 to 410) is executed.
[0342] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the DCT operation unit 902 divides each frame of
the moving image data 101 into the M'.times.N' (M' and N' are
natural numbers) pixels units, then calculates the DCT coefficient
by executing the DCT operation in the M'.times.N' pixels unit (in
the step 1004).
[0343] And, the detecting unit 501 detects the cut point of the
moving image data 101 as the information 502 of the cut point (in
the step 602).
[0344] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 302 for extracting the
information of the feature quantity of the compressed moving image
and the detecting unit 701 for detecting the cut point. And, the
following process (the steps 402 to 410) is executed.
[0345] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame and the information of the motion
compensation of each frame, from the compressed moving image data
301 (in the step 402). And, the detecting unit 701 detects the cut
point of the compressed moving image data 301 as the information
702 of the cut point (in the step 802).
[0346] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. On the other hand, the extracting unit 703 calculates
values such as a motion vector, a mean motion vector between frames
before and behind, a total motion vector, a difference vector
and/or a motion vector on a whole frame from the information of the
motion compensation. And the extracting unit 703 arranges at least
one value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 703 extracts the
picked out values in each shot unit of the moving image data 101
divided by using the information 502 or in each shot unit of the
compressed moving image data 301 divided by using the information
702 as the information 705 of the unique identification of the
compressed moving image to be used for identifying the compressed
moving image (in the step 803).
[0347] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0348] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804 (in FIGS. 12A and 12B).
[0349] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0350] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0351] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0352] As mentioned above, according to the sixteenth embodiment of
the invention, whether the inputted digital moving image data 2601
is the uncompressed moving image data 101 or the compressed moving
image data 301 can be judged automatically. Consequently, identical
moving image can be retrieved and identified from the moving image
at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0353] An apparatus and a method for retrieving moving image in the
seventeenth preferred embodiment according to the invention will be
explained in FIGS. 34, 35A and 35B.
[0354] FIG. 34 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 34 with FIG. 13. In FIG. 34, the apparatus for
retrieving moving image comprises an extracting unit 901 for
extracting information of motion compensation in the M.times.N (M
and N are natural numbers) pixels unit of moving image data 101, an
extracting unit 1301 for calculating an average pixel value, such
as information of average color (R, G, B), and/or information of
average luminance and average color difference (Y, Cb, Cr), in the
M'.times.N' (M' and N' are natural numbers) pixels unit, an
extracting unit 1302 for extracting information 1303 of the feature
quantity of compressed moving image, which includes DC (Direct
Current) components of DCT (Discrete Cosine Transform) coefficients
of each frame and information of motion compensation of each frame,
from compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, an
extracting unit 1304 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame and the information of the motion compensation of each
frame or the average pixel value, which are included in the
information 1303 extracted by the extracting unit 901 and the
extracting unit 1301, or the extracting unit 1302, on the time axis
and for extracting information 305 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis, a storing unit 306 for storing the information 305
of the unique identification of the compressed moving image
extracted by the extracting unit 1304, a retrieving unit 309 for
retrieving identical information with the information 305 of the
unique identification of the compressed moving image stored in the
storing unit 306 from information 308 of the unique identification
of compressed moving image beforehand extracted from other moving
image data, a storing unit 110 for storing the retrieving result
provided by the retrieving unit 309, an output unit 111 for
displaying and/or printing the retrieving result stored in the
storing unit 110, and a judging unit 2602 for judging whether
inputted digital moving image data 2601 is the moving image data
101 or the compressed moving image data 301 and for outputting the
moving image data 101 to the extracting unit 901 and the extracting
unit 1301, and the compressed moving image data 301 to the
extracting unit 1302.
[0355] FIGS. 35A and 35B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 34. Still, an identical step number is
appended to the step having the identical process in FIGS. 35A and
35B with FIGS. 15A and 15B. In FIGS. 34, 35A and 35B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0356] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 901 for extracting the information of the motion
compensation and the extracting unit 1301 for extracting the
average value of the block unit. And, the following process (the
steps 1003 to 410) is executed.
[0357] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). On
the other hand, the extracting unit 1301 divides each frame of the
moving image data 101 into the M'.times.N' (M' and N' are natural
numbers) pixels units, then calculates the average pixel value in
the M'.times.N' pixels unit (in the step 1502).
[0358] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 1302 for extracting
the information of the feature quantity of the compressed moving
image. And, the following process (the steps 1503 to 410) is
executed.
[0359] The extracting unit 1302 extracts the information 1303 of
the feature quantity of the compressed moving image, which includes
the DC components of the DCT coefficients of each frame and the
information of the motion compensation of each frame, from the
compressed moving image data 301 (in the step 1503).
[0360] Next, the extracting unit 1304 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or values of the average pixel
value of each frame, which are included in the information 1303,
mean value of the values, total value of the values and/or
difference value of the values. On the other hand, the extracting
unit 1304 calculates values such as a motion vector, a mean motion
vector between frames before and behind, a total motion vector, a
difference vector and/or a motion vector on a whole frame from the
information of the motion compensation. And the extracting unit
1304 arranges at least one value from the above mentioned values on
the time axis and then picks out all values arranged on the time
axis, or some values from the values arranged on the time axis at
uniform intervals or irregular intervals. And then, the extracting
unit 1304 extracts the picked out values as the information 305 of
the unique identification of the compressed moving image to be used
for identifying the compressed moving image (in the step 1504).
[0361] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 1304 (in the step
404).
[0362] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404 (in FIGS. 15A and 15B).
[0363] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0364] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0365] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image *99 extracted from
the other moving image data (in the step 410).
[0366] As mentioned above, according to the seventeenth embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0367] An apparatus and a method for retrieving moving image in the
eighteenth preferred embodiment according to the invention will be
explained in FIGS. 36, 37A and 37B.
[0368] FIG. 36 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 36 with FIG. 16. In FIG. 36, the apparatus for
retrieving moving image comprises an extracting unit 901 for
extracting information of motion compensation in the M.times.N (M
and N are natural numbers) pixels unit of moving image data 101, an
extracting unit 1301 for calculating an average pixel value such as
information of average color (R, G, B) and/or information of
average luminance and average color difference (Y, Cb, Cr) in the
M'.times.N' (M' and N' are natural numbers) pixels unit, a
detecting unit 501 for detecting a cut point of the moving image
data 101 as information 502 of a cut point, an extracting unit 1302
for extracting information 1303 of the feature quantity of
compressed moving image, which includes DC (Direct Current)
components of DCT (Discrete Cosine Transform) coefficients of each
frame and information of motion compensation of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, a
detecting unit 701 for detecting a cut point of the compressed
moving image data 301 as information 702 of a cut point, an
extracting unit 1603 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame and the information of the motion compensation of each
frame or the average pixel values, which are included in the
information 1303 extracted by the extracting unit 901 and the
extracting unit 1301, or the extracting unit 1302, on the time axis
and for extracting information 705 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis in each shot unit of the moving image data 101
divided by using the information 502 of the cut point or in each
shot unit of the compressed moving image data 301 divided by using
the information 702 of the cut point, a storing unit 306 for
storing the information 705 of the unique identification of the
compressed moving image extracted by the extracting unit 1603, a
retrieving unit 309 for retrieving identical information with the
information 705 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 708 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a storing unit 110 for
storing the retrieving result provided by the retrieving unit 309,
an output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110, and a judging unit 2602 for
judging whether inputted digital moving image data 2601 is the
moving image data 101 or the compressed moving image data 301 and
for outputting the moving image data 101 to the extracting unit
901, the extracting unit 1301 and the detecting unit 501, and the
compressed moving image data 301 to the extracting unit 1302 and
the detecting unit 701.
[0369] FIGS. 37A and 37B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 36. Still, an identical step number is
appended to the step having the identical process in FIGS. 37A and
37B with FIGS. 17A and 17B. In FIGS. 36, 37A and 37B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0370] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 901 for extracting the information of the motion
compensation, the extracting unit 1301 for extracting the average
value of the block unit and the detecting unit 501 for detecting
the cut point. And, the following process (the steps 1003 to 410)
is executed.
[0371] The extracting unit 901 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the information of the motion
compensation in the M.times.N pixels unit (in the step 1003). And,
the extracting unit 1301 divides each frame of the moving image
data 101 into the M'.times.N' (M' and N' are natural numbers)
pixels units, then calculates the average pixel value in the
M'.times.N' pixels unit (in the step 1502). And more, the detecting
unit 501 detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0372] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 1302 for extracting
the information of the feature quantity of the compressed moving
image and the detecting unit 701 for detecting the cut point. And,
the following process (the steps 1503 to 410) is executed.
[0373] The extracting unit 1302 extracts the information 1303 of
the feature quantity of the compressed moving image, which includes
the DC components of the DCT coefficients of each frame and the
information of the motion compensation of each frame, from the
compressed moving image data 301 (in the step 1503). And then, the
detecting unit 701 detects the cut point of the compressed moving
image data 301 as the information of the cut point 702 (in the step
802).
[0374] Next, the extracting unit 1603 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of the compressed moving image from the DC components of
the DCT coefficients of each frame and/or values of the average
pixel value of each frame, which are included in the information
1303, mean value of the values, total value of the values and/or
difference value of the values. On the other hand, the extracting
unit 1603 calculates values such as a motion vector, a mean motion
vector between frames before and behind, a total motion vector, a
difference vector and/or a motion vector on a whole frame from the
information of the motion compensation. And the extracting unit
1603 arranges at least one value from the above mentioned values on
the time axis and then picks out all values arranged on the time
axis, or some values from the values arranged on the time axis at
uniform intervals or irregular intervals. And then, the extracting
unit 1603 extracts the picked out values in each shot unit of the
moving image data 101 divided by using the information 502 of the
cut point or in each shot unit of the compressed moving image data
301 divided by using the information 702 of the cut point as the
information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 1702).
[0375] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 1603 (in the step
804).
[0376] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804 (in FIGS. 17A and 17B).
[0377] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0378] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0379] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0380] As mentioned above, according to the eighteenth embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0381] An apparatus and a method for retrieving moving image in the
nineteenth preferred embodiment according to the invention will be
explained in FIGS. 38, 39A and 39B.
[0382] FIG. 38 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 38 with FIG. 18. In FIG. 38, the apparatus for
retrieving moving image comprises a DCT (.Discrete Cosine
Transform) operation unit 902 for obtaining a DCT coefficient by
executing a DCT operation in the M.times.N (M and N are natural
numbers) pixels unit of moving image data 101 as information 303 of
the feature quantity of compressed moving image, an extracting unit
302 for extracting information 303 of the feature quantity of
compressed moving image, which includes DCT coefficients of each
frame, from compressed moving image data 301 provided by the
standard compression encoding process for moving image such as
MPEG, an extracting unit 304 for arranging at least one information
of feature quantity from the DCT coefficients of each frame, which
is included in the information 303 extracted by the DCT operation
unit 902 or by the extracting unit 302, on the time axis and for
extracting information 305 of the unique identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of feature quantity on the
time axis, a storing unit 306 for storing the information 305 of
the unique identification of the compressed moving image extracted
by the extracting unit 304, a retrieving unit 309 for retrieving
identical information with the information 305 of the unique
identification of the compressed moving image stored in the storing
unit 306 from information 308 of the unique identification of
compressed moving image beforehand extracted from other moving
image data, a storing unit 110 for storing the retrieving result
provided by the retrieving unit 309, an output unit 111 for
displaying and/or printing the retrieving result stored in the
storing unit 110, and a judging unit 2602 for judging whether
inputted digital moving image data 2601 is the moving image data
101 or the compressed moving image data 301 and for outputting the
moving image data 101 to the DCT (Discrete Cosine Transform)
operation unit 902 and the compressed moving image data 301 to the
extracting unit 302.
[0383] FIGS. 39A and 39B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 38. Still, an identical step number is
appended to the step having the identical process in FIGS. 39A and
39B with FIGS. 19A and 19B. In FIGS. 38, 39A and 39B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0384] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the DCT
operation unit 902. The following process (the steps 1902 to 410)
is executed.
[0385] The DCT operation unit 902 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the DCT coefficient by executing the
DCT operation in the M.times.N pixels unit as the information 303
of the feature quantity of the compressed moving image (in the step
1902).
[0386] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 302 for extracting the
information of the feature quantity of the compressed moving image.
The following process (the steps 1903 to 410) is executed.
[0387] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame, from the compressed moving image
data 301 as (in the step 1903).
[0388] Next, the extracting unit 304 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303 extracted by the
extracting unit 902, mean value of the values, total value of the
values and/or difference value of the values. And the extracting
unit 304 arranges at least one value from the above mentioned
values on the time axis and then picks out all values arranged on
the time axis, or some values from the values arranged on the time
axis at uniform intervals or irregular intervals. And then, the
extracting unit 304 extracts the picked out values as the
information 305 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 1994).
[0389] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 304 (in the step
404).
[0390] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404 (in FIGS. 19A and 19B).
[0391] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0392] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0393] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0394] As mentioned above, according to the nineteenth embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0395] An apparatus and a method for retrieving moving image in the
twentieth preferred embodiment according to the invention will be
explained in FIGS. 40, 41A and 41B.
[0396] FIG. 40 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 40 with FIG. 20. In FIG. 40, the apparatus for
retrieving moving image comprises a DCT (Discrete Cosine Transform)
operation unit 902 for obtaining a DCT coefficient as information
303 of the feature quantity of compressed moving image by executing
a DCT operation in the M.times.N (M and N are natural numbers)
pixels unit of moving image data 101, a detecting unit 501 for
detecting a cut point of the moving image data 101 as information
502 of a cut point, an extracting unit 302 for extracting
information 303 of the feature quantity of compressed moving image,
which includes DCT coefficients of each frame, from compressed
moving image data 301 provided by the standard compression encoding
process for moving image such as MPEG, a detecting unit 701 for
detecting a cut point of the compressed moving image data 301 as
information 702 of a cut point, an extracting unit 703 for
arranging at least one information of feature quantity from the DCT
coefficients of each frame, which is included in the information
303 extracted by the DCT operation unit 902 or the extracting unit
302, on the time axis and for extracting information 705 of the
unique identification of compressed moving image to identify the
compressed moving image from the arrangement of the information of
the feature quantity on the time axis in each shot unit of the
compressed moving image data 301 divided by using the information
502 of the cut point and/or the information 702 of the cut point, a
storing unit 306 for storing the information 705 of the unique
identification of the compressed moving image extracted by the
extracting unit 703, a retrieving unit 309 for retrieving identical
information with the information 705 of the unique identification
of the compressed moving image stored in the storing unit 306 from
information 708 of the unique identification of compressed moving
image beforehand extracted from other moving image data, a storing
unit 110 for storing the retrieving result provided by the
retrieving unit 309, an output unit 111 for displaying and/or
printing the retrieving result stored in the storing unit 110, and
a judging unit 2602 for judging whether inputted digital moving
image data 2601 is the moving image data 101 or the compressed
moving image data 301 and for outputting the moving image data 101
to the DCT (Discrete Cosine Transform) operation unit 902 and the
detecting unit 501, and the compressed moving image data 301 to the
extracting unit 302 and the detecting unit 701.
[0397] FIGS. 41A and 41B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 40. Still, an identical step number is
appended to the step having the identical process in FIGS. 41A and
41B with FIGS. 21A and 21B. In FIGS. 40, 41A and 41B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0398] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the DCT
operation unit 902 and the detecting unit 501 for detecting the cut
point. And, the following process (the steps 1902 to 410) is
executed.
[0399] The DCT operation unit 902 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the DCT coefficient by executing the
DCT operation in the M.times.N pixels unit as the information 303
of the feature quantity of the compressed moving image (in the step
1902). On the other hand, the detecting unit 501 for detecting the
cut point detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0400] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 302 for extracting the
information of the feature quantity of the compressed moving image
and the detecting unit 701 for detecting the cut point. The
following process (the steps 1903 to 410) is executed.
[0401] The extracting unit 302 extracts the information 303 of the
feature quantity of the compressed moving image, which includes the
DCT coefficients of each frame, from the compressed moving image
data 301 (in the step 1903). And, the detecting unit 701 detects
the cut point of the compressed moving image data 301 as the
information 702 of the cut point (in the step 802).
[0402] Next, the extracting unit 703 for extracting the information
of the unique identification of the compressed moving image
calculates values of physical information of the feature quantity
of the compressed moving image from the DCT coefficients of each
frame, which is included in the information 303, mean value of the
values, total value of the values and/or difference value of the
values. And the extracting unit 703 arranges at least one value
from the above mentioned values on the time axis and then picks out
all values arranged on the time axis, or some values from the
values arranged on the time axis at uniform intervals or irregular
intervals. And then, the extracting unit 703 extracts the picked
out values in each shot unit of the moving image data 101 divided
by using the information 502 or in each shot unit of the compressed
moving image data 301 divided by using the information 702 as the
information 705 of the unique identification of the compressed
moving image to be used for identifying the compressed moving image
(in the step 2102).
[0403] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 703 (in the step
804).
[0404] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804 (in FIGS. 21A and 21B).
[0405] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0406] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0407] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0408] As mentioned above, according to the twentieth embodiment of
the invention, whether the inputted digital moving image data 2601
is the uncompressed moving image data 101 or the compressed moving
image data 301 can be judged automatically. Consequently, identical
moving image can be retrieved and identified from the moving image
at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0409] An apparatus and a method for retrieving moving image in the
twenty-first preferred embodiment according to the invention will
be explained in FIGS. 42, 43A and 43B.
[0410] FIG. 42 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 42 with FIG. 22. In FIG. 42, the apparatus for
retrieving moving image comprises an extracting unit 1301 for
calculating an average pixel value, such as information of average
color (R, G, B), and/or information of average luminance and
average color difference (Y, Cb, Cr), in the M.times.N (M and N are
natural numbers) pixels unit, an extracting unit 1302 for
extracting information 1303 of the feature quantity of compressed
moving image, which includes DC (Direct Current) components of DCT
(Discrete Cosine Transform) coefficients of each frame, from
compressed moving image data 301 provided by the standard
compression encoding process for moving image such as MPEG, an
extracting unit 1304 for arranging at least one information of
feature quantity from the DC components of the DCT coefficients of
each frame or the average pixel value, which is included in the
information 1303 extracted by the extracting unit 1301 and/or the
extracting unit 1302, on the time axis and for extracting
information 305 of the unique identification of compressed moving
image to identify compressed moving image from the arrangement of
the information of the feature quantity on the time axis, a storing
unit 306 for storing the information 305 of the unique
identification of the compressed moving image extracted by the
extracting unit 1304, a retrieving unit 309 for retrieving
identical information with the information 305 of the unique
identification of the compressed moving image stored in the storing
unit 306 from information 308 of the unique identification of
compressed moving image beforehand extracted from other moving
image data, a storing unit 110 for storing the retrieving result
provided by the retrieving unit 309, an output unit 111 for
displaying and/or printing the retrieving result stored in the
storing unit 110, and a judging unit 2602 for judging whether
inputted digital moving image data 2601 is the moving image data
101 or the compressed moving image data 301 and for outputting the
moving image data 101 to the extracting unit 1301 and the
compressed moving image data 301 to the extracting unit 1302.
[0411] FIGS. 43A and 43B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 42. Still, an identical step number is
appended to the step having the identical process in FIGS. 43A and
43B with FIGS. 23A and 23B. In FIGS. 42, 43A and 43B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0412] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 1301 for extracting the average value of the block
unit. The following process (the steps 1502 to 410) is
executed.
[0413] The extracting unit 1301 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the average pixel value in the
M.times.N pixels unit as the information 1303 of the feature
quantity of the compressed moving image (in the step 1502).
[0414] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 1302 for extracting
the information of the feature quantity of the compressed moving
image. The following process (the steps 1503 to 410) is
executed.
[0415] The extracting unit 1302 extracts the information 1303 of
the feature quantity of the compressed moving image, which includes
the DC components of the DCT coefficients of each frame, from the
compressed moving image data 301 (in the step 1503).
[0416] Next, the extracting unit 1304 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or values of the average pixel
value of each frame, which are included in the information 1303,
mean value of the values, total value of the values and/or
difference value of the values. And the extracting unit 1304
arranges at least one value from the above mentioned values on the
time axis and then picks out all values arranged on the time axis,
or some values from the values arranged on the time axis at uniform
intervals or irregular intervals. And then, the extracting unit
1304 extracts the picked out values as the information 305 of the
unique identification of the compressed moving image to be used for
identifying the compressed moving image (in the step 1504).
[0417] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305 of the unique identification of the compressed
moving image extracted by the extracting unit 1304 (in the step
404).
[0418] On the other hand, the information 308 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 404 (in FIGS. 23A and 23B).
[0419] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0420] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0421] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0422] As mentioned above, according to the twenty-first embodiment
of the invention, whether the inputted digital moving image data
2601 is the uncompressed moving image data 101 or the compressed
moving image data 301 can be judged automatically. Consequently,
identical moving image can be retrieved and identified from the
moving image at the higher speed regardless of the type
(compressed/uncompressed) of retrieving image data.
[0423] An apparatus and a method for retrieving moving image in the
twenty-second preferred embodiment according to the invention will
be explained in FIGS. 44, 45A and 45B.
[0424] FIG. 44 is a block diagram showing an apparatus for
retrieving moving image according to the present invention. Still,
an identical code is appended to the thing having the identical
composition in FIG. 44 with FIG. 24. In FIG. 44, the apparatus for
retrieving moving image comprises an extracting unit 1301 for
calculating an average pixel value such as information of average
color (R, G, B) 5 and/or information of average luminance and
average color difference (Y, Cb, Cr) in the M.times.N (M and N are
natural numbers) pixels unit of moving image data 101 as
information 1303 of the feature quantity of compressed moving
image, a detecting unit 501 for detecting a cut point of the moving
image data 101 as information 502 of a cut point, an extracting
unit 1302 for extracting 10 information 1303 of the feature
quantity of compressed moving image, which includes DC (Direct
Current) components of DCT (Discrete Cosine Transform) coefficients
of each frame, from compressed moving image data 301 provided by
the standard compression encoding process for moving image such as
MPEG, a detecting unit 701 for detecting a cut point of the
compressed moving image data 15 301 as information 702 of a cut
point, an extracting unit 1603 for arranging at least one
information of feature quantity from the DC components of the DCT
coefficients of each frame or the average pixel value, which is
included in the information 1303 extracted by the extracting unit
1301 and/or the extracting unit 1302, on the time axis and for
extracting information 705 of the unique 20 identification of
compressed moving image to identify the compressed moving image
from the arrangement of the information of the feature quantity on
the time axis in each shot unit of the moving image data 101
divided by using the information 502 of the cut point or in each
shot unit of the compressed moving image data 301 divided by using
the information 702 of the cut point, a storing 25 unit 306 for
storing the information 705 of the unique identification of the
compressed moving image extracted by the extracting unit 1603, a
retrieving unit 309 for retrieving identical information with the
information 705 of the unique identification of the compressed
moving image stored in the storing unit 306 from information 708 of
the unique identification of compressed moving image beforehand
extracted from other moving image data, a storing unit 110 for
storing the retrieving result provided by the retrieving unit 309,
an output unit 111 for displaying and/or printing the retrieving
result stored in the storing unit 110, and a judging unit 2602 for
judging whether inputted digital moving image data 2601 is the
moving image data 101 or the compressed moving image data 301 and
for outputting the moving image data 101 to the extracting unit and
the detecting unit 501, and the compressed moving image data 301 to
the extracting unit 1302 and the detecting unit 701.
[0425] FIGS. 45A and 45B are flowcharts of the process in the
apparatus for retrieving the moving image according to the present
invention described in FIG. 44. Still, an identical step number is
appended to the step having the identical process in FIGS. 45A and
45B with FIGS. 25A and 25B. In FIGS. 44, 45A and 45B, at first the
compressed/uncompressed judging unit 2602 acquires the digital
moving image data 2601 (in the step 2701). Then, the
compressed/uncompressed judging unit 2602 judges whether the
acquired digital moving image data 2601 is the uncompressed moving
image data 101 or the compressed moving image data 301 (in the step
2702).
[0426] In case the digital moving image data 2601 is the
uncompressed moving image data 101, the compressed/uncompressed
judging unit 2602 outputs the moving image data 101 to the
extracting unit 1301 for extracting the average value of the block
unit and the detecting unit 501 for detecting the cut point. And,
the following process (the steps 1502 to 410) is executed.
[0427] The extracting unit 1301 divides each frame of the moving
image data 101 into the M.times.N (M and N are natural numbers)
pixels units, then calculates the average pixel value in the
M.times.N pixels unit (in the step 1502). And, the detecting unit
501 detects the cut point of the moving image data 101 as the
information 502 of the cut point (in the step 602).
[0428] On the other hand, in case the digital moving image data
2601 is the compressed moving image data 301, the
compressed/uncompressed judging unit 2602 outputs the compressed
moving image data 301 to the extracting unit 1302 for extracting
the information of the feature quantity of the compressed moving
image and the detecting unit 701 for detecting the cut point. And,
the following process (the steps 1503 to 410) is executed.
[0429] The extracting unit 1302 extracts the information 1303 of
the feature quantity of the compressed moving image, which includes
the DC components of the DCT coefficients of each frame, from the
compressed moving image data 301 (in the step 1503). And, the
detecting unit 701 detects the cut point of the compressed moving
image data 301 as the information 702 of the cut point (in the step
802).
[0430] Next, the extracting unit 1603 for extracting the
information of the unique identification of the compressed moving
image calculates values of physical information of the feature
quantity of compressed moving image from the DC components of the
DCT coefficients of each frame and/or the average pixel value of
each frame, which are included in the information 1303, mean value
of the values, total value of the values and/or difference value of
the values. And the extracting unit 1603 arranges at least one
value from the above mentioned values on the time axis and then
picks out all values arranged on the time axis, or some values from
the values arranged on the time axis at uniform intervals or
irregular intervals. And then, the extracting unit 1603 extracts
the picked out values in each shot unit of the moving image data
101 divided by using the information 502 of the cut point or in
each shot unit of the compressed moving image data 301 divided by
using the information 702 of the cut point as the information 705
of the unique identification of the compressed moving image to be
used for identifying the compressed moving image (in the step
1702).
[0431] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 705 of the unique identification of the compressed
moving image extracted by the extracting unit 1603 (in the step
804).
[0432] On the other hand, the information 708 of the unique
identification of the compressed moving image is extracted from
other compressed moving image data (the object compressed moving
image data for retrieving) (in the step 405). This extracting
process can be executed by the processing equal to the foregoing
steps 1001 to 804 (in FIGS. 25A and 25B).
[0433] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 705 of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 708 of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0434] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0435] The process of the above mentioned steps 406 to 209 is
executed for all of the information 708 of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0436] As mentioned above, according to the twenty-second
embodiment of the invention, whether the inputted digital moving
image data 2601 is the uncompressed moving image data 101 or the
compressed moving image data 301 can be judged automatically.
Consequently, identical moving image can be retrieved and
identified from the moving image at the higher speed regardless of
the type (compressed/uncompressed) of retrieving image data.
[0437] An apparatus and a method for retrieving moving image in the
twenty-third preferred embodiment according to the invention will
be explained in FIGS. 46, 47A and 47B.
[0438] FIG. 46 is a block diagram showing an apparatus for
monitoring television commercials, which is a specific example of
the second embodiment according to the present invention. Still, an
identical code is appended to the thing having the identical
composition in FIG. 46 with FIG. 3.
[0439] In FIG. 46, the apparatus for monitoring television
commercials comprises an extracting unit 302A for extracting
information (a DC component) 303A of the feature quantity of
compressed moving image, which includes the only DC (Direct
Current) components of DCT (Discrete Cosine Transform) coefficients
of each frame, from MPEG-2 stream (television broadcasting) 301A
compressed by MPEG-2 process, an extracting unit (a calculating
unit for calculating average color information) 304A for arranging
an average value of the DC components of each frame, which is
included in the information 303A extracted by the extracting unit
302A, on the time axis and for extracting the arrangement of the
average values on the time axis as information (time series
information of average color information) 305A of the unique
identification of compressed moving image to identify the
compressed moving image, a storing unit 306 for storing the
information 305A of the unique identification of the compressed
moving image extracted by the extracting unit 304A, a retrieving
unit 309 for retrieving identical information with the information
305A of the unique identification of the compressed moving image
stored in the storing unit 306 from information (time series
information of average color information of a television
commercial) 308A of the unique identification of compressed moving
image beforehand extracted from other moving image data, a storing
unit 110 for storing the retrieving result provided by the
retrieving unit 309, and an output unit 111 for displaying and/or
printing the retrieving result stored in the storing unit 110.
[0440] The MPEG-2 stream (the television broadcasting) 301A is
provided by encoding the moving image of the current television
broadcasting.
[0441] On the apparatus for monitoring television commercials
according to the present invention constituted like the above, an
operation will be explained in the following.
[0442] FIGS. 47A and 47B are flowcharts of the process in the
apparatus for monitoring television commercials to the present
invention described in FIG. 46. Still, an identical step number is
appended to the step having the identical process in FIGS. 47A and
47B with FIGS. 4A and 4B. In FIGS. 46, 47A and 47B, at first the
extracting unit 302A for extracting the DC component fetches the
MPEG-2 stream (the television broadcasting) 301A to be a retrieval
key for identifying the compressed moving image (in the step 401A).
Then, the extracting unit 302A extracts the information (the DC
component) 303A of the feature quantity of the compressed moving
image, which includes the only DC components of the DCT
coefficients of each frame, from the MPEG-2 stream 301A (in the
step 402A).
[0443] Next, the extracting unit (the calculating unit for
calculating the average color information) 304A for extracting the
information of the unique identification of the compressed moving
image calculates an average value by using the DC components of
each frame, which is included in the information 303A. Then, the
extracting unit 304A arranges the average values on the time axis
and extracts the arrangement of the average values on the time axis
as the information (the time series information of the average
color information) 305A of the unique identification of the
compressed moving image to be used for identifying the compressed
moving image (in the step 403A).
[0444] The storing unit 306 for storing the information of the
unique identification of the compressed moving image stores the
information 305A extracted by the extracting unit 304A (in the step
404).
[0445] In the twenty-third embodiment, the information (the time
series information of the average color information) 305A of the
unique identification of the compressed moving image consists of
three kinds of time series information, which are information of
luminance and color difference (Y, Cb, Cr). By making a graph of
each time series information, which it makes the horizontal line to
be time and makes the vertical line to be a value of the average
color information, the information 305A is shown by three
graphs.
[0446] On the other hand, the information (the time series
information of the average color information of the television
commercial) 308A of the unique identification of the compressed
moving image is extracted from the MPEG-2 stream of the television
commercial (the object compressed moving image data for retrieving)
(in the step 405). This extracting process can be executed by the
processing equal to the foregoing steps 401A to 404.
[0447] The MPEG-2 stream of the television commercial is provided
by encoding the current television broadcasting equal to the above
mentioned MPEG-2 stream 301A.
[0448] Then, the retrieving unit 309 for retrieving the information
of the unique identification of the compressed moving image
compares the information 305A of the unique identification of the
compressed moving image stored in the storing unit 306 with the
information 308A of the unique identification of the compressed
moving image extracted from the other moving image data (in the
step 406).
[0449] The apparatus for retrieving the moving image of the
twenty-third embodiment is considered that is realized using the
simple facility as PC (Personal Computer). Consequently, in the
present television broadcasting, ghost phenomenon, in which image
is multiply reflected by reflection and multipath of the television
radio wave, and color slippage and the like state arise. A little
effect is also generated in the information 305A extracted using
the television broadcasting.
[0450] As mentioned above, the information 305A is shown by 3
graphs, and effect is also generated in the graph. However, since
there is sufficient information of unique identification of moving
image to be used for identifying the moving image in the graph, it
is possible to be difficult to receive the effect by expanding and
reducing the graph.
[0451] This comparing result is stored in the storing unit 110 for
storing the retrieving result (in the step 207). In the case of
displaying and/or printing the comparing result (in the step 208),
it is outputted by the output unit 111 for displaying and/or
printing the retrieving result (in the step 209).
[0452] The process of the above mentioned steps 406 to 209 is
executed for all of the information 308A of the unique
identification of the compressed moving image extracted from the
other moving image data (in the step 410).
[0453] The apparatus and the method for retrieving the moving image
according to the invention are explained the above.
[0454] The experimental system shown in FIG. 46 was mounted, and
the experiment testing the effectiveness of this invention was
executed. In the following, the result of the experiment is
shown.
[0455] In the experiment for retrieving the television commercials
that are 152 in number and 37 in type from the television
broadcasting of 26 hours 15 minutes which is ground wave television
broadcasting, the number of omission was 0 and the number of
over-detection was 0.
[0456] In the above mentioned methods and apparatuses for
retrieving the moving image, which are referred from in the first
embodiment to in the twenty-third embodiment, a process for
dividing an image into a plurality of frames is executed. It is
also possible to execute a process for dividing each of the frames
into "n" ("n" is a natural number 2 or over) regions in addition to
the above mentioned process. In this case, the information of the
unique identification of the moving image and/or the information of
the unique identification of the compressed moving image for
identifying the moving image are extracted respectively from the
divided regions, and then, the extracted information is applied to
the information 105 of the unique identification of the moving
image and/or the information 305 and 305A of the unique
identification of the compressed moving image. And, a retrieving
unit 109 for retrieving the information of the unique
identification of the moving image and/or a retrieving unit 309 for
retrieving the information of the unique identification of the
compressed moving image retrieve the identical information with the
information 105 and or the information 305 and 305A from the
information 108 of the unique identification of the moving image
and/or the information 308 and 308A of the unique identification of
the compressed moving image beforehand extracted. In this
retrieving process, the identical information with the information
105 and/or the information 305 and 305A may be extracted with
corresponding in all regions of the information 105 and/or the
information 305 and 305A. Like the above, by retrieving a frame
with a plurality of information values, the precision of the
retrieving process may be further improved.
[0457] And more, by dividing each of the frames into "n" regions
and extracting the information 105 and/or the information 305 and
305A from each of the regions, it is possible to shorten time
series information included in the information 105 and/or the
information 305 and 305A.
[0458] Moreover, when the information 105 and/or the information
305 and 305A are extracted from "n" regions of each of the frames
and a process of extracting moving image is executed by using the
information 105 and/or the information 305 and 305A of limited
regions over "n-m" (1.ltoreq.m.ltoreq.(n-1)), it is possible to
retrieve similar moving image with the difference in a part of
regions. That is to say, the retrieving process may be executed
with corresponding in the part of the regions selected as objects
for retrieving, consequently, it is possible to retrieve similar
moving image.
[0459] Until now, the methods and the apparatuses for retrieving
the moving image, which are referred from in the first embodiment
to in the twenty-third embodiment, are explained. A program for
executing processes of the above mentioned methods for retrieving
the moving image may be executed by a computer system.
[0460] Still, in the above mentioned embodiments according to the
invention, although either the uncompressed moving image data or
the compressed moving image data was applied as the object moving
image data for retrieving, both the uncompressed moving image data
and the compressed moving image data may be applied as the object
moving image data for retrieving.
[0461] And, in the above mentioned embodiments according to the
invention, the information 105 or 305 was extracted by using over
one information from each of the frames. However, the information
105 or 305 will be extracted by using over two information from
each of the frames, consequently, the precision of the retrieving
process may be improved.
[0462] In addition, when the information 105 or 305 is extracted by
using over two information from each frame, the retrieving unit 109
or 309 may retrieve similar moving image according to comparing the
information 105 or 305 by using a part of the information values
from each of the frames.
[0463] Although the invention has been described in its preferred
form with a certain degree of particularity, it is understood that
the present disclosure of the preferred form has been changed in
the details of construction and the combination and arrangement of
parts may be resorted to without departing from the spirit and the
scope of the invention as hereinafter claimed.
* * * * *