U.S. patent application number 12/597158 was filed with the patent office on 2010-05-27 for method and apparatus for retrieving multimedia contents.
Invention is credited to Mi-Ran Choi, Jeong Heo, Myung-Gil Jang, Hyun-Ki Kim, Soo-Jong Lim, Hee-Cheol Seo, Kyoung-Ro Yoon, Yeo-Chan Yoon.
Application Number | 20100131557 12/597158 |
Document ID | / |
Family ID | 40154958 |
Filed Date | 2010-05-27 |
United States Patent
Application |
20100131557 |
Kind Code |
A1 |
Seo; Hee-Cheol ; et
al. |
May 27, 2010 |
METHOD AND APPARATUS FOR RETRIEVING MULTIMEDIA CONTENTS
Abstract
Disclosed is an apparatus and method for retrieving multimedia
contents represented in a Moving Picture Experts Group (MPEG) 7 by
transforming a user query into an MPEG-7 query format. The method
for retrieving multimedia contents includes: representing a user
query by using an indicator indicating a specific region of a
Moving Picture Experts Group 7 (MPEG-7) document and a reference
for referring to the indicator; analyzing a meaning of the user
query represented by using the indicator and the reference to
thereby produce an analysis result; and retrieving multimedia
contents according to the analysis result. The present research can
satisfy more than two retrieval conditions within the same
structure in an MPEG-7 query format and it can also clearly
represent that two different MPEG-7 documents are referred to.
Since the meaning of a user query is analyzed accurately during
retrieval process, it is possible to precisely retrieve multimedia
contents.
Inventors: |
Seo; Hee-Cheol; (Daejon,
KR) ; Choi; Mi-Ran; (Daejon, KR) ; Kim;
Hyun-Ki; (Daejon, KR) ; Jang; Myung-Gil;
(Daejon, KR) ; Heo; Jeong; (Daejon, KR) ;
Lim; Soo-Jong; (Daejon, KR) ; Yoon; Yeo-Chan;
(Seoul, KR) ; Yoon; Kyoung-Ro; (Seoul,
KR) |
Correspondence
Address: |
BLAKELY SOKOLOFF TAYLOR & ZAFMAN LLP
1279 OAKMEAD PARKWAY
SUNNYVALE
CA
94085-4040
US
|
Family ID: |
40154958 |
Appl. No.: |
12/597158 |
Filed: |
April 23, 2008 |
PCT Filed: |
April 23, 2008 |
PCT NO: |
PCT/KR08/02285 |
371 Date: |
October 22, 2009 |
Current U.S.
Class: |
707/771 ;
707/E17.014 |
Current CPC
Class: |
G06F 16/68 20190101;
G06F 16/48 20190101; G06F 16/632 20190101; G06F 16/58 20190101;
G06F 16/70 20190101 |
Class at
Publication: |
707/771 ;
707/E17.014 |
International
Class: |
G06F 17/30 20060101
G06F017/30 |
Foreign Application Data
Date |
Code |
Application Number |
Apr 23, 2007 |
KR |
10-2007-0039475 |
Apr 18, 2008 |
KR |
10-2008-0035896 |
Claims
1. A method for retrieving multimedia contents, comprising:
representing a user query by using an indicator indicating a
specific region of a Moving Picture Experts Group 7 (MPEG-7)
document and a reference for referring to the indicator; analyzing
a meaning of the user query represented by using the indicator and
the reference to thereby produce an analysis result; and retrieving
multimedia contents according to the analysis result.
2. The method of claim 1, wherein the indicator includes: an
indicator identification (ID) code used for the reference to refer
to the indicator; an indicator region descriptor for describing the
region indicated by the indicator; and an indicator limiting
descriptor for describing a condition limiting the region indicated
by the indicator.
3. The method of claim 2, wherein the indicator: region descriptor
designates an uppermost node of the region indicated by the
indicator.
4. The method of claim 1, wherein the user query is described in an
eXtensible Markup Language (XML) format in the representing a user
query by using an indicator and a reference.
5. The method of claim 4, wherein the analyzing a meaning of the
user query represented by using the indicator and the reference
includes: parsing the user query by using an XML parser to thereby
produce a parsing result; processing the indicator and the
reference based on the parsing result; and analyzing a meaning of
the user query by using the processed indicator and reference.
6. The method of claim 5, wherein in the processing the indicator
and the reference, a value inside a same region is referred to for
references referring to a same indicator.
7. A method for processing a user query to retrieve multimedia
contents, comprising: receiving a query for retrieving multimedia
contents from a user; representing the user query by using an
indicator for indicating a specific region of an MPEG-7 document
and a reference for referring to the indicator.
8. The method of claim 7, wherein the indicator includes: an
indicator ID code used for the reference to refer to the indicator;
an indicator region descriptor for describing the region indicated
by the indicator; and an indicator limiting descriptor for
describing a condition limiting the region indicated by the
indicator.
9. The method of claim 8, wherein the indicator region descriptor
for designating designates an uppermost node of the region
indicated by the indicator.
10. The method of claim 7, wherein the user query is described in
an XML format in the representing a user query by using an
indicator and a reference.
11. An apparatus for retrieving multimedia contents, comprising: a
query input unit for receiving a query for retrieving multimedia
contents from a user; a query representation unit for representing
the user query inputted through the query input unit by using an
indicator for indicating a specific region of an MPEG-7 document
and a reference for referring to the indicator; a query analysis
unit for analyzing a meaning of the user query represented in the
query representation unit by using the indicator and the reference
to thereby produce an analysis result; and a contents retrieval
unit for retrieving multimedia contents according to the analysis
result.
12. The apparatus of claim 11, wherein the indicator includes: an
indicator ID code used for the reference to refer to the indicator;
an indicator region descriptor for describing the region indicated
by the indicator; and an indicator limiting descriptor for
describing a condition limiting the region indicated by the
indicator.
13. The apparatus of claim 12, wherein the indicator region
descriptor designates an uppermost node of the region indicated by
the indicator.
14. The apparatus of claim 11, wherein the user query is described
in an XML format in the query representation unit.
15. The apparatus of claim 14, wherein the query analysis unit
includes: an XML parser for parsing the user query to thereby
produce a parsing result; a descriptor processor for processing the
indicator and the reference based on the parsing result of the XML
parser; a meaning analyzer for analyzing a meaning of the user
query by using the processed indicator and reference which are
obtained in the descriptor processor.
16. A data structure for representing a user query to retrieve
multimedia contents, comprising: an indicator for indicating a
specific region of an MPEG-7 document; and a reference for
referring to the indicator.
17. The data structure of claim 16, wherein the indicator includes:
an indicator ID code used for the reference to refer to the
indicator; an indicator region descriptor for describing the region
indicated by the indicator; and an indicator limiting descriptor
for describing a condition limiting the region indicated by the
indicator.
18. The data structure of claim 17, wherein the indicator region
descriptor for designating designates an uppermost node of the
region indicated by the indicator.
19. The data structure of claim 16, wherein the data structure is
described in an XML format.
Description
TECHNICAL FIELD
[0001] The present invention relates to an apparatus and method for
retrieving multimedia contents; and, more particularly, to
multimedia contents retrieving apparatus that can retrieve
multimedia contents represented based on Moving Picture Experts
Group 7 (MPEG-7) by transforming a user query into an MPEG-7 query
format, and a method thereof.
[0002] This work was supported by the IT R&D program of
MIC/IITA [2005-S-117-03, "Development of Intelligent Personal Media
Managing Technology for Ubiquitous Environment"].
BACKGROUND ART
[0003] Moving Picture Experts Group 7 (MPEG-7) is an international
standardization on the architectures of metadata representing
multimedia information, such as image, audio and moving picture. An
MPEG-7 query format is used to retrieve multimedia contents
represented based on the MPEG-7. An MPEG-7 multimedia contents
retrieving system retrieves multimedia contents related to a query
inputted in an MPEG-7 query format.
[0004] The MPEG-7 query format defines syntaxes for retrieving
MPEG-7 documents. The syntaxes can represent diverse types of
queries that can be used for the retrieval of MPEG-7 documents. For
example, they can represent not only natural sentence-type query
such as "an image with mountain" but also example-based query using
a multimedia file as a query and MPEG-7 textual description-based
query.
[0005] While representing such diverse queries, referring to the
same or different portions of an MPEG-7 document occurs frequently.
To be specific, there is a case where more than one retrieval
condition should be all satisfied in the same structure. For
example, to retrieve moving picture segments with "mountain" and
"sea", the presence of "mountain" and "sea" could be represented
for one region. As for joint operation, two different MPEG-7
documents should be referred to. For this, it should be clearly
represented that two different documents are referred to.
[0006] Conventional MPEG-7 query formats may satisfy more than two
retrieval conditions within the same architecture, but they have a
shortcoming that they cannot clearly represent reference to two
different MPEG-7 documents.
DISCLOSURE
Technical Problem
[0007] An embodiment of the present invention, which is invented to
resolve the problem, is directed to providing a Moving Picture
Experts Group 7 (MPEG-7) query format that can satisfy more than
two retrieval conditions within the same structure and clearly
represent that different MPEG-7 documents are referred to.
[0008] Another embodiment of the present invention is directed to
providing an apparatus and method that can accurately retrieve
multimedia contents by precisely analyzing the meaning of a user
query in a retrieving process.
[0009] Other objects and advantages of the present invention can be
understood by the following description, and become apparent with
reference to the embodiments of the present invention. Also, it is
obvious to those skilled in the art of the present invention that
the objects and advantages of the present invention can be realized
by the means as claimed and combinations thereof.
Technical Solution
[0010] In accordance with an aspect of the present invention, there
is provided a method for retrieving multimedia contents, which
includes: representing a user query by using an indicator
indicating a specific region of a Moving Picture Experts Group 7
(MPEG-7) document and a reference for referring to the indicator;
analyzing a meaning of the user query represented by using the
indicator and the reference to thereby produce an analysis result;
and retrieving multimedia contents according to the analysis
result.
[0011] In accordance with another aspect of the present invention,
there is provided a method for processing a user query to retrieve
multimedia contents, which includes: receiving a query for
retrieving multimedia contents from a user; representing the user
query by using an indicator for indicating a specific region of an
MPEG-7 document and a reference for referring to the indicator.
[0012] In accordance with another aspect of the present invention,
there is provided an apparatus for retrieving multimedia contents,
which includes: a query input unit for receiving a query for
retrieving multimedia contents from a user; a query representation
unit for representing the user query inputted through the query
input unit by using an indicator for indicating a specific region
of an MPEG-7 document and a reference for referring to the
indicator; a query analysis unit for analyzing a meaning of the
user query represented in the query representation unit by using
the indicator and the reference to thereby produce an analysis
result; and a contents retrieval unit for retrieving multimedia
contents according to the analysis result.
[0013] In accordance with another aspect of the present invention,
there is provided a data structure for representing a user query to
retrieve multimedia contents, which includes: an indicator for
indicating a specific region of an MPEG-7 document; and a reference
for referring to the indicator.
Advantageous Effects
[0014] The present invention described above provides an MPEG-7
query format that can satisfy more than two retrieval conditions
within the same structure and clearly represent that different
MPEG-7 documents are referred to. Also, since the meaning of a user
query is precisely analyzed during a retrieving process, it is
possible to retrieve multimedia contents that accurately agree with
the user query.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] FIG. 1 is a flowchart describing a multimedia contents
retrieving method in accordance with an embodiment of the present
invention.
[0016] FIG. 2 illustrates an eXtensible Markup Language (XML)
schema of an indicator in accordance with an embodiment of the
present invention.
[0017] FIG. 3 illustrates an XML schema of a reference in
accordance with an embodiment of the present invention.
[0018] FIG. 4 is a flowchart describing a query representation step
S10 of FIG. 1 in detail.
[0019] FIG. 5 is a flowchart describing a query processing step S20
of FIG. 1 in detail.
[0020] FIG. 6 illustrates an XML schema of an indicator in
accordance with another embodiment of the present invention.
[0021] FIG. 7 is a block view showing a structure of multimedia
contents retrieving apparatus in accordance with an embodiment of
the present invention.
BEST MODE
[0022] The advantages, features and aspects of the invention will
become apparent from the following description of the embodiments
with reference to the accompanying drawings, which is set forth
hereinafter. When it is considered that detailed description on a
related art may obscure a point of the present invention, the
description will not be provided herein. Hereinafter, specific
embodiments of the present invention will be described with
reference to the accompanying drawings.
[0023] FIG. 1 is a flowchart describing a multimedia contents
retrieving method in accordance with an embodiment of the present
invention.
[0024] In step S10, a user query is represented as a query for
retrieving multimedia contents. The user query is represented using
an indicator and a reference for referring to the indicator to
precisely represent the meaning of the user query. The indicator
denotes a specific region of a Moving Picture Experts Group 7
(MPEG-7) document, and the reference is used to refer to the
indicator. For example, when moving picture segments with
"mountain" and "sea" is retrieved for, there is an indicator for a
moving picture segment and a reference of the indicator may
represent the presence of "mountain" and another reference, the
presence of "sea." In subsequent joint operation, two indicators
may be established for two different MPEG-7 documents,
respectively, and each of the two indicators may have references to
clearly represent the two different MPEG-7 documents from each
other.
[0025] In step S20, a query processor analyzes the user query
represented using the indicator and references. In step S30, a
retrieval engine retrieves multimedia contents related to the user
query analyzed in the query processor and, in step S40, provides a
retrieval result.
[0026] FIG. 2 illustrates an eXtensible Markup Language (XML)
schema of an indicator in accordance with an embodiment of the
present invention. As shown in the drawing, an indicator includes
an indicator identification (ID) number 101, an indicator region
descriptor 102, and an indicator limiting descriptor 103. The
indicator region descriptor 102 may include a reference 104 for
referring to another indicator. The indicator limiting descriptor
103 includes a part 105 describing conditions for limiting an
indicator.
[0027] An MPEG-7 document is described in an XML format, and an
indicator indicates a specific region of the MPEG-7 document. For
this, the indicator region descriptor 102 is used to designate an
uppermost node of the specific region. The indicator limiting
descriptor 103 is used when an additional limiting condition is
needed in connection with a region represented by indicator region
descriptor. The indicator ID number 101 is used when an indicator
is referred to.
[0028] The following Table 1 shows FIG. 2 described in the format
of an XML schema. In the Table 1, a "path" element is a part for
describing an indicator region, and a "selector" element is a part
for describing limitation of an indicator. The "id" denotes the
unique number of an indicator. To allow referring to other
indicators within an indicator, a "ref" attribute is used. In a
part limiting an indicator, "ConditionalType" is defined as a
limiting condition to describe specific condition.
TABLE-US-00001 TABLE 1 <complexType name="IndicatorType">
<sequence> <element name="Path"> <complexType>
<simpleContent> <extension base="mpeg7:xPathType">
<attribute name="ref" type="IDREF" use="optional"/>
<extension> </simpleContent> </complexType>
</element> <element name="Selector"
type="mp7qf:ConditionType" minOccurs="0"/> </sequence>
<attribute name="id" type="ID" use="required"/>
</complexType>
[0029] FIG. 3 illustrates an XML schema of a reference in
accordance with an embodiment of the present invention. An
indicator may refer to a specific indicator, and it is possible to
refer to a node inside a specific region which is indicated by the
indicator.
[0030] In FIG. 3, an indicator may include a "ref" attribute for
referring to a specific indicator, and represent a region related
to the indicator by the attribute value. XML schema related to FIG.
3 may be described as the following Table 2, where the "ref"
attribute refers to the indicator and "xPathType" describes a part
related to the indicator.
TABLE-US-00002 TABLE 2 <complexType name="FeatureNameType">
<simpleContent> <extension base="mpeg7:xPathType">
<attribute name="ref" type="IDREF" use="optional"/>
</extension> </simpleContent> </complexType>
[0031] FIG. 4 is a flowchart describing a query representation step
S10 of FIG. 1 in detail. In step S402, a query for retrieving
multimedia contents is inputted from a user. In step S404, the
inputted user query is represented as an indicator for indicating a
specific region of an MPEG-7 document and a reference for referring
to the indicator.
[0032] For example, a query for "retrieving images whose horizontal
length.times.vertical length is greater than 1024.times.768" can be
represented as the following Table 3 based on the XML schema
defined in the Tables 1 and 2. In the Table 3, an indicator is
referred to by using a reference "href," and a specific part
related to a region indicated by the indicator can be indicated by
describing an additional path.
TABLE-US-00003 TABLE 3 <mp7qf:RetrieveData>
<mp7qf:Indicator id="M7DocID">
<mp7qf:Path>/Mpeg7</mp7qf:Path>
</mp7qf:Indicator> <mp7qf:Indicator
id="VisualCodingFrameID"> <mp7qf:Path
ref="M7DocID">//VisualCoding/Frame</mp7qf:Path>
</mp7qf:Indicator> <mp7qf:Condition>
<mp7qf:ConditionBag operator="AND"> <!-- target content :
Image --> <mp7qf:FeatureCondition operator="equalTo">
<mp7qf:SourceFeature ref="M7DocID">
//MediaFormat/Content/Name </mp7qf:SourceFeature>
<mp7qf:TargetConstantValue
xsi:type="mp7qf:FeatureStringType">
<mp7qf:value>Image</mp7qf:value>
</mp7qf:TargetConstantValue> </mp7qf:FeatureCondition>
<!-- sizes are greater than or equal to 1024*768 pixels (width *
height) --> <mp7qf:FeatureCondition
operator="greaterThanOREqualTo">
<mp7qf:SourceFeatureExpression operator="multiply">
<mp7qf:FeatureName ref="VisualCodingFrameID"> @height
</mp7qf:FeatureName> <mp7qf:FeatureName
ref="VisualCodingFrameID"> @width </mp7qf:FeatureName>
</mp7qf:SourceFeatureExpression>
<mp7qf:TargetFeatureExpression operator="multiply">
<mp7qf:ConstantValue xsi:type="mp7qf:FeatureDecimalType">
<mp7qf:value>1024</mp7qf:value>
</mp7qf:ConstantValue> <mp7qf:ConstantValue
xsi:type="mp7qf:FeatureDecimalType">
<mp7qf:value>768</mp7qf:value>
</mp7qf:ConstantValue> </mp7qf:TargetFeatureExpression>
</mp7qf:FeatureCondition> </mp7qf:ConditionBag>
</mp7qf:Condition> </mp7qf:RetrieveData>
[0033] FIG. 5 is a flowchart describing a query processing step S20
of FIG. 1 in detail. The meaning of the user query represented
using an indicator and a reference is analyzed in the query
processing step S20. First, in step S502, an XML parser parses a
user query described in an XML format. Subsequently, in step S504,
the indicator and the reference are processed based on a parsing
result. In step S506, the meaning of the user query is analyzed
using the processed indicator and reference.
[0034] In the step S504 where the indicator and the reference are
processed, references referring to the same indicator are regarded
as values for referring to a value in the inside of the same region
to analyze the meaning of the user query. For example, since
"@height" and "@width" refer to "VisualCodingFrameID" in the user
query, it is analyzed that the two refer to a value in the inside a
region indicated by the "VisualCodingFrameID."
[0035] FIG. 6 illustrates an XML schema of an indicator in
accordance with another embodiment of the present invention. An
indicator ID number 601 is the same as the indicator ID number 101
of FIG. 2. An indicator region descriptor of FIG. 6 is an optional
element whereas an indicator limiting descriptor 603 is essential
element, and it does not have "attribute." FIG. 6 may be described
in an XML schema, which is presented in the following Table 4.
TABLE-US-00004 TABLE 4 <mp7qf:RetrieveData>
<mp7qf:Indicator id="M7DocID">
<mp7qf:Path>/Mpeg7</mp7qf:Path>
</mp7qf:Indicator> <mp7qf:Indicator
id="VisualCodingFrameID"> <mp7qf:Path
ref="M7DocID">//VisualCoding/Frame</mp7qf:Path>
</mp7qf:Indicator> <mp7qf:Condition>
<mp7qf:ConditionBag operator="AND"> <!-- target content :
Image --> <mp7qf:FeatureCondition operator="equalTo">
<mp7qf:SourceFeature ref="M7DocID">
//MediaFormat/Content/Name </mp7qf:SourceFeature>
<mp7qf:TargetConstantValue
xsi:type="mp7qf:FeatureStringType">
<mp7qf:value>Image</mp7qf:value>
</mp7qf:TargetConstantValue> </mp7qf:FeatureCondition>
<!-- sizes are greater than or equal to 1024*768 pixels (width *
height) --> <mp7qf:FeatureCondition
operator="greaterThanOREqualTo">
<mp7qf:SourceFeatureExpression operator="multiply">
<mp7qf:FeatureName ref="VisualCodingFrameID"> @height
</mp7qf:FeatureName> <mp7qf:FeatureName
ref="VisualCodingFrameID"> @width </mp7qf:FeatureName>
</mp7qf:SourceFeatureExpression>
<mp7qf:TargetFeatureExpression operator="multiply">
<mp7qf:ConstantValue xsi:type="mp7qf:FeatureDecimalType">
<mp7qf:value>1024</mp7qf:value>
</mp7qf:ConstantValue> <mp7qf:ConstantValue
xsi:type="mp7qf:FeatureDecimalType">
<mp7qf:value>768</mp7qf:value>
</mp7qf:ConstantValue> </mp7qf:TargetFeatureExpression>
</mp7qf:FeatureCondition> </mp7qf:ConditionBag>
</mp7qf:Condition> </mp7qf:RetrieveData>
[0036] FIG. 7 is a block view showing a structure of multimedia
contents retrieving apparatus in accordance with an embodiment of
the present invention. As shown in the drawing, the multimedia
contents retrieving apparatus 700 includes a query input unit 702,
a query representation unit 704, a query analysis unit 706, a
contents retrieval unit 708, and an output unit 710.
[0037] The query input unit 702 receives a query for retrieving
multimedia contents from a user. The query representation unit 704
represents the user query inputted through the query input unit 702
into an MPEG-7 query format by using an indicator indicating a
specific region of an MPEG-7 document and a reference for referring
to the indicator. An indicator includes an indicator ID number used
for a reference to refer to the indicator, a descriptor for
describing limiting conditions for the region indicated by the
indicator, and a descriptor for designating an uppermost node of
the region indicated by the indicator. The user query is
represented in an XML format.
[0038] The query analysis unit 706 analyzes the meaning of the user
query represented using the indicator and the reference in the
query representation unit 704. The query analysis unit 706 includes
an XML parser 712 for parsing a user query, a descriptor processor
714 for processing an indicator and a reference based on the
parsing result of the XML parser 712, and a meaning analyzer 716
for analyzing the meaning of the user query based on the indicator
and the reference processed in the descriptor processor 714. The
contents retrieval unit 708 retrieves multimedia contents according
to the analysis result of the user query analysis unit 706. The
contents retrieval unit 708 may retrieve a database 718 or search
the internet 722 through a communication unit 720. The database 718
may be set up inside or outside the multimedia contents retrieving
apparatus 700. The output unit 710 provides multimedia contents
retrieved by the contents retrieval unit 708 to the user.
[0039] More for the Invention
[0040] The method of the present invention described above may be
realized as a program and stored in a computer-readable recording
medium, such as CD-ROM, RAM, ROM, floppy disks, hard disks,
magneto-optical disks and the like. Since this process can be
easily implemented by those skilled in the art to which the present
invention belongs, further description will not be provided
herein.
[0041] While the present invention has been described with respect
to the specific embodiments, it will be apparent to those skilled
in the art that various changes and modifications may be made
without departing from the spirit and scope of the invention as
defined in the following claims.
* * * * *