U.S. patent application number 12/785806 was filed with the patent office on 2011-05-26 for method and apparatus for detecting specific human body parts in image.
This patent application is currently assigned to Electronics and Telecommunications Research Institute. Invention is credited to Byeong Cheol CHOI.
Application Number | 20110123076 12/785806 |
Document ID | / |
Family ID | 44062108 |
Filed Date | 2011-05-26 |
United States Patent
Application |
20110123076 |
Kind Code |
A1 |
CHOI; Byeong Cheol |
May 26, 2011 |
METHOD AND APPARATUS FOR DETECTING SPECIFIC HUMAN BODY PARTS IN
IMAGE
Abstract
An apparatus for detecting specific human body parts in an image
includes: a texture energy analysis unit for analyzing energy
distribution in the image and generating texture energy maps; a
candidate region-of-interest extraction unit for extracting
candidate regions-of-interest for the specific body parts on a
given texture energy map by applying a threshold to the given
texture energy map, the given texture energy map being selected
among the texture energy maps; a candidate mask application unit
for performing convolution between candidate masks for the specific
body parts and the candidate regions-of-interest and selecting
candidate body parts based on results of the convolution; and a
body part detection unit for detecting the specific body parts on
the image by performing verification on the candidate body parts.
The verification is performed by using machine-learning models for
the specific body parts.
Inventors: |
CHOI; Byeong Cheol;
(Daejeon, KR) |
Assignee: |
Electronics and Telecommunications
Research Institute
Daejeon
KR
|
Family ID: |
44062108 |
Appl. No.: |
12/785806 |
Filed: |
May 24, 2010 |
Current U.S.
Class: |
382/128 |
Current CPC
Class: |
G06K 9/00362 20130101;
G06K 9/4609 20130101 |
Class at
Publication: |
382/128 |
International
Class: |
G06K 9/62 20060101
G06K009/62 |
Foreign Application Data
Date |
Code |
Application Number |
Nov 23, 2009 |
KR |
10-2009-0113467 |
Claims
1. An apparatus for detecting specific human body parts in an
image, the apparatus comprising: a texture energy analysis unit for
analyzing energy distribution in the image and generating texture
energy maps; a candidate region-of-interest extraction unit for
extracting candidate regions-of-interest for the specific body
parts on a given texture energy map by applying a threshold to the
given texture energy map, the given texture energy map being
selected among the texture energy maps; a candidate mask
application unit for performing convolution between candidate masks
for the specific body parts and the candidate regions-of-interest
and selecting candidate body parts based on results of the
convolution; and a body part detection unit for detecting the
specific body parts on the image by performing verification using
machine-learning models for the specific body parts on the
candidate body parts.
2. The apparatus of claim 1, wherein the texture energy analysis
unit analyzes the energy distribution by using texture energy
measures.
3. The apparatus of claim 1, wherein the texture energy measures
are texture energy measures of combined matrixes generated by using
texture energy measures of basic matrixes.
4. The apparatus of claim 3, wherein the texture energy maps are
generated via convolution between the texture energy measures of
combined matrixes and the image on a macro window basis.
5. The apparatus of claim 1, wherein the candidate
region-of-interest extraction unit selects the given texture energy
map based on weights assigned to a texture energy measure used in
generating the given energy map, the weights being specialized for
the specific body parts.
6. The apparatus of claim 5, wherein the threshold is calculated
based on minimum and maximum energy values on a quantized texture
energy map generated by quantizing the given texture energy map by
an intensity indexing.
7. The apparatus of claim 1, wherein the candidate masks include at
least one candidate face mask, at least one candidate breast mask,
at least one candidate genitals mask and at least one combination
of two or more of said at least one candidate face mask, said at
least one candidate breast mask and said at least one candidate
genitals mask.
8. The apparatus of claim 7, wherein said at least one candidate
face mask and said at least one genitals mask are configured to be
used in eight-directional masking, and said at least one breast
mask is configured to be used in nine-directional masking.
9. The apparatus of claim 1, further comprising: a normalization
unit for performing a normalization process on the candidate
regions-of-interest extracted by the candidate region-of-interest
and then providing the candidate regions-of-interest having been
normalized to the candidate mask application unit.
10. The apparatus of claim 9, wherein the normalization process
includes splitting and resizing of the candidate
regions-of-interest.
11. The apparatus of claim 1, further comprising: an analysis
result database for storing therein body parts detection result of
the body part detection unit.
12. The apparatus of claim 11, wherein the body parts detection
result includes a texture energy map on which locations and names
of the specific body parts are denoted.
13. A method for detecting specific human body parts in an image,
the method comprising: generating a texture energy map by analyzing
texture energy distribution of the image; extracting candidate
regions-of-interest for the specific body parts on the texture
energy map, the candidate regions-of-interest having energy values
equal to or higher than a threshold; performing convolution between
candidate masks for the specific body parts and the candidate
regions-of-interest to select candidate body parts based on results
of the convolution; and detecting the specific body parts on the
image by performing verification using machine-learning models for
the specific body parts on the candidate body parts.
14. The method of claim 13, wherein said generating the texture
energy map includes: generating texture energy measures of combined
matrixes by using texture energy measures of basic matrixes; and
generating the texture energy map by performing convolution between
the texture energy measures of combined matrixes and the image on a
macro window basis.
15. The method of claim 13, wherein said extracting the candidate
regions-of-interest includes: generating a quantized texture energy
map by quantizing the texture energy map by an intensity indexing;
and calculating the threshold based on minimum and maximum energy
values on the quantized texture energy map.
16. The method of claim 13, further comprising: normalizing the
candidate regions-of-interest before said performing the
convolution between the candidate masks and the candidate
regions-of-interest.
17. The method of claim 13, further comprising: generating, after
said detecting the specific body parts, body parts detection result
including a texture energy map on which locations and names of the
specific body parts are denoted.
18. The method of claim 13, wherein the candidate masks include at
least one candidate face mask, at least one candidate breast mask,
at least one candidate genitals mask and at least one combination
of two or more of said at least one candidate face mask, said at
least one candidate breast mask and said at least one candidate
genitals mask.
19. The method of claim 18, wherein said at least one candidate
face mask and said at least one genitals mask are configured to be
used in eight-directional masking, and said at least one breast
mask is configured to be used in nine-directional masking.
20. The method of claim 16, wherein said normalizing the candidate
regions-of-interest includes splitting and resizing of the
candidate regions-of-interest.
Description
CROSS-REFERENCE(S) TO RELATED APPLICATION(S)
[0001] The present invention claims priority to Korean Patent
Application No. 10-2009-0113467, filed on Nov. 23, 2009, which is
incorporated herein by reference.
FIELD OF THE INVENTION
[0002] The present invention relates to a region-of-interest
detection technique; and more particularly, to a method and
apparatus for detecting specific human body parts in an image, the
specific human body parts arousing sexuality.
BACKGROUND OF THE INVENTION
[0003] Adult image identification techniques can be used in various
fields, e.g., medical image analysis, Internet-based multimedia
services, multimedia contents broadcasting and privacy
protection.
[0004] However, since the adult image identification techniques
being used or being developed classify an image into an adult image
by estimating overall sexuality or provocativeness of the image
without detecting regions-of-interest in the image, they have
limitations in actual applications to various services.
SUMMARY OF THE INVENTION
[0005] In view of the above, the present invention provides a
method and apparatus for detecting specific human body parts in an
image, the specific human body parts arousing sexuality, wherein
candidate body parts on a texture energy map of the image are found
by using candidate masks and thus found candidate body parts are
verified by using machine-learning body part models so that
existence of the specific body parts on the image can be accurately
detected.
[0006] In accordance with an aspect of the present invention, there
is provided an apparatus for detecting specific human body parts in
an image, the apparatus including:
[0007] a texture energy analysis unit for analyzing energy
distribution in the image and generating texture energy maps;
[0008] a candidate region-of-interest extraction unit for
extracting candidate regions-of-interest for the specific body
parts on a given texture energy map by applying a threshold to the
given texture energy map, the given texture energy map being
selected among the texture energy maps;
[0009] a candidate mask application unit for performing convolution
between candidate masks for the specific body parts and the
candidate regions-of-interest and selecting candidate body parts
based on results of the convolution; and
[0010] a body part detection unit for detecting the specific body
parts on the image by performing verification using
machine-learning models for the specific body parts on the
candidate body parts.
[0011] Preferably, the texture energy analysis unit analyzes the
energy distribution by using texture energy measures.
[0012] Preferably, the texture energy measures are texture energy
measures of combined matrixes generated by using texture energy
measures of basic matrixes.
[0013] Preferably, the texture energy maps are generated via
convolution between the texture energy measures of combined
matrixes and the image on a macro window basis.
[0014] Preferably, the candidate region-of-interest extraction unit
selects the given texture energy map based on weights assigned to a
texture energy measure used in generating the given energy map, the
weights being specialized for the specific body parts.
[0015] Preferably, the threshold is calculated based on minimum and
maximum energy values on a quantized texture energy map generated
by quantizing the given texture energy map by an intensity
indexing.
[0016] Preferably, the candidate masks include at least one
candidate face mask, at least one candidate breast mask, at least
one candidate genitals mask and at least one combination of two or
more of said at least one candidate face mask, said at least one
candidate breast mask and said at least one candidate genitals
mask.
[0017] Preferably, said at least one candidate face mask and said
at least one genitals mask are configured to be used in
eight-directional masking, and said at least one breast mask is
configured to be used in nine-directional masking.
[0018] The apparatus may further include a normalization unit for
performing a normalization process on the candidate
regions-of-interest extracted by the candidate region-of-interest
and then providing the candidate regions-of-interest having been
normalized to the candidate mask application unit.
[0019] Preferably, the normalization process includes splitting and
resizing of the candidate regions-of-interest.
[0020] The apparatus may further include an analysis result
database for storing therein body parts detection result of the
body part detection unit.
[0021] Preferably, the body parts detection result includes a
texture energy map on which locations and names of the specific
body parts are denoted.
[0022] In accordance with another aspect of the present invention,
there is provided a method for detecting specific human body parts
in an image, the method including:
[0023] generating a texture energy map by analyzing texture energy
distribution of the image;
[0024] extracting candidate regions-of-interest for the specific
body parts on the texture energy map, the candidate
regions-of-interest having energy values equal to or higher than a
threshold;
[0025] performing convolution between candidate masks for the
specific body parts and the candidate regions-of-interest to select
candidate body parts based on results of the convolution; and
[0026] detecting the specific body parts on the image by performing
verification using machine-learning models for the specific body
parts on the candidate body parts.
[0027] Preferably, said generating the texture energy map includes
generating texture energy measures of combined matrixes by using
texture energy measures of basic matrixes; and generating the
texture energy map by performing convolution between the texture
energy measures of combined matrixes and the image on a macro
window basis.
[0028] Preferably, said extracting the candidate
regions-of-interest includes generating a quantized texture energy
map by quantizing the texture energy map by an intensity indexing;
and calculating the threshold based on minimum and maximum energy
values on the quantized texture energy map.
[0029] The method may further include normalizing the candidate
regions-of-interest before said performing the convolution between
the candidate masks and the candidate regions-of-interest.
[0030] The method may further include generating, after said
detecting the specific body parts, body parts detection result
including a texture energy map on which locations and names of the
specific body parts are denoted.
[0031] Preferably, the candidate masks include at least one
candidate face mask, at least one candidate breast mask, at least
one candidate genitals mask and at least one combination of two or
more of said at least one candidate face mask, said at least one
candidate breast mask and said at least one candidate genitals
mask.
[0032] Preferably, said at least one candidate face mask and said
at least one genitals mask are configured to be used in
eight-directional masking, and said at least one breast mask is
configured to be used in nine-directional masking.
[0033] Preferably, said normalizing the candidate
regions-of-interest includes splitting and resizing of the
candidate regions-of-interest.
[0034] According to the present invention, candidate body parts
arousing sexuality are found by using candidate masks and thus
found candidate body parts are verified by using machine-learning
body part models. Therefore, existence of specific body parts on an
image can be accurately detected.
BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The above and other objects and features of the present
invention will become apparent from the following description of
embodiments, given in conjunction with the accompanying drawings,
in which:
[0036] FIG. 1 illustrates a block diagram of an apparatus for
detecting regions-of-interest in a human body image in accordance
with an embodiment of the present invention;
[0037] FIG. 2 illustrates an exemplary view of texture energy
measures for use in generating text energy maps by the texture
energy analysis unit of FIG. 1;
[0038] FIG. 3 illustrates an exemplary view of texture energy maps
generated by using the texture energy measures of FIG. 2;
[0039] FIG. 4 illustrates an exemplary view for explaining
candidate regions-of-interest extraction procedure performed by the
candidate region-of-interest extraction unit of FIG. 1;
[0040] FIG. 5 illustrates an exemplary view of candidate masks for
body parts for use in selecting candidate body parts by the
candidate mask application unit of FIG. 1;
[0041] FIG. 6 illustrates an exemplary view for explaining
procedures through which body part detection result data is
obtained from the texture energy map generated by the texture
energy analysis unit; and
[0042] FIG. 7 illustrates a flowchart of a method for detecting
regions-of-interest in a human body image in accordance with an
embodiment of the present invention.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0043] Hereinafter, embodiments of the present invention will be
described in detail with reference to the accompanying drawings,
which form a part hereof.
[0044] FIG. 1 illustrates a block diagram of an apparatus for
detecting regions-of-interest in a human body image in accordance
with an embodiment of the present invention.
[0045] The apparatus includes a texture energy analysis unit 102, a
candidate region-of-interest extraction unit 104, a normalization
unit 106, a candidate mask application unit 108, a body part
detection unit 110 and an analysis result database (DB) 112.
[0046] The texture energy analysis unit 102 analyzes texture energy
of an original image to generate texture energy maps.
[0047] For example, the texture energy analysis unit 102 measures
energy distribution for each pixel on the original image by using
fifteen 5.times.5 texture energy measures shown in FIG. 2 and
generates texture energy maps shown in FIG. 3.
[0048] In FIG. 2, five left-side elements L5, E5, S5, R5 and W5
represent texture energy measures of 1.times.5 basic matrixes, and
fifteen right-side elements L5E5, E5L5, L5R5, R5L5, E5S5, S5E5,
S5S5, R5R5, L5S5, S5L5, E5E5, E5R5, R5E5, S5R5 and R5S5 represent
texture energy measures of 5.times.5 combined matrixes. Here, L, E,
S, R and W indicate a locality component, an edge component, a spot
component, a ripple component and a wave component, respectively.
Further, numbers in square brackets of each component represent
weights to be multiplied with pixel values. Due to the similarity
in properties of the ripple component R and the wave component W,
only one of them may be used for generating fifteen 5.times.5
texture energy measures. In FIG. 2, the locality component L, the
edge component E, the spot component S and the ripple component R
are used in generating the 5.times.5 texture energy measures.
[0049] The texture energy analysis unit 102 generates fifteen
5.times.5 texture energy measures by using four 1.times.5 texture
energy measures, and then generates the texture energy maps as
shown in FIG. 3 via convolution between the fifteen 5.times.5
texture energy measures and the original image on a 15.times.15
macro window basis.
[0050] The candidate region-of-interest extraction unit 104
extracts candidate regions-of-interest for specific body parts by
applying a threshold to a texture energy map generated using a
given measure, e.g., R5E5 of FIG. 3, among the texture energy maps
generated by the texture energy analysis unit 102. Here, the given
measure can be selected based on weights assigned to the given
measure, the weights being specialized for specific body parts. For
example, R5E5 of FIG. 3 is a measure having a weight specialized
for a face.
[0051] FIG. 4 illustrates an exemplary view for explaining
candidate regions-of-interest extraction procedure performed by the
candidate region-of-interest extraction unit 104 of FIG. 1. In FIG.
4, candidate regions-of-interest C are extracted by using a
threshold T.
[0052] The threshold T is calculated as in Equation 1:
T = E min + E max - E min 2 = E max + E min 2 , Equation 1
##EQU00001##
wherein E.sub.min, and E.sub.max respectively denote minimum and
maximum energy values on a texture energy map quantized by 16-level
or 32 level intensity indexing. The candidate regions-of-interest C
are regions having energy values equal to or higher than the
threshold T.
[0053] The normalization unit 106 performs a normalization process
on the candidate regions-of-interest extracted by the candidate
region-of-interest extraction unit 104. To be specific, the
normalization unit 106 splits and resizes the candidate
regions-of-interest. Through the normalization process performed by
the normalization unit 106, normalized candidate
regions-of-interest for specific body parts can be obtained.
[0054] The candidate mask application unit 108 performs convolution
between candidate masks for specific body parts and the candidate
regions-of-interest having been subjected to the normalization
process and then selects, for each body part, a candidate
region-of-interest having the highest convolution result as a
candidate body part.
[0055] The body part detection unit 110 detects specific body parts
by performing verification using machine-learning body part models
on the candidate body parts selected by the candidate mask
application unit 108.
[0056] FIG. 5 illustrates an exemplary view of candidate masks for
body parts for use in selecting candidate body parts by the
candidate mask application unit of FIG. 1.
[0057] As shown in FIG. 5, the candidate masks may include
candidate face masks, candidate breast masks and candidate genitals
masks. In FIG. 5, the candidate face masks and the candidate
genitals masks are configured to be used in eight-directional
masking, while the candidate genitals masks are configured to be
used in nine-directional masking. The candidate genitals masks may
include candidate masks for female genitals, male genitals and
combined genitals. Further, two or more candidate masks can be
combined to detect various scenes. For example, combination of the
candidate face masks and the candidate genitals masks may be used
to detect an oral sex scene.
[0058] Each of the candidate masks for specific body parts may have
a size of 32.times.32 pixels. Also, the size can vary depending on
analysis environment.
[0059] The analysis result DB 112 stores therein body parts
detection result data, i.e., processing results of the body part
detection unit 110.
[0060] FIG. 6 illustrates an exemplary view for explaining
procedures through which the body part detection result data is
obtained from the texture energy map generated by the texture
energy analysis unit 102. As shown in FIG. 6, the body parts
detection result data may be a texture energy map on which
locations and names of the detected specific body parts are
denoted.
[0061] Hereinafter, a method for detecting regions-of-interest in a
human body image in accordance with an embodiment of the present
invention will be described with reference to FIGS. 1 to 7.
[0062] FIG. 7 illustrates a flowchart of a method for detecting
regions-of-interest in a human body image in accordance with an
embodiment of the present invention.
[0063] The texture energy analysis unit 102 generates fifteen
texture energy measures of 5.times.5 combined matrixes by using
four texture energy measures of 1.times.5 combined matrixes, the
texture energy measures being as shown in FIG. 2 (step S700).
[0064] The texture energy analysis unit 102 generates texture
energy maps via convolution between the fifteen texture energy
measures generated in the step S700 and an original image on a
15.times.15 macro window basis (step S702).
[0065] The candidate region-of-interest extraction unit 104
calculates a threshold by using a minimum energy value and a
maximum energy value on a given texture energy map among the
texture energy maps generated in the step S702 (step S704). Here,
the given texture energy map may be selected based on weights
specialized for specific body parts. The threshold may be
calculated as in Equation 1.
[0066] The candidate region-of-interest extraction unit 104
extracts regions-of-interest by using the threshold calculated in
the step S704 (step S706). To be specific, the candidate
region-of-interest extraction unit 104 may extract as the
regions-of-interest regions having an energy value equal to or
higher than the threshold, as shown in FIG. 4.
[0067] The normalization unit 106 performs a normalization process
on the candidate regions-of-interest extracted in the step S706
(step S708). The normalization process may include splitting and
resizing of the candidate regions-of-interest.
[0068] The candidate mask application unit 108 performs convolution
between candidate masks for specific body parts and the candidate
regions-of-interest normalized in the step S708 and then selects
candidate body parts (step S710). The candidate masks may be as
shown in FIG. 5. The candidate mask application unit 108 may
select, for each body part, a candidate region-of-interest having
the highest convolution result as a candidate body part.
[0069] The body part detection unit 110 performs verification using
machine-learning body part models on the candidate
regions-of-interest selected in the step S710 to detect specific
body parts on the original image (step S712).
[0070] Body parts detection result data verified by the body part
detection unit 110 in the step S712 is stored in the analysis DB
112 (step S714). As shown in FIG. 6, the body part detection result
data may be a texture energy map on which locations and names of
the detected body parts are denoted.
[0071] While the invention has been shown and described with
respect to the particular embodiments, it will be understood by
those skilled in the art that various changes and modification may
be made.
* * * * *