U.S. patent application number 14/222679 was filed with the patent office on 2014-12-11 for image capture system and operation method thereof.
This patent application is currently assigned to Etron Technology, Inc.. The applicant listed for this patent is Etron Technology, Inc.. Invention is credited to Chi-Feng Lee.
Application Number | 20140363097 14/222679 |
Document ID | / |
Family ID | 52005140 |
Filed Date | 2014-12-11 |
United States Patent
Application |
20140363097 |
Kind Code |
A1 |
Lee; Chi-Feng |
December 11, 2014 |
IMAGE CAPTURE SYSTEM AND OPERATION METHOD THEREOF
Abstract
An image capture system includes a depth information generation
unit, a feature extraction unit, and a merging unit. The depth
information generation unit generates a depth information
corresponding to at least one object of an original image. The
feature extraction unit generates a feature information
corresponding to the at least one object of the original image. The
merging unit is coupled to the depth information generation unit
and the feature extraction unit, and merges the depth information
and the feature information into a feature depth map and outputs
the feature depth map to an application unit.
Inventors: |
Lee; Chi-Feng; (Hsinchu
County, TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Etron Technology, Inc. |
Hsinchu |
|
TW |
|
|
Assignee: |
Etron Technology, Inc.
Hsinchu
TW
|
Family ID: |
52005140 |
Appl. No.: |
14/222679 |
Filed: |
March 23, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61831620 |
Jun 6, 2013 |
|
|
|
Current U.S.
Class: |
382/284 |
Current CPC
Class: |
G06T 19/20 20130101;
H04N 13/00 20130101; H04N 2213/003 20130101; H04N 2013/0081
20130101; G06T 7/593 20170101 |
Class at
Publication: |
382/284 |
International
Class: |
G06T 7/00 20060101
G06T007/00; G06T 19/20 20060101 G06T019/20 |
Claims
1. An image capture system comprising: a depth information
generation unit generating a depth information corresponding to at
least one object of an original image; a feature extraction unit
generating a feature information corresponding to the at least one
object of the original image; and a merging unit coupled to the
depth information generation unit and the feature extraction unit,
the merging unit merging the depth information and the feature
information into a feature depth map and outputting the feature
depth map to an application unit.
2. The image capture system of claim 1, wherein the merging unit
gives a first weight to the depth information and a second weight
to the feature information, and the merging unit merges the depth
information and the feature information into the feature depth map
according to the first weight and the second weight.
3. The image capture system of claim 1, wherein the feature
information corresponds to high frequency parts of the feature
depth map and the depth information corresponds to low frequency
parts of the feature depth map.
4. The image capture system of claim 3, wherein the application
unit utilizes the high frequency parts of the feature depth map to
execute face recognition corresponding to the at least one
object.
5. The image capture system of claim 3, wherein the application
unit utilizes the low frequency parts of the feature depth map to
execute gesture recognition corresponding to the at least one
object.
6. The image capture system of claim 3, wherein the application
unit utilizes the low frequency parts of the feature depth map to
determine a distance between the at least one object and the image
capture system.
7. The image capture system of claim 3, wherein the application
unit utilizes the low frequency parts of the feature depth map to
execute gesture recognition corresponding to the at least one
object and determine a distance between the at least one object and
the image capture system.
8. An operation method of an image capture system, wherein the
image capture system comprises a depth information generation unit,
a feature extraction unit, and a merging unit, the operation method
comprising: the depth information generation unit generating a
depth information corresponding to at least one object of an
original image; the feature extraction unit generating a feature
information corresponding to the at least one object of the
original image; and the merging unit merging the depth information
and the feature information into a feature depth map and outputting
the feature depth map to an application unit.
9. The operation method of claim 8, wherein the merging unit
merging the depth information and the feature information into the
feature depth map comprises: the merging unit giving a first weight
to the depth information and a second weight to the feature
information; and the merging unit merging the depth information and
the feature information into the feature depth map according to the
first weight and the second weight.
10. The operation method of claim 8, wherein the feature
information corresponds to high frequency parts of the feature
depth map and the depth information corresponds to low frequency
parts of the feature depth map.
11. The operation method of claim 10, wherein the application unit
utilizes the high frequency parts of the feature depth map to
execute face recognition corresponding to the at least one
object.
12. The operation method of claim 10, wherein the application unit
utilizes the low frequency parts of the feature depth map to
execute gesture recognition corresponding to the at least one
object.
13. The operation method of claim 10, wherein the application unit
utilizes the low frequency parts of the feature depth map to
determine a distance between the at least one object and the image
capture system.
14. The operation method of claim 10, wherein the application unit
utilizes the low frequency parts of the feature depth map to
execute gesture recognition corresponding to the at least one
object and determine a distance between the at least one object and
the image capture system.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional
Application No. 61/831,620, filed on Jun. 6, 2013 and entitled
"Depth Map Post Process System," the contents of which are
incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] The present invention relates to an image capture system and
an operation method thereof, and particularly to an image capture
system and an operation method thereof that can generate and output
a feature depth map simultaneously including a depth information
and a feature information corresponding to at least one object of
an original image to decrease transmission data amount and
bandwidth for the feature depth map.
[0004] 2. Description of the Prior Art
[0005] If a gesture application provided by the prior art wants to
determine whether an operator is an effective operator, the
simplest method executed by the gesture application is face
detection or face determination. Generally speaking, the gesture
application provided by the prior art utilizes gray level
information or color information of an original image to execute
face detection, text recognition, or any pattern recognition (e.g.
quick response (QR) code.) and utilizes depth information generated
from the original image to execute gesture detection. However, the
gesture application provided by the prior art simultaneously needs
information of the original image and depth information, so a
disadvantage of the gesture application provided by the prior art
is that the gesture application needs more transmission data amount
and bandwidth. Therefore, the gesture application provided by the
prior art is not a good choice for a user.
SUMMARY OF THE INVENTION
[0006] An embodiment provides an image capture system. The image
capture system 100 includes a depth information generation unit, a
feature extraction unit, and a merging unit. The depth information
generation unit generates a depth information corresponding to at
least one object of an original image. The feature extraction unit
generates a feature information corresponding to the at least one
object of the original image. The merging unit is coupled to the
depth information generation unit and the feature extraction unit,
and merges the depth information and the feature information into a
feature depth map and outputting the feature depth map to an
application unit.
[0007] Another embodiment provides an operation method of an image
capture system, wherein the image capture system comprises a depth
information generation unit, a feature extraction unit, and a
merging unit. The operation method includes the depth information
generation unit generating a depth information corresponding to at
least one object of an original image; the feature extraction unit
generating a feature information corresponding to the at least one
object of the original image; and the merging unit merging the
depth information and the feature information into a feature depth
map and outputting the feature depth map to an application
unit.
[0008] The present invention provides an image capture system and
an operation method thereof. The image capture system and the
operation method utilize a depth information generation unit of the
image capture system to generate a depth information corresponding
to at least one object of an original image, a feature extraction
unit of the image capture system to generate a feature information
corresponding to the at least one object of the original image, and
a merging unit of the image capture system to generate a feature
depth map by merging the depth information and the feature
information. Compared to the prior art, because the feature depth
map simultaneously includes the depth information and the feature
information, transmission data amount and bandwidth for the feature
depth map can be decreased.
[0009] These and other objectives of the present invention will no
doubt become obvious to those of ordinary skill in the art after
reading the following detailed description of the preferred
embodiment that is illustrated in the various figures and
drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] FIG. 1 is a diagram illustrating an image capture system for
outputting depth and feature information of at least one object in
an original image according to an embodiment.
[0011] FIG. 2 is a diagram illustrating the original image.
[0012] FIG. 3 is a diagram illustrating depth information
corresponding to an object of the original image.
[0013] FIG. 4 is a diagram illustrating a feature depth map.
[0014] FIG. 5 is a diagram illustrating a recognition result
generated by the application unit after the application unit
executes face recognition and gesture recognition on the feature
depth map.
[0015] FIG. 6 is flowchart illustrating an operation method of the
image capture system 100 according to another embodiment.
DETAILED DESCRIPTION
[0016] Please refer to FIG. 1, FIG. 2, FIG. 3, and FIG. 4. FIG. 1
is a diagram illustrating an image capture system 100 according to
an embodiment, FIG. 2 is a diagram illustrating an original image
OIM, FIG. 3 is a diagram illustrating depth information
corresponding to an object of the original image OIM, and FIG. 4 is
a diagram illustrating a feature depth map. As shown in FIG. 1, the
image capture system 100 includes a depth information generation
unit 102, a feature extraction unit 104, and a merging unit 106.
The depth information generation unit 102 is used for generating a
depth information 108 (as shown in FIG. 3) corresponding to an
object 110 of the original image OIM (as shown in FIG. 2). But, the
present invention is not limited to the original image OIM only
including the object 110. That is to say, the original image OIM
can include at least one object. The feature extraction unit 104 is
used for generating a feature information corresponding to the
object 110 according to the original image OIM. For example, the
feature extraction unit 104 can generate the feature information
corresponding to the object 110 according to eye edges, a face
edge, or a lip of the object 110 of the original image OIM. But,
the present invention is not limited to the feature extraction unit
104 generating the feature information corresponding to the object
110 according to the eye edges, the face edge, or the lip of the
object 110. As shown in FIG. 1, the merging unit 106 is coupled to
the depth information generation unit 102 and the feature
extraction unit 104. After the merging unit 106 receives the depth
information 108 from the depth information generation unit 102 and
the feature information from the feature extraction unit 104, the
merging unit 106 gives a first weight to the depth information 108
and a second weight to the feature information corresponding to the
object 110. Then, the merging unit 106 can merge the depth
information 108 and the feature information corresponding to the
object 110 into a feature depth map 112 (as shown in FIG. 4) and
output the feature depth map 112 to an application unit 114
according to the first weight and the second weight, wherein the
feature information corresponds to high frequency parts of the
feature depth map 112 and the depth information 108 corresponds to
low frequency parts of the feature depth map 112. In addition, the
image capture system 100 can utilize a high-pass filter (not shown
in FIG. 1) to filter the feature depth map 112 to generate the high
frequency parts of the feature depth map 112, and a low-pass filter
(not shown in FIG. 1) to filter the feature depth map 112 to
generate the low frequency parts of the feature depth map 112.
[0017] As shown in FIG. 1, after the application unit 114 receives
the feature depth map 112, the application unit 114 utilizes the
high frequency parts of the feature depth map 112 to execute face
recognition corresponding to the object 110 and the low frequency
parts of the feature depth map 112 to execute gesture recognition
corresponding to the object 110. In addition, the application unit
114 can also utilize the high frequency parts of the feature depth
map 112 to recognize patterns corresponding to the object 110, or
characters shown in the original image OIM. In addition, in another
embodiment of the present invention, after the application unit 114
receives the feature depth map 112, the application unit 114 can
utilize the high frequency parts of the feature depth map 112 to
execute face recognition, text recognition, QR code recognition,
pattern recognition, or profile recognition corresponding to the
object 110.
[0018] Please refer to FIG. 5. FIG. 5 is a diagram illustrating a
recognition result 116 generated by the application unit 114 after
the application unit 114 executes face recognition and gesture
recognition on the feature depth map 112. As shown in FIG. 5, the
recognition result 116 includes a face profile 1162 and a body
profile 1164 corresponding to the object 110. Then, the application
unit 114 can utilize the recognition result 116 to execute
corresponding operation. Further, in another embodiment of the
present invention, the application unit 114 can utilize the low
frequency parts of the feature depth map 112 to determine a
distance between the object 110 and the image capture system 100.
Further, in another embodiment of the present invention, the
application unit 114 can simultaneously utilize the low frequency
parts of the feature depth map 112 to execute gesture recognition
corresponding to the object 110 and determine a distance between
the object 110 and the image capture system 100.
[0019] Please refer to FIGS. 1 to 6. FIG. 6 is flowchart
illustrating an operation method of the image capture system 100
according to another embodiment. The operation method in FIG. 6 is
illustrated using the image capture system 100 in FIG. 1. Detailed
steps are as follows:
[0020] Step 600: Start.
[0021] Step 602: The depth information generation unit 102
generates depth information 108 corresponding to the object 110
according to the original image OIM.
[0022] Step 604: The feature extraction unit 104 generates feature
information corresponding to the object 110 according to the
original image OIM.
[0023] Step 606: The merging unit 106 gives a first weight to the
depth information 108 and a second weight to the feature
information corresponding to the object 110.
[0024] Step 608: The merging unit 106 merges the depth information
108 and the feature information corresponding to the object 110 to
generate a feature depth map 112 according to the first weight and
the second weight.
[0025] Step 610: End.
[0026] In Step 604, the feature extraction unit 104 can generate
the feature information corresponding to the object 110 according
to eye edges, a face edge, or a lip of the object 110 of the
original image OIM. But, the present invention is not limited to
the feature extraction unit 104 generating the feature information
corresponding to the object 110 according to the eye edges, the
face edge, or the lip of the object 110. In Step 606, after the
merging unit 106 receives the depth information 108 from the depth
information generation unit 102 and the feature information from
the feature extraction unit 104, the merging unit 106 gives the
first weight to the depth information 108 and the second weight to
the feature information corresponding to the object 110. Then, in
Step 608, the merging unit 106 can merge the depth information 108
and the feature information corresponding to the object 110 to
generate and output the feature depth map 112 (as shown in FIG. 4)
to the application unit 114 according to the first weight and the
second weight, wherein the feature information corresponds to high
frequency parts of the feature depth map 112 and the depth
information 108 corresponds to low frequency parts of the feature
depth map 112. In addition, the image capture system 100 can
utilize a high-pass filter (not shown in FIG. 1) to filter the
feature depth map 112 to generate the high frequency parts of the
feature depth map 112, and a low-pass filter (not shown in FIG. 1)
to filter the feature depth map 112 to generate the low frequency
parts of the feature depth map 112.
[0027] As shown in FIG. 1, after the application unit 114 receives
the feature depth map 112, the application unit 114 can utilize the
high frequency parts of the feature depth map 112 to execute face
recognition corresponding to the object 110 and the low frequency
parts of the feature depth map 112 to execute gesture recognition
corresponding to the object 110.
[0028] In addition, in another embodiment of the present invention,
after the application unit 114 receives the feature depth map 112,
the application unit 114 can utilize the high frequency parts of
the feature depth map 112 to execute face recognition, text
recognition, QR code recognition, pattern recognition, or profile
recognition corresponding to the object 110.
[0029] In addition, the application unit 114 can also utilize the
high frequency parts of the feature depth map 112 to recognize
veins corresponding to the object 110 or characters shown in the
original image OIM.
[0030] In addition, after the application unit 114 executes face
recognition and gesture recognition on the feature depth map 112,
the application unit 114 can generate a recognition result 116. As
shown in FIG. 5, the recognition result 116 includes a face profile
1162 and a body profile 1164 corresponding to the object 110. Then,
the application unit 114 can utilize the recognition result 116 to
execute corresponding operation. Further, in another embodiment of
the present invention, the application unit 114 can utilize the low
frequency parts of the feature depth map 112 to determine a
distance between the object 110 and the image capture system 100.
Further, in another embodiment of the present invention, the
application unit 114 can simultaneously utilize the low frequency
parts of the feature depth map 112 to execute gesture recognition
corresponding to the object 110 and determine a distance between
the object 110 and the image capture system 100.
[0031] To sum up, the image capture system and the operation method
thereof the depth information generation unit to generate a depth
information corresponding to at least one object of an original
image, the feature extraction unit to generate a feature
information corresponding to the at least one object of the
original image, and the merging unit to generate a feature depth
map by merging the depth information and the feature information.
Compared to the prior art, because the feature depth map
simultaneously includes the depth information and the feature
information, transmission data amount and bandwidth for the feature
depth map can be decreased.
[0032] Those skilled in the art will readily observe that numerous
modifications and alterations of the device and method may be made
while retaining the teachings of the invention. Accordingly, the
above disclosure should be construed as limited only by the metes
and bounds of the appended claims.
* * * * *