U.S. patent application number 13/787369 was filed with the patent office on 2013-07-18 for method and system for biometric recognition.
This patent application is currently assigned to Eyelock, Inc.. The applicant listed for this patent is EyeLock, Inc.. Invention is credited to Keith J. Hanna.
Application Number | 20130182915 13/787369 |
Document ID | / |
Family ID | 39875916 |
Filed Date | 2013-07-18 |
United States Patent
Application |
20130182915 |
Kind Code |
A1 |
Hanna; Keith J. |
July 18, 2013 |
METHOD AND SYSTEM FOR BIOMETRIC RECOGNITION
Abstract
High quality, high contrast images of an iris and the face of a
person are acquired in rapid succession in either sequence by a
single sensor and one or more illuminators, preferably within less
than one second of each other, by changing the data acquisition
settings or illumination settings between each acquisition.
Inventors: |
Hanna; Keith J.; (New York,
NY) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
EyeLock, Inc.; |
Caguas |
PR |
US |
|
|
Assignee: |
Eyelock, Inc.
Caguas
PR
|
Family ID: |
39875916 |
Appl. No.: |
13/787369 |
Filed: |
March 6, 2013 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12596019 |
|
|
|
|
PCT/US08/60791 |
Apr 18, 2008 |
|
|
|
13787369 |
|
|
|
|
60925259 |
Apr 19, 2007 |
|
|
|
Current U.S.
Class: |
382/116 |
Current CPC
Class: |
G06F 21/32 20130101;
G06K 9/00288 20130101; G06K 9/00604 20130101; H04N 5/2352 20130101;
G06K 9/3208 20130101; G06T 7/74 20170101; G06K 9/00892 20130101;
H04N 5/2256 20130101; G06K 9/00261 20130101; G06K 9/00248 20130101;
A61B 5/117 20130101; G06K 9/00617 20130101; G06T 7/521 20170101;
H04N 5/23296 20130101; G06K 9/00255 20130101 |
Class at
Publication: |
382/116 |
International
Class: |
G06K 9/00 20060101
G06K009/00 |
Foreign Application Data
Date |
Code |
Application Number |
Apr 18, 2008 |
US |
PCT/US08/60791 |
Claims
1. A method for determining a rotational adjustment angle from an
image having biometric information, for image adjustment prior to
biometric matching against stored biometric information, the method
comprising: acquiring, by a sensor, a first image comprising one or
both of a subject's eyes; determining an orientation of at least
one feature in the first image, in relation to the first image's
orientation; determining a rotational adjustment angle based at
least in part on the determined orientation, the rotational
adjustment angle comprising an angle of tilt of the subject's face
within the first image; and adjusting stored or acquired biometric
information based on the determined rotational adjustment angle, to
substantially remove orientation difference if present, between the
stored and the acquired biometric information.
2. The method of claim 1, wherein adjusting the stored or the
acquired biometric information comprises substantially removing
orientation difference to reduce further adjustments if any during
biometric matching in searching for an improved match.
3. The method of claim 1, further comprising performing biometric
matching between the stored and the acquired biometric information
responsive to the adjustment, detecting that an orientation
difference remains, and further adjusting one of the stored and the
acquired biometric information to search for an improved match.
4. The method of claim 1, comprising adjusting the stored or the
acquired biometric information, the acquired biometric information
comprising information from a second image acquired within a
predetermined time of the first image.
5. The method of claim 1, comprising adjusting the stored or the
acquired biometric information, the acquired biometric information
comprising information from a second image acquired by a second
sensor.
6. The method of claim 1, comprising adjusting the stored or the
acquired biometric information, the acquired biometric information
comprising information from the first image.
7. The method of claim 1, comprising adjusting the stored or the
acquired biometric information, the acquired biometric information
comprising at least one of: iris and face data.
8. The method of claim 1, further comprising adjusting, based on
the first image, an illumination level, a zoom level or a focus
setting for acquiring a second image.
9. The method of claim 1, further comprising extracting, from the
first image, an image portion comprising an iris of the
subject.
10. The method of claim 1, wherein determining the orientation of
the at least one feature comprises determining a relative position
of the subject's eyes, in relation to the first image's
orientation.
11. The method of claim 1, further comprising determining a pose of
the subject's face, comprising one or more of: pan, tilt, yaw and
translation of the subject's face in three-dimensional space.
12. A system for determining a rotational adjustment angle from an
image having biometric information, for image adjustment prior to
biometric matching against stored biometric information, the system
comprising: a sensor acquiring a first image comprising one or both
of a subject's eyes; and a biometric system: determining an
orientation of at least one feature in the first image, in relation
to the first image's orientation; determining a rotational
adjustment angle based at least in part on the determined
orientation, the rotational adjustment angle comprising an angle of
tilt of the subject's face within the first image; and adjusting
stored or acquired biometric information based on the determined
rotational adjustment angle, to substantially remove orientation
difference if present, between the stored and the acquired
biometric information.
13. The system of claim 12, wherein the biometric system adjusts
the stored or the acquired biometric information to reduce further
adjustments if any during biometric matching in searching for an
improved match.
14. The system of claim 12, wherein the biometric system performs
biometric matching between the stored and the acquired biometric
information responsive to the adjustment, detects that an
orientation difference remains, and further adjusts one of the
stored and the acquired biometric information to search for an
improved biometric match.
15. The system of claim 12, wherein the biometric system adjusts
the stored or the acquired biometric information, the acquired
biometric information comprising information from a second image
acquired within a predetermined time of the first image.
16. The system of claim 12, wherein the biometric system adjusts
the stored or the acquired biometric information, the acquired
biometric information comprising information from a second image
acquired by a second sensor.
17. The system of claim 12, wherein the biometric system adjusts
the stored or the acquired biometric information, the acquired
biometric information comprising information from the first
image.
18. The system of claim 12, wherein the biometric system adjusts
the stored or the acquired biometric information, the acquired
biometric information comprising at least one of: iris and face
data.
19. The system of claim 12, wherein the biometric system adjusts,
based on the first image, an illumination level, a zoom level or a
focus setting for acquiring a second image.
20. The system of claim 12, wherein the biometric system extracts,
from the first image, an image portion comprising an iris of the
subject.
21. The system of claim 12, wherein the biometric system determines
the orientation of the at least one feature by determining a
relative position of the subject's eyes, in relation to the first
image's orientation.
22. The system of claim 12, wherein the biometric system determines
a pose of the subject's face, comprising one or more of: pan, tilt,
yaw and translation of the subject's face in three-dimensional
space.
Description
RELATED APPLICATIONS
[0001] This application is a continuation of, and claims priority
to U.S. application Ser. 12/596,019, filed under 35 U.S.C. 111(a)
on Oct. 15, 2009, which claims priority to International
application PCT/US08/60791, filed Apr. 18, 2008, which claims
priority to U.S. provisional application 60/925259, filed Apr. 19,
2007, all of which are hereby incorporated by reference in their
entireties for all references.
BACKGROUND
[0002] This disclosure relates generally to systems and methods
wherein imagery is acquired primarily to determine or verify the
identity of an individual person using biometric recognition.
[0003] Biometric recognition methods are widespread and are of
great interest in the fields of security, protection, financial
transaction verification, airports, office buildings, but prior to
the invention their ability to correctly identify individuals, even
when searching through a small reference database of faces or
irises, has always been limited, most notably when more than one
biometric type is acquired from a single sensor or more generally
when poor quality biometric data is acquired. In such cases there
are typically false positives (which means that the incorrect
person was identified) or false negatives (meaning that the correct
person was not identified).
[0004] There are several reasons for such poor performance of
biometric recognition methods.
[0005] First, when comparing a probe face to a reference face, it
is important that the biometric templates or features are
registered so that corresponding features (nose position for
example) can be compared accurately. Even small errors in
registration can result in matching errors even if the faces being
compared are from the same person.
[0006] Second, for facial or iris recognition, it is important that
the recognized face or iris and reference face or iris have the
same, or very similar, pose. Pose in this context means orientation
(pan, tilt, yaw) and zoom with respect to the camera. Variations in
pose between the images again results in matching errors even if
the faces being compared are from the same person.
[0007] Third, the dynamic range or sensitivity of the camera
sensor, optical system and digitization system (the Data
Acquisition System) may not be sufficient to capture biometric
information. For example, some biometric systems are multi-modal,
which means that they use several biometrics (for example, iris and
face) either to improve the accuracy of recognition or to provide a
quality image of the face for human inspection. In such multiple
biometric systems and methods there are problems in assuring that
each of the sets of data are from the same person, for example the
system may unintentionally capture the face of a first individual
and iris of a second individual, resulting in an identification or
match failure or incorrect association of the face from one person
with the iris of another person, for example. Another problem with
such multiple biometric systems is difficulty of obtaining good
data for each of the separate biometrics, e.g., face and iris
because, for example, the albedo or reflectance of one biometric
material (the iris for example) may be very different to the albedo
of a second biometric (the face for example). The result is that
the signal detected of one of the two biometrics is outside the
dynamic range or sensitivity of the Data Acquisition System and are
either saturated or in the dark current region of the Acquisition
System's sensitivity or simply appears as a uniform gray scale with
very poor contrast of biometric features, while the second
biometric signal is within the dynamic range or sensitivity of the
Data Acquisition System and has sufficient signal to noise ratio to
enable accurate biometric or manual recognition.
[0008] Fourth, the illumination may vary between the images being
matched in the biometric recognition system. Changes in
illumination can result in poor match results since detected
differences are due to the illumination changes and not to the fact
that a different person is being matched. In addition, due to the
variability in illumination and due to the limited dynamic range or
sensitivity of the Data Acquisition System, only some features of
the entire biometric (fingerprint or face for example) may be
within range of the Data Acquisition System and therefore suitable
for biometric matching. This can reduce the number of features
available for matching and also greatly reduces biometric
accuracy.
[0009] Since reflectance of a face is different from that of an
iris, acquiring an image of an iris and a face from the same person
with a single sensor according to prior methods and systems has
yielded poor results. Past practice required two cameras or sensors
or, in the cases of one sensor, the sensor and illuminators were
operated at constant settings.
[0010] For example, Adam, et al., US Pat. Publ. 20060050933 aims to
address the problem of acquiring data for use in face and iris
recognition using one sensor, but does not address the problem of
optimizing the image acquisition such that that the data acquired
is optimal for each of the face and iris recognition components
separately.
[0011] Determan, et al., U.S. Pat. Publ. 20080075334 and Saitoh, et
al., U.S. Pat. Publ. 20050270386 disclose acquiring face and iris
imagery for recognition using a separate sensor for the face and a
separate sensor for the iris. Saitoh claims a method for performing
iris recognition that includes identifying the position of the iris
using a face and iris image, but uses two separate sensors that
focus separately on the face and iris respectively and acquires
data simultaneously such that user motion is not a concern.
[0012] Determan also discusses using one sensor for both the face
and iris, but does not address the problem of optimizing the image
acquisition such that that the data acquired is optimal for each of
the face and iris recognition components separately.
[0013] Jacobson, et al., in US Pat. Publ. 20070206840 also
describes a system that includes acquiring imagery of the face and
iris, but does not address the problem of optimizing the image
acquisition such that that the data acquired is optimal for each of
the face and iris recognition components separately.
SUMMARY
[0014] We have discovered a method and related system for carrying
out the method which captures a high quality image of an iris and
the face of a person with single sensor or camera having a sensor
by acquiring at least two images with small time elapse between
each acquisition by changing the sensor or camera settings and/or
illumination settings between the iris acquisition(s) and the face
acquisition(s).
[0015] The system comprises a sensor, illuminator, and processor
adapted to acquire a high quality image of an iris of the person at
a first set of parameters. The parameters which can be varied
between the first biometric recognition step and the second
biometric recognition step, for example between iris and face
recognition steps, can include one or more of the following
parameters of the Data Acquisition System, by means of example:
illumination power setting, camera integration time, wavelengths,
frame grabber gain and offset, and frame grabber look-up-table. The
acquisitions of the first biometric and the second biometric are
within one second of each other, preferably within less than one
second of each other. For example, the elapsed time between
recognition steps where the parameters are varied can be as little
as 0.5, 0.25, 0.1,0.05, or even less, depending on the capability
of the sensor, illuminators, and processor.
[0016] The settings on the illuminator and/or sensor are also
changed within one second, and within one half, one quarter, or
even faster than one tenth of a second, depending on the
embodiment.
[0017] Some embodiments include the steps of, and related system
components or modules for, identifying one or more acquired images
containing the iris or face, performing registration over a
captured sequence between the identified acquired image,
constraining the search for the iris or face in the remainder of
the sequence in response to the results of the original identified
image, and the recovered registration parameters across the
sequence.
[0018] Certain embodiments of the invention include determining a
distance from the sensor by comparing a diameter of the iris in the
iris image with a reference table and/or comparing an separation
value between two eyes of the person with a reference table.
[0019] The system and method in some cases can adjust focus or zoom
as a function of a measured distance between two eyes of the person
and/or adjust illumination based on the distance from the sensor,
the distance calculated by comparing a diameter of the iris in an
iris image with a reference table.
[0020] In certain cases the method comprises changing one or more
sensor settings of the Data Acquisition System between the
acquisition of the face and iris selected from the group consisting
of sensor integration time, illumination, shutter speed, aperture,
gain and offset of the camera, gain and offset of the frame
grabber, and look-up tables in the frame grabber that may select
bits of precision that are different from that available from the
camera sensor output. The parameters which can be varied between
the first image and the second image can be, for example,
illumination pulse setting, illumination amplitude setting, camera
integration time, camera gain setting, camera offset setting,
camera wavelength, and frame grabber settings such as the look-up
table.
[0021] It is sometimes beneficial for the system to compute the
diameter, eccentricity and orientation of the iris upon acquisition
of the image of the iris with the sensor, to estimate eye
separation, pose of the iris, and/or pose the face.
BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The features and advantages of the invention will be
appreciated by reference to the detailed description when
considered in connection with the attached drawings wherein:
[0023] FIG. 1 is a schematic of a face of a person wherein the face
features are captured within the dynamic range of the sensor or
image grabbing device (the Data Acquisition System) with sufficient
contrast for accurate facial recognition, while on the other hand
the iris features are captured either outside the dynamic range of
the sensor or image grabbing device, or without sufficient contrast
for accurate iris recognition.
[0024] FIG. 2 is a schematic of the face of the person of FIG. 1
wherein the iris features are captured within the dynamic range of
the sensor or image grabbing device and with sufficient contrast
for accurate iris recognition, while on the other hand the face
features are captured either outside the dynamic range of the
sensor or image grabbing device, or without sufficient contrast for
accurate facial recognition.
[0025] FIG. 3 is a plan view of an image acquisition system
comprising a single sensor in a camera and a set of two
illuminators.
DETAILED DESCRIPTION
[0026] In the following detailed description, certain embodiments
will be illustrated by reference to the drawings, although it will
be apparent that many other embodiments are possible according to
the invention.
[0027] Referring first to FIG. 1, a face 10 is illustrated wherein
face features, including corners of eyes 11, wrinkles 12, 13, 16,
and 17, and corners of mouth 14 and nose 18 are visible but iris 15
is of low contrast and in poor detail. Such an image could be
obtained with low illumination settings with the sensor.
[0028] FIG. 2 illustrates the same face 10 as in FIG. 1 but with
the iris 15 of high contrast and the face features seen in FIG. 1
not acquired by the sensor but rather the facial features are of
low contrast 19.
[0029] FIG. 3 is a side view of an embodiment of a system according
to the invention wherein a subject 10 is to the left of the figure.
Images of the subject are captured by means of the camera 31 and
illumination is provided by means of an illuminator or, in the
illustrated embodiment, two sets of illuminators wherein first
illuminator set 32 are infra-red wavelength illuminators, for
example an Axis ACC IR Illuminator model 20812, and second
illuminator 33 is a set of Light Emitting Diode (LED) illuminators
which have a broad wavelength spectrum, for example Silicon Imaging
white LED illuminator 2-61617. In the illustrated embodiment a 2
megapixel resolution CCD camera 31 such as the Aegis PL-B956F model
is used with the two sets of illuminators 32, 33. The illumination
and camera settings are controlled by the camera and illumination
setting controller. The specific parameters controlled by the
controller include but are not limited to: camera gain, camera
offset, camera integration time, camera look-up table selection,
illumination pulse width for illuminator 1, illumination pulse
width for illuminator 2, illumination amplitude for illuminator 1,
and illumination amplitude for illuminator 2.
[0030] An optional range measurement module 30 can also be
included. This module 30 measures the approximate distance between
the cameras and/or illumination and the subject 10. There are many
devices known for measuring range. These include stereo depth
recovery methods, such as that described by Horn in "Robot Vision",
MIT Press, pages 202-242, or an acoustic/ultrasonic range sensor
such as that supplied by Campbell Scientific Inc, model number
SR50.
[0031] A second method to determine range is to measure the eye
separation or iris diameter.
[0032] In one embodiment, the system comprises a sensor, lens
system, illuminator, and processor adapted to acquire a high
quality image of an iris of the person at a first set of
illumination power setting, camera integration time, wavelengths
and lens settings; and to acquire with the same first sensor a high
quality image of the face of the person at a second set of
illumination power setting, camera integration time, wavelengths
and lens settings, wherein acquisitions of the face image and iris
image are within one second of each other, preferably within less
than one second of each other.
[0033] The settings on the illuminator and/or sensor and/or lens
system are also changed within one second, and within one half, one
quarter, or even faster than one tenth of a second, depending on
the embodiment.
[0034] Some embodiments include the steps of, and related system
components or modules for, identifying one or more acquired images
containing the biometric data, for example the iris or the face,
performing registration over a captured sequence between the
identified acquired image, constraining the search for the
biometric data, including the iris or face, in the remainder of the
sequence in response to the results of the original identified
image, and the recovered registration parameters across the
sequence. The recovered motion between the images may be due to the
motion of the person in the scene as they approach or recede from
the camera, or may be from motion induced by changes in the lens
parameters, such as zooming of the lens or pan and tilt control of
the camera.
[0035] Certain embodiments of the invention include determining a
distance from the sensor by comparing a diameter of the iris in the
iris image with a reference table and/or comparing an separation
value between two eyes of the person with a reference table.
[0036] The system and method in some cases can adjust focus or zoom
as a function of a measured distance between two eyes of the person
and/or adjust illumination based on the distance from the sensor,
the distance calculated by comparing a diameter of the iris in an
iris image with a reference table.
[0037] In certain cases the method comprises changing one or more
sensor and lens settings selected from the group consisting of
integration time, illumination, shutter speed, aperture, and gain
between the acquisitions of the face and the iris.
[0038] It is sometimes beneficial for the system to compute the
diameter, eccentricity and orientation of the iris upon acquisition
of the image of the iris with the sensor, to estimate eye
separation, pose of the iris, and/or pose the face. In one
embodiment, the wavelength and brightness of the illumination are
varied. More specifically, the camera and illumination parameters
are controlled as follows: The visible illuminator 33 is set to
provide constant illumination for all acquired frames with a
magnitude of 50 milliamps. The remaining IR illuminator 32 is set
to a constant pulse width of 6 msecs, but to a pulse magnitude that
varies between the two values of 0 milliamps and 400 milliamps
between alternate frames that are acquired by the camera. The
camera may be set to acquire frames at 3 frames a second The camera
integration time may be set at 6 msecs, and the camera gain and
offset and camera lookup table may be set to constant values. Those
constant values are chosen such that the image of the iris captured
when the current is being passed to the Infra-Red illuminator has
enough signal to noise for accurate biometric matching. By this
embodiment, the images acquired when the current is not being
passed to the infrared illuminator are suitable for accurate facial
recognition.
[0039] In a second embodiment, the camera integration time is
varied. More specifically, the camera and illumination parameters
are controlled as follows: The visible illuminator is set to
provide no illumination for any frame, or a constant illumination
for all acquired frames with a magnitude of 50 milliamps. The
remaining IR illuminator is set to a constant pulse width of 6
msecs, and to a constant pulse magnitude of 400 milliamps. The
camera may be set to acquire frames at 3 frames a second. The
camera integration time is set to alternate between adjacent frames
between the two values of 1.5 msecs and 6 msecs, and the camera
gain and offset and frame grabber lookup table (and other
parameters of the Data Acquisition System) may be set to constant
values. Those constant values are chosen such that the image of the
iris captured when the camera uses the longer integration time has
enough signal to noise for accurate biometric matching. In this
embodiment, the images acquired when the shorter integration time
are suitable for accurate facial recognition.
[0040] A third embodiment is the same as the first embodiment,
excepting that the magnitude of one or both of the visible
illuminators (set at 50 milliamps in embodiment 1) and IR
illuminators (set at 400 milliamps in embodiment 1) is adjusted in
response to an output of either the processor and/or the depth
measurement sensor. More specifically, the processor or range
measurement sensor provides an estimate of the range of the
subject. This estimate is then used to look-up a preferred
intensity magnitude for each of the visible and infra-red
illuminators which is then provided to the illuminators. These
preferred values are selected by acquiring data from a wide range
of subjects under different intensity magnitude settings and at
different distances, and by empirically finding the settings that
provide the best performance for biometric recognition of the face
and iris respectively.
[0041] A fourth embodiment first acquires data as described in the
first embodiment. In a second step however the images that are
optimized for acquiring data of each of the face and iris are
aligned using the methods described in this invention, in order to
remove any subject or camera motion that may have occurred between
the two time instants that each of the optimized data was acquired
from the sensor. In this way the features that are optimal for
facial recognition in one image can be corresponded to features
that are optimal for iris recognition in the other image. This
allows processing performed on one image to be used to constrain
the results of processing on the other image. For example, recovery
of the approximate position and orientation of the face in one
image can then be used to constrain the possible position and
orientation of the iris in the second image. Similarly, recovery of
the position of the iris in one image constrains the possible
position of the face in the second image. This can assist in
reducing the processing time for one or other of the biometric
match processes, for example. In another example, some facial
features are most accurately localized and have best signal to
noise properties under one set of camera or illumination
conditions, whereas another set of facial features are most
accurately localized and have best signal to noise properties under
another set of camera or illumination settings. This method allows
the most accurately localized features of all facial features to be
used for facial recognition, thereby providing improved recognition
performance. More specifically, the features can be combined by
selecting which features from which image have highest signal to
noise. There are several methods for feature selection, for
example, a contrast measure such as an edge detector can be
performed over the image (for example see Sobel, I., Feldman, G.,
"A 3.times.3 Isotropic Gradient Operator for Image Processing",
Pattern Classification and Scene Analysis, Duda, R. and Hart, P.,
John Wiley and Sons, '73, pp 271-272), and the magnitude of the
result can be used to select the feature from either image with the
largest contrast.
[0042] The image resolution typically required for face recognition
is recognized to be approximately 320.times.240 pixels, as
documented in an ISO standard. As part of the image acquisition
system, however, we use a camera capable of imaging the face with a
much higher resolution, for example 1024.times.1024 pixels. This
higher resolution data enables the detection and localization of
features that cannot be detected reliably in the lower resolution
data, and also enables more precise and robust detection of
features that could be seen in the lower resolution imagery. For
example, the precise location of the pupil boundary can be
recovered in the high resolution imagery and typically cannot be
recovered accurately in the lower resolution imagery. One method
for detecting the pupil/iris boundary is to perform a Hough
transform, for example. U.S. Pat. No. 3,069,654. The face
recognition algorithm may use the same high resolution data that is
being captured or by using an additional low resolution camera. An
additional method for performing registration is to perform
alignment algorithms over the eye region. In this case the eye
region in one face image is aligned to sub-pixel precision to the
eye region in another face image. Registration can also be done
over the entire image or over the face region. Registration can be
performed for example by methods described in Horn, "Robot Vision",
MIT Press p 278-299. The precise localization information can be
passed to the face recognition algorithm in order to improve its
performance.
[0043] In addition to eye location, the image acquisition system
also recovers the zoom or distance of the person. This is
accomplished by setting the high resolution camera to have a very
narrow depth of field. This means that features of the face only
appear sharply in focus at a specific distance from the camera.
Methods can be performed to detect when those features are sharply
in focus, and then only those images are selected for face
recognition. If a second lower resolution camera is used for
acquiring the data used for face recognition, then processing
performed on the high-resolution imagery to detect sharply-focused
features is used to trigger image acquisition on the lower
resolution camera. This ensures that the face images used for face
recognition are all at the identical scale. There are several
methods available to detect sharply-focused features. For example,
an edge filter can be performed over the image (see Sobel, I.,
Feldman, G., "A 3.times.3 Isotropic Gradient Operator for Image
Processing", Pattern Classification and Scene Analysis, Duda, R.
and Hart, P., John Wiley and Sons, '73, pp 271-272) and then
squared at each pixel, and then averaged over the image in order to
compute an edge energy score. When the score is maximal or exceeds
a threshold, then the person is within the depth of field of the
high resolution camera.
[0044] Knowledge of the eye location as well as the zoom of the
face allows specific sub-regions of the face to be selected and
used for face recognition. For example, one or more rectangular
regions of a certain size (in pixels) can be cut out from the
high-resolution imagery and used as an input to a face recognition
engine, even though only part of the face is being presented. The
locations of certain areas, such as the nose, can be predicted
using a model of a standard face, and using knowledge of the eye
location and the zoom. The face recognition engine is informed that
only one or more specific subsets of the face are being presented.
In this case we only provide a face recognition database to the
face recognition engine that comprises only the same specific
subset regions.
[0045] If a second camera is used to acquire data for face
recognition (or any other biometric recognition, such as ear
recognition) then because the location of the first camera is
different to that of the second camera, then recovering a precise
pixel location in the first camera does not simply translate into a
corresponding pixel location in the second camera. We accomplish
this using knowledge of the location of the depth of field of the
first camera, which in turn provides a very precise depth of the
face with respect to the first camera. Given a pixel location in
the first camera, and the depth of the person, as well as camera
intrinsics (such as focal length, and relative camera translation)
that can be calibrated in advance (see for example, "An Efficient
and Accurate Camera Calibration Technique for 3D Machine Vision",
Roger Y. Tsai, Proceedings of IEEE Conference on Computer Vision
and Pattern Recognition, Miami Beach, Fla., 1986, pages 364-374),
then it is known how to compute the precise pixel location of the
corresponding feature in the second camera (see Horn, "Robot
Vision", MIT Press, p 202-242 for example).
[0046] In addition to ensuring consistent zoom, we also take steps
to ensure consistent pose by detecting features in the
high-resolution image that would not otherwise be visible with
precision in the low resolution imagery. For example, the pupil
boundary is only near-circular if the person is looking in the
direction of the camera. A method for detecting circular or
non-circular boundaries is the Hough Transform, for example see
U.S. Pat. No. 3,069,654. If imagery of the iris is near circular in
the narrow field of view imagery, then imagery of the face is in
the lower resolution camera is more likely to be of a frontal view
and is passed to the facial recognition module. Similarly, the pose
of the face can be recovered and used to constrain the expected
pose of the iris for subsequent processing.
[0047] The image acquisition system also has a dynamic range
control module. This module addresses the problem where data from
two different biometrics (e.g. iris and face) cannot be reliably
acquired because the dynamic range of the sensor, and the Data
Acquisition System in general, is limited. We address this by two
methods.
[0048] First, we acquire data at two different but controlled times
in such a way that at the first time instance we expect that the
first biometric imagery (e.g., face) imagery will be within the
dynamic range of the sensor given the specific illumination
configuration. We then acquire data at a second time instance where
we expect the second biometric imagery (e.g. iris imagery) to be
within the dynamic range or sensitivity of the sensor. For example,
consider a configuration where a camera and an illuminator lie
close to each other, and a person is approaching the configuration.
Images are continuously captured. As the person approaches the
configuration, then the reflectance off the biometric tissue (face
or iris) increases since the distance from the person to the camera
and illumination configuration is decreasing. At one distance it
can be expected that data corresponding to one biometric will be
within the dynamic range (e.g. face) while at a different distance,
it can be expected that data corresponding to a second biometric
can be within the dynamic range (e.g. iris). The camera may have a
small depth of field due to the resolution requirements of
obtaining one biometric (e.g. the iris). However, the resolution
required for the other biometric may be much coarser so that
blurring due to imagery lying outside the depth of field has
negligible impact on the quality of data acquired for the other
biometric (e.g. the face).
[0049] A specific implementation of this approach is to a) Acquire
all images into a stored buffer, b) detect the presence of an eye
in the depth of field of the camera using the methods described
earlier, c) compute the number of frames back in time where the
person was situated at a further distance from the depth of field
region (and therefore illuminated less), based on a prediction of
their expected motion (which can be, for example, a fixed number
based on walking speed), and d) select that imagery from the buffer
to be used for face recognition. The eye and face location can be
registered over time in the buffer to maintain knowledge of the
precise position of the eyes and face throughout the sequence.
Registration can be performed for example by Horn, "Robot Vision",
MIT Press p 278-299
[0050] The second method for ensuring that data lies within the
dynamic range of the camera is to modulate the magnitude of the
illumination over a temporal sequence. For example, in one frame
the illumination can be controlled to be much brighter than in a
subsequent frame. In one implementation, images are always acquired
at a low illumination level suitable for one biometric. Features
are detected that would only be observed when the face is fully in
focus and within the depth of field region. For example, the Sobel
image focus measure previously cited can be used. When the face is
near or within the depth of field region, based for example on a
threshold of the focus measure, then the illumination can be
increased in order to obtain imagery of the second biometric (e.g.
the iris) within the dynamic range of the camera. When the face has
left the depth of field region, then the illumination can revert
back to the lower magnitude level.
[0051] In addition to modulating the magnitude of the illumination,
we also modulate the wavelength of the illumination. This allows
multiple datasets corresponding to the same biometric to be
acquired and matched independently, but using the constraint that
the data belongs to the same person. For example, person A may
match dataset Band C using data captured at one wavelength, but
person A may match dataset C and D using data captured at a second
wavelength. This gives evidence that person A matches to dataset C.
This approach is extended to not only include fusing the results of
face recognition after processing, but also by including the
multi-spectral data as a high dimensional feature vector as an
input to the face recognition engine.
[0052] In some embodiments there is an advantage to aligning the
acquired images of the face and iris with the processor, thereby
reducing the effect of camera or subject motion that may have
occurred between the two time instants that each of the images was
acquired from the sensor. In this way the features that are optimal
for facial recognition in one image can be corresponded to features
that are optimal for iris recognition in the other image. This
allows processing performed on one image to be used to constrain
the results of processing on the other image. For example, recovery
of the approximate position and orientation of the face in one
image can then be used to constrain the possible position and
orientation of the iris in the second image. Similarly, recovery of
the position of the iris in one image constrains the possible
position of the face in the second image. This can assist in
reducing the processing time for one or other of the biometric
match processes, for example. In another example, some facial
features are most accurately localized and have best signal to
noise properties under one set of camera or illumination
conditions, whereas another set of facial features are most
accurately localized and have best signal to noise properties under
another set of camera or illumination settings. This method allows
the most accurately localized features of all facial features to be
used for facial recognition, thereby providing improved recognition
performance.
[0053] The present invention, therefore, is well adapted to carry
out the objects and attain the ends and advantages mentioned, as
well as others inherent therein. While the invention has been
depicted and described and is defined by reference to particular
preferred embodiments of the invention, such references do not
imply a limitation on the invention, and no such limitation is to
be inferred. The invention is capable of considerable modification,
alteration and equivalents in form and function, as will occur to
those ordinarily skilled in the pertinent arts. The depicted and
described preferred embodiments of the invention are exemplary only
and are not exhaustive of the scope of the invention. Consequently,
the invention is intended to be limited only by the spirit and
scope of the appended claims, giving full cognizance to equivalents
in all respects.
* * * * *