U.S. patent application number 14/855073 was filed with the patent office on 2016-01-14 for efficient texture comparison.
The applicant listed for this patent is Apple Inc.. Invention is credited to Byron B. Han, Craig A. Marciniak, Wayne C. Westerman.
Application Number | 20160012273 14/855073 |
Document ID | / |
Family ID | 49581341 |
Filed Date | 2016-01-14 |
United States Patent
Application |
20160012273 |
Kind Code |
A1 |
Westerman; Wayne C. ; et
al. |
January 14, 2016 |
Efficient Texture Comparison
Abstract
A scannable object is sensed and scanned. A map is constructed
based on the scan results. The map is compared to one or more
stored templates. Results of the comparison are provided. In some
implementations, a secured processor may construct the map and may
provide reduced resolution (and/or other versions that contain less
information) versions of the map and/or the stored templates to one
or more other processors. The one or more other processors may
determine a match-set based on matching between the reduced
resolution map and stored templates. The secured processor may then
identify whether or not a match exists between the map and any
stored template based on the match-set.
Inventors: |
Westerman; Wayne C.;
(Cupertino, CA) ; Han; Byron B.; (Cupertino,
CA) ; Marciniak; Craig A.; (Cupertino, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Apple Inc. |
Cupertino |
CA |
US |
|
|
Family ID: |
49581341 |
Appl. No.: |
14/855073 |
Filed: |
September 15, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13797970 |
Mar 12, 2013 |
9135496 |
|
|
14855073 |
|
|
|
|
61649210 |
May 18, 2012 |
|
|
|
Current U.S.
Class: |
382/124 |
Current CPC
Class: |
G06F 21/602 20130101;
G06F 3/041 20130101; G06K 9/0008 20130101; G06K 9/00067 20130101;
G06K 9/00013 20130101; G06K 9/00006 20130101; G06F 21/32
20130101 |
International
Class: |
G06K 9/00 20060101
G06K009/00; G06F 3/041 20060101 G06F003/041 |
Claims
1-20. (canceled)
21. A system, comprising: a fingerprint sensor configured to
capture fingerprint data; a secure processor operably connected to
the fingerprint sensor, the secure processor configured to:
construct a fingerprint map based on the fingerprint data; generate
a lower resolution fingerprint pattern that represents the
fingerprint map; and generate a lower resolution template pattern
that represents a fingerprint template; and a second processor
operably connected to the secure processor, the second processor
configured to: compare the lower resolution fingerprint pattern
with the lower resolution template pattern to determine if the
lower resolution fingerprint pattern matches the lower resolution
template pattern; and provide a result of the comparison to the
secure processor.
22. The system of claim 21, wherein the secure processor is
configured to determine if the fingerprint map matches the
fingerprint template when the result of the comparison indicates
the lower resolution fingerprint pattern matches the lower
resolution template pattern.
23. The system of claim 21, further comprising a memory operably
connected to the secure processor, the memory storing the
fingerprint template.
24. The system of claim 23, wherein the secure processor is
configured to encrypt the fingerprint template and store the
encrypted fingerprint template in the memory.
25. The system of claim 24, wherein the secure process is
configured to decrypt the fingerprint template prior to generating
the lower resolution template pattern that represents the
fingerprint template.
26. The system of claim 21, wherein the second processor comprises
an unsecured processor.
27. The system of claim 21, wherein the fingerprint sensor, the
memory, the secure processor, and the second processor are included
in an electronic device.
28. The system of claim 21, wherein the memory, the secure
processor, and the second processor are included in an electronic
device and the fingerprint sensor is operably connected to the
electronic device.
29. The system of claim 21, wherein: the reduced resolution
fingerprint pattern comprises at least one of a checksum generated
from the fingerprint map, a hash generated from the fingerprint
map, or a histogram generated from the fingerprint map; and the
reduced resolution template pattern comprises at least one of a
checksum generated from the fingerprint template, a hash generated
from the fingerprint template, or a histogram generated from the
fingerprint template.
30. A method, comprising: producing, by a secure processor, a
fingerprint pattern based on a fingerprint map, the fingerprint
pattern containing less information than the fingerprint map;
producing, by the secure processor, one or more template patterns
based on associated fingerprint templates, each template pattern
containing less information than an associated fingerprint
template; transmitting the fingerprint pattern and the one or more
template patterns from the secure processor to a second processor;
identifying, by the second processor, at least one template pattern
that matches the fingerprint pattern; identifying, to the secure
processor, the at least one template pattern that matches the
fingerprint pattern; and determining, by the secure processor, if
the fingerprint map associated with the fingerprint pattern matches
at least one fingerprint template associated with each identified
template pattern.
31. The method of claim 30, further comprising: encrypting, by the
secure processor, the one or more fingerprint templates; and
storing the encrypted one or more fingerprint templates in a
memory.
32. The method of claim 31, further comprising: prior to producing
the one or more template patterns, reading, by the secure
processor, the one or more encrypted fingerprint templates from the
memory; and decrypting, by the secure processor, the one or more
fingerprint templates.
33. The method of claim 31, wherein the memory comprises a secure
memory.
34. The method of claim 30, wherein the fingerprint pattern
comprises at least one of a checksum generated from the fingerprint
map, a hash generated from the fingerprint map, or a histogram
generated from the fingerprint map.
35. The method of claim 30, wherein each template pattern comprises
at least one of a checksum generated from the fingerprint template,
a hash generated from the fingerprint template, or a histogram
generated from the fingerprint template.
36. A system, comprising: a fingerprint sensor configured to
capture fingerprint data; a secure processor operably connected to
the fingerprint sensor, the secure processor configured to:
construct a fingerprint map based on the fingerprint data; produce
fingerprint pattern based on the fingerprint map, the fingerprint
pattern containing less information than the fingerprint map; and
produce one or more template patterns based on associated
fingerprint templates, each template pattern containing less
information than an associated fingerprint template; a second
processor operably connected to the secure processor, the second
processor configured to: compare the fingerprint pattern with each
template pattern; identify at least one template pattern that
matches the fingerprint pattern; and provide a result of the
comparison to the secure processor, wherein, based on the result of
the comparison, the secure processor is configured to determine if
the fingerprint map associated with the fingerprint pattern matches
at least one fingerprint template associated with each identified
template pattern.
37. The system of claim 36, wherein the secure processor is
configured to: encrypt each fingerprint template and store the
encrypted fingerprint template in a memory operably connected to
the secure processor; and decrypt the associated fingerprint
templates prior to generating the one or more template
patterns.
38. The system of claim 36, wherein the fingerprint sensor, the
memory, the secure processor, and the second processor are included
in an electronic device.
39. The system of claim 36, wherein the memory, the secure
processor, and the second processor are included in an electronic
device and the fingerprint sensor is operably connected to the
electronic device.
40. The system of claim 36, wherein: the fingerprint pattern
comprises at least one of a checksum generated from the fingerprint
map, a hash generated from the fingerprint map, or a histogram
generated from the fingerprint map; and each template pattern
comprises at least one of a checksum generated from the fingerprint
template, a hash generated from the fingerprint template, or a
histogram generated from the fingerprint template.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. patent
application Ser. No. 13/797,970, filed Mar. 12, 2013, entitled
"Efficient Texture Comparison," which claims the benefit under 35
U.S.C. .sctn.119(e) to U.S. Provisional Patent Application No.
61/649,210 filed May 18, 2012, entitled "Efficient Texture
Comparison," both of which are incorporated by reference in their
entirety as if fully disclosed herein.
TECHNICAL FIELD
[0002] Embodiments described herein relate generally to a device
and process for efficient texture pattern comparison and matching,
and more specifically to fingerprint matching on a portable
device.
BACKGROUND DESCRIPTION
[0003] Fingerprint sensing technology has become widespread in use
and is often used to provide secure access to sensitive electronic
devices and/or data. Generally, capacitive fingerprint sensors may
be used to determine an image of a fingerprint through measuring
capacitance through each pixel of a capacitive sensor. The higher
the capacitance, the nearer the surface of an adjacent or overlying
finger to the pixel. Thus, fingerprint ridges provide a higher
capacitance in an underlying pixel than do fingerprint valleys.
There are other types of fingerprint sensors, such as optical
sensors.
[0004] Typically, fingerprint sensors have been tied to relatively
powerful computers, such as PCs or laptops, or incorporated in
specialty devices specifically designed for fast processing and
sufficient battery life of the scanner.
[0005] Portable user devices, such as smart phones and tablets, are
more and more common, and include more and more features and
functions. Such devices become more powerful and less battery
intensive all the time, but still have relatively smaller
computational resources and a constant concern over battery
consumption rates.
[0006] Accordingly, there is a need for an improved functionality
in highly mobile devices, and a need for a computationally
efficient implementation of the improved functionality.
SUMMARY
[0007] The present disclosure provides systems, methods, and
apparatuses for efficient texture comparison. A scannable object
may be sensed and scanned. A map may be constructed based on the
scan results. The map may be compared to one or more stored
templates. Results of the comparison may be provided.
[0008] In some implementations, a secured processor may construct
the map and may provide reduced resolution (and/or other versions
that contain less information) versions of the map and/or the
stored templates to one or more other processors. The one or more
other processors may determine a match-set based on matching
between the reduced resolution map and stored templates. The
secured processor may then identify whether or not a match exists
between the map and any stored template based on the match-set.
BRIEF DESCRIPTION OF THE FIGURES
[0009] FIG. 1 depicts a block diagram of a sample capacitive
sensing array.
[0010] FIG. 2 depicts a sample electronic device incorporating the
embodiment of a capacitive sensing array.
[0011] FIG. 3 is a cross-sectional view taken along line 4-4 of
FIG. 2, showing the embodiment of a capacitive sensing array
incorporated into a stack-up with an input device.
[0012] FIG. 4 is an exemplary process for efficiently matching a
scanned pattern according to one exemplary embodiment.
[0013] FIG. 5 is an exemplary system for efficiently matching a
scanned pattern according to one exemplary embodiment.
[0014] FIG. 6 is an exemplary process for efficiently and securely
matching a scanned pattern according to one exemplary
embodiment.
DETAILED DESCRIPTION
[0015] Generally, embodiments discussed herein may provide
efficient and secure texture sensing on a device, such as a smart
phone. For example, a smart phone touch screen can be configured
with a fingerprint sensor (e.g., a capacitive sensor) over part or
all of the touch screen interface, the device housing, and/or other
device inputs.
[0016] The present disclosure provides systems, methods, and
apparatuses for efficient texture comparison. A scannable object
may be sensed and scanned. A map may be constructed based on the
scan results. The map may be compared to one or more stored
templates. Results of the comparison may be provided. It should be
appreciated that embodiments described herein may be used with any
suitable fingerprint sensor, including swipe or strip sensors,
two-dimensional array sensors, and the like.
[0017] In some implementations, a secured processor may construct
the map and may provide reduced resolution (and/or other versions
that contain less information) versions of the map and/or the
stored templates to one or more other processors. The one or more
other processors may determine a match-set based on matching
between the reduced resolution map and stored templates. The
secured processor may then identify whether or not a match exists
between the map and any stored template based on the match-set.
[0018] FIG. 2 depicts an electronic device 200 that may incorporate
a fingerprint sensor, e.g., a capacitive sensor. The electronic
device may be a mobile telephone, a tablet computing device, a
notebook computer, a personal digital assistant, a desktop
computer, a portable media player, and the like. The sensor pad may
be placed anywhere on device 200, such as below an input mechanism,
e.g., button 210, an input and/or output mechanism, e.g., screen
220, and/or a casing/housing, e.g., device housing 230 of the
electronic device. The sensor may occupy part of an area (e.g.,
part of button 210), a whole area (e.g., all of screen 220), or an
area that spans part/all of more than one of the areas. For
example, a sensor may cover screen 220 and extend past the edge,
covering all or part of forehead area 232 and/or chin area 234.
Essentially, any portion of the electronic device's enclosure may
house the fingerprint sensor.
[0019] In certain exemplary embodiments, the device can include a
separate attachment, such as external scan accessory 240. Accessory
240 is shown connected to device I/O port 245, which could be via a
flexible wire connection, a ridged connection (e.g., simulating an
extension of the device housing via a fastening mechanism (e.g., a
snap together interface)). In other exemplary embodiments, this
connection can be wireless via a proprietary protocol or a common
protocol (e.g., Bluetooth, WiFi, GSM/CDMA/4G, etc.).
[0020] In certain exemplary embodiments, as mentioned above, the
sensor may be included within the device housing, display, or other
area, such as input button 210. FIG. 3 illustrates one exemplary
embodiment of a fingerprint scanner/sensor disposed beneath button
210. FIG. 3 is a cross-sectional view of the electronic device of
FIG. 2, taken along line 4-4 of FIG. 2, which may include the
layers: cover dielectric 301, ink 302, liquid adhesive 303, silicon
TSV (3um pass) 304, solder 305, flex 306, air gap 307, stiffener
308, adhesive 309, flex 310, tact 311, shim 312, adhesive 313, and
bracket 314. As shown in FIG. 3, the fingerprint sensor chip
(including both sensor pad and drive ring) may be positioned
beneath the button (e.g., 210), which may be the cover dielectric
301. As illustrated, the top layer cover dielectric 301 is concave,
as exemplary button 210 may be concave. A similar illustration,
with differing dimensions and features, could show a flat screen in
this layer extending to a housing, etc. In the exemplary
embodiments of a button sensor, an ink layer and/or adhesive may be
placed between the button's bottom surface and the sensor chip's
top surface. The adhesive may bond the chip to the button, for
example. One or more solder balls may affix the fingerprint sensor
chip to a flex conductor. The solder balls may generally be placed
near the center of the fingerprint sensor chip to reduce the
likelihood of cracking due to stress.
[0021] The exemplary scanner shown in FIG. 3, accessory 240, and/or
any other configuration incorporating a texture sensor/scanner with
a user device may include a capacitive sensor (e.g., the same,
similar, or different than the capacitive sensor shown in FIG. 1),
or any number of other types of sensors capable of sensing a
texture/pattern of an adjacent or proximate object (e.g., an
optical sensor) can be used in one or more exemplary
embodiments.
[0022] Regardless of the location or configuration of the sensor,
the exemplary device, including the exemplary sensor, can execute
an exemplary process for matching a scanned texture with stored
templates. FIG. 4 illustrates one such exemplary process. The
exemplary process may start at 410 by sensing or detecting a
scannable object. This may be a low power state, where power
consumption is reduced while waiting for a sensed object. A
scannable object can be one close to the device scanner or in
contact with the device scanner. In other exemplary embodiments,
the object may be "scannable" if it has a texture that can be
detected, and in other exemplary embodiments an object may be
scannable based on proximity, while the texture (or lack thereof)
can be detected later in the exemplary process.
[0023] Once a scannable and/or proximate object has been detected,
the exemplary process (e.g., using the exemplary device and sensor)
can scan the object at 420. The sensor results, which may vary
depending on the type of sensor used (e.g., capacitive, optical,
etc.), can then be used to construct a map associated with (e.g.,
descriptive of) the scanned features of the objects texture at
430.
[0024] One such exemplary map can include a ridge flow map or
direction map, which represents the direction of ridge flow within
the scanned fingerprint image. As just one example of how a ridge
flow map can be computed and stored: the exemplary map may contain
a grid of integer directions, where each cell in the grid
represents, e.g., an 8.times.8 pixel neighborhood in the image.
Ridge flow angles can be quantized into, e.g., 16 integer
bi-directional units equally spaced on a semicircle. In this
example, starting with vertical direction 0, direction units can
increase clockwise and represent incremental jumps of 11.25
degrees, stopping at direction 15 which is 11.25 degrees shy of
vertical. Using this scheme, direction 8 is horizontal. A value of
-1 in this map represents a neighborhood where no valid ridge flow
was determined. Other exemplary methods of producing a ridge flow
map are also possible, including different sizes, value ranges,
matrix configurations, etc. Further, other map types are also
possible, such as a quality map, contrast map, etc.
[0025] FIG. 5 shows an exemplary system that can be used to execute
one or more exemplary processes. The exemplary system can include a
sensor 540, which can be sensor 100, sensor 240, the sensor of FIG.
3, or any number of other exemplary sensors. This sensor can
include a separate encryption/security feature/module (not shown)
or send data to processor block 500 without a separate security
module. The processor 500 can include an application processor (AP)
510 and a secure enclave processor (SEP) 520. Each of these
processors can include multiple processors, multiple cores, or
reside on the same processor. The application processor 510 can be
a general processor, responsible for several processing tasks of
the device it resides within. The secure enclave processor 520 can
be specially and/or specifically designed/configured to perform
encrypted tasks, such as encrypting data associated with an
authorized user's fingerprint/ID-pattern.
[0026] Processor block 500 can be connected to sensor 540 by any
number or wired or wireless connections, using any number of
transmission protocols, such as a serial peripheral interface
(SPI). Processor block 500 can also be connected to a data storage
repository 550, which can include any number of mediums (e.g.,
magnetic material, solid state memory, etc.) Data repository 550
can include a secure data repository 555, which can include
encrypted data, e.g., data associated with an authorized user's
fingerprint/ID-pattern. Secure repository 555 can be separate from
the main repository 550 or a part of the main repository 550. In
the example of fingerprint patterns (e.g., maps based on a scanned
fingerprint pattern), the repository can store files for multiple
authorized users, files for multiple fingers (e.g., 10) of each
user, multiple files for each finger, etc.
[0027] In a first exemplary operation, sensor 540 can scan a
texture of an object. This texture can be translated into an
associated map by sensor 540, AP 510, or SEP 520. The SEP 520 can
then retrieve encrypted templates (e.g., based on patterns
associated with authorized users), and match the translated map
with the encrypted templates. The SEP (e.g., via the AP, operating
system (OS), and input/output devices (I/O)) can then provide a
result, such as maintaining the screen lock (no match), or
unlocking the device. Personal settings associated with the
particular authorized user can also be pre-loaded at unlock.
[0028] The SEP 520 may have less computational resources than the
more general processor AP 510, and thus be some degree slower. In
order to provide efficient and faster matching, certain exemplary
embodiments may push some or all of the matching operations to the
AP 510. The AP 510 can identify a match and provide a result or
identify the match so that a result can be provided. In one
exemplary embodiments, the SEP 520 may decrypt the match templates
and pass them to the AP 510 for match processing. While the SEP 520
may be needed for encryption/decryption (as AP 510 may be
unsecured), the process can be greatly sped up, as the SEP 520 only
has to perform tasks it was designed for (encryption/decryption),
while the more powerful AP 510 can perform the more computationally
intensive matching procedures.
[0029] A potential drawback of the above described exemplary
embodiment can be that the AP 510 is unsecured or partially
unsecured, and certain exemplary template maps may contain
sufficient information that a malicious unauthorized user (e.g.,
someone who steals the device) could reverse engineer the exemplary
template to construct a pattern that could unlock the device (e.g.,
sufficiently mimic an authorized user's fingerprint pattern). For
example, an unauthorized user could intercept a decrypted template
from the unsecured AP 510, and use the template data to construct
an artificial object with associated properties (e.g. properties
that when scanned would produce data that matched the intercepted
template).
[0030] To overcome this potential security drawback, another
exemplary embodiment of the present disclosure can include a
process of collapsing the full maps into a sort of checksum, hash
function, or histogram. For example, each encrypted ridge map
template can have some lower resolution pattern computed and
associated with the ridge map. One exemplary pattern could be a
histogram of, e.g., the most common angles (e.g., a 2 dimensional
(2D) array of common angles). The exemplary pattern could include
in each slot an average value over a respective vector of the map.
The exemplary pattern could include in each slot a sum of the
values over a respective vector of the map. The exemplary pattern
could include the smallest or largest value within a respective
vector of the map, or could be a difference between a largest and a
smallest value within the respective vector of the map. The
exemplary pattern could simply be a particular vector, e.g., the
pattern is merely the Nth vector of the map. Exemplary patterns can
include more than one vector. For example, for an N by N map, the
exemplary pattern could be the four edge vectors (e.g., the
1.sup.st and Nth column, and the 1.sup.st and Nth row), or any
other sampling, positions, or calculated reduction. Numerous other
exemplary embodiments are also possible, and any other exemplary
pattern calculation can be used, where the exemplary pattern
includes enough associated information to narrow the candidate
list, while omitting enough associated information that the
unsecured pattern cannot or cannot easily be reverse engineered
into a matching texture.
[0031] In an exemplary process for this exemplary embodiment, a
scanned object can have a ridge map calculated from the scanner
input, e.g., in the SEP 520. This encrypted ridge map can then have
an unencrypted pattern calculated (according to the implemented
protocol) and sent to the AP 510. This pattern can be compared to
patterns associated with the stored encrypted templates, which can
be calculated in real-time or preferably be stored to reduce
computation. Several of the templates may be different, but have
the same or similar associated patterns, since two different
templates may have values the same or similar in the areas used to
determine the lower resolution patterns. Thus, the AP 510 may
return multiple positive results (and might also return a single
match or no matches as determined with the scanned pattern to be
compared). The SEP 520 can then access the encrypted ridge maps
associated with any patterns identified by the AP 510 as matching.
The SEP 520 can then compare the ridge map of the scanned pattern
with the small subset of possible matches, instead of the entire
library of possible matches. This exemplary embodiment can
therefore P1 greatly speed up the computation of map matching by
leveraging the powerful AP, while maintaining encrypted security of
the stored ridge maps.
[0032] As mentioned earlier, any number of other exemplary
embodiments are also possible, and the above example is presented
with certain specific implementations (e.g., using ridge maps for
patterns) for illustration purposes, but could be applied to any
number of other exemplary embodiments having other exemplary
implementations.
[0033] FIG. 6 illustrates an exemplary embodiment of this exemplary
process. At 610, the exemplary process can sense or detect an
object to scan. At 620 the exemplary process scans the object.
Secure process 604 then constructs an input map based on the scan
results at 630. The secure process 604 can then construct a lower
resolution pattern 635. Secure process 604 can then load,
determine, or otherwise provide stored template patterns
(associated with stored template maps) and the input pattern to a
process 602, which can be unsecured, partially secured, secured
with a different protocol, or secured in the same manner as process
604. Process 602 can then run a match comparison of the input
pattern and the received template patterns at 650. At 655, process
602 can provide the identity of possible match results to secured
process 604. This can be a pointer, an identification, or the
actual matching pattern. The secure process 604 can then run (e.g.,
at 660) a full match comparison of the input map and the stored
templates associated with those possible matches identified at 655.
Finally, the exemplary process can provide the results at 670.
[0034] Although embodiments have been described herein with respect
to particular configurations and sequences of operations, it should
be understood that alternative embodiments may add, omit, or change
elements, operations and the like. Accordingly, the embodiments
disclosed herein are meant to be examples and not limitations.
* * * * *