U.S. patent application number 17/170183 was filed with the patent office on 2021-09-09 for authentication of a suspect object using extracted native features.
The applicant listed for this patent is ALITHEON, INC.. Invention is credited to Brian Elmenhurst, John Forbes, David Ross, Heather Wheelock Ross, Mark Tocci.
Application Number | 20210279462 17/170183 |
Document ID | / |
Family ID | 1000005611545 |
Filed Date | 2021-09-09 |
United States Patent
Application |
20210279462 |
Kind Code |
A1 |
Ross; David ; et
al. |
September 9, 2021 |
AUTHENTICATION OF A SUSPECT OBJECT USING EXTRACTED NATIVE
FEATURES
Abstract
A forgery detection system includes a computer server and a
database system of digital fingerprint records corresponding to
forged or altered objects of a given object type. Using the
computer server, a digital image of a suspect object of the given
object type is accessed, an authentication region is selected, and
a native feature within the authentication region is extracted. The
native feature describes physical characteristics of the
authentication region without recognizing content that appears in
the authentication region. The computer server forms a feature
vector to represent the native feature in a compact form and
queries the database system to obtain a result responsive to
digital fingerprint records that match the feature vector. Each
matching digital fingerprint record is counted, and if the count of
fraud indicator matches crosses a predetermined threshold
indicating a confidence level that the suspect object is forged or
altered, a report based is generated and communicated to a user
interface.
Inventors: |
Ross; David; (Bellevue,
WA) ; Elmenhurst; Brian; (Bellevue, WA) ;
Tocci; Mark; (Bellevue, WA) ; Forbes; John;
(Bellevue, WA) ; Ross; Heather Wheelock;
(Bellevue, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
ALITHEON, INC. |
Bellevue |
WA |
US |
|
|
Family ID: |
1000005611545 |
Appl. No.: |
17/170183 |
Filed: |
February 8, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16041710 |
Jul 20, 2018 |
10915749 |
|
|
17170183 |
|
|
|
|
15044034 |
Feb 15, 2016 |
10043073 |
|
|
16041710 |
|
|
|
|
14531307 |
Nov 3, 2014 |
9582714 |
|
|
15044034 |
|
|
|
|
14290653 |
May 29, 2014 |
9350552 |
|
|
14531307 |
|
|
|
|
13410753 |
Mar 2, 2012 |
8774455 |
|
|
14290653 |
|
|
|
|
13618362 |
Sep 14, 2012 |
9152862 |
|
|
14531307 |
|
|
|
|
61898780 |
Nov 1, 2013 |
|
|
|
61914722 |
Dec 11, 2013 |
|
|
|
61448465 |
Mar 2, 2011 |
|
|
|
61535084 |
Sep 15, 2011 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 16/583 20190101;
G06K 2209/01 20130101; G06F 16/2468 20190101; G06K 9/3216 20130101;
H04L 9/3247 20130101; G06K 9/00483 20130101; G06T 7/90 20170101;
G06F 16/5846 20190101; G06T 7/60 20130101; B07C 5/3422 20130101;
G06Q 50/32 20130101; G06K 9/52 20130101; G06K 9/4652 20130101 |
International
Class: |
G06K 9/00 20060101
G06K009/00; G06F 16/583 20060101 G06F016/583; G06F 16/2458 20060101
G06F016/2458; G06K 9/32 20060101 G06K009/32; G06T 7/90 20060101
G06T007/90; B07C 5/342 20060101 B07C005/342; H04L 9/32 20060101
H04L009/32; G06K 9/46 20060101 G06K009/46; G06K 9/52 20060101
G06K009/52; G06T 7/60 20060101 G06T007/60 |
Claims
1.-20. (canceled)
21. A system, comprising: at least one processor; at least one
processor-readable medium that stores processor-executable
instructions which, when executed by the at least one processor,
cause the at least one processor to: extract from a first digital
image of a reference physical object identifying information from
one or more regions of interest, the identifying information
including at least one feature of the regions of interest of the
reference physical object; generate a digital fingerprint of the
reference physical object based at least in part on the at least
one feature extracted for each region of interest of the reference
physical object; store the digital fingerprint in a dataset,
wherein the dataset comprises a plurality of digital fingerprints
for each of one or more physical objects; subsequently, extract
from a second digital image of a physical object to be
authenticated identifying information from one ore more region of
interest, the identifying information including at least one
feature of the regions of interest of the physical object to be
authenticated; generate a digital fingerprint of the physical
object to be authenticated based at least in part on the at least
one feature extracted for each region of interest of the physical
object to be authenticated; compare the digital fingerprint of the
physical object to be authenticated to one or more of the digital
fingerprints stored in the database including the digital
fingerprint of the reference physical object; and responsive to the
comparison, return an indication as to whether the physical object
to be authenticated is a subsequent appearance of the reference
physical object.
22. The system of claim 21 wherein, when executed by the at least
one processor, the processor-executable instructions cause the at
least one processor further to: parse the first digital image of
the reference physical object into the one or more regions of
interest based at least in part on a type of object in which the
reference physical object is classified.
23. The system of claim 21 wherein, when executed by the at least
one processor, the processor-executable instructions cause the at
least one processor further to: parse the second digital image of
the physical object to be authenticated into the one or more
regions of interest based at least in part on a type of object in
which the physical object to be authenticated is classified.
24. The system of claim 21 wherein, when executed by the at least
one processor, the processor-executable instructions cause the at
least one processor further to: parse the second digital image of
the physical object to be authenticated into the one or more
regions of interest using an object template selected based at
least in part on a type of object in which the physical object to
be authenticated is classified.
25. The system of claim 21 wherein the at least one feature is
identified by a respective location in the first or the second
digital images; and the at least one feature location is stored in
the database in relation to the digital fingerprints.
26. The system of claim 21 wherein, when executed by the at least
one processor, the processor-executable instructions cause the at
least one processor further to: select the at least one region of
interest based on relatively high variability of image information
in that region among different instances of physical objects like
either the reference physical object or the physical object to be
authenticated.
27. The system of claim 21 wherein, when executed by the at least
one processor, the processor-executable instructions cause the at
least one processor further to: select a region of interest based
on pre-defined regions of interest for a type of object that the
reference physical object or the physical object to be
authenticated is classified.
28. The system of claim 21 wherein the digital fingerprint
comprises a defined pattern of multiple features extracted from the
image.
29. The system of claim 28 wherein the pattern of features is
pre-defined for the regions of interest for the type of the
physical object.
30. The system of claim 28 wherein the pattern of features is
user-defined.
31. The system of claim 28 wherein the defined pattern of features
includes information on how to apply the pattern either alone or in
a defined and prioritized order with other defined patterns.
32. The system of claim 28 wherein the defined pattern of features
specifies what generic and specific information to return in
response to the comparison.
33. The system of claim 21 wherein to compare the digital
fingerprint of the physical object to be authenticated to one or
more of the digital fingerprints stored in the database in
processor-executable instructions cause the at least one processor
to: compare a number of individual elements and combine a results
of comparisons of the individual elements.
34. The system of claim 21 wherein the indication as to whether the
physical object to be authenticated is a subsequent appearance of
the reference physical object is based on a deterministic threshold
range, probabilistic, or both.
35. The system of claim 21 wherein the indication as to whether the
physical object to be authenticated is a subsequent appearance of
the reference physical object is based on at least one of (a) image
data associated with the physical object to be authenticated and
(b) a virtual representation of the physical object to be
authenticated.
36. The system of claim 21 wherein the digital fingerprint defines
a spatial relationship between one or more features, artifacts,
and/or indicia appearing or existing on the reference physical
object and on the physical object to be authenticated.
37. The system of claim 36 wherein the one or more features,
artifacts, and/or indicia may be visual, textual, and/or audible in
nature.
Description
RELATED APPLICATIONS
[0001] This application is a divisional application of U.S.
application Ser. No. 15/044,034, filed Feb. 15, 2016 (attorney
docket 110266.402D1), which is a divisional application of U.S.
application Ser. No. 14/531,307, filed Nov. 3, 2014, now U.S. Pat.
No. 9,582,714, which is a non-provisional of, and claims priority
pursuant to 35 USC .sctn. 119(e) to, U.S. provisional application
No. 61/898,780 filed Nov. 1, 2013, and to U.S. provisional
application No. 61/914,722 filed Dec. 11, 2013. U.S. application
Ser. No. 14/531,307 is also a continuation of U.S. application Ser.
No. 14/290,653 filed May 29, 2014, now U.S. Pat. No. 9,350,552,
which is a continuation of U.S. application Ser. No. 13/410,753
filed Mar. 2, 2012, now U.S. Pat. No. 8,774,455, which claims
benefit of 61/448,465 filed on Mar. 2, 2011. U.S. application Ser.
No. 14/531,307 is also a continuation of U.S. application Ser. No.
13/618,362 filed on Sep. 14, 2012, now U.S. Pat. No. 9,152,862,
which claims benefit of 61/535,084 filed Sep. 15, 2011. All of the
aforementioned applications are hereby incorporated by reference as
though fully set forth.
[0002] Copyright .COPYRGT. 2011-2018 Alitheon, Inc. A portion of
the disclosure of this patent document contains material which is
subject to copyright protection. The copyright owner has no
objection to the facsimile reproduction by anyone of the patent
document or the patent disclosure, as it appears in the Patent and
Trademark Office patent file or records, but otherwise reserves all
copyright rights whatsoever. 37 CFR .sctn. 1.71(d).
BACKGROUND
[0003] Counterfeiting of manufactured goods is a worldwide problem,
with recent studies estimating that 8% of the world's total GDP is
now generated by the manufacturing and sales of counterfeit
products. Many classes of counterfeit goods create substantial
risks to public health including counterfeit pharmaceutical drugs,
auto parts, pesticides, and children's toys. In addition,
counterfeit computer chips, aerospace parts, and identification
documents present significant risks to national security.
[0004] Authentication alone is not enough to stop counterfeiting.
Counterfeiters use a variety of strategies, including diverting
unfinished products from factories that make authentic goods and
then adding their own counterfeit brand identifiers such as labels
and tags. Counterfeit items can enter the supply chain at any
point, including at the original manufacturing facility, at the
shipper, in distribution, or in retail stores. Unless the
manufacturer or supplier can identify exactly where and when the
item entered the supply chain, identifying and eliminating the
counterfeit goods can be almost impossible.
[0005] Many different approaches have been tried to uniquely
identify and authenticate objects, including labeling and tagging
strategies using serial numbers, bar codes, holographic labels,
RFID tags, and hidden patterns using security inks or special
fibers. All of these methods can be duplicated, and many add a
substantial extra cost to the production of the goods sought to be
protected. Physical labels and tags can also be easily lost,
modified, or stolen.
SUMMARY OF THE DISCLOSURE
[0006] The following is a summary of the present disclosure in
order to provide a basic understanding of some features and
context. This summary is not intended to identify key/critical
elements of the invention or to delineate the scope of the
invention. Its sole purpose is to present some concepts of the
present disclosure in a simplified form as a prelude to the more
detailed description that is presented later.
[0007] In an embodiment, individual objects are scanned and a
unique digital signature is generated by a digital fingerprinting
method that utilizes the object's natural structure or features.
The object is registered in the system database. Once the object is
registered, the Digital Fingerprinting Track and Trace System can
track the location of any object as it passes through a supply
chain, distribution network, or sales channel. The system maintains
a database record for each unique object, and can store any
relevant data related to the object over the course of its
lifetime. The system can be queried, generate reports, and analyze
data on individual objects or on sets of objects. Applications of
the system include but are not limited to object authentication,
determining the provenance of an object, creating audit trails, and
identifying where counterfeit goods are entering manufacturing,
distribution or sales networks.
[0008] In other applications, an original digital fingerprint can
be compared to a subsequent digital fingerprint of an object to
establish that the object is the original, without regard to its
history of locations or possession since the original digital
fingerprint was acquired and stored.
[0009] Another aspect of this disclosure relates to detecting a
counterfeit or forged object, for example a document such as a
drivers license or passport. In this case, there may be no
"original" or source object digital fingerprint for comparison.
Rather, "fingerprints" of known indicia of counterfeit or forged
objects can be acquired and stored. For example, a large number of
bogus New York State driver's licenses might be obtained by law
enforcement officials in a raid or the like. Digital images of
those forged documents can be acquired, and analyzed to form
digital fingerprints, as described in more detail below.
[0010] In an embodiment, "Forgery feature vectors" can be collected
and stored in a database, for example, sharp, non-bleeding edges
where a photograph has been replaced or torn paper fibers where an
erasure occurred. These fingerprints can be searched and compared
to detect a forged document. A count of "fraud indicator matches"
can be compared to an empirical threshold to determine a confidence
that a document is forged (or not). Additional aspects and
advantages of this invention will be apparent from the following
detailed description of preferred embodiments, which proceeds with
reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] In order to describe the manner in which the above-recited
and other advantages and features of the disclosure can be
obtained, a more particular description follows by reference to the
specific embodiments thereof which are illustrated in the appended
drawings.
[0012] Understanding that these drawings depict only typical
embodiments of the invention and are not therefore to be considered
to be limiting of its scope, the invention will be described and
explained with additional specificity and detail through the use of
the accompanying drawings in which:
[0013] FIG. 1A is a simplified flow diagram illustrating a method
for creating a storing a digital fingerprint of an object in a
database.
[0014] FIG. 1B illustrates a process that includes more robust
feature extraction.
[0015] FIG. 2 is a simplified flow diagram illustrating a method
for matching a digital fingerprint of a target object to a database
of existing digital fingerprints.
[0016] FIG. 3 is a simplified conceptual diagram showing scanning
of an object at various times and places along a manufacture and
distribution chain.
[0017] FIG. 4 is a simplified conceptual diagram illustrating use
of a mobile device application to query authentication information
related to an object.
[0018] FIG. 5 is a simplified flow diagram illustrating a method
for tracking an object to verify its provenance.
[0019] FIG. 6 illustrates an example of authentication region and
object feature definition for a U.S. passport.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0020] In this application, we use the term "scan" in a broad
sense. We refer to any means for capturing an image or set of
images, which may be in digital form or transformed into digital
form. The images may be two dimensional, three dimensional, or be
in the form of a video. Thus a "scan" may refer to an image (or
digital data that defines an image) captured by a scanner, a
camera, a specially-adapted sensor array such as CCD array, a
microscope, a smart phone camera, a video camera, an x-ray machine,
etc. Broadly, any device that can sense and capture electromagnetic
radiation that has traveled through an object, or reflected off of
an object, is a candidate to create a "scan" of the object. Various
means to extract "fingerprints" or features from an object may be
used; for example, through sound, physical structure, chemical
composition, or many others. The remainder of this application will
use terms like "image" but when doing so, the broader uses of this
technology should be implied. In other words, alternative means to
extract "fingerprints" or features from an object should be
considered equivalents within the scope of this disclosure.
Authentication Regions
[0021] Because digital fingerprinting works with many different
types of objects, it is necessary to define what parts of the
digital images of the objects are to be used for the extraction of
features for authentication purposes. This can vary widely for
different classes of objects. In some cases it is the image of the
entire object; in other cases it will be a specific sub-region of
the image of the object. For instance, for a photograph we may want
to use the digital image of the entire photograph for feature
extraction. Each photograph is different, and there may be unique
feature information anywhere in the photograph. So in this case,
the authentication region will be the entire photograph.
[0022] Multiple regions may be used for fingerprints for several
reasons, two of which are particularly important. It may be that
there are several regions where significant variations take place
among different similar objects that need to be distinguished
while, in the same objects, there may be regions of little
significance. In that case a template may be used (see below)
primarily to eliminate regions of little interest.
[0023] A bank note, for example, can be authenticated if a few
small arbitrary regions scattered across the surface are
fingerprinted, along with recognizing the contents of a region
telling the value of the bank note and one containing the bank
note's serial number. In such a case the fingerprints of any region
(along with sufficient additional information to determine the bank
note's value and its purported identity) may be sufficient to
establish the authenticity of the bill and multiple fingerprinted
regions are used solely in the event that one or more regions may
be absent (through, for example, tearing) when the bill is later
presented for authentication. Sometimes, however, all regions of an
item must be authenticated to ensure the item is both authentic and
has not been altered.
[0024] A passport provides an example of feature extraction from an
authentication region; see FIG. 6. On a passport, the features that
we may want to use for authentication may be extracted from regions
containing such specific identification information as the passport
number, recipient name, and recipient photo, as illustrated in FIG.
6. In that case one may define a feature template specifying those
regions whose alteration from the original would invalidate the
passport, such regions including the passport holder's photo and
unique personal data.
[0025] The ability to define and store the optimal authentication
region for a given class of objects offers significant benefits to
the user, although it is not mandatory. In many cases it is much
easier to scan a limited region of an object than the entire
object. For instance, in the case of an article of designer
clothing, it is much easier to take a picture of the manufacturer's
label than it is to take a picture of the entire garment. Further,
defining such regions enable the detection of partial alteration of
the object.
[0026] Once an authentication region is defined, specific
applications can be created for different markets and classes of
objects that can assist the user in locating and scanning the
optimal authentication region. For instance, an appropriately sized
location box and crosshairs can automatically appear in the
viewfinder of a smartphone camera application to help the user
center the camera on the authentication region, and automatically
lock onto the region and take the picture when the camera is
focused on the correct area. It should be noted that while some
examples discussed above are essentially two-dimensional objects
(passport, bank note); the present disclosure is fully applicable
to three-dimensional objects as well. Scanning or image capture may
be 2-D, 3-D, stereoscopic, HD etc. Image capture is not limited to
the use of visible light.
[0027] In many cases, objects may have permanent labels or other
identifying information attached to them. These can also be used as
features for digital fingerprinting. For instance, wine may be put
into a glass bottle and a label affixed to the bottle. Since it is
possible for a label to be removed and reused, simply using the
label itself as the authentication region is often not sufficient.
In this case we may define the authentication region to include
both the label and the substrate it is attached to--in this case
some portion of the glass bottle. This "label and substrate"
approach may be useful in defining authentication regions for many
types of objects, such as consumer goods and pharmaceutical
packaging. If a label has been moved from its original position,
this can be an indication of tampering or counterfeiting. If the
object has "tamper-proof" packaging, this may also be useful to
include in the authentication region.
[0028] In some cases, we will want to use multiple authentication
regions to extract unique features. For a firearm, for example, we
might extract features from two different parts of the weapon. It
is, of course, important that both match the original but since the
two parts may both have been taken from the original weapon and
affixed to a weapon of substandard quality, it may also be
important to determine whether their relative positions have
changed as well. In other words it may be necessary to determine
that the distance (or other characteristic) between Part A's
authentication region and Part B's authentication region is
effectively unchanged, and only if that is accomplished can the
weapon be authenticated. Specifications of this type can be stored
with or as part of a digital fingerprint of the firearm.
[0029] Once a suitable digital fingerprint of an object is
acquired, the object (actually some description of it) and
corresponding fingerprint may be stored or "registered" in a
database. For example, in some embodiments, the fingerprint may
comprise one or more feature vectors. The database should be
secure. In some embodiments, a unique ID also may be assigned to an
object. An ID may be a convenient index in some applications.
However, it is not essential, as a digital fingerprint itself can
serve as a key for searching a database. In other words, by
identifying an object by the unique features and characteristics of
the object itself, arbitrary identifiers, labels, tags, etc. are
unnecessary and, as noted, inherently unreliable.
[0030] FIG. 1 is a simplified flow diagram illustrating a method
100 for creating and storing or "registering" a digital fingerprint
of an object in a database. The process in one embodiment includes
acquiring a digital image of the object, block 102, as discussed
above. A variety of image capture technologies and devices may be
used as noted. Next, features are extracted, block 104, from the
digital image data. As explained, specific features or regions of
interest (authentication regions) may be selected in support of
subsequent identification or authentication of the object. The
extracted features are analyzed and feature vectors are extracted
to form a digital fingerprint--a digital file or record associated
with the original image data, indicated at block 106. The digital
fingerprint preferably may be stored in a database record. Other
forms of searchable digital data storage should be deemed
equivalents. Further, at block 110, initialization data should be
added to the database record, or associated with it in a related
table. This data is associated with the physical object that was
scanned. For example, a description, manufacturer, model number,
serial number, contents--a wide variety of data, selected as
appropriate or useful depending on the type of object.
[0031] FIG. 1B illustrates a process that includes more robust
feature extraction. In this example, we again begin with acquiring
digital image data, block 120. We select at least one
authentication region, block 122. This may be done by analysis of
the image data, analysis of related image data, by reference to a
predetermined template that defines at least one authentication
region, or other means. The next block 124 calls for extracting a
feature vector from the selected authentication region. A feature
vector may be used to represent features of a region in a more
compact form. For example, a feature vector may comprise an array
of color or gray scale numeric values corresponding to areas within
the selected authentication region. The values may each comprise a
sum, average, maximum or other function of the individual values of
a corresponding group of pixels forming a sub-part of the region.
In some applications, a feature vector may identify a location and
shape of a distinctive aspect within a selected region. In decision
126, there may be additional feature vectors to be extracted from
the same image data. In that case, the flow returns, path 130, to
repeat the feature extraction step 124. This loop 130 may repeat
until all desired feature vectors are collected. Optionally, there
may be another authentication region to process in the same image
data, see decision 132. In that case, the outer loop 133 is
traversed back to block 122 for further feature extraction with
respect to one or more additional authentication regions. Then some
or all of the extracted feature vectors may be combined to form a
digital fingerprint, block 134, which is then stored, block 136,
along with related data, block 138, as mentioned above. The process
returns or concludes at block 140.
[0032] A database of digital fingerprints can form the basis of a
system to track and trace the object through a supply chain,
distribution network, or sales channel. A track and trace system
based on digital fingerprinting has unique advantages and provides
unique capabilities that are not available with track and trace
systems based on traditional methods.
[0033] Holograms, bar codes and serial numbers can all be
duplicated with varying degrees of effort. This means that if the
code or tag can be duplicated, then counterfeit objects or two
objects with the same identifier can exist in the supply chain or
distribution network. They can then be registered in a traditional
track and trace system. All such systems rely on determining that
the anti-counterfeit item (label, hologram, RFID tag) is
legitimate, not that the item itself is.
[0034] Due to this weakness, track and trace systems based on
traditional approaches like bar codes or serial numbers cannot
prevent the resulting corruption of the system database. A
counterfeit object may be mistakenly identified as genuine, and
generate a false audit trail as it is tracked through the supply
chain. Two or more objects with the same ID (one genuine, one or
more counterfeit) may exist at the same time. Without physically
examining the objects it is impossible to tell which item is
genuine. Once identification is made as to which object is genuine,
the false trails must be removed from the database to restore
integrity. This can be extremely difficult depending on the
structure of the database and the complexity of the tracking data.
In some cases the objects may not have any further contact with the
track and trace system (for instance if they are purchased by a
consumer), and the record will never be identified as false,
leaving the database permanently corrupted.
[0035] In one embodiment of the Digital Fingerprinting Track and
Trace System, an item may be scanned and identified at initial
manufacture. Alternatively, an item may be scanned and identified
at any subsequent time or location for entry into a tracking
system. This point of identification preferably is done when the
item is either in the possession of its manufacturer or has been
transferred by secure means to the current holder so that its
legitimacy at the point of identification is adequately
established.
[0036] The system then identifies the object every time it is
scanned again, typically at discrete steps in manufacturing,
distribution, and sale. FIG. 2 is a simplified flow diagram
illustrating a method 200 for matching a digital fingerprint of a
target object to a database of existing digital fingerprints. Here,
we acquire image data of a "target object" i.e., the object we want
to identify or authenticate by finding a match in the database, see
block 202. We extract features from the target object image data,
block 204, as discussed above. Then we create a new (second)
digital fingerprint based on the extracted features, block 206. The
next step is querying the database, block 208, for a record that
matches the second digital fingerprint record. "Matching" in this
context may be relative to a threshold confidence level rather than
a binary decision. The requisite confidence level may vary
depending on the specific application. The confidence level
required may be varied dynamically responsive to the data and
experience with a given system. If no "matching" record is
returned, decision 210, update the second record (the digital
fingerprint of the target object), block 212, to reflect that no
match was found. If a match is returned, the matching record is
updated to reflect the match, for example, it may be linked to the
second record. The results may be returned to the user.
[0037] Typical tracking steps might include scanning at the point
of manufacture, when packaged, when placed in inventory, when
shipped, and at a retail point of purchase (upon arrival and again
when sold), as illustrated in the tracking process 300 of FIG. 3.
Each scan can be used to update a remote database.
[0038] As mentioned earlier, a "scan" may refer to an image (or
digital data that defines an image) captured by a scanner, a
camera, a specially-adapted sensor array such as CCD array, a
microscope, a smart phone camera, a video camera, an x-ray machine,
etc. Broadly, any device that can sense and capture electromagnetic
radiation that has traveled through an object, or reflected off of
an object, is a candidate to create a "scan" of the object. It is
critical to capture at least one native feature of the object, as
distinguished from a feature added to the object for
identification, such as a label, bar code, RFID tag, serial number,
etc.
[0039] A "native feature" in this description is not concerned with
reading or recognizing meaningful content. For example, a label on
a scanned object with a printed serial number may give rise to
various features in fingerprint processing, some of which may
become part of a digital fingerprint feature set or vector that is
associated with the object. The features may refer to light and
dark areas, locations, spacing, ink blobs, etc. This information
may refer to the printed serial number on the label, but there is
no effort to actually "read" or recognize the printed serial number
(which may be bogus). Similarly, an RFID tag applied to an object
may give rise to a fingerprint vector responsive to its appearance
and location on the object. However, no effort is made to actually
stimulate or "read" data or signals from the tag.
[0040] While the most common application of track and trace systems
is in manufactured goods, the present system and methods, in
various different embodiments, may be applied to any object that
can be identified with a digital fingerprint and tracked. These
include but are not limited to mail pieces, parcels, art, coins,
currency, precious metals, gems, jewelry, apparel, mechanical
parts, consumer goods, integrated circuits, firearms,
pharmaceuticals and food and beverages. Tracking may consist of any
sequence of actions where the object is scanned, such as each time
an object is appraised, authenticated, certified, auctioned,
displayed, or loaned. The system may store both positive and
negative authentication transactions. In an embodiment, the system
may store location information (associated with a scan or
fingerprint), which provides a profile of where counterfeit goods
may be encountered.
[0041] FIG. 4 is a simplified conceptual diagram illustrating use
of a mobile device application to query authentication information
related to an object. Here, various computing devices or terminals
402 may have access over a network, for example, the Internet 404,
to cloud computing facilities/services such as a cloud
server/datastore 406. For example, the devices 402 may be located
at various points along a distribution chain as illustrated in FIG.
3, each location scanning an object and updating the cloud
server/datastore 406.
[0042] A server 412 may be provisioned to provide tracking and/or
tracing data analysis and reporting. The server 412 has access to a
datastore 420 which may be used to store digital fingerprints and
related data. The server can query or search the database 420 for
digital fingerprint search and matching. The database 420
preferably is coupled to the cloud server 406 in some embodiments.
A mobile user device 410 such as a smartphone, tablet, laptop
computer or dedicated device may be configured for communications
with the server 412 to request and receive a reply or
authentication report for an object of interest. This architecture
is simplified and in any event is merely illustrative and not
intended to be limiting.
[0043] Continuous and Discrete Tracking
[0044] In some implementations, sensors may be attached to the
object, and sensor data can flow back to the database in either a
continuous fashion (near real time), or in discrete data transfer
events. For example, data transfer may occur when an authentication
event occurs. For instance, if there is a GPS chip attached to the
object, data flow can start when the object is first registered in
the system, and continue to flow as the object changes location.
Continuous (frequent) data updates can also be buffered in local
memory in a sensor attached to the item, and then downloaded the
next time the object is scanned and authenticated. This provides a
record of where the object has traveled (its itinerary).
[0045] As an example of the potential uses of sensor data, many
products like food and beverages can degrade with exposure to
certain environmental factors over the course of their storage and
shipment. Examples of sensor data could include temperature, light
exposure, altitude, oxygen level, or other factors, as well as
location such as GPS data.
[0046] FIG. 5 is a simplified flow diagram illustrating one
embodiment of a process 500 for tracking an object to verify its
provenance. Here, an expected itinerary of an object (a series of
locations) may be stored in a datastore if known, block 502. The
methods and systems described above may be used to track the object
to the next location, block 504. If the object does not arrive as
expected (where and when expected according to the itinerary), the
failure may be reported to a user. In an embodiment, an object that
arrives later than expected may be subjected to closer matching
scrutiny to ensure its identity.
[0047] The next step, block 510, is to query the database for the
next valid or expected location. A unique itinerary may not be
known, but a set of valid or expected locations may be known. The
next actual location of the object (as determined by imaging and
matching digital fingerprints) may be compared to the expected
location(s) returned by the database, block 512. If that comparison
indicates a departure from the expected or authorized route,
decision 520, the result may be reported to a user, block 522. (A
report that the object is on track may be reported as well.) Other
options may be implemented such as a quantity check, block 524. The
process returns or terminates at block 526.
[0048] Most existing track and trace systems are only designed to
be accessed by manufacturers or their authorized distributors, and
often require specialized scanners or equipment. However, the
consumer also has a vested interest in determining whether the
items that they are buying are authentic. In some embodiments, the
present system is designed to enable anyone along the supply,
distribution, or sales chain, from manufacturer to the retail
consumer, to access the system and determine whether the item is
authentic. A specialized scanner is not required in all cases. For
example, in one embodiment a mobile phone application designed for
the consumer can be used to scan an object, query the database, and
determine if the object is authentic.
[0049] Finally, data collected by a digital fingerprinting system
offers a variety of useful information to people along the supply,
distribution and sales chain. Reports can be generated on
individual items, or on sets of items. These reports can include
but are not limited to the locations of items over time, audit
trails, points of entry of counterfeit goods, and exposure to
environmental variables over the course of an object's useful
lifetime.
[0050] Tags and Bar Codes
[0051] A tag may be added to an item, a barcode to a mail piece,
etc. for two reasons. First, the human may need it to know what the
item is. This is the identification function. It may identify the
item to a store clerk as a particular style and size of clothing of
a particular manufacturer; it may tell a postal carrier where to
deliver a mail piece. Second, however, are tags that are only
useful for a machine. Thus a four-state bar code on a mail piece
(unreadable by humans) is used to route the mail piece by machine.
This entire class of machine readable tags can be replaced by the
methods of this patent. The first set may still be needed for human
use but are now divorced from their authentication function.
[0052] Because we are exploiting natural features and often
scanning the object under variable conditions, it is highly
unlikely that two different "reads" will produce the exact same
fingerprint. We therefore have to introduce the ability to look up
items in the database when there is a near-miss. For example, two
feature vectors [0, 1, 5, 5, 6, 8] and [0, 1, 6, 5, 6, 8] are not
identical but (given the proper difference metric) may be close
enough to say with certainty that they are from the same item that
has been seen before. This is particularly true if, otherwise, the
nearest feature vector of a different item is [5, 2, 5, 8, 6, 4].
For example, a distance between vectors of n-dimensions is easily
calculated, and may be used as one metric of similarity or
"closeness of match" between the vectors. One may also consider the
distance to the next nearest candidate.
[0053] Obviating the Chain of Custody
[0054] Many systems rely on a known "chain of custody" to verify
authenticity of an object. The rules of evidence in court, for
example, typically require proof of a chain of custody to
demonstrate authenticity of a hair sample, weapon or other piece of
physical evidence. From the time an object is collected at a crime
scene, for example, it is typically bagged, tagged, and moved into
a locked box or evidence room for safekeeping. Each person who
removes it must attest to returning the original item unchanged.
Custody of the object from the crime scene to the evidence locker
to the courtroom must be accounted for as an unbroken chain.
Digital fingerprinting techniques as disclosed herein can be used
to obviate most of that process. Provided an original object is
under the control and custody of an authorized or trusted entity at
least once, and a digital fingerprint, or an image suitable for
forming a digital fingerprint of the object, is acquired under such
circumstances, and stored, the object is uniquely identifiable
thereafter by that fingerprint for the lifetime of the object.
[0055] Because digital fingerprinting works by extracting key
features of an object, it may be used to identify or authenticate
objects even after a good deal of wear and tear. At any subsequent
time, a suspect or "target" object can be similarly "fingerprinted"
and the subsequent fingerprint compared to the stored fingerprint
of the original object. If they match, authenticity is established,
regardless of where or in whose custody the object may have
traveled in the meantime. Returning to the crime scene example, if
a digital fingerprint is acquired of a weapon taken from the crime
scene, and the digital fingerprint stored, the weapon can be
fingerprinted again at any subsequent time, and the digital
fingerprints compared to authenticate to weapon. Custody of the
weapon in the interim is no longer an issue. Likewise when a coin
or piece of art is stolen our technology gives us the ability to
continue the original provenance even though chain of custody has
been lost (i.e. we know it is the same coin we saw before even
though it has not been in our direct possession during the time
between theft and recovery).
[0056] Global vs. Regional Feature Matching
[0057] In a case where we have the original document or other
object fingerprinted, our techniques allow region-by-region
matching so that we can tell what (if any) regions have been
changed. Thus, for example, we might get a really good overall
match on a passport but none of the matches happen in the
photograph--so we know the photograph probably was changed.
Further, if some individual or group, say Al Qaeda, has a certain
pattern or regularity to altering passports--change the photo, the
date of birth and one digit of the passport number, say--then this
ability to find altered regions also gives us the ability to
discern the pattern of changes and thus develop a signature of the
group making the changes. Thus aspects of the present technology
can be applied not only to detect a forged or altered document, but
to identify in some cases the source of the bogus document.
[0058] It will be obvious to those having skill in the art that
many changes may be made to the details of the above-described
embodiments without departing from the underlying principles of the
invention. The scope of the present invention should, therefore, be
determined only by the following claims.
* * * * *