U.S. patent application number 13/355240 was filed with the patent office on 2012-05-17 for methods and systems to help detect identity fraud.
Invention is credited to William Y. Conwell, Bruce L. Davis.
Application Number | 20120123959 13/355240 |
Document ID | / |
Family ID | 38234123 |
Filed Date | 2012-05-17 |
United States Patent
Application |
20120123959 |
Kind Code |
A1 |
Davis; Bruce L. ; et
al. |
May 17, 2012 |
Methods and Systems to Help Detect Identity Fraud
Abstract
The disclosed technology generally relates to methods and
systems to aid in verifying a person's identity, e.g., in
connection with applying for an identity document (such as a
passport or driver's license), or in connection with qualifying to
enter a secured area (such as at an airport). Many arrangements
involve testing the person concerning specific knowledge with which
he or she should be familiar, e.g., by reason of living in a
particular residence and neighborhood, by reason of their
particular employment, or by reason of their particular education.
An appendix particularly addresses crowdsourcing technology,
including its applicability in redressing some of the shortcomings
of fingerprint-based content identification.
Inventors: |
Davis; Bruce L.; (Lake
Oswego, OR) ; Conwell; William Y.; (Portland,
OR) |
Family ID: |
38234123 |
Appl. No.: |
13/355240 |
Filed: |
January 20, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
12114612 |
May 2, 2008 |
|
|
|
13355240 |
|
|
|
|
11613891 |
Dec 20, 2006 |
|
|
|
12114612 |
|
|
|
|
60753652 |
Dec 23, 2005 |
|
|
|
Current U.S.
Class: |
705/325 |
Current CPC
Class: |
G06Q 50/18 20130101;
H04L 63/08 20130101; G06K 9/62 20130101; G06F 16/683 20190101; G06F
16/58 20190101; G06F 16/783 20190101; H04L 9/3247 20130101; G06Q
50/01 20130101; G06Q 50/265 20130101; G06N 5/048 20130101; G06Q
10/00 20130101; G06F 16/40 20190101; G06F 16/43 20190101 |
Class at
Publication: |
705/325 |
International
Class: |
G06Q 99/00 20060101
G06Q099/00 |
Claims
1. A method comprising: collecting information about an applicant;
storing said information in a first data structure; by reference to
the information stored in the data structure, identifying a
knowledge domain with which the applicant is likely familiar;
selecting a question relating to said knowledge domain, the answer
to said question not earlier having been provided by the applicant;
posing the selected question to the applicant; receiving an
applicant response to the posed question; and generating a score
based at least in part on an assessment of the received applicant
response.
2. The method of claim 1 that includes using certain of said
collected information to obtain additional information relating to
the applicant from an external database, and storing said
additional information in the data structure.
3. The method of claim 1 in which the collected information
indicates that the applicant is a lawyer, and in which the question
posed to the applicant is one related to a legal subject.
4. The method of claim 3, wherein the question is one to which a
lawyer-applicant is more likely to know the answer than a
non-lawyer-applicant, due to the lawyer-applicant's legal
training.
5. The method of claim 4, wherein the question is one relating to
the meaning of a Latin phrase.
6. The method of claim 1 wherein the knowledge domain comprises
facts with which lawyers are more likely to be familiar than
non-lawyers.
7. The method of claim 1 wherein the knowledge domain comprises
facts with which the applicant is likely to be more familiar than
members of the general public, by reason of applicant's
employment.
8. The method of claim 1 wherein the knowledge domain comprises
facts with which the applicant is likely to be more familiar than
other members of the general public, by reason of applicant's
education.
9. The method of claim 1 wherein the knowledge domain comprises
facts with which the applicant is likely to be more familiar than
other members of the general public, by reason of applicant's
current or prior residence address.
10. The method of claim 9 wherein said posing includes presenting
to the applicant a photograph or map depicting a neighborhood
scene.
11. The method of claim 10 wherein the method includes posing to
the applicant plural questions testing the applicant's familiarity
with the neighborhood depicted in said photograph or map.
12. The method of claim 1 that further includes storing data
relating to a corpus of plural questions in a second data
structure, each question data having at least two classification
data associated therewith, said classification data being useful in
identifying applicants who are likely to be able to answer
questions based on said question data.
13. The method of claim 12 wherein one classification data relates
to a geographic location.
14. The method of claim 12 wherein one classification data relates
to an expertise in law.
15. The method of claim 12 wherein one classification data relates
to age.
16. The method of claim 12 wherein one classification data relates
to education.
17. The method of claim 12 that further includes: modeling a
multi-dimensional vector space in which a distance between a
particular applicant and a particular question data can be gauged,
the gauging of said distance including assessing information about
said particular applicant stored in the first data structure, in
conjunction with classification data associated with said
particular question data stored in the second data structure;
wherein relative proximity between a particular applicant and
particular question data suggests that said applicant is relatively
likely to be able to correctly answer questions based on said
question data; gauging the relative distance between the particular
applicant and plural question data in the corpus; determining
thereby a subset of question data stored in the corpus, said subset
comprising question data particularly suited for said applicant;
and posing questions to said particular applicant, based on
question data in said determined subset.
Description
RELATED APPLICATION DATA
[0001] This application is a division of copending application Ser.
No. 12/114,612, filed May 2, 2008 (published as US20080208849),
which is a division of abandoned application Ser. No. 11/613,891,
filed Dec. 20, 2006 (published as US20070162761), which claims
priority benefit to provisional application 60/753,652, filed Dec.
23, 2005.
[0002] Some of the subject matter herein is related to that in
other patent applications, including Ser. No. 10/723,240, filed
Nov. 26, 2003 (published as US20040213437); Ser. No. 10/979,770,
filed Nov. 1, 2004 (now U.S. Pat. No. 7,314,162); and Ser. No.
11/132,724, filed May 18, 2005 (published as US20050288952).
TECHNICAL FIELD
[0003] The technology detailed herein generally relates to methods
and systems to aid in verifying a person's identity, e.g., in
connection with applying for an identity document (such as a
passport or driver's license), or in connection with qualifying to
enter a secured area (such as at an airport).
BACKGROUND
[0004] Traditionally, applicants for identity documents have been
required to present only a few items of collateral identification,
such as a birth certificate, a social security card, and/or a study
body ID card. (Such collateral documents are sometimes termed
"breeder documents;" a fuller list of commonly-accepted breeder
documents is detailed in application Ser. No. 10/979,770, published
as US20070205266.) With the proliferation of low-cost and
high-quality scanning and printing technologies, as well as simple
image editing software, such breeder documents have become easier
to counterfeit. Thus, there is a need for techniques by which the
identity of an applicant can more reliably be determined.
[0005] Application Ser. No. 10/979,770 notes that the risk of
identity fraud in the issuance of ID documents varies, with some
types of breeder documents being more reliable in establishing a
person's identity (e.g., US passports) than other types of breeder
documents (e.g., student body ID cards). Data on the incidence of
discovered fraud can be collected, and correlated back to the types
of breeder documents submitted in each case, e.g., using factor
analysis. This historical data permits a risk score to be generated
for each new applicant, based on the particular types of breeder
documents he or she presents. Applicants with relatively high
breeder document risk scores can be scrutinized relatively more
closely than applicants with relatively low risk scores. Such
techniques allow security personnel to focus their efforts where
they will do the most good.
[0006] Application Ser. No. 11/132,724 (published as US20050288952)
notes that some parts of the applicant enrollment process can be
performed from the applicant's home. A state Department of Motor
Vehicles (DMV), for example, may have a web site through which an
applicant for a driver's license can enter their name, address,
birth date, hair color, organ donor preferences, and other
background information. Scans of breeder documents that the
applicant intends to present (e.g., birth certificate and passport)
can also be submitted from home. In some systems the applicant may
even be allowed to submit a proposed portrait photograph for
printing on their license. This data-entry web session can conclude
by allowing the applicant to schedule an appointment to visit a
nearby DMV office to complete the enrollment and license issuance
process.
[0007] By receiving this applicant information in advance, the DMV
can undertake more thorough vetting of an applicant's identity than
if they simply appear at the DMV office. Such vetting generally
involves researching the applicant and his/her purported identity,
and checking any breeder document data, to make sure that nothing
appears amiss. For example, the DMV may check third party
databases, such as credit bureaus, telephone directories, social
security databases, etc., to verify that the information submitted
by the applicant, and the information represented by the breeder
documents, is consistent with data maintained by these third
parties. Any portrait photograph submitted by the applicant can
likewise be checked against an archive of previous driver license
images to determine whether a person of similar appearance has
already been issued a driver license. If these checks give any
ground for suspicion, the DMV can contact the applicant to solicit
further information. If issues are not satisfactorily addressed
prior to the appointment, the appointment may be canceled.
[0008] Published application US20040213437 details technologies by
which the photograph of a new applicant for a driver license can be
checked against a database of photographs on previously-issued
driver licenses. If a suspected match is found, the circumstances
can be investigated to determine whether the applicant may be
engaged in fraud.
[0009] Application Ser. No. 10/979,770 details how a risk score may
be generated, to give an indication of the relative possibility of
fraud associated with a given applicant (e.g., by considering past
fraud experiences correlated with different types of breeder
documents). Other risk scoring techniques are known the art.
Examples are shown in published applications and patents such as
US20030052768 (a trusted-traveler card system, in which trust is
scored based on various factors, including how long the traveler
has lived at a given address); US20030099379 and US20030115459
(various ID card attributes are sensed, and facial features and
third party databases are checked, to yield a score by which
confidence in a person's asserted identity is assessed);
US20030216988 (the validity of an applicant's telephone number is
checked, and combined with other indicators, to produce a risk
score for a proposed transaction); US20040059953 (a transportation
worker identity card system in which various personal data is
collected from the applicant and checked against third party
databases, to determine if confidence in the asserted identity
exceeds a threshold); US20040064415 (a traveler's identity is
checked by reference to various public and private databases--which
may include birth certificate data, social security number data,
and INS records--and a resulting score is produced); US20040153663
(in processing a change-of-address request, a credit card company
compares demographics of the applicant's previous and new
neighborhoods, e.g., average income, average net worth, percentage
of renters, etc., looking for unexpected disparities; a polynomial
equation is applied to compute an associated risk score);
US20040230527 (before completing a wired money transfer,
circumstances of the transfer are compared against historical data
and referred to third party evaluation services, to generate a
score indicative of the risk of charge-back); US20040245330 (before
completing a financial transaction, various parameters are
considered and contribute--positively or negatively--to a net
score, which is used to determine whether the transaction should
proceed); US20050039057 (during enrollment, a person is questioned
re opinions and trivial facts, e.g., "I carry my car keys in my (a)
pocket; (b) purse; (c) briefcase; (d) backpack", "The phone number
of a childhood friend is XXX-YYY-ZZZZ," and the given answers are
stored in a database; when the person's identity is later to be
checked, a random subset of these questions is posed, some with
different weightings, until a given confidence of identity is met);
US20050132235 and US20050171851 (a user's speech is biometrically
analyzed to determine degree of match against earlier-captured
voice data, to yield an identification factor; this can be combined
with other checks to generate a score indicating confidence in the
speaker's identity); and U.S. Pat. No. 5,679,938 and U.S. Pat. No.
5,679,940 (different attributes of a financial check and an
associated transaction are weighted to compute a score indicating
degree of confidence that the check will be honored).
[0010] U.S. Pat. No. 6,513,018 details empirically-based scoring
technologies employed by Fair, Isaac and Company in computing
credit scores. Particularly detailed are arrangements by which
different applicant characteristics are either positively- or
negatively-correlated with certain performance results, and methods
for determining and applying such correlative information. U.S.
Pat. No. 6,597,775 details Fair, Isaac's extrapolation and
improvement of such methodologies to predictive modeling, and
mitigation, of telecommunications fraud.
[0011] Many ID verification systems rely on challenge-response
testing concerning information that could become known to an
imposter, e.g., by stealing a person's wallet or purse, or by
removing mail from a mailbox. This gives rise to systems based on
"out-of-wallet" information--the most common of which is "mother's
maiden name." More elaborate "out-of-wallet" systems for confirming
identity are detailed, e.g., in some of the patents and
publications referenced above, as well as in the patent
publications such as: US20040189441 (which checks identity by
testing applicant's knowledge of inherent attributes--such as
mother's maiden name, as well as voluntary attributes--such as
favorite color; the techniques provide some error tolerance in
assessing answers, e.g., answer of "Smith" vs. expected answer of
"Mr. Smith); and 20040205030 (provides a lengthy catalog of
"out-of-wallet" information that may be used in confirming
applicant identity, e.g., name of a signatory on a particular
document, alimony payments, surgical records, medication currently
prescribed, judicial records, etc.).
[0012] Some identity-verification systems employ multi-factor
approaches. An example is US20050154924, which validates a user
based on a collection of user-provided factors, such as a
combination of `what the user knows` (e.g., knowledge-based data),
`who the user is` (e.g., biometric-based data), `what the user
possesses` (e.g., token-based data), `where the user is` (i.e.,
location-based data), and when the user is seeking validation
(i.e., time-based data).
[0013] Biometrics can be useful in checking identity. But
biometrics can only check for a match between an earlier-collected
and presently-collected set of data. Unless there is confidence
about the identity of the person from whom the earlier biometric
data was collected, such technologies are of limited utility.
BRIEF DESCRIPTIONS OF THE DRAWINGS
[0014] FIGS. 1-11 illustrate information that can be used in
testing an applicant to help determine relative confidence in the
applicant's asserted identity.
DETAILED DESCRIPTION
[0015] For expository convenience, most of the following detailed
description focuses on one particular application of applicants'
technology: verifying the identity of an applicant for a driver
license. It will be recognized that such technologies can likewise
be employed to help verify the identity of persons in myriad other
contexts.
[0016] The reader is presumed to be familiar with driver license
issuance systems and procedures. (The commonly-owned patent
applications identified above provide useful information in this
regard.)
[0017] In one arrangement, profile data about an applicant is
collected in an XML-based data structure, based on a collection of
standardized tags. Part of an illustrative collection of data for
an applicant may be as follows
TABLE-US-00001 <BIRTHDATE> 12/19/1945
<CURRENT_RES_ADDRESS> 6299 SW Tower Way, Portland, OR 97221
<PRIOR_RES_ADDRESS_1> 3609 SW Admiral St., Portland, OR 97221
<PRIOR_RES_ADDRESS_2> 5544 SW 152.sup.nd Ave., Portland, OR
97226 <HIGH_SCHOOL_1> Summit High School, Summit, NJ
<HIGH_SCHOOL_2> Wilson High School, Portland, OR
<COLLEGE_1> Washington State University, Pullman, WA
<COLLEGE_2> University of Waterloo, Ontario, Canada
<COLLEGE_STUDY_1> Geology <OCCUPATION_1> Professor
<OCCUPATION_2> School of Earth Sciences <OCCUPATION_3>
University of Portland <CITIZEN> USA
[0018] Such a collection of data can be seeded by information
provided by the applicant, e.g., name, address, phone number, and
social security number. The collection can then be supplemented by
further information obtained from public sources (e.g., the web and
public databases), as well as private data collections.
[0019] For example, credit agency databases (e.g., Experian,
Equifax and Transunion) typically store prior addresses for myriad
people, in addition to their current addresses. Based on address
data, a wealth of additional information can be obtained. Various
public web sites, for example, can provide corresponding provide
latitude/longitude information. Online municipal tax databases can
be queried to obtain information about the home (three bedrooms,
two full baths, cedar shake roof, etc.). Third party commercial
databases can provide statistical demographics about the
neighborhood (e.g., average income, age distribution, percentage of
renters, etc.). All of this information can be added to the profile
(or accessed online, as needed).
[0020] Knowing alleged facts about the applicant allows the DMV to
pose questions that help establish confidence in the applicant's
asserted identity. The questions are generally of a sort that can
be correctly answered by the applicant--if the facts collected
concerning the applicant are correct, but would be more difficult
to answer otherwise.
[0021] One knowledge domain with which every applicant should be
familiar is facts about their residence. Given a residence address,
online tools can be used to mine substantial information about the
building, its construction, its features, its age, etc., as well as
about the surrounding neighborhood.
[0022] Consider an applicant who asserts that his or her residence
address is 6200 SW Tower Way, Portland, Oreg. By providing this
address to the web page at portlandmaps.com or zillow.com, a wealth
of information--parts of which are shown in FIGS. 1A and 1B, can be
obtained. Questions can be posed to the applicant based on the
facts provided from this web resource. For example [0023] How many
bathrooms does 6200 SW Tower Way have? [0024] 1. One [0025] 2. One
and a half [0026] 3. Two [0027] 4. Two and a half [0028] 5. Three
[0029] 6. Three and a half [0030] When was 6200 SW Tower Way built?
[0031] 1. Prior to 1940 [0032] 2. Between 1940 and 1990 [0033] 3.
Between 1991 and 2000 [0034] 4. After 2001 [0035] What type of
heating system does 6200 SW Tower Way have? [0036] 1. Forced air
[0037] 2. Radiant floor heat [0038] 3. Baseboard hot water If the
applicant identifies himself as the owner of the property (rather
than a renter), then the applicant might be further asked: [0039]
In what year did you purchase the property? [0040] 1. 1996 [0041]
2. 2000 [0042] 3. 2002 [0043] 4. 2004 [0044] 5. 2005 [0045] What
was the purchase price for the property? [0046] 1. Less than
$180,000 [0047] 2. Between $180,000 and $220,000 [0048] 3. Between
$220,001 and $260,000 [0049] 4. Between $260,001 and $300,000
[0050] 5. More than $300,000 Each of these answers can be readily
checked from the information available on-line, and depicted in
FIGS. 1A and 1B.
[0051] If the applicant answers each of these questions correctly,
it lends credence to their assertion that their residence is 6200
SW Tower Way. If the applicant answers most of the questions
incorrectly, it raises a serious doubt as to at least their
residence address. (Other information provided by such applicant
may also be called into doubt.)
[0052] The web increasingly offers a rich trove of geospatial data,
such as maps, aerial imagery, and curbside imagery, which is
readily accessible by street addresses or other location data. The
maps are of different types, e.g., street maps, zoning maps,
topographic maps, utility (e.g., water, sewer, electrical) maps,
hazard maps (e.g., flood plain, earthquake), etc. (The
portlandmaps.com site referenced above includes all these
categories of information.) Such resources offer a rich source of
information about which an applicant can be questioned--based on
current and/or prior residence addresses. Consider, e.g., the
following questions which may be posed to an applicant who asserts
his address is 6200 SW Tower Way, Portland, Oreg.: [0053] Refer to
the map shown in FIG. 2 (which includes your residence address) for
the following questions. [0054] What is the name of the park ("E")?
[0055] 1. Albert Kelly Park [0056] 2. April Hill Park [0057] 3.
Custer Park [0058] 4. Dickinson Park [0059] 5. Gabriel Park [0060]
6. Hillsdale Park [0061] 7. Pendleton Park [0062] 8. None of the
above [0063] What is the name of the store located at the
intersection of SW Vermont St and SW 45.sup.th avenue? [0064] 1.
7-11 [0065] 2. Express Mart [0066] 3. Jackpot Food Mart [0067] 4.
Plaid Pantry [0068] 5. Swan Mart [0069] 6. Uptown Market [0070] Is
that store located at corner A, B or C? [0071] 1. A [0072] 2. B
[0073] 3. C [0074] Is there a traffic light, or a four-way stop, at
the intersection of SW Vermont St and SW 45.sup.th avenue? [0075]
1. Traffic light [0076] 2. Four-way stop [0077] Is location D
uphill from location E? [0078] 1. Yes [0079] 2. No [0080] Wilson
High School is off the map. In what direction, from the map, is it?
[0081] 1. Off the top side [0082] 2. Off the right side [0083] 3.
Off the bottom side [0084] 4. Off the left side Again, an applicant
who truly lives at 6200 SW Tower Way would have little difficulty
with such questions. However, an imposter would fare poorly.
[0085] Typically, the map about which the applicant is questioned
would be presented on an electronic display device, e.g., on a
testing kiosk at a DMV office. Alternatively, the question could be
posed to the applicant at a remote terminal (e.g., at the
applicant's home)--provided certain safeguards are put in place to
prevent the applicant from researching the answer. (E.g., the
applicant would have a limited period of time after the
presentation of each question to provide the answer.)
[0086] Similarly, consider the following questions, which may be
posed to an applicant, and which proceed with reference to two
aerial photographs: [0087] Refer to the aerial photograph shown in
FIG. 3 (which includes your residence address) for the following
questions. [0088] Which of the buildings is your residence (A-V)?
[0089] What is the last name of one of your neighbors? [0090] In
what building does that neighbor live (A-V)? [0091] What is the
name of the street labeled W? [0092] 1. SW Tower Way [0093] 2. SW
Dakota St [0094] 3. SW Idaho St [0095] 4. SW Caldew Dr [0096] If
you drove the street labeled X from the top of the map towards the
bottom, would you be going uphill or downhill? [0097] 1. Uphill
[0098] 2. Downhill [0099] FIG. 4 is an aerial photograph showing
one of the intersections nearest your residence (about 0.4 miles
away). In the map: [0100] What business occupies building Y? [0101]
1. Multnomah Community Center [0102] 2. OHSU Clinic [0103] 3. Big 5
Sports [0104] 4. Tursi's Soccer Store [0105] What business occupies
building Z? [0106] 1. Gerber Labs [0107] 2. Marquam Gymnasium
[0108] 3. Southwest Community Center [0109] 4. Whole Foods
Market
[0110] It will be recognized that many such questions follow a
standard form (e.g., a template) that can be recalled and
customized by an automated process that inserts data unique to that
particular neighborhood.
[0111] (In the examples given above, the maps/photographs used in
some of the questions give away answers to other questions.
Naturally, in practical application, this would be avoided.)
[0112] Source material for the foregoing queries can come from
commercial providers (e.g., Google Earth, Terraserver.com, Yahoo!,
MapQuest, etc.), or from government databases (national, state,
municipal, etc.) The overlay of "A" "B" "C" markings, etc., can be
provided--as desired--by the DMV or other entity.
[0113] Yet another source of information useful in quizzing
applicants is Amazon's Yellow Pages search, and its A9 search
engine. Amazon has compiled a library of curbside-scene images,
accessible by street address or business name. Various public
taxation districts also publish such information. Again, such
imagery can be used to test an applicant's familiarity with the
neighborhood in which she claims to reside.
[0114] To illustrate, imagine that the applicant gives an address
that is between two town centers, e.g., Portland and Beaverton. A
mapping database can be used to identify the fastest route between
the stated address and one of the towns. A major intersection along
this route (preferably near the applicant's residence) can be
identified. A curbside imagery database (e.g., Amazon Yellow Pages
A9) can then be queried to obtain a scene from this intersection.
This image, or an unrelated image, can be presented to the
applicant with a question such as: [0115] Referring to FIG. 5, is
this scene: [0116] 1. Between your residence and Beaverton? [0117]
2. Between your residence and Portland? or [0118] 3. Not familiar
to you.
[0119] Microsoft's Windows Live Local service (aka Virtual Earth)
is yet another source of geographic image/map data against which
the knowledge of an applicant may be tested. Microsoft's service
offers "Bird's eye" (oblique), in addition to "aerial" (straight
down), photographic imagery of many city locations. FIG. 6, for
example, shows a "Bird's eye" view of part of the campus of the
Georgia Institute of Technology, obtained through the Microsoft
service. Such imagery can be used in the foregoing examples.
[0120] Another class of resources that might be tapped, in this
example as in others, are online services provided by certain
municipal bus services. Portland's "Tri-Met" transit system, for
example, offers a web-based service (at the domain trimet.org) by
which users can specify their start points and desired
destinations, and the service identifies buses and trains, and
their schedules, that the user can take to commute between the
locations. The system's web services also offer local maps, showing
which bus routes travel along which streets, and the location of
bus stops. A user can specify an intersection, and obtain a map of
bus routes near that intersection. Such a map, and the associated
user interface, is shown in FIG. 7.
[0121] Such transit system information can be used to assess an
applicant's knowledge about their alleged residence. An applicant
may be asked, for example: [0122] S.W. Dakota Street is near your
home. Do mass transit buses run down that street? [0123] 1. Yes.
[0124] 2. No.
[0125] The DMV may sometimes ask questions expecting that the
applicant will not know the answer. For example, a person living in
the neighborhood depicted in the bus map of FIG. 7 may be asked to
name one or more bus routes that travel SW Beaverton-Hillsdale
Highway (e.g., 54, 56, 61 and 92). If the applicant cannot answer
this question, he is not faulted; the answer is likely unfamiliar
to many neighborhood residents. If, however, the applicant can
answer this difficult question correctly, such correct answer may
help his score more than a correct answer to a more routine
question. (The ability to identify two or more of the buses along
this route could boost his score still further.) Thus, answers to
different questions may be weighted differently in scoring
confidence in the applicant's asserted identity.
[0126] Google offers a Transit Trip Planner service
(google.com/transit) that seeks to standardize delivery of mass
transit schedule and route information across many cities'
transportation services. It, too, may be used as a fact resource in
compiling challenge questions for applicants.
[0127] The neighborhood-based questions noted earlier are just a
start. Others may include the following: [0128] Leaving your home,
what street intersection do you first encounter? What if you go the
opposite way? [0129] What cable TV service provider serves your
house/neighborhood? (DSL provider? Electric utility company?)
[0130] What big mall is closest to your home? (Its anchor tenants?)
What supermarket(s)? [0131] Which town(s) adjoin the town where you
live/work? [0132] --What bus route is close to your home/work?
[0133] What is the name of the local mass transit system? [0134]
Does your side of your home street have a curb? A sidewalk? [0135]
Are power poles/lines on side of street where you live? (Or are
they buried in your neighborhood?) [0136] On what street is the
Post Office in your community? [0137] What school is near your
home? [0138] What intersection with traffic light is close to your
home? [0139] Name/number of thoroughfare that links town to
adjoining town of [X] [0140] On what day(s) does garbage get picked
up from your home? [0141] In what year did you purchase (or
mortgage, or refinance) home? If not a homeowner, what is name of
landlord/agency to whom you pay rent? [0142] Name a person who
lives next door.
[0143] The foregoing discussion has focused on a single knowledge
domain--information that an applicant may be expected to know by
reason of their residence address. Many other knowledge domains
exist. One, for example, is information an applicant may be
expected to know by reason of their employment. A few examples of
facts on which an applicant may be quizzed, based on their
employment, follow: [0144] If a truck driver [0145] Validity period
of commercial driving license? [0146] Meaning of acronym GVW?
[0147] Location of highway permit offices? [0148] Location of
highway weigh stations? [0149] Weight restrictions on major
roads/bridges? [0150] Phone number of company's freight dispatcher?
[0151] Tolls on local roads? [0152] If a lawyer [0153] Names of
local judges? [0154] Web site of county bar association? [0155]
Knowledge of CLE requirements (e.g., how often must you report; how
many hours/year required)? [0156] Town where state bar association
has its headquarters?
[0157] It will be recognized that many of these questions are not
strictly based on employment, but also depend on location of
employment. A truck driver in Portland, for example, may be able to
name seven bridges in Portland, but none in Seattle. Generally
speaking, the more closely tailored the questions are to the
applicant profile (e.g., residence/employment/age/etc), the more
useful they will be in establishing (or refuting) confidence in the
asserted identity.
[0158] Another knowledge domain comprises facts an applicant may be
expected to know by reason of their education. For example: [0159]
If a HS student or graduate [0160] Name of principal of HS
where/when graduated (or teacher)? [0161] Size of HS graduating
class (in tiered ranges)? [0162] Sports team name/mascot? [0163] If
a college graduate [0164] (Above questions for HS, adapted to
college) [0165] Town where college is located? [0166] Zip code of
that town? [0167] Is school on schedule of 2 semester or 3
quarter/yr? [0168] Name of a dorm on campus? [0169] Name of a
professor? [0170] If degreed in pharmacology [0171] Generic name
for active ingredient in Benadryl? [0172] Name of company that is
successor to A.H. Robins? [0173] City and/or state where Merck is
headquartered? [0174] Name of college where received pharmacology
degree? [0175] If a lawyer [0176] Name of an early Supreme Court
justice? [0177] Meaning of force majeure? [0178] Name 3
traditionally-required first year courses?
[0179] Some of this information may be readily mined from public
databases. For example, the XML profile excerpted above specified
that the applicant attended Wilson High School in Portland, Oreg.
Querying the online database Google with the input "wilson high
school portland mascot" gives, as the first search result, a "hit"
from the online encyclopedia Wikipedia revealing that that the team
mascot is the "Trojans." See FIG. 8.
[0180] Some of this information may not appear as the first hit of
a Google search, but can be researched with little effort. For
example, information relating to high schools and colleges,
including size and faculty (currently and historically) can be
obtained by web sites maintained by the respective high schools and
universities, and also from databases maintained by independent
providers, such as classmates.com.
[0181] Some of this information can be garnered from independent
human searchers, e.g., using Amazon's Mechanical Turk service.
Amazon's Turk web site explains: [0182] Amazon Mechanical Turk
provides a web services API for computers to integrate Artificial
Artificial Intelligence directly into their processing by making
requests of humans. Developers use the Amazon Mechanical Turk web
services API to submit tasks to the Amazon Mechanical Turk web
site, approve completed tasks, and incorporate the answers into
their software applications. To the application, the transaction
looks very much like any remote procedure call--the application
sends the request, and the service returns the results. In reality,
a network of humans fuels this Artificial Intelligence by coming to
the web site, searching for and completing tasks, and receiving
payment for their work. [0183] All software developers need to do
is write normal code. The pseudo code below illustrates how simple
this can be.
TABLE-US-00002 [0183] read (photo); photoContainsHuman =
callMechanicalTurk(photo); if (photoContainsHuman == TRUE){
acceptPhoto; } else { rejectPhoto; }
More information about Amazon's Mechanical Turk service is provided
in Appendix B (Amazon Mechanical Turk Developer Guide, 2006, 165
pp., API Version Oct. 31, 2006).
[0184] Alternatively, automated processes, e.g., using more
traditional artificial intelligence techniques, can be applied to
generate and check questions from available online resources, such
as the maps and databases noted above, given profile data on the
applicant (e.g., current and prior residence addresses, education,
city where attended high school, etc.).
[0185] There are many other domains of knowledge about which an
applicant can be questioned. One is facts the applicant may be
expected to know by reason of who they know. For example, they may
be asked to provide four memorized telephone numbers, and name the
person to which each corresponds. Another, as noted earlier, is
names of neighbors.
[0186] Still another knowledge domain comprises facts the applicant
may be expected to know by reason of their age. For example: [0187]
Who was Nixon's VP? [0188] In what state did Rosa Parks become
famous? [0189] What was Muhammad Ali's real name? [0190] Who did
Elizabeth Taylor many more than once? [0191] Who did Brad Pitt
recently break up with? [0192] What was Dustin Hoffman's first
famous role? [0193] How many stars did the US flag have during
WWII? [0194] What animal is on the back of old nickels?
[0195] Yet another knowledge domain is facts the applicant may be
expected to know by reason of activities they earlier performed.
For example, readily accessible databases reveal answers to the
following questions on which an applicant may be tested: [0196] Did
you cast a ballot in the November, 2004, election? How about the
presidential race in 2000? In what county/state? [0197] Did you
donate money to the electoral campaign of any official? Who? [0198]
Have you had a motor vehicle violation in past 5 years? (In what
state?) [0199] Have you ever been summoned to jury duty?
[0200] While quizzing an applicant, some questions may be asked to
compile data that can be a source for questions in future
checking--either for the applicant himself/herself, or for another
person. Examples include: [0201] Name your first pet/your favorite
teacher/your favorite frequent flier account number/your first
telephone number/your earliest-remembered zip code. [0202] Name a
non-family member with whom you've lived (e.g., a college
roommate); name a former neighbor; name a person who sits near you
at work. [0203] Name a person who was with you when you learned of
the 9/11 attack, and where you were when you heard the news.
[0204] Another factor that can help confirm identity is an on-line
account. Many individuals have accounts with on-line entities.
Examples include Amazon, PayPal, EBay, Orbitz, Expedia, NetFlix,
etc. Such accounts can provide verification information--either
with the companies' cooperation, or without.
[0205] Consider Amazon. Its web site allows registered users to
sign-in and display historical information, including a list of all
orders ever placed with the company (at last back to 1997, as of
this writing). For each order, Amazon provides information
including the address to which it was shipped, the shipping date,
and the billing address.
[0206] As part of a verification protocol, a DMV clerk may direct a
web browser on a DMV computer to the Amazon Sign-In page. The
applicant can then type his/her account information (e.g., email
address and password), and navigate from the resulting "My Account"
page to select "View by Item--all items ordered" (FIG. 9). A report
like that shown in FIG. 10 is produced, listing all items ordered
through that account. Clicking on an order number generates a
screen like that shown in FIG. 11, showing both the address to
which the item was shipped, as well as the billing address for the
credit card used. If this address information is consistent with
the address given by the applicant, this tends to confirm the
applicant's credibility. In embodiments in which such factors
influence an applicant's score, this would tend to increase that
score. Naturally, the further back in time the applicant can
demonstrate residence at the current address, the more the
applicant's score might be increased. (Evidence that Amazon shipped
a book to the stated address a month ago would be less assuring
than evidence that Amazon shipped a book to that address a year, or
five years, ago.)
[0207] Other on-line services can also be used for this purpose.
EBay reports the date on which people joined and became members,
and their location (e.g., "Member since Apr. 18, 1998; Location:
United States"). Again, a demonstrable history with such an online
vendor can be factored into the analysis of assessing whether the
applicant is who they purport to be.
[0208] Moreover, EBay and other sites provide various mechanisms by
which members are critiqued by their peers or others users. EBay
thus reports a feedback score, indicating the percentage of unique
users who have scored their transactions with a particular member
as favorable. In one example, member "mgmill" has a feedback score
of 100%: 2773 members left positive feedback; none left negative
feedback. (A 100% score based on thousands of feedbacks is
naturally more meaningful than a 100% score based on a dozen
feedbacks.) Such peer/user rankings serve as an additional factor
on which an applicant can be scored (albeit one that might be given
relative little weight--absent some demonstrated correlation
between EBay feedback scores, and the metric that the agency is
trying to assess).
[0209] Expedia, Orbitz, and other online travel companies have
histories that a user can access giving information useful in
verification. From the Expedia home page, for example, a user can
click on "My Account" and sign-in by entering a login name and
password. An Account Overview page is then produced. This page
lists the account holder and others for whom that person has made
travel arrangements. Each name is hyperlinked to a further screen
with information about that traveler, such as home phone number,
work phone number, cell phone number, passport number and country,
frequent flier numbers, etc. At the bottom of the screen there is a
listing of credit cards associated with the account--including
billing addresses and telephone numbers. Again, all such
information can be accessed--with the cooperation of the
applicant--and factored into a possibility-of-fraud fraud
assessment for that applicant.
[0210] PayPal offers further data useful in checking a person.
(PayPal currently has over 80 million accounts.) Again, going to
the PayPal home page yields a screen on which the applicant can
enter their email address and PayPal password. From the resulting
"Overview" screen, the user can navigate to the "Profile" page.
Here again, a great wealth of information useful in corroborating
an applicant is available, e.g., links to email, street address,
telephone, credit card accounts registered with PayPal, bank
accounts registered with PayPal, etc. Clicking on the credit card
account link produces a screen giving the last digits of the credit
card number, and the credit card billing address. Clicking on the
"History" tab allows historical transactions to be reviewed. Like
Amazon, histories going back several years are available. As
before, a long-tenured account may be taken as a favorable factor
in assessing an applicant.
[0211] Again, for any particular PayPal transaction, further
details can be retrieved (e.g., by clicking the "Details" link).
These details include the address to which the purchased item was
to be sent, and information about the credit card or bank account
that was debited. Again, this information can be collected as part
of the verification process, and checked for consistency with other
information about the applicant.
[0212] By such techniques, ID verification can be augmented by
allowing the applicant to demonstrate knowledge of accounts with
which the applicant is associated. Knowledge of passwords, order
history, and consistency between shipping addresses and applicant's
asserted address, all tend to boost a confidence score.
[0213] Still richer data may be obtained by establishing a
commercial relationship with companies having accounts with large
numbers of consumers, so that additional information--not generally
available--might also be made available.
[0214] (Naturally, safeguards may be put into place to assure
privacy of the password and other data entered into the computer by
the applicant to access such information, as well as of the account
information thereby revealed.)
[0215] Such techniques serve to enlarge the "neighborhood" of those
who can vouch for an individual, to encompass entities involved in
commercial transactions.
[0216] Still another source from which question data can be derived
is information provided by prior applicants. Applicants can be
queried for information (questions and answers, or facts) that
would most likely be known to other applicants having a similar
background qualification, but would likely be unknown to others.
Thus, a person who is a lawyer in Portland, Oreg., may offer the
question, "Who is the old federal courthouse named after?" and
offer the answer: "Gus Solomon." Likewise, a bus driver in the same
city may offer "Where is the bus garage on the west side of the
Willamette River? Answer: Merlo Street." A teacher at Hayhurst
Elementary School in Portland may offer "To what middle school do
Hayhurst students go? Answer: Robert Gray Middle School."
[0217] A nineteen year old student at the University of Portland
may offer, "What live music club is closest to school? Answer:
Portsmouth Club." A sixty year old faculty member at the same
school may offer, "What is the Catholic order with which the
University is affiliated? Answer: Holy Cross."
[0218] It will be recognized that the nineteen year old student at
the University of Portland may be less likely to know the answer to
the latter question, and the sixty year old faculty member may be
less likely to know the answer to the former question. Thus, in
many cases, questions should not be selected for presentation to an
applicant based on just a single factor (e.g., a connection to the
University of Portland, or a residence on S.W. Dakota Street).
Rather, the questions may be assigned based on two or more factors
(e.g., age plus school affiliation; profession plus address;
ethnicity plus location where grew up; etc.).
[0219] One way of conceptualizing this is as a multidimensional
vector space in which both questions and applicants can be
located.
[0220] The questions can each be located based on two or more
classification data that help identify applicants who are likely to
be able to answer such questions. Examples of possible
classification data includes geographic location to which question
relates, professional expertise (e.g., law) useful in answering
question, age of persons most likely to be familiar with question,
education of persons most likely to be familiar with questions,
etc.
[0221] The applicant can be similarly located in this vector space,
again based on factors such as noted above.
[0222] The questions nearest the applicant in this vector space are
thus those that most closely match that applicant's background and
other credentials, and are thus most likely to be useful in
confirming that the applicant's background/credentials are as he or
she states.
[0223] In cases where candidate questions are proposed by
applicants, such questions can be tagged with classification data
based on the proposer's attributes. E.g., if a Portland, Oreg.
lawyer, aged 60, proposed the question about the name of the old
courthouse, then that question might be stored with such applicant
attributes (e.g., lawyer, Oregon, age 60) as classification data.
(Classification data may be of the same type, or may even be drawn
from the same vocabulary, as applicant attributes, but this need
not be the case.)
[0224] In one arrangement, question data from which questions are
drawn is stored in a large database, each record tagged with plural
of the classification data noted earlier. Proximity between plural
of the question data, and the particular applicant is then
determined, so as to identify a subset of the original question
space from which questions might usefully be posed. The method can
then select questions from this subset (e.g., randomly) for
presentation to the applicant.
[0225] It should be noted that question data need not be a single
query to which there is a single answer. Question data can span a
larger body of information, e.g., names of familiar Nobel laureates
in physics from 1910-1935, and can provide fodder for many
different, but related, questions.
[0226] In some arrangements, new question information collected
from applicants can be vetted prior to use in verifying further
applicants. Such vetting can be performed by DMV personnel, or by
contractors ("professional vetting," which may include the
Mechanical Turk service). Computer verification may be employed in
some instances.
[0227] Sometimes, the vetting may be performed by presenting such
questions/answers to new applicants, on a trial basis. Answers to
such questions would likely not be included in a substantive
assessment as to the new applicant's identity or risk. Rather, such
answers would be used to judge whether the question/answer is a
useful tool.
[0228] For example, each proposed new question might be posed to
five new applicants, each of whom appears to have a background that
would allow them to correctly answer same. If at least three of
them (or, alternatively, if at least four of them, or if all of
them) give the answer offered by the proposer of the question, then
the question can be moved to the pool of real questions--and used
to substantively assess applicants.
[0229] (If the trial question is proposed to an applicant who does
not know the answer, and other circumstances give rise to
uncertainty whether such applicant's identity is bona fide or
fraudulent, then such data point may be disregarded in assessing
whether the question is a useful tool. Conversely, if the question
is answered correctly by a person whose identity is otherwise
established to be fraudulent, then this suggests that the question
is not a useful discriminator tool.)
[0230] Combinations of professional and applicant vetting can also
be used. For example, a candidate new question may be posed to five
new applicants on a trial basis--as outlined above. If a sufficient
number answer correctly, it can be further vetted by a
professional. Or, such order may be reversed.
[0231] Tests of the sort detailed above might be posed routinely to
applicants. If an applicant does not demonstrate knowledge of an
expected sort, this can trigger further scrutiny of other aspects
of the applicant's application. For example, a DMV agent might
respond by more closely inspecting breeder documents presented by
the dubious applicant, or by requesting additional information not
required from routine applicants.
[0232] Alternatively, the checks presented above might be posed
only if an applicant's profile otherwise suggests further checking
is prudent (e.g., if a risk score based on the ensemble of
presented breeder documents exceeds a threshold).
[0233] In most embodiments, an incorrect answer isn't fatal to
applicant verification. There may be many credible explanations for
an incorrect answer. Rather, answers are typically used to adjust
an aggregate score--with no one question being determinative.
[0234] Applicants' technology detailed herein may be regarded as a
knowledge-based system--utilizing a knowledge base of information
to perform or facilitate an applicant-verification process. Earlier
work in such knowledge-based systems is shown, e.g., by U.S. Pat.
Nos. 6,968,328, 6,965,889 and 6,944,604.
[0235] As referenced above, the Mechanical Turk service can be
widely used in gathering and vetting information used in the
foregoing. (Additionally, the Mechanical Turk service can be used
in conjunction with the systems detailed in the earlier-referenced
patent documents to facilitate some of the operations required by
those systems, e.g., making judgments and undertaking tasks that
computers are ill-suited to perform--such as performing fuzzy
matching, applying common-sense knowledge, and interpreting
documents. Thus, for example, such a system understands that the
names "Bill," "Will," "Wm.," and the like, all can be acceptable
matches to the name "William;" likewise that address lines
"Apartment 3A," "Apt. 3A," "Unit 3A," and the like, all can be
acceptable matches to the address line "Apt. 3-A," etc. Similarly,
the Turk service can be used to harvest public records data that
can be used in verification operations. A number of such
applications of the Mechanical Turk service to the arrangements in
the cited documents are within the capabilities of the artisan from
the teachings herein. Appendix A further details some further
inventive uses of the Mechanical Turk service, and similar
"crowdsourcing" technologies.)
[0236] As noted, the arrangements detailed herein are not limited
to verifying identity prior to issuance of identity documents; they
have applicability in many other contexts.
[0237] To give but one example, consider a passport control
checkpoint in an airport, where a government official inspects
passports of travelers. When the passport is swiped or scanned by a
passport reader, the official is presented with a screen of
information pertaining to the person. This same screen, or another,
can be used to present one or more verification checks like those
detailed above, e.g., showing a map of the passport holder's
neighborhood, and requesting the traveler to identify her home.
[0238] Naturally, in cases where the applicant data is received in
advance of applicant testing, e.g., from a home web session, then
additional time is available to prepare questions customized to
that applicant.
[0239] No particular methodology for generating a score has been
detailed above, because same depends on the facts of particular
cases. One technique for generating a numeric score is a polynomial
approach (such as that detailed in US20040153663), where different
factors are weighted differently, and summed to produce an
aggregate score. Scores may be produced based just on the
information collected through the procedures detailed in this
detailed description, or the scored can be based on a larger set of
data, e.g., including factors on which scores are computed in the
prior art.
[0240] Implementation of systems employing the foregoing principles
is straightforward to artisans, e.g., using standard computer-,
database-, software- and network-technology.
[0241] To provide a comprehensive disclosure without unduly
lengthening this specification, applicants incorporate-by-reference
the documents referenced in this disclosure (including Appendices).
It is expressly contemplated that the technologies, features and
analytical methods detailed herein can be incorporated into the
methods/systems detailed in such other documents. Moreover, the
technologies, features, and analytical methods detailed in those
documents can be incorporated into the methods/systems detailed
herein. (It will be recognized that the brief synopses of prior
documents provided above naturally do not reflect all of the
features found in such disclosures.)
[0242] In view of the wide variety of embodiments to which the
principles and features discussed above can be applied, it should
be apparent that the detailed embodiments are illustrative only and
should not be taken as limiting the scope of the disclosed
technology. Rather, we claim all such modifications as may come
within the scope and spirit of the following claims and equivalents
thereof.
APPENDIX A
Some Further Applications of Crowdsourcing Strategies
[0243] The Mechanical Turk service (detailed in Appendix B) may be
regarded as a structured implementation of a technology commonly
termed "crowdsourcing"--employing a group of outsiders to perform a
task. Wikipedia explains: [0244] "Crowdsourcing" is a neologism for
a business model that depends on work being done outside the
traditional company walls: while outsourcing is typically performed
by lower paid professionals, crowdsourcing relies on a combination
of volunteers and low-paid amateurs who use their spare time to
create content, solve problems, or even do corporate R&D. The
term was coined by Wired magazine writer Jeff Howe and editor Mark
Robinson in June 2006. [0245] Crowds targeted for crowdsourcing
include garage scientists, amateur videographers, freelancers,
photo enthusiasts, data companies, writers, smart mobs and the
electronic herd.
[0246] Overview [0247] While not a new idea, crowdsourcing is
becoming mainstream. Open source projects are a form of
crowdsourcing that has existed for years. People who may not know
one another work together online to create complex software such as
the Linux kernel, and the Firefox browser. In recent years internet
technology has evolved to allow non-technical people to participate
in online projects. Just as important, crowdsourcing presumes that
a large number of enthusiasts can outperform a small group of
experienced professionals.
[0248] Advantages [0249] The main advantages of crowdsourcing is
that innovative ideas can be explored at relatively little cost.
Furthermore, it also helps reduce costs. For example if customers
reject a particular design, it can easily be scrapped. Though
disappointing, this is far less expensive than developing high
volumes of a product that no one wants. Crowdsourcing is also
related to terms like Collective Customer Commitment (CCC) and Mass
Customisation. Collective Customer Commitment (CCC) involves
integrating customers into innovation processes. It helps companies
exploit a pool of talent and ideas and it also helps firms avoid
product flops. Mass Customisation is somewhat similar to collective
customer commitment; however, it also helps companies avoid making
risky decisions about what components to prefabricate and thus
avoids spending for products which may not be marketable later.
[0250] Types of Crowdsourced Work [0251] Steve Jackson Games
maintains a network of MIB (Men In Black), who perform secondary
jobs (mostly product representation) in exchange for free product.
They run publicly or semi-publicly announced play-tests of all
their major books and game systems, in exchange for credit and
product. They maintain an active user community online, and have
done so since the days of BBSes. [0252] Procter & Gamble
employs more than 9000 scientists and researchers in corporate
R&D and still have many problems they can't solve. They now
post these on a website called InnoCentive, offering large cash
rewards to more than 90,000 `solvers` who make up this network of
backyard scientists. P&G also works with NineSigma, YourEncore
and Yet2. [0253] Amazon Mechanical Turk co-ordinates the use of
human intelligence to perform tasks which computers are unable to
do. [0254] YRUHRN used Amazon Mechanical Turk and other means of
crowdsourcing to compile content for a book published just 30 days
after the project was started. [0255] iStockphoto is a website with
over 22,000 amateur photographers who upload and distribute stock
photographs. Because it does not have the same margins as a
professional outfit like Getty Images it is able to sell photos for
a low price. It was recently purchased by Getty Images. [0256]
Cambrian House applies a crowdsourcing model to identify and
develop profitable software ideas. Using a simple voting model,
they attempt to find sticky software ideas that can be developed
using a combination of internal and crowdsourced skills and effort.
[0257] A Swarm of Angels is a project to utilize a swarm of
subscribers (Angels) to help fund, make, contribute, and
distribute, a .English Pound.1 million feature film using the
Internet and all digital technologies. It aims to recruit earlier
development community members with the right expertise into paid
project members, film crew, and production staff. [0258] The
Goldcorp Challenge is an example of how a traditional company in
the mining industry used a crowdsource to identify likely veins of
gold on its Red Lake Property. It was won by Fractal Graphics and
Taylor-Wall and Associates of Australia but more importantly
identified 110 drilling targets, 50% of which were new to the
company. [0259] CafePress and Zazzle, customized products
marketplaces for consumers to create apparel, posters, cards,
stamps, and other products. [0260] Marketocracy, to isolating top
stock market investors around the world in head to head competition
so they can run real mutual funds around these
soon-to-be-discovered investment super-stars. [0261] Threadless, an
internet-based clothing retailer that sells t-shirts which have
been designed by and rated by its users. [0262] Public Insight
Journalism, A project at American Public Media to cover the news by
tapping the collective and specific intelligence of the public.
Gets the newsroom beyond the usual sources, uncovers unexpected
expertise, stories and new angles.
EXTERNAL LINKS AND REFERENCES
[0262] [0263] The Rise of Crowdsourcing, Wired June 2006. [0264]
Crowdsourcing: Consumers as Creators, BusinessWeek July 2006.
[0265] One use of the Mechanical Turk service (or similar
crowdsourcing engines) is in connection with computationally
difficult tasks, such as identification of audio, video and imagery
content. These tasks are sometimes addressed by so-called
"fingerprint" technology, which seeks to generate a "robust hash"
of content (e.g., distilling a digital file of the content down to
perceptually relevant features), and then compare the thus-obtained
fingerprint against a database of reference fingerprints computed
from known pieces of content, to identify a "best" match. Such
technology is detailed, e.g., in Haitsma, et al, "A Highly Robust
Audio Fingerprinting System," Proc. Intl Conf on Music Information
Retrieval, 2002; Cano et al, "A Review of Audio Fingerprinting,"
Journal of VLSI Signal Processing, 41, 271, 272, 2005; Kalker et
al, "Robust Identification of Audio Using Watermarking and
Fingerprinting," in Multimedia Security Handbook, CRC Press, 2005,
and in patent documents WO02/065782, US20060075237, US20050259819,
and US20050141707.
[0266] A particular example of such technology is in facial
recognition--matching an unknown face to a reference database of
facial images. Again, each of the faces is distilled down to a
characteristic set of features, and a match is sought between an
unknown feature set, and feature sets corresponding to reference
images. (The feature set may comprise eigenvectors or shape
primitives.) Patent documents particularly concerned with such
technology include US20020031253, U.S. Pat. No. 6,292,575, U.S.
Pat. No. 6,301,370, U.S. Pat. No. 6,430,306, U.S. Pat. No.
6,466,695, and U.S. Pat. No. 6,563,950.
[0267] These are examples of technology that relies on "fuzzy"
matching. The fingerprint derived from the unknown content often
will not exactly match any of the reference fingerprints in the
database. Thus, the database must be searched not just for the
identical content fingerprint, but also for variants.
[0268] Expanding the search to include variants hugely
complicates--and slows--the database search task. To make the
search tractable, one approach is to prune the
database--identifying excerpts thereof that are believed to be
relatively likely to have a match, and limiting the search to those
excerpts (or, similarly, identifying excerpts that are believed
relatively unlikely to have a match, and not searching those
excerpts).
[0269] The database search may locate several reference
fingerprints that are similar to the fingerprint of the unknown
content. The identification process then seeks to identify a "best"
match, using various algorithms.
[0270] Such content identification systems can be improved by
injecting a human into the process--by the Mechanical Turk service
or similar systems.
[0271] In one particular arrangement, the content identification
system makes an assessment of the results of its search, e.g., by a
score. A score of 100 may correspond to a perfect match between the
unknown fingerprint and a reference fingerprint. Lower scores may
correspond to successively less correspondence. (At some lower
score, S.sub.x, (perhaps 60) the system may decide that there is no
suitable match, and a "no-match" result is returned, with no
identification made.)
[0272] Above some threshold score, S.sub.y, (perhaps 70) the system
may be sufficiently confident of the result that no human
intervention is necessary. At scores below S.sub.y, the system may
make a call through the Mechnical Turk service for assistance.
[0273] The Mechanical Turk can be presented the unknown content (or
an excerpt thereof), and some reference content, and asked to make
a comparison. (The reference content may be stored in the
fingerprint database, or may be readily obtainable through use of a
link stored in the reference database.)
[0274] A single item of reference content can be provided for
comparison with the unknown content, or several items of reference
content can be provided. (Again, excerpts may be used instead of
the complete content objects. Depending on the application, the
content might be processed before sending to the crowdsource
engine, e.g., removing metadata (such as personally identifiable
information: name, driver license number, etc.) that is printed on,
or conveyed with, the file.)
[0275] The requested comparison can take different forms. The
service can be asked simply whether two items appear to match. Or
it can be asked to identify the best of several possible matches
(or indicate that none appears to match). Or it can be asked to
give a relative match score (e.g., 0-100) between the unknown
content and one or more items reference content.
[0276] In many embodiments, a query is referred to several
different humans (e.g., 2-50) through the Mechanical Turk service,
and the returned results are examined for consensus on a particular
answer. In some queries (e.g., does Content A match Content B? Or
is Content A a better match to Content C?), a "vote" may be taken.
A threshold of consensus (e.g., 51%, 75%, 90%, 100%) may be
required in order for the service response to be given weight in
the final analysis. Likewise, in queries that ask the humans to
provide a subjective score, the scores returned from plural such
calls may be combined to yield a net result. (The high and/or low
and/or outlier scores may be disregarded in computing the net
result; weighting can sometimes be employed, as noted below.)
[0277] As suggested, the data returned from the Mechanical Turk
calls may serve as a biasing factor, e.g., pushing an
algorithmically determined output one way or another, to yield a
final answer (e.g., a net score). Or the data returned from the
Mechanical Turk calls may be treated as a definitive answer--with
results from preceding processes disregarded.
[0278] Sometimes the database search may reveal several candidate
matches, all with comparable scores (which may be above the
threshold S.sub.y). Again, one or more calls to the Mechanical Turk
service may be invoked to decide which match is the best, from a
subjective human standpoint.
[0279] Sometimes the Mechanical Turk service can be invoked even in
situations where the original confidence score is below the
threshold, S.sub.x, which is normally taken as indicating "no
match." Thus, the service can be employed to effectively reduce
this threshold--continuing to search for potential matches when the
rote database search does not yield any results that appear
reliable.
[0280] The service can also be invoked to effect database pruning.
For example, a database may be organized with several partitions
(physical or logical), each containing information of a different
class. In a facial recognition database, the data may be segregated
by subject gender (i.e., male facial portraits, female facial
portraits), and/or by age (15-40, 30-65, 55 and higher--data may
sometimes be indexed in two or more classifications), etc. In an
image database, the data may be segregated by topical
classification (e.g., portrait, sports, news, landscape). In an
audio database, the data may be segregated by type (spoken word,
music, other). Each classification, in turn, can be further
segregated (e.g., "music" may be divided into classical, country,
rock, other). And these can be further segregated (e.g., "rock" may
be classified by genre, such as soft rock, hard rock, Southern
rock; by artist, e.g., Beatles, Rolling Stones, etc).
[0281] A call to the Mechanical Turk can be made, passing the
unknown content object (or an excerpt thereof) to a human reviewer,
soliciting advice on classification. The human can indicate the
apparent class to which the object belongs (e.g., is this a male or
female face? Is this music classical, country, rock, or other?).
Or, the human can indicate one or more classes to which the object
does not belong.
[0282] With such human advice (which, again, may involve several
human reviewers, with a voting or scoring arrangement), the system
can focus the database search where a correct match--if any--is
more likely to be found (or avoid searching in unproductive
database excerpts). This focusing can be done at different times.
In one scenario it is done after a rote search is completed, in
which the search results yield matches below the desired confidence
level of S.sub.y. If the database search space is thereafter
restricted by application of human judgment, the search can be
conducted again in the limited search space. A more thorough search
can be undertaken in the indicated subset(s) of the database. Since
a smaller excerpt is being searched, a looser criteria for a
"match" might be employed, since the likelihood of false-positive
matches is diminished. Thus, for example, the desired confidence
level S.sub.y might be reduced from 70 to 65. Or the threshold
S.sub.x at which "no match" is concluded, may be reduced from 60 to
55. Alternatively, the focusing can be done before any rote
searching is attempted.
[0283] The result of such a human-focused search may reveal one or
more candidate matches. The Mechanical Turk service may be called a
second time, to vet the candidate matches--in the manner discussed
above. This is one of several cases in which it may be desirable to
cascade Mechanical Turk calls--the subsequent calls benefiting from
the former.
[0284] In the example just-given, the first Mechanical Turk call
aids in pruning the database for subsequent search. The second call
aids in assessing the results of that subsequent search. In other
arrangements, Mechanical Turk calls of the same sort can be
cascaded.
[0285] For example, the Mechanical Turk first may be called to
identify audio as music/speech/other. A second call may identify
music (identified per the first call) as
classical/country/rock/other. A third call may identify rock
(identified per the second call) as Beatles/Rolling Stones/etc.
Here, again, by iterative calling of a crowdsourcing service, a
subjective judgment can be made that would be very difficult to
achieve otherwise.
[0286] In some arrangements, human reviewers are pre-qualified as
knowledgeable in a specific domain (e.g., relatively expert in
recognizing Beatles music). This qualification can be established
by an online examination, which reviewers are invited to take to
enable them to take on specific tasks (often at an increased rate
of pay). Some queries may be routed only to individuals that are
pre-qualified in a particular knowledge domain. In the cascaded
example just given, for example, the third call might be routed to
one or more users with demonstrated expertise with the Beatles
(and, optionally, to one or more users with demonstrated expertise
with the Rolling Stones, etc). A positive identification of the
unknown content as sounding like the Beatles would be given more
relative weight if coming from a human qualified in this knowledge
domain. (Such weighting may be taken into account when aggregating
results from plural human reviewers. For example, consider an
unknown audio clip sent to six reviewers, two with expertise in the
Beatles, two with expertise in the Rolling Stones, and two with
expertise in the Grateful Dead. Assume the Beatles experts identify
it as Beatles music, the Rolling Stones experts identify it as
Grateful Dead music, and the Grateful Dead experts identify it as
Rolling Stones music. Despite the fact that there are tie votes,
and despite the fact that no selection earned a majority of the
votes, the content identification service that made these calls and
is provided with these results may logically conclude that the
music is Beatles.)
[0287] Calls to the Mechanical Turk service may request the human
to provide metadata relevant to any content reviewed. This can
include supposed artist(s), genre, title, subject, date, etc. This
information (which may be ancillary to a main request, or may
comprise the entirety of the request) can be entered into a
database. For example, it can be entered into a fingerprint
database--in association with the content reviewed by the
human.
[0288] Desirably, data gleaned from Mechanical Turk calls are
entered into the database, and employed to enrich its data--and
enrich information that can be later mined from the database. For
example, if unknown content X has a fingerprint F.sub.x, and
through the Mechanical Turk service it is determined that this
content is a match to reference content Y, with fingerprint
F.sub.y, then a corresponding notation can be added to the
database, so that a later query on fingerprint F.sub.x (or close
variants thereof) will indicate a match to content Y. (E.g., a
lookup table initially indexed with a hash of the fingerprint
F.sub.x will point to the database record for content Y.)
[0289] Calls to outsourcing engines involve a time lag before
results are returned. The calling system can generally cope, or be
adapted to cope, with such lags.
[0290] Consider a user social networking site such as YouTube (now
Google) that distributes "user generated content" (e.g., video
files), and employs fingerprinting to recognize media content that
should not be distributed. The site may check a video file at the
time of its uploading with a fingerprint recognition system (e.g.,
of the sort offered by Audible Magic, or Gracenote). If no clear
match is identified, the video may be indexed and stored on
YouTube's servers, available for public downloading. Meanwhile, the
content can be queued for review by one our more crowdsource
reviewers. They may recognize it as a clip from the old TV sitcom
"I Love Lucy"--perhaps digitally rotated 3 degrees to avoid
fingerprint detection. This tentative identification is returned to
YouTube from the API call. YouTube can check the returning metadata
against a title list of works that should not be distributed (e.g.,
per the request of copyright owners), and may discover that "I Love
Lucy" clips should not be distributed. It can then remove the
content from public distribution. (This generally follows a
double-check of the identification by a YouTube employee.)
Additionally, the fingerprint database can be updated with the
fingerprint of the rotated version of the I Love Lucy, allowing it
to be immediately recognized the next time it is encountered.
[0291] If the content is already being delivered to a user at the
moment the determination is made (i.e., the determination that the
content should not be distributed publicly), then the delivery can
be interrupted. An explanatory message can be provided to the user
(e.g., a splash screen presented at the interruption point in the
video).
[0292] Rotating a video by a few degrees is one of several hacks
that can defeat fingerprint identification. (It is axiomatic that
introduction of any new content protection technology draws hacker
scrutiny. Familiar examples include attacks against Macrovision
protection for VHS tapes, and against CSS protection for packaged
DVD discs.) If fingerprinting is employed in content protection
applications, such as in social networking sites (as outlined
above) or peer-to-peer networks, its vulnerability to attack will
eventually be determined and exploited.
[0293] Each fingerprinting algorithm has particular weaknesses that
can be exploited by hackers to defeat same. An example will help
illustrate.
[0294] A well known fingerprinting algorithm operates by repeatedly
analyzing the frequency content of a short excerpt of an audio
track (e.g., 0.4 seconds). The method determines the relative
energy of this excerpt within 33 narrow frequency bands that
logarithmically span the range 300 Hz-2000 Hz. A corresponding
32-bit identifier is then generated from the resulting data. In
particular, a frequency band corresponds to a data bit "1" if its
energy level is larger than that of the band above, and a "0" if
its energy level is lower. (A more complex arrangement can also
take variations over time into account, outputting a "1" only if
the immediately preceding excerpt also met the same test, i.e.,
having a band energy greater than the band above.)
[0295] Such a 32 bit identifier is computed every hundredth of a
second or so, for the immediately preceding 0.4 second excerpt of
the audio track, resulting in a large number of "fingerprints."
This series of characteristic fingerprints can be stored in a
database entry associated with the track, or only a subset may be
stored (e.g., every fourth fingerprint).
[0296] When an unknown track is encountered, the same calculation
process is repeated. The resulting set of data is then compared
against data earlier stored in the database to try and identify a
match. (As noted, various strategies can be employed to speed the
search over a brute-force search technique, which yields
unacceptable search times.)
[0297] While the just-described technique is designed for audio
identification, a similar arrangement can be used for video.
Instead of energies in audio subbands, the algorithm can use
average luminances of blocks into which the image is divided as the
key perceptual features. Again, a fingerprint can be defined by
determining whether the luminance in each block is larger or
smaller than the luminance of the preceding block.
[0298] While little has been written about attacks targeting
fingerprinting systems, a casual examination of possible attack
scenarios reveals several possibilities. A true hacker will
probably see many more. Four simple approaches are discussed
below.
Radio Loudness Profiling
[0299] The reader may be familiar with different loudness profiles
selectable on car radios, e.g., Jazz, Talk, Rock, etc. Each applies
a different frequency equalization profile to the audio, e.g.,
making bass notes louder if the Rock setting is selected, and
quieter if the Talk setting is selected, etc. The difference is
often quite audible when switching between different settings.
[0300] However, if the radio is simply turned on and tuned to
different stations, the listener is generally unaware of which
loudness profile is being employed. That is, without the ability to
switch between different profiles, the frequency equalization
imposed by a particular loudness profile is typically not noticed
by a listener. The different loudness profiles, however, yield
different fingerprints.
[0301] For example, in the Rock setting, the 300 Hz energy in a
particular 0.4 second excerpt may be greater than the 318 Hz
energy. However, in the Talk setting, the situation may be
reversed. This change prompts a change in the leading bit of the
fingerprint.
[0302] In practice, an attacker would probably apply loudness
profiles more complex than those commonly available in car
radios--increasing and decreasing the loudness at many different
frequency bands (e.g., 32 different frequency bands). Significantly
different fingerprints may thus be produced. Moreover, the loudness
profile could change with time--further distancing the resulting
fingerprint from the reference values stored in a database.
Multiband Compression
[0303] Another process readily available to attackers is audio
multiband compression, a form of processing that is commonly
employed by broadcasters to increase the apparent loudness of their
signal (most especially commercials). Such tools operate by
reducing the dynamic range of a soundtrack--increasing the loudness
of quiet passages on a band-by-band basis, to thereby achieve a
higher average signal level. Again, this processing of the audio
changes its fingerprint, yet is generally not objectionable to the
listeners.
Psychoacoustic Processing
[0304] The two examples given above are informal attacks--common
signal processing techniques that yield, as side-effects, changes
in audio fingerprints. Formal attacks--signal processing techniques
that are optimized for purposes of changing fingerprints--are
numerous.
[0305] Some formal attacks are based on psychoacoustic masking.
This is the phenomena by which, e.g., a loud sound at one instant
(e.g., a drum beat) obscures a listener's ability to perceive a
quieter sound at a later instant. Or the phenomena by which a loud
sound at one frequency (e.g., 338 Hz) obscures a listener's ability
to perceive a quieter sound at a nearby frequency (e.g., 358 Hz) at
the same instant. Research in this field goes back decades. (Modern
watermarking software employs psychoacoustic masking in an
advantageous way, to help hide extra data in audio and video
content.)
[0306] Hacking software, of course, can likewise examine a song's
characteristics and identify the psychoacoustic masking
opportunities it presents. Such software can then automatically
make slight alterations in the song's frequency components in a way
that a listener won't be able to note, yet in a way that will
produce a different series of characteristic fingerprints. The
processed song will be audibly indistinguishable from the original,
but will not "match" any series of fingerprints in the
database.
Threshold Biasing
[0307] Another formal attack targets fingerprint bit determinations
that are near a threshold, and slightly adjusts the signal to swing
the outcome the other way. Consider an audio excerpt that has the
following respective energy levels (on a scale of 0-99), in the
frequency bands indicated:
TABLE-US-00003 300 Hz 318 Hz 338 Hz 358 Hz 69 71 70 68
[0308] The algorithm detailed above would generate a fingerprint of
{011 . . . } from this data (i.e., 69 is less than 71, so the first
bit is `0`; 71 is greater than 70, so the second bit is `1`; 70 is
greater than 68, so the third bit is `1`).
[0309] Seeing that the energy levels are somewhat close, an
attacker tool could slightly adjust the signal's spectral
composition, so that the relative energy levels are as follows:
TABLE-US-00004 300 Hz 318 Hz 338 Hz 358 Hz [69] 70 [71] 69 70
68
[0310] Instead of {011 . . . }, the fingerprint is now {101 . . .
}. Two of the three illustrated fingerprint bits have been changed.
Yet the change to the audio excerpt is essentially inaudible.
Exploiting Database Pruning
[0311] Other fingerprint hacking vulnerabilities arise from
shortcuts employed in the database searching strategy--seeking to
prune large segments of the data from further searching. For
example, the system outlined above confines the large potential
search space by assuming that there exists a 32 bit excerpt of the
unknown song fingerprint that exactly matches (or matches with only
one bit error) a 32 bit excerpt of fingerprint data in the
reference database. The system looks at successive 32 bit excerpts
from the unknown song fingerprint, and identifies all database
fingerprints that include an excerpt presenting a very close match
(i.e., 0 or 1 errors). A list of candidate song fingerprints is
thereby identified that can be further checked to determine if any
meets the looser match criteria generally used. (To allow non-exact
fingerprint matches, the system generally allows up to 2047 bit
errors in every 8192 bit block of fingerprint data.)
[0312] The evident problem is: what if the correct "match" in the
database has no 32 bit excerpt that corresponds--with just 1 or 0
bit errors--to a 32 bit excerpt from the unknown song? Such a
correct match will never be found--it gets screened out at the
outset.
[0313] A hacker familiar with the system's principles will see that
everything hinges on the assumption that a 32 bit string of
fingerprint data will identically match (or match with only one bit
error) a corresponding string in the reference database. Since
these 32 bits are based on the strengths of 32 narrow frequency
bands between 300 Hz and 2000 Hz, the spectrum of the content can
readily be tweaked to violate this assumption, forcing a
false-negative error. (E.g., notching out two of these narrow bands
will force four bits of every 32 to a known state: two will go to
zero--since these bands are lower in amplitude than the preceding
bands, and two will go to one--since the following bands are higher
in amplitude that these preceding, notched, bands). On average,
half of these forced bits will be "wrong" (compared to the
untweaked music), leading to two bit errors--violating the
assumption on which database pruning is based.)
[0314] Attacks like the foregoing require a bit of effort. However,
once an attacker makes the effort, the resulting hack can be spread
quickly and widely.
[0315] The exemplary fingerprinting technique noted above (which is
understood to be the basis for Gracenote's commercial
implementation, MusicID, built from technology licensed from
Philips) is not unique in being vulnerable to various attacks. All
fingerprinting techniques (including the recently announced
MediaHedge, as well as CopySense and RepliCheck) are similarly
believed to have vulnerabilities that can be exploited by hackers.
(A quandary for potential adopters is that susceptibility of
different techniques to different attacks has not been a focus of
academic attention.)
[0316] It will be recognized that crowdsourcing can help mitigate
the vulnerabilities and uncertainties that are inherent in
fingerprinting systems. Despite a "no-match" returned from the
fingerprint-based content identification system (based on its rote
search of the database for a fingerprint that matches that of the
altered content), the techniques detailed herein allow human
judgment to take a "second look." Such techniques can identify
content that has been altered to avoid its correct identification
by fingerprint techniques. (Again, once such identification is
made, corresponding information is desirably entered into the
database to facilitate identification of the altered content next
time.)
[0317] It will be recognized that the "crowdsourcing" methodologies
detailed above also have applicability to other tasks involved in
the arrangements detailed in the specification, including all the
documents incorporated by reference.
* * * * *