U.S. patent application number 10/912433 was filed with the patent office on 2005-02-10 for system and method for evaluating job candidates.
Invention is credited to Ladouceur, David, LaPasta, Douglas G., Mincer, Martha.
Application Number | 20050033633 10/912433 |
Document ID | / |
Family ID | 34193125 |
Filed Date | 2005-02-10 |
United States Patent
Application |
20050033633 |
Kind Code |
A1 |
LaPasta, Douglas G. ; et
al. |
February 10, 2005 |
System and method for evaluating job candidates
Abstract
Human resource system for collecting and analyzing survey data
from reference providers identified by a job candidate for use by
an employer. The system sends an electronic communication to the
reference providers to request completion of survey questions and
electronically receives the survey data from the reference
providers, preferably through webpages. An analysis module combines
the received survey data from the reference providers and generates
a confidential candidate report for an employer which excludes
identification of any ratings or comments by any reference
providers. The system also generates customized interview probe
questions for use during job interviews and coaching tactics for
use after the hiring, based on the weak areas that have been
identified from the completed surveys in order to assist the hiring
manager to bring the new hires up to speed quickly and
effectively.
Inventors: |
LaPasta, Douglas G.;
(Hawley, PA) ; Mincer, Martha; (Milford, PA)
; Ladouceur, David; (Westminster, CO) |
Correspondence
Address: |
REED SMITH, LLP
ATTN: PATENT RECORDS DEPARTMENT
599 LEXINGTON AVENUE, 29TH FLOOR
NEW YORK
NY
10022-7650
US
|
Family ID: |
34193125 |
Appl. No.: |
10/912433 |
Filed: |
August 4, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60492457 |
Aug 4, 2003 |
|
|
|
Current U.S.
Class: |
434/362 ;
705/1.1; 705/321 |
Current CPC
Class: |
G06Q 10/10 20130101;
G06Q 10/1053 20130101 |
Class at
Publication: |
705/011 ;
705/001 |
International
Class: |
G06F 017/60 |
Claims
What is claimed is:
1. A human resource information system employing competency based
performance evaluation parameters for providing an analysis of an
object person's competency to a client through interrogation of
reference individuals, comprising: a data processor, a
predetermined set of competency based questions requesting
quantitative responses to provide a set of predetermined competency
parameters for the object person, an I/O interface operable to
transmit through the internet a predetermined subset of said
competency based questions to a set of reference individuals and to
receive through the internet the responses from members of said set
of reference individuals, said data processor coupled to said I/O
interface to merge the responses from the members of said set of
reference individuals to provide a continuing cumulative
quantitative analysis of each parameter in said subset of
competency based questions, a client terminal coupled to said
client computer to permit client interrogation of said cumulative
quantitative analysis, said analysis being decoupled from the
reference individuals to provide anonymity for said reference
individuals.
2. The system of claim 1, wherein: the client inputs a portion of
the contents of said predetermined subset of competency based
questions.
3. The system of claim 1, wherein: said client interrogation is
restricted to a stage wherein said cumulative analysis is based on
more than a predetermined number of responses received from the
reference individuals.
4. The system of claim 2, wherein: said client interrogation is
restricted to a stage wherein said cumulative analysis is based on
more than a predetermined number of responses received from the
reference individuals.
5. The system of claim 1, wherein said data processor is further
operable to provide a comparison of said cumulative quantitative
analysis of any one or more of said parameters to the corresponding
parameter analysis to a predetermined set of individuals whose
performance assessment parameters have been processed by the
system.
6. The system of claim 2, wherein said data processor is further
operable to provide a comparison of said cumulative quantitative
analysis of any one or more of said parameters to the corresponding
parameter analysis to a predetermined set of individuals whose
performance assessment parameters have been processed by the
system.
7. The system of claim 3, wherein said data processor is further
operable to provide a comparison of said cumulative quantitative
analysis of any one or more of said parameters to the corresponding
parameter analysis to a predetermined set of individuals whose
performance assessment parameters have been processed by the
system.
8. The system of claim 4, wherein said data processor is further
operable to provide a comparison of said cumulative quantitative
analysis of any one or more of said parameters to the corresponding
parameter analysis to a predetermined set of individuals whose
performance assessment parameters have been processed by the
system.
9. The system of claim 1, wherein said data processor merges the
responses from the members of said set of reference individuals to
provide a continuing cumulative quantitative analysis upon a
request from the client.
10. A method of providing personnel analysis for a client based on
competency based performance evaluation questions of an object
individual comprising the steps of: providing a set of competency
based questions requesting quantitative responses to provide a set
of predetermined competency parameters, transmitting said set of
questions over the internet to each member of a set of reference
individuals provided by the object individual for use by a client
interested in determining the object individual's competency for a
given position, receiving responses to said set of questions over
the internet from members of said set of reference individuals,
merging said responses to provide a quantitative score of each of
said parameters based on the quantitative responses provided for
each of the parameters, executing said step of merging continuously
to provide continuous up-to-date quantitative scores, providing
said quantitative scores to a client terminal to permit continuous
client interrogation of said quantitative scores subsequent to
processing responses from a predetermined number of reference
individuals in said step of merging.
11. The method of claim 10, wherein said step of providing said
quantitative scores to a client terminal is limited to a state in
said step of executing wherein responses from more than a
predetermined number of reference individuals have been merged.
12. The method of claim 10, wherein executing said step of merging
continuously to provide continuous up-to-date quantitative scores
is performed upon a request from the client.
13. In a computer network, a computer implemented system for
collecting and analyzing survey data from a plurality of reference
providers identified by a job candidate for use by an employer, the
system comprising: a candidate database that stores survey data
provided from a plurality of reference providers for a job
candidate; a collection module operable to send an electronic
communication to the plurality of reference providers to request
completion of a survey containing survey questions and to
electronically receive through a computer network the survey data
from the plurality of reference providers for storage in the
candidate database; an analysis module operable to combine the
received survey data from the plurality of reference providers and
generate a candidate report for an employer.
14. The computer implemented system according to claim 13, further
comprising: a survey database that stores a plurality of survey
questions for different job types; a set-up module that allows the
employer to select a job type and customize a survey for the
selected job type.
15. The computer implemented system according to claim 13, wherein
the survey questions are related to competencies that represent the
characteristics of an individual.
16. The computer implemented system according to claim 13, wherein
the collection module is further operable to generate a web page
containing the survey questions and to receive the survey data
through the web page.
17. The computer implemented system according to claim 16, wherein
the survey includes a plurality of questions and an answer to each
of the plurality of questions is a number within a numerical range
and the collection module generates a graphical button for each
number in the numerical range through which the survey data are
collected.
18. The computer implemented system according to claim 13, wherein
the collection module is operable to provide status information
regarding the progress of completing the survey questions by the
reference providers.
19. The computer implemented system according to claim 13, wherein
the analysis module generates the candidate report which contains a
score for each survey question, the score representing an average
of all answers to the each question by the plurality of reference
providers.
20. The computer implemented system according to claim 13, wherein
the survey questions are grouped into categories, and the analysis
module generates a candidate report which contains a score for each
category, the score being based on answers from the plurality of
reference providers to all questions that belong to the each
category.
21. The computer implemented system according to claim 13, wherein
the analysis module generates the candidate report which contains
an identification of weak areas for the job candidate based on
benchmark scores.
22. The computer implemented system according to claim 13, wherein
the analysis module generates the candidate report which contains
an identification of weak areas for the job candidate based on
benchmark scores and which contains interview questions that allow
the employer to further explorer the identified weak areas during a
subsequent interview with the job candidate.
23. The computer implemented system according to claim 13, wherein
the analysis module generates the candidate report which contains
an identification of weak areas for the candidate based on
benchmark scores and which contains coaching suggestions that allow
the employer to improve the identified weak areas after the
candidate is hired.
24. The computer implemented system according to claim 13, wherein
the analysis module generates a group candidate report which ranks
multiple candidates.
25. The computer implemented system according to claim 13, wherein
the analysis module upon request is operable to generate an interim
candidate report that includes all of the survey data that have
been provided by the reference providers.
26. The computer implemented system according to claim 13, wherein
the analysis module generates a report that correlates the survey
questions to performance of hired candidates.
27. In a computer network, a computer implemented system for
collecting and analyzing survey data from a plurality of reference
providers identified by a job candidate for use by an employer, the
system comprising: a candidate database that stores survey data
provided from a plurality of reference providers for a job
candidate, the survey data representing responses to a survey
containing a predetermined set of survey questions; a benchmark
database that contains benchmark data; a collection module operable
to send an electronic communication to the plurality of reference
providers to request completion of survey questions and to
electronically receive survey data from the plurality of reference
providers through a computer network; an analysis module operable
to compare the received survey data against the benchmark database
and generate a candidate report based on the comparison, the
candidate report being anonymous so that the identities of the
reference providers are excluded from the candidate report.
28. The computer implemented system according to claim 27, wherein
the collection module is further operable to dynamically generate a
web page containing the survey questions and to receive the survey
data through the web page.
29. The computer implemented system according to claim 28, wherein
the survey includes a plurality of questions and an answer to each
of the plurality of questions is a number within a numerical range
and the collection module generates a graphical button for each
number in the numerical range through which the survey data are
collected.
30. The computer implemented system according to claim 27, wherein
the analysis module generates the candidate report which contains a
score for each survey question, the score representing an average
of all answers to the each question by the plurality of reference
providers.
31. The computer implemented system according to claim 27, wherein
the analysis module generates the candidate report which contains
an identification of weak areas for the job candidate based on the
benchmark data.
32. The computer implemented system according to claim 27, wherein
the analysis module analyzes performance data of hired candidates
and generates a report that correlates the survey questions to
performance of the hired candidates.
33. A computer implemented method of collecting and analyzing
survey data from a plurality of reference providers identified by a
job candidate for use by an employer, the method comprising:
sending an electronic communication to a plurality of reference
providers for a job candidate to request completion of survey
questions; electronically receiving through a computer network the
survey data from the plurality of reference providers for storage
in the candidate database; and combining the received survey data
from the plurality of reference providers and generating a
candidate report for an employer.
34. The method according to claim 33, further comprising generating
a web page containing the survey questions and the step of
electronically receives the survey data through the web page.
35. The computer implemented system according to claim 33, wherein
the step of generating a candidate report includes generating the
candidate report which contains an identification of weak areas for
the job candidate based on benchmark scores.
36. The computer implemented system according to claim 33, wherein
the step of generating a candidate report includes generating the
candidate report which contains an identification of weak areas for
the job candidate based on benchmark scores and which contains
interview questions that allow the employer to further explorer the
identified weak areas during a subsequent interview with the job
candidate.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. provisional patent
application No. 60/492,457, filed Aug. 4, 2003, which is
incorporated herein by reference.
FIELD OF THE INVENTION
[0002] This invention relates to human resource management system,
and more particularly to a system for collecting and analyzing
information from references identified by job candidates.
BACKGROUND OF THE INVENTION
[0003] It has been estimated that errors in hiring cost companies
more than 50 billion dollars per year in lost revenue, decreased
productivity, squandered training expenses, legal liabilities, high
turnover and other undesirable consequences. The result, in spite
of the dramatic increase in the use of background checks and
psychological profiles by many organizations, and the universality
of selection interviews, is that annual turnover in U.S. companies
of over 5,000 employees continues to be 25%. According to experts
in the field, nearly 80% of turnover is caused by poor selection
decisions.
[0004] Therefore, background checks have become even more important
than in the past. One part of the background check, and more
generally the hiring process, is the gathering of information from
references, that is those individuals identified by a job candidate
as being knowledgeable about the candidate's character and
qualifications.
[0005] Unfortunately, traditional reference checking methods such
as telephone interviews are very costly and time-consuming, require
extensive training for interviewers and generally do not yield
useful information due to lack of precision, lack of
confidentiality and possible errors of filtering, amplification and
interpretations by the intermediaries. Most important, perhaps, is
the fact that in today's litigious society, business and
professional reference givers are usually unwilling to provide more
than basic information such as employment dates and positions held.
Such information has little value in helping companies make
effective hiring decisions.
[0006] Another problem with the conventional reference checking is
that it's done very late in the hiring process, which is typically
done after the candidate is hired. Ideally, it should be done
during the screening and selection process. Moreover, the
conventional background checking provides no guidance for the
hiring manager to further explore areas of weakness in the
candidate during the hiring process.
[0007] Therefore, it would be desirable to provide a more effective
and inexpensive system and method for collecting and evaluating
information provided by reference providers for job candidates. It
would be also desirable to provide such a system that is
substantially automated and that is used early in the hiring
process.
SUMMARY OF THE DISCLOSURE
[0008] A system for collecting and analyzing survey data from
reference providers identified by a job candidate for use by an
employer is provided. The system includes a candidate database that
stores survey data which are provided by the reference providers. A
collection module running in the system sends an electronic
communication to the reference providers requesting them to
complete the survey questions and electronically receives the
survey data. The electronic communication preferably contains a URL
link that takes the reference provider to a dynamically generated
webpage through which the survey data are entered.
[0009] An analysis module running in the system combines the
received survey data from the reference providers and generates a
candidate report. In one aspect, the candidate report is a
confidential report which excludes identification of any ratings or
comments by any reference providers. In another aspect, the system
also generates customized interview probe questions for use during
job interviews and coaching tactics for use after the hiring, based
on the weak areas that have been identified from the completed
surveys in order to assist the hiring manager to bring the new
hires up to speed quickly and effectively.
[0010] The human resource system provides substantially automated
collection and analysis which is inexpensive and yet accurate and
useful.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] FIG. 1 is a block diagram of a job candidate evaluation
system according to the present invention.
[0012] FIG. 2 is a simplified process flow of a candidate
evaluation process according to the present invention.
[0013] FIG. 3 illustrates a detailed process flow of a purchase and
set-up step of FIG. 2.
[0014] FIG. 4 illustrates a detailed process flow of a candidate
and reference identification step of FIG. 2.
[0015] FIG. 5 illustrates a detailed process flow of a collection
step of FIG. 2.
[0016] FIG. 6 illustrates a detailed process flow of an analysis
step of FIG. 2.
[0017] FIG. 7 illustrates a detailed process flow of a continuous
update step of FIG. 2.
[0018] FIGS. 8A and 8B illustrate a sample survey form.
[0019] FIG. 9 is a sample email sent to a job candidate.
[0020] FIG. 10 is a sample email sent to each reference identified
by the job candidate.
[0021] FIGS. 11A-11D illustrate a sample candidate report for use
by a hiring manager.
[0022] FIGS. 12A-12B illustrate a sample set of interview questions
for use by the hiring manager in a subsequent interview with the
job candidate.
[0023] FIGS. 13-13C illustrate a sample set of coaching tactics for
use by the hiring manager after the job candidate is hired.
[0024] FIG. 14 illustrates a group report which ranks multiple job
candidates.
[0025] FIGS. 15A and 15B illustrate a sample vendor report that
evaluates vendors supplying goods and services to a company.
[0026] FIG. 16 illustrates a sample report that shows the
correlation between survey questions/competencies and performance
of hired candidates.
DETAILED DESCRIPTION OF THE INVENTION
[0027] As an overview, the present invention uses a computer
network such as the Internet and the resources of the network
including emails and webpages to set up initial survey questions,
send out emails to references identified by job candidates, collect
confidential competency-based survey information via webpages from
the identified references, analyze the collected information and
generate candidate reports for use by a hiring manager. The system
has integrated the screening and selection process with a
competency based survey database that allows the comparative review
of reference information against one or more candidates, the
company's own employees, the industry or other normalized database
by job type, organization or company competency.
[0028] Referring now to FIG. 1, a job candidate evaluation system 1
of the present invention involves a number of computers 10, 15 that
are connected to each other through a computer network such as the
Internet. The computers 10, 15 of the system 1 cooperate with each
other to provide comprehensive collection and analysis of reference
information that are made through the network 2. Computers 15 are
similar to the computer 10, with the exception of some of the
databases and software modules.
[0029] As illustrated in FIG. 1, the computer 10 is connected to
the Internet 2 through, for example, an I/O interface 12, such as
for a LAN, WAN, or fiber optic, wireless or cable link, which
receives information from and sends information to other computers
15. The computer 10 is also connected to a keyboard 14 for
controlling the computer.
[0030] The computer 10 includes, for example, memory storage 16,
processor (CPU) 18, program storage 20, and data storage 22, all
commonly connected to each other through a bus 24. The program
storage 20 stores, among others, software programs such as set-up
module 26, collection module 28, and analysis module 30 as will be
explained in detail later herein. The data storage 22 stores, among
others, candidate database 34, benchmark database 36 and survey
database 318, all preferably stored in a relational database that
relates all of the databases stored in the data storage. Any of the
software program modules in the program storage 20 and data from
the data storage 22 are transferred to the memory 32 as needed and
is executed by the processor 18.
[0031] The computer 10 can be any computer such as a WINDOWS-based
or UNIX-based personal computer, server, workstation or a
mainframe, or a combination thereof. While the computer 10 is
illustrated as a single computer unit for purposes of clarity,
persons of ordinary skill in the art will appreciate that the
system may comprise a group of computers which can be scaled
depending on the processing load and database size and which can be
remotely located to provide localized non-stop service.
[0032] FIG. 2 illustrates a high level process flow of the
evaluation process according to the present invention. In step 42,
a client company sets up an order for a job candidate or multiple
candidates, and prepares one or more surveys for use in the hiring
process as will be explained in detail with reference to FIG. 3.
This step is executed by the set-up module 26. In the same step,
the order for the job candidates is done by purchasing a certain
number of candidate reports and specifying the purchase
information. Purchase of one report provides unlimited generation
of reports for each job candidate until the time of hire or
rejection.
[0033] In step 44, the hiring manager identifies a job candidate
and receives information about the references or reference
providers that are identified by the job candidate, which include
an email address for each reference as will be explained in more
detail with reference to FIG. 4. A reference provider should be
someone who has worked extensively with the job candidate in the
past which include customers, supervisors, and peers. Step 44 is
executed by the collection module 28.
[0034] In step 46, which is also executed by the collection module
28, the system 1 sends emails to all of the references that were
identified by the job candidate. The email requests each reference
to fill out the survey prepared by the hiring manager. The survey
information is then collected from the identified references
through web pages and stored in the survey database 38. Step 46 is
more fully explained with reference to FIG. 5.
[0035] In step 48, executed by the analysis module 30, the system 1
analyzes the collected information and generates a report that
includes the overall assessment of the candidate's competency in
each of the several competency areas and includes any comments
supplied by the references. Competency is a well known concept that
represents a particular characteristic of an individual or
organization performing a task, function or project at a particular
point in time that leads to successful performance. The report can
be a final report with all surveys completed by the references, or
it can be a real time interim report with analysis of partially
completed survey information which can be requested by the hiring
manager at any time. Based on the analysis, the system 1 also
generates "interview probes" for those areas where the candidate
did not score as highly as others, a sample of which is shown in
FIGS. 12A-12B. The probes guide an interviewer to obtain more
information about the candidate's level of accomplishments or
experience with regard to specific lower scoring competencies. The
report may also include coaching tactics to manage and develop the
candidate assuming the candidate is hired, a sample of which is
shown in FIGS. 13A-13C. Like the interview probes, the coaching
tactics are also based on analysis of those areas where the
candidate did not score as highly as others. The coaching tactics
are suggested "micro-behaviors" that the hiring manager can use to
help the candidate to develop his strengths in lower-scoring
competencies. Step 48 is more fully explained with reference to
FIG. 6.
[0036] In step 50, the system 1 continuously monitors the job
candidates after the hiring process. The system tracks the progress
of the hired candidates and collects additional data such as
performance levels of the hired candidates. The additional data for
all the candidates are then analyzed to generate additional reports
containing the correlation between various competencies and high
retention/performance. The reports are preferably graphical in
nature and graphically illustrate the competencies that are most
closely correlated with the high retention/performance of the
candidates. The reports can also be customized by a user to specify
whether the correlation is based on position-specific, company-wide
or industry-wide benchmarks as will be explained in detail with
reference to FIG. 7.
[0037] FIG. 3 illustrates a detailed process flow of the purchase
and set-up step 42 of FIG. 2. In step 52, a client company uses an
Internet enabled computer 15 to access web pages of the system 1
through the Internet 2. The web pages are generated by a
conventional database web page generating engine such as PHP
(Hypertext Preprocessor) in conjunction with a relational database
program stored in the program storage 20 and the web engine is
executed by the processor 18. The Internet enabled computer 15 is
equipped with a web browser capable of handling forms.
[0038] In step 54, order information such as the client company's
address and contact information of various hiring managers working
for the client company are filled out in the web page form that was
generated by the computer 10. In step 56, purchase information such
as the number of reports purchased and credit card data are also
entered through the web page. The data entered by the client
company are stored in the data storage 22. In step 58, the credit
card information provided by the client company is verified and in
step 60 a client record is created in a client database in the
storage 22 with the contact and purchase information. The client
record includes an allocation of reports to specific hiring
managers and the user id and password for each hiring manager. In
step 62, the computer 10 generates a confirmation message
confirming the number of reports purchased and the set-up of the
client company in the system 1.
[0039] As can be appreciated by persons of ordinary skill in the
art, the above steps 52-62 are optional and can be omitted by using
a billing arrangement where the client company is billed on a
periodic basis for the candidate evaluation services that have been
rendered.
[0040] FIG. 4 illustrates a detailed process flow of the candidate
and reference identification step 44 of FIG. 2. When a hiring
manager needs to fill a particular position, the manager accesses
the computer 10 through a web browser. In step 66, the hiring
manager designates a survey that is to be used for that position.
The manager can choose from a set of pre-designed or pre-selected
surveys stored in the data storage 22, design his own by selecting
survey questions from an existing set of questions stored in the
data storage, create his own set of questions, or modify an
existing survey. The questions are stored in a master table in data
storage 22. Each survey also has a corresponding record in the
database, which points to the questions in the master table that
are included for that survey. Each survey question relates to a
specific job-related and validated competency, selected from a bank
of competencies that have been derived, tested and validated from
experience and research.
[0041] FIGS. 8A-8B illustrate a sample survey that is used for a
management position. The survey of FIG. 8A contains 16 questions
that relate to various competencies that are known to be important
for a management position. For example, the fist three questions
relate to a competency known as "Managing the Business". Each of
the 16 questions requires the reference to select a value of "1"
through "7" by clicking on an appropriate radio button. The survey
also contains two comment boxes as shown in FIG. 8B. It includes
one for describing the candidate's strengths and one for describing
the candidate's weaknesses.
[0042] Once a particular survey is selected or created, the hiring
manager enters the job candidates' information in step 68 through
the computer 15 and sets the required minimum number of references
that must be provided by each job candidate. The entered
information is stored by the computer 10 in the candidate database
34 of the data storage 22. In step 68, the computer 10 also
generates a unique 16 character alphanumeric identifier for that
job candidate which is also stored in the candidate database 34.
The alphanumeric identifiers are used for security purposes since
they ensure that only known and authorized job candidates can enter
or access the information in the system 1.
[0043] In step 70, the computer dynamically generates a web page
asking whether the reference information will be provided by the
job candidate. If the hiring manager answers yes, the computer 10
in step 72 generates and send an email message to the job candidate
with a URL link to a dynamically generated web page and requesting
the job candidate to click on the link to provide information on
the references he chooses. A sample email to a job candidate is
shown in FIG. 9.
[0044] In step 74, the job candidate receives the email and
accesses the web page generated by the computer 10 by clicking on
the link provided in the email. In response, the computer transmits
through the Internet 2 a sample of the survey questions for display
on the candidate's computer 15 along with a dynamically generated
web page form to provide information on the references. The sample
survey questions assist the candidate in determining which
individuals would be appropriate references. In step 76, the job
candidate enters via the web page form the names, email addresses
and relationship of the references. The relationship field only
allows "Business" or "Professional" as "Personal" references tend
to give scores that are severely skewed towards the positive, and
may not have specific knowledge about the job-related competencies
of the candidate. The candidate also indicates the dates and
location of the relevant employment. The candidate then submits the
form to the computer 10.
[0045] The collection module 28 of the system 10 then verifies that
each email address is in a valid format and that there are no
duplications. As part of the validity check, the collection module
28 checks the domain portion of each email address against the
registered location using industry standard databases (WHOIS) to
provide the hiring manager additional information if needed. Once
the candidate is determined to have submitted a valid list, the
computer 10 stores the data on references in the candidate database
34.
[0046] If the answer to step 70 is no, however, then the hiring
manager already has the information of references. That information
is entered by the hiring manager in step 78. The same type of data
checking that are performed in step 76 is also performed in step 78
to ensure that no errors are made.
[0047] FIG. 5 illustrates a detailed process flow of the collection
step of FIG. 2. In step 80, the computer 10 generates a unique
identifier for each reference and send an email message to each
reference explaining the purpose of the email and directing the
reference to click on a URL link to a dynamically generated web
page. The unique identifier is generally used internally to
uniquely identify the reference within the system 1. A sample email
to each reference is shown in FIG. 10. The sample email contains a
statement that the operator of the system 1 will maintain strict
confidentiality of responses provided by the references and that
their responses will be aggregated and analyzed so that all of the
information generated in a report for the hiring manager will be
confidential. This statement is important because it encourages the
references to provide more honest responses.
[0048] In step 82, the email is received by the computer 15 of the
reference. In step 84, the reference accesses the web page
generated by the computer 10 by clicking on the link provided in
the email. In response, the computer transmits through the Internet
2 a dynamically generated web page form for display on the
reference's computer 15 along with instructions on how to properly
complete the form, a sample of which is shown in FIGS. 8A-8B.
[0049] In step 86, the job candidate enters via the web page form
answers to the questions in the survey. For each question, the
reference indicates the level of competency possessed by the job
candidate using a seven-point scale. The reference is also shown
the employment information submitted by the job candidate and is
asked to confirm whether the information is accurate. The candidate
then submits the completed form to the computer 10 in step 88.
[0050] In step 90, the collection module 28 of the system 10 stores
the survey data in the survey database 38. In step 92, the
collection module 28 determines whether there is a sufficient
number of completed surveys to provide a meaningful report to the
hiring manager. For example, in one case that requires seven
references, four references might be considered to provide a
meaningful report. If no, then the collection module 28 waits for
additional surveys to be completed. If yes, however, step 94 is
executed.
[0051] Alternatively, the hiring manager has three additional
options at this stage. The first option is to override the minimum
number of completed surveys and to request an interim candidate
report reports regardless. A second option is to set a
predetermined time period from the job candidate identifies the
references and checking to see whether the predetermined time
period has passed. If it has, then step 94 is executed. The third
option is to simply allow the hiring manger to close the job
candidate's record. That option may be convenient in situations
such as the job candidate voluntarily withdrawing from the job
opening.
[0052] In step 94, an email to the hiring manager is generated to
let him know that at least an interim report is available, which
email is received by him in step 98. In step 96, reminder emails
are sent to those references that have not provided the survey data
within a predetermined time period.
[0053] FIG. 6 illustrates a detailed process flow of the analysis
step of FIG. 2. In step 100, the hiring manager accesses the web
page generated by the computer 10 by either clicking on the link
provided in the email or logging on to the website of the computer
10 independent of the email. At this stage, the analysis module 30
displays a "dashboard" for the current status of the job opening.
The status includes the state of each candidate's progress for the
completion of the reference information. In step 102, a request for
report on a particular candidate is made.
[0054] In response, the analysis module 30 in step 104 analyzes the
received survey data stored in the survey database 38 and generates
a candidate report. A sample report is shown in FIGS. 11A-11D.
[0055] As can be seen in FIG. 11A, the report includes a list of
references, email addresses, identification of company and relation
to the job candidate, dates worked by the candidate and whether the
survey was completed. The report also notes any changes or
discrepancies between the information provided by the candidate and
the reference. For example, reference named "Roger Brown" reported
that the dates worked by the candidate of "02/00 To 02/03" is
different from "01/00 To 02/03". FIG. 11B explains the score for
each competency in FIG. 11C which is based on benchmark scores that
are stored in the benchmark database 36. The benchmark scores
represent competencies that are stored on an industry-wide basis,
company-wide basis or company-specific job position type basis.
[0056] As seen in FIG. 11C, scores for each question are averaged
and converted to "very low" to a "very high" score. The scores to
questions that are related to the same competency are averaged into
the same "very low" to a "very high" score. For example, the
average scores for questions 1-3 have been converted to a "Medium",
"High" and "Medium", respectively. Also, the three questions are
grouped into the competency of "Managing the business" and the
average scores for each of questions 1-3 are averages and converted
to the score of "Medium".
[0057] One third of the questions where the candidate has received
the lowest raw numerical averages are weak areas the analysis
module has identified and are indicated using asterisks which are
used as the basis for generating interview probe questions and
coaching tactics as discussed below in step 106. Alternatively, the
weak areas are identified by comparing an averaged score for each
question against a benchmark score from the benchmark database 36
and those scores that fall below the benchmark score by a
predetermined amount are identified as the weak areas and are
indicated as such using asterisks.
[0058] To further make the report useful, it may contain three
additional columns: Company Comparison, Industry Comparison and
Overall Rating. The Company Comparison and Industry Comparison
correspond to the benchmark scores on company-wide job type basis,
and industry-wide job type basis. The Overall Rating is derived
from taking the average of the raw scores from each normalized
database for that job type for the company, industry or other
organization and the average of responses from the candidate's
reference providers. This score is converted to a verbal descriptor
from very-low to very-high based on lookup table for that value.
For example a 4.2-5.9 inclusive, could return a "High", each
descriptor range can be set based on selection standards.
[0059] As shown in FIG. 11C, questions 3, 8, 11 and 16 are
highlighted using a rectangular box. The highlighted questions mean
that those questions are most closely associated with high
performance and retention of job candidates that have been hired
which are based on a statistical analysis of performance data of
the hired candidates as will be explained in detail with reference
to FIG. 7.
[0060] The report also contains an overall average score ("High" in
FIG. 11C) which is an average of scores from all the references for
all of the questions. Alternatively, the analysis module 30 can use
the correlation data from the continuous update step 50 of FIG. 2
and generate an overall score on a weighted scale in which the
weight used for each question or competency is based on the
correlation to the performance data with higher weight being used
for higher correlation.
[0061] FIG. 11D contains the "Strengths" and "Could Improve"
comments provided by the references. It is important to note that
the candidate report maintains strict confidentiality of responses
provided by the references. In other words, the candidate report
decouples the reference individuals from the responses the
reference individuals provide so as to provide anonymity of the
reference individuals from the hiring company/hiring manager. This
is important as it encourages the references to provide more candid
responses.
[0062] FIG. 14 illustrates a group report which ranks multiple
candidates. For each candidate, the group report includes averaged
scores for each question, averaged score for each competency, and
an overall summary score. The group report also includes questions
3, 8 and 16 which are highlighted using a rectangular box. The
highlighted questions mean that those questions are most closely
associated with high performance and retention of job candidates
that have been hired which are based on a statistical analysis of
performance data of the hired candidates.
[0063] In step 106, the hiring manager, after reviewing the report,
determines whether to continue with the hiring process for the
candidate. If the answer is no, then the analysis step 48 ends at
step 111. If the answer is yes, the analysis step 48 continues with
step 108. In step 108, the analysis module generates interview
questions (see FIGS. 12A-12B) based on the identified weak areas in
the report. Specifically, the interview questions are associated
with the questions in the survey and are stored in the data storage
22. For each weak area, the analysis module retrieves those
interview questions that are associated with the questions that
correspond to the weak areas.
[0064] For example, the report in FIG. 11C has identified questions
1, 2, 7, 9, 12 and 13 as the weak areas. For those questions, the
analysis module 30 retrieves associated interview questions from
the data storage 22 as shown in FIGS. 12A-12B.
[0065] In step 110, the hiring manager, after having interviewed
the candidate, determines whether to hire the candidate. If the
decision is no, then the analysis step 48 ends. If the decision is
a yes, then the analysis module in step 112 generates coaching
suggestions that allow the employer to improve the identified weak
areas after the candidate is hired. Like the interview questions,
the coaching suggestions are associated with the questions in the
survey and are stored in the data storage 22. For each weak area,
the analysis module 30 retrieves those coaching suggestions that
are associated with the questions that correspond to the weak areas
as shown in FIGS. 13A-13C.
[0066] It is important to note that the steps of survey collection
46 and analysis 48 can be used in an iterative process to screen
out job candidates. For example, a hiring manager might use a
simple survey containing 4 questions against 100 job candidates to
narrow the list down to 10 candidates, then design a new survey
containing 16 questions to narrow the 10 candidates down to 3
candidates, and then design another survey containing 20 questions
to select one candidate for hire. Accordingly, the present
invention can be used to both as a screening tool and a selection
tool.
[0067] FIG. 7 illustrates a detailed process flow of the continuous
update step of FIG. 2 which is also part of the analysis module 30.
In step 120, the computer 10 waits for a predetermined time period
after the final candidate report was generated. In the embodiment
shown, the predetermined time period is one month. In step 122, the
computer 10 prepares and sends an email to the hiring manager with
a URL link to a dynamically generated web page. The web page asks
whether a particular job candidate is hired. Alternatively, the
email may include two links asking the hiring manager to click on
one link if the candidate was hired and to click on the other link
is the candidate was not hired.
[0068] In step 124, the computer 10 receives the response of the
hiring manager and determines whether the job candidate was hired.
If no, then that fact is noted and stored in the candidate database
34 in step 126 for later analysis and reporting. If the candidate
was hired, then control passes to step 128. At step 128, the
computer 10 waits for a predetermined time period after the final
candidate report was generated. In the embodiment shown, the
predetermined time period is one year from the final report. In
step 130, the computer 10 prepares and sends an email to the hiring
manager with a URL link to a dynamically generated survey web page.
The survey web page asks retention and performance information. In
the embodiment shown, the survey web page asks two questions: (1)
is the candidate still employed; and (2) how well the candidate has
performed based on a survey containing multiple questions or based
on a single question on a predefined scale, e.g., scale of 1-10. In
step 126, the response to the two questions from the hiring manager
are stored in the candidate database 34 for later analysis and
reporting.
[0069] In step 132, the analysis module 30 analyzes the hiring data
stored in the candidate database 34. Specifically, the retention
and performance data are statistically correlated with the various
scores received by that candidate in the surveys to identify the
questions where high ratings are most closely associated with high
performance and retention. The correlation can be calculated on an
industry-wide position type basis, on a company-wide basis without
regard to the position type or on a company-wide position type
basis. Thus, when the candidate reports such as shown in FIG. 11C
are generated, the questions where high ratings are most closely
associated with high performance and retention are graphically
indicated based on the latest data accumulated up to that
point.
[0070] FIG. 16 illustrates a sample report that shows the
correlation between survey questions/competencies and performance
of hired candidates. As shown, the report includes raw correlation
scores and corresponding ratings. For example, question 3 has a raw
score of 4.7 and a "Very High" rating.
[0071] These reports can be used by the hiring manager to
continuously improve the survey. For example, the hiring manager
can choose to delete the two questions that have the lowest
correlation to the performance data and add two new questions from
the competency area that has the highest correlation to the
performance data. In the example shown, the two lowest scoring
questions are questions 1 and 2, and the competency area having the
highest score of 4.66 is "Teamwork".
[0072] Once step 132 is executed, control passes to step 128 where
the computer waits a programmable amount of time usually 6 to 15
months to repeat the steps 130 to 132 to continuously monitor the
performance of the job candidates that have been hired in order to
continuously improve the survey questions and competency
categories.
[0073] Application of the principles of the present invention are
many. For example, principles of the survey selection, collection
of responses and analysis of the responses can be used to evaluate
a large number of vendors who supply products and services to a
company through a group of buyers working for the company. The
buyers for the company are acting as "reference providers". A
sample vendor report as shown in FIGS. 15A-15B can be used to
better manage the large number of vendors.
[0074] The foregoing specific embodiments represent just some of
the ways of practicing the present invention. Many other
embodiments are possible within the spirit of the invention.
Accordingly, the scope of the invention is not limited to the
foregoing specification, but instead is given by the appended
claims along with their full range of equivalents.
* * * * *