U.S. patent application number 16/196022 was filed with the patent office on 2019-10-24 for method, system, and apparatus for validation.
This patent application is currently assigned to Nuance Communications, Inc.. The applicant listed for this patent is Nuance Communications, Inc.. Invention is credited to Keith W. Boone, Sunitha Chaparala, Cameron Fordyce, Sean Gervais, Jeffrey G. Hopkins, Roubik Manoukian, Harry J. Ogrinc, Robert G. Titemore.
Application Number | 20190325024 16/196022 |
Document ID | / |
Family ID | 33451464 |
Filed Date | 2019-10-24 |
United States Patent
Application |
20190325024 |
Kind Code |
A1 |
Boone; Keith W. ; et
al. |
October 24, 2019 |
Method, System, and Apparatus for Validation
Abstract
In a method for validating data, a text of a document is
received. At least one fact is extracted from the text. At least
one expert refinement is merged with the at least one fact to
create at least one modified fact. The at least one modified fact
is provided for a review. An expert refinement to the at least one
modified fact is captured in response to the review. A superset
document based on the at least one pre-existing refinement and the
expert refinement is stored.
Inventors: |
Boone; Keith W.; (Randolph,
MA) ; Chaparala; Sunitha; (South Weymouth, MA)
; Gervais; Sean; (Dorchester, MA) ; Titemore;
Robert G.; (Lexington, MA) ; Ogrinc; Harry J.;
(Medfield, MA) ; Hopkins; Jeffrey G.; (Lincoln,
RI) ; Manoukian; Roubik; (Belmont, MA) ;
Fordyce; Cameron; (Providence, RI) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Nuance Communications, Inc. |
Burlington |
MA |
US |
|
|
Assignee: |
Nuance Communications, Inc.
Burlington
MA
|
Family ID: |
33451464 |
Appl. No.: |
16/196022 |
Filed: |
November 20, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13345123 |
Jan 6, 2012 |
10133726 |
|
|
16196022 |
|
|
|
|
13313718 |
Dec 7, 2011 |
10127223 |
|
|
13345123 |
|
|
|
|
10448317 |
May 30, 2003 |
8095544 |
|
|
13313718 |
|
|
|
|
10448317 |
May 30, 2003 |
8095544 |
|
|
13345123 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 40/30 20200101;
G16H 10/60 20180101; G06F 40/226 20200101; G06Q 50/24 20130101;
G06F 19/00 20130101; G06F 40/56 20200101 |
International
Class: |
G06F 17/27 20060101
G06F017/27; G06F 17/28 20060101 G06F017/28; G16H 10/60 20060101
G16H010/60 |
Claims
1-28. (canceled)
29. A method comprising: extracting from an electronic text
document, using at least one processor, at least a first medical
fact of a patient, the electronic text document containing text
that is a transcription of a clinician's dictations regarding an
encounter between the clinician and the patient; extracting from
the electronic text document, using the at least one processor, a
time associated with the first medical fact; determining a temporal
significance of the first medical fact, based at least in part on
the time that was extracted from the electronic text document and
is associated with the first medical fact; storing the first
medical fact in a record in a memory accessible by the at least one
processor; and indicating the temporal significance of the first
medical fact in the record.
30. The method of claim 29, wherein: the record includes a previous
medical fact and a time associated with the previous medical fact,
and the determining determines a temporal significance of the
previous medical fact based on the first medical fact, the time
associated with the previous medical fact, and the time associated
with the first medical fact.
31. The method of claim 30, wherein the method further comprises:
updating a relevance of the previous medical fact stored in the
record, if the first medical fact is a counter-example of the
previous medical fact.
32. The method of claim 30, further comprising: providing a
graphical user interface to enable a user to view the electronic
text document and the record, and to input refinement information,
wherein the refinement information includes any one or any
combination of: a validity of the first medical fact, a relevance
of the first medical fact, a modification to the first medical
fact, a validity of a previous medical fact stored in the record, a
relevance of the previous medical fact, and a modification to the
previous medical fact.
33. The method of claim 32, further comprising: storing the
refinement information in a refinement record in the memory, the
refinement record being stored in the memory in association with
the record.
34. The method of claim 32, wherein the graphical user interface
enables the user to indicate text in the electronic text document
to be extracted, and wherein the method further comprises:
extracting a second medical fact, corresponding to the indicated
text, and a time associated with the second medical fact, and
storing the second medical fact and the time associated with the
second medical fact in the record.
35. An apparatus comprising: at least one processor; and a memory
accessible by the at least one processor, wherein the processor is
programmed to perform a method comprising: extracting from an
electronic text document at least a first medical fact of a
patient, the electronic text document containing text that is a
transcription of a clinician's dictations regarding an encounter
between the clinician and the patient, extracting from the
electronic text document a time associated with the first medical
fact, determining a temporal significance of the first medical
fact, based at least in part on the time that was extracted from
the electronic text document and is associated with the first
medical fact, causing the first medical fact to be stored in a
record in a memory accessible by the at least one processor, and
indicating the temporal significance of the first medical fact in
the record.
36. The apparatus of claim 35, wherein: the record includes a
previous medical fact and a time associated with the previous
medical fact, and the determining determines a temporal
significance of the previous medical fact based on the first
medical fact, the time associated with the previous medical fact,
and the time associated with the first medical fact.
37. The apparatus of claim 36, wherein the method further
comprises: updating a relevance of the previous medical fact stored
in the record, if the first medical fact is a counter-example of
the previous medical fact.
38. The apparatus of claim 35, further comprising a display,
wherein the method further comprises providing a graphical user
interface to the display to enable a user to view the electronic
text document and the record, and to input refinement information,
and wherein the refinement information includes any one or any
combination of: a validity of the first medical fact, a relevance
of the first medical fact, a modification to the first medical
fact, a validity of a previous medical fact stored in the record, a
relevance of the previous medical fact, and a modification to the
previous medical fact.
39. The apparatus of claim 38, wherein the method further
comprises: storing the refinement information in a refinement
record in the memory, the refinement record being stored in the
memory in association with the record.
40. The apparatus of claim 38, wherein the graphical user interface
enables the user to indicate text in the electronic text document
to be extracted, and wherein the method further comprises:
extracting a second medical fact, corresponding to the indicated
text, and a time associated with the second medical fact, and
storing the second medical fact and the time associated with the
second medical fact in the record.
41. A non-transitory computer-readable storage medium storing a
program that, when executed by a computer, causes the computer to
perform a method comprising: extracting from an electronic text
document at least a first medical fact of a patient, the electronic
text document containing text that is a transcription of a
clinician's dictations regarding an encounter between the clinician
and the patient; extracting from the electronic text document a
time associated with the first medical fact; determining a temporal
significance of the first medical fact, based at least in part on
the time that was extracted from the electronic text document and
is associated with the first medical fact; causing the first
medical fact to be stored in a record in a memory accessible by the
computer; and indicating the temporal significance of the first
medical fact in the record.
42. The non-transitory computer-readable storage medium of claim
41, wherein: the record includes a previous medical fact and a time
associated with the previous medical fact, and the determining
determines a temporal significance of the previous medical fact
based on the first medical fact, the time associated with the
previous medical fact, and the time associated with the first
medical fact.
43. The non-transitory computer-readable storage medium of claim
42, wherein the method further comprises: updating a relevance of
the previous medical fact stored in the record, if the first
medical fact is a counter-example of the previous medical fact.
44. The non-transitory computer-readable storage medium of claim
41, wherein the method further comprises: providing a graphical
user interface to a display to enable a user to view the electronic
text document and the record, and to input refinement information,
and wherein the refinement information includes any one or any
combination of: a validity of the first medical fact, a relevance
of the first medical fact, a modification to the first medical
fact, a validity of a previous medical fact stored in the record, a
relevance of the previous medical fact, and a modification to the
previous medical fact.
45. The non-transitory computer-readable storage medium of claim
44, wherein the method further comprises: storing the refinement
information in a refinement record in the memory, the refinement
record being stored in the memory in association with the
record.
46. The non-transitory computer-readable storage medium of claim
44, wherein the graphical user interface enables the user to
indicate text in the electronic text document to be extracted, and
wherein the method further comprises: extracting a second medical
fact, corresponding to the indicated text, and a time associated
with the second medical fact, and storing the second medical fact
and the time associated with the second medical fact in the record.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application relates to co-pending U.S. patent
application Ser. No. 10/413,405, entitled, "INFORMATION CODING
SYSTEM AND METHOD", filed April 15, 2003; co-pending U.S., patent
application Ser. No. 10/XX,XXX, entitled, "SYSTEM AND METHOD FOR
UTILIZING NATURAL LANGUAGE PATIENT RECORDS", filed on May 29, 2003;
co-pending U.S. patent application Ser. No. 10/XX,XXX, entitled,
"METHOD, SYSTEM, AND APPARATUS FOR DATA REUSE", filed on May 30,
2003; and co-pending U.S. patent application Ser. No. 10/XX,XXX,
entitled, "METHOD, SYSTEM, AND APPARATUS FOR VIEWING DATA", filed
on May 30, 2003, all of which co-pending applications are hereby
incorporated by reference in their entirety.
BACKGROUND OF THE INVENTION
[0002] The present invention relates generally to validating data
from text extracted from a set of records. More specifically, the
present invention relates to capturing and applying refinements
made by a domain expert to the validity, relevance, and temporal
significance of "facts" (extractions of discreet data elements,
their location within the document, their normalizations, and their
ontological classifications) automatically extracted from
electronic text.
[0003] In the medical field, health care providers (e.g.,
physicians, medical technicians or administrators) typically
dictate diagnoses, medications and other patient medical reports in
a free form manner. These dictations are then transcribed into
documents. The transcribed documents are typically then submitted
to the provider for review and approval. The transcribed documents
will likely contain data that is relevant to different users at
different times. Additionally, many legacy databases contain
documents that include data with varying degrees of relevancy.
[0004] Automatic extraction of specified data from electronic
medical records has been known for some time. It is well known in
the art that computation algorithms may be employed to process text
of an electronic document to extract specific data from the
document. However, validating the relevancy, relevance,
classification, and temporal significance of the data has not been
possible heretofore.
[0005] Presently, users are required to manually review extracted
data in order to validate the data. The manual process requires
review of the text document, a time consuming review process in
which the user may edit and approve the text for ultimate storage
in a database where the text may be reviewed at a later time.
Manual operation may include data entry using drop down menus,
mouse pointing clicks, typing and time consuming records review. It
is therefore desirable to provide users with a validation process
that utilizes automatically extracted, relevant data items from
free form dictated and transcribed documents.
[0006] The significance of facts can change over time. A deficiency
in current systems that perform extraction is that they do not
account for the temporal significance of the fact. For example, a
problem that is relevant today may he resolved tomorrow, and thus
the fact that the problem exists is true only when the context of
the time period (today) is provided.
[0007] An additional problem exists relating to nomenclature. There
are several ways to describe many different physical ailments. More
particularly, users of such systems often use different phrases to
describe a single type of event. For example, one physician may use
`myocardial infarction` while another physician may use `heart
attack` to describe a problem for a patient. In this example, there
may be up to 25 phrases that describe the same or similar ailment
to the heart. As such, a searcher who wishes to find a group of
records that involve a particular term of art would have to know
and use of all the variants of those phrases in order to ensure a
complete search. It would be desirable to provide a grouping of
like and similar variants of key medical facts, medical concepts,
and present those in a user interface along with extractions of the
discrete data elements.
[0008] Health care providers are often responsible for maintaining
lists of current problems, medications, allergies, and procedures
for patients. Problems in this context can be anything that is
relevant to the physician or affects the care and treatment of the
patient. Facts on the current list are significant over a
particular time period, after which the problem may no longer be
relevant to the patient's treatment and care, or the patient's
problem may have been resolved, or the medication discontinued, et
cetera.
[0009] Manual processes for maintaining these lists often include
paper forms wherein the provider writes in new items on the list,
dates it, and signs or through dictation wherein the provider
dictates the actual insertions and removals, where these changes
are then made by clerical personnel at the time the dictated report
is transcribed. Automated processes found in electronic medical
record systems require data entry of the items on the current
list.
[0010] The deficiencies inherent in manual processes are numerous.
When a paper form is used, only one copy of it is available,
whereas when this information is stored electronically, multiple
viewers can access the information at the same time. It is
difficult to locate information on paper forms or even in
electronic documents as these storage mechanisms do not provide
sorting and filtering features that might be available when the
information is stored in a database. A further problem is that when
the provider dictates changes to the list, there are time lags
introduced by the transcription and editing process that create a
delay between the dictation of these changes and the actual
implementation of these changes on the storage media. This imposes
a delay on the availability of changes to the provider and to the
rest of the medical community providing patient care.
[0011] When current lists are maintained in electronic medical
record systems, the user must manually enter the information in the
list, rather than have the system suggest to them changes that
might be made to the current list based on extracted facts.
[0012] Finally, when current lists are maintained on forms, through
dictated changes, or even in electronic medical records, the
context in which the problem, medication, allergy, or procedure
mentioned for the patient is not available. Therefore, the only
information available to the medical community is the item on the
current list, without more detailed context that might provide for
better medical care.
[0013] Thus, present systems do not have the ability to integrate
information in real time to a current lists report and cannot
provide context for that information. It is desirable to provide a
system that presents discrete data elements for approval in real
time by a user with the ability to determine the context of a
report, namely, the creation point of the report, the creator, the
time frame and the relevance of the discrete element for
extraction.
OBJECTS OF THE INVENTION
[0014] In light of the above-identified prior art deficiencies, it
is an object of the present invention to provide a system and
method to validate a freeform text document for certain facts as
true or relevant to a case before they are stored in a database and
marked as such.
[0015] It is another object of the present invention to provide a
system and method by which a user may approve or validate extracted
data prior to sending it to the database for a subsequent retrieval
and viewing inquiry.
[0016] It is still another object of the present invention to
provide a system and method for validating extracted data
applicable to third party systems, such as a hospital information
system or an EMR.
[0017] It is another object of the present invention to provide a
system and method for validating extracted data and maintaining a
current list.
[0018] It is another object of the present invention to provide a
system and method for validating extracted data and maintaining a
current list indexed and searchable by multiple degrees, namely, to
determine the status of a record as of a specified date.
[0019] It is another object of the present invention to provide a
system and method for validating extracted data where a user may
review specific extracted data elements to further refine the
extracted information.
[0020] It is another object of the present invention to provide a
system and method for validating extracted data and maintaining a
current list by carrying forward the information pre-determined as
relevant or true until a user specified change.
[0021] It is another object of the present invention to capture
information about the time that a fact was observed or reported
upon, and/or the time that a counter-example to the fact was
observed or reported upon, in order to maintain information about
the temporal significance of said fact.
SUMMARY OF THE INVENTION
[0022] An advantage exists in the present invention, which
facilitates the determination of validity, relevance,
classification, and temporal significance of facts, automatically
extracted from electronic text for capturing and applying
refinements made by a domain expert.
[0023] In a first aspect, the present invention includes a method
of reviewing data. The method includes receiving the text of a
document and at least one fact, capturing an expert refinement to
the at least one fact in response to the review, and storing a
superset document based on the at least one pre-existing fact and
the expert refinement. The method may also include the at least one
fact from the text being subsequently merged with a previously
stored expert refinement to produce at least one modified fact and
the capturing of expert refinements is applied to modified facts.
The receiving of the text of the document may also include
receiving the document by one of electronic mail, file transport
protocol, and a network file transfer protocol. The providing of
the review document for the review may also include providing a
graphical user interface adapted to display the at least one
modified fact and highlighting a selected fact displayed on the
graphical user interface. The method may also include displaying at
least one category of facts, the selected fact being a member of
the at least one category of facts, displaying a related details
category for the selected fact, and displaying the selected text
and surrounding text (i.e., the context) of the selected fact in
the graphical user interface. The method may also include
displaying a relevancy indicator for each fact in the at least one
category of facts. The method may also include displaying a
truthfulness indicator for each fact in the at least one category
of facts. The method may also include providing the at least one
modified fact and the text to a domain expert and determining the
expert refinement based on a review of the at least one modified
fact and the at least one expert refinement by the domain expert.
The method may also include storing the expert refinement as an
expert refinement file, collecting a set of related documents based
on an index, extracting the at least one fact based on the set of
related documents, and providing the at least one fact to a domain
expert. The related documents may be of similar date, topic or
clustered by similar content using any number of document
clustering and classification algorithms well known to those
practiced in the art (e.g., K-nearest neighbor algorithm, or cosine
similarity metric). The method may also include determining a set
of normalized facts based on the at least one fact, for example, by
classifying facts to a taxonomy such as SNOMED or to the ICD-9-CM,
or CPT, or other such taxonomy, not necessarily limited to the
medical domain. The method may also include providing the set of
normalized facts with the at least one modified fact for the
review, and determining a temporal significance for the at least
one modified fact, for example by recording the date the fact was
observed based on metadata included with the medical record. The
method may also include determining a relevancy factor for the at
least one modified fact and providing the relevancy factor with the
at least one modified fact for the review.
[0024] In a second aspect, the present invention includes a system
for validation. The system includes an extraction module configured
to extract a set of facts from a captured electronic document, a
storage device configured to interface with the extraction module
and the validation module, and a validation module configured to
provide a graphical user interface to validate the facts, wherein
the validation module is configured to receive a set of facts from
the extraction module, apply a set of expert facts retrieved from
storage device to the set of facts to create a set of modified
facts, and provide the set of modified facts to an author for
review. The validation module may be further configured to
determine a set of normalized facts for the set of facts. The
validation module may be further configured to determine a temporal
significance for the set of facts. The validation module may be
further configured to determine a relevancy factor for the set of
facts. The validation module may be further configured to provide
at least one of a set of normalized facts, a temporal significance,
and a relevancy factor with the set of facts to a domain expert.
The validation module may be further configured to capture
modifications to the set of facts as the set of expert facts based
on a review of the at least one of the set of normalized facts, the
temporal significance, and relevancy factor with the set of facts
by the domain expert. The validation module may be further
configured to store the set of expert facts.
[0025] The above advantages and features are of representative
embodiments only, and are presented only to assist in understanding
the invention. It should be understood that they are not to be
considered limitations on the invention as defined by the claims,
or limitations on equivalents to the claims. Additional features
and advantages of the invention will become apparent from the
drawings, the following description, and the claims.
BRIEF DESCRIPTION OF THE DRAWINGS
[0026] While the specification concludes with claims particularly
pointing out and distinctly claiming the present invention, it is
believed the same will be better understood from the following
description taken in conjunction with the accompanying drawings,
which illustrate, in a non-limiting fashion, the best mode
presently contemplated for carrying out the present invention, and
in which like reference numerals designate like parts throughout
the figures, wherein:
[0027] FIG. 1 illustrates an exemplary architecture of a validation
client module in accordance with an embodiment of the present
invention;
[0028] FIG. 2 illustrates an exemplary flow diagram for the
validation client module in accordance with another embodiment of
the present invention;
[0029] PIG. 3 illustrates a natural language patient record (NLPR)
system utilizing the validation client module shown in FIG. 1 in
accordance with yet another embodiment of the present
invention;
[0030] FIG. 4 illustrates a validation viewer GUI provided by the
validation client module in accordance with yet another embodiment
of the present invention;
[0031] FIG. 5 illustrates the target viewer component in greater
detail in accordance with yet another embodiment of the present
invention;
[0032] FIG. 5A illustrates an expanded view of a current list
included in the target viewer component in accordance with another
embodiment of the present invention;
[0033] FIG. 6 illustrates a more detailed view of the record viewer
component in accordance with yet another embodiment of the present
invention;
[0034] FIG. 7 illustrates a more detailed view of the extractions
viewer component in accordance with yet another embodiment of the
present invention;
[0035] FIG. 8 illustrates a more detailed flow diagram for
validating extractions for the validation viewer GUI (shown in
FIGS. 4-7) in accordance with yet another embodiment;
[0036] FIG. 9 illustrates a more detailed flow diagram for
validating extractions for the validation viewer GUI (shown in
FIGS. 4-7) in accordance with yet another embodiment of the present
invention; and
[0037] FIG. 10 illustrates an exemplary block diagram of a computer
system where an embodiment of the present invention may be
practiced.
DETAILED DESCRIPTION OF TIME EMBODIMENTS
[0038] For simplicity and illustrative purposes, the principles of
the present invention are described by referring mainly to
exemplary embodiments thereof. However, one of ordinary skill in
the art would readily recognize that the same principles are
equally applicable to, and can be implemented in, all types of
network systems, and that any such variations do not depart from
the true spirit and scope of the present invention. Moreover, in
the following detailed description, references are made to the
accompanying figures, which illustrate specific embodiment
Electrical, mechanical, logical and structural changes may be made
to the embodiments without departing from the spirit and scope of
the present invention. The following detailed description is,
therefore, not to be taken in a limiting sense and the scope of the
present invention is defined by the appended claims and their
equivalents.
[0039] Embodiments relate to validating data extracted from a
document. In one embodiment, a host application instantiates a
validation client module and forwards a document to the validation
client module. The validation client module is configured to
capture a document. The document may be in an electronic format
such as commercial word processing file, ASCII, mark-up language,
or other similar format. The validation client module is also
configured to extract a set of discrete data elements (e.g., facts,
keywords, or other similar data) from the captured electronic text.
It will be understood by those skilled in the art that the present
invention can be applied to freeform dictated documents as well as
to any electronic text, free narrative or otherwise.
[0040] More particularly, the validation module may use parsing
engines to parse for relevant facts within the captured electronic
text. The validation client module may be further configured to
merge a previously determined set of validated facts to the
extracted set of facts as a preliminary set of facts. The
validation client module may be further configured to normalize the
extracted facts, determine the temporal significance for the
preliminary set of facts, and/or to determine the relevance of the
modified set of facts as preliminary metadata. The validation
client module may record the time that a fact or its
counter-example was observed or reported upon in order to determine
the temporal significance of said fact.
[0041] The validation client module may be further configured to
provide the preliminary metadata, the preliminary set of facts, and
the text of the document to a domain expert for review. The
validation client module may then receive expert refinements, i.e.,
changes, based on a review of the preliminary metadata, the
preliminary set of facts, and the text of the document. The
validation client module may then be configured to store the
changes to the preliminary metadata and set of facts as expert
refinements. The expert refinements are associated with the
document and returned to the host application. In one embodiment,
the validation client module may be configured to maintain a delta
file that captures the changes that occurred during the review of
the preliminary metadata and the preliminary set of facts. The
validation client module may be further configured to maintain and
permanently store the delta files for each document. In another
embodiment, the validation client module may be configured to
provide preliminary metadata and the preliminary set of facts on a
set of related documents. The set of related documents may be
related chronologically, subject, or other similar indexing key.
The validation client module then accepts expert refinements based
on the review of the domain expert for the set of related
documents.
[0042] Accordingly, the validation client module may provide a
mechanism for a user to quickly evaluate and validate facts from a
document. By associating the validated facts with a document, the
search capability for the document may be increased. More
specifically, the validated facts may become search terms for the
document and thus increase the precision of the search.
[0043] FIG. 1 illustrates an exemplary architecture of a validation
client module 100 in accordance with an embodiment. It should be
readily apparent to those of ordinary skill in the art that the
exemplary architecture depicted in FIG. 1 represents a generalized
schematic illustration and that other elements may be added or
existing elements may be removed or modified.
[0044] As shown in FIG. 1, the validation client module 100
includes a validation module 110, an input/output (I/O) module 120
(labeled as `I/O module` in FIG. 1), an extraction module 130, and
a storage interface module 140. The validation module 110 may be
configured to provide the functionality for the validation client
module 100. For example, the validation module 110 may invoke the
I/O module 120 to provide for a validation graphical user interface
(GUI) in response to initiating the validation client module 100.
As another example, the validation module 110 may invoke the
extraction module 130 to extract at least one fact from a selected
document. As yet another example, the validation module 110 may
merge extracted facts with a set of facts extracted from a previous
version and/or group of documents. The validation module 110 may
also determine similar terms for a selected fact, i.e., normalize
the selected fact.
[0045] The I/O module 120 may be configured to provide a mechanism
for a user to communicate with the validation client module 100.
For example, the I/O module 120 may be invoked to provide a GUI for
a domain expert to review extracted facts. The 110 module 120 may
also provide another GUI to receive revisions to extracted
facts.
[0046] The extraction module 130 may be configured to extract facts
from a selected document when invoked by the validation module 110.
The extraction module 130 may be implemented by conventional
extraction software (e.g., those implemented by applying a
collection of regular expressions to a document). The extraction
module 130 may return the extracted facts to the validation module
110.
[0047] The storage interface module 140 may be configured to
provide access to storage devices by the validation module 110. The
storage interface module 140 may retrieve and store previous
validated facts for a document (or group of documents),
normalization data for facts, categorization data for facts,
versions of the validated facts for a selected document, etc., for
the validation module 110. The storage interface module 140 may be
implemented as a physical drive interface (e.g., IDE, SCSI,
IEEE1394, etc.), a device driver library or other similar
interfacing technique.
[0048] Accordingly, the validation client module 100 may he adapted
to he invoked by a host application. The validation client module
100 may be configured to receive a document or a pointer to the
document from the host application. The validation module 110 may
be configured to invoke the extraction module 130 to extract facts
from the document. The extraction module 130 may be configured to
return the extracted facts, to the validation module 110.
[0049] The validation module 110 may he configured to retrieve
previous expert refinements, if any, through the storage interface
module 140. The validation module 110 combines the current facts
with any previous expert refinements to create a preliminary set of
facts. The validation module 110 may then invoke the 110 module to
provide for a graphical user interface (GUT) that displays the
preliminary set of facts, the text of the current document and the
preliminary metadata. The validation module 110 may be further
configured to capture any changes implemented by a domain expert,
i.e., a user with proper authority, on the GUI, as an expert
refinement file. The validation module 110 may be further
configured to maintain delta file of the changes made by the domain
expert.
[0050] The validation module 110 is configured to associate the
expert refinement file with the document and return the files (by
copy or link) to the host application. The validation module 110
may he further configured to store the expert refinement file and
delta file by passing the files to the storage interface module
140. Accordingly, the validation module 110 may retrieve the expert
refinement file to perform validation on new versions of the
document.
[0051] It should be readily apparent to those skilled in the art
that the individual functions, as described above and in further
details below, embodied by the respective I/O module 120,
extraction module 130, and storage interface module 140 may be
performed by the validation module 110. Conversely, the individual
functions, as described above and in further details below, of the
validation module 110 may be moved to the I/O module 120,
extraction module 130, and storage interface module 140.
[0052] The validation client module 100 may be implemented as a
software program, a utility, a subroutine, or other similar
programming entity. In this respect, the validation client module
100 may be implemented using software languages such as C, C++,
JAVA, etc. Alternatively, the validation client module 100 may be
implemented as an electronic device utilizing an
application-specific integrated circuit, discrete components,
solid-state components or a combination thereof.
[0053] FIG. 2 illustrates an exemplary flow diagram 200 for the
validation client module 100 in accordance with another embodiment.
It should be readily apparent to those of ordinary skill in the art
that this method 200 represents a generalized illustration and that
other steps may be added or existing steps may be removed or
modified.
[0054] As shown in FIG. 2, the validation client module 100 may be
invoked by a host application (not shown), in step 205. For
example, the host application may receive activation of a menu item
that represents the validation client module 100, perform a
function call to the validation client module 100, or a user may
execute a command line to instantiate the validation client module
100. Alternatively, the validation client module 100 may be a
standalone application program.
[0055] In step 210, the validation module 110 may invoke the
extraction module 130 to extract facts from a selected document.
The document or a pointer to the document may have passed to the
validation module 110 when the validation client 100 was invoked.
The extraction module 130 may utilize a conventional extraction
module to extract the facts (or keywords, concepts, etc.) from the
selected document. The extraction module 130 may be configured to
return the extracted facts to the validation module 110.
[0056] In step 215, the validation module 110 may invoke the i./0
module 120 to provide a validation viewer GUI (not shown). The
validation viewer may provide a mechanism to review the extracted
facts along with access to previous validated facts. The validation
viewer GUI may comprise a target viewer component, a record viewer
component, and an extraction viewer component. The target viewer
component may present the extracted facts into target groups (e.g.,
Problems, Medications, Allergies). The extraction viewer GUI
presents an extracted fact in the context of a single line of the
report. This enables an authorized user to quickly determine
whether or not the selected fact is valid. The record viewer
displays the location of a selected fact within the document in
response to the fact being selected.
[0057] In step 220, the I/O module 120 may detect a change in the
facts on the validation viewer GUI. If the change to the fact is
validation of an extracted fact, the change is updated to the list
of validated facts in step 225. Otherwise, if the I/O module 120
does not detect a change in the facts, the validation module would
proceed to step 235.
[0058] In step 230, the validation module 110 may determine whether
there is a change to the facts. For example, a user may add a fact
by `swiping` a portion of the text of the document, i.e.,
highlighting the selected fact. If the validation module 110
determines that there has been a change to the extracted facts, the
validation module 110 may determine whether or not a new extraction
is needed in step 240.
[0059] If the validation module 110 determines that a new
extraction is needed, the validation module 110 may be configured
to call the extraction module 130 to receive the extracted facts to
perform the processing in step 230. Otherwise, the validation
module 110 may validate the extracted facts, in step 245.
[0060] Returning to step 235, if the validation module 110
determines that there is no change to the extracted facts, the
validation module 110 may determine whether or not a change to a
current list.
[0061] If the validation module 110 determines a change in the
current list, the validation module 110 may be configured to update
the current list with the latest change in step 255.
[0062] Otherwise, in step 260, the validation module 110 may
determine whether or not to save the changes implemented by the
user. If the validation module 110 determines that data is to be
saved, the validation module 110 may create a revision file, which
is passed onto to a storage device through the storage interface
module 140. The revision file may be comprised of the original
document, facts made by the software, and changes to the validation
status of those facts, changes to the current list, and/or changes
made to the temporal status of a fact made during the validation
steps described above. The revision file may be used to update
later facts. Otherwise, if the validation module 110 determines not
to save the changes, the validation module 110 may determine
whether or not the user has completed the validation process in
step 270. If the user has not completed the validation process, the
validation module 110 may return to the processing of step 220.
Otherwise, the validation module may invoke the I/O module 120 to
close the validation viewer GUI, in step 275.
[0063] FIG. 3 illustrates a natural language patient record (NLPR)
system 300 utilizing a validation client module in accordance with
yet another embodiment. As shown in FIG. 3, the NLPR system 300
includes a plurality of workstations 305 interconnected by a
network 310. The NLPR system 300 also includes a server 315
executing a computer readable version 32(3 of the NLPR system and
data storage 325. The NLPR system 300 is a system for maintaining
electronic medical records of patients, which is described in
greater detail in co-pending U.S. patent application Ser. No.
10/XX,XXX, entitled, "SYSTEM AND METHOD FOR UTILIZING NATURAL
LANGUAGE PATIENT RECORDS," filed May 29, 2003, which has been
incorporated by reference in its entirety.
[0064] The workstations 305 may be personal computers, laptops, or
other similar computing element. The workstations 305 execute a
physician workstation (PWS) client 330 from the NLPR system 300.
The PWS client 325 provides the capability for a physician to
dictate, review, and/or edit medical records in the NLPR system
300. While FIG. 3 is described in the realm of the medical field,
it will be understood by those skilled in the art that the present
invention can be applied to other fields of endeavor where users
dictate, review and edit records in any domain.
[0065] The workstations 305 also execute a transcriptionist client
335 for a transcriptionist to access and convert audio files into
electronic text. The NLPR system 300 may also use speech
recognition engines to automatically convert dictations from
dictators into electronic text.
[0066] The network 310 is configured to provide a communication
channel between the workstations 305 and the server 315. The
network 310 may be a wide area network, local area network or
combination thereof. The network 310 may implement wired protocols
(e.g., TCP/IP, X.25, IEEE802.3, IEEE802.5, etc.), wireless
protocols (e.g., IEEE802.11, CDPD, etc.) or combination
thereof.
[0067] The server 315 may be a computing device capable of
providing services to the workstations 305. The server 315 may be
implemented using any commonly known computing platform. The server
315 is configured to execute a computer readable version of the
NLPR software 320. The NLPR software provides functionality for the
NLPR system 300. The NLPR system 300 may receive audio files and/or
documents by other network access means such as electronic mail,
file transfer protocols, and other network transferring
protocols,
[0068] The data storage 325 may be configured to interface with
network 310 and provide storage services to the workstations 305
and the server 315. The data storage 325 may also be configured to
store a variety of files such as audio, documents, and/or
templates. In some embodiments, the data storage 325 includes a
file manager (not shown) that provide services to manage and access
the files stored therein. The data storage 325 may be implemented
as a network-attached storage or through an interface through the
server 315.
[0069] The server 315 may be further configured to interface with
an embodiment of the validation client module 100. A user may
invoke the validation client module 100 by through a PWS client
320. For example, the validation client module 100 may be a menu
item on a graphical user interface of the PWS client 320.
Alternatively, a user may use a command line prompt at the PWS
client 320 to invoke the validation client module 100. Once
invoked, the validation client module 100 may display a validation
viewer GUI as shown in FIG. 4.
[0070] FIG. 4 illustrates a validation viewer GUI 400 provided by
the validation client module 100 in accordance with yet another
embodiment. It should be readily apparent that the elements of the
validation viewer GUI 400 may be deleted and/or modified and new
elements added.
[0071] As shown in FIG. 4, the validation viewer GUI 400 includes a
target viewer component 410, a record viewer component 420, and an
extraction viewer 430 as generated by the I/O module 120. The
target viewer component 410 may be configured to allow editing of
validation attributes for each extracted fact (or keyword, concept,
term, etc.) through checkboxes and current list icons. Selecting an
icon on the target viewer component 410 highlights the associated
fact and its corresponding extractions in the extractions viewer
430.
[0072] FIG. 5 illustrates the target viewer component 410 in
greater detail in accordance with yet another embodiment. It should
be readily apparent that the elements of the target viewer
component 410 may be deleted and/or modified and new elements
added.
[0073] As shown in FIG. 5, the target viewer component 410 may
include a control bar 502 that includes a `Finish` button 504, a
`Save` button 506, and an `Exit` button 508. The Finish button 504
may be configured to save the domain expert's changes to a
database, mark the revision of the document as being finished in
the database, and close the validation viewer GUI 400, returning
the document and its facts to the host application. The Save button
506 may be configured to save the current state of the validation
viewer GUI 400 in a database for later completion by the user. The
Exit button 508 may be configured to provide the user with the
options of exiting the validation viewer GUI 400 without saving or
exiting the validation viewer GUI 400 and saving. The options may
be presented in a dialog box by the I/O module 120.
[0074] When the user is finished validating the facts the set of
facts that have been deleted, added, modified, and validated are
sent to the database through the storage interface module 140.
[0075] The target viewer component 410 may present the facts in
target groups (e.g., as shown in FIG. 5: Problems 510, Medications
512, Allergies 514, Procedures 516, and History 518). Under each
target group, the associated facts are displayed. A relevancy
checkbox 520 is associated with each fact. If activated, a selected
relevancy checkbox 520 may indicate that the associated fact is
material to the selected document (or report). The I/O module 120
may also place a status change marker 522 to indicate that the
relevancy of the associated fact has changed from a previous
report.
[0076] The target viewer component 410 also includes a current list
icon for each associated fact, as shown in an expanded view in FIG.
5A. The current list icon 524 may be configured to indicate the
status of the fact on the current list. By activating the
associated current list icon 524 for a selected fact, a user may
elect to make the fact Active, Inactive or view the current
list.
[0077] Returning to FIG. 4, the record viewer component 420 may be
configured to display the current document (or record) while the
extraction and target viewer components, 430 and 410, respectively,
display the extractions and facts for the selected document.
[0078] FIG. 6 illustrates a more detailed view of the record viewer
component 420 in accordance with yet another embodiment. It should
be readily apparent that the elements of the record viewer
component 420 may be deleted and/or modified and new elements
added.
[0079] As shown in FIG. 6, the record viewer component 420 may
include mention buttons, previous 602 and next 604. The mention
buttons, 602 and 604, may be configured to activate when a selected
fact in the target viewer component 410 has multiple mentions in
the current report. The context and spans of texts associated with
the selected extraction may also be displayed in the extraction
viewer 430. Otherwise, if a selected fact has a single mention, the
mention buttons, 602 and 504, may be `ghosted` or deactivated.
[0080] When activated, the mention buttons, 602 and 604, may be
configured to navigate the report by highlighting the occurrences
of the selected fact. Simultaneously, the context for the highlight
occurrences will also highlight in the extraction viewer 430.
[0081] In the record viewer component 420, a user may add
extractions. More particularly, the user may select a whole word(s)
within the same sentence. The validation module 110 may be
configured not to permit the user to select text in the headings.
After selection of text, a user may right-click on the selected
text to provide options to send the selected text to as an
extraction. For example, the I/O module 120 may display a dialog
box that lists the target groups (e.g., Add Problem, Add
Medication, Add Procedure, Add Allergy) in the target viewer
component.
[0082] Returning to FIG. 4, the extraction viewer component 430 may
be configured to display the detailed extractions from a
highlighted fact in the target viewer component 410. The extraction
viewer component 430 may also be configured to simultaneously
highlight selected text in the extraction viewer component 430 and
the corresponding text in the record viewer component 420.
[0083] FIG. 7 illustrates a more detailed view of the extractions
viewer component 430 in accordance with yet another embodiment. It
should be readily apparent that the elements of the extractions
viewer component 430 may be deleted and/or modified and new
elements added.
[0084] As shown in FIG. 7, the extractions viewer component 430 may
display an extraction 702 in one of three states: new, correct or
incorrect. A new extraction is one generated by the extraction
module 130 that has not yet been validated in any document version.
A correct (or validated) extraction has been checked and approved
by a user with the appropriate authority to approve the extraction.
An incorrect (or deprecated) extraction is one that the user with
proper authority has deemed as incorrect.
[0085] Associated with each extraction is a status checkbox 704. If
a user has placed a check in the checkbox 702, this indicates that
the status of the extraction is valid. If a user has placed an `X`
mark in the checkbox 702, this indicates an incorrect or
depreciated status for the selected extraction. The checkbox 702
for a new extraction may be defaulted to a state that configured by
the user. The extraction viewer component 430 may toggle between a
check and `X` mark in the checkbox 702.
[0086] A specific mention can be displayed in context for specific
extraction. The span of the text displayed can be any number of
characters as desired by the user however it is preferable to
display a limited number of characters in width (e.g., 100) so as
to limit the context to something easily understood by the user,
while achieving and appropriate aspect ratio of leading context to
following context based on the characteristics of the language of
the text (e.g., 2:1 for English). The actual specific extraction
may be distinguished from the surrounding context via font effects.
Whole words or partial words may be displayed. When a user selects
a particular mention or any part of the mention word string, the
line may become highlighted and the corresponding mention may be
displayed in the record viewer component 430.
[0087] FIG. 8 illustrates a more detailed flow diagram 800 for
validating facts for the validation viewer GUI 400 (shown in FIGS.
4-7) in accordance with yet another embodiment. It should be
readily apparent to those of ordinary skill in the art that this
flow diagram 800 represents a generalized illustration and that
other steps may be added or existing steps may be removed or
modified.
[0088] As shown in FIG. 8, a user with proper authority, e.g., a
domain expert, may instantiate the process of validating a fact by
selecting the fact (e.g., 530 on FIG. 5), in step 805 in step 810,
the validation module 110 may determine whether the selected fact
has the correct relevance by the action of the user. More
specifically, if the user indicates in the relevancy checkbox 520
that the selected fact is not relevant, the user may activate the
status marker icon 522 in step 815. Otherwise, the validation
module 110 may proceed to the processing of step 835, which is
described below.
[0089] In step 820, the validation module 110 may determine whether
the selected fact was relevant by waiting for a user selection on
the status marker icon 522. More particularly, if validation module
receives indication from the user that the selected fact is
relevant, the user may select the inactive status to make the fact
not relevant, in step 825. Subsequently, the validation module 110
proceeds to the processing of step 835.
[0090] Otherwise, if the selected fact was deemed relevant, the
user may select the Active status to make the fact relevant, in
step 830. Subsequently, the validation module 110 may determine
whether the user has selected additional facts for validation, in
step 835. If the user selects another fact, the validation module
110 returns to the processing of step 815. Otherwise, the
validation module 110 waits for an exit event, in step 840.
[0091] FIG. 9 illustrates a more detailed flow diagram 900 for
validating extractions for the extraction viewer component 430 of
the validation viewer GUI 400 (shown in FIGS. 4-7) in accordance
with yet another embodiment. It should be readily apparent to those
of ordinary skill in the art that this flow diagram 900 represents
a generalized illustration and that other steps may be added or
existing steps may be removed or modified.
[0092] As shown in FIG. 9, a user with proper authority, e.g., a
domain expert, may instantiate the process of validating an
extraction by selecting the extraction (e.g., 702 on FIG. 7), in
step 905
[0093] In step 910, the validation module 110 waits for an
indication from the user on whether the selected extraction is
correct. If the selected extraction is correct, the validation
module 110 proceeds to the processing of step 960, as described in
greater detail below. Otherwise, if the user indicates that the
selected extraction is incorrect, the user may activate (or click)
on associated status checkbox 704 (shown in FIG. 7), in step 915.
In step 920, the validation module 110 may wait for an indication
from the user that the extraction was correct. If the selected
extraction was incorrect, the user may change the status of the
selected extraction as incorrect by toggling the associated status
checkbox 704, in step 925. In step 930, the system may not require
additional user feedback. If the system determines that all the
extractions have been marked as incorrect, the may automatically
mark the associated fact as incorrect. Alternatively, the
validation module 110 may wait for an indication from the user on
whether or not all the fact extractions were incorrect. If the all
the fact extractions were not incorrect, the validation module 110
may proceed to the processing of step 960. Otherwise, if all the
fact extractions are incorrect, the system may mark the associated
fact as incorrect by marking the status to Incorrect in check box
704. Subsequently, the validation module may proceed to the
processing of step 960.
[0094] Returning to step 920, if the user determines that the
extraction was correct, the user may toggle the associated status
checkbox 704 as correct, in step 945. The validation module 110
then waits for an indication from the user on whether or not the
fact was incorrect in step 950. The user may correct the fact in
step 955. Subsequently, the validation module 110 proceeds to the
processing of step 960.
[0095] Otherwise, if the user determines that the fact was correct,
the validation module 110 may wait for an indication from the user
on whether or not to select additional extractions, in step 960. If
there are additional extractions, the validation module 110 returns
to the processing of step 905. Otherwise, the validation module 110
waits for an exit event, in step 965.
[0096] FIG. 10 illustrates an exemplary block diagram of a computer
system 1000 where an embodiment may be practiced. The functions of
the validation client module 100 may be implemented in program code
and executed by the computer system 1000. The validation client
module 100 and the NLPR system 300 may be implemented in computer
languages such as PASCAL, C, C++, JAVA, etc.
[0097] As shown in FIG. 10, the computer system 1000 includes one
or more processors, such as processor 1002, that provide an
execution platform for embodiments of the expressway routing
module. Commands and data from the processor 1002 are communicated
over a communication bus 1004. The computer system 1000 also
includes a main memory 1006, such as a Random Access Memory (RAM),
where the software for the validation client module 100 may be
executed during runtime, and a secondary memory 1008. The secondary
memory 1008 includes, for example, a hard disk drive 1020 and/or a
removable storage drive 1022, representing a floppy diskette drive,
a magnetic tape drive, a compact disk drive, or other removable and
recordable media, where a copy of a computer program embodiment for
the validation client module may be stored. The removable storage
drive 1022 reads from and/or writes to a removable storage unit
1024 in a well-known manner. A user interfaces with the validation
client module 100 with a keyboard 1026, a mouse 1028, and a display
1020. The display adaptor 1022 interfaces with the communication
bus 1004 and the display 1020 and receives display data from the
processor 1002 and converts the display data into display commands
for the display 1020.
[0098] Certain embodiments may be performed as a computer program.
The computer program may exist in a variety of forms both active
and inactive. For example, the computer program can exist as
software program(s) comprised of program instructions in source
code, object code, executable code or other formats; firmware
program(s); or other known program. Any of the above can be
embodied on a computer readable medium, which include storage
devices and signals, in compressed or uncompressed form. Exemplary
computer readable storage devices include conventional computer
system RAM (random access memory), ROM (read-only memory), EPROM
(erasable, programmable ROM), EEPROM (electrically erasable,
programmable ROM), and magnetic or optical disks or tapes.
Exemplary computer readable signals, whether modulated using a
carrier or not, are signals that a computer system hosting or
running the present invention can be configured to access,
including signals arriving from the Internet or other networks.
Concrete examples of the foregoing include distribution of
executable software program(s) of the computer program on a CD-ROM
or via Internet download. In a sense, the Internet itself, as an
abstract entity, is a computer readable medium. The same is true of
computer networks in general.
[0099] While the invention has been described with reference to the
exemplary embodiments thereof, those skilled in the art will be
able to make various modifications to the described embodiments
without departing from the true spirit and scope. The terms and
descriptions used herein are set forth by way of illustration only
and are not meant as limitations. In particular, although the
method has been described by examples, the steps of the method may
be performed in a different order than illustrated or
simultaneously. Those skilled in the art will recognize that these
and other variations are possible within the spirit and scope as
defined in the following claims and their equivalents.
* * * * *