U.S. patent application number 10/938314 was filed with the patent office on 2005-06-23 for method and system for quantitatively analyzing biological samples.
This patent application is currently assigned to Bioimagene, Inc.. Invention is credited to Gholap, Abhijeet S., Gholap, Gauri A..
Application Number | 20050136509 10/938314 |
Document ID | / |
Family ID | 34316479 |
Filed Date | 2005-06-23 |
United States Patent
Application |
20050136509 |
Kind Code |
A1 |
Gholap, Gauri A. ; et
al. |
June 23, 2005 |
Method and system for quantitatively analyzing biological
samples
Abstract
A method and system analyzing tissue or cell samples stored on
laboratory slides or other media to identify properties of medical
predictive and diagnostic relevance. The method and system includes
automatically analyzing plural digital images created from plural
biological tissue samples to which a staining reagent or a an
immunohistochemical (IHC) compound has been applied, automatically
quantitatively analyzing the relevant properties of the digital
images, and creating interpretive data, images and reports
resulting from such analysis.
Inventors: |
Gholap, Gauri A.; (San Jose,
CA) ; Gholap, Abhijeet S.; (San Jose, CA) |
Correspondence
Address: |
Lesavich High-Tech Law Group, P.C.
Suite 325
39 S. LaSalle Street
Chicago
IL
60603
US
|
Assignee: |
Bioimagene, Inc.
San Jose
CA
|
Family ID: |
34316479 |
Appl. No.: |
10/938314 |
Filed: |
September 10, 2004 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60501412 |
Sep 10, 2003 |
|
|
|
60515582 |
Oct 30, 2003 |
|
|
|
Current U.S.
Class: |
435/40.5 ;
382/128; 702/19 |
Current CPC
Class: |
G01N 33/5091 20130101;
G16H 50/20 20180101; G06T 2207/30072 20130101; G06K 9/0014
20130101; G06T 2207/10056 20130101; G06T 2207/30024 20130101; G06T
7/0012 20130101; G06T 7/41 20170101; G06T 7/11 20170101; Y10S
128/922 20130101; G06K 9/48 20130101; G06T 2200/24 20130101; G06K
9/00127 20130101; G01N 33/5005 20130101; G01N 1/30 20130101 |
Class at
Publication: |
435/040.5 ;
382/128; 702/019 |
International
Class: |
C12Q 001/68; G06F
019/00; G01N 033/48; G01N 033/50; G01N 001/30 |
Claims
We claim:
1. A pathological analysis system, comprising in combination: a
digital image analysis module for automatically analyzing a
plurality of digital images created from a plurality of biological
tissue samples to which an staining reagent has been applied to
determine one or more areas of interest and for automatically
analyzing a plurality of digital images created from a plurality of
biological tissues samples to which an immunohistochemical compound
has been applied to determine one or more areas of interest; an
medical analysis module for automatically quantitatively analyzing
the one or more determined areas of interest to automatically
generate additional interpretive images, medical data, medial
statistics or medical reports of predictive value or diagnostic
value; a display module to display the plurality of digital images,
the additional interpretive images, the medical data, medical
statistics or medical reports on a graphical user interface
display; and a recorder module to automatically record, store and
apply knowledge generated by the quantitatively analysis of the one
or more determined areas of interest.
2. The pathological analysis of claim 1 in which the digital image
analysis module automatically analyzes a plurality of digital
images which an staining reagent has been applied to determine one
or more areas of interest in which one or more human cancer cells
may be present.
3. The pathological analysis system of claim 1 in which the digital
image analysis module includes automatically analyzing a plurality
of digital images to which a staining reagent has been applied to
determine one or more areas of interest in which one or more human
cancer cells may be present using automatic chromatin pattern
analysis, automatic nucleoar pattern analysis or automatic mitotic
activity pattern analysis.
4. The pathological analysis system of claim 1 in which the digital
image analysis module includes automatically analyzing a plurality
of digital images created from a plurality of biological tissues
samples to which an immunohistochemical (IHC) compound has been
applied to determine one or more areas of interest in which one or
more human cancer cells may be present using automatic nuclear
pattern analysis, automatic cytoplasmic pattern analysis or
automatic membrane pattern analysis.
5. The pathological analysis system of claim 1 further comprising:
a microscope; and a digital camera.
6. A pathological analysis system, comprising in combination: a
means for automatically analyzing a plurality of digital images
created from a plurality of biological tissue samples to which an
staining reagent has been applied to determine one or more areas of
interest and for automatically analyzing a plurality of digital
images created from a plurality of biological tissues samples to
which an immunohistochemical compound has been applied to determine
one or more areas of interest; a means for automatically
quantitatively analyzing the one or more determined areas of
interest to automatically generate additional interpretive images,
medical data, medial statistics or medical reports of predictive
value or diagnostic value; a means for displaying the plurality of
digital images, the additional interpretive images, the medical
data, medical statistics or medical reports on a graphical user
interface display; and a means for automatically recording, storing
and applying knowledge generated by the quantitatively analysis of
the one or more determined areas of interest.
7. The pathological analysis of claim 6 in which the a means for
automatically analyzing a plurality of digital images automatically
analyzes a plurality of digital images which an staining reagent
has been applied to determine one or more areas of interest in
which one or more human cancer cells may be present.
8. The pathological analysis system of claim 6 in which the means
for automatically analyzing a plurality of digital images includes
automatically analyzing a plurality of digital images to which a
staining reagent has been applied to determine one or more areas of
interest in which one or more human cancer cells may be present
using automatic chromatin pattern analysis, automatic nucleoar
pattern analysis or automatic mitotic activity pattern
analysis.
9. The pathological analysis system of claim 6 in which the means
for automatically analyzing a plurality of digital images includes
automatically analyzing a plurality of digital images created from
a plurality of biological tissues samples to which an
immunohistochemical (IHC) compound has been applied to determine
one or more areas of interest in which one or more human cancer
cells may be present using automatic nuclear nuclear pattern
analysis, automatic cytoplasmic pattern analysis or automatic
membrane pattern analysis.
10. The pathological analysis system of claim 6 further comprising:
a means for magnifying microscopic biological tissue samples; and a
means for creating digital photographs.
11. A method for automatically creating a medical diagnosis,
comprising: acquiring a plurality of digital images created from
the plurality of biological tissue samples to which a staining
reagent has been applied; automatically pre-processing the
plurality of digital images to adjust, if necessary, a contrast
level and a color level; automatically performing a histogram
analysis on the plurality of digital images using grey and red,
green, blue (RGB) luminosity values to locate one or more areas of
interest; automatically determining one or more areas of interest
in which one or more cancer cells may be present using automatic
chromatin pattern analysis, automatic nucleoar pattern analysis or
automatic mitotic activity pattern analysis; and automatically
classifying the determined one or more areas of interest in which
one or more cancer cells may be present, thereby creating a medical
diagnosis.
12. The method of claim 11 further comprising a computer readable
medium having stored therein instructions for causing a processor
to execute the steps of the method.
13. The method of claim 11 in which the staining reagent includes a
Hematoxillin and Eosin (H&E) staining reagent.
14. The method of claim 11 further comprising: creating new medical
knowledge using the determined one or more areas of interest which
one or more cancer cells may be present; and automatically
adjusting the processing completed on the plurality of digital
images to further refine the ability to automatically detect and
classify one or more cancer cells to create a medical
diagnosis.
15. A method for automatically creating a medical diagnosis,
comprising: acquiring a plurality of digital images created from
the plurality of biological tissue samples to which an
immunohistochemical (IHC) compound has been applied; automatically
pre-processing the plurality of digital images to adjust, if
necessary, a contrast level and a color level; automatically
performing a histogram analysis on the plurality of digital images
using grey and red, green, blue (RGB) luminosity values to locate
one or more areas of interest; automatically determining one or
more areas of interest in which one or more cancer cells may be
present using automatic nuclear pattern analysis, automatic
cytoplasmic pattern analysis or automatic membrane pattern
analysis; and automatically classifying the determined one or more
areas of interest in which one or more cancer cells may be present,
thereby creating a medical diagnosis.
16. The method of claim 15 further comprising a computer readable
medium having stored therein instructions for causing a processor
to execute the steps of the method.
17. The method of claim 15 further comprising: creating new medical
knowledge using the determined one or more areas of interest which
one or more cancer cells may be present; and automatically
adjusting the processing completed on the plurality of digital
images to further refine the ability to automatically detect and
classify one or more cancer cells to create a medical
diagnosis.
18. A method for completing a pathological diagnosis with a
pathiam, comprising: automatically analyzing a plurality of digital
image intensity values from a plurality of digital images created
from a plurality of biological tissue samples to which a staining
reagent or a an immunohistochemical (IHC) compound has been applied
to determine one or more areas of interest; automatically
classifying the determined one or more areas of interest into one
or more patterns used to locate one or more morphological features
from the biological tissue samples in which one or more human
cancer cells may be present; automatically creating a pathological
diagnosis using the classified one or more patterns.
19. The method of claim 18 further comprising a computer readable
medium having stored therein instructions for causing a processor
to execute the steps of the method.
20. The method of claim 18 staining reagent includes a Hematoxillin
and Eosin (H&E) staining reagent.
Description
CROSS REFERENCES TO RELATED APPLICATIONS
[0001] This patent application claims priority to U.S. Provisional
Patent Application No. 60/501,142, filed Sep. 10, 2003, and U.S.
Provisional Patent Application No. 60/515,582 filed Oct. 30, 2003,
the contents of both of which are incorporated by reference.
FIELD OF THE INVENTION
[0002] This invention is related to analysis of laboratory data.
More specifically, it relates to a method and system for
quantitatively analyzing biological samples.
BACKGROUND OF THE INVENTION
[0003] Molecular methods are one standard approach used to diagnose
and treat a number of diseases. Microarray based (e.g., High
Content Screening (HCS)) analysis High Throughput Screening (HTS)
analysis and gene expression profiling studies can be used to
identify diagnosis markers, analyze disease progression and aid
prognosis in developing cures (e.g., cancer progression and tumor
analysis). This results in the availability of new and improved
diagnostic tests to patients and physicians.
[0004] Such molecular analysis also provides the beginning of
personalized medicine. Molecular-based therapies are becoming more
targeted, such that knowledge of specific genetic variations
present in individual patients guides selection of the best
possible therapy, while minimizing chances of adverse reactions.
This concept has also found its way to the drug development
process, which is likely to result in an increasing number of new
therapeutic agents that are prescribed in conjunction with
"theranostic" genetic tests that guide use of the therapy in
appropriate patient populations.
[0005] Gene expression using microarray technology allows the
simultaneous assessment of the transcription of tens of thousands
of genes, and of their relative expression between normal cells and
malignant cells. Tissue Microarray Analysis (TMA) significantly
impacts a researcher's ability to explore the genetic changes
associated with cancer etiology and development and ultimately lead
to the discovery of new biomarkers for disease diagnosis, prognosis
and new therapeutic tools. Such onco-analysis at molecular level is
much more specific and accurate than in the past.
[0006] Also the microarray genetic profile of patients can reveal
individual patients response to chemotherapy. Comparative genome
hybridization studies based on microarray technology may also
predict alterations in cancer patients' genetic profiles.
Microarray based expression-profiling studies and its correlation
to clinical/histopathological/cystopathological findings still
needs to be greatly fine-tuned to increase its significance. More
sophisticated and high throughput validation approaches are
required to allow the screening of hundreds of potential biomarkers
onto thousands of tumor samples.
[0007] The use of newer imaging tools such as automated microscopes
and digital imaging in the laboratory setting has resulted in a
need for more sophisticated diagnostic software tools. Currently
histological, cytological and imunohistochemical specimens are
stored on glass slides, and these slides are read manually by
technicians. Such analysis is, necessarily, based on subjective,
human interpretation and is subject to human error. Clinicians and
oncologists desire higher level of precision, objectivity,
reproducibility and standardization in the pathological
results.
[0008] Analyzing the tissue samples stained with
immunohistochemical (IHC) reagents has been the key development in
the practice of pathology. Both normal and diseased cells have
certain physical characteristics that can be used to differentiate
them. These characteristics include complex patterns, rare events,
and subtle variations in color and intensity. These variants are
what a pathologist looks for when scanning a slide with a manual
microscope.
[0009] Hematoxillin and Eosin (H&E) is a method of staining is
used to study the morphology of tissue samples. Oncologists attempt
to identify particular types of cancer by detecting variations in
the patterns from the normal tissue. H&E staining can also be
used to determine the pathological grading/staging of cancer (e.g.,
the Richardson and Bloom Method).
[0010] This pathological grading of cancer is important from both a
diagnostic and predictive perspective. Currently, pathologists must
rely on manually analyzed samples without the benefit of any
software tool. It is desirable to provide automated objective and
reproducible results with fewer variations from
pathologist-pathologist and lab-to-lab.
[0011] Until recently, most computer-driven systems could match the
human eye in its ability to recognize complex patterns. There are
limits, however, to the power of the human eye. Fatigue,
repetitiveness of the task, large numbers of cells inside a given
sample can be a problem for pathologists, as can the inability to
distinguish between similar colors. Also computers typically can
surpass the human eye in their ability to detect rare events and
recognize subtle variations in color and intensity. Computer
scientists first approached the challenge of automating the reading
of samples on glass slides from a pattern-recognition angle,
attempting to write a software program to duplicate the actions of
the human brain.
[0012] Looking for a single object within a range of objects of
similar but different sizes and orientations is very difficult.
Additionally, to find the object when it is partially obscured or
broken further increases the complexity. If a human looks at the
same scene for approximately 5 to 10 minutes the brain acclimatizes
to it and loses the ability to detect subtle changes in the scene.
It is as though the observer is seeing a memory, and not the actual
scene. Having a person spend a long period of time scanning slides
that have similar features leads to fatigue and increases the
chance of missing important information in the slides. Computers do
not experience this fatigue and so are effective at detecting
subtle changes in specific criteria. Therefore, rare-event
detection is an appropriate use of a computer's power.
[0013] Additionally, because the human brain calculates brightness
and color by comparison with the local context, a level of
subjectivity is incorporated into reading stained slides manually.
Thus, IHC/H&E slides read by manual microscopy are scored only
semi quantitatively, in accordance with the subjective and
approximate nature of the human eye's color perception.
[0014] Based on the foregoing considerations, there is a need for
reliable, information driven histopathological and
cystopathological image analysis software system for
quantification, image management and retrieval based on a
self-learning framework using color, intensity, morphological
patterns, artificial intelligence and expert system,
personalization and remote communications.
[0015] Thus, it is desirable to provide a reliable, information
driven software system for analyzing tissue and cell samples, which
can create, retrieve, manage, and statistically and/or
quantitatively analyze such images to increase their diagnostic and
predictive value.
SUMMARY OF THE INVENTION
[0016] In accordance with preferred embodiments of the invention,
some of the problems associated with human interpretation and error
associated with manually analyzing images created from biological
samples are overcome. A method and system for quantitatively
analyzing images created from biological samples is presented.
[0017] The method and system includes, but is not limited to,
automatically analyzing plural digital images created from plural
biological tissue samples to which a staining reagent or a an
immunohistochemical (IHC) compound has been applied, automatically
quantitatively analyzing the relevant properties of the digital
images, and creating interpretive data, images and reports
resulting from such analysis.
[0018] The foregoing and other features and advantages of preferred
embodiments of the present invention will be more readily apparent
from the following detailed description. The detailed description
proceeds with references to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Preferred embodiments of the present invention are described
with reference to the following drawings, wherein:
[0020] FIG. 1 is a block diagram illustrating an exemplary
biological sample analysis processing system;
[0021] FIG. 2 is a block diagram illustrating applications of the
biological sample analysis processing system of FIG. 1;
[0022] FIG. 3 is a flow diagram illustrating a method for
automatically creating a medical diagnosis;
[0023] FIG. 4 is a block diagram illustrating a data flow for the
method of FIG. 3;
[0024] FIG. 5 is a flow diagram illustrating a method for
automatically creating a medical diagnosis;
[0025] FIG. 6 is a block diagram illustrating a data flow for the
method of FIG. 4;
[0026] FIG. 7 is a flow diagram illustrating a method completing a
pathological diagnosis with a pathiam; and
[0027] FIG. 8 is a block diagram illustrating another data flow for
analyzing digital images.
DETAILED DESCRIPTION OF THE INVENTION
[0028] Exemplary Biological Sample Analysis System
[0029] FIG. 1 is a block diagram illustrating an exemplary
biological sample analysis processing system 10. The exemplary
biological sample analysis processing system 10 includes, but is
not limited to, one or more computers 12 with a computer display
14. The computer display 14 presents a windowed graphical user
interface (GUI) 16 with multiple windows to a user. The one or more
computers 12 may be replaced with client terminals in
communications with one or more servers, a personal digital/data
assistant (PDA), a laptop computer, a mobile computer, an Internet
appliance, one or two-way pagers, or other similar mobile or
hand-held electronic device.
[0030] The one or more computers are associated with one or more
databases 18 (one of which is illustrated) includes biological
sample information in various digital image or digital data
formats. The one or more databases 18 may be integral to a memory
system on the computer 12 or in secondary storage such as a hard
disk, floppy disk, optical disk, or other non-volatile mass storage
devices. The computer 12 includes one or more applications 20 for
analyzing biological samples.
[0031] The one or more computers 12 are also in communications with
a communications network 22 such as the Internet, an intranet, a
Local Area Network (LAN) or other computer network. Functionality
of the medical data analyzing system 10 can also be distributed
over plural computers 12 via the communications network 22.
[0032] The communications network 22 includes, but is not limited
to, the Internet, an intranet, a wired Local Area Network (LAN), a
wireless LAN (WiLAN), a Wide Area Network (WAN), a Metropolitan
Area Network (MAN), Public Switched Telephone Network (PSTN) and
other types of communications networks 18 providing voice, video
and data communications.
[0033] The communications network 22 may include one or more
gateways, routers, or bridges. As is known in the art, a gateway
connects computer networks using different network protocols and/or
operating at different transmission capacities. A router receives
transmitted messages and forwards them to their correct
destinations over the most efficient available route. A bridge is a
device that connects networks using the same communications
protocols so that information can be passed from one network device
to another.
[0034] The communications network 22 may include one or more
servers and one or more web-sites accessible by user to send and
receive information useable by the one or more computers 12. The
communications network 22 includes, but is not limited to data
networks using the Transmission Control Protocol (TCP), User
Datagram Protocol (UDP), Internet Protocol (IP) and other data
protocols.
[0035] As is know in the art, TCP provides a connection-oriented,
end-to-end reliable protocol designed to fit into a layered
hierarchy of protocols which support multi-network applications.
TCP provides for reliable inter-process communication between pairs
of processes in network devices attached to distinct but
interconnected networks. For more information on TCP see Internet
Engineering Task Force (ITEF) Request For Comments (RFC)-793, the
contents of which are incorporated herein by reference.
[0036] As is know in the art, UDP provides a connectionless mode of
communications with datagrams in an interconnected set of computer
networks. UDP provides a transaction oriented datagram protocol,
where delivery and duplicate packet protection are not guaranteed.
For more information on UDP see IETF RFC-768, the contents of which
incorporated herein by reference.
[0037] As is known in the art, IP is an addressing protocol
designed to route traffic within a network or between networks. IP
is described in IETF Request For Comments (RFC)-791, the contents
of which are incorporated herein by reference. However, more fewer
or other protocols can also be used on the communications network
28 and the present invention is not limited to TCP/UDP/IP.
[0038] The communications network 22 may also include portions of a
Public Switched Telephone Network (PSTN) or cable television
network (CATV) that connects the one or more computers 12 via one
or more twisted pairs of copper wires, coaxial cable, fiber optic
cable, other connection media or other connection interfaces with
corresponding wired connection protocols (e.g., DSL, ADSL, ISDN,
etc.) The PSTN is any public switched telephone network provided by
AT&T, GTE, Sprint, MCI, SBC, Verizon and others.
[0039] Preferred embodiments of the present invention includes
network devices and interfaces that are compliant with all or part
of standards proposed by the Institute of Electrical and Electronic
Engineers (IEEE), International Telecommunications
Union-Telecommunication Standardization Sector (ITU), European
Telecommunications Standards Institute (ETSI), Internet Engineering
Task Force (IETF), U.S. National Institute of Security Technology
(NIST), American National Standard Institute (ANSI), Wireless
Application Protocol (WAP) Forum, Data Over Cable Service Interface
Specification (DOCSIS) Forum, Bluetooth Forum, or the ADSL Forum.
However, network devices and interfaces based on other standards
could also be used.
[0040] IEEE standards can be found on the World Wide Web at the
Universal Resource Locator (URL) "www.ieee.org." The ITU, (formerly
known as the CCITT) standards can be found at the URL "www.itu.ch."
ETSI standards can be found at the URL "www.etsi.org." IETF
standards can be found at the URL "www.ietf.org." The NIST
standards can be found at the URL "www.nist.gov." The ANSI
standards can be found at the URL "www.ansi.org." The DOCSIS
standard can be found at the URL "www.cablemodem.com." Bluetooth
Forum documents can be found at the URL "www.bluetooth.com." WAP
Forum documents can be found at the URL "www.wapforum.org." ADSL
Forum documents can be found at the URL "www.adsl.com."
[0041] The digital images include digital images of biological
samples taken via a microscope with a digital camera and stored in
a variety of digital image formats including, bit-mapped, joint
pictures expert group (JPEG), graphics interchange format (GIF),
etc. However, the present invention is not limited to these digital
image formats and other digital image or digital data formats can
also be used to practice the invention.
[0042] The digital images are typically obtained by magnifying the
biological samples with a microscope or other magnifying device and
capturing a digital image of the magnified biological sample (e.g.,
groupings of plural magnified cells, etc.).
[0043] One embodiment includes a microscope or other magnifying
device and a digital or analog camera. Another embodiment
optionally includes a microscope with a digital camera 24.
[0044] The term "sample" includes cellular material derived from a
biological organism. Such samples include but are not limited to
hair, skin samples, tissue samples, cultured cells, cultured cell
media, and biological fluids. The term "tissue" refers to a mass of
connected cells (e.g., CNS tissue, neural tissue, or eye tissue)
derived from a human or other animal and includes the connecting
material and the liquid material in association with the cells. The
term "biological fluid" refers to liquid material derived from a
human or other animal. Such biological fluids include, but are not
limited to, blood, plasma, serum, serum derivatives, bile, phlegm,
saliva, sweat, amniotic fluid, and cerebrospinal fluid (CSF), such
as lumbar or ventricular CSF. The term "sample" also includes media
containing isolated cells. The quantity of sample required to
obtain a reaction may be determined by one skilled in the art by
standard laboratory techniques. The optimal quantity of sample may
be determined by serial dilution.
[0045] An operating environment for the devices biological sample
analysis processing system 10 include a processing system with one
or more high speed Central Processing Unit(s) ("CPU"), processors
and one or more memories. In accordance with the practices of
persons skilled in the art of computer programming, the present
invention is described below with reference to acts and symbolic
representations of operations or instructions that are performed by
the processing system, unless indicated otherwise. Such acts and
operations or instructions are referred to as being
"computer-executed," "CPU-executed," or "processor-executed."
[0046] It will be appreciated that acts and symbolically
represented operations or instructions include the manipulation of
electrical signals or biological signals by the CPU or processor.
An electrical system or biological system represents data bits
which cause a resulting transformation or reduction of the
electrical signals or biological signals, and the maintenance of
data bits at memory locations in a memory system to thereby
reconfigure or otherwise alter the CPU's or processor's operation,
as well as other processing of signals. The memory locations where
data bits are maintained are physical locations that have
particular electrical, magnetic, optical, or organic properties
corresponding to the data bits.
[0047] The data bits may also be maintained on a computer readable
medium including magnetic disks, optical disks, organic memory, and
any other volatile (e.g., Random Access Memory ("RAM")) or
non-volatile (e.g., Read-Only Memory ("ROM"), flash memory, etc.)
mass storage system readable by the CPU. The computer readable
medium includes cooperating or interconnected computer readable
medium, which exist exclusively on the processing system or can be
distributed among multiple interconnected processing systems that
may be local or remote to the processing system.
[0048] The digital images include digital images of biological
samples taken with a camera such as a digital camera and stored in
a variety of digital image formats including, bit-mapped, joint
pictures expert group (JPEG), graphics interchange format (GIF),
etc. However, the present invention is not limited to these digital
image formats and other digital image or digital data formats can
also be used to practice the invention.
[0049] The digital images are typically obtained by magnifying the
biological samples with a microscope or other magnifying device and
capturing a digital image of the magnified biological sample (e.g.,
groupings of plural magnified cells, etc.).
[0050] The term "sample" includes cellular material derived from a
biological organism. Such samples include but are not limited to
hair, skin samples, tissue samples, cultured cells, cultured cell
media, and biological fluids. The term "tissue" refers to a mass of
connected cells (e.g., CNS tissue, neural tissue, or eye tissue)
derived from a human or other animal and includes the connecting
material and the liquid material in association with the cells. The
term "biological fluid" refers to liquid material derived from a
human or other animal. Such biological fluids include, but are not
limited to, blood, plasma, serum, serum derivatives, bile, phlegm,
saliva, sweat, amniotic fluid, and cerebrospinal fluid (CSF), such
as lumbar or ventricular CSF. The term "sample" also includes media
containing isolated cells. The quantity of sample required to
obtain a reaction may be determined by one skilled in the art by
standard laboratory techniques. The optimal quantity of sample may
be determined by serial dilution.
[0051] An operating environment for the devices biological sample
analysis processing system 10 include a processing system with one
or more high speed Central Processing Unit(s) ("CPU"), processors
and one or more memories. In accordance with the practices of
persons skilled in the art of computer programming, the present
invention is described below with reference to acts and symbolic
representations of operations or instructions that are performed by
the processing system, unless indicated otherwise. Such acts and
operations or instructions are referred to as being
"computer-executed," "CPU-executed," or "processor-executed."
[0052] It will be appreciated that acts and symbolically
represented operations or instructions include the manipulation of
electrical signals or biological signals by the CPU or processor.
An electrical system or biological system represents data bits
which cause a resulting transformation or reduction of the
electrical signals or biological signals, and the maintenance of
data bits at memory locations in a memory system to thereby
reconfigure or otherwise alter the CPU's or processor's operation,
as well as other processing of signals. The memory locations where
data bits are maintained are physical locations that have
particular electrical, magnetic, optical, or organic properties
corresponding to the data bits.
[0053] The data bits may also be maintained on a computer readable
medium including magnetic disks, optical disks, organic memory, and
any other volatile (e.g., Random Access Memory ("RAM")) or
non-volatile (e.g., Read-Only Memory ("ROM"), flash memory, etc.)
mass storage system readable by the CPU. The computer readable
medium includes cooperating or interconnected computer readable
medium, which exist exclusively on the processing system or can be
distributed among multiple interconnected processing systems that
may be local or remote to the processing system.
[0054] Analyzing Biological Samples to Create a Medical
Diagnosis
[0055] FIG. 2 is a block diagram 26 illustrating applications 20 of
the biological sample analysis processing system 10, that include,
but are not limited to a digital image analysis module 28 for
automatically analyzing a plural digital images created from a
plural biological tissue samples to which an staining reagent has
been applied to determine one or more areas of interest and for
automatically analyzing a plural digital images created from a
plural biological tissues samples to which an immunohistochemical
(IHC) compound has been applied to determine one or more areas of
interest, a medical analysis module 30 for automatically
quantitatively analyzing the one or more determined areas of
interest to automatically generate additional interpretive images,
medical data, medical statistics or medical reports of predictive
value or diagnostic value, a display module 32 to display the
plural digital images, the additional interpretive images, the
medical data, medical statistics or medical reports on a graphical
user interface display, a recorder module 34 to automatically
record, store and apply knowledge generated by the quantitatively
analysis of the one or more determined areas of interest.
[0056] In one embodiment the applications 20 are software
applications. However the invention is not limited
[0057] FIG. 3 is a flow diagram illustrating a Method 36 for
automatically creating a medical diagnosis. At Step 38, plural
digital images created from plural biological tissue samples to
which a staining reagent has been applied are acquired. At Step 40,
the plural digital images are pre-processed to adjust, if
necessary, a contrast level and a color level. At Step 42, a
histogram analysis is performed on the plural digital images using
grey and red, green, blue (RGB) luminosity values to locate one or
more areas of interest. At Step 44, one or more areas of interest
in which one or more human cancer cells may be present are
determined using automatic chromatin pattern analysis, automatic
nucleoar pattern analysis or automatic mitotic activity pattern
analysis. At Step 46, the determined one or more areas of interest
in which one or more human cancer cells may be present are
classified and used to create a medical diagnosis.
[0058] In one embodiment, Method 36 further comprises creating new
medical knowledge using the determined one or more areas of
interest and automatically adjusting the processing completed at
Steps 40-46 to further refine the ability to automatically detect
and classify one or more human cancer cells to create a medical
diagnosis.
[0059] FIG. 4 is a block diagram 48 illustrating a data flow for
Method 36. FIG. 4 illustrates as analysis of digital images are
completed new medical diagnostic knowledge is obtained. The new
medical diagnostic knowledge is then automatically re-applied to
the analysis techniques used to further the further refine the
ability to automatically detect and classify one or more cancer
cells to create a medical diagnosis.
[0060] FIG. 5 is a flow diagram illustrating a Method 50 for
automatically creating a medical diagnosis. At Step 52, plural
digital images created from plural biological tissue samples to
which an immunohistochemical (IHC) compound has been applied are
acquired. At Step 54, the plural digital images are pre-processed
to adjust, if necessary, a contrast level and a color level. At
Step 56, a histogram analysis is performed on the plural digital
images using grey and red, green, blue (RGB) luminosity values to
locate one or more areas of interest. At Step 58, one or more areas
of interest in which one or more cancer cells may be present are
determined using automatic nuclear pattern analysis, automatic
cytoplasmic pattern analysis or automatic membrane pattern
analysis. At Step 60, the determined one or more areas of interest
in which one or more cancer cells may be present are classified and
used to create a medical diagnosis.
[0061] In one embodiment, Method 50 further comprises creating new
medical knowledge using the determined one or more areas of
interest and automatically adjusting the processing completed at
Steps 52-60 to further refine the ability to automatically detect
and classify one or more cancer cells to create a medical
diagnosis.
[0062] FIG. 6 is a block diagram 62 illustrating a data flow for
Method 50. FIG. 5 illustrates as analysis of digital images are
completed new medical diagnostic knowledge is obtained. The new
medical diagnostic knowledge is then automatically re-applied to
the analysis techniques used to further the further refine the
ability to automatically detect and classify one or more cancer
cells to create a medical diagnosis.
[0063] A PATHIAM is a tool used by pathologists to assist in
quantification and reporting a pathological diagnosis. A PATHIAM
automatically uses digital image intensity values and classifies
them into useable identifiable patterns to locate biological
components such as cell membranes and other shapes decoded as
morphological feature patterns and other types of patterns. A
PATHIAM is thus used an aid or enabling tool for pathologists and
other medical or research professionals to help them in analyzing
biological tissue and other biological samples with more precision
and accuracy using automated processes.
[0064] FIG. 7 is a flow diagram illustrating a Method for 64
completing a pathological diagnosis with a pathiam. At Step 66,
plural digital image intensity values from a plural digital images
created from plural biological tissue samples to which a staining
reagent or an immunohistochemical (IHC) compound has been applied
are automatically analyzed to determine one or more areas of
interest. At Step 68, the determined one or more areas of interest
are automatically classified into one or more patterns used to
locate one or more morphological features from the biological
tissue samples in which one or more human cancer cells may be
present. At Step 70, a pathological diagnosis is automatically
created using the classified one or more patterns.
[0065] FIG. 8 is a block diagram 72 illustrating another data flow
for analyzing digital images. The applications 20 are capable of
acquiring images from digital microscopic equipment or from image
databases, either managed or unmanaged.
[0066] In one embodiment, the applications 20 are based on an
expert system framework for analyzing Histopathological and
Cytopathological biopsy slides, which may be stained with, for
example, Hematoxillin-Eosin or with Imunohistochemical reagents.
The stained slides can then subjected to microscopic
examinations.
[0067] The parameters used for digital image analysis are
configurable by a user and default parameters may be modified and
refined per set of images processed via self-learning engine
collecting knowledge across the framework.
[0068] The analyzed digital images also are
grouped/clustered/classified based on similar trends and features
and each formed cluster can then aid the knowledge extraction step
indicated by a possible knowledge base (KB) creation. The KB
creation collects information thus received as a part of analysis,
which it may correlate to the statistical libraries and other
database to generate knowledge rules.
[0069] The system includes but is not limited to the steps of
creating or capturing images of the cell or tissue or other
biological samples. The biological samples may, but are not
required to be, treated with chemical reagents to stain or enhance
the visibility of certain properties of the biological samples.
[0070] The properties of the captured images are quantitatively
analyzed by the software to generate additional interpretive
images, data and/or reports of predictive and diagnostic value
having various content, appearance and format.
[0071] Exemplary applications of the invention may include but are
not limited to: (1) Oncopathology Diagnosis: a PATHIAM may be
useful in Oncopathological or other diagnosis of biopsy slides. For
example, digital images of H&E and IHC stained breast biopsy
slides can be analyzed using PATHIAM for pathological TNM scoring
and common IHC breast panel analysis such as ER, PR, HER-2
positivity tests. These analysis results may be useful not only
from diagnostic viewpoint but may have prognostic value associated
with it; (2) Oncopathology Research: an immunostain database data
with statistical values and tissue microarray features are useful
to research community characteristics. For example, a PATHIAM may
help the pathologist choose the best panel of immunostains, which
may in turn help differentiate between the tumors. This may be a
useful application considering the market availability of a large
pool of antibody reagents. The knowledge database lists the
antibodies that can differentiate between the type of tumors
entered by the user (e.g., ductal ca vs. lobular ca). The system
may also grade antibodies with respect to their ability to
differentiate between the tumors; (3) Cytopathogocial Diagnosis:
high power image analysis of H&E stained slides is beneficial
for the cytopathologists. Nuclear grading based on nuclear
pleomorphism with graphical display 14 and quantitification of
chromatin and nucleolar pattern may be of significance in
cystopathological diagnosis of surgical/FNA slides. This assists in
distinguishing benign and malignant tumors1 (4) Cytopathogocial
Research: comparison of graphs and values of nuclear pleomorphism,
chromatin and nucleolar pattern with prior cases having similar or
closely similar results may assist cytopathology researchers; (5)
Histotopathological Diagnosis: A PATHIAM is useful for
histopathological or other diagnosis of surgical biopsy slides. The
H&E module may help in morphological analysis while IHC marker
may be useful for standardization and quantification of biomarker
levels on tissue samples. The peer review model is also useful from
a histopathological or other diagnostic point of view; (6)
Histopathogocial Research: useful features for histopathical
research may include image management and retrieval and peer
review; (7) Molecular Diagnostics: The system includes functional
capabilities such as object identification, object counting,
staining-intensity measurement, and morphometric characterization.
The functions are applied individually or in combinations to enable
licensed laboratory professionals or others to analyze any slides
and other media stained with H&E, IHC or ICC reagents, such as
those stained for ER/PR, HER2, EGFR, Ki-67, p53, and other stains.
PATHIAM will also assist Microarray based expression profiling
studies and its correlation to clinical/
histopathological/cytopathological findings, which is the basis of
molecular diagnostics. This potentially may allow the screening of
hundreds of potential biomarkers onto thousands of tumor or other
samples simultaneously; (8) Pharmacogenomics, Drug Discovery, and
Clinical Trials: It is anticipated that growth in the field and
emergence of advanced bio-informatics tools for data mining and
analysis with simpler array designs may facilitate this technology
in diagnosis and prognosis of cancers. And very soon the screening
tests based on microarray findings may one of the many tests
performed on cancer patients in a multidisciplinary team patient
management approach to cancer diagnosis and prognosis.
[0072] It should be understood that the programs, processes,
methods and system described herein are not related or limited to
any particular type of computer or network system (hardware or
software), unless indicated otherwise. Various combinations of
general purpose, specialized or equivalent computer components
including hardware, software, and firmware and combinations thereof
may be used with or perform operations in accordance with the
teachings described herein.
[0073] In view of the wide variety of embodiments to which the
principles of the present invention can be applied, it should be
understood that the illustrated embodiments are exemplary only, and
should not be taken as limiting the scope of the present invention.
For example, the steps of the flow diagrams may be taken in
sequences other than those described, and more fewer or equivalent
elements may be used in the block diagrams.
[0074] The claims should not be read as limited to the described
order or elements unless stated to that effect. In addition, use of
the term "means" in any claim is intended to invoke 35 U.S.C.
.sctn.112, paragraph 6, and any claim without the word "means" is
not so intended.
[0075] Therefore, all embodiments that come within the scope and
spirit of the following claims and equivalents thereto are claimed
as the invention.
* * * * *