U.S. patent application number 11/037914 was filed with the patent office on 2008-11-13 for video and audio monitoring for syndromic surveillance for infectious diseases.
Invention is credited to David Volk Beard, Dorin Comaniciu, Stephen P. Masticola, Justinian Rosca.
Application Number | 20080279420 11/037914 |
Document ID | / |
Family ID | 39916545 |
Filed Date | 2008-11-13 |
United States Patent
Application |
20080279420 |
Kind Code |
A1 |
Masticola; Stephen P. ; et
al. |
November 13, 2008 |
VIDEO AND AUDIO MONITORING FOR SYNDROMIC SURVEILLANCE FOR
INFECTIOUS DISEASES
Abstract
We present, in exemplary embodiments of the present invention,
novel systems and methods for syndromic surveillance that can
automatically monitor symptoms that may be associated with the
early presentation of a syndrome (e.g., fever, coughing, sneezing,
runny nose, sniffling, rashes). Although not so limited, the novel
surveillance systems described herein can be placed in common areas
occupied by a crowd of people, in accordance with local and
national laws applicable to such surveillance. Common areas may
include public areas (e.g., an airport, train station, sports
arena) and private areas (e.g., a doctor's waiting room). The
monitored symptoms may be transmitted to a responder (e.g., a
person, an information system) outside of the surveillance system,
such that the responder can take appropriate action to identifying,
treat and quarantine potentially infected individuals, as
necessary.
Inventors: |
Masticola; Stephen P.;
(Kingston, NJ) ; Beard; David Volk; (Inkom,
ID) ; Comaniciu; Dorin; (Princeton Jct., NJ) ;
Rosca; Justinian; (West Windsor, NJ) |
Correspondence
Address: |
Siemens Corporation;Intellectual Property Department
170 Wood Avenue South
Iselin
NJ
08830
US
|
Family ID: |
39916545 |
Appl. No.: |
11/037914 |
Filed: |
January 18, 2005 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60538347 |
Jan 22, 2004 |
|
|
|
Current U.S.
Class: |
382/103 ;
382/128 |
Current CPC
Class: |
A61B 5/7275 20130101;
A61B 5/1123 20130101; G16H 50/80 20180101; A61B 5/1113 20130101;
A61B 5/01 20130101; A61B 7/003 20130101; A61B 5/0823 20130101; A61B
5/1128 20130101; G08B 31/00 20130101 |
Class at
Publication: |
382/103 ;
382/128 |
International
Class: |
G06K 9/00 20060101
G06K009/00 |
Claims
1. A method for syndromic surveillance, comprising: detecting
syndromic data associated with infectious disease and sickness
among a plurality of people using a plurality of monitoring
devices; probabilistically fusing the syndromic data from the
plurality of monitoring devices; locating individuals exhibiting
the syndromic data; verifying the locations of the individuals and
the syndromic data for consistency among the plurality of
monitoring devices; and transmitting the location of a potentially
infected individual exhibiting the probabilistically fused
syndromic data to a responder for tracking the potentially infected
individual.
2. The method of claim 1, wherein the step of probabilistically
fusing the syndromic data comprises detecting a combination of
symptoms exhibited by a single individual, the combination of
symptoms being associated with a syndrome.
3. The method of claim 1, wherein the step of detecting syndromic
data comprises detecting at least one of temperature-related
symptoms, audible symptoms, and visual symptoms.
4. The method of claim 1, further comprising: enhancing sensitivity
and selectivity of the plurality of monitoring devices using the
results of the step of verifying.
5. The method of claim 1, wherein the step of transmitting the
location of a potentially infected individual to a responder
comprises wirelessly transmitting the location of the potentially
infected individual to the responder.
6. The method of claim 1, wherein the step of transmitting the
location of a potentially infected individual to a responder
comprises transmitting the location of the potentially infected
individual to a portable receiver operated by the responder.
7. The method of claim 1, further comprising: transmitting the
probabilistically fused syndromic data to the responder.
8. The method of claim 1, further comprising: transmitting the
syndromic data to the responder.
9. A system for syndromic surveillance, comprising: a plurality of
symptom monitors, wherein the plurality of symptom monitors detect
a plurality of symptoms among a plurality of people, and wherein at
least one of the plurality of symptom monitors detects a location
for at least one candidate from the plurality of people exhibiting
at least one of the plurality of symptoms; a syndrome identifier
operatively connected to the plurality of symptom monitors, wherein
the syndrome identifier identifies a combination of symptoms
exhibited by a potentially infected individual in the at least one
candidate, wherein the combination of symptoms indicates a
syndrome, and wherein the syndrome identifier verifies the
plurality of symptom monitors; a position tracker operatively
connected to the syndrome identifier for continuously tracking the
potentially infected individual; and a response system for
transmitting the tracking information of the potentially infected
individual and the results of the syndrome identifier to a
responder.
10. The system of claim 9, wherein the plurality of symptom
monitors comprise at least one of a temperature monitor, a motion
monitor, and an audio monitor.
11. The system of claim 10, wherein the temperature monitor
comprises an infrared video camera.
12. The system of claim 10, wherein the motion monitor comprises a
conventional video camera.
13. The system of claim 10, wherein the audio monitor comprises a
phased audio array.
14. The system of claim 9, wherein the results of verifying the
plurality of symptom monitors is transmitted to the plurality of
symptom monitors for enhancing sensitivity and selectivity of the
plurality of symptom monitors.
15. The system of claim 9, wherein the syndrome identifier
continuously transmits the location of the potentially infected
individual to the position tracker.
16. The system of claim 9, wherein one of the plurality of symptom
monitors receives assistance from the other of the plurality of
symptom monitors for detecting the plurality of symptoms.
17. The system of claim 9, wherein the system monitor verifies the
plurality of symptom monitors comprises the system monitor verifies
one of the plurality of symptom monitors based on the others of the
plurality of symptom monitors.
18. The system of claim 9, wherein the response system wirelessly
transmits the tracking information of the potentially infected
individual and the results of the syndrome identifier to the
responder.
19. The system of claim 9, wherein the response system transmits
the tracking information of the potentially infected individual and
the results of the syndrome identifier to a portable receiving unit
operated by the responder.
20. A machine-readable medium having instructions stored thereon
for execution by a processor to perform a method for syndromic
surveillance, the method comprising: detecting syndromic data
associated with infectious disease and sickness among a plurality
of people using a plurality of monitoring devices;
probabilistically fusing the syndromic data from the plurality of
monitoring devices; locating individuals exhibiting the syndromic
data; verifying the locations of the individuals and the syndromic
data for consistency among the plurality of monitoring devices; and
transmitting the location of a potentially infected individual
exhibiting the probabilistically fused syndromic data to a
responder for tracking the potentially infected individual.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. Provisional
Application No. 60/538,347, which was filed on Jan. 22, 2004, and
which is fully incorporated herein by reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] The present invention relates generally to the field of
syndromic surveillance, and, more particularly, to automated
behavioral video and audio monitoring for infectious diseases.
[0004] 2. Description of the Related Art
[0005] Fear of infectious diseases and bioterrorism events is a
cause for concern among many in today's social landscape. For
example, during November 2002 through July 2003, a total of 8,098
people worldwide became sick with severe acute respiratory syndrome
("SARS") that was accompanied by either pneumonia or respiratory
distress syndrome (probable cases), according to the World Health
Organization ("WHO"). Of these, 774 died. For another example,
between September and October 2001, letters containing Bacillus
anthracis (known commonly as "anthrax") were received by mail in
several parts of the United States. A widespread scare of handling
United States mail soon ensued.
[0006] Fortunately, infectious outbreaks for both of the above
examples were contained before the public health was substantially
jeopardized. However, it is well-accepted that future outbreaks of
infectious disease and bioterrorism events, which can rapidly
spread without notice, may not be so easily contained without
earlier detection and response. Controlling rapidly-spreading,
infectious diseases and sicknesses require that infected
individuals be promptly identified, treated and quarantined, if
necessary, to prevent further outbreak.
[0007] Syndromic surveillance is a methodology for finding
individuals who manifest a syndrome (i.e., a collection of
symptoms) associated with a disease or sickness. For infectious
diseases, syndromic surveillance can provide a source of infection
and prevent further outbreak. For bioterrorism events, syndromic
surveillance can provide early detection of widespread
dissemination of bioterrorism agents, prompting life-saving
treatment of those infected and timely containment of the deadly
agents.
[0008] Traditional syndromic surveillance technology generally
relies on manual reporting of symptoms, syndromes, and diagnoses by
health care workers. One difficulty with this approach is that a
patient may wait before diagnosis and treatment of the infectious
disease or sickness. If the patient waits too long, the patient may
not receive potentially life-saving treatment in time. If a patient
has an infectious disease, the wait may allow the patient to infect
several other people prior to seeking adequate health care. If a
patient is sick because of a bioterrorism event, the wait may
postpone the determination by health officials of the bioterrorism
event. These difficulties, among others, substantially limit the
effectiveness of traditional syndromic surveillance.
SUMMARY OF THE INVENTION
[0009] In one aspect of the present invention, a method for
syndromic surveillance is provided. The method includes the steps
of detecting syndromic data associated with infectious disease and
sickness among a plurality of people using a plurality of
monitoring devices; probabilistically fusing the syndromic data
from the plurality of monitoring devices; locating individuals
exhibiting the syndromic data; verifying the locations of the
individuals and the syndromic data for consistency among the
plurality of monitoring devices; and transmitting the location of a
potentially infected individual exhibiting the probabilistically
fused syndromic data to a responder for tracking the potentially
infected individual.
[0010] In another aspect of the present invention, a system for
syndromic surveillance is provided. The system includes a plurality
of symptom monitors, wherein the plurality of symptom monitors
detect a plurality of symptoms among a plurality of people, and
wherein at least one of the plurality of symptom monitors detects a
location for at least one candidate from the plurality of people
exhibiting at least one of the plurality of symptoms; a syndrome
identifier operatively connected to the plurality of symptom
monitors, wherein the syndrome identifier identifies a combination
of symptoms exhibited by a potentially infected individual in the
at least one candidate, wherein the combination of symptoms
indicates a syndrome, and wherein the syndrome identifier at least
one of verifies and assists the plurality of symptom monitors; a
position tracker operatively connected to the syndrome identifier
for continuously tracking the potentially infected individual; and
a response system for transmitting the tracking information of the
potentially infected individual and the results of the syndrome
identifier to a responder.
[0011] In yet another aspect of the present invention, a
machine-readable medium having instructions stored thereon for
execution by a processor to perform a method for syndromic
surveillance is provided. The method includes the steps of
detecting syndromic data associated with infectious disease and
sickness among a plurality of people using a plurality of
monitoring devices; probabilistically fusing the syndromic data
from the plurality of monitoring devices; locating individuals
exhibiting the syndromic data; verifying the locations of the
individuals and the syndromic data for consistency among the
plurality of monitoring devices; and transmitting the location of a
potentially infected individual exhibiting the probabilistically
fused syndromic data to a responder for tracking the potentially
infected individual.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The invention may be understood by reference to the
following description taken in conjunction with the accompanying
drawings, in which like reference numerals identify like elements,
and in which:
[0013] FIG. 1 depicts a method for syndromic surveillance, in
accordance with one exemplary embodiment of the present invention;
and
[0014] FIG. 2 depicts a system for syndromic surveillance, in
accordance with one exemplary embodiment of the present
invention.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0015] Illustrative embodiments of the invention are described
below. In the interest of clarity, not all features of an actual
implementation are described in this specification. It will of
course be appreciated that in the development of any such actual
embodiment, numerous implementation-specific decisions must be made
to achieve the developers' specific goals, such as compliance with
system-related and business-related constraints, which will vary
from one implementation to another. Moreover, it will be
appreciated that such a development effort might be complex and
time-consuming, but would nevertheless be a routine undertaking for
those of ordinary skill in the art having the benefit of this
disclosure.
[0016] While the invention is susceptible to various modifications
and alternative forms, specific embodiments thereof have been shown
by way of example in the drawings and are herein described in
detail. It should be understood, however, that the description
herein of specific embodiments is not intended to limit the
invention to the particular forms disclosed, but on the contrary,
the intention is to cover all modifications, equivalents, and
alternatives falling within the spirit and scope of the invention
as defined by the appended claims.
[0017] It is to be understood that the systems and methods
described herein may be implemented in various forms of hardware,
software, firmware, special purpose processors, or a combination
thereof. In particular, at least a portion of the present invention
is preferably implemented as an application comprising program
instructions that are tangibly embodied on one or more program
storage devices (e.g., hard disk, magnetic floppy disk, RAM, ROM,
CD ROM, etc.) and executable by any device or machine comprising
suitable architecture, such as a general purpose digital computer
having a processor, memory, and input/output interfaces. It is to
be further understood that, because some of the constituent system
components and process steps depicted in the accompanying Figures
are preferably implemented in software, the connections between
system modules (or the logic flow of method steps) may differ
depending upon the manner in which the present invention is
programmed. Given the teachings herein, one of ordinary skill in
the related art will be able to contemplate these and similar
implementations of the present invention.
[0018] We present, in exemplary embodiments of the present
invention, novel systems and methods for syndromic surveillance
that can automatically monitor symptoms that may be associated with
the early presentation of a syndrome (e.g., fever, coughing,
sneezing, runny nose, sniffling, rashes). Although not so limited,
the novel surveillance systems described herein can be placed in
common areas occupied by a crowd of people, in accordance with
local and national laws applicable to such surveillance. Common
areas may include public areas (e.g., an airport, train station,
sports arena) and private areas (e.g., a doctor's waiting room).
The monitored symptoms may be transmitted to a responder (e.g., a
person, an information system) outside of the surveillance system,
such that the responder can take appropriate action for
identifying, treat and quarantine potentially infected individuals,
as necessary.
[0019] Referring now to FIG. 1, a method 100 for syndromic
surveillance is shown, in accordance with one exemplary embodiment
of the present invention. Syndromic data associated with infectious
disease and sickness is detected (at 105) among a plurality of
people using a plurality of monitoring devices. The detected data
may include physical data associated with the infectious disease or
sickness. For example, temperature-related symptoms (e.g., fever,
hypothermia) may be detected using an infrared video device,
audible symptoms (e.g., coughing, sneezing, sniffling) may be
detected using an audio monitoring device, and visual symptoms
(e.g., head movement associated with coughing and sneezing, runny
nose, rashes, watery eyes) may be detected using a video monitoring
device. It should appreciated that many symptoms, such as coughing
and sneezing, exhibit themselves in a variety of manners (e.g.,
audio, visual).
[0020] Although not so limited, coughing and fever are used
throughout the present disclosure for the sake of simplicity. It
should be appreciated that any of a variety of symptoms associated
with any of a variety of infectious diseases and sicknesses may be
monitored using any of a variety of monitoring devices, as
contemplated by those skilled in the art.
[0021] The syndromic data from the plurality of monitoring devices
is probabilistically fused (at 110). For example, a subset of the
syndromic data detected from a single individual may indicate a
syndrome with the individual. Not all individuals will indicate all
possible symptoms of a syndrome. Further, certain symptom
indications may not be as severe as others. Thus, the accuracy of
the syndrome determination may be indicated using a probabilistic
scale. Information necessary to determine the probabilistic scale
may be obtained from any of a variety of resources, such as medical
literature.
[0022] The individuals exhibiting the syndromic data are located
(at 115). In one embodiment, as described in greater detail below,
the individuals may be located, for example, using data collected
from a plurality of monitoring devices, such as a audio monitoring
device and a video monitoring device.
[0023] The locations of the individuals and the syndromic data are
verified (at 120) for consistency among the plurality of monitoring
devices. For example, an audio monitoring device and a video
monitoring device may each independently determine the location of
a coughing source. The audio monitoring device may determine
location, for example, by positioning a sufficient number of
receivers in a room to determine the angle and direction of the
cough. The video monitoring device may determine location, for
example, by dividing the monitored area into a coordinate system.
In one embodiment, the coordinate-based system and the positioning
of the receivers may be synchronized accordingly. By comparing the
results from the audio monitoring device with the video monitoring
device, the accuracy and reliability of the detected data can be
verified. For example, if the determined angle and direction of a
cough from the audio monitoring device is consistent with the
location from the video monitoring device of a jerking head
associated with the cough, then the coughing data may be determined
to be accurate and reliable. It should be appreciated that an
increased number of monitoring devices in a particular area may
provide more accurate and reliable results.
[0024] The location of a potentially infected individual exhibiting
the probabilistically fused syndromic data is transmitted (at 125)
to a responder for tracking the potentially infected individual. In
one embodiment the responder may be a health official receiving the
location of the potentially infected individual and the syndromic
data through a portable response system that continuously tracks
the potentially infected individuals. In an alternate embodiment,
the responder may be an automated response system. By tracking the
infected individuals, the responder can easily identify, treat and
quarantine, if necessary, the potentially infected individuals.
[0025] Referring now to FIG. 2, a syndromic surveillance system 200
is shown, in accordance with one embodiment of the present
invention. The syndromic surveillance system 200 includes an audio
cough tracker 205, a cough head motion tracker 210, and a fever
monitor 215. A syndrome identifier 220 is operatively connected to
the audio cough tracker 205, the cough head motion tracker 210, and
the fever monitor 215. The syndrome identifier 220, as shown in
FIG. 2, can receive data from and transmit data to the audio cough
tracker 205, the cough head motion tracker 210, and the fever
monitor 215. The syndrome identifier 220 is operatively connected
to a position tracker 225. The position tracker 225 is operatively
connected to a response system 230. The response system 230 is
operatively connected to a position system 235 and responder.
Although not so limited, as shown in FIG. 1, the position system
235 wirelessly transmits data to the responder.
[0026] As shown in FIG. 2, the audio cough tracker 205 is
operatively connected to phased audio array 240. The phased audio
array 240 includes a plurality of receivers for receiving sound
candidates. The phased audio array 240 may be present, for example,
in a room in a three-dimensional configuration. It should be
appreciated that the phased audio array 240 may be configured in
any of a variety of configurations, as contemplated by those
skilled in the art. Ideally, the phased audio array 240 should be
configured in such a manner that necessary information can be
properly derived from the sound candidates. For example, in one
embodiment, the phased audio array 240 may be configured such that
the angle and direction of the sound candidates can be accurately
determined.
[0027] When the audio cough tracker 205 receives a sound candidate,
the sound candidate may be a sound mixture that includes background
noise, such as crowd noise, in addition to a plurality of audible
symptoms. In such a case, the audible symptoms (in this case,
coughing sounds) may be separated from each other and from the
background noise using any of a variety of demixing and noise
reduction techniques known to those skilled in the art, such as
blind source separation. The accuracy of blind source separation
techniques may depend on a good directionality of the sound
candidate in combination with the number of sound sources active at
a given time. Thus, a good configuration of the phased audio array
240 may be imperative for accurate blind source separation.
[0028] The audio cough tracker 205 may differentiate between
different types of coughs as well as between a cough sound and a
non-cough sound by comparing a detected audible symptom with known
data, such as a database of audible cough samples. Alternate
embodiments of the present invention may include neural networks,
as well as other artificial intelligence mechanisms and techniques,
for "learning" audible symptoms in a variety of applications.
[0029] The audio cough tracker 205 may determine the location of
the source/sources of the audible symptoms using any of a variety
of source localization techniques known to one skilled in the art.
For example, the phased audio array 240 may be configured in such a
manner that the location of the source can be determined by
analyzing the direction and angle of the sound candidate. The audio
cough tracker 205 may also record the time when each cough
occurs.
[0030] The cough head motion tracker 210 identifies individuals who
show jerking head motions related to coughing. As shown in FIG. 2,
the cough head motion tracker 210 is operatively connected to a
motion video camera. In one embodiment, the cough head motion
tracker 210 may analyze the recorded video for head motions related
to coughing. Heads of individuals in the video may be isolated
using any of a variety of video recognition techniques known to
those skilled in the art.
[0031] The cough head motion tracker 210 may differentiate between
different types of coughing head motions as well as between cough
head motions and non-cough head motions by comparing the recorded
video with known data, such as a database of cough head motions.
Alternate embodiments of the present invention may include neural
networks, as well as other artificial intelligence mechanisms and
techniques, for "learning" cough head motions in a variety of
applications.
[0032] The cough head motion tracker 210 may be an application of
an object tracking system, such as the one provided in U.S. patent
application Ser. No. 10/325,413, entitled "REAL-TIME VIDEO OBJECT
GENERATION FOR SMART CAMERAS," filed on Dec. 20, 2002, which is
incorporated herein by reference.
[0033] The cough head motion tracker 210 may determine the location
of the source/sources of the cough by using any of a variety of
techniques known to one skilled in the art. For example, the entire
physical area recorded by the cough head motion tracker 210 may be
divided into a three-dimensional grid. Thus, the location of the
source may be recorded using a coordinate system based on the
three-dimensional grid. The cough head motion tracker 210 may also
record the time when each cough occurs.
[0034] The fever monitor 215 identifies individuals who may be
showing signs of a fever. As shown in FIG. 2, the fever monitor 215
is operatively connected to an infrared video camera, which can
determine an individual's temperature. The fever monitor 215 may
also locate individuals. For example, the entire physical area
recorded by the fever monitor 215 may be divided into a
three-dimensional grid. Thus, the location of the source may be
recorded using a coordinate system based on the three-dimensional
grid. The fever monitor 215 may also record the time when each
temperature reading is recorded.
[0035] Data from the audio cough tracker 205, the cough head motion
tracker 210, and the fever monitor 215 is transmitted to the
syndrome identifier 220. In one embodiment, the audio cough tracker
205, the cough head motion tracker 210 and the fever monitor 215
may each utilize the data of the other monitoring devices for
verification and assistance via the syndrome identifier 220. For
example, the location of a cougher determined by the cough head
motion tracker 210 may be compared to the location of the cougher
determined by the audio cough tracker 205. Further, data from the
verification may be used to enhance the sensitivity and selectivity
of the monitoring devices. In one embodiment, the location system
used by the audio cough tracker 205 and the cough head motion
tracker 210 may be synchronized. For another example, the location
of a cougher determined by the cough head motion tracker 210 may be
used to assist the audio cough tracker 205 in determining the
location of the cougher.
[0036] The syndrome identifier 220 may analyze the collected data
from the various monitoring devices (e.g., the audio cough tracker
205, the cough head motion tracker 210, and the fever monitor 215)
to determine whether detected individuals have combined symptoms
(i.e., a syndrome), which potentially indicates a more serious
illness. Whether the same individual is detected by more than one
monitoring device may be determined using any of a variety of
factors, such as the location of the detected individual and the
time recorded when the symptom was exhibited.
[0037] The position tracker 225 receives the analyzed data from the
syndrome identifier 220. The position tracker 225 continuously
tracks the individuals identified to have the syndrome(s) detected
by the syndrome identifier. The position tracker 225 may track the
individuals using, for example, the location determining
capabilities of the various monitoring devices operatively
connected to the syndrome identifier 220. As described in greater
detail above, the various monitoring devices may utilize the
capabilities of each other to verify and assist tracking the
location of the identified individuals.
[0038] The output of the position tracker is transmitted to the
response system 230, which, as shown in FIG. 2, transmits the
locations of the identified individuals to a portable position
systems 235 held by the responder. Although not so limited, the
locations of the identified individuals are transmitted wirelessly.
In alternate embodiments, the locations of the identified
individuals are transmitted by other means, such as telephone lines
and Ethernet. The responder may also receive the results of the
syndrome identifier such that the responder knows what type of
syndrome(s) the identified individuals may possess. The responder
can track and rendezvous with possibly-infected individuals.
[0039] It should be appreciated that the components shown in FIG. 2
are only exemplary. For example, components may be combined or
divided depending on a particular implementation. Further, the
number and types of monitoring and tracking devices may differ
depending on a particular implementation.
[0040] The particular embodiments disclosed above are illustrative
only, as the invention may be modified and practiced in different
but equivalent manners apparent to those skilled in the art having
the benefit of the teachings herein. Furthermore, no limitations
are intended to the details of construction or design herein shown,
other than as described in the claims below. It is therefore
evident that the particular embodiments disclosed above may be
altered or modified and all such variations are considered within
the scope and spirit of the invention. Accordingly, the protection
sought herein is as set forth in the claims below.
* * * * *