U.S. patent application number 15/397088 was filed with the patent office on 2017-07-06 for system and method of identification of the heart valve signals.
This patent application is currently assigned to AventuSoft, LLC. The applicant listed for this patent is AventuSoft, LLC. Invention is credited to Mahdi Esfahanian, Luis Gonzalo Sanchez Giraldo, Kaustubh Kale, Diego Pava.
Application Number | 20170188868 15/397088 |
Document ID | / |
Family ID | 59235187 |
Filed Date | 2017-07-06 |
United States Patent
Application |
20170188868 |
Kind Code |
A1 |
Kale; Kaustubh ; et
al. |
July 6, 2017 |
SYSTEM AND METHOD OF IDENTIFICATION OF THE HEART VALVE SIGNALS
Abstract
A system for identifying heart valve signals includes a
non-invasive sensor unit for capturing electrical signals and
composite vibration objects, a memory containing computer
instructions, and one or more processors coupled to the memory,
where the one or more processors are configured to perform the
steps of separating a plurality of individual heart vibration
events from the composite vibration objects by using at least one
among bin-wise clustering and permutation alignment, or
non-negative matrix factorization, or deep belief networks and
tagging the plurality of individual heart vibration events using at
least one among principal component analysis, Gabor filtering,
generalized cross correlation, phase transform, smoothed coherent
transformation, Roth correlation or band filtering.
Inventors: |
Kale; Kaustubh; (Royal Palm
Beach, FL) ; Giraldo; Luis Gonzalo Sanchez; (Miami,
FL) ; Pava; Diego; (Coral Springs, FL) ;
Esfahanian; Mahdi; (Boca Raton, FL) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
AventuSoft, LLC |
Boca Raton |
FL |
US |
|
|
Assignee: |
AventuSoft, LLC
Boca Raton
FL
|
Family ID: |
59235187 |
Appl. No.: |
15/397088 |
Filed: |
January 3, 2017 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62274765 |
Jan 4, 2016 |
|
|
|
62274761 |
Jan 4, 2016 |
|
|
|
62274763 |
Jan 4, 2016 |
|
|
|
62274766 |
Jan 4, 2016 |
|
|
|
62274770 |
Jan 4, 2016 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A61B 5/6801 20130101;
A61B 5/725 20130101; A61B 5/7203 20130101; A61B 5/1102 20130101;
A61B 5/04014 20130101; A61B 5/046 20130101; A61B 5/7225 20130101;
A61B 7/04 20130101; A61B 5/0816 20130101; A61B 5/1107 20130101;
A61B 5/02438 20130101; A61B 5/04325 20130101; A61B 2562/0204
20130101; A61B 5/04017 20130101; A61B 5/0468 20130101; A61B 2562/06
20130101; A61B 5/0456 20130101; A61B 5/02055 20130101; A61B 5/0536
20130101; A61B 5/7275 20130101; A61B 5/0404 20130101; A61B 5/7214
20130101; G16H 40/67 20180101; A61B 5/6802 20130101; G16H 50/30
20180101; A61B 5/6833 20130101; A61B 5/02444 20130101; A61B 5/0022
20130101; A61B 5/0472 20130101; A61B 5/0044 20130101; A61B 5/7282
20130101; A61B 5/0205 20130101; A61B 5/02028 20130101; A61B 5/0006
20130101; A61B 2562/0219 20130101; A61B 5/7246 20130101; A61B
5/0008 20130101; A61B 5/7257 20130101; A61B 5/024 20130101; A61B
5/6831 20130101 |
International
Class: |
A61B 5/0452 20060101
A61B005/0452; A61B 5/0432 20060101 A61B005/0432; A61B 5/024
20060101 A61B005/024; A61B 5/04 20060101 A61B005/04; A61B 5/00
20060101 A61B005/00 |
Claims
1. A system for identifying heart valve signals, comprising: a
non-invasive sensor unit for capturing electrical signals and
composite vibration objects; a memory containing computer
instructions; and one or more processors coupled to the memory, an
execution of the computer instructions by the one or more
processors causing the one or more processors to perform operations
comprising: separating a plurality of individual heart vibration
events from the composite vibration objects by using at least one
among bin-wise clustering and permutation alignment, or
non-negative matrix factorization, or deep belief networks; and
tagging the plurality of individual heart vibration events using at
least one among principal component analysis, Gabor filtering,
generalized cross correlation, phase transform, smoothed coherent
transformation, Roth correlation or band filtering.
2. The system of claim 1, wherein the non-invasive sensor unit
comprises at least one or more sensors using a tri-axial
accelerometer configured for placement on different points of a
torso.
3. The system of claim 1, wherein the system uses one or more among
spectral information, relations among channels, and relations among
events in a form of relative times of occurrence of heart valve
events.
4. The system of claim 1, wherein the non-invasive sensor unit
provides sensor signals that are filtered, cross correlated, and
peak detected to provide an estimate of a delay
5. The system of claim 1, further comprising a sensor for
performing an electrocardiogram and wherein the non-invasive sensor
unit comprises at least one sensor for sensing a heart valve
opening.
6. The system of claim 1, wherein a permutation alignment is used
for source separation.
7. The system of claim 1, wherein the system uses Gabor analysis on
source separated signals and then source tags the source separated
signals.
8. The system of claim 1, wherein the processor is configured to
perform Gabor analysis on source separated signals using a finite
Gabor dictionary of fixed frequencies and with variable phase
delay.
9. The system of claim 1, wherein the system comprises a Gabor
dictionary that serves as a basis representation of the source
signals, an optimization algorithm that determines a delay to
minimize reconstruction error, and collection and organization of
features extracted from the optimization algorithm.
10. The system for measuring cardiac time intervals of claim 1,
wherein the non-invasive sensor unit comprises at least one sensor
for sensing vibrations corresponding to a heart valve opening.
11. The system for measuring cardiac time intervals of claim 1,
wherein a number of vibration sensors in the non-invasive sensor
unit is less than the number of the separated vibration
sources.
12. The system of claim 1, wherein the processor uses Principal
Component Analysis (PCA) to find timing information and delay
between channels and to determine which source is associated with
which heart valve signal event.
13. The system of claim 1, wherein the marking of individual valve
events comprises at least the marking of one or more among a Mitral
valve opening (MO), Aortic valve opening (AO), Tricuspid valve
opening (TO), or Pulmonary valve opening (PO).
15. The system of claim 1, wherein the system uses a feature
extraction algorithm based on multichannel Gabor basis
decomposition using matching pursuit, and a second algorithm that
uses eigen-decomposition of a covariance matrix extracted from the
source signals to obtain a cross correlation function between
channels.
16. A sensor array device, comprising: a wearable device configured
to capture an electrocardiogram signal synchronized with composite
vibration objects from at least one accelerometer and further
configured to communicate with a wireless node; wherein the at
least one accelerometer is configured for capturing the composite
vibration objects; an electrode for sensing the electrocardiogram
signal; and one or more processors configured for: separating a
plurality of individual heart vibration events from the composite
vibration objects by using at least one among bin-wise clustering
and permutation alignment, or non-negative matrix factorization, or
deep belief networks; and tagging the plurality of individual heart
vibration events using at least one among principal component
analysis, Gabor filtering, generalized cross correlation, phase
transform, smoothed coherent transformation, Roth correlation or
band filtering.
17. The sensor array device of claim 16, wherein the at least one
accelerometer are configurable for measuring a lower frequency
range vibration signal and a higher frequency range vibration
signal.
18. The sensor array device of claim 16, wherein the processor is
operatively coupled to the at least one accelerometer, the
processor further being configured for: identifying the plurality
of individual heart vibration events from the composite vibration
objects; transmitting the composite vibration signals or the
plurality of individual heart vibration events to a remote device;
and marking and presenting individual valve events from the
plurality of individual heart vibration events with respect to the
electrocardiogram signal.
19. The sensor array device of claim 16, wherein the one or more
processors are configured to uses Gabor analysis on source
separated signals.
20. The sensor array device of claim 16, wherein the sensor array
device is portable and captures synchronized sensor data to a
memory and wherein the processor is configurable in the process of
separating the plurality of individual heart vibration events from
the composite vibration objects into separate vibration sources and
further identifying the individual heart vibration events among at
least one of a mitral valve opening, a tricuspid valve opening, an
aortic valve opening, or a pulmonary valve opening.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the priority benefit of Provisional
Application Nos. 62274765, 62274761, 62274763, 62274766, and
62274770, each of which were filed on Jan. 4, 2016, the entire
disclosure each are incorporated herein by reference.
FIELD
[0002] The embodiments herein relate generally to cardiopulmonary
health monitoring and more particularly to analysis software
combined with transducers to capture multi-channel vibration
signals along with an electrocardiogram signal for the measurement
of heart functions.
BACKGROUND
[0003] Heart disease is the leading cause of death accounting for
more than one-third (33.6%) of all U.S. deaths. Overall cardiac
health can be significantly improved by proper triage. Low invasive
and non-invasive ultrasound techniques (e.g. echocardiogram) are
standard procedures, but the requirement of expensive devices and
skilled operators limit their applicability. The following are the
various types of heart disease that can be diagnosed and treated
using the separated signal, namely, Coronary artery disease, Heart
murmurs and valve abnormalities, Heart failure, Heart rhythm
abnormalities (arrhythmias), Vascular disease, congenital heart
disease, Cardiac resynchronization and Risk factor modification. A
physician can work with patients to perform a comprehensive
evaluation and design a personalized plan of care aimed at keeping
them healthy.
[0004] The cardio pulmonary system which consists of the
respiratory components, snoring components, and cardiac components,
creates vibrations during each cardiac cycle. The vibrations are
the result of the lung sounds, heart sounds, acceleration and
deceleration of blood due to abrupt mechanical opening and closing
of the heart valves during the cardiac cycle.
SUMMARY
[0005] The exemplary embodiments herein provide a method and system
based on a technique to identify the separated cardiopulmonary
signals and to extract information contained in vibration objects.
In some embodiments, machine learning, auditory scene analysis, or
spare coding are used as approaches to the source separation
problem. Data is obtained using a tri-axial accelerometer or
multiple tri-axial accelerometers placed on different points of a
torso. Further note that the techniques and methods herein are not
limited to acoustic, electrical or vibrational data as might be
used in some stethoscopes, but can also be applied to other forms
of monitoring such as echo imaging or sonograms, magnetic resonance
imaging (MRI), computed tomography (CT) scanning, positron emission
tomography (PET) scanning, and monitoring using various forms of
catheterization. The techniques and methods herein are primarily
applicable to monitoring of heart valve events, but can be
alternatively applied to other types of involuntary biological
signaling emanating from the brain, intrauterine, pre-natal
contractions, or elsewhere within both humans and other
species.
[0006] Examples of cardiac vibration objects are the first sound,
the second sound, the third sound, the fourth sound, ejection
sounds, opening sounds, murmurs, heart wall motions, coronary
artery sounds, and valve sounds of the Mitral valve opening and
closing, Aortic valve opening and closing, Pulmonary valve opening
and closing, Tricuspid valve opening and closing. Examples of the
pulmonary vibration objects are the respiratory lung sounds,
breathing sounds, tracheobronchial sounds, vesicular sounds,
Broncho vesicular sounds, snoring sounds. A portion of the energy
produced by these vibrations lies in the infra-sound range, which
falls in the inaudible and low sensitivity human hearing range. For
example, the vibration objects from the Mitral, Tricuspid, Aortic,
and Pulmonary valve openings fall in a lower range of vibrations
such as 0 to 60 Hertz, whereas vibration objects from the Mitral,
Tricuspid, Aortic, and Pulmonary valve closings fall in a higher
range of vibrations such as 50 to 150 Hertz. A portion of the
energy produced by these vibrations falls in the audible hearing
range. Accelerometer transducers placed on the chest capture these
vibrations from both these ranges.
[0007] Source identification analysis in accordance with the
methods described herein identify individual vibration objects
described above from the source separated vibration signals. The
individual vibration signals are identified to be from the mitral
valve, aortic valve, tricuspid valve, the pulmonary valve, coronary
artery, murmurs, third sound, fourth sound, respiratory sound,
breathing, and snoring during individual heart beats. The
identified signals are marked to indicate their start with respect
to the start of the electrocardiogram or EKG.
DESCRIPTION OF THE DRAWINGS
[0008] FIG. 1A illustrates a system for the extraction,
identification, marking and display of the heart valve signals in
accordance with one embodiment;
[0009] FIGS. 1B and 1C illustrate cardio pulmonary signal capture
at the chest in accordance with various embodiments;
[0010] FIG. 2 is a flowchart of a method practiced by the system in
accordance with one embodiment;
[0011] FIG. 3 illustrates multichannel signals captured from the
sensor array on the chest shown in accordance with one
embodiment;
[0012] FIG. 4 illustrates a cardiac cycle in relation with
Electrocardiogram, acoustic and accelerometer sensors of the system
in accordance with one embodiment;
[0013] FIG. 5 illustrates a schematic for the source identification
in accordance with one embodiment;
[0014] FIGS. 6A, 6B, 6C, 6D, 6E, 6F and 6G illustrate the
identification of various sources from separated sources in
accordance with one embodiment;
[0015] FIGS. 7A, 7B, and 7C illustrate the source identification of
various heart valve signals across the different heart beats in
accordance with one embodiment;
[0016] FIGS. 8A and 8B illustrate the source identification of
various heart valve signals with reference to an electrocardiogram
in accordance with one embodiment;
[0017] FIG. 9 illustrates the pseudo code for the classification
algorithm for source identification in accordance with one
embodiment.
DETAILED DESCRIPTION
[0018] The exemplary embodiments may be further understood with
reference to the following description and the appended drawings,
wherein like elements are referred to with the same reference
numerals. The exemplary embodiments describe a system and method of
source identification of the individual cardiopulmonary signals
from the composite vibrations captured on the chest wall.
Specifically, psychoacoustics are considered in identifying the
cardiac vibration signals captured through the transducers. The
system, the psychoacoustics, and a related method will be discussed
in further detail below.
[0019] The embodiments can include different source identification
techniques specifically used for tagging individual cardiopulmonary
signals for application in a non-linear time variant system, such
as Principal component analysis, Gabor filtering, Generalized Cross
Correlation (GCC), Phase transform (PHAT), ROTH, SCOT and Band
Filtering. The techniques herein can use one or more among 1)
Spectral information, 2) Relations among channels, and 3) Relations
among events in the form of relative times of occurrence.
[0020] The exemplary embodiments provide a novel approach for
small, portable, robust, fast and configurable source separation
based software with transducer hardware 103, 203. The use of the
vibration signal pattern and novel psychoacoustics help bypass
conventional issues faced by linear time invariant systems.
[0021] The following are the various types of heart disease that
can be diagnosed and treated using the identifies signals, namely,
Coronary artery disease, Heart murmurs and valve abnormalities,
Heart failure, Heart rhythm abnormalities (arrhythmias), Vascular
disease, congenital heart disease, and Risk factor modification. A
physician can work with patients to perform a comprehensive
evaluation and design a personalized plan of care aimed at keeping
them healthy.
[0022] The exemplary embodiments of the system and method proposed
here are shown in FIGS. 1A, 1B, and 1C. System 100 shown in FIG. 1
is an embedded platform which can be any smart processing platform
with digital signal processing capabilities, application processor,
data storage, display, input modality like touch-screen or keypad,
microphones, speaker, Bluetooth, and connection to the internet via
WAN, Wi-Fi, Ethernet or USB. This embodies custom embedded
hardware, smartphone, iPad-like and iPod-like devices. Area 101 in
FIGS. 1A and 1B is the auditory scene at the chest locations. Array
102 in FIGS. 1A and 1B is a transducer array used to capture the
heart signal. In some embodiments, the transducer array includes a
pad that includes a vibration sensor such as a vibration sensor
102b and an electrode 102a for an ECG sensor. In some embodiments,
the transducer array can include a single pad, two pads as shown in
FIG. 1B or more than two pads as shown in FIG. 1C. In the
particular embodiment of FIG. 1C, a transducer array 110 includes
three pads (102) where each pad includes the vibration sensor 102b
and the ECG electronic 102a. Other embodiments can include three or
more pads where each pad would have at least a vibration sensor and
optionally an electrode for the ECG sensor. Hardware 103 in FIGS.
1A-C is the wearable microprocessor hardware with digital signal
processing capabilities, application processor, analog-to-digital
frontend, data storage, input modality like buttons, and wireless
connection via Bluetooth, Bluetooth low energy, near field
communication transceiver, Wi-Fi, Ethernet or USB. Processor 112
shown in FIG. 1C comprises of the signal processing module on the
wearable device that captures synchronized sensor data from the
transducer array 102. The processor 112 is configured to save the
synchronized sensor data to memory and communicate it with the
system 100 for data transfer. 105 in FIG. 1A is the module that
calculates vital sign from the input sensor stream coming from
hardware 103 for the Heart rate, breathing rate, EKG signal, skin
temperature, and associated vitals. The hardware 103 can optionally
encrypt the raw sensor data for transmission to the cloud computing
module 106. It can also communicate with a dashboard on a computing
device 106 for data exchange, login, alerts, notifications, display
of processed data. Computing device 106 in FIG. 1A can be the cloud
module that processes the individual streams for eventual source
separation, identification and marking of the heart valve signals.
The system 100 allows a user to visually see the individual streams
and information from the heart valves. In some embodiments, the
system 100 could include a connected display or any other modality
of display or presentation device. In some embodiments, the system
100 allows a user to visually see the individual streams and
information from the heart valves and in some embodiments the
system could present streams or information on a connected display
or any other modality of display. The transducer array 102 can
include multiple sensor transducers that capture the composite
signal that includes the electrocardiogram signals, heart sounds,
lung sounds and snoring sounds for example. The module 103 can be
in the form of wearable hardware that synchronously collects the
signals across the transducers and is responsible for the analog to
digital conversion, storage and transmission to a portable unit
104. Note that the embodiments herein are not limited to processing
the individual streams for source separation, identification and
marking of the heart valve signals at the cloud computing module
106 only. Given sufficient processing power, the aforementioned
processing can occur at the microprocessor hardware module 103, at
the module 105, or at the cloud-computing module 106, or such
processing can be distributed among such modules 103, 105, or
106.
[0023] The exemplary embodiments of the system and method proposed
here for the source identification of the cardiopulmonary signals
200 are shown in FIG. 2. Block 201 indicates the separation of
sources from the composite signals. Block 202 represents the phase
estimation between the separated sources at each of the sensor
position. Block 203 represents calculating the time stamps of
individual sources at each heartbeat with respect to the
synchronized EKG signal and the other sensor or sensors. Block 204
represents the source identification module responsible for tagging
each of the separated source in individual heart beats to be one of
the heart valve event, namely Mitral valve closing and opening,
Tricuspid valve closing and opening, Aortic valve opening and
closing, and the Pulmonic valve opening and closing. Block 205
represents the time marking module to estimate the time of
occurrence of the above mentioned valve events with respect to the
start of the EKG signal.
[0024] The exemplary embodiments of the system and method proposed
here for the source identification of the cardiopulmonary signals
from the composite signal 300 are shown in FIG. 3. Area(s) 101 in
FIG. 1B indicate the locations at which the composite heart signal
can be captured. A vibration signal 302 as charted on the first
line in FIG. 3 represents a signal captured at the aortic
auscultation location. A vibration signal 303 shows the vibration
signal captured at the pulmonic auscultation location. A vibration
signal 304 shows the vibration signal captured at the tricuspid
auscultation location. A vibration signal 305 represents a
vibration signal captured at the mitral auscultation location. The
last or bottom line in FIG. 3 represents an electrocardiogram
signal 306 captured. In some embodiments, note that the number of
sensors used (such as in the sensor array 102 of FIG. 1), are less
than the number of vibration sources. For example, 3 sensors can be
used to ultimately extract signals for 4 (or more) vibration
sources; or 2 sensors can be used to ultimately extract signals for
3 or 4 (or more) vibration sources; or 1 sensor can be used to
ultimately extract signals for 2, or 3, or 4 (or more) vibration
sources.
[0025] The exemplary embodiments of the system and method proposed
here draw inspirations from biology with respect to the cardiac
cycle in-relation with electrocardiogram and accelerometer
transducer captured cardiac signal. A timeline chart 400 in FIG. 4
shows a cardiac cycle. Lines or signals 401a, 401b, and 401c
represent or indicate the pressure changes during a cardiac cycle
for aortic pressure (401a), atrial pressure (401b) and ventricular
pressure (401c) measured in measured in millimeters of mercury
(mmHg). Line or signal 402 represents or indicates the volume
changes during a cardiac cycle in milliliters (ml). Line or signal
403 represents or indicates the electrical changes during a cardiac
cycle captured by an electrocardiogram. Line or signal 404
represents or indicates the acoustic changes during a cardiac cycle
captured by an acoustic sensor such as a phonocardiogram or PCG. S1
represents the first heart sound or the "lub" sound and the S2
represents the second heart sound or "dub" sound. Line or signal
405 represents or indicates the vibration changes during a cardiac
cycle captured by an accelerometer transducer at the location of
the device. Pattern 406 in FIG. 4 indicates the different valve
opening and closing seen in line or signal 405 as captured by the
accelerometer sensor or sensors. More specifically, a closer
inspection of the pattern 406 reveals the closing of the mitral
valve (M1) and tricuspid valve (T1) during the S1 or first heart
sound and the closing of the aortic valve (A2) and pulmonary valve
(P2). The bottom half of FIG. 4, goes on to further show a
representation of the anatomy of the human heart relevant for the
generation of the sounds and a corresponding graph representing the
sounds belonging to coronary artery, murmurs, first sound, second
sound, third sound, fourth sound, ejection sounds, opening sounds,
respiratory sound, breathing, and snoring during individual heart
beats, with respect to the electrocardiogram signal.
[0026] The exemplary embodiments of the system and method proposed
here provide a source identification analysis algorithm that allows
for the tagging of the vibrations from the cardiopulmonary system.
The sensors can be modeled as:
x.sub.1(t)=s.sub.1(t)+n.sub.1(t)
x.sub.2(t)=.alpha.s.sub.1(t+D)+n.sub.2(t),
[0027] Where x are the sensor signals and s and n are the sources
and the noise respectively (assumed to be uncorrelated). A system
500 can include filtering or weighting (using transforms 501 (H1)
and 502 (H2)) of the respective input sensor signals X1 and X2.
Cross correlating the filtered sensor signals (using blocks 504,
505, and 506) and finding the peak using peak detector 507 will
result in the estimate of the delay D as can be seen in the block
diagram of FIG. 5. The cross correlation between x1(t) and x2(t) is
related to the cross power spectral density function by the
well-known Fourier transform relationship. The process takes the
filters 501 and 502 into account, which can be considered a
weighting function. This weighting function and the delay 503 is
used to compensate for the smearing effect of the Fourier
Transform. The weighting function can be of different forms
depending on the conditions of the signal, predominant between the
weighting functions are: GCC-ROTH, GCC-SCOT, GCC-PHAT, PASSBAND
FILTERING (The weighting function is an ideal pass band filter that
eliminates all frequencies above and bellow the expected
range).
.psi. R ( f ) = 1 G x 1 x 1 ( f ) .psi. s ( f ) = 1 / G x 1 x 1 ( f
) G x 2 x 2 ( f ) . .psi. p ( f ) = 1 G x 1 x 2 ( f ) , .
##EQU00001##
The chart 600 of FIG. 6A, shows the solid lines with manual marking
of the M1 and T1 (the first and second vertical lines,
respectively) showing peaks in the composite Mitral auscultation
site. The adjacent broken lines correspond to the maximum point of
the signals obtained by source separation. A feature similar to
GCC-PHAT can be obtained from the aligned cluster centers shown in
chart 610 of FIG. 6B. Each signal is represented with a different
color or shading in grayscale. The peak of each one of the separate
colored curves (represented by the vertical lines in FIG. 6B)
corresponds to the delay estimate of arrival between two sensors
for each one of the sources. Permutation alignment is important for
source separation. It can also help with the identification
problem. Since the cluster centers are related to source locations.
If we compute the phase differences between two sensors, we can
obtain a histogram of the phase differences for the full recording
as depicted for a mixture of 4 sources as shown in FIG. 620 of FIG.
6C. The relative times of arrival are clearly different for the M1
and T1 sounds as shown in the charts 630 of FIG. 6D and chart 640
of FIG. 6E. This is a clear indicator of the different location of
the sources. Analysis of the propagation of sources can be based on
signal markings. First notice the periodic pattern, which is more
evident for the relative TOA of the T1 sound. This pattern
correlates with the breathing intervals of the subject as shown in
charts 650 of FIG. 6F and 660 of FIG. 6G. It suggests that
propagation speed is directly affected by breathing. Propagation is
not time invariant while working with cardiopulmonary signals.
[0028] One of the proposed embodiments of the source identification
is using Gabor analysis on the source separated signals. After the
signals have been separated the next step is to identify them, this
process is usually known as source tagging. The need to do source
tagging arises from the permutation problem caused by the source
separation algorithm. The separated sources obtained by the source
separation are similar to the original sources up to a scale and a
permutation factor.
[0029] In another embodiment, the source can be identified by
manually tagging them against a synchronized EKG signal and taking
advantage of the timings relative to the QRS wave. This way,
however is usually slow and time consuming and an automatic tagging
algorithm is thus preferable. Since the different heart sounds
comes from different locations in the heart, it is expected that
each source will have a unique phase relation between the sensors
that are located at fixed points during the data gathering phase.
This phase relation can be made evident by using signal
representation with a dictionary suited for highlighting frequency
and phase relations with a greedy algorithm such as the matching
pursuit algorithm. One of the proposed algorithms herein performs a
Gabor analysis in the source separated signals using a finite Gabor
Dictionary of fixed frequencies and with variable phase delay. The
system finds the delay that minimizes the reconstruction error for
each source and creates a group of features that is incorporated
into a decision-making and classification algorithm. The
classification algorithm will combine the features extracted by the
Gabor analysis with other features that comes from the Principal
Component Algorithm (PCA) and cross correlation analysis that uses
a set of manually tagged patient's tracks as training. The system
is composed of three modules or stages: A suitable Gabor dictionary
is created that can serve as the basis representation of the
signals. An optimization algorithm aims to find the delay that will
minimize the reconstruction error in a giving delay range.
Collection and organization of all the features extracted from the
second stage. For each one of the stages different techniques were
tried in order to achieve the best results. The Gabor dictionary
selected has the function:
G ( .theta. , n ) = e - 1 ( n - .theta. ) 2 2 .sigma. 2 cos ( 4
.pi. n - .theta. f ) ##EQU00002##
In the equation 8 is the delay, a is the Gaussian decay of the atom
and f is the frequency of operation of the Gabor. The system
initially searches for the Gabor atoms that best represent the
signal by doing a sweep in frequency and selecting those Gabor
atoms that produced a larger weight matrix after the matching
pursuit. The system was subsequently changed to a group of fixed
frequency Gabor atoms exploiting the fact that for the M1, T1, A2
and P2 sounds vibration reside mainly in the range of 50-300 Hz and
the Aortic opening vibration resides mainly in the low frequency
range (from 0-50 Hz). The fixed Gabor approach sacrifices a little
bit of reconstruction error in favor of a faster computation and
easier analysis since the frequencies were fixed. The first
optimization technique used was the, Gradient Descent algorithm: A
first-order optimization algorithm that looks for the minimum of a
given function by taking steps proportional to the negative of the
gradient. The system is very dependent on the initial guess and
number of minima of the system. A second approach was using the
minimize function, which uses the Polack-Ribiere algorithm of
conjugate gradients to compute search directions, and a line search
using quadratic and cubic polynomial approximations. The system is
also dependent on the initial value of the search but is more
efficient in choosing the parameters to find the minimum. Finally,
a brute force approach was implemented that sweeps the signal with
different delays and then selects the one that resulted in a
minimum function. In order to improve speed on this algorithm, a
broad search is done first and then a refined search is done around
the minimum point found by the broad search.
[0030] The exemplary embodiments of the system and method proposed
here provide a source identification algorithm for the vibrations
from the cardiopulmonary system. Referring to the chart 700 of FIG.
7A, in order to find the time stamps for events such as Mitral
closing & opening, Tricuspid closing & opening, Aortic
opening & closing, Pulmonic opening and closing, we look at all
the individual source separated signals 701 to 706 of the composite
signal 707 and first try to find the location of max peak in the
source separated (SS) signal for each source and then find delay
between two channels. The chart 710 of FIG. 7B shows the frequency
spectrum of the source separated signals. Cross correlated
vibrations in aortic and pulmonic channels for each interval for
each source are calculated to find a consistent delay between two
channels as represented by chart 720 of FIG. 7C. Given the start of
QRS complex (indicative of ventricular depolarization triggered by
contraction of the ventricles) and end point of each vibration, the
vibration(s) within this interval is cross correlated with all
vibrations in each source. This is done for both aortic and
pulmonic channels. At the end, Principle Component Analysis (PCA)
was applied to find the timing information and delay between two
channels. PCA uses (separated source) SS signal from each source in
each channel to find the template which represents a majority of
the vibrations within that source. The template is then cross
correlated with the whole source and maximum of PCA signal in each
interval is found and compared with the start of QRS. In another
implementation, in the second attempt, SS signals from two channels
but the same source are fed into the PCA to find the template. Then
the aortic template is used for both channels' cross correlation to
identify the different vibrations into valve events, breathing
sounds, and vibrations of the heart walls. To accurately estimate
M1, T1, A2, P2, AO, PO from the frequency signal captured by
digital accelerometer on the wearable, several sub-frequencies are
considered. Some are noted here for example: 0-30 Hz, 0-60 Hz,
30-150 hz, 30-250 Hz.
[0031] The exemplary embodiments of the system and method proposed
here provide a source marking algorithm for the vibrations from the
cardiopulmonary system. Next step is to use the PCA to determine
which source is associated with which event (Mitral closing &
opening, Tricuspid closing & opening, Aortic opening &
closing, Pulmonic opening and closing). The architecture for
automatic source tagging and timing of valvular events is further
described. One way to identify which events are relevant to a
source is by manually tagging the sources against the synchronized
EKG signal and taking advantage of the timings relative to the QRS
wave (identification of the S1 and S2 sounds using the EKG signal
as the reference has been widely researched in studies. Another
approach is an automatic tagging algorithm. The tagging is composed
of a classifier preceded by a feature extraction algorithm. For the
timing, the system can exploit the computations of one of the
feature extraction algorithms to obtain an energy contour from
which the time location of a given event can be inferred. Because
the embodiments herein build upon having the ability to capture the
signal at different locations simultaneously, to the system
exploits the relations among channels to extract additional
information about the sources. Likewise, since some source
separation algorithms where channels relations are associated with
location, the system can leverage on the intrinsic relations among
the channels to extract relevant information that helps distinguish
among the events. The system herein hypothesizes that phase
information between channels is relevant for distinguishing among
cardiac events since valves are located at different positions
within the heart. Perhaps, one of the most distinctive features of
the cardiac events is their relative order of occurrence, which
repeats periodically with each heartbeat. Time information
extracted from the set of sources can be utilized to localize the
occurrence of each source signal within the heart cycle. Therefore,
the features proposed here are conceived to provide three aspects:
1) Spectral information, 2) Relations among channels, and 3)
Relations among events in the form of relative times of occurrence.
In some embodiments, the system includes a feature extraction
algorithm based on multichannel Gabor basis decomposition using
matching pursuit, and a second algorithm that uses
eigen-decomposition of a covariance matrix extracted from the
source signals to obtain a cross correlation function between
channels. For the Gabor basis, consider the reconstruction of a
2-channel signal x (t). In this approach to the system can extract
a phase relation between the sensors which are located at standard
aortic and pulmonic auscultation positions. As mentioned above, the
system is interested on features that reflect spectral content as
well as the channel interrelations. In the two channel case, each
basis function is a pair of Gabor functions with equal frequency f
and envelope width h and a phase difference .theta.. The tunable
parameter .theta. can be swept across a range of values for which
the matching pursuit reconstruction error can be obtained. The
behavior of the reconstruction error for a source signal over
different values of .theta. provides information about the phase
different between the two channels for the particular source
signal. In particular, the value of .theta. that attains the
minimum error within the defined range of values is taken as the
optimal channel delay, and the time average of the activation
coefficients obtained by matching pursuit at the optimal .theta.
provide and spectral characterization of the source. For the
feature Extraction by Self-Similarity Template Based-on PCA, we
define a self-similarity using the concept of eigenfilter. Let yi
the a two-channel source signal defined in (1), and let Yi its
discretized version of size Nx2. For filter of length Nw, we
compute the tap-delay matrices .DELTA.(j). The joint-channel
eigenfilter correspond to the leading right-singular vector u of
the tap-delay centered matrix .DELTA.i obtained by removing the
mean of each column of .DELTA.i. The eigenfilter u can be split
into the first and second channel components. The first channel
component u(1) corresponds to the first Nw entries of u, and u(2)
to the remaining entries going from index Nw+1 to last index 2Nw.
The proposed feature is the cross-correlation function between u(1)
and u(2). Note that this function contains the main frequency
components of the source signals expressed in the time domain and
the peak provides information about the channel relations. For
different delays between the two channels the cross correlation
peaks at different locations, accordingly. To extract relative time
information contains cues for the classification of the cardiac
events, we adopt a simple approach that uses the energy contours.
The process consists of three basic steps: Compute energy contours
for all source signals, Compute timings for each source signal,
Compute features of source i by averaging the sum of the remainder
sources centered at the timings. The energy of each source signal
Yi is calculated using the leading, quadrature pair of right
singular vectors of their corresponding tap-delay matrix. For
classification, a set of training exemplars (sources) have been
manually tagged with the respective events. Each source signal is
then represented by the feature vectors described above. Tests have
been carried out on pooled covariance linear discriminant analysis,
quadratic discriminant analysis, k-nearest neighbours and support
vector machines both linear kernel and Gaussian kernel. The
features described in the previous sections can be used in
different manners for tagging, for example: Tagging using only
Gabor-based features, Tagging using only self-similarity features,
Tagging using only time-based features, Tagging using all
Gabor-based, self-similarity, and time-based features. There are
even sub cases of the situations considered above, For instance,
the combination of Gabor and self-similarity features can be done
on a single classifier or using ensembles. The first validation
examples only address the cases in the bullet items. In addition to
the classifier, preprocessing of the feature vectors is also
performed. A centering vector x.sub.cent is calculated using the
mean over the training set x.sub.train. This step is followed by a
linear transformation based on PCA to produce a much smaller
dimension feature vector. The preprocessing is summarized in
Algorithm 1, in FIG. 9. The choice of p is typically driven by the
training data itself. The singular values of x.sub.train can be
used to decide the dimensionality of the transformed feature
vectors after.
[0032] The exemplary embodiments of the system and method proposed
here provide a source marking algorithm that allows from the
explanation earlier for the marking of the Mitral valve closing
(MC), Mitral valve opening (MO), Aortic valve opening (AO), Aortic
valve closing (AC), Tricuspid valve closing (TC), Tricuspid valve
opening (TO), Pulmonary valve closing (PC) and Pulmonary valve
opening (PO) signals. The extracted individual valve vibration
objects are aligned into a signal for each of the four valves
across multiple heart beats. The chart 800 in FIG. 8A shows the
source separation of heart valve opening and closing signals. Line
801 indicates length or duration of the vibration signal for the
Mitral valve closing (M1). Line 802 indicates length or duration of
the vibration signal for the Tricuspid valve closing (T1). Line 803
indicates the length or duration of the vibration signal for the
Aortic valve closing (A2). Line 804 indicates the length or
duration the vibration signal for the Pulmonic valve closing (P2).
Line 805 indicates the composite vibration signal captured by a
particular transducer. Signal 806 indicates the EKG signal captured
by the system. Referring to chart 810 of FIG. 8B, the line 807
indicates the length or duration of the vibration of the Aortic
valve opening (AO). Line 808 indicates the length or duration of
the vibration of the Pulmonic valve opening (PO). Further note that
the lines or signals 809 in FIG. 8A or 811 in FIG. 8B are actually
several separated superimposed signals representing the vibration
signals from separate sources coming from the mitral valve,
tricuspid valve, aortic valve, and pulmonary valve (using less than
4 vibration sensors to extract such separated signals in some
embodiments).
[0033] The exemplary embodiments of the system and method proposed
here provide a source marking algorithm for the vibrations from the
cardiopulmonary system, using information about the time of
occurrence of the event. Automated finding of M1, T1, A2 & P2:
After going through the timing plots of different patients, it was
decided to first determine A2 and P2 timing and their corresponding
sources and then find M1 and T1 from the rest of sources. Automatic
A2 & P2 finding: 1) The number of zero-crossings is found for
each SS source. Then the noisy source that is associated with the
maximum zero-crossing is discarded. 2) Automated calculation of QRS
onset points for all heartbeats in each source. 3) Automated
calculation of beginning and ending points of all vibrations in
each source. 4) Applying PCA approach on all sources that outputs
the time difference between QRS onset and peak of PCA signal
(timing vector) as well as delay between aortic and pulmonic
channels. 5) Based on probability density estimation of each
source's timing vector, strong peak(s) that correspond(s) to
timing(s) more than 300 ms are accepted as A2 and P2 candidates and
those sources that don't satisfy this condition are discarded. Also
new candidate timings are sorted in ascending order. 6) Variation,
length of samples more than 300 ms (L) and some other measurements
are calculated from accepted timing vectors of accepted sources and
are stored in a structure for further analysis. 7) If there is only
one strong peak in probability density estimation plot whose timing
is more than 300 ms, then a variable called "ind_both" is set to 1.
If there are two or more strong peaks whose timing is more than 300
ms, if max peak's timing is over 300 ms, then "ind_both" variable
is set to 1 but if max peak's timing is lower than 300 ms but
another strong peak's timing is more than 300 ms, then "ind_both"
is set to 0. If there are two or more sources whose "both_indic"
variables are equal to 1, then "both_indic"=1.8) In case of having
no valid timing, A2 and P2 are set to zero. 9) In case of having
one valid timing, if L>6 then it is marked as A2 and its
corresponding source number is also saved. P2 is set to zero. 10)
In case of having two valid timings:
[0034] if both_indic=1,
[0035] if L1>=6 samples, then A2 is set to first timing in
sorted timing vector Else A2=0
[0036] if A2=0 and L2>=6, then A2 is set to second timing in
sorted timing vector Else P2=0
[0037] if A2.about.=0 and L2>=6, then P2 is set to second timing
in sorted timing vector Else P2=0
if both_indic.about.=1,
[0038] if L1>=4 samples & V1<=800 then A2 is set to first
timing in sorted timing vector Else A2=0
[0039] if A2=0 and L2>=4 & V1<=800, then A2 is set to
second timing in sorted timing vector Else P2=0
[0040] if A2.about.=0 and L2>=6 & V2<=800 then P2 is set
to second timing in sorted timing vector Else P2=0
[0041] 11) In case of having more than two valid timings,
if both_indic=1 for loop: if L1>=6 & V1<=800, then A2 is
set to current timing in sorted timing vector; break Else A2=0; End
of for loop
[0042] if A2=0, then P2=0;
[0043] if A2.about.=0 & last timing in the sorted timing
vector
[0044] if L>=6 & V1<=800, then P2 is set to last timing
in the sorted timing vector
[0045] if A1.about.=0 & there are two timings after A1
timing,
[0046] if V2<V1, then P2 is set to second timing after A2.
[0047] Else P2 is set to first timing after A2.
[0048] if A1.about.=0 & there are more than two timings after
A1 timing,
[0049] if L1>=6 & V1<=800, then P2 is set.
if both_indic .about.=1, for loop: if L1>=8 samples &
V1<=1000 then A2 is set to current timing in sorted timing
vector Else A2=0; End of for loop.
[0050] if A2=0, then P2=0;
[0051] if A2.about.=0
for loop: if L>=8 & V1<=1000,
[0052] then P2 is set to current timing in the sorted timing
vector; break
[0053] Else P=0;
End of for loop At the end, A2 and P2 timings and their
corresponding source numbers can be saved in an excel spreadsheet
along with the patient name. Automatic M1 & T1 finding: [0054]
1) The same previous steps from 1 to 7 are implemented in this
phase [0055] 8) In case of having no valid timing, M1 and T1 are
set to zero. [0056] 9) In case of having one valid timing, if
L>6 and timing is less than 120 ms, then it is marked as M1 and
its corresponding source number is also saved. [0057] P2 is set to
zero. [0058] 10) In case of having two valid timings, if
both_indic=1,
[0059] if L1>=6, then M1 is set to first timing in the sorted
timing vector Else M1=0
[0060] if M1=0 & L2>=6, then M1 is set to second timing in
the sorted timing vector and T1=0
[0061] if M1.about.=0 & L2>=6 and second timing is less than
double the first timing
[0062] then T1=second timing in the sorted timing vector Else
T1=0
if both_indic .about.=1 L1>=4 & V1<=800, then M1 is set
to first timing in the sorted timing vector Else M1=0 If M1=0 &
L2>=4 & V2<=800, then M1=second timing in the sorted
timing vector Else T1=0 If M1.about.=0 & L2>=4 &
V2<=800, then T1=second timing in the sorted timing vector Else
T1=0
[0063] 11) In case of having more than two valid timings,
if both_indic .about.=1, for loop: if L>=8 & V<=1000,
then M1 is set to current timing in the sorted timing vector Else
M1=0; break; end of for loop
[0064] if M1=0, then T1=0;
for loop: if M1.about.=0 & L>=6 & V<=1000, then T1 is
set to current timing in the sorted timing vector; break, end of
for loop if M1.about.=0 & L2>=6 and second timing is less
than double the first timing
[0065] then T1=second timing in the sorted timing vector Else
T1=0
if both_indic=1 low_var=Find sources with variations less than 15.
From these low_var sources, find those whose length are more than 9
samples (len_low_var). If length of low_var>=2 &
len_low_var>=2
[0066] M1 is set to the timing corresponding to the lowest
variation
[0067] T1 is set to the timing corresponding to the second lowest
variation
Else
[0068] for loop: If L>=5 & V<800, then M1 is set to the
current timing in the sorted timing vector Else M1=0 If M1=0, then
T1=0
If M1.about.=0
[0069] for loop: If there is only one timing after M1 & L>=5
& V<=800, then T1=current timing in the sorted timing
vector; break If there are two timings after M1, if V1<300, then
T1=first timing after M1 in the sorted timing vector; break Else if
V2<V1, then T1=second timing after M1 in the sorted timing
vector; break Else, T1=first timing after M1 in the sorted timing
vector; break If there are more than two timings after M1, if
V1<5, then T1=current timing after M1 in the sorted timing
vector; break Else if V2<V1, then T1=current timing after M1 in
the sorted timing vector; break Else, T1=current timing after M1 in
the sorted timing vector; break
Automated Finding of Aortic Opening (AO):
[0070] Track 1 with 2 channels 8 by 2 by N [0071] M1, T1 timings
and EKG times [0072] Remove the noisy source using max
zero-crossing. Data down to 7 by 2 by N [0073] Sub-source markings
including start and end of vibrations, envelope and max peak [0074]
Save to non-zero-source vector for both channels [0075] Remove
missing start and end points and too large ones by comparing
between two channels as well as considering each source [0076]
Calculate PCA for two channels and all vibrations. Outputs are
delay_vec, PCA signal and delay between two channels
Begin AO Detection
[0076] [0077] Select above data for AO channel [0078] Remove zero
delay values [0079] Remove delays between two channels that are
greater than 50 ms and count <4 [0080] Remove positive delays
between two channels and count <5 [0081] Calculate ksdensity
[0082] Find max peak as well as other peaks that their distance
together is more than 5 samples [0083] Keep negative delays if
delay is less than 150 ms [0084] Calculate ksdensity of event_vec
[0085] Find more max peak and peaks >half of max peak [0086]
Find peaks >10 ms and with 0.05 of max probability value [0087]
Save all the candidate sources with negative delay btw two channels
[0088] Select peaks that fall in range of M1 and M1+95 ms (if
M1=T1=0, in range 20 to 150 ms) [0089] Sort all timings and save
[0090] Find AO
First Case:
[0091] If there is only on candidate source: [0092] 1) If delay
variation <500 ms but source variation >1000 ms, [0093] 2)
remove timings >200 ms, [0094] 3) If length of samples >4,
remove samples whose timings relative to median are >60 ms
[0095] 4) If empty timing vector: I. then get the original timing
vector, calculate kdensity and find max and other peaks whose
probability is more than half of max's probability II. If time diff
btw AO candidate and max peak's timing <10 ms,
[0096] set max's peak timing as ref time, remove samples with diff
more than 50 ms relative to ref time and update the AO candidate
and its variation.
III. If time diff btw AO candidate and max peak's timing >10 ms,
find a qualified peak and its corresponding timings that is closest
to AO candidate, set that peak's timing as ref time, remove samples
with diff more than 50 ms relative to ref time and update the AO
candidate and its variation. [0097] 5) If non-empty timing vector:
[0098] I. calculate kdensity and find max and other peaks whose
probability is more than half of max's probability. Find the one
that is closest to candidate, set is as ref time, remove samples
with diff more than 50 ms relative to ref time and update the AO
candidate and its variation. [0099] 6) Check if var_delay <500
&& var_SRC<900 && AO_candidate <=M1+96
Second Case:
[0100] If there are two candidate sources:
The following steps from the first case are implemented: [0101] 1,
2, 3 with 50 ms threshold, and 6 [0102] After founding all valid AO
timings from all candidate sources, the closet AO candidate to
"M1+30 ms" is selected.
Third Case:
[0103] If there are more than two candidate sources:
The following steps from the first case are implemented: [0104] 1,
2, 3 with 90 ms threshold, 4 with 90 ms threshold, and 6 [0105]
After founding all valid AO timings from all candidate sources, the
closet AO candidate to "M1+30 ms" is selected.
[0106] To improve the results in the exemplary embodiment the
system can perform AO detection using a new PCA approach such as:
Mean removal, Min-Max removal, or Different window length for
correlation. Detailed analysis on delay calculation from
cross-correlation template. Source tagging using PCA features and
different classifiers on both tagged and untagged tracks for both
High Pass and Low Pass. Defined two timing calculation algorithm 1)
with threshold 2) without threshold and got the timing
distributions for different test cases. To improve AO detection
using new PCA algorithm, a few different tests were done such as:
Removed the mean of each source's template (X matrix) before
calculating covariance matrix. For some sources, cross-correlation
template shows zero delay at a discontinued point although there is
a delay between peaks of templates. Removed min-max of eigenvector
before performing cross-correlation showing a continuous behavior
with a delay. Then ran different SSs on the same patient to see if
these observations are consistent. There are different scenarios
when calculating the delay such as: Maximum peak is positive or
negative, The Delay of Max peak is positive or negative. The
approach herein is to select positive maximum peak with the
positive delay. The goal in some embodiments is to tag or label the
sources with an event (HP: M1, T1, A2, P2, LP: AO). The
cross-correlation signals generated by new PCA algorithm are used
as features to train the classifiers and then tag sources of a test
signal. A similar approach can be extended for the identification
of other cardiopulmonary signals, like coronary artery sounds,
snoring, murmurs, respiration sounds in one embodiment.
[0107] The exemplary embodiments develop a novel method of source
identification, which in one embodiment using the Pulmonary and
Aortic, and in addition possibly the Tricuspid and Mitral
auscultation locations, lends the system capable of identifying
individual cardiopulmonary events from the vibrations.
[0108] The exemplary embodiment of system and method described is
the development on an embedded hardware system, the main elements
required to capture body sounds are the sensor unit that captures
the body sounds, digitization, and digital processing of the body
sounds for noise reduction, filtering and amplification. Of course,
more complicated embodiments using the techniques described herein
can use visual sensors, endoscopy cameras, ultrasound sensors, MRI,
CT, PET, EEG and other scanning methods alone or in combination
such that the monitoring techniques enable improvement in terms of
source separation or identification, and/or marking of events such
as heart valve openings, brain spikes, contractions, or even
peristaltic movements or vibrations. Although the focus of the
embodiments herein are for non-invasive applications, the
techniques are not limited to such non-invasive monitoring. The
techniques ultimately enable diagnosticians to better identify or
associate or correlate detected vibrations or signaling with
specific biological events (such as heart valve openings and
closings, brain spikes, fetal signals, or pre-natal
contractions.)
[0109] It will be apparent to those skilled in the art that various
modifications may be made in the present invention without
departing from the spirit or scope of the invention. Thus, it is
intended that the present invention cover the modifications and
variations of this invention provided they come within the scope of
the method and system described and their equivalents.
* * * * *