U.S. patent application number 13/716532 was filed with the patent office on 2014-03-27 for wound measurement on smart phones.
This patent application is currently assigned to INNOVATIVE THERAPIES, INC.. The applicant listed for this patent is INNOVATIVE THERAPIES, INC.. Invention is credited to Tianning Xu.
Application Number | 20140088402 13/716532 |
Document ID | / |
Family ID | 50339529 |
Filed Date | 2014-03-27 |
United States Patent
Application |
20140088402 |
Kind Code |
A1 |
Xu; Tianning |
March 27, 2014 |
WOUND MEASUREMENT ON SMART PHONES
Abstract
A smart phone is used to accomplish measurement of a wound
surface area, when at least one flat marker device, which is a
predefined reference, is placed near the wound and photographed by
a smart phone.
Inventors: |
Xu; Tianning; (Atlanta,
GA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
INNOVATIVE THERAPIES, INC. |
Gaithersburg |
MD |
US |
|
|
Assignee: |
INNOVATIVE THERAPIES, INC.
Gaithersburg
MD
|
Family ID: |
50339529 |
Appl. No.: |
13/716532 |
Filed: |
December 17, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61705404 |
Sep 25, 2012 |
|
|
|
Current U.S.
Class: |
600/407 |
Current CPC
Class: |
A61B 5/6898 20130101;
G06T 7/0012 20130101; A61B 2576/02 20130101; A61B 5/7475 20130101;
A61B 5/0077 20130101; A61B 5/742 20130101; H04N 5/44 20130101; A61B
5/445 20130101; A61B 5/1079 20130101; A61B 5/1072 20130101 |
Class at
Publication: |
600/407 |
International
Class: |
A61B 5/00 20060101
A61B005/00 |
Claims
1. A system for measuring a wound of a patient, comprising: (A) a
set of at least one flat marker device to be placed on the
patient's unwounded skin near the wound, each flat marker device
having a top surface with a predefined pattern; (B) a computerized
system in which is stored in computer-readable form the predefined
pattern and/or the known size of the flat marker device having the
predefined pattern, wherein the computerized system comprises an
imaging device, and wherein the computerized system performs steps
comprising: capturing an image that includes the wound and the set
of at least one flat marker device nearby; processing the captured
image and recognizing a part of the captured image as the
predefined pattern; processing the captured image and recognizing a
part of the captured image as the wound; attributing a boundary to
the recognized wound; after the boundary has been attributed to the
wound, computing a surface area of the wound based on the known
size of the flat marker device.
2. The system of claim 1, wherein the set consists of exactly one
flat marker device.
3. The system of claim 1, wherein the set consists of between 1
flat marker device and 6 flat marker devices.
4. The system of claim 1, wherein the flat marker devices in the
set are identical.
5. The system of claim 1, wherein each flat marker device has a top
surface that consists of an interior surrounded by a contrasting
exterior.
6. The system of claim 1, wherein each flat marker device is square
or rectangular in shape, and has a top surface that consists of an
interior solid color surrounded by a contrasting exterior.
7. The system of claim 6, wherein the top surface consists of a
white interior solid color surrounded by a black contrasting
exterior.
8. The system of claim 1, wherein the step of attributing a
boundary to the wound comprises carrying out an edge-searching
algorithm.
9. The system of claim 1, further comprising a screen or a
display.
10. (canceled)
11. (canceled)
12. (canceled)
13. (canceled)
14. (canceled)
15. The system of claim 1, wherein the computerized system performs
a computing step using a watershed algorithm.
16. (canceled)
17. (canceled)
18. The system of claim 1, wherein the marker on a bottom side
comprises an adhesive backing.
19. The system of claim 1, wherein the marker on a bottom side
includes an adhesive section that only occupies a center section of
the bottom side.
20. (canceled)
21. (canceled)
22. The system of claim 1, wherein the automated system performs
watershed processing followed by contour finding.
23. A wound measurement method, comprising steps each of which is
performed by an automated system of: initializing a camera
including initialized buffers for image processing, obtaining
captured frame data and converting to Bitmap format, converting to
a gray picture and filtering out noise, edge detection processing,
contour finding, and finding marker objects and computing an
average area of the marker objects.
24. The method of claim 23, wherein the marker objects are square
objects.
25. (canceled)
26. (canceled)
27. The method of claim 26, including using an obtained touch point
as a wound sample, using neighboring location of the squares as a
normal skin sample, and performing Watershed edge detection.
28. The method of claim 27, including finding contours and choosing
all convex contours.
29. The method of claim 28, including drawing a result on a scratch
buffer, finding a contour that encloses all other contours, and
using the found contour as a boundary of the wound.
30. The method of claim 29, including computing an area of the
wound, comparing the computed area to an average area of the
squares, and obtaining an absolute wound area measurement.
31. (canceled)
32. (canceled)
33. (canceled)
34. (canceled)
35. (canceled)
36. A wound measurement method, comprising: positioning a set of at
least two marker devices near a wound on a patient but not touching
the wound; taking an image including the wound and the thus
positioned set of marker devices; wherein the image is taken via a
camera or imaging device; and processing the image to arrive at a
computed surface area of the wound, wherein the image-processing
step is performed by a smart phone or other automated device or
processor.
37. (canceled)
Description
FIELD OF THE INVENTION
[0001] The invention relates to the medical arts, more
particularly, to wound measurement.
BACKGROUND OF THE INVENTION
[0002] Medical treatment of a patient with a wound typically calls
for assessment of wound sizes, repeated over time to provide an
indication of the patient's progress. However, existing methods of
wound size assessment suffer from various disadvantages and
inadequacies.
[0003] Conventionally, a medical professional's assessment of wound
size has involved placement of a ruler near the wound. However,
when so using a ruler, often there is not a flat surface and error
is thereby added. A further complication is that a wound tends to
be irregularly shaped, multi-colored, and not susceptible of having
its perimeter automatically defined. A further aspect is that what
is used needs to be sterile.
SUMMARY OF THE INVENTION
[0004] In a preferred embodiment, the invention provides a system
for measuring a wound of a patient, comprising: (A) a set of at
least one flat marker device to be placed on the patient's
unwounded skin near the wound, each flat marker device having a top
surface with a predefined pattern; (B) a computerized system in
which is stored in computer-readable form the predefined pattern
and/or the known size of the flat marker device having the
predefined pattern, wherein the computerized system comprises an
imaging device, and wherein the computerized system performs steps
comprising: capturing an image that includes the wound and the set
of at least one flat marker device nearby; processing the captured
image and recognizing a part of the captured image as the
predefined pattern; processing the captured image and recognizing a
part of the captured image as the wound; attributing a boundary to
the recognized wound (such as, e.g., a step of attributing a
boundary to the wound that comprises carrying out an edge-searching
algorithm); after the boundary has been attributed to the wound,
computing a surface area of the wound based on the known size of
the flat marker device; such as, e.g., an inventive system that
further comprises a screen or a display; an inventive system
comprising disposable marker devices consisting of a material
selected from the group consisting of paper; plastic film; and a
water-resistant material; an inventive system wherein the marker
device is sterile; an inventive system wherein the set of marker
devices is contained within packaging that maintains the marker
devices as sterile until ready for use on the patient; an inventive
system wherein the computerized system is a smart phone; an
inventive system wherein the imaging device is a camera in a smart
phone; an inventive system wherein the computerized system performs
a computing step using a watershed algorithm; an inventive system
wherein the computerized system performs a computing step using
image segmentation; an inventive system including a touch screen;
an inventive system wherein the marker on a bottom side comprises
an adhesive backing; an inventive system wherein the marker on a
bottom side includes an adhesive section that only occupies a
center section of the bottom side; inventive systems wherein the
automated system performs watershed processing followed by contour
finding; and other inventive systems.
[0005] In another preferred embodiment the invention provides a
wound measurement method, comprising steps each of which is
performed by an automated system of: initializing a camera
including initialized buffers for image processing; obtaining
captured frame data and converting to Bitmap format; converting to
a gray picture and filtering out noise; edge detection processing;
contour finding; and finding marker objects (such as, e.g., marker
objects that are square objects; marker objects that are
rectangle-shaped; etc.) and computing an average area of the marker
objects; such as, e.g., inventive methods that include a step of
determining whether the squares are acceptable, and if not
acceptable, then repeating a step of obtaining captured frame data,
and if acceptable, then proceeding to a step of determining whether
a screen is touched (such as methods that include at the step of
determining whether a screen is touched, if not touched, then
repeating a step of obtaining captured frame data, and if touched,
then proceeding to a step of obtaining a touch point); inventive
methods that include using an obtained touch point as a wound
sample, using neighboring location of the squares as a normal skin
sample, and performing Watershed edge detection; inventive methods
that include finding contours and choosing all convex contours;
inventive methods that include drawing a result on a scratch
buffer, finding a contour that encloses all other contours, and
using the found contour as a boundary of the wound; inventive
methods that include computing an area of the wound, comparing the
computed area to an average area of the squares, and obtaining an
absolute wound area measurement; inventive methods that include
displaying the absolute wound area measurement as a displayed
result; inventive methods wherein the camera is contained within a
smart phone; inventive methods wherein the automated system
includes a display screen that is finger-paintable; inventive
methods wherein the automated system performs steps of displaying a
highlighted area proposed as the wound area on the display screen,
followed by receiving a finger-painted command from a medical staff
person whereby the displayed highlighted area is revised; inventive
methods wherein the image is captured via a camera within a smart
phone operated by a medical practitioner who positioned the set of
markers near the wound, positioned the camera perpendicular to the
wound and at a distance where the wound and the markers were
visible in the screen but not extending beyond the screen, and then
operated the smart phone to take the photograph; and other
inventive methods.
[0006] The invention in another preferred embodiment provides a
wound measurement method, comprising: positioning a set of at least
two marker devices near a wound on a patient but not touching the
wound; taking an image including the wound and the thus positioned
set of marker devices; wherein the image is taken via a camera or
imaging device; and processing the image to arrive at a computed
surface area of the wound, wherein the image-processing step is
performed by a smart phone or other automated device or processor;
such as, e.g., inventive methods wherein the image-processing
includes recognizing the marker devices in the image and further
includes recognizing the wound in the image, wherein the
recognizing is performed by the smart phone or other automated
device or processor; and other inventive methods.
BRIEF DESCRIPTION OF FIGURES
[0007] FIG. 1 is a top view of exemplary markers 3a, 3b used in
practicing the invention in an embodiment, positioned near a wound
1.
[0008] FIG. 2 is a perspective view of a medical staff person 20
and a patient 10 with a wound 1 while an inventive wound
measurement method being practiced in an embodiment.
[0009] FIG. 3 is a top view of a display screen of a device 6 used
in practicing an inventive wound measurement method in an
embodiment.
[0010] FIG. 4 (left) was prepared from a photographic top view of a
display screen of a phone device running an application according
to the invention; FIG. 4 (right) represents an initial photograph
of a wound that was itself photographed by the phone in FIG. 4
during development of the invention.
[0011] FIG. 5 was prepared from a photographic top view of a
display screen of a phone device running an application according
to the invention.
[0012] FIG. 6 is a process flow diagram of steps performed by a
computer in an inventive embodiment.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENT OF THE INVENTION
[0013] The inventive methods, systems, products and kits are useful
for measuring a patient's wound, such as wound 1 in FIGS. 1-2, to
give a measurement especially a surface area measurement.
[0014] The invention provides a set of at least one flat marker. In
FIGS. 1-3, a set of two markers is shown for illustrative purposes,
and in some embodiments an inventive set of markers may consist of
exactly one marker. Examples of an inventive set of at least flat
marker are, e.g., a set that consists of exactly one flat marker
device; a set that consists of between 1 flat marker device and 6
flat marker devices; a set wherein the flat marker devices in the
set are identical; etc.
[0015] A marker used in inventive systems, methods, products or
kits should be readily imaged by an imaging device, preferably
readily imaged by a smart phone, iPhone, etc. Examples of a
preferred marker are, e.g., a flat marker device that has a top
surface that consists of an interior surrounded by a contrasting
exterior; a flat marker device that is square or rectangular in
shape, and has a top surface that consists of an interior solid
color surrounded by a contrasting exterior (such as, e.g., a top
surface consisting of a white interior solid color surrounded by a
black contrasting exterior); etc.
[0016] Preferred examples of a size of a marker used in the
invention, are, e.g., that the marker has a surface area of at
least one square inch; that the marker has a surface area of
exactly one square inch; etc. The preferred size of the marker is
one square inch, because, while a bigger marker tends to improve
accuracy, if a bigger marker is used, there might not be enough
room. For European usage, a marker size of 2 square cm might be
preferred.
[0017] An inventive wound measurement method is begun by placing
(such as placing performed by medical personnel) a set of at least
one marker near a wound, such as placing markers 3a, 3b on
unwounded skin 2 near wound 1.
[0018] Markers 3a, 3b have interior sections 4a, 4b and contrast
rims 5a, 5b. A preferred example of a marker is a white-colored
interior and a black contrast rim. Another example of a marker is a
black interior and a light contrast rim.
[0019] Examples of a shape for a marker used in the invention are,
e.g., a square shape, a rectangle shape, etc. In FIG. 1, markers
3a, 3b are depicted as square, which is a preferred shape for the
markers.
[0020] After the markers 3a, 3b are placed on the patient 10 near
the wound 1, a medical professional 20 (such as a doctor, nurse,
etc.) positions device 6 (such as, e.g., a smart phone, an iPhone,
etc.) over the wound 1 and after the wound 1 is positioned in the
screen of the device 6, takes a photo.
[0021] A minimum number of markers to practice the invention is one
marker. Using more than six markers to practice the invention would
not be preferred. The reason to use more than one marker when
practicing the invention is to better handle a situation in which
the camera is not straight up from the area being imaged. Having
more than one marker reduces error, by taking an average.
[0022] For the marker to be flat, and to be kept flat when
positioned on the patient's skin, is preferred, to reduce error. A
marker that conforms to the contours of the patient and loses its
flatness tends to contribute to measurement error.
[0023] Preferably the marker is constructed of a material that
resists curling-up. As a material for constructing the marker, a
synthetic material that does not absorb water is preferred, so that
the marker resists curling-up.
[0024] When constructing a marker, an adhesive backing section on a
marker is preferred. When an adhesive backing is included on a
marker, preferably the adhesive backing is included only at a
center section of the marker.
[0025] For the user (such as the doctor or nurse) photographing the
wound and the set of markers to position the camera straight up
(perpendicular) with respect to the wound is preferred. The medical
professional holding the camera positions the camera as close as
possible to the wound being imaged without losing the wound and the
marker(s) off the screen.
[0026] When writing an application to be loaded on a device for use
in practicing the invention, preferably image processing used for
practicing in the invention includes performance of an image
segmentation method by a processor, computer, or other machine. A
preferred example of an image segmentation method for use in
practicing the invention is an image segmentation method that
comprises computing according to a watershed algorithm. Imaging
segmentation using a watershed algorithm is a method previously
used by others to segment tissue such as segmenting cancer and
non-cancerous tissue to get an edge. A watershed method is
preferred for use in practicing the invention, because a wound
tends to be multi-color, without obvious grey level, and in such a
situation, a watershed method works.
[0027] After watershed processing, preferably contour-finding
processing is performed.
[0028] A preferred device onto which an application is loaded, for
use practicing the invention, is, e.g., a smart phone with a touch
screen.
[0029] Preferably a finger painting application loaded on a device
(such as, e.g., a smart phone, etc.) is used for practicing the
invention.
[0030] The place of practicing inventive methods is not
particularly limited, and includes, e.g., a hospital setting, at a
patient's home, etc. For example, at a patient's home an image of a
wound may be taken using a device, which then emails the image
included in a PDF file to a doctor or other medical
professional.
[0031] In the invention, preferably an image of a patient's wound
only resides in RAM and is not otherwise saved, for patient
privacy.
[0032] The invention may be further appreciated with reference to
the following examples, without the invention being limited
thereto.
Example 1
[0033] The inventor constructed an app that runs on iPhone, Android
phone and Windows phone platforms with an on board camera. A user
is provided with an easy method to access the area of a 2D object
with the captured object image and a reference marker.
[0034] The reference can be a ruler, in this case the user uses
functions of the app to mark the length of the ruler, to thus
obtain the calibration/scale data for the 2D object at the same
distance.
[0035] The app also has automatic recognition capabilities towards
markers of a certain shape (e.g., square). In this case, the
calibration/scale data are obtained whenever such a marker is
within the view of the camera.
[0036] The of-interest 2D object inside the camera view can be
segmented from the background with finger painting/erasing on the
smart phone's touch screen. Once done, the area measurement of the
object can be derived from the scale data and the size of the
painted area.
[0037] The app also provides semi-automatic segmentation to
separate the object from the background. The user can tap the
object on the screen and the app uses the tapping point as an
example to find areas with the same imaging characteristics and
mark them accordingly. For a regular wound surface which usually
has similar color and visible edge the app will mark the whole
wound area until the edge.
[0038] The user can lay more than one marker--usually around the
wound area, the app will average the markers' scale, which is a
good way to reduce the error caused by off axis (i.e., camera lens
axis not perpendicular to the wound surface).
Example 1A
[0039] The inventor performed experimentation with an app that he
had created and loaded onto a T-Mobile phone. Three flat, square,
black-colored markers were placed on a surface around a
to-be-measured test surface area. The phone (with the app created
by the inventor thereon) was used to take a photograph of the three
black markers, and the device processed the photographed images of
the three markers and the to-be-measured test surface area. On the
screen, each square black marker is outlined in pink. The app
green-highlighted the to-be-measured test surface area, computed
its surface area, and displayed its computed surface area as 9.47
in.sup.2 on the screen. The screen was photographed, and is
represented by FIG. 5 (in which cross-hatching was substituted in
some areas of the photograph, for better reproducibility).
Example 1B
[0040] The inventor performed experimentation with an app that he
had created and loaded onto a T-Mobile phone. Two flat, square
markers were placed on a photograph of a wound having a
predominantly yellow-colored section and also some red sections.
Each marker had a white interior with purple-lavender-colored trim.
The phone was positioned by the user so as to image the photograph
of the wound, and the app computed a surface area of the wound and
displayed a result on screen of 3.44 in.times.in.
Example 1C
[0041] The inventor performed experimentation with an app that he
had created and loaded onto a T-Mobile phone. A red wound area
poorly defined against a red background area was the subject of the
experimentation. The user used the phone, and the phone captured an
image. The app displayed, in green color, a proposed perimeter of
the wound, and also displayed, in green color, a large circular
dot, to communicate to the user what area of the image would be
defined as being the wound and included when surface area was
computed. The screen was photographed, and is represented by FIG. 4
(in which cross-hatching was substituted in some areas of the
photograph, for better reproducibility).
Example 2
[0042] In this inventive example, which may be further appreciated
with reference to FIG. 6, a computer, a processor, an automated
device, a machine, or a combination thereof, performs steps as
follows:
[0043] a step 100 of initializing a camera (such as a camera of a
device 6) and initializing buffers for image processing; followed
by
[0044] a step 110 of getting captured frame data, and converting to
Bitmap format; followed by
[0045] a step 120 of converting to grey picture, and filtering out
noise; followed by
[0046] a step 130 of performing edge detection; followed by
[0047] a step 140 of finding contours; followed by
[0048] a step 150 of finding square objects, and calculating an
average of their area; followed by
[0049] determining 151 whether the squares are good, and if NO,
then returning to step 110, and if YES, determining 152 whether the
screen is touched, and if NO, then returning to step 110, and if
YES, then proceeding to
[0050] a step 160 of getting a touch point, using the obtained
touch point as a wound sample, using a neighboring location of the
squares as the normal skin sample, and performing Watershed edge
detection; followed by
[0051] a step 170 of finding contours, and choosing all convex
contours; followed by
[0052] a step 180 of drawing the result on a scratch buffer,
finding the contour that encloses all the other contours, and using
it as the boundary of the wound; followed by
[0053] a step 190 of calculating the area of the wound, comparing
it to the average area of the squares, and getting the absolute
wound area measurement; followed by
[0054] a step 191 of displaying the result (such as displaying a
computed numerical quantity that is a wound surface area onto a
display screen of device 6).
[0055] While the invention has been described in terms of a
preferred embodiment, those skilled in the art will recognize that
the invention can be practiced with modification within the spirit
and scope of the appended claims.
* * * * *