U.S. patent application number 14/134957 was filed with the patent office on 2014-07-03 for object information obtaining device, display method, and non-transitory computer-readable storage medium.
This patent application is currently assigned to CANON KABUSHIKI KAISHA. The applicant listed for this patent is CANON KABUSHIKI KAISHA. Invention is credited to Hiroshi Abe, Koichi Suzuki.
Application Number | 20140182383 14/134957 |
Document ID | / |
Family ID | 51015648 |
Filed Date | 2014-07-03 |
United States Patent
Application |
20140182383 |
Kind Code |
A1 |
Suzuki; Koichi ; et
al. |
July 3, 2014 |
OBJECT INFORMATION OBTAINING DEVICE, DISPLAY METHOD, AND
NON-TRANSITORY COMPUTER-READABLE STORAGE MEDIUM
Abstract
An object information obtaining device includes a light source
which emits light, an acoustic wave detecting unit which detects a
photoacoustic wave generated by irradiation of an object with the
light, and outputs an electric signal in response to detection of
the photoacoustic wave, and a processing unit configured to perform
two or more types of processing to photoacoustic signal data based
on the electric signal to obtain object information corresponding
to each of the two or more types of processing, and to display on a
display unit the object information corresponding to at least one
processing selected by a user out of the two or more types of
processing.
Inventors: |
Suzuki; Koichi;
(Kodaira-shi, JP) ; Abe; Hiroshi; (Kyoto-shi,
JP) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
CANON KABUSHIKI KAISHA |
Tokyo |
|
JP |
|
|
Assignee: |
CANON KABUSHIKI KAISHA
Tokyo
JP
|
Family ID: |
51015648 |
Appl. No.: |
14/134957 |
Filed: |
December 19, 2013 |
Current U.S.
Class: |
73/655 |
Current CPC
Class: |
A61B 5/0095 20130101;
G01H 9/00 20130101 |
Class at
Publication: |
73/655 |
International
Class: |
G01H 9/00 20060101
G01H009/00 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 28, 2012 |
JP |
2012-286685 |
Claims
1. An object information obtaining device, comprising: a light
source configured to emit light; an acoustic wave detecting unit
configured to detect a photoacoustic wave generated by irradiation
of an object with the light, and to output an electric signal in
response to detection of the acoustic wave; and a processing unit
configured to perform two or more types of processing to
photoacoustic signal data based on the electric signal, and to
display on a display unit object information corresponding to at
least one processing selected by a user out of the two or more
types of processing.
2. The object information obtaining device according to claim 1,
wherein the processing unit is configured to perform the two or
more types of processing including signal processing to transform
the photoacoustic signal data into different photoacoustic signal
data based on the photoacoustic signal data.
3. The object information obtaining device according to claim 2,
wherein the signal processing include probe response correction
processing or noise removal processing.
4. The object information obtaining device according to claim 1,
wherein the processing unit is configured to perform the two or
more types of processing including reconstruction processing to
transform the photoacoustic signal data into the object information
based on the photoacoustic signal data.
5. The object information obtaining device according to claim 4,
wherein the reconstruction processing include at least one of time
domain reconstruction processing, Fourier domain reconstruction
processing, and model base reconstruction processing.
6. The object information obtaining device according to claim 1,
wherein the processing unit is configured to perform the two or
more types of processing including image processing to transform
the object information obtained based on the photoacoustic signal
data into object information different from the object
information.
7. The object information obtaining device according to claim 6,
wherein the image processing includes resolution improvement
processing.
8. The object information obtaining device according to claim 1,
wherein the photoacoustic signal data used in each of the two or
more types of processing includes same data based on the electric
signal which the acoustic wave detecting unit outputs after
detecting the photoacoustic wave at certain time.
9. The object information obtaining device according to claim 1,
wherein the processing unit is configured to start the at least one
processing when the at least one processing is selected by the
user.
10. The object information obtaining device according to claim 1,
further comprising: a storage unit configured to store the object
information corresponding to each of the two or more types of
processing, wherein the processing unit displays the object
information corresponding to the at least one processing stored in
the storage unit on the display unit when the at least one type of
processing is selected by the user.
11. The object information obtaining device according to claim 1,
further comprising: the display unit.
12. The object information obtaining device according to claim 1,
further comprising: an input unit configured to allow the user to
select the at least one type of processing from the two or more
types of processing.
13. The object information obtaining device according to claim 1,
further comprising: the display unit; and an input unit configured
to allow the user to select the at least one type of processing
from the two or more types of processing, wherein the display unit
and the input unit are integrally formed as a single unit.
14. The object information obtaining device according to claim 1,
wherein the processing unit is configured to perform three or more
types of processing to the photoacoustic signal data and display
object information corresponding to at least two types of
processing selected by the user out of the three or more types of
processing on the display unit.
15. A method of displaying object information obtained by an object
information obtaining device, the method comprising: inputting
information of at least one type of processing out of two or more
types of processing; performing the at least one type of processing
which is input to photoacoustic signal data obtained by detecting a
photoacoustic wave generated by a photoacoustic effect; and
displaying object information corresponding to the at least one
type of processing.
16. The method of displaying according to claim 15, wherein the
step of performing the at least one type of processing is executed
after the step of inputting the information of the at least one
type of processing, and the step of displaying the object
information corresponding to the at least one type of processing is
executed after the step of performing the at least one type of
processing.
17. The method of displaying according to claim 15, wherein the
step of inputting the information of the at least one type of
processing is executed after the step of performing the at least
one type of processing, and the step of displaying the object
information corresponding to the at least one type of processing is
executed after the step of inputting the at least one type of
processing.
18. A non-transitory computer-readable storage medium storing
thereon a program for allowing a computer to execute the method of
displaying according to claim 15.
Description
BACKGROUND OF THE INVENTION
[0001] 1. Field of the Invention
[0002] The present invention relates to technology to obtain object
information based on a photoacoustic wave generated by irradiation
of light to an object.
[0003] 2. Description of the Related Art
[0004] Photo acoustic imaging (PAI) in an optical imaging technique
developed based on the photoacoustic effect. In photo acoustic
imaging, for example, an object such as a living body is irradiated
with pulsed light and a light absorber such as a blood vessel
absorbs energy of the pulsed light to generate a photoacoustic
wave. An acoustic wave detecting unit detects the photoacoustic
wave generated by the photoacoustic effect. Then, a detection
signal output from the acoustic wave detecting unit is analyzed by
image processing, for example, and object information is
obtained.
[0005] As an example of photo acoustic imaging, Non-Patent Document
1 entitled "Universal back-projection algorithm for photoacoustic
computed tomography", disclosed by Xu et al., PHYSICAL REVIEW E
71,016706 (2005), discloses obtaining initial sound pressure
distribution as the object information by applying universal
back-projection reconstruction processing (hereinafter, referred to
as "UBP processing") to the detection signal of the photoacoustic
wave.
SUMMARY OF THE INVENTION
[0006] An object information obtaining device disclosed in this
specification is provided with a light source configured to emit
light, an acoustic wave detecting unit configured to detect a
photoacoustic wave generated by irradiation of an object with the
light, and to output an electric signal in response to detection of
the acoustic wave, and a processing unit configured to perform two
or more types of processing to photoacoustic signal data based on
the electric signal to obtain object information corresponding to
each of the two or more types of processing, and to display on a
display unit the object information corresponding to at least one
processing selected by a user out of the two or more types of
processing. Further features of the present invention will become
apparent from the following description of exemplary embodiments
(with reference to the attached drawings).
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] FIG. 1 is a view illustrating an object information
obtaining device according to this embodiment.
[0008] FIG. 2 is a view illustrating a processing unit according to
this embodiment in detail.
[0009] FIG. 3 is a view illustrating a flow of a method of
obtaining object information according to this embodiment.
[0010] FIG. 4A is a view illustrating a simulation model according
to this embodiment.
[0011] FIG. 4B is a view illustrating a simulation result of a
Fourier domain reconstruction processing according to this
embodiment.
[0012] FIG. 4C is a view illustrating a simulation result of a time
domain reconstruction processing according to this embodiment.
[0013] FIG. 4D is a view illustrating a simulation result of a
model base reconstruction processing according to this
embodiment.
[0014] FIG. 5 is a view illustrating a flow of a method of
obtaining object information according to Example 1 of the present
invention.
[0015] FIG. 6 is a view illustrating a processing unit according to
Example 1 of the present invention in detail.
[0016] FIG. 7 is a view illustrating a screen displayed on a
display according to Example 1 of the present invention.
[0017] FIG. 8 is a view illustrating a flow of a method of
obtaining object information according to Example 2 of the present
invention.
[0018] FIG. 9 is a view illustrating a screen displayed on a
display according to Example 2 of the present invention.
DESCRIPTION OF THE EMBODIMENTS
[0019] Object information according to one embodiment includes
initial sound pressure of a photoacoustic wave generated by a
photoacoustic effect, optical energy absorption density derived
from the initial sound pressure, an absorption coefficient, density
of a substance forming tissue and the like. Herein, density of a
substance may be determined by levels of oxygen saturation,
oxyhemoglobin density, deoxyhemoglobin density, total hemoglobin
density and the like. The total hemoglobin density is a sum of the
oxyhemoglobin density and the deoxyhemoglobin density.
[0020] The object information in this embodiment may be not
numerical data but distribution information of each position in an
object. That is to say, the distribution information such as
absorption coefficient distribution and oxygen saturation
distribution may be used as the object information.
[0021] Further improvement in method of displaying the object
information obtained only by specific processing (UBP
reconstruction processing) as disclosed in Non-Patent Document 1 is
desired from a diagnostic viewpoint.
[0022] For example, a real image corresponding to the object might
be displayed in a different manner depending on a type of the
processing. Therefore, usefulness in diagnosis of an observation
object might be different depending on the type of the
processing.
[0023] A virtual image referred to as an artifact might be present
in a diagnostic image obtained through the reconstruction
processing. The artifact might preclude appropriate diagnosis. As
it is known, depending on the type of the reconstruction
processing, artifacts appear differently in a reconstructed
image.
[0024] Therefore, display of object information obtained by the
specific processing alone might be insufficient at the time of
diagnosis.
[0025] In accordance with at least one embodiment of the present
invention, at least one processing is selected by a user from two
or more types of processing to photoacoustic signal data (also
referred to as raw data). According to this, the user may confirm
the object information obtained by desired processing, so that the
user may selectively use the image corresponding to the processing
determined to be useful according to a symptom in the
diagnosis.
[0026] With the object information obtaining device capable of
executing only one specific processing, there is a case in which
processing requiring long processing time should be executed even
though the user wants to see a diagnostic result in a short time.
With the object information obtaining device capable of executing
only the specific processing, there also is a case in which
processing based on a simple model should be executed even though
the user wants to observe detailed information even if it takes
long processing time.
[0027] Therefore, according to an embodiment disclosed herein, the
user may also select the desired processing in consideration of
acceptable processing time to the user. That is to say, according
to this embodiment, the user may select the object information
corresponding to the desired processing determined by the user to
be highly useful within the acceptable processing time to the
user.
[0028] The present embodiment is hereinafter described with
reference to the drawings. In the drawings, the same reference sign
is assigned to the same component, and the description thereof is
not repeated.
[0029] A basic configuration of the object information obtaining
device (information obtaining apparatus) according to this
embodiment illustrated in FIG. 1 is first described.
[0030] The object information obtaining device illustrated in FIG.
1 includes a light source 110, an optical system 120, an acoustic
wave detecting unit 130, a processing unit 140 as a computer, an
input unit 150, and a display unit 160 in order to obtain
information of a living body 100 as the object.
[0031] FIG. 2 is a block diagram illustrating relevant parts of a
computer, which is an example of a data processing apparatus
including the processing unit 140 and peripheral elements of the
processing unit 140. As illustrated in FIG. 2, the processing unit
140 is provided with an arithmetic unit 141 and a storage unit 142.
An example of the processing unit 140 includes, but is not limited
to, a microprocessor chip, such as a CPU (central processing unit)
or MPU (micro processing unit). An example of storage unit 140
includes, but is not limited to, RAM or ROM memory.
[0032] The arithmetic unit 141 controls operation of each component
forming the object information obtaining device through a data
network 200. The arithmetic unit 141 reads a program in which
processing steps for (a method of) obtaining object information to
be described later is saved in the storage unit 142 and allows the
object information obtaining device to execute the method of
obtaining object information.
[0033] Each component of the object information obtaining device
according to this embodiment is hereinafter described in
detail.
(Light Source 110)
[0034] The light source 110 is preferably a pulse light source
capable of emitting light pulses lasting a few nanoseconds to few
microseconds. Specifically, the light source 110 is preferably
capable of emitting light having a pulse width of approximately 10
nanoseconds in order to efficiently generate the photoacoustic
wave. A wavelength of the light which can be emitted by the light
source 110 is desirably the wavelength at which the light
propagates into the object. Specifically, when the object is a
living body, such as a human or animal body, a preferable
wavelength is not shorter than 500 nm and not longer than 1500
nm.
[0035] A laser or a light-emitting diode are examples of a light
source that may be used in some embodiments disclosed herein. As
the laser, various lasers such as a solid-state laser, a gas laser,
a dye laser, and a semiconductor laser may be used. For example,
the laser used in this embodiment includes an alexandrite laser, an
yttrium-aluminum-garnet laser, a titanium-sapphire laser and the
like.
(Optical System 120)
[0036] The light emitted from the light source 110 is typically
guided to the living body 100 while being shaped into a desired
light distribution shape by means of an optical component such as a
lens and a mirror. In addition, it is also possible to propagate
the pulsed light by using a waveguide or an optical fiber. The
optical component used to shape the light distribution includes,
for example, a mirror reflecting the light, a lens collecting and
magnifying the light or changing a focusing shape thereof, a prism
dispersing, refracting, and reflecting the light, the optical fiber
propagating the light, a diffusion plate dispersing the light and
other like optical components or combinations thereof. Any type or
number of such optical components may be used as long as the object
is irradiated with the light emitted from the light source 110 in
the desired manner.
[0037] However, when the light emitted by the light source 110 may
be guided directly to the object as desired light, it may not be
necessary to use the optical system 120.
(Acoustic Wave Detecting Unit 130)
[0038] The acoustic wave detecting unit 130 is provided with one or
more opto-acoustic transducers and a housing enclosing the
transducer(s). An opto-acoustic transducer, as used herein, is an
element capable of detecting an acoustic wave.
[0039] The transducer receives the acoustic wave such as the
photoacoustic wave and an ultrasonic echo to transform it to an
electric signal being an analog signal. Any transducer may be used
as long as the transducer is configured to receive the acoustic
wave. Examples of transducer include a transducer using a
piezoelectric phenomenon, a transducer using optical resonance, a
transducer using change in capacitance, and other like transducers.
The acoustic wave detecting unit 130 is preferably provided with a
plurality of transducers arranged in an array.
(Processing Unit 140)
[0040] The processing unit 140 is provided with the arithmetic unit
141 and the storage unit 142 as illustrated in FIG. 2.
[0041] The arithmetic unit 141 is typically formed of an arithmetic
element such as a CPU, a GPU, an A/D converter, a FPGA (field
programmable gate array) card, and an ASIC (application specific
integrated circuit) chip. Meanwhile, the arithmetic unit 141 may be
formed not only of one arithmetic element but also of a plurality
of arithmetic elements. Any arithmetic element may be used to
perform the disclosed process.
[0042] The storage unit 142 is typically formed of a storage medium
such as a ROM memory, a RAM memory, a hard disk drive, or a
combination thereof. That is, the storage unit 142 may be formed
not only of one storage medium but also of a plurality of storage
media.
[0043] The arithmetic unit 141 may make a gain adjustment to
increase or decrease an amplification gain according to time that
elapses from irradiation of the light to arrival of the acoustic
wave at the element of the acoustic wave detecting unit 130 in
order to obtain the image having a uniform contrast regardless of a
depth in the living body.
[0044] The arithmetic unit 141 may control light emission timing of
the pulsed light emitted from the light source 110, and may also
control operation start timing of the acoustic wave detecting unit
130 by using the pulsed light as a trigger signal. The arithmetic
unit 141 may control display operations of the display unit
160.
[0045] The arithmetic unit 141 is preferably configured to
simultaneously perform pipeline processing of a plurality of
signals when a plurality of detecting signals is obtained from the
acoustic wave detecting unit 130. According to this, time that
elapses before the object information is obtained may be
shortened.
[0046] Preferably, each processing operation performed by the
processing unit 140 may be saved in the storage unit 142 as part of
the program to be executed by the arithmetic unit 141. The storage
unit 142 in which the program is saved is a non-transitory
computer-readable recording medium.
[0047] The processing unit 140 and the acoustic wave detecting unit
130 may be provided as an integrated unit. Then, the processing
unit provided on the acoustic wave detecting unit may perform a
part of signal processing, and the processing unit provided outside
the acoustic wave detecting unit may perform the remainder of
signal processing. In this case, the processing unit provided on
the acoustic wave detecting unit and the processing unit provided
outside the acoustic wave detecting unit may be collectively
referred to as the processing unit according to this
embodiment.
(Input Unit 150)
[0048] The input unit 150 is a user interface (I/F) configured to
accept an operation (e.g., input) by the user. Information input by
the user is input from the input unit 150 to the processing unit
140.
[0049] For example, a pointing device such as a mouse and a
keyboard, a graphics tablet type and the like may be adapted as the
input unit 150. A mechanical device such as a button and a dial
provided on a device forming the object information obtaining
device, or other I/F device may also be adapted as the input unit
150. When a touch panel display is used as the display unit 160,
the display unit 160 may also be adapted to function as the input
unit 150.
[0050] Naturally, the input unit 150 may be provided as a user I/F
disposed separately from the object information obtaining device
and connected thereto via the data network 200.
(Display Unit 160)
[0051] The display unit 160 is a device which displays the object
information output from the processing unit 140.
[0052] Although a liquid crystal display (LCD) and the like is
typically used as the display unit 160, another type of display
such as a plasma display, an organic EL display, and a FED may also
be used. It is also possible to integrally form the input unit 150
and the display unit 160 by adopting the touch panel display as the
display unit 160.
[0053] The display unit 160 may also be provided separately from
the object information obtaining device according to this
embodiment.
[0054] Next, the method of obtaining object information according
to this embodiment using the object information obtaining device
illustrated in FIGS. 1 and 2 is described with reference to a flow
illustrated in FIG. 3. The flow process illustrated in FIG. 3 is
example of an algorithm executed by the processing unit 140.
(S301: Step of Obtaining Photoacoustic Signal Data)
[0055] At step S301, the light emitted by the light source 110 is
applied to the living body 100 as pulse light 121 through the
optical system 120. Then, a light absorber 101 absorbs the pulse
light 121 and a photoacoustic wave 102 is generated by the
photoacoustic effect.
[0056] Next, the acoustic wave detecting unit 130 transforms the
photoacoustic wave 102 to the electric signal being the analog
signal to output to the processing unit 140. The arithmetic unit
141 saves the electric signal output from the acoustic wave
detecting unit 130 in the storage unit 142 as the photoacoustic
signal data.
[0057] In this embodiment, data obtained when the electric signal
output from the acoustic wave detecting unit 130 is saved in the
storage unit 142 is made into the photoacoustic signal data. The
photoacoustic signal data may be read from the storage unit 142 by
the arithmetic unit 141 to be used in the two or more types of
processing to be described later.
[0058] The electric signal output from the acoustic wave detecting
unit 130 is typically amplified and subjected to the A/D conversion
to be saved in the storage unit 142 as the photoacoustic signal
data. The electric signal output from the acoustic wave detecting
unit 130 may also be saved in the storage unit 142 as the
photoacoustic signal data after being averaged.
[0059] The photoacoustic signal data is saved in the storage unit
142 in this manner. The arithmetic unit 141 may use the
photoacoustic signal data including the same photoacoustic signal
data corresponding to the photoacoustic wave detected at certain
time in a plurality of types of processing to be described
later.
[0060] In photo acoustic imaging, it is possible to apply different
types of processing to the photoacoustic signal data including the
same photoacoustic signal data obtained by detecting the
photoacoustic wave at certain time. According to this, the object
information at the same time corresponding to each of the different
types of processing may be obtained.
[0061] That is, the object information corresponding to the desired
processing out of pieces of object information at the same time
obtained by applying each of the two or more types of processing to
the photoacoustic signal data including the same data may be
selectively displayed.
[0062] The arithmetic unit 141 may also obtain the object
information corresponding to each processing by performing the two
or more types of processing to the photoacoustic signal data not
including the same data.
(S302: Step of Selecting Information of Desired Processing from Two
or More Types of Processing)
[0063] At step S302, the user selects the desired processing from
two or more types of processing by using the input unit 150. Then,
the input unit 150 outputs the information of the processing
selected by the user to the processing unit 140. At that time, the
information of the selected processing is saved in the storage unit
142.
[0064] An example of the input unit 150 for the user to select the
desired processing from the two or more types of processing is
hereinafter described. That is, an example of a method of inputting
the information of the desired processing by the user is
described.
[0065] For example, the user may select the desired processing by
pressing a mechanical button as the input unit 150 corresponding to
each of the two or more types of processing. Alternatively, the
user may select the desired processing by turning a mechanical dial
as the input unit 150 corresponding to each of the two or more
types of processing.
[0066] As another example, the user may also select the desired
processing by selecting an item indicating the processing displayed
on the display unit 160 by means of a pointing device (mouse), the
keyboard and the like as the input unit 150. At that time, the
display unit 160 may display the items indicating the processing
next to one another as icons or display them as a menu. The item
related to the processing displayed on the display unit 160 may be
always displayed beside the image of the object information or may
be configured to be displayed when the user performs some operation
by using the input unit 150. For example, the display unit 160 may
be configured such that the item indicating the processing is
displayed on the display unit 160 by a click of the mechanical
button provided on the mouse as the input unit 150.
[0067] The method is not limited to the above-described method and
any method may be adopted as long as the user may select the
desired processing out of the two or more types of processing.
[0068] The object information obtaining device is preferably
configured such that progress of each processing is visually
presented to the user. For example, it is possible to configure the
object information obtaining device such that the progress of each
processing is visually presented by displaying a progress bar or
displaying a predicted calculation termination time on the display
unit 160. In addition, it is also possible to use a circular
progress mark in which an angle of a part with changed color
changes as the processing advances. Alternatively, a color of the
item corresponding to the processing may be changed according to a
progress status such as completion of the processing or the
progress status may be displayed in characters in the vicinity of
the item.
[0069] The object information obtaining device according to this
embodiment is preferably configured such that the progress of the
processing may be grasped and the user may optionally stop the
processing currently being calculated. Such configuration allows
the user to start a different process operation when the user sees
the progress bar and determines that the progress of the processing
currently being calculated is not convenient (e.g., the processing
is taking too long, the processing is not good due to a processing
error, the type of processing was chosen in error, etc.).
[0070] Image reconstruction processing selected by default may be
set in advance in a file in the storage unit 142. In this case, the
arithmetic unit 141 may read default processing at the beginning of
step S302 and execute the processing selected by default if the
user does not especially select other processing. It is also
possible that the user may intentionally select the processing set
by default.
[0071] The desired processing selected by the user may be at least
one type of processing. In this embodiment, at least two types of
processing may be selected from three or more types of processing.
At that time, the object information obtaining device according to
this embodiment may be configured such that a plurality of
combinations of at least two types of processing may be selected.
According to this, the user may select the desired processing with
a high degree of freedom and it becomes possible to display the
object information useful in the diagnosis.
(S303: Step of Obtaining Object Information by Performing Desired
Processing)
[0072] At step S303, the arithmetic unit 141 obtains the object
information by performing the desired processing selected at S200
based on the photoacoustic signal data saved in the storage unit
142. Herein, the object information obtained by performing the
desired processing is referred to as "object information
corresponding to the desired processing".
[0073] Meanwhile, the arithmetic unit 141 may read the program in
which an algorithm of the processing is described stored in the
storage unit 142 and apply this processing to the photoacoustic
signal data to obtain the object information.
[0074] In this embodiment, three-dimensional voxel data and
two-dimensional pixel data as the object information may be
obtained by the processing.
[0075] Herein, the processing according to this embodiment is
intended to mean every processing performed during transform from
the photoacoustic signal data to the object information having a
pathological value. For example, the processing according to this
embodiment includes signal processing such as probe response
correction processing and noise removal processing to generate
different photoacoustic signal data based on the photoacoustic
signal data stored in the storage unit 142. There also is, for
example, reconstruction processing such as time domain
reconstruction processing, Fourier domain reconstruction
processing, and model base reconstruction processing to generate
the object information from the photoacoustic signal data stored in
the storage unit 142 as the processing according to this
embodiment. For example, the processing according to this
embodiment includes image processing such as resolution improvement
processing to generate different object information based on the
object information generated by the above-described reconstruction
processing.
[0076] An example of each processing is hereinafter described.
[0077] The probe response correction processing (hereinafter,
referred to as "BD processing") as the signal processing according
to this embodiment is the processing to correct signal
deterioration due to band limitation of a probe by applying
processing based on a blind deconvolution algorithm to the
photoacoustic signal data (refer to Patent Document 1 (Japanese
Patent Application Laid-Open No. 2012-135462)). When the
photoacoustic wave is transformed to the electric signal by the
acoustic wave detecting unit 130, there is limitation in receiving
bandwidth of the acoustic wave detecting unit 130, so that a
waveform of the electric signal might change to generate ringing.
This ringing causes the artifact appearing in the vicinity of the
light absorber on the image to deteriorate resolution. Probe
response correction has an effect of decreasing the ringing by the
acoustic wave detecting unit, thereby decreasing the artifact and
improving the resolution.
[0078] The noise removal processing (hereinafter, referred to as
"wavelet processing") as the signal processing according to this
embodiment is the processing to remove a noise component of the
photoacoustic signal data through basis pursuit by a wavelet
function of the photoacoustic signal data. A waveform of the signal
resulting from the photoacoustic wave is known to be an N-shaped
waveform under an ideal condition (refer to Non-Patent Document 2
(Sergey A. Ermilov, RedaGharieb, Andre Conjusteau, Tom Miller,
Ketan Mehta, and Alexander A. Oraevsky, "Data Processing and
quasi-3D optoacoustic imaging of tumors in the breast using a
linear arc-shaped array of ultrasonic transducers", Proc. of SPIE,
Vol. 6856). On the other hand, random noise being an irregular
waveform mixed from an electric system and the like of the device
is superimposed on the signal resulting from the photoacoustic
wave. Therefore, the signal resulting from the noise is
discriminated from the signal resulting from the photoacoustic wave
by applying a discrete wavelet transform to the photoacoustic
signal data and removing a coefficient having a small absolute
value from a result thereof. The wavelet processing has a large
effect when the signal resulting from the photo acoustic wave has
the waveform close to the ideal waveform. On the other hand, when a
frequency of the photoacoustic wave is significantly different from
a bandwidth of the acoustic wave detecting unit, when the noise is
too large, and when a plurality of waveforms are superimposed due
to a feature of the object, there is a case in which an effect of
improving an image quality by the wavelet processing is small.
[0079] The time domain reconstruction processing (hereinafter,
referred to as "TD processing") as the reconstruction processing is
the processing to estimate a sonic wave source by superimposing
sonic wave signals in a real space by using a property that the
photoacoustic wave is a spherical wave to generate the voxel data
(refer to Patent Document 2 (Japanese Patent Application Laid-Open
No. 2010-35806)). The TD processing specifically includes UBP
processing disclosed in Non-Patent Document 1. The TD processing is
performed in the real space, so that an effect of a measurement
system is easily introduced as compared to the Fourier domain
reconstruction processing and the like to be described later. For
example, it is possible to decrease a side-lobe artifact by
applying weighted correction processing of a solid angle and the
like in consideration of a state of the acoustic wave detecting
unit 130, for example.
[0080] The Fourier domain reconstruction processing (hereinafter,
referred to as "FD processing") as the reconstruction processing is
the processing to estimate the sonic wave source by superimposing
the detection signals in a frequency domain by using a Fourier
transform and an inverse Fourier transform to generate the voxel
data (refer to Japanese Patent Application Laid-Open No.
2010-35806). The processing may be performed in a short time by
using a fast Fourier transform. However, the effect of the
measurement system is not easily introduced in a frequency space as
compared to the real space. Therefore, it is difficult to apply the
weighted correction processing of the solid angle and the like in
consideration of the state of the acoustic wave detecting unit
which may be performed in the TD processing, for example, and there
is a case in which the side-lobe artifact is generated.
[0081] The model base reconstruction processing (hereinafter,
referred to as "MBP processing") as the reconstruction processing
is the processing to estimate the sonic wave source such that
difference between a calculation result based on a propagation
model of an ideal photoacoustic wave and the photoacoustic signal
data is minimum to generate the voxel data (refer to Patent
Document 3 (Japanese Patent Application Laid-Open No.
2011-143175)). By using a descriptive model of a phenomenon, the
measurement system may be more strictly described than in the TD
processing and the FD processing. According to this, an image with
few artifacts may be obtained. However, since it is required to
repetitively calculate such that the difference between the
photoacoustic signal data and the calculation result is minimum in
the MBP processing, longer processing time than that in the TD
processing and the FD processing is typically required. It is
difficult to reflect all the phenomena in the model, and when a
generated phenomenon cannot be reflected in the model, there is a
case in which quantitativeness of the object information obtained
by the MBP processing is deteriorated.
[0082] The resolution improvement processing (hereinafter, referred
to as "CF processing") as the image processing is the processing to
reduce the artifact generated by limitation of a viewing angle of
the probe by using a coherent filter in the object information
obtained by the above-described reconstruction processing (refer to
Patent Document 4 (Japanese Patent Application Laid-Open No.
2011-120765). According to this, a high-resolution image of the
object information may be obtained. This is the processing to
calculate a coefficient which is set to 1 when phase signals of the
photoacoustic wave are in phase and set to 0 when they are out of
phase for each voxel and multiply distribution of the coefficients
by the image. The CF processing is especially effective when sound
speed distribution of the object is nearly constant. On the other
hand, when variation in the sound speed distribution of the object
is large, an effect of improving the image quality by the CF
processing might be small.
[0083] Hereinafter, a result obtained when various types of
reconstruction processing are applied to the photoacoustic signal
data obtained by simulating step S301 using a model having
absorption coefficient distribution illustrated in FIG. 4A is
described. Herein, an x-axis corresponds to a horizontal direction
and a z-axis corresponds to a vertical direction in FIG. 4A.
Herein, a case in which a one-dimensional transducer array in an
x-axis direction is arranged on a lowest part in FIG. 4A and the
one-dimensional transducer array detects the photoacoustic wave
propagated from an upper side of the z-axis is simulated.
[0084] FIG. 4B illustrates initial sound pressure distribution when
the FD processing is performed. FIG. 4C illustrates the initial
sound pressure distribution when the TD processing is performed.
FIG. 4D illustrates the initial sound pressure distribution when
the MBP processing is performed.
[0085] As is understood from the images in FIGS. 4B to 4D,
different images are obtained for the same absorption coefficient
distribution depending on the type of the processing.
[0086] For example, in the image obtained by the FD processing
illustrated in FIG. 4B, an arc-like artifact is confirmed. In the
image obtained by the TD processing illustrated in FIG. 4C, the
artifact extending in the x-axis direction is confirmed. It is
understood that the artifact in the image obtained by the MBP
processing illustrated in FIG. 4D is entirely suppressed as
compared to the artifact in the images illustrated in FIGS. 4B and
4C.
[0087] In the images illustrated in FIGS. 4B and 4C, it is
understood that connection of the initial sound pressure
distribution corresponding to the absorption coefficient
distribution extending in a z-axis direction decreases as compared
to that in the image illustrated in FIG. 4D.
[0088] As described above, appropriate processing differs according
to a measurement environment and a site wanted to be observed. Time
required for the processing differs depending on the type of
processing.
(S304: Step of Displaying Object Information Corresponding to
Desired Processing)
[0089] At this step, the arithmetic unit 141 calculates display
data to be displayed on the display unit 160 based on the voxel
data (or the pixel data) of the object information saved in the
storage unit 142 to display the display data on the display unit
160.
[0090] Meanwhile, the display data of desired dimension out of one
dimension, two dimensions, and three dimensions may be obtained
from the voxel data (or the pixel data). The object information
obtaining device may be configured such that the user may set the
dimension of the display data by using the input unit 150.
[0091] As described above, in the object information obtaining
device according to this embodiment, it is possible to display the
object information corresponding to the desired processing selected
by the user out of the two or more types of processing on the
display unit. According to this, it is possible to diagnose by
using the image meeting needs of the user such as the processing
time and the image quality from the images at the same time
obtained by each image reconstruction.
[0092] The configuration in which the object information
corresponding to the desired processing is obtained after the
information of the desired processing is obtained is described
above. However, the object information obtaining device according
to this embodiment may also be configured such that the information
of the desired processing is obtained and the object information
corresponding to the information of the desired processing is
displayed in a state in which the object information corresponding
to the desired processing is obtained in advance. That is to say,
step S302 may be executed after step S303 is executed and step S304
may be executed thereafter.
[0093] In this case, the image itself obtained after the processing
is applied to the photoacoustic signal data may be adopted as the
item indicating the processing. That is to say, the user may select
the processing by selecting the image. For example, it is possible
that the images whose processing is finished are sequentially
displayed on the display unit 160 and when the user selects one of
a plurality of images, the image is displayed in an enlarged
manner. By this method, the user may compare results of a plurality
of types of processing and may select the desired image even when
the user does not have knowledge of the processing.
[0094] In this case, the object information obtaining device is
preferably configured such that the user cannot select the
processing not finished yet.
[0095] In this case, the object information obtaining device is
preferably further provided with notifying means of notifying the
user of whether the processing is finished processing. Further, the
notifying means is preferably configured such that the user may
visually recognize whether the processing is the finished
processing.
[0096] For example, when the item indicating the object information
is displayed on the display unit 160, it is possible to display the
item of the finished processing and the item of the processing not
finished yet in different colors and the like as the notifying
means.
[0097] It is also possible to provide a lamp as the notifying means
corresponding to each processing on the device forming the object
information obtaining device or another device. In this case, it is
possible to notify the user of whether the processing is the
finished processing by setting such that the lamp corresponding the
finished processing is turned on, for example.
[0098] It is also possible that the information of the desired
processing is obtained and the object information corresponding to
the desired processing is displayed on the display unit 160 in a
state in which the object information different from the object
information corresponding to the desired processing is displayed on
the display unit 160. At that time, the object information
corresponding to the desired processing may be displayed so as to
be superimposed on the object information displayed in advance or
may be displayed next to the same. It is also possible to switch
from the object information displayed in advance to the object
information corresponding to the desired processing to display on
the display unit 160. That is to say, it is possible to hide the
object information displayed in advance from the display unit 160
and display the object information corresponding to the desired
processing in an area on the display unit 160 in which the object
information is displayed. The display method may be set in advance
before shipping or may be set by the user by means of the input
unit 150.
[0099] In this manner, the user may grasp pathological information
which may be grasped from the object information displayed in
advance and the pathological information which may grasped from the
object information corresponding to the desired processing to
diagnose in a comprehensive manner. It is also possible to diagnose
in a comprehensive manner by grasping a plurality of pieces of
pathological information without a time interval.
Example 1
[0100] Example 1 according to the present invention is subsequently
described with reference to FIGS. 1, 5, and 6. FIG. 5 is a flow
diagram of a method of obtaining object information according to
this example. FIG. 6 is a schematic diagram illustrating a computer
140 as a processing unit according to this example in detail and a
peripheral device. As illustrated in FIG. 6, the computer 140 is
provided with a CPU 641, a FPGA 642, and a GPU 643 as an arithmetic
unit, and a ROM 644 and a RAM 645 as a storage unit. Herein, the
ROM 644 is used as a non-transitory computer-readable recording
medium.
[0101] In this example, the CPU 641 controls operation of each
component forming an object information obtaining device through a
data network 200, which is similar to that shown in FIG. 2. The CPU
641 reads a program in which the method of obtaining object
information according to this example is described saved in the ROM
644 to allow the object information obtaining device to execute the
method of obtaining object information. That is to say, the
computer 140 executes a flow illustrated in FIG. 5.
[0102] At step S501, a user operated an input unit 150 to input a
measurement parameter. The measurement parameter was saved in the
RAM 645 as the storage unit. At this step, the user set a
wavelength of laser light used in measurement and the number of
irradiation times of the laser light to a breast 100 of a subject
as an object in one measurement as the measurement parameters.
Meanwhile, in this example, the user set the wavelength of the
laser light used in the measurement to 797 nm and the number of
irradiation times of the laser light to 30.
[0103] Subsequently, at step S502, the CPU 641 issues an
instruction based on the measurement parameter to a
titanium-sapphire laser 110 as a light source to allow the same to
emit the laser light. The laser light was applied to the breast 100
as pulse light 121 with a pulse width of 50 nm through an optical
fiber 120. Then, the breast 100 absorbed the pulse light 121 and a
photoacoustic wave reflecting absorption coefficient distribution
in the breast 100 was generated. Meanwhile, the titan-sapphire
laser 110 in this example includes a flash lamp and a Q-switch as
means of exciting an internal laser medium and light emission
timing was controlled by the instruction from the CPU 641.
[0104] Subsequently, at step S503, a CMUT array 130 as an acoustic
wave detecting unit transformed the photoacoustic wave to an
electric signal and output the electric signal to the processing
unit 140.
[0105] Meanwhile, the CPU 641 instructs the CMUT array 130 to
detect the photoacoustic wave in synchronization with the
instruction to emit the laser light at step S502. In this example,
ultrasonic gel whose acoustic impedance is close to that of the
breast 100 was provided as an acoustic matching medium between the
CMUT array 130 and the breast 100.
[0106] Subsequently, at step S504, the FPGA 642 amplified the
electric signal and performed A/D conversion thereof. The CPU 641
saved the signal amplified and subjected to the A/D conversion in
the RAM 645 as photoacoustic signal data.
[0107] Subsequently, at step S505, it was determined whether the
measurement of the object was completed. When the measurement of
the object is completed, the procedure shifts to step S506. When
the measurement of the object is not completed, the procedure
shifts to step S502. In this example, since the number of
irradiation times of the laser light was set to 30 at step S501,
the measurement is completed when the procedure from step S502 to
step S504 is repeated 30 times.
[0108] A screen displayed on a liquid crystal display 160 as a
display unit used in a following step is illustrated in FIG. 7. The
user may select a desired item from an item 701 corresponding to BD
processing, an item 702 corresponding to UBP processing, an item
703 corresponding to MBP processing, and an item 704 corresponding
to CF processing.
[0109] The items 701 to 704 corresponding to each processing and
progress bars 711 to 714 indicating a progress situation of each
processing are displayed next to one another. When a black bar of
each of the progress bars 711 to 714 is located on a left end,
progress of the corresponding processing is indicated to be 0% and
when this reaches a right end, the progress of the corresponding
processing is indicated to be 100%. By this configuration, the user
may grasp the progress situation and time remained of the
corresponding processing from a position and a speed of the
progress bar.
[0110] FIG. 7 illustrates the screen displayed when the item 701
corresponding to the BD processing is selected at step S508 to be
described later and the item 703 corresponding to the MBP
processing is selected at step S512 thereafter. At that time, the
black bar of the progress bar 713 corresponding the MBP processing
does not reach the right end as illustrated in FIG. 7. Therefore,
it is understood that the MBP processing is not finished at that
time.
[0111] Subsequently, at step S506, the CPU 641 referred to the RAM
645 and displayed a list of pieces of the saved photoacoustic
signal data in a data selection window 720. Then, the user selected
one of the pieces of photoacoustic signal data displayed in the
data selection window 720.
[0112] Meanwhile, in this example, an ID number of the subject and
photographing time of the photoacoustic signal data are displayed
in the data selection window 720 such that they may be
selected.
[0113] Subsequently, at step S507, the CPU 641 read the measurement
parameter corresponding to the photoacoustic signal data selected
by the user at step S506 and displayed the object information which
may be displayed in an object information selection window 730.
Then, the user selected the item corresponding to initial sound
pressure.
[0114] The initial sound pressure, an absorption coefficient, and
oxygen saturation are displayed in the object information selection
window 730. However, since it was measured by using only one
wavelength 797 nm, the oxygen saturation being spectral
characteristics cannot be selected in this example. The initial
sound pressure and the absorption coefficient which may be selected
by the user and the oxygen saturation which cannot be selected by
the user are displayed in different colors.
[0115] Subsequently, at step S508, the CPU 641 determines whether
the item 701 corresponding to the BD processing is selected by the
user. When the item 701 corresponding to the BD processing is
selected, the procedure shifts to step S509. When the item 701
corresponding to the BD processing is not selected, the procedure
shifts to step S510. Meanwhile, since the user selects the item 701
corresponding to the BD processing, the procedure shifts to step
S509 in this example.
[0116] Subsequently, at step S509, the CPU 641 read the
photoacoustic signal data selected by the user from the RAM 645 and
applied the above-described BD processing to the photoacoustic
signal data. Then, the photoacoustic signal data to which the BD
processing was applied was saved in the RAM 645.
[0117] Subsequently, at step S510, the CPU 641 determined whether
the item 702 corresponding to the UBP processing was selected. When
the item 702 corresponding to the UBP processing is selected, the
procedure shifts to step S511. When the item 702 corresponding to
the UBP processing is not selected, the procedure shifts to step
S512. Since the user does not select the item 702 corresponding to
the UBP processing, the procedure shifts to step S512 in this
example.
[0118] Subsequently, at step S512, the CPU 641 determined whether
the item 703 corresponding to the MBP processing was selected. When
the item 703 corresponding to the MBP processing is selected, the
procedure shifts to step S513. When the item 703 corresponding to
the MBP processing is not selected, the procedure shifts to step
S510. Meanwhile, since the user selects the item 703 corresponding
to the MBP processing, the procedure shifts to S513 in this
example.
[0119] Subsequently, at step S513, the CPU 641 instructed the GPU
643 to perform the MBP processing. Then, the GPU 643 applied the
MBP processing to the photoacoustic signal data to which the BD
processing was applied at step S509 to generate three-dimensional
voxel data related to the initial sound pressure. The
three-dimensional voxel data was saved in the RAM 645.
[0120] Subsequently, at step S514, the CPU 641 determines whether
the item 704 corresponding to the CF processing is selected. When
the item 704 corresponding to the CF processing is selected, the
procedure shifts to step S515. When the item 704 corresponding to
the CF processing is not selected, the procedure shifts to step
S516. In this example, the user selects the item 704 corresponding
to the CF processing, so that the procedure shifts to step
S515.
[0121] Subsequently, at step S515, the CPU 641 applied the CF
processing to the three-dimensional voxel data related to the
initial sound pressure stored in the RAM 645 to generate the
three-dimensional voxel data related to the initial sound pressure
subjected to the CF processing. Then, the three-dimensional voxel
data related to the initial sound pressure after being subjected to
the CF processing was saved in the RAM 645. By applying the CF
processing at this step, resolution of the three-dimensional voxel
data related to the initial sound pressure was improved.
[0122] Subsequently, at step S516, the GPU 643 applied scan
transform processing to the three-dimensional voxel data related to
the initial sound pressure stored in the RAM 645 to generate
display data. Then, the CPU 641 output the display data to the
liquid crystal display 160 and initial sound pressure distribution
was displayed in an image display window 740.
[0123] Subsequently, the procedure shifts to step S508 again to
determine whether the item corresponding to each processing is
selected and when any item is selected, the processing
corresponding to the item is executed.
[0124] According to this example, the user may execute the desired
processing to display the object information. Therefore, the user
may diagnose by using an image obtained by the processing meeting
needs of the user such as processing time and an image quality by
using the photoacoustic signal data obtained at certain time.
Example 2
[0125] Subsequently, Example 2 of the present invention is
described. This example is different from Example 1 in that a
plurality of types of processing is started in parallel and desired
processing may be selected from finished processing.
[0126] In this example, an object information obtaining device
illustrated in FIGS. 1 and 6 was used as in Example 1. Hereinafter,
a method of obtaining object information of this example is
described with reference to a flow illustrated in FIG. 8.
Meanwhile, the flow illustrated in FIG. 8 is executed by a computer
140.
[0127] In this example, a CPU 641 issued an instruction to a GPU
643 to execute UBP processing at step S511 to photoacoustic signal
data to which BD processing was applied after steps up to step
S509. Further, the CPU 641 issued an instruction to the GPU 643 to
execute MBP processing at step S513 in parallel with step S511.
[0128] Meanwhile, reconstruction processing is stored in a ROM 644
as a different thread program. Each processing is executed by each
of a plurality of processors assigned in the GPU 643.
[0129] A screen displayed on a liquid crystal display 160 at that
time is illustrated in FIG. 9. When a progress bar 712 indicating
progress of the UBP processing is confirmed, the progress is
indicated to be 100%, so that initial sound pressure distribution
corresponding to the UBP processing may be selected.
[0130] On the other hand, when a progress bar 713 indicating the
progress of the MBP processing is confirmed, the progress is not
indicated to be 100%. Therefore, a user cannot select an item 703
corresponding to the MBP processing. This is because the MBP
processing requires longer processing time than that of the UBP
processing.
[0131] In this example, an item 702 corresponding to the UBP
processing which may be selected is displayed with white background
and the item 703 corresponding to the MBP processing which cannot
be selected is displayed with gray background, so that the user
could visually recognize whether the processing may be
selected.
[0132] Subsequently, at step S510, the CPU 641 determined whether
the item 702 corresponding to the UBP processing was selected. When
the item 702 corresponding to the UBP processing is selected, the
procedure shifts to step S516. When the item 702 corresponding to
the UBP processing is not selected, the procedure shifts to step
S512. In this example, the user selects the item 702 corresponding
to the UBP processing which previously becomes selectable, so that
the procedure shifts to step S516.
[0133] Subsequently, the procedure shifts to step S510 again to
determine whether the item corresponding to each processing is
selected, and when any item is selected, the object information
corresponding to the item is displayed.
[0134] As described above, the number of finished processing
increases with time, so that the types of processing which the user
may select also increase with time in this example. Therefore, a
diagnostic method of confirming the object information obtained in
a short time by the UBP processing and the like first and
confirming the object information obtained by the MBP processing
and the like when further detail is required during reading as in
this example becomes possible.
Other Embodiments
[0135] Embodiments of the present invention can also be realized by
a computer of a system or apparatus that reads out and executes
computer executable instructions recorded on a storage medium
(e.g., non-transitory computer-readable storage medium) to perform
the functions of one or more of the above-described embodiment(s)
of the present invention, and by a method performed by the computer
of the system or apparatus by, for example, reading out and
executing the computer executable instructions from the storage
medium to perform the functions of one or more of the
above-described embodiment(s). The computer may comprise one or
more of a central processing unit (CPU), micro processing unit
(MPU), or other circuitry, and may include a network of separate
computers or separate computer processing units. The computer
executable instructions may be provided to the computer, for
example, from a network or the storage medium. The storage medium
may include, for example, one or more of a hard disk, a
random-access memory (RAM), a read only memory (ROM), a storage of
distributed computing systems, an optical disk (such as a compact
disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD).TM.),
a flash memory device, a memory card, and the like.
[0136] While the present invention has been described with
reference to exemplary embodiments, it is to be understood that the
invention is not limited to the disclosed exemplary embodiments.
The scope of the following claims is to be accorded the broadest
interpretation so as to encompass all such modifications and
equivalent structures and functions.
[0137] This application claims the benefit of Japanese Patent
Application No. 2012-286685, filed Dec., 28, 2012 which is
incorporated by reference herein in its entirety.
* * * * *