U.S. patent application number 10/815759 was filed with the patent office on 2004-11-25 for combining first and second image data of an object.
Invention is credited to Grimm, Marcus, Sakas, Georgios.
Application Number | 20040236206 10/815759 |
Document ID | / |
Family ID | 32865002 |
Filed Date | 2004-11-25 |
United States Patent
Application |
20040236206 |
Kind Code |
A1 |
Sakas, Georgios ; et
al. |
November 25, 2004 |
Combining first and second image data of an object
Abstract
The invention relates to a method of combining first and second
image data of an object (3), wherein the first image data of the
object (3) are repeatedly generated by an ultrasound detector (1),
wherein the second image data of the object (3) are generated
separately, for example the second image data have been recorded
earlier by a computer tomography, a magnetic resonance or a
positron emission tomography imaging device, wherein the first
image data are transferred from the ultrasound detector (1) to a
combination device (5) and wherein the combination device (5) is
adapted to combine the first and second image data of the object
(3). Geometry data is transferred in addition to the first image
data from the ultrasound detector (1) to the combination device
(5), wherein the geometry data comprise information for
establishing a link between the geometry of the first image data
and the geometry of the second image data.
Inventors: |
Sakas, Georgios; (Darmstadt,
DE) ; Grimm, Marcus; (Egelsbach, DE) |
Correspondence
Address: |
NEIFELD IP LAW, PC
2001 JEFFERSON DAVIS HIGHWAY
ARLINGTON
VA
22202
US
|
Family ID: |
32865002 |
Appl. No.: |
10/815759 |
Filed: |
April 2, 2004 |
Current U.S.
Class: |
600/407 |
Current CPC
Class: |
A61B 8/483 20130101;
G06T 2207/30004 20130101; G06T 2207/30108 20130101; A61B 8/00
20130101; G06T 7/33 20170101; G06T 2207/10072 20130101; A61B 8/4254
20130101; G06T 2207/10132 20130101; G06T 7/38 20170101; A61B 8/5238
20130101 |
Class at
Publication: |
600/407 |
International
Class: |
A61B 005/05 |
Foreign Application Data
Date |
Code |
Application Number |
Apr 11, 2003 |
EP |
03008448.7-2218 |
Claims
1. An apparatus for combining first and second image data of an
object, comprising: an ultrasound detector for repeatedly
generating first image data of the object; a combination device; at
least one data connection; structure configured for at least one of
storing and receiving second image data of the object; wherein said
combination device is adapted to combine said first image data and
said second image data; wherein said at least one data connection
is designed to transfer from said ultrasound detector to said
combination device at least said geometry data; wherein said
geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an
image unit of the first image data, b) information concerning an
image position of at least a part of an image, which is represented
by the first image data, relative to a reference point of the
ultrasound detector or relative to a reference point or reference
object in the ultrasound image; c) information concerning an
orientation of the ultrasound image relative to a reference point
or a reference object of the ultrasound detector; and d)
information concerning a region or an area, which is actually
covered by an ultrasound image that is represented by the first
image data.
2. The apparatus of claim 1 wherein said at least one data
connection is designed to transfer from said ultrasound detector to
said combination device said first image data and said geometry
data.
3. The apparatus of claim 1 wherein said at least one data
connection comprises a first data connection which is designed to
transfer from said ultrasound detector to said combination device
said geometry data and a second data connection which is designed
to transfer from said ultrasound detector to said combination
device said first image data.
4. The apparatus of claim 1 wherein said at least one data
connection is adapted to directly transfer said first image data in
a digital format from said ultrasound detector to the combination
device.
5. The apparatus of claim 1 wherein said ultrasound detector
comprises a control unit for controlling an image data generation
of said ultrasound detector and wherein said control unit is
adapted to generate at least a part of said geometry data.
6. The apparatus of claim 1 wherein said geometry data comprises
data defining information concerning at least one spatial dimension
of a pixel of the first image data.
7. The apparatus of claim 1 wherein said second image data
comprises at least one of image data generated by a computer
tomography, a magnetic resonance, a positron emission tomography an
X-ray and a three-dimensional ultrasound imaging device.
8. The apparatus of claim 1 wherein said geometry data comprise
data defining information concerning at least one spatial dimension
of an image unit of said first image data.
9. The apparatus of claim 1 wherein said geometry data comprise
data defining information concerning at least an image position of
at least a part of an image, which is represented by the first
image data relative to a reference point of the ultrasound detector
or relative to a reference point or reference object in the
ultrasound image.
10. The apparatus of claim 1 wherein said geometry data comprise
data defining information concerning at least an orientation of the
ultrasound image relative to a reference point or a reference
object of the ultrasound detector.
11. The apparatus of claim 1 wherein said geometry data comprise
data defining information concerning at least a region or an area
which is actually covered by an ultrasound image that is
represented by said first image data.
12. The apparatus of claim 1 wherein said first image data is
derived from said ultrasound detector.
13. The apparatus of claim 1 wherein said second image data is
derived from a second detector other than said ultrasound
detector.
14. The apparatus of claim 13 wherein said second detector is not
an ultrasound detector.
15. A method of making an apparatus for combining first and second
image data of an object, comprising: providing an ultrasound
detector for repeatedly generating first image data of the object;
providing a combination device; providing at least one data
connection; providing structure configured for at least one of
storing and receiving second image data of the object; wherein said
second image data of the object comprises at least one of image
data generated by a computer tomography, a magnetic resonance, a
positron emission tomography, and an X-ray imaging device; wherein
said combination device is adapted to combine said first image data
and said second image data; wherein said at least one data
connection is designed to transfer from said ultrasound detector to
said combination device at least said geometry data; wherein said
geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an
image unit of the first image data; b) information concerning an
image position of at least a part of an image, which is represented
by the first image data, relative to a reference point of the
ultrasound detector or relative to a reference point or reference
object in the ultrasound image; c) information concerning an
orientation of the ultrasound image relative to a reference point
or a reference object of the ultrasound detector; and d)
information concerning a region or an area, which is actually
covered by an ultrasound image that is represented by the first
image data.
16. The method of claim 15 wherein said at least one data
connection is designed to transfer from said ultrasound detector to
said combination device said first image data and said geometry
data.
17. The method of claim 15 wherein said at least one data
connection comprises a first data connection designed to transfer
from said ultrasound detector to said combination device said
geometry data and a second data connection designed to transfer
from said ultrasound detector to said combination device said first
image data.
18. The method of claim 15 wherein said geometry data comprise data
defining information concerning at least one spatial dimension of
an image unit of said first image data.
19. The method of claim 15 wherein said geometry data comprise data
defining information concerning at least an image position of at
least a part of an image, which is represented by the first image
data relative to a reference point of the ultrasound detector or
relative to a reference point or reference object in the ultrasound
image.
20. The method of claim 15 wherein said geometry data comprise data
defining information concerning at least an orientation of the
ultrasound image relative to a reference point or a reference
object of the ultrasound detector.
21. The method of claim 15 wherein said geometry data comprise data
defining information concerning at least a region or an area which
is actually covered by an ultrasound image that is represented by
said first image data.
22. The method of claim 15 wherein said first image data is derived
from said ultrasound detector.
23. The method of claim 15 wherein said second image data is
derived from a second detector other than said ultrasound
detector.
24. The method of claim 23 wherein said second detector is not an
ultrasound detector.
25. A method of using an apparatus for combining first and second
image data of an object, comprising: repeatedly generating first
image data of the object using an ultrasound detector; at least one
of storing and receiving second image data of the object; combining
said first image data and said second image data using a
combination device; transferring from said ultrasound detector to
said combination device said first image data and geometry data
using at least one data connection; wherein said geometry data
comprise data defining at least one of the following: a)
information concerning at least one spatial dimension of an image
unit of the first image data; b) information concerning an image
position of at least a part of an image, which is represented by
the first image data, relative to a reference point of the
ultrasound detector or relative to a reference point or reference
object in the ultrasound image; and c) information concerning an
orientation of the ultrasound image relative to a reference point
or a reference object of the ultrasound detector; and d)
information concerning a region or an area, which is actually
covered by an ultrasound image that is represented by the first
image data.
26. The method of claim 25 wherein said second image data comprises
at least one of image data generated by a computer tomography, a
magnetic resonance, a positron emission tomography, an X-ray and a
three-dimensional ultrasound imaging device.
27. The method of claim 25 wherein said geometry data comprise
information concerning a detector position of the ultrasound
detector relative to at least one of a position sensor and a signal
source, for determining at least one of a location and an
orientation of said ultrasonic detector.
28. The method of claim 25 wherein said geometry data are at least
one of generated by and transferred from a control unit of the
ultrasound detector and wherein the control unit is adapted to
control an image data generation of the ultrasound detector.
29. The method of claim 25 wherein the combination device
continuously displays a first image of the object corresponding to
the repeatedly generated first image data and continuously displays
a second image of the object corresponding to the second image
data, wherein at least one of the orientation and scaling of at
least a part of the object is identical in the first and in the
second image.
30. The method of claim 25 wherein at least a part of the geometry
data is repeatedly transferred to the combination device.
31. The method of claim 25 wherein said geometry data comprise data
defining information concerning at least one spatial dimension of
an image unit of said first image data.
32. The method of claim 25 wherein said geometry data comprise data
defining information concerning at least an image position of at
least a part of an image, which is represented by the first image
data relative to a reference point of the ultrasound detector or
relative to a reference point or reference object in the ultrasound
image.
33. The method of claim 25 wherein said geometry data comprise data
defining information concerning at least an orientation of the
ultrasound image relative to a reference point or a reference
object of the ultrasound detector.
34. The method of claim 25 wherein said geometry data comprise data
defining information concerning at least a region or an area which
is actually covered by an ultrasound image that is represented by
said first image data.
35. The method of claim 25 wherein said at least one data
connection comprises a first data connection designed to transfer
from said ultrasound detector to said combination device said
geometry data and a second data connection designed to transfer
from said ultrasound detector to said combination device said first
image data.
36. The method of claim 25 wherein said first image data is derived
from said ultrasound detector.
37. The method of claim 25 wherein said second image data is
derived from a second detector other than said ultrasound
detector.
38. The method of claim 37 wherein said second detector is not an
ultrasound detector.
39. A method of combining first and second image data of an object,
comprising: repeatedly generating first image data of the object
using an ultrasound detector; at least one of storing and receiving
second image data of the object; combining said first image data
and said second image data using geometry data; wherein said
geometry data comprise data defining at least one of the following:
a) information concerning at least one spatial dimension of an
image unit of the first image data; b) information concerning an
image position of at least a part of an image, which is represented
by the first image data, relative to a reference point of the
ultrasound detector or relative to a reference point or reference
object in the ultrasound image; and c) information concerning an
orientation of the ultrasound image relative to a reference point
or a reference object of the ultrasound detector; and d)
information concerning a region or an area, which is actually
covered by an ultrasound image that is represented by the first
image data.
40. The method of claim 39 wherein said second image data comprises
at least one of image data generated by a computer tomography, a
magnetic resonance, a positron emission tomography, an X-ray and a
three-dimensional ultrasound imaging device.
41. The method of claim 39 wherein the geometry data comprise data
defining information concerning a detector position of the
ultrasound detector relative to at least one of a position sensor
and a signal source, for determining at least one of a location and
an orientation of the ultrasound detector.
42. The method of claim 39 wherein the geometry data are at least
one of generated by and transferred from a control unit of the
ultrasound detector and wherein the control unit is adapted to
control an image data generation of the ultrasound detector.
43. The method of claim 39 wherein the combination device
continuously displays a first image of the object corresponding to
the repeatedly generated first image data and continuously displays
a second image of the object corresponding to the second image
data, wherein the at least one of the orientation and scaling of at
least a part of the object is identical in the first and in the
second image.
44. The method of claim 39 wherein at least a part of the geometry
data is repeatedly transferred to the combination device.
45. The method of claim 39 wherein said geometry data comprise data
defining information concerning at least one spatial dimension of
an image unit of said first image data.
46. The method of claim 39 wherein said geometry data comprise data
defining information concerning at least an image position of at
least a part of an image, which is represented by the first image
data relative to a reference point of the ultrasound detector or
relative to a reference point or reference object in the ultrasound
image.
47. The method of claim 39 wherein said geometry data comprise data
defining information concerning at least an orientation of the
ultrasound image relative to a reference point or a reference
object of the ultrasound detector.
48. The method of claim 39 wherein said geometry data comprise data
defining information concerning at least a region or an area which
is actually covered by an ultrasound image that is represented by
said first image data.
49. The method of claim 39 wherein said first image data is derived
from said ultrasound detector.
50. The method of claim 39 wherein said second image data is
derived from a second detector other than said ultrasound
detector.
51. The method of claim 50 wherein said second detector is not an
ultrasound detector.
Description
BACKGROUND OF THE INVENTION
[0001] The invention relates to an apparatus and a method for/of
combining first and second image data of an object. An ultrasound
(US) detector repeatedly generates the first image data of the
object and the second image data of the object are generated
separately. In the apparatus, means for storing and/or receiving
the second image data of the object (such as an image data storage
and/or an interface) are provided. For example, the second image
data may have been recorded earlier by a computer tomography (CT),
a magnetic resonance (MR), a positron emission tomography (PET), an
X-ray and/or a three-dimensional (3D) US imaging device. In
particular, any 3D image information can be used as the second
image data. A combination device combines the first and second
image data of the object. The combined image data may be displayed
in separate areas of a screen and/or may be superimposed on a
screen. More particularly, the invention may be applied in the
field of stereographic diagnosis concerning human or animal bodies,
but also concerning material research and/or material
examination.
[0002] Ultrasound detectors are comparatively easy to handle and
are able to deliver image information quasi-continuously and,
approximately, in real-time. However, in many applications, other
imaging technologies (such as the ones mentioned above) provide
better results. Therefore, it has been proposed earlier to combine
image information of better quality, which has been recorded
earlier, with real-time ultrasound image information.
[0003] In the scientific publication of Pagoulatos et al.:
"Interactive 3-D Registration of Ultrasound and Magnetic Resonance
Images Based on a Magnetic Position Sensor", published in IEEE
TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE, VOL. 3, NO.
4, December 1999, describes an apparatus of the type mentioned
above. It is proposed in the publication to use an MR imaging
device and to register the imaging object relative to the MR
imaging device. The term "register" means that the geometry of the
object and the coordinate system of the MR imaging device are
brought into a defined relation. The apparatus comprises a position
sensor, which is firmly coupled to an ultrasound probe. Using the
position sensor and due to the fact that its relative position to
the ultrasound probe does not vary, it is possible to track the
position and the orientation of the ultrasound probe. However, it
is necessary to calibrate the position sensor relative to the
ultrasound probe and to register the imaging object relative to the
coordinate system of the ultrasound probe. As a result, the MR
imaging information and the ultrasound imaging information can be
combined.
[0004] Using systems of the type described before, it is possible
to display MR image information of the same orientation and/or
scaled in the same manner as the most recent US image. In other
words: it can be simulated that the MR image is recorded in
real-time, provided that the image data processing is fast
enough.
[0005] However, the various calibration and registration procedures
are time-consuming and need to be performed with care in order to
obtain good results. For example, the position of a set of
landmarks on the surface of the object and/or anatomical markers
(in case of a human or animal body) is to be detected during
registration.
[0006] Furthermore, modern US systems enable the user to vary the
penetration depth of the US image in the object and/or to change
the ultrasound probe. As a consequence, the calibration of the
position sensor relative to the ultrasound probe and the
registration of the imaging object relative to the coordinate
system of the ultrasound probe are no longer valid.
[0007] There are prior art ultrasound devices with pre-calibrated
pixel sizes wherein the pixel size can be varied by the user.
However, only step-like increases or decreases of the pixel size
(or of a spatial dimension of the ultrasound image) can be
performed with these prior art devices.
[0008] U.S. Pat. No. 6,546,279 B1 discloses a method and an
arrangement for locating, vectoring and inserting a needle-like
medical device toward and into a targeted patient anatomic feature
while the patient is being imaged with multi-modality medical
imaging equipment. In the second embodiment of the document at
least a portion of the patient is imaged with a first imaging
technique (such as computed tomography) to provide a first set of
imaging data, which has a fixed frame of reference. Ultrasound
imaging data is obtained. The ultrasound imaging data is not fixed
relative to the fixed frame of reference. Position data is
determined for the ultrasound device. Using the determined position
data and the ultrasound imaging data, a converted set of imaging
data is provided which is referenced to the fixed frame of
reference.
[0009] The publication "sensor fusion for surgical applications" by
Jim Leonhard (15.sup.th Annual AESS/IEEE Dayton Section Symposium.
Sensing the world: Analog sensors and systems across the spectrum
(Cat. No.98EX178), pages 37-44, XP002253643, New York, N.Y., USA,
IEEE, USA) discloses a surgical navigation system which combines
preoperate 3d imagery and intra-operate localisation to register a
patient.
[0010] U.S. Pat. No. 2002/0128550 A1 discloses a diagnostic imaging
system. Magnetic resonance image and ultrasound images are
registered in a common reference frame.
[0011] It is an object of the present invention to provide an
apparatus and a method of the type indicated above, which allow
lessening the effects of the disadvantages mentioned before, which
facilitate the handling by the user and which allow using a greater
variety of features of the ultrasound system without loosing time
for calibration and/or registration. In particular, it is desirable
to adjust the settings of the ultrasound system and/or to change
the ultrasound probe and to continue with the ultrasound imaging
process without interruption.
SUMMARY OF THE INVENTION
[0012] A method is provided of combining first and second image
data of an object, wherein the first image data of the object are
repeatedly generated by an ultrasound detector, wherein the second
image data of the object are generated separately, wherein the
first image data are transferred from the ultrasound detector to a
combination device and wherein the combination device is adapted to
combine the first and second image data of the object. Geometry
data is transferred in addition to the first image data from the
ultrasound detector to the combination device.
[0013] In particular, the geometry data comprise information for
establishing a link between the geometry of the first image data
and the geometry of the second image data.
[0014] Preferably, the geometry data are transferred directly from
the ultrasound detector to the combination device, e.g. via an
additional cable which connects the detects and the device.
[0015] Further, an apparatus is provided for combining first and
second image data of an object, the apparatus comprising an
ultrasound detector for repeatedly generating the first image data
of the object; means for storing and/or receiving the second image
data of the object; a combination device which is adapted to
combine the first and second image data of the object; and an image
data connection from the ultrasound detector to the combination
device for transferring the first image data. The ultrasound
detector is connected to the combination device by a geometry data
connection, wherein the geometry data connection, the ultrasound
detector and the combination device are adapted to transfer
geometry data additionally to the first image data from the
ultrasound detector to the combination device.
[0016] The geometry data may, for example, comprise information for
establishing a link between the geometry of the first image data
and the geometry of the second image data. The "link" (or, in other
words, a processable relation between the first and the second
image data) allows, for example, the determination of an
overlapping region of a first image represented by the first image
data and of a second image represented by the second image data.
According to a preferred application, the combination device
continuously displays a first image of the object corresponding to
the repeatedly generated first image data and continuously displays
a second image of the object corresponding to the second image
data, wherein the orientation and/or scaling of at least a part of
the object is identical in the first and in the second image. In
particular, the second image data may be displayed, as if the
second image data have repeatedly been recorded instead of or in
addition to the first image data.
[0017] As a consequence of the fact that the geometry data are
available to the combination device, the effort for re-calibrating
the ultrasound detector relative to a tracking sensor (if any)
and/or for re-registrating the object relative to the coordinate
system of the ultrasound detector can be reduced or eliminated.
[0018] The geometry data may comprise one or more than one of the
following type of information. In particular, any combination
and/or aggregation of the following type of information can be
included in the geometry data:
[0019] a) information concerning at least one spatial dimension of
an image unit of the first image data, in particular of a pixel
(preferably separately for different directions of a coordinate
system);
[0020] b) information concerning an image position of at least a
part of an image, which is represented by the first image data,
relative to a reference point of the ultrasound detector or
relative to a reference point or reference object in the ultrasound
image. This information is particularly useful, if a user can
adjust a zoom factor of the ultrasound image. For example, this
information comprises a distance in image units (e.g. pixels). In
combination with the scaling information of item a), the distance
may be defined in cm or another unit of length;
[0021] c) information concerning an orientation of the ultrasound
image relative to a reference point or a reference object of the
ultrasound detector (in particular an ultrasound probe of the
detector). For example, this information may comprise the
orientation of at least one axis of a coordinate system of the
ultrasound image;
[0022] d) information concerning a region or an area, which is
actually covered by an ultrasound image that is represented by the
first image data; and
[0023] e) information concerning a detector position of the
ultrasound detector relative to a position sensor for determining a
location and/or an orientation of the ultrasound detector. Instead
of or in addition to a position sensor, a signal source may be
coupled to the ultrasound probe, wherein the signal can be
evaluated in order to determine the position of the probe. For
example, such information may be collected once in advance and may
be saved individually for each ultrasound probe, which can be
connected to the ultrasound system/device. In this case, it is
sufficient during operation to transfer simply an identification
signal, which enables to identify the probe that is used. The
combination device can select the respective geometry information
using the identification information. In a specific embodiment, the
information concerning the relative position, which is transferred
or saved, may be a calibration matrix.
[0024] Preferably, all of these types of information are
transferred from the ultrasound detector to the combination
device.
[0025] Since information concerning at least one spatial dimension
of an image unit of the first image data (e. g. the pixel size) can
be transferred from the ultrasound device to the combination
device, according to one embodiment of the present invention, the
user can choose the spatial dimension within the respective range
of the ultrasound device. Consequently, a continuously adjustable
pixel size can be implemented in the ultrasound device.
[0026] The information concerning an image position relative to a
reference point of the ultrasound detector facilitates the
combination of the two different types of image data and makes it
possible to adjust the position of the ultrasound image, for
example by using a control knob provided at the ultrasound
probe.
[0027] The information concerning an orientation of the ultrasound
image relative to a reference point of the ultrasound detector
enable the user to change the orientation by operating a control
element at the ultrasound probe.
[0028] Similarly to the information concerning the pixel size, the
information concerning a region or an area, which is actually
covered by an ultrasound image facilitates the combination of the
ultrasound image data with the other image data.
[0029] According to a preferred embodiment at least a part of the
geometry data is repeatedly transferred to the combination device,
in particular every time when the first image data are generated
and/or when the first image data are transferred to the ultrasound
detector. The geometry data may be transferred on request and/or
without request from the combination device and/or from another
part or unit (for example of a central control unit) of the
apparatus.
[0030] There are further possibilities to use the additional data
connection between the ultrasound detector and the combination
device, or to use a further data connection. For example, a mode
and/or a user setting of the ultrasound detector may be transferred
from the ultrasound detector to the combination device, on request
or without request of the combination device or another unit of the
apparatus. E.g. a colour of the ultrasound image to be used in
displaying the ultrasound image, a repetition frequency of the
ultrasound image generation (for example in order to give an
indication to the user, if the frequency is too low) and/or
information representing the object may be transferred.
[0031] If the ultrasound detector comprises a control unit for
controlling an image data generation of the ultrasound detector,
the control unit may be adapted to generate at least a part of the
geometry data. For example, the control unit can adjust a
penetration depth of the ultrasound image, using a velocity value
of the ultrasound waves in the object, by setting a time limit for
detection of US echo signals. In this case, the control unit can
calculate the penetration depth and can transfer information about
the penetration depth to the combination device. Further, the width
of an image recording area of an ultrasound probe may be available
to the control unit for control purposes and the control unit can
transfer this information to the combination device.
[0032] In a preferred embodiment of the apparatus the image data
connection is adapted to directly transfer the first image data in
a digital format from the ultrasound detector to the combination
device. This saves time for transferring the first image data and
reduces costs and effort of the apparatus. Furthermore, the
ultrasound detector, the combination device and (optionally)
further parts or units of an imaging system may be integrated in
one and the same device. For example, several or all of the units
of such a device may be connected to a data bus system for
transferring data.
[0033] Generally, the image data connection and/or the geometry
data connection may be realised by a data bus (e.g. USB or
FireWire, IEEE 1394) and/or may be part of a data network.
Preferably, the embodiment of the connection or connections shall
allow displaying a first image (represented by the first image
data) and a second image (represented by the second image data)
approximately in real-time of the generation of the first image
data, for example within less than 100 ms after the generation.
Ultrasound is particularly useful in order to generate the first
image data, since the generation process can be performed with high
repetition frequency (quasi-continuously). The corresponding data
connection and the following data processing should not delay the
displaying in way, which can be noticed by the user.
BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In the following, examples and possible further features of
the invention are described by way of reference to the accompanied
drawing. The embodiment shown in the FIGS. 1 to 3 and 5 represent
the currently known best mode of the invention. However, the
invention is not limited to the features described in the following
description. The figures of the drawing schematically show:
[0035] FIG. 1 an arrangement 2 comprising an apparatus for
combining ultrasound image data with a second type of data, e.g. CT
image data;
[0036] FIG. 2 a more detailed view of the ultrasound detector shown
in FIG. 1;
[0037] FIG. 3 a first example of an arrangement of components,
which contribute to a processing of the ultrasound image data;
[0038] FIG. 4 a second example of an arrangement of components,
which contribute to a processing of the ultrasound image data;
[0039] FIG. 5 a flow-chart of processing image data.
DETAILED DESCRIPTION
[0040] Images of an object 3 (shown in FIG. 1) are to be displayed
on a screen 6. An ultrasound detector 1 generates first image data
of the object 3 and transfers the first image data to a combination
device 5 via an image data connection 10. The combination device 5
comprises a data storage 4, which contains second image data that
have previously been generated by a separate device (not shown in
FIG. 1). The combination device 5 is adapted to combine the first
and second image data and to display them on a screen 6, which is
connected to the combination device 5. For example, the first and
second image data may be displayed separately on a split screen or
may be superimposed. In any case, it is preferred that a first
image, which is generated using the first image data, and a second
image, which is generated using the second image data, precisely
show at least partially the same area or region of the object 3 in
the same orientation (angle of view) and scaling (dimensions).
[0041] The ultrasound detector 1 and the combination device 5 are
connected to each other by an additional data connection 12 for
transferring geometry data from the ultrasound detector 1 to the
combination device 5. In particular, the geometry data connection
12 may be connected (as shown in FIG. 2). to a control unit 14 of
the ultrasound detector 1.
[0042] In practice, the data connections 10, 12 may be realised by
separate data connection links or by the same data connection link.
For example, a "link" may comprise a connection line, a plurality
of connection lines and/or a digital data bus or bus system.
[0043] An ultrasound probe 16 (FIG. 2) of the ultrasound detector 1
is firmly coupled to a position sensor 18 of a tracking system. The
determination of the orientation and the location of such a
position sensor and, thereby, of the ultrasound probe is known in
the art (see the above-cited publication of Pagoulatos et al.). For
example, magnetic and/or optical (e.g. infrared) signals may be
used by the tracking system. The position sensor 18 is connected to
a tracking system control unit 8 and the control unit 8 is
connected to the combination device 5. During operation of the
arrangement 2, the control unit 8 repeatedly or quasi-continuously
transfers information concerning the position and concerning the
orientation of the ultrasound probe 16 to the combination unit 5.
Alternatively, this information may be transferred from the US
detector to the combination device. I.e. this information might be
at least partially included in the geometry data, which are
transferred.
[0044] As shown in FIG. 2, the ultrasound device 1 may, for
example, comprise an ultrasound probe 16, which is connected to the
ultrasound control unit 14 via a flexible cord 17 for transferring
echo signals to the control unit 14. On the other hand, the control
unit 14 transfers control signals to the ultrasound probe via the
cord 17. Also, it is possible that at least a part of the geometry
information is transferred from the ultrasound probe 16 to the
control unit 14 and/or that at least a part of the geometry
information generated by the control unit 14 is based on and/or
derived from information, which is transferred from the ultrasound
probe 16 to the control unit 14. For example, the ultrasound probe
16 may be replaced and, therefore, transfers information concerning
its identity to the combination device 5. In the combination device
5, in an additional unit of the arrangement 2, and/or in the
combination device 5, information concerning the relative position
and/or orientation of the specific ultrasound probe 16 relative to
the position sensor 18 may be saved. For example, the ultrasound
probe 16 may comprise a clip for attaching the position sensor 18.
Therefore, it is possible to precisely position and orientate the
position sensor 18 relative to the ultrasound probe 16 and to
determine as well as to save the respective geometry data in
advance. After replacement of the ultrasound probe 16, the identity
information and the saved geometry information can be combined. As
a consequence, it is not necessary to re-calibrate the arrangement
consisting of the ultrasound detector 1 and the position sensor
18.
[0045] An input unit 20 is connected to the ultrasound control unit
14, for example for inputting settings of the ultrasound detector,
such as a penetration depth or range of the ultrasound image.
Further, the user may change the orientation of the ultrasound
image via the input unit 20.
[0046] According to a preferred embodiment of the invention, the
ultrasound image data are generated from analog signals in a unit
22 of the ultrasound detector. The unit 22 may, for example, be
controlled by the control unit 14 (as shown in the figure), be a
part of the control unit 14 or be realised by a combination of the
ultrasound probe 16 and the control unit 14. The unit 22 is
connected to an A/D-converter 24, which converts the analog image
signal to a digital image signal. The A/D-converter 24 is connected
to an image data processing unit 26, which is also connected to the
control unit 14 or to an alternative unit of the ultrasound
detector that transfers geometry information to the image data
processing unit 26. Thus, the image data processing unit 26 can
combine the geometry information with the image data and/or can
transferred both information/data to a further unit (not shown in
FIG. 3). For example, the image data processing unit 26 may be
identical to or part of the combination device 5. In this case, the
image data processing unit 26 also combines the first and second
image information and the unit 26 may be connected to a screen 6
(as shown in the figure).
[0047] The embodiment of FIG. 3 is of particular advantage, if the
ultrasound detector and the combination device are integrated in
one device.
[0048] In an alternative arrangement, the ultrasound detector may
directly generate digital image data, without conversion from
analog to digital.
[0049] An alternative arrangement is shown in FIG. 4. The same
reference numeral denote units or devices having the same or a
corresponding function. A D/A-converter 23 converts the digital
image signal received via a connection from the unit 22 or from a
corresponding unit to an analog image signal, for example to a
signal in standard video format. An A/D-converter 25 that may be
part of the combination device (e.g. a video capture card) is
connected to the unit 23. The unit 25 converts the analog signal to
a digital signal, e.g. in pixel format. Then, the unit 26 processes
the digital signal.
[0050] An example of processing and combining ultrasound image
information is described in the following with reference to FIG.
5.
[0051] In step S1, geometry information concerning the scaling of
the ultrasound image information are combined with the ultrasound
image information. For example, the ultrasound image data contains
information concerning the intensity of image units (e.g. pixels)
before step S1, but no information concerning the dimension of the
image units. The dimensional information is added by performing
step S1.
[0052] In step S2, calibration information concerning the relative
position of the ultrasound detector to the position sensor is added
and/or combined with the information resulting from step S1. The
calibration information may comprise as well information concerning
the orientation of the ultrasound image, which may be changed by
the user. As a result, it is now possible to identify the
orientation and location of specific areas of the ultrasound
image.
[0053] In step S3, the geometry information concerning the
registration of the object within the coordinate system of the
ultrasound system or the tracking system is added.
[0054] In step S4, the ultrasound image data resulting from step S3
can be combined with the second image data.
[0055] In practice, at least some of the processing operations of
steps S1 to S4 may be realised by performing a polygon scanline
method, which is described for example in the book "Computer
Graphics. Principles and Practice." by James D. Foley, Andries
VanDam, Steven K. Feiner, published by Addison-Wesley, Boston,
U.S.A. The combination device 5 shown in FIG. 1 may perform the
data processing, for example.
[0056] For example, it is first calculated whether an overlapping
region of the respective images exists. If this is the case, the
corresponding overlapping region of the second image data is read
out from the data storage and is prepared for displaying (step S5).
The corresponding data of the second image data are processed
before displaying so that the second image, which is based on the
second image data, can be displayed in the same orientation and
scaling as the ultrasound image.
[0057] The procedure described before is preferred, if the
ultrasound image is two-dimensional. For 3D-ultrasound image
information it is as well possible, to choose the orientation (i.e.
the angle of the view) of the ultrasound image to be shown, in
particular to adapt the orientation according to other criteria. In
any case, it is possible to adapt the scaling of the ultrasound
image before displaying. Instead of, or additionally to displaying,
the combined first and second image information may be processed in
another way. For example, they can be stored and/or evaluated in a
different manner.
* * * * *