U.S. patent application number 11/538511 was filed with the patent office on 2007-04-05 for data distribution system.
This patent application is currently assigned to ALGOTEC SYSTEMS LTD.. Invention is credited to Ran Bar-Sella, Menashe Benjamin, Michael Elad, Jacob Margolin, Yosef Reichman.
Application Number | 20070075999 11/538511 |
Document ID | / |
Family ID | 11069426 |
Filed Date | 2007-04-05 |
United States Patent
Application |
20070075999 |
Kind Code |
A1 |
Benjamin; Menashe ; et
al. |
April 5, 2007 |
Data Distribution System
Abstract
An interactive method for allowing a user to obtain image data
for diagnostic purposes from a server having access to stored data,
comprising: connecting a user's computer to the server over a
communication network; receiving from the server image
reconstruction software for the user's computer; requesting
specific image data for transmission from the server to the user's
computer; progressively transmitting the requested specific image
data over the network from the server to user's computer; and
repeatedly reconstructing diagnostic quality images, from the
progressively received image data, at different quality levels,
using the reconstruction software on the user's computer.
Inventors: |
Benjamin; Menashe; (Raanana,
IL) ; Elad; Michael; (Haifa, IL) ; Bar-Sella;
Ran; (Haifa, IL) ; Reichman; Yosef;
(Kfar-Saba, IL) ; Margolin; Jacob; (Atlanta,
GA) |
Correspondence
Address: |
WOLF, BLOCK, SCHORR & SOLIS-COHEN LLP
250 PARK AVENUE
NEW YORK
NY
10177
US
|
Assignee: |
ALGOTEC SYSTEMS LTD.
P.O. Box 46
Raanana
IL
|
Family ID: |
11069426 |
Appl. No.: |
11/538511 |
Filed: |
October 4, 2006 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09300490 |
Apr 28, 1999 |
|
|
|
11538511 |
Oct 4, 2006 |
|
|
|
PCT/IL97/00349 |
Oct 29, 1997 |
|
|
|
09300490 |
Apr 28, 1999 |
|
|
|
Current U.S.
Class: |
345/428 |
Current CPC
Class: |
H04N 21/47202 20130101;
H04N 21/23439 20130101; H04N 21/8153 20130101; G16H 80/00 20180101;
G16H 10/60 20180101; G16H 40/67 20180101; G16H 30/40 20180101; G16H
30/20 20180101; H04N 7/17318 20130101; H04N 1/217 20130101; G16H
40/20 20180101 |
Class at
Publication: |
345/428 |
International
Class: |
G06T 17/00 20060101
G06T017/00 |
Foreign Application Data
Date |
Code |
Application Number |
Oct 30, 1996 |
IL |
119523 |
Claims
1. An interactive method for allowing a user to obtain image data
for diagnostic purposes from a server having access to stored data,
comprising: connecting a user's computer to the server over a
communication network; receiving from the server image
reconstruction software for the user's computer; requesting
specific image data for transmission from the server to the user's
computer; progressively transmitting the requested specific image
data over the network from the server to user's computer; and
repeatedly reconstructing diagnostic quality images, from the
progressively received image data, at different quality levels,
using the reconstruction software on the user's computer.
2. A method according to claim 1, comprising image processing said
reconstructed image using said reconstruction software on said
user's computer.
3. A method according to claim 1, comprising: receiving from the
server image selection software for the user's computer, wherein
said image selection software is used for said requesting.
4. A method according to claim 3, wherein said image selection
software controls the transmission of the image data.
5. A method according to claim 3, wherein said image selection
software displays images from said server.
6. A method according to claim 3, wherein said image selection
software and said reconstruction software are received
together.
7. A method according to claim 3, wherein said image selection
software and said reconstruction software comprise a single
software unit.
8. A method according to claim 3, wherein said image selection
software is operative to stop the transmission of the data, after
at least a low quality image is reconstructed and viewed from said
data.
9. A method according to claim 8, wherein said image selection
software is operative to restart the transmission of the data of
the entire image, after said stopping.
10. A method according to claim 3, wherein said image selection
software controls processing of data at said server, prior to its
transmission.
11. A method according to claim 10, wherein said processing
comprises reducing said data from a large bit-per-pixel ratio to a
small bit-per-pixel ratio, independently of details of said
request, except an identification of the data requested.
12. A method according to claim 10, comprising interactively
providing user input to said image selection software, to affect
said control.
13. A method according to claim 3, wherein said image selection
software controls said server to selectively transmit only portions
of the image data.
14. A method according to claim 3, wherein the image selection
software comprises application software coded using a device
independent network programming language.
15. A method according to claim 3, wherein the reconstruction
software comprises application software coded using a device
independent network programming language.
16. A method according to claim 14, wherein said language comprises
Java.
17. A method according to claim 14, wherein said language comprises
ActiveX.
18. A method according to claim 1, wherein reconstructing
comprises: reconstructing images of progressively improving quality
from the progressively received data; using the produced improved
images of progressively produced quality to decide on processing of
the images, wherein said processing comprises gray-level windowing;
and processing said images, prior to the progressively received
data being completely received.
19. A method according to claim 1, wherein reconstructing
comprises: reconstructing images of progressively improving quality
from the progressively received data; using the produced improved
images of progressively produced quality to decide on processing of
the images; and interactively selecting regions of interests in the
images based on said progressively improved images, prior to the
progressively received data being completely received.
20. A method according to claim 1, wherein said data is
progressively transmitted while a lower quality image is being
reconstructed from said data.
21. A method according to claim 11, wherein reducing the
bit-per-pixel ratio comprises: calculating an average "M" of the
gray values in the image and a standard deviation "S" of said gray
values; and resealing these values in the range [(M-S/2)..(M+S/2)]
to obtain a new lower number of bits per pixel.
22. A method according to claim 11, wherein reducing the
bit-per-pixel ratio comprises: estimating the mean and standard
deviation of the gray levels locally; and resealing these values to
obtain a new lower number of bits per pixel.
23. A method according to claim 1, wherein progressively
transmitting the requested data over the network comprises:
recomposing the image into a pyramidal structure comprised of
layers, said layers ranging sequentially from a layer having the
least amount of data to a layer having the most data; and
transmitting the layers making up the pyramid individually starting
with the layer with the least amount of data to enable the user to
view a progressively improving image to decide on further
transmission of the image.
24. A method according to claim 23, wherein recomposing the image
into a pyramidal structure comprises reducing the image to provide
the different layers at the transmitting end for progressive
transmittal.
25. A method according to claim 24, wherein reducing comprises
discarding alternate rows and columns to create an image that is a
quarter of the size of the original image.
26. A method according to claim 23, comprising: providing a first
layer with reduced resolution in the pyramidal structure; providing
remaining layers that contain residual values with increased
resolution; and progressively receiving the data used to provide
images based on the received data of progressively improved
resolution.
27. A method according to claim 1, comprising: compressing the
requested data transmitted over the network; and decompressing the
received required data to provide images.
28. A method according to claim 27, wherein compressing comprises
spatially decorrelating the data by predicting each pixel at the
current resolution using its spatial casual neighbors.
29. A method according to claim 27, wherein compressing comprises
temporally decorrelating each pixel by predicting each pixel value
at the current resolution using the values of temporal neighbors
from previous images.
30. A method according to claim 28, wherein a predictor X used in
predicting each pixel value for a single image is equal to f(a, b,
c), wherein a, b and c are previously predicted neighboring
pixels.
31. A method according to claim 29, wherein a predictor X used in
predicting each pixel value for a group of images equals f(a, b, c,
a1, b1, cl, x1) wherein a, b and c are previously predicted
neighboring pixels in a same image and a1, b1, c1 and x1 are
corresponding pixels in a previously predicted image of the image
group.
32. A method according to claim 27, wherein said compressing and
said decompressing use entropy coding and decoding
respectively.
33. A method according to claim 32, wherein said entropy coding and
decoding are accomplished using Golomb Rice entropy coding and
decoding.
34. A method according to claim 1 wherein connecting the user
computer to the server over a communication network comprises
connecting over the Internet.
35. A method according to claim 1 wherein connecting the user
computer to the server over a communication network comprises using
a dial up communication system.
36. A method according to claim 1 wherein connecting the user
computer to the server over the communication network comprises
using networking facilities.
37. A method according to claim 1, wherein the stored data
comprises data for a plurality of "postage stamp" images.
38. A method according to claim 37, comprising using "postage
stamp" images as a catalog for selecting those images for which no
further data is to be transmitted and those images for which
further data is to be transmitted.
39. A method according to claim 37, wherein said postage stamps
comprise a lowest level in a pyramidal representation of said
images.
40. A method according to claim 1, wherein progressively
transmitting comprises serially transmitting a sequence of images
of increasing resolution, each image being progressively
transmitted.
41. A method according to claim 1, wherein progressively
transmitting comprises transmitting data operative to reconstruct
images of increasing resolution.
42. A method according to claim 1, wherein progressively
transmitting the requested data over the network comprises
segmenting an image into background parts and tissue parts, and
transmitting the tissue parts first.
43. A method according to claim 1, wherein said requesting
comprises requesting after said receiving and during a same
session.
44. A method according to claim 1, wherein said reconstructing
comprises showing progressively reconstructed images.
45. A method according to claim 10, wherein said server processes
said image data for enhancing transmission thereof.
46. A method according to claim 1, wherein said server only
transmits image data received from a data store connected to said
server by a network connection.
47. A method according to claim 2, wherein the image reconstruction
and processing software comprises application software coded using
a device independent network programming language.
48. A method according to claim 3, wherein said image selection
software controls said server to selectively transmit only a region
of interest of said image.
49. A method according to claim 1, wherein said connecting and said
receiving are performed via an industry standard browser
software.
50. A method according to claim 1, wherein receiving from the
server image reconstruction software is performed in each session
in which the user receives progressively transmitted images.
51. A method according to claim 1, wherein receiving from the
server image reconstruction software is performed after requesting
image data by the user.
Description
RELATED APPLICATION DATA
[0001] The present application is a continuation of U.S. Ser. No.
09/300,490 filed on Apr. 28, 1999, which is a continuation of
PCT/IL97/00349 filed on Oct. 29, 1997, the disclosures of which are
fully incorporated herein by reference.
FIELD OF THE INVENTION
[0002] This invention is concerned with interactive communication
systems linking central locations having access to stores of data
and images used for medical purposes and a plurality of outlying
users of the images and data for medical review, processing,
assessment and diagnostics.
BACKGROUND OF THE INVENTION
[0003] Modern hospitals and health centers today usually have
several computerized systems for medical information gathering,
exchange, storage and processing. Herein such a system is referred
to as a "data source". Medical information may come in textual,
voice, sound, graphical, and image modalities. Such medical
information may be required by authorized personnel, including
those located outside the hospital premises, and equipped with
computers of some sort. Herein the requiring side is referred to as
the "user". In present systems the users are equipped with their
own software to access the data source. Difficulties in the use of
such computerized systems are caused by such things as the varied
networking procedures required to fetch the data, the lack of an
industry standard, the lack of an easy to use user interface and,
in the case of image data transfer, the channel bandwidth
requirements along with the typically large volumes of the image
information, which in turn translates into very long transmission
periods. In addition to that, a typical user might be required to
master the skills of operating a large number of software systems
like those used with various data processors, the varied
communication software, software installations procedures, etc. The
system administrator needs to install the different types of
application software in large numbers of computers, and update this
software, in each computer, every time a new version is used. This
proliferation of software and hardware in the medical data
processing systems make such systems difficult to maintain and a
burden to update.
[0004] Presently, more and more hospitals and clinics are uniting
for economic reasons to form healthcare enterprises with
consolidated resources, having a single headquarters for managing
the organization. The consolidation of resources also takes place
inside individual hospitals, with the primary goal of facilitating
data exchanges inside the hospital, with hospital personnel outside
the hospital premises, as well as with other related facilities and
with the enterprise headquarters. Generally, most individual
facilities that make up the enterprise operate special systems to
store and manage various parts of their clinical data. One can
generally view these systems as being composed of data acquisition
devices, data storage devices (data banks), and data management and
communication modules. The users are connected to the data banks
via various networking procedures and communications protocols.
These users may operate a variety of computer hardware systems.
Access of each user to stored patient data is presently done
through the use of special application software on the user's
computer. Since presently, most health and medical organizations
have constructed their information systems and communications
network over a period of time, access to these systems is often
complicated, and sometimes requires the user to master several
application, software and communication protocols. Typically, no
common access method or user interface is available to the user,
and users are often confined to the use of particular hardware at a
specific location to access the data. The need to access image data
further complicates the situation. The large bandwidth required
from the communications link, the large data volumes, and the
special processing that is usually needed, often requires the use
of special software and hardware on the user's side .
[0005] Thus, one problem encountered with the present server-user
communication systems for transferring medical data is the many
different interfaces, software applications, and communication
protocols required and the many different types of work stations
that make up the "installed base". Due to this proliferation of
different work stations requiring different software applications,
interfaces and communications protocols, then whenever a new
improved system or a new data type become available, the many
different work stations have to be equipped with the software for
utilizing the new systems or data. This is not only expensive, but
time consuming in that the installation of the software in each of
the many different work stations and the central server requires
time and usually requires expertise beyond that of the doctor or
medical professional using the workstation.
[0006] A second and equally troublesome problem is encountered when
the data requested by the user includes images that must be
transmitted over a given enterprise network. This is due to the
long time required for transmitting image data as compared to other
forms of data. Image compression is used to reduce transmission
times. For clinical image data, special precautions must be taken
if lossy compression is implemented, due to the potential loss of
possibly vital findings. Lossless compression schemes are therefore
employed, which provide a relatively small reduction of image
transmission time (a factor of 2-3 for radiology images).
Interactive compression schemes, that optimize the transmission
time for any given user and user type are currently not available
in existing healthcare information systems. Such an interactive
compression scheme is presented as part of this invention.
[0007] The above can be summarized in a conceptual diagram (FIG.
1). An enterprise wide healthcare information system 11 may be
conceptually conceived as comprising several local facilities,
12-14, connected to a central facility 16. Each local facility
comprises data sources 17 connected through appropriate interfaces
such as interface 18, to the local network, to which the various
local users 19 are connected too. The local network of each local
facility is in turn connected to the central facility through
another interface (possibly including firewalls and security
features). The central facility comprises a similar structure, with
the addition of central repositories 23, data bases, and data
management tools. This structure of the presently available systems
suffers from the problems described above. Thus those skilled in
the art are still searching for effective solution to the existing
problems.
BRIEF DESCRIPTION OF THE INVENTION
[0008] This invention provides systems and methods for largely
overcoming the above problems, among other things, by adding data
distribution servers, such as local server 24 and central server 26
as indicated in FIG. 2. The concept detailed in FIG. 2 is logically
summarized in FIG. 3. The various clinical data acquisition devices
and data banks are conceptually grouped into a "data source" block
28. A server 29 is introduced as an intermediate level between the
data source and the users. The introduction of the server, with the
appropriate functionality and data handling algorithms, alleviates
many of the problems presented above.
[0009] While the concept and method introduced here is applicable
for the distribution of any type of clinical and non-clinical
information, this invention will focus on solving the problem of
distributing clinical images over the network, which poses one of
the major obstacles in implementing a complete and comprehensive
healthcare clinical information system.
[0010] The present medical image data communications and networking
system overcomes the above described and other problems by
providing an interactive and efficient method for a user to obtain
images for diagnostic, review and processing purposes from a server
having access to a plurality of stored images. The method comprises
the following steps: [0011] connecting a user's computer to a
server over a communications networks, [0012] interfacing the
computer and the user (e.g. installing an industry standard browser
software, such as Netscape, Explorer, etc., on the user's
computer), [0013] using a program in the server to receive from the
server special software (applet) for the user's computer necessary
for data selection and processing, [0014] selecting data for
transmission from the server to the user's computer, [0015]
compressing the data for transmission by selectively using either
lossy or loss-less compression algorithms, [0016] progressively
transmitting the requested data over the network, [0017] receiving
and decompressing the compressed data using the applet and
producing preview images of progressively improving resolution, and
[0018] using the preview images for selecting parts or a subset of
the images for continued transmission to reduce the required
transmission time.
[0019] The above enumerated and other problems are overcome, by
using, for example, the Internet to connect the user and the
server. The user's computer does not need any software for
receiving or processing the requested image except the standard
browser software. Instead, the necessary software is transmitted
from the server along with the image using a network computing
language such as Java or ActiveX or similar language. Accordingly,
if and when there are changes or improvements made in the data, the
software or the image processing procedure, it is no longer
required for a technician to go and install the necessary software
or modify the existing software on each user's computer. Instead
this is accomplished using the Java or ActiveX type language.
[0020] The second problem, that is the time required for
transmitting of images is overcome by progressively sending
"layers" of the data so as to make preview images which are not
necessarily of high resolution quality, but as more and more layers
of data are sent the resolution progressively improves . The user
will be able to decide to get only a certain portion (Region of
Interest--ROI) of the image rather than the whole image, or to
entirely discard the image by stopping the transmission. In
addition, based on the preview images, the user may decide to
request only a subset of an image set. Further time savings result
from using segmentation whereby background in the image is
automatically detected and omitted. Also, a lossy compression can
be used for the preview image, and upgrading to full resolution and
quality can be done on specific request. Therefore, in the process
of image viewing, much less data may be transmitted than in the
present systems and time savings can be considerable. This greatly
alleviates the problem faced when transmitting clinical images from
a server to a user for diagnostic purposes.
[0021] Hence, the system described herein overcomes the above
mentioned difficulties by applying a coding scheme which includes
lossy and loss-less algorithms, with pyramidal structuring of the
images for progressive transmission. A simple yet general man to
machine interface (MMI) enables the user to activate the
compression algorithms interactively, according to on-line
requirements. Today's network computing software (such as Java or
ActiveX) enable a simple procedure by which the user's software is
installed by a networking service program on the server, and there
is no installation requiring user intervention on the user side
(except a one time installation of an industry standard browser
software).
[0022] A sample compression-decompression scheme for the treated
images to be used on a wide range of communication networks is
presented. This scheme can treat both volumes (group of images) and
single images applying loss-less or lossy coding, according to
requirements posed by the user and the available resources. A
progressive approach is applied within the
compression-decompression scheme, which enables the user to get
previews or overviews of the transmitted image long prior to the
time required to transmit the entire image. The quality of the
overviews improves over time thus enabling the user to get basic
impressions of the transmitted data long before the entire image
has arrived. The basic impressions enable the user to interact with
the server so that only the actually necessary data is really sent
to the user. A typical session of requesting a medical image from a
medical image store involves a series of decisions regarding image
and region of interest selection. The progressive, interactive
approach enables the user to make many of these decisions before
the entire image-set is received. This further reduces the time
required to get the needed images and improves the resource
utilization by transmitting, in full description, only that part of
the data which is really of interest.
[0023] The compression-decompression scheme is designed to be
asymmetric, namely, the computational requirements on the
server/coding side are much greater compared to the requirements on
the user/decoding side. This is in accordance with the assumption
that the server's side of the system is implemented on a relatively
powerful computer while the user's hardware requirements are
minimal (the user's work station can also be a simple Personal
Computer).
[0024] The coding scheme described herein utilizes various
properties of the typical medical images, for coding benefit.
Medical images often contain an informative part surrounded by a
background which is of less medical importance. The proposed coding
scheme takes advantage of this particular image structure by using
segmentation of the image prior to transmission, in order to avoid
transmitting the information-less background. The background
information is coded and transmitted only upon a specific user's
request and/or after the informative part of the images has arrived
at the user.
[0025] Medical images are typically acquired using more than 8 bits
per pixel. Many if not most of potential users are only able to
display color or gray level information at 8 bits per pixel. For
that reason a lossy version of the image can be obtained by dynamic
range reduction using simple well known techniques generally
referred to as "windowing", or more complicated adaptive
techniques, such as adaptive "windowing". Those lossy image
versions are much smaller in size than the original image and can
be transmitted relatively fast, enabling the user to display a
lossy version of the image in a short time. This lossy version
serves as a preview or overview image and also as a basis for
further improvements of image quality. As an overview image it
enables the user to decide what part of the image set and image
region is really required. The interactive nature of the
user/server protocol enables the client to dynamically specify
which part of the image set is needed. Only the needed part will be
fully transmitted; thus, reducing the required transmission
periods.
[0026] Because of the progressive nature of the coding schemes, the
images at the user's side are available for on-line processing
during transmission. Various enhancement, display, and analysis
techniques are supported. Interactive graphics on the user's side
enables the viewer to define a region of interest in order to
confine the transmission to that specific area.
[0027] It should be understood that, in a preferred embodiment, the
interactive communication system also interfaces to clinical
information systems available to the server enabling the transfer
of data that includes, for example, medical reports, medical
history, laboratory results and test results.
[0028] There is therefore provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0029] connecting a user's computer to the server over a
communication network;
[0030] receiving from the server image reconstruction software for
the user's computer;
[0031] requesting specific image data for transmission from the
server to the user's computer;
[0032] progressively transmitting the requested specific image data
over the network from the server to user's computer; and
[0033] reconstructing a diagnostic quality image, from the image
data, using the reconstruction software.
[0034] Preferably, the method comprises image processing said
reconstructed image using said reconstruction software.
Alternatively or additionally, the method comprises:
[0035] receiving from the server image selection software for the
user's computer,
[0036] wherein said image selection software is used for said
requesting. Preferably, said image selection software controls the
transmission of the image data.
[0037] Alternatively or additionally, said image selection software
displays images from said server. Alternatively or additionally,
said image selection software and said reconstruction software are
received together. Alternatively or additionally, said image
selection software and said reconstruction software comprise a
single software unit. Alternatively or additionally, said image
selection software is operative to stop the transmission of the
data. Preferably, said image selection software is operative to
restart the transmission of the data, after said stopping.
[0038] In a preferred embodiment of the invention, said image
selection software controls processing of data at said server,
prior to its transmission. Preferably, said processing comprises
reducing said data from a large bit-per-pixel ratio to a small bit
per pixel ratio.
[0039] Alternatively or additionally, said image selection software
controls said server to selectively transmit only portions of the
image data. Alternatively or additionally, the method comprises
interactively providing user input to said image selection
software, to affect said control.
[0040] Alternatively or additionally, the image selection software
comprises application software coded using a device independent
network programming language.
[0041] Alternatively or additionally, the reconstruction software
comprises application software coded using a device independent
network programming language.
[0042] Preferably, said language comprises Java. Alternatively,
said language comprises ActiveX.
[0043] In a preferred embodiment of the invention, reconstructing
comprises:
[0044] reconstructing images of progressively improving quality
from the progressively received data;
[0045] using the produced improved images of progressively produced
quality to decide on processing of the images; and
[0046] processing said images, prior to the progressively received
data being completely received.
[0047] Alternatively or additionally, reconstructing comprises:
[0048] reconstructing images of progressively improving quality
from the progressively received data;
[0049] using the produced improved images of progressively produced
quality to decide on processing of the images; and
[0050] interactively selecting regions of interests in the images
based on said progressively improved images, prior to the
progressively received data being completely received.
[0051] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0052] connecting a user's computer to the server over a
communication network;
[0053] requesting specific image data for transmission from the
server to the user's computer;
[0054] progressively transmitting the requested specific data over
the network to the user's computer;
[0055] reconstructing images of progressively improving quality
from the progressively received data;
[0056] using the produced images of progressively produced quality
to decide on processing of the images; and
[0057] processing said images, prior to the progressively received
data being completely received.
[0058] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0059] connecting a user's computer to the server over a
communication network;
[0060] requesting specific image data for transmission from the
server to the user's computer;
[0061] progressively transmitting the requested specific data over
the network to the user's computer;
[0062] reconstructing images of progressively improving quality
from the progressively received data;
[0063] using the produced improved images of progressively produced
quality to decide on processing of the images; and
[0064] interactively selecting regions of interests in the images
based on said progressively improved images, prior to the
progressively received data being completely received.
[0065] In a preferred embodiment of the invention, said image data
represents a set of images and wherein said using the produced
images to decide comprises using the produced images to decide on
processing images which have not yet been received.
[0066] In a preferred embodiment of the invention, the method
comprises deciding on a termination of transmission of said
progressive image data, responsive to said reconstructed images.
Alternatively or additionally, said processing comprises converting
said image data to image data representing an image with fewer bits
per pixel. Alternatively or additionally, said processing comprises
selecting a region of interest of said images. Alternatively or
additionally, said processing said images is performed on said
image data, at said server, prior to their being transmitted to
said user's computer.
[0067] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain data for diagnostic purposes from a server having
access to stored data, said method comprising:
[0068] connecting a user's computer to the server over a
communication network;
[0069] receiving from the server reconstruction software for the
user's computer;
[0070] requesting specific data for transmission from the server to
the user's computer;
[0071] progressively transmitting the requested specific data over
the network;
[0072] progressively receiving the data using the reconstruction
software to process the received data and produce images of
progressively improving quality; and
[0073] using the produced progressively improved images to decide
on further transmission of the image data.
[0074] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0075] connecting a user's computer to the server over a
communication network;
[0076] requesting specific image data for transmission from the
server to the user's computer;
[0077] reducing the bit-per-pixel ratio of parts of the data being
transmitted, responsive to said request; and
[0078] transmitting the reduced data.
[0079] Preferably, said reduction in bit-per-pixel ratio is
performed responsive to user input at said user's computer.
Preferably, said user input comprises selection of an image
portion.
[0080] In a preferred embodiment of the invention, reducing the
bit-per-pixel ratio comprises:
[0081] calculating an average "M" of the gray values in the image
and a standard deviation "S" of said gray values; and
[0082] rescaling these values in the range [(M-S/2) . . . (M+S/2)]
to obtain a new lower number of bits per pixel.
[0083] Alternatively, reducing the bit-per-pixel ratio
comprises:
[0084] estimating the mean and standard deviation of the gray
levels locally; and
[0085] rescaling these values to obtain a new lower number of bits
per pixel.
[0086] In a preferred embodiment of the invention, progressively
transmitting the requested data over the network comprises:
[0087] recomposing the image into a pyramidal structure comprised
of layers, said layers ranging sequentially from a layer having the
least amount of data to a layer having the most data; and
[0088] transmitting the layers making up the pyramid individually
starting with the layer with the least amount of data to enable the
user to view the progressively improving image to decide on further
transmission of the image.
[0089] Preferably, recomposing the image into a pyramidal structure
comprises reducing the image to provide the different layers at the
transmitting end for progressive transmittal. Preferably, reducing
comprises discarding alternate rows and columns to create an image
that is a quarter of the size of the original image.
[0090] In a preferred embodiment of the invention, the method
comprises:
[0091] providing a first layer with reduced resolution in the
pyramidal structure;
[0092] providing remaining layers that contain residual values with
increased resolution; and
[0093] progressively receiving the data using to provide images
based on the received data of progressively improved
resolution.
[0094] In a preferred embodiment of the invention, the method
comprises:
[0095] compressing the requested data transmitted over the network;
and
[0096] decompressing the received required data to provide
images.
[0097] Preferably, compressing comprises spatially decorrelating
the data by predicting each pixel at the current resolution using
its spatial casual neighbors. Alternatively, compressing comprises
temporally decorrelating each pixel by predicting each pixel value
at the current resolution using the values of temporal neighbors
from previous images. Preferably, a predictor X used in predicting
each pixel for a single image is equal to f(a,b,c), where a, b and
c are previously predicted neighboring pixels.
[0098] Alternatively, a predictor X used in predicting each pixel
for a group of images equals f(a,b,c,a1,b1,c1,x1) where a, b and c
are previously predicted neighboring pixels in a same image and a1,
b1,c1 and x1 are corresponding pixels in a previously predicted
image of the image group.
[0099] In a preferred embodiment of the invention, said compressing
and said decompressing use entropy coding and decoding
respectively. Preferably, said entropy coding and decoding are
accomplished using Golomb Rice entropy coding and decoding.
[0100] In a preferred embodiment of the invention, the method
comprises using adaptive slicing and entropy coding and decoding of
each slice for progressively transmitting the requested specific
image data, wherein said entropy coding generates a residual
matrix.
[0101] Preferably, using adaptive slicing comprises:
[0102] scanning the obtained residual matrix into a residual
vector; and
[0103] partitioning the residual vector into variable length sub
vectors with a relatively homogeneous probability distribution
function.
[0104] Preferably, partitioning comprises:
[0105] estimating the local mean and variance on the
sub-vector;
[0106] sectioning the vector on high transients; and
[0107] coding each sub vector separately.
[0108] Alternatively or additionally, said compression does not
increase the size of said data.
[0109] In a preferred embodiment of the invention, connecting the
user computer to the server over a communication network comprises
connecting over the Internet. Alternatively, connecting the user
computer to the server over a communication network comprises using
a dial up communication system. Alternatively, connecting the user
computer to the server over the communication network comprises
using networking facilities.
[0110] Alternatively or additionally, the stored data comprises
data for a plurality of "postage stamp" images. Preferably, the
method comprises using "postage stamp" images as a catalog for
selecting those images for which no further data is to be
transmitted and those images for which further data is to be
transmitted. Alternatively or additionally, said postage stamps
comprise a lowest level in a pyramidal representation of said
images.
[0111] In a preferred embodiment of the invention, the method
comprises stopping the transmission at the user's request.
Preferably, the method comprises restarting the transmission at the
user's request.
[0112] In a preferred embodiment of the invention, progressively
transmitting comprising serially transmitting a sequence of images.
Alternatively or additionally, progressively transmitting
comprising transmitting data operative to reconstruct images of
increasing resolution.
[0113] In a preferred embodiment of the invention, progressively
transmitting the requested data over the network comprises
segmenting an image into background parts and tissue parts, and
transmitting the tissue parts first.
[0114] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0115] connecting a user's computer to the server over a
communication network;
[0116] segmenting an image into background parts and tissue parts;
and
[0117] transmitting the tissue parts first.
[0118] Preferably, the method comprises requesting said specific
image data for transmission from the server to the user's
computer.
[0119] In a preferred embodiment of the invention, the method
comprises stopping the transmission before transmitting the
background part. Alternatively, the method comprises transmitting
the background part to achieve loss-less transmission of the
image.
[0120] There is also provided in accordance with a preferred
embodiment of the invention, a method of adaptive slice
compression, for compressing progressively transmitted multi-slice
image data, which data is progressively transmitted as pyramid
layers, comprising
[0121] encoding said data using entropy encoding, which encoding
generates a residual matrix;
[0122] scanning the obtained residual matrix into a vector; and
[0123] partitioning the residual vector into variable length sub
vectors with a relatively homogeneous probability distribution
function.
[0124] Preferably, partitioning comprises:
[0125] estimating the local mean and variance on the vector;
[0126] sectioning the vector on high transients; and
[0127] coding each sub vector separately.
[0128] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0129] connecting a user's computer to the server over a
communication network;
[0130] requesting specific image data for transmission from the
server to the user's computer;
[0131] transmitting the requested specific image data over the
network from the server to user's computer;
[0132] stopping said transmission, by command from a user at said
user's computer; and
[0133] continuing said transmission after a time, on command from
said user.
[0134] Preferably, said continued transmission is modified by said
user, responsive to images reconstructed from said stopped
transmission. Alternatively or additionally, stopping said
transmission stops compression of images at said server.
Alternatively or additionally, stopping said transmission comprises
stopping said transmission after a partial-resolution
representation of the image data is transmitted.
[0135] There is also provided in accordance with a preferred
embodiment of the invention, an interactive method for allowing a
user to obtain image data for diagnostic purposes from a server
having access to stored data, comprising:
[0136] connecting a user's computer to the server over a
communication network;
[0137] receiving from the server image reconstruction software for
the user's computer;
[0138] requesting specific image data for transmission from the
server to the user's computer;
[0139] transmitting the requested specific image data over the
network from the server to user's computer; and
[0140] reconstructing a diagnostic quality image, from the image
data, using the reconstruction software.
BRIEF DESCRIPTION OF THE DRAWINGS
[0141] For a more complete understanding of the invention,
reference should be made to the following detailed description
which is given in conjunction with the accompanying drawings, of
which:
[0142] FIG. 1--is a conceptual representation of a present prior
art enterprise wide healthcare information system, FIG. 2--adds
data distribution servers to the enterprise healthcare information
system to facilitate enterprise wide data transfer, FIG.
3--presents a logical representation of the data distribution
server concept of FIG. 2.
[0143] FIG. 4--is a general block diagram of a preferred
compression-decompression scheme;
[0144] FIG. 5--is a showing of the reduce and enlarge operations in
the pyramidal decomposition;
[0145] FIG. 6--is a showing of the pyramidal structure concept;
[0146] FIGS. 7a and 7b--illustrate the background transmission
approach;
[0147] FIG. 8--illustrates the order of transmission;
[0148] FIG. 9a--illustrates a predictor for a single image;
[0149] FIG. 9b--illustrates a predictor for a group of images;
and
[0150] FIG. 10--illustrates an example of vector partitioning.
GENERAL DESCRIPTION OF THE PREFERRED EMBODIMENT
1. System Overview
[0151] The system consists of a server that has access to data
banks and distributes the data on demand. Several users can
connect, simultaneously, to the server, over communication lines.
In this system the server is also responsible for image
pre-processing and for distributing user software. The user's
function is to manage the medical image acquisition and processing
through the use of an intuitive Man-Machine Interface, a special
protocol and the available hardware and communication
resources.
[0152] A typical medical image acquisition session will start by a
simple data request, made by the user, to the system's server
communication site. This generic request can be accomplished using
any of standard communication protocols and, for example, through
an HTTP (Hyper Text Transmission Protocol) connection to the server
(which can be designed for access purposes as a Web (WWW) site.
There are no requirements on the user's hardware and browser
software other than the basic capability to communicate over the
chosen communication line and for the browser to support a network
computing language such as Java or ActiveX. Using the Web, these
requirements will include a link to the Internet and a standard Web
browser as described above. Upon such a request, the server will
download, to the user's machine, a network application applet. This
network application will serve as the user's application in all
future interactions with the server. The network application is a
generic, platform independent application written in a suitable
network application language such as, but not limited to, Java or
ActiveX. The network language may also be any other software that
utilizes the communication capabilities of the user's machine.
After a short authorization and authentication procedure, the user
will be presented with an opportunity to request medical data. The
communication can also be accomplished using "dial-up" or other
"networking" schemes.
[0153] Medical data includes Medical Image Data, throughout this
description. It may comprise a number of medical images, of various
modalities, which are available for transmission through the
server. The user may define the specific medical case of interest
through the use of network application queries into the server's
database. Selecting the case is done using case identifiers which
are usually, but not limited to, textual, image icons, etc. A
typical CT (Computerized Tomography) case may contain 50-100
medical images. The actual transmission of the medical image
information is accomplished through the use of a
compression/decompression algorithm and a powerful client/server
protocol. The transmission is relatively fast owing to a smart
utilization of the available hardware and network resources and
focusing on the needed medical information by providing the user
with interim information, thus letting the user refine the
information query parameters during the acquisition process itself.
The compression-decompression algorithm is basic to the explanation
of the user/server acquisition protocol. Therefore, this general
description will start with an explanation of the
compression-decompression algorithm followed by a discussion of the
acquisition protocol and conclude with a more detailed review of
the Man-Machine Interface.
2. The Compression-Decompression Algorithm
[0154] The goal of the compression-decompression algorithm is to
achieve maximal compression ratios but at the same time supply the
user with visually adequate interim images. The algorithm should
also support loss-less as well as lossy interim and final results,
be suited to the medical image processing common to these images
and as much as possible be asymmetric and easy to implement using
the network computing language.
[0155] FIG. 4 presents an overview of the compression-decompression
algorithm for use with the system described. Compression starts by
(optional) segmentation (block A1 in the FIG. 4), where the
background of the image (if it exists) is separated from the actual
image.
[0156] FIGS. 7a and 7b show graphical presentations of such
possible background segmentations. The regions denoted A, B, C, and
D in FIG. 7a are background regions. The proposed segmentation
bounds the region of actual tissue by a rectangle. Only the inner
part is progressively transmitted. Other methods of segmentation
are possible as shown in FIG. 7b where the actual tissue is shown
peripherally bounded by the dashed lines.
[0157] The second step (optional) in image coding for compression
is a windowing operation (block B1 in FIG. 4), where the dynamic
range of the input image is reduced to a lower number of bits per
pixel. The new number of bits can represent the client's display
capabilities or be derived from the communication bandwidth
restrictions. The windowing operation could be done, for example,
by estimating the average M and the standard deviation S of the
image values, and resealing these values in the range [(M-S/2) . .
. (M+S/2)] using the required new number of bits. As an
alternative, an improved locally adaptive windowing method can be
applied, which estimates the mean and standard deviation locally.
Other well known windowing procedures can be used.
[0158] Since one of the goals to be accomplished is to supply the
user with meaningful interim results, the medical images are sent
progressively. This requirement in turn implies that a pyramidal
re-structuring of the image is required (block C1 in FIG. 4). The
concept of pyramidal decomposition of an image is shown in FIG. 6,
where the two basic operations--Reduce and Enlarge--are further
described in FIG. 5. The reduce operation revises or decomposes the
image by, for example, simply discarding all even rows and columns,
creating an array that is a quarter of the original size. The
enlarge operation, for example, bilinearly interpolates the image,
resulting in an array four times larger. The interpolation process
is not limited to a bilinear interpolation. The exact type of
interpolation is selected based on the user's computational and
display capabilities. The pyramidal structure contains at the
zero-level one small image with reduced resolution. All the
remaining levels contain residual values with increased resolution.
The pyramidal decomposition of the image could also be achieved
through the use of other pyramidal decomposition algorithms. The
pyramidal data structures consist of several versions of the
original image. Each version is of different size and nature. The
pyramidal information is ordered such that the top of the pyramid
is the version of the original image which least resembles the
original image. If the pyramid is loss-less the final level of the
pyramid is an exact replication of the original image. It is clear
that after decompressing a specific level we can reconstruct the
image up to that level and get an interim result. This interim
result resembles the original image according to the level of the
pyramid.
[0159] In order to facilitate an efficient coding scheme, further
decorrelation of the data is required. This is achieved by spatial
and temporal decorrelation operation (block D1 in FIG. 4). At this
stage, each pixel in the current resolution level is predicted by
its spatial casual (already transmitted) neighbors. If groups of
images are being coded together, temporal neighbors from previous
images are used to compute a second predictor, and the best
predicator is chosen for each block of pixels. At the end of the
prediction stage, the residuals are rescanned into a vector. If the
user selected only part of the image to be transmitted (ROI--Region
of Interest) only that part of the residual image is scanned and
the ROI parameters are added to the header of the image
[0160] Following is an example of a predictor for a single image
(FIG. 9a): x = f .function. ( a , b , c ) = { max .function. ( a ,
b ) c < min .function. ( a , b ) min .function. ( a , b ) c >
max .function. ( a , b ) a + b + c otherwise } ##EQU1##
[0161] Using similar reasoning a predictor for a group of images
can be effective in case there is correlation between successive
images (FIG. 9b):
[0162] x=f(a,b,c,a1,b1,c1,x1)
[0163] The residual vector is partitioned into variable length
sub-vectors with a relatively homogeneous probability distribution
function (block E1 in FIG. 4). The adaptive partitioning is
accomplished by estimating the local mean and variance on the
vector, and sectioning the vector on high transients. Each
sub-vector is then coded using an entropy coder. One example of
such coding is a Golomb-Rice code (block F1 in FIG. 4). An example
of a possible partitioning is shown in FIG. 10.
[0164] The decompression algorithm is basically the compression
operations in inverse order. First, a header is obtained, stating
whether segmentation and/or windowing operations were applied, the
size of the images and their number, the pyramid depth, etc. (block
A2 in FIG. 4). A zeros pyramid is then constructed in order to be
filled during the decoding process (block B2 in FIG. 4). Each
sub-sector is decoded using inverse entropy coding, i.e.,
Golomb-Rice code (block C2 in FIG. 4), and all these sub-blocks are
rearranged into matrix form. The spatial/temporal prediction is
then computed and added to the residuals (block D2 in FIG. 4), and
the obtained values are loaded into the pyramid (block E2 in FIG.
4). The pyramid can be restructured to an image at any time during
this operation, yielding the obtained image so far.
[0165] If segmentation is applied, the background will be
transmitted at the end of the transmission of the inner image part.
This is for loss-less transmission. For lossy transmission the user
can stop the transmission, thus disregarding the background.
Transmitting the background is supported by dividing the background
into four parts as indicated in FIG. 7a or by mapping the image as
indicated in FIG. 7b. Each such part is raster scanned into a
vector and the same coding operations presented above apply again,
namely, decorrelation, adaptive sectioning, and entropy coding
(i.e., Golomb-Rice coding). Other compression-decompression methods
of course can be applied within the scope of this invention.
[0166] If windowing is applied, the received image at the user's
location is a lossy representation of the original image. Upon the
user's request, the error image (the difference between the
original and the windowed image) should be coded and transmitted.
This error image is coded using the same methodology as presented
for the background transmission--decorrelation, adaptive
sectioning, and entropy coding (i.e., Golomb-Rice coding).
[0167] For image group/series, the order of transmission is as
shown in FIG. 8. First, all the low-resolution levels are sent. At
the end of this stage, the client may view all the required images
in an overview form using the basic version of the entire image
set. At the second stage, each of the images is updated by sending
the next resolution level. As soon as a resolution level for a
specific image is received, the image can be updated to the next
interim version which is better than the current version. After
several such steps, the images are obtained in error-less form on
the user's display
[0168] Another option, instead of the loss-less entropy coding
described above, may be implemented by applying the already
existing JPEG routines within the browser software. This approach
consists of optional (as before) windowing and/or segmentation
steps, followed by a pyramidal decomposition of the obtained image.
Each resolution level is then compressed using the lossy JPEG
algorithm. At the user, each such level is decompressed
accordingly. Since lossy compression-decompression results in
deeper compression ratios, and since the decompression routines are
written in the computer native language, much shorter waiting
periods are obtained at the user's end. As described in detail for
the windowing and segmentation operations, one final step of
residuals transmission is required in order to support final
loss-less representation of the original images at the user's
workstation.
[0169] In addition to the above described techniques, other methods
can be applied within the scope of the present invention. All the
above are examples of the invention, which is not limited to those
methods.
3. Image Acquisition Process--"Stamps"
[0170] Using the insight gained in the explanations rendered until
now, the image acquisition process itself can now be described in
greater detail. As presented herein above, the user defines the
specific medical case (patient, study, series, images) of interest
through the use of network application queries into the server's
database. Selecting the case is done using the case identifiers
which are usually, but not limited to, text or image icons. A
typical CT case may contain 50-100 medical images. Out of all these
images the goal is to supply the user with the images really needed
for the purpose of drawing conclusions (diagnostic, second opinion,
etc.) as fast as possible. Usually, out of the entire image case
the user will require only a limited number of images and only a
specific region of interest (ROI) in the limited number of images.
Typically these requirements are case dependent and the user cannot
decide which images and what part of these are really needed until
the images are viewed. The protocol thus should let the user
specify these requirements as soon as possible by supplying the
user with interim information which will arrive fast and be
sufficiently adequate to make these decisions.
[0171] Upon selection of the medical image case the server starts
to prepare (as an option) a very basic version of the entire
medical case. This basic version of the images will be referred to
as "stamps" or icons and will consist of a reduced version, which
is visually similar, for every image in the case. The size, in
bits, of these "stamps" is small compared with the size of the
original images. The entire "stamp" collection is thus a reduced
representation of the entire image case. Its size is selected to
enable the user to visually select which images are of interest
while retaining the small total size. This will assure that the
entire reduced representation of the image case will arrive at the
user in a relatively short time. Having presented the entire
"stamp" collection to the user, the server awaits the user
selection of a sub-set of the entire image case. The sub-set can
include the entire case but will typically include only several
images. This sub-set of the image case will be referred to
hereinafter as the "image group". In place of the icons, text can
be used to describe the images.
[0172] After the user selects the image group, the server prepares
a pyramidal decomposition for each and every image in the group or
volume process for the whole series. If segmentation and/or
windowing were selected, the server performs these operations at
this stage. It then goes on and performs the rest of the
compression chain (D1 through F1 in FIG.4) for the top level of the
pyramid. This level is the most reduced version of the image and
thus is also the smallest. As an option, the server can utilize the
"icons" which have been prepared in the previous stage for this
purpose. Optionally, as a first stage, only the smallest
representation of the image is sent from the server to the user.
The user receives and displays the images. After, or even during,
this recuperation stage the user can select either a smaller sub
set of the group images and/or smaller region of interest out of
the image space. This serves as a finger query into the entire
image data base and is sent from the user to the server over the
communication line. If no finer selection is required the user is
enabled to specify whether the visual level obtained so far is
sufficient, thus ending the image acquisition process. However, if
a better visual level is needed the acquisition process is combined
to obtain the next level in the pyramid. Alternatively if the user
does nothing the next level is sent. As soon as the server gets the
request it performs blocks D1 through F2 on the next level in the
pyramid. This is done only to those images which are required and
within these images only to that part of the image which is of
interest (the ROI). Within the image group the order of compression
and communication is presented in FIG. 8. The protocol preferably
works on a resolution first basis. All the images in the image
group may be brought to the same resolution level and only then the
server advances to the next resolution level. Other orders of
operation can be used without losing the generality of the
invention.
[0173] The above process is iterated for all resolution levels. The
process is stopped either when the user indicates that the visual
level is adequate or the entire image has been sent resulting in a
perfect, loss-less, replication of the original image on the user's
screen.
[0174] The type of temporal prediction, (block D1 in FIG. 4) is
selected by the server according to the user's computational
capabilities.
[0175] If segmentation and/or windowing and/or lossy compression
was performed on the images, the user can request the server to
complete the images to their loss-less representation. In such a
case, the server will compress and transmit the needed information
for the user to complete the images to their loss-less version.
[0176] At each and every stage the user can choose to broaden the
information query requirements, for example, by enlarging the
number of images rather than reducing it. In that case, the server
will "backtrack" and send the required information to the user.
[0177] When the needed information has arrived and been presented
to the user, the user is presented with the option to acquire
another medical image case from the server.
4. The Man-Machine-Interface
[0178] The man-machine interface (MMI) of the user serves as the
means by which the user interacts with the system and as a display
surface for the medical images. Being a medical images
communication network based system, the MMI combines the known and
familiar user interface environment of communication software with
the tools needed for medical image processing. The goal is to give
the user the tools to be part of the described image acquisition
process as well as to enable the user to perform tasks regarding
the medical image information. The MMI should achieve these goals
with minimal to zero intervention or requirements of the user. For
that end the entire user software is completely downloaded from the
server to the user's machine and for the most part uses part of the
communication software already part of the user's machine. All this
is done without any user intervention. This also makes user
software updates and improvements irrelevant to the end-user. The
user software relies heavily on the communication software (e.g.
browser) already installed on the user's machine. This enables the
user to operate on different machines with different computational
and display capabilities. The first task of the user, upon loading
the user software into the user's machine is to automatically
profile the machine and the network capabilities. This information
is then relayed to the server and is used to select various
parameters for the rest of the session. This is done without any
user intervention.
[0179] The MMI, the user is presented with, contains controls which
are part of the image acquisition process as well as typical
medical image processing tools. The image acquisition tools include
case specification tools, image selection tools, resolution level
advancement tools, tools for windowing, zooming, panning, graphics
and annotations, CINE, and so on.
[0180] At all times, the user has full information as to what part
of the entire medical image case is currently being viewed on the
user's display screen. This information includes, but is not
limited to, image number, resolution level, loss-less indicator,
region of interest indication, segmentation and window parameters,
and so on. By these, the system makes sure the user is fully aware
of what exactly is being presented at all times.
[0181] While the invention has been particularly shown and
described with reference to preferred embodiments thereof, it is to
be understood by those skilled in the art that various changes may
be made in form and details without departing from the spirit and
scope of the invention as defined in the appended claims.
* * * * *