U.S. patent application number 10/165446 was filed with the patent office on 2003-03-27 for system and method for the lossless progressive streaming of images over a communication network.
This patent application is currently assigned to RTIMAGE, LTD.. Invention is credited to Dekel, Shai, Goldberg, Nitzan.
Application Number | 20030059096 10/165446 |
Document ID | / |
Family ID | 22731646 |
Filed Date | 2003-03-27 |
United States Patent
Application |
20030059096 |
Kind Code |
A1 |
Dekel, Shai ; et
al. |
March 27, 2003 |
System and method for the lossless progressive streaming of images
over a communication network
Abstract
Disclosed is a lossless image streaming system for the
transmission of images over a communication network. The system
eliminates the necessity to store a compressed version of the
original image, by losslessly streaming ROI data using the original
stored image. The imaging system of the present invention also
avoids the computationally intensive task of compression of the
full image. When a user wishes to interact with a remote image, the
imaging server performs a fast preprocessing step in near real time
after which it can respond to any ROI requests also in near real
time. When a ROI request arrives at the server, a sophisticated
progressive image encoding algorithm is performed, but not for the
full image. Instead, the encoding algorithm is performed only for
the ROI. Since the size of the ROI is bounded by the size and
resolution of the viewing device at the client and not by the size
of the image, only a small portion of the full progressive coding
computation is performed for a local area of the original
image.
Inventors: |
Dekel, Shai; (Or-Yehuda,
IL) ; Goldberg, Nitzan; (Beit-Shemesh, IL) |
Correspondence
Address: |
NEEDLE & ROSENBERG P C
127 PEACHTREE STREET N E
ATLANTA
GA
30303-1811
US
|
Assignee: |
RTIMAGE, LTD.
|
Family ID: |
22731646 |
Appl. No.: |
10/165446 |
Filed: |
June 7, 2002 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
10165446 |
Jun 7, 2002 |
|
|
|
09837862 |
Apr 17, 2001 |
|
|
|
60198017 |
Apr 18, 2000 |
|
|
|
Current U.S.
Class: |
382/131 ;
375/E7.013; 375/E7.04; 375/E7.044; 375/E7.049; 375/E7.056;
375/E7.06; 375/E7.062; 375/E7.153; 375/E7.166; 375/E7.182;
375/E7.239; 382/282 |
Current CPC
Class: |
H04N 21/4728 20130101;
H04N 19/186 20141101; H04N 1/3873 20130101; H04N 19/147 20141101;
H04N 19/36 20141101; H04N 21/4621 20130101; H04N 21/234345
20130101; H04N 19/146 20141101; H04N 19/115 20141101; H04N 19/17
20141101; H04N 19/1883 20141101; H04N 19/63 20141101; H04N 21/25808
20130101; H04N 21/2662 20130101; H04N 19/129 20141101; H04N 19/635
20141101 |
Class at
Publication: |
382/131 ;
382/282 |
International
Class: |
G06K 009/00; G06K
009/20 |
Claims
What is claimed:
1. A method for transmitting visual data over a communication
network, comprising: (a) performing a wavelet transform upon the
visual data to produce wavelet coefficients, each wavelet
coefficient having an associated spatial location, resolution and
quality; (b) grouping wavelet coefficients into data blocks, each
data block described by a two-dimensional spatial coordinate pair,
a resolution coordinate and an accuracy coordinate; (c) selecting a
region of interest of the visual data and a quality at which the
visual data is to be rendered; and (d) transmitting over the
communication network only those data blocks corresponding to the
selected region of interest and quality.
2. The method claimed in claim 1, wherein the accuracy coordinates
of the data blocks define the quality of visual data the
coefficients represent, coefficients having a greater number of
bits representing the visual data at a higher quality than
coefficients having a fewer number of bits.
3. The method claimed in claim 1, wherein the visual data comprises
a digital image.
4. The method claimed in claim 1, wherein the visual data comprises
digital video.
5. The method claimed in claim 1, wherein the visual data comprises
a sequence of images.
6. The method claimed in claim 5, wherein the visual data comprises
computed tomography (CT) medical images.
7. The method claimed in claim 1, wherein the step of selecting a
quality at which the visual data is to be rendered includes
selecting lossless quality.
8. The method claimed in claim 1, wherein the coefficients of the
accuracy blocks described by resolution coordinates defining a
highest resolution are the result of local Discrete Cosine
Transforms regarded as wavelet-like coefficients.
9. The method claimed in claim 1, wherein: the step of selecting a
quality at which the visual data is to be rendered comprises a
client computer coupled to the communication network generating at
least one request for interaction with the visual data and at least
one subsequent request for data blocks identifying a region of
interest at a selected quality and a selected resolution within the
visual data; and the transmitting step comprises a server computer
coupled to the communication network receiving the requests and
progressively transmitting data blocks corresponding to the region
of interest to the client computer via the communication
network.
10. The method claimed in claim 9, wherein the step of a client
computer coupled to the communication network generating at least
one request for interaction with the visual data and at least one
subsequent request for data blocks identifying a region of interest
comprises the client computer selecting a progressive mode from the
group consisting of: resolution, quality and spatial.
11. The method claimed in claim 1, wherein: the step of selecting a
quality at which the visual data is to be rendered comprises a
client computer coupled to the communication network generating at
least one request for interaction with the visual data and at least
one subsequent request identifying a region of interest at a
selected quality and a selected resolution within the visual data;
and the transmitting step comprises a server computer coupled to
the communication network receiving the request identifying the
region of interest and selecting and progressively transmitting
data blocks corresponding to the region of interest to the client
computer via the communication network
12. The method claimed in claim 11, wherein the step of a server
computer coupled to the communication network receiving the request
identifying the region of interest and selecting and progressively
transmitting data blocks corresponding to the region of interest to
the client computer via the communication network is performed in
response to the client computer selecting a progressive mode from
the group consisting of: resolution, quality and spatial.
13. The method claimed in claim 1, further comprising the client
computer progressively performing the steps of: decompressing
compressed data blocks corresponding to the region of interest
received from the server computer; and rendering the region of
interest.
14. The method claimed in claim 13, wherein the rendering step
comprises the client computer rendering the region of interest on a
video display.
15. The method claimed in claim 13, wherein the rendering step
comprises the client computer rendering the decompressed region of
interest on a printer.
16. The method claimed in claim 13, further comprising the step of
the client computer storing the data blocks in a cache associated
with the client computer.
17. The method claimed in claim 16, wherein: the step of selecting
a quality at which the visual data is to be rendered comprises a
client computer coupled to the communication network generating at
least one request for interaction with the visual data and at least
one subsequent request for data blocks identifying a region of
interest; and the transmitting step comprises a server computer
coupled to the communication network receiving the requests and
progressively transmitting data blocks corresponding to the region
of interest to the client computer via the communication network,
and wherein the client computer only requests the data blocks
associated with a portion of a region of interest that was not
previously stored in the cache.
18. A system for transmitting visual data over a communication
network, comprising: (a) an image storage device for digitally
storing visual data; (b) a client computer coupled to the
communication network; and (c) a server computer, coupled to the
communication network and the image storage device, wherein the
server computer performs the steps of: (i) performing a wavelet
transform upon the visual data to produce wavelet coefficients,
each wavelet coefficient having an associated spatial location,
resolution and quality; (ii) grouping wavelet coefficients into
data blocks, each data block described by a two-dimensional spatial
coordinate pair, a resolution coordinate and an accuracy
coordinate; (iii) selecting a region of interest of the visual data
and a quality at which the visual data is to be rendered; and (iv)
transmitting over the communication network only those data blocks
corresponding to the selected region of interest and quality.
19. The system claimed in claim 18, wherein the accuracy
coordinates of the data blocks define the quality of visual data
the coefficients represent, coefficients having a greater number of
bits representing the visual data at a higher quality than
coefficients having a fewer number of bits.
20. The system claimed in claim 18, wherein the step of selecting a
quality at which the visual data is to be rendered includes
selecting lossless quality.
21. The system claimed in claim 18, wherein the coefficients of the
accuracy blocks described by resolution coordinates defining a
highest resolution are the result of local Discrete Cosine
Transforms regarded as wavelet-like coefficients.
22. The system claimed in claim 18, wherein: the client computer
generates at least one request for interaction with the visual data
and at least one subsequent request for data blocks identifying a
region of interest at a selected quality and a selected resolution
within the visual data; and the server computer receives the
requests and progressively transmits data blocks corresponding to
the region of interest to the client computer via the communication
network.
23. The system claimed in claim 22, wherein the client computer
selects a progressive mode from the group consisting of:
resolution, quality and spatial.
24. The system claimed in claim 18, wherein: the client computer
generates at least one request for interaction with the visual data
and at least one subsequent request identifying a region of
interest at a selected quality and a selected resolution within the
visual data; and the server computer receives the request
identifying the region of interest and selects and progressively
transmits data blocks corresponding to the region of interest to
the client computer via the communication network.
25. The system claimed in claim 24, wherein the client computer
selects a progressive mode from the group consisting of:
resolution, quality and spatial.
26. The system claimed in claim 18, further comprising the client
computer progressively performing the steps of: decompressing
compressed data blocks corresponding to the region of interest
received from the server computer; and rendering the region of
interest.
27. The system claimed in claim 26, further comprising the step of
the client computer storing the data blocks in a cache associated
with the client computer.
28. The system claimed in claim 27, wherein: the client computer
generates at least one request for interaction with the visual data
and at least one subsequent request for data blocks identifying a
region of interest; and the server computer receives the requests
and progressively transmits data blocks corresponding to the region
of interest to the client computer via the communication network,
and wherein the client computer only requests the data blocks
associated with a portion of a region of interest that was not
previously stored in the cache.
29. A server computer for transmitting visual data over a
communication network to a client computer, comprising: (a) an
image storage device for digitally storing visual data; and (b) a
processing system, coupled to the communication network and the
image storage device, wherein the processing system effects the
steps of: (i) performing a wavelet transform upon the visual data
to produce wavelet coefficients, each wavelet coefficient having an
associated spatial location, resolution and quality; (ii) grouping
wavelet coefficients into data blocks, each data block described by
a two-dimensional spatial coordinate pair, a resolution coordinate
and an accuracy coordinate; (iii) selecting a region of interest of
the visual data and a quality at which the visual data is to be
rendered; and (iv) transmitting over the communication network only
those data blocks corresponding to the selected region of interest
and quality.
30. The server computer claimed in claim 29, wherein the accuracy
coordinates of the data blocks define the quality of visual data
the coefficients represent, coefficients having a greater number of
bits representing the visual data at a higher quality than
coefficients having a fewer number of bits.
31. The server computer claimed in claim 29, wherein the step of
selecting a quality at which the visual data is to be rendered
includes selecting lossless quality.
32. The server computer claimed in claim 29, wherein the
coefficients of the accuracy blocks described by resolution
coordinates defining a highest resolution are the result of local
Discrete Cosine Transforms regarded as wavelet-like
coefficients.
33. The server computer claimed in claim 29, wherein the server
computer receives requests from the client computer for interaction
with the visual data and at least one subsequent request for data
blocks identifying a region of interest at a selected quality and a
selected resolution within the visual data, and the server computer
progressively transmits data blocks corresponding to the region of
interest to the client computer via the communication network.
34. The server computer claimed in claim 29, wherein the server
computer receives requests from the client computer for interaction
with the visual data and at least one subsequent request
identifying a region of interest at a selected quality and a
selected resolution within the visual data, and the server computer
selects and progressively transmits data blocks corresponding to
the region of interest to the client computer via the communication
network.
Description
[0001] This is a continuation of U.S. patent application Ser. No.
09/837,862, filed Apr. 17, 2001, entitled "SYSTEM AND METHOD FOR
THE LOSSLESS PROGRESSIVE STREAMING OF IMAGES OVER A COMMUNICATION
NETWORK," and further claims priority to U.S. patent application
Ser. No. 09/837,862, filed Apr. 7, 2001, entitled "SYSTEM AND
METHOD FOR THE LOSSLESS PROGRESSIVE STREAMING OF IMAGES OVER A
COMMUNICATION NETWORK," U.S. patent application Ser. No.
09/386,264, filed Aug. 31, 1999, entitled "SYSTEM AND METHOD FOR
TRANSMITTING A DIGITAL IMAGE OVER A COMMUNICATION NETWORK," and
U.S. Provisional Patent Application No. 60/198,017, filed Apr. 18,
2000, entitled "LOSSLESS PROGRESSIVE STREAMING OF IMAGES OVER THE
INTERNET," and the specifications thereof are incorporated herein
in their entireties by this reference.
BACKGROUND OF THE INVENTION
[0002] 1. Field of the Invention
[0003] This invention relates to systems and methods for
transmission of still images over relatively low-speed
communication channels. More specifically the invention relates to
progressive image streaming over low speed communication lines, and
may be applied to a variety of fields and disciplines, including
commercial printing and medical imaging, among others.
[0004] 2. Brief Description of the Prior Art
[0005] In a narrow bandwidth environment, a simple transfer to the
client computer of any original image stored in the server's
storage is obviously time consuming. In many cases the user only
wishes to view a low resolution of the image and perhaps a few more
high-resolution details, in these instances it would be inefficient
to transfer the full image. This problem can be overcome by storing
images in some compressed formats. Examples for such formats are
standards such as Progressive JPEG (W. Pennebaker and J. Mitchel,
"JPEG, still image data compression standard", VNR, 1993) or the
upcoming JPEG2000 (D. Taubman, "High performance scalable image
compression with EBCOT", preprint, 1999). These formats allow
progressive transmission of an image such that the quality of the
image displayed at the client computer improves during the
transmission.
[0006] In some application such as medical imaging, it is also
necessary that whenever the user at the client computer is viewing
a portion of the highest resolution of the image, the progressive
streaming will terminate at lossless quality. This means that at
the end of progressive transmission the pixels rendered on the
screen are exactly the pixels of the original image. The current
known "state-of-the-art" wavelet algorithms for progressive
lossless streaming all have a major drawback: their rate-distortion
behavior is very inferior to the "lossy" algorithms. The
implications are serious:
[0007] 1. Whenever the user is viewing any low resolution of the
image (at low resolutions the term "lossless" is not well defined)
more data needs to be sent for the same visual quality.
[0008] 2. During the progressive transmission of the highest
resolution, before lossless quality is achieved, more data needs to
be sent for the same visual quality.
[0009] Researchers working in this field are troubled by these
phenomena. As F. Sheng, A. Bilgin, J. Sementilli and M. W.
Marcellin say in [SBSM]: ". . . . Improved lossy performance when
using integer transforms is a pursuit of our on-going work." Here
is an example:
1TABLE 1 Comparison of the lossy compression performances
(implemented by the (7, 9) Wavelet) to a lossless compression
(implemented by a reversible (4, 4) Wavelet) of "Barabara" image
(PSNR (dB)) ([SBSM]). Rate (bit per pixel) Wavelet 0.1 0.2 0.5 0.7
1.0 Floating Point 7 .times. 9 24.18 26.65 31.64 34.17 36.90
Reversible (4, 4) 23.89 26.41 31.14 33.35 35.65
[0010] As one can see from Table 1, state of the art progressive
lossless coding is inferior to lossy coding by more than 1 dB at
the high bit-rate.
[0011] Indeed, intuitively, the requirement for lossless
progressive image transmission should not effect the rendering of
lower resolutions or the progressive "lossy" rendering of the
highest resolution before lossless quality is obtained. The final
lossless quality should be a layer that in some sense is added to a
lossy algorithm with minor (if any) effect on its performance.
[0012] The main problem with known lossless wavelet algorithms,
such as SPIHT [SP1] and CREW [ZASB], is that they use special
"integer To Integer" transforms (see "Wavelet transforms that map
integers to integers", A. Calderbank, I. Daubechies, W. Sweldens,
B. L. Yeo, J. Fourlier Anal. Appl., 1998). These transforms mimic
"mathematically proven" transforms that work well in lossy
compression using floating-point arithmetic implementations. But
because they are constraint to be lossless, they do not approximate
their floating-point ancestors sufficiently well. Although in all
previous work there have been attempts to correct this
approximation in the progressive coding stage of the algorithm, the
bad starting point, an inefficient transform, prevented previous
authors to obtain decent rate-distortion behavior.
[0013] Our algorithm solves the rate-distortion behavior problem.
Using the fact that images are two-dimensional signals, we
introduce new 2D lossless Wavelet transforms that approximate much
better their lossy counterparts. As an immediate consequence our
lossless progressive coding algorithm has the same rate-distortion
of a lossy algorithm during the lossy part of the progressive
transmission.
SUMMARY OF THE INVENTION
[0014] The imaging system that is described below is directed to a
lossless image streaming system that is different from traditional
compression systems and overcomes the above problems. By utilizing
a lossless means of progressive transmission means the pixels
rendered on the screen at the end of transmission are exactly the
pixels of the original image that was transmitted. The imaging
system disclosed herein eliminates the necessity to store a
compressed version of the original image, by streaming ROI data
using the original stored image. The imaging system of the present
invention also avoids the computationally intensive task of
compression of the full image. Instead, once a user wishes to
interact with a remote image, the imaging server performs a fast
preprocessing step in near real time after which it can respond to
any ROI requests also in near real time. When a ROI request arrives
at the server, a sophisticated progressive image-encoding algorithm
is performed, but not for the full image. Instead, the encoding
algorithm is performed only for the ROI. Since the size of the ROI
is bounded by the size and resolution of the viewing device at the
client and not by the size of the image, only a small portion of
the full progressive coding computation is performed for a local
area of the original image. This local property is also true for
the client. The client computer performs decoding and rendering
only for ROT and not the full image. This real time streaming or
Pixels-On-Demand.TM. architecture requires different approaches
even to old ideas. For example, similarly to some prior art, the
present imaging system is based on wavelets. But while in other
systems wavelet bases are selected according to their coding
abilities, the choice of wavelet bases in the present imaging
system depends more on their ability to perform well in the real
time framework. The system of the present invention supports
several modes of progressive transmission: by resolution, by
accuracy and by spatial order.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] FIG. 1 is a system architecture block diagram.
[0016] FIG. 2 is an imaging system workflow diagram.
[0017] FIG. 3 is a flow diagram representing a "lossless
progressive by accuracy" request list for a ROI.
[0018] FIG. 4 is a diagram depicting the client "progressive by
accuracy" workflow.
[0019] FIG. 5 is a diagram depicting the server workflow.
[0020] FIG. 6 is a diagram describing the server-preprocessing
step.
[0021] FIG. 7 is a diagram describing the low resolution encoding
process.
[0022] FIG. 8 is a diagram describing the ROI high resolution
processing.
[0023] FIG. 9 is a diagram depicting the local forward lossless
wavelet transform.
[0024] FIG. 10 is a diagram depicting the local inverse lossless
wavelet transform.
[0025] FIG. 11 is a diagram depicting the progressive rendering
steps.
[0026] FIG. 12 is a diagram depicting a lossless subband tile
wherein the spatial grouping of subband coefficients are at a given
resolution and the halfbit matrix is associated with the hh,
subband.
[0027] FIG. 13 a diagram depicting the RGB<->YUV reversible
conversion.
[0028] FIG. 14 a diagram depicting a lossless last bit plane data
block in which only hl and hh subband coefficients are scanned.
[0029] FIG. 15 is a sample pseudo-code of the encoding algorithm
represented by: (a) Least significant bit plane scan pseudo-code
(b) Half bit plane scan pseudo-code.
[0030] FIG. 16 is a sample pseudo-code of the decoding algorithm
represented by: (a) Least significant bit plane scan pseudo-code
(b) Half bit plane scan pseudo-code.
[0031] FIG. 17 is a diagram depicting the curve defining the
mapping from Original_Image_Depth-bit image to
Screen_Depth-bit.
[0032] FIG. 18 is a diagram depicting the decomposition of
one-dimensional signal x to the Low-subband s and the High-subband
d and the separable decomposition of two-dimensional signal X into
4 matrices (subbands): LL, HL, LH and HH.
[0033] FIG. 19 is a diagram depicting the first stage of the 2D
separable transform step in the X-direction.
[0034] FIG. 20 is a diagram depicting the second stage of the 2D
separable transform step in the Y-direction.
[0035] FIG. 21 is a diagram depicting the application of the full
2D Wavelet transform.
[0036] FIG. 22 is a flow diagram representing the least significant
bit plane scan of the encoding algorithm.
[0037] FIG. 23 is a flow diagram representing the least significant
bit plane scan of the decoding algorithm.
[0038] FIG. 24 is a flow diagram describing a bit plane
significance scan of the server-encoding algorithm.
[0039] FIG. 25 is a flow diagram describing the subband tile
extraction of the client progressive rendering process.
[0040] FIG. 26 is a diagram depicting the preprocessing
multi-resolution structure.
DETAILED DESCRIPTION OF THE INVENTION
[0041] 1. Notation and Terminology
[0042] The following notation is used throughout this document.
2 Term Definition 4D Four dimensional FIR Finite Impulse Response
FWT Forward Wavelet Transform GUI Graphical User Interface ID
Identification tag IWT Inverse Wavelet Transform ROI Region Of
Interest URL Uniform Resource Locator LSB Least Significant Bit RMS
Root Square Error FP Floating Point PDF Probability Distribution
Function
[0043] The following terminology and definitions apply throughout
this document.
3 Term Definition Rendering Procedure to output/display a ROI of an
image into a device such as a monitor, printer, etc.
Multiresolution A pyramidal structure representing an image at
dyadic resolutions, beginning with the original image as the
highest resolution. Subband Transform/ A method of decomposing an
image (time domain) to frequency subband coefficients components
(frequency domain). A representation of an image as a sum of
differences between the dyadic resolutions of the image's
multiresolution. Wavelet Transform/ A special case of Subband
Transform. Wavelet coefficients Progressive Transmitting a given
image in successive steps, where each step Transmission adds more
detail to the rendering of the image Progressive Rendering A
sequence of rendering operations, each adding more detail.
Progressive by accuracy Transmit/render strong features first
(sharp edges), less significant features (texture) last Progressive
by Transmit/render low resolution first, high resolution last
resolution Progressive by spatial Transmit/render top of image
first, bottom of image last order Distributed database A database
architecture that can be used in a network environment
Subband/Wavelet tile A group of subband/wavelet coefficients
corresponding to a time- frequency localization at some given
spatial location and resolution/frequency Subband/Wavelet data An
encoded portion of a subband/wavelet tile that corresponds to block
some accuracy layer
[0044] 2. Overview of the Invention
[0045] Referring to FIG. 1, a block diagram is provided depicting
the various components of the imaging system in one embodiment. A
client computer 110 is coupled to a server computer 120 through a
communication network 130.
[0046] In one embodiment, the client computer 110 and server
computer 120 may comprise a PC-type computer operating with a
Pentium-class microprocessor, or equivalent. Each of the computers
110 and 120 may include a cache 111, 121 respectively as part of
its memory. The server may include a suitable storage device 122,
such as a high-capacity disk, CD-ROM, DVD, or the like.
[0047] The client computer 110 and server computer 120 may be
connected to each other, and to other computers, through a
communication network 130, which may be the Internet, an Intranet
(e.g., a local area network), a wide-area network, or the like.
Those having ordinary skill in the art will recognize that any of a
variety of communication networks may be used to implement the
present invention.
[0048] With reference to FIG. 2, the system workflow is described.
Using any browser type application, the user of the client computer
110 connects to the Web Server 140 or directly to the Imaging
server 120 as described in FIG. 1. He/she then selects, using
common browser tools an image residing on the Image file storage
122. The corresponding URL request is received and processed by the
Imaging Server 120. In case results of previous computations
previous computations on the image are not present in the Imaging
Cache 121, the server performs a fast preprocessing algorithm (see
.sctn.2.1) in a lossless mode. The result of this computation is
inserted into the cache 121. Unlike other more "traditional"
applications or methods which perform full progressive encoding of
the image in an "offline" type method, the goal of the
preprocessing step is to allow the server, after a relatively fast
computational step, to serve any ROI specified by the user of the
client computer. For example, for a 15M grayscale medical image,
using the described (software) server, installed on computer with a
Pentium processor, fast disk, running Windows NT, the preprocessing
step 501 will typically take 3 seconds. This is by order of
magnitude faster than a "full" compression algorithm such as [S],
[SP1], [T]. Serving ROI is sometimes called "pixels-on-demand"
which means a progressive transmission of any ROI of the image in
"real-time", where the quality of the view improves with the
transfer rate until lossless view is received in the client side.
Once the preprocessing stage is done, the server sends to the
client a notification that the "image is ready to be served". The
server also transmits the basic parameters associated with the
image such as dimensions, color space, etc. Upon receiving this
notification, the client can select any ROI of the image using
standard GUI. The ROI is formulated in step 203 into a request list
that is sent to the server. Each such request corresponds to a data
block (.sctn.4). The order of requests in the list corresponds to
some progressive mode selected in the context of the application
such as "progressive by accuracy" rendering of the ROI. Upon
receiving the ROI request list, the server processes the requests
according to their order. For each such request the server checks
if the corresponding data block exists in the Cache 121. If not,
the server then computes the data block, stores it in the cache and
immediately sends it to the client. Once a data block that was
requested arrives at the client, it is inserted into the cache 111.
At various points in time during the transfer process, a decision
rule invokes a rendering of the ROI. Obviously, if some of the data
blocks required for a high quality rendering of the ROI, were
requested from the server, but have not arrived yet, the rendering
of the ROI will be of lower quality. But, due to the progressive
request order, the rendering quality will improve with each
received data block in an "optimal" way. In case the user changes
the ROI, the rendering task at the client is canceled and a new
request list corresponding to a new ROI, sent from the client to
the server, will notify the server to terminate the previous
computation and transmission task and begin the new one.
[0049] 3. New Reversible Wavelet Transform
[0050] We will now explain in detail why the rate-distortion
behavior of our progressive lossless algorithm is better than other
known algorithms. Lossless wavelet transform, must be
integer-to-integer-transform, such that round-off errors are
avoided. In order to demonstrate the difference between lossy and
lossless transforms, let us look at the simplest wavelet, the Haar
wavelet. Let x(k) be the k-th component of the one-dimensional
discrete signal x. The first forward Haar transform step, in its
accurate "mathematical" form, is defined by: 1 { s ( n ) = 1 2 ( x
( 2 n + 1 ) + x ( 2 n ) ) , d ( n ) = 1 2 ( x ( 2 n + 1 ) - x ( 2 n
) ) , ( 3.1 )
[0051] where s is a low-resolution version of x, and d is the
"difference" between s and x. In the case of lossless transform,
applying the above transform results in round-off error. One
possibility is to apply the transform step suggested by [CDSI]: 2 {
s ( n ) = x ( 2 n + 1 ) + x ( 2 n ) 2 , d ( n ) = x ( 2 n + 1 ) - x
( 2 n ) . ( 3.2 )
[0052] The symbol .left brkt-bot..smallcircle..right brkt-bot. is
the floor function meaning "greatest integer less than or equal to
.smallcircle.", e.g.
.left brkt-bot.0.5.right brkt-bot.=0, .left brkt-bot.-1.5.right
brkt-bot.=-1, .left brkt-bot.2.right brkt-bot.=2, .left
brkt-bot.-1.right brkt-bot.=-1.
[0053] The one-dimensional transform step is generalized to 2D
separable transform step, by applying the 1D transform step twice,
first in the X-direction and than (on the first stage output) in
the Y-direction as described in FIG. 18, 19 and 20 (see also [M]
7.7). The full 2D Wavelet transform is applied by using the 2D
Wavelet transform step iteratively in the classic Mallat
decomposition of the image (FIG. 21) ([M] 7.7).
[0054] In (3.2) two properties are kept:
[0055] 1. Reversibility, i.e., one can restore x(2n) and x(2n+1),
by knowing s(n)and d(n), as follows: 3 { x ( 2 n ) = s ( n ) - d (
n ) 2 , x ( 2 n + 1 ) = d ( n ) + x ( 2 n ) . ( 3.3 )
[0056] 2. De-correlation, i.e., s(n) and d(n) contains the minimal
number of bits required in order to follow property 1. For example,
if the transform would have been defined by: 4 { s ( n ) = x ( 2 n
+ 1 ) + x ( 2 n ) , d ( n ) = x ( 2 n + 1 ) - x ( 2 n ) , ( 3.4
)
[0057] then the least significant bit of s(n) and d(n) would have
been the same and saved twice. In other words, there is a
correlation between s(n) and d(n) in (3.4). From the view point of
coding this should be avoided since there is a redundancy of
transmitting this bit.
[0058] On the other hand, the scaling property, that is a very
important one, is not kept in (3.2). Observe that the value of s(n)
computed by (3.2), is smaller than its "real mathematical value" as
computed in (3.1), by factor of {square root}{square root over
(2)}. Since s(n) should be rounded to an integer number, the fact
that s(n) is smaller than what it should be, increases the
round-off error. In low resolutions, the error is accumulated
through the wavelet steps.
[0059] If we take the error as a model of "white noise" added to
the i-th resolution in a multi-resolution representation of the
image, i.e. X, in FIG. 21, it can be proved that the variance of
this noise exponentially increases as a function of i. This
"contamination" to the multi-resolution image damages the coding
efficiency at low bit-rates. Let us describe this in detail for the
case of the Haar wavelet. We have two assumptions in our
analysis:
[0060] 1. The parity (least significant bit) of an arbitrary
coefficient c, in any of the wavelet steps is a uniformly
distributed random variable, i.e. 5 P r ( c 0 m o d 2 ) = P r ( c 1
m o d 2 ) = 1 2 .
[0061] 2. This parity is independent of other coefficient's parity
(Identically independent distributed).
[0062] Our referenced computation, i.e. the accurate computation,
is the Haar transform step defined in (3.1). We concentrate on the
LL-subband coefficients, because the low-resolution subbands are
computed from them. LL-subband coefficients are the result of a
2D-transform step (FIG. 18) 6 l l ( a c c u r a t e ) ( m , n ) = x
( 2 m + 1 , 2 n + 1 ) + x ( 2 m , 2 n + 1 ) + x ( 2 m + 1 , 2 n ) +
x ( 2 m , 2 n ) 2 , .
[0063] where m and n are the indices of the row and column of the
coefficient respectively.
[0064] As described in FIG. 18, 19 and 20, according to the step
defined in (3.2), we first apply the X-direction step 7 s ( k , n )
= x ( k , 2 n + 1 ) + x ( k , 2 n ) 2 ,
[0065] for each input row x(k,.multidot.).
[0066] Under assumption 1 mentioned above, we can write s(k,n) as 8
s ( k , n ) = x ( k , 2 n + 1 ) + x ( k , 2 n ) 2 + e , ( 3.5 )
[0067] where e is a random variable with a probability distribution
function (P.D.F.) p (.multidot.)defined by 9 { p ( - 0.5 ) = P r (
e = - 0.5 ) = 1 2 , p ( 0 ) = P r ( e = 0 ) = 1 2 . Therefore , (
3.6 ) E ( e ) = - 1 4 , Var ( e ) = 1 16 . ( 3.7 )
[0068] We then apply the Y-direction transform by 10 l l ( C D S I
) ( m , n ) = s ( 2 m + 1 , n ) + s ( 2 m , n ) 2 = s ( 2 m + 1 , n
) + s ( 2 m , n ) 2 + e ' . ( 3.8 )
[0069] As in (3.5) we can represent s(2m+1,n) and s(2m,n) by: 11 s
( 2 m + 1 , n ) = x ( 2 m + 1 , 2 n + 1 ) + x ( 2 m + 1 , 2 n ) 2 +
e 1 , ( 3.9 ) s ( 2 m , n ) = x ( 2 m , 2 n + 1 ) + x ( 2 m , 2 n )
2 + e 2 . ( 3.10 )
[0070] Now we can write: 12 l l ( C D S I ) ( m , n ) = x ( 2 m + 1
, 2 n + 1 ) + x ( 2 m + 1 , 2 n ) 2 + e 1 + x ( 2 m , 2 n + 1 ) + x
( 2 m , 2 n ) 2 + e 2 2 + e ' = x ( 2 m + 1 , 2 n + 1 ) + x ( 2 m +
1 , 2 n ) + x ( 2 m , 2 n + 1 ) + x ( 2 m , 2 n ) 4 + e 1 2 + e 2 2
+ e ' = x ( 2 m + 1 , 2 n + 1 ) + x ( 2 m + 1 , 2 n ) + x ( 2 m , 2
n + 1 ) + x ( 2 m , 2 n ) 4 + e ( 3.11 )
[0071] where e.sub.1,e.sub.2,e' are independent (assumption 2
above) random variables with 13 e x p e c t a t i o n 1 4 , V a r i
a n c e 1 16 , a n d e = e 1 2 + e 2 2 + e ' . T h e r e fo r e , E
( e ) = 1 2 E ( e 1 ) + 1 2 E ( e 2 ) + E ( e ' ) = - 1 2 , V a r (
e ) = V a r ( e 1 2 + e 2 2 + e ' ) = 1 4 V a r ( e 1 ) + 1 4 V a r
( e 2 ) + V a r ( e ' ) = = 1 4 1 16 + 1 4 1 16 + 1 16 = 3 32 . (
3.12 ) T h u s , l l ( C D S I ) ( m , n ) = l l ( a c c u r a t e
) ( m , n ) 2 + e . ( 3.13 )
[0072] e represents the approximation error of the LL-subband
coefficients, results from one 2D transform step. The error relates
to the accurate floating-point computation.
[0073] This was a description of a single 2D-transform step
assuming that the input coefficients are without any error. Now we
wish to evaluate the error accumulated after several steps.
[0074] At an arbitrary step i.gtoreq.0, we can assume that an input
coefficient can be written as:
x.sub.1(k,l)=x.sub.1.sup.(accurate)(k,l)+e.sub.1,
[0075] where x.sub.1.sup.(accurate) (k,l) is the accurate value
achieved by floating-point computation for all the previous steps,
i.e., a step defined by 14 { s ( n ) = x ( 2 n + 1 ) + x ( 2 n ) 2
, d ( n ) = x ( 2 n + 1 ) - x ( 2 n ) , ( 3.14 )
[0076] instead of the integer-to-integer computation in (3.2).
Observe that if x.sub.1.sup.(accurate) (k,l) is the i-th resolution
image coefficient, using (3.14) as the 1D Wavelet step, then 15 x i
( a c c u r a t e ) ( k , l ) = l l i - 1 ( a c c u r a t e ) ( k ,
l ) 2 ' , i 1 , ( 3.15 )
[0077] where ll.sub.l-1.sup.(accurate) (k,l) is the normalized
(L.sub.2-norm) LL-subband coefficient resulting from the i-th 2D
transform step using (3.1) as the 1D Wavelet step (see Figure ).
e.sub.1 , is the difference between x.sub.1(k,l) and
x.sub.1.sup.(accurate) (k,l) (I.e., the approximation error of the
integer computation made until now). E.g. e.sub.0=0(x.sub.0(k,l) is
an original image pixel), while e.sub.1 is a random number with 16
e x p e c t a t i o n - 1 2 a n d v a r i a n c e 3 32
[0078] (see (3.12)). 0.
[0079] Using (3.11), we get: 17 l l i ( C D S I ) ( m , n ) = x i (
a c c ) ( 2 m + 1 , 2 n + 1 ) + e i 1 + x i ( a c c ) ( 2 m + 1 , 2
n ) + e i 2 + x i ( a c c ) ( 2 m , 2 n + 1 ) + e i 3 + x ( a c c )
( 2 m , 2 n ) + e i 4 4 + e = x i ( a c c ) ( 2 m + 1 , 2 n + 1 ) +
x i ( a c c ) ( 2 m + 1 , 2 n ) + x i ( a c c ) ( 2 m , 2 n + 1 ) +
x i ( a c c ) ( 2 m , 2 n ) 4 + e i 1 + e i 2 + e i 3 + e i 4 4 e =
x i ( a c c ) ( 2 m + 1 , 2 n + 1 ) + x i ( a c c ) ( 2 m + 1 , 2 n
) + x i ( a c c ) ( 2 m , 2 n + 1 ) + x i ( a c c ) ( 2 m , 2 n ) 4
+ e i + 1 = x i + 1 ( a c c u r a t e ) + e i + 1 ,
[0080] where e.sub.1+1 is defined by 18 e i + 1 = e i 1 + e i 2 + e
i 3 + e i 4 4 + e ,
[0081] and corresponds to LL, subband.
[0082] Consequently 19 V a r ( e i + 1 ) = V a r ( e i ) 4 + V a r
( e ) .
[0083] Observe that 20 E ( e ) = - 1 2 , V a r ( e ) = 3 32 .
[0084] As a result, we can write recursive formulas for the error
expectation and variance after i steps. 21 { E ( e 0 ) = 0 , E ( e
i + 1 ) = E ( e i ) - 1 2 , { V a r ( e 0 ) = 0 , V a r ( e i + 1 )
= V a r ( e i ) 4 + 3 32 , ( 3.16 )
[0085] The explicit solutions to these formulas are 22 E ( e i ) =
- i 2 , V a r ( e i ) = 1 8 - 1 2 4 i + 1 . ( 3.17 )
[0086] By replacing 23 x i ( a c c u r a t e ) ( m , n ) with l l i
- 1 ( a c c u r a t e ) ( m , n ) 2 i
[0087] we get 24 l l i ( C D S I ) ( m , n ) = l l i - 1 ( a c c u
r a t e ) ( m , n ) 2 i + 1 + e i + 1 . ( 3.18 )
[0088] Thus, the approximation to ll.sub.1.sup.(accurate) (m,n)
is
2.sup.l+1ll.sub.l.sup.(CDSI)(m,n)=ll.sub.l.sup.(accurate)(m,n)+2.sup.l+1e.-
sub.l+1.
[0089] The approximation error expectation is 25 E ( 2 i e i ) = 2
i E ( e i ) = 2 i ( - i 2 ) = - i2 i - 1 .
[0090] The approximation error variance and standard deviation are
26 Var ( 2 i e i ) = 4 i Var ( e i ) = 4 i ( 1 8 - 1 2 4 i + 1 ) =
4 i - 1 8 4 i 8 = 2 2 i - 3 . Hence Std ( 2 i e i ) = Var ( 2 i e i
) 2 i - 1 2 .
[0091] Let us now evaluate the approximation error of the 3 other
subbands: 27 l h i ( C D S I ) ( m , n ) = l l i - 1 ( CDSI ) ( 2 m
+ 1 , 2 n + 1 ) + l l i - 1 ( CDSI ) ( 2 m + 1 , 2 n ) 2 - l l i -
1 ( CDSI ) ( 2 m , 2 n + 1 ) + l l i - 1 ( CDSI ) ( 2 m , 2 n ) 2 =
lh i ( accurate ) ( m , n ) 2 i + e i 1 + e i 2 2 + e ' - ( e i 3 +
e i 4 2 + e " ) = lh i ( accurate ) ( m , n ) 2 i + e i L H
[0092] where
[0093] e.sub.1.sup.k1.ltoreq.k .ltoreq.4 are identical to the
random variable whose expectation and variance are given in (3.17).
28 e i L H = e i 1 + e i 2 2 + e ' - ( e i 3 + e i 4 2 + e " )
[0094] e' and e" are identical to the random variable whose
expectation and variance are given in (3.7).
[0095] Thus,
E(e.sub.1.sup.Lll)0,
[0096] 29 Var ( e i L H ) 1 4 ( 1 8 - 1 2 4 i + 1 + 1 8 - 1 2 4 i +
1 ) + 1 16 + 1 4 ( 1 8 - 1 2 4 i + 1 + 1 8 - 1 2 4 i + 1 ) + 1 16 =
1 4 - 1 2 4 i + 1 . ( 3.19 )
[0097] The approximation to lh.sup.(accurate) (m,n) is
2'lh.sub.l.sup.(CDSI)(m,n)=lh.sub.l.sup.(accurate)(m,n)+2'e.sub.l.sup.LH.
[0098] The approximation error variance and standard deviation are:
30 Var ( 2 i e i L H ) = 4 i Var ( e i L H ) = 4 i ( 1 4 - 1 2 4 i
+ 1 ) = 4 i - 1 - 1 8 4 i - 1 . Therefore Std ( 2 i e i L H ) = Var
( 2 i e i L H ) 4 i - 1 = 2 i - 1 .
[0099] A similar approximation error estimation, it can be done
with the HL-subband and the HH-subband.
[0100] The approximation error evaluation results are summarized in
the following table where the error is the difference between the
normalized (in L.sub.2-norm) coefficients according to [CDSI]
reversible transform and the "mathematical" transform (defined in
(3.1)).
4TABLE 2 Normalized (in L.sub.2 -norm) approximation errors of the
Wavelet coefficients at resolution i .gtoreq. (1 = 0 is the highest
resolution) using the (CDS1) reversible Haar transform. Expectation
Variance Std LL.sub.i - error --(i + 1)2 31 4 i + 1 8 2 2 i - 1
.quadrature. 0.707 .multidot. 2.sup.i LH.sub.i - error 0 32 2 4 i -
1 8 4 i - 1 .quadrature. 0.5 .multidot. 2.sup.i HL.sub.i - error 33
- 1 4 2 i 34 3 4 i - 2 16 3 4 i - 2 .quadrature. 0.433 .multidot.
2.sup.i HH.sub.i - error 0 35 4 i - 1 8 2 2 i - 3 .quadrature.
0.354 .multidot. 2.sup.i
[0101] Assuming a low-bit rate transmission where only the
coefficients whose absolute value belongs to the range .left
brkt-bot.2.sup.b,2.sup.b +1) are encoded, for every resolution i,
where i is greater than b (less or more). It must be noted that the
large error implies a significant loss of coding efficiency.
[0102] Instead, we propose a new family of reversible transforms.
The proposed family of integer wavelet transforms has all three
properties:
[0103] 1. Reversibility
[0104] 2. De-correlatioin
[0105] 3. Scaling--i.e. improved approximation of the
"mathematical" transform.
[0106] Our 2D transform step is separable also, but the
one-dimensional transform step, which the 2D transform is based on,
is different for the X-direction (step 1901), the Y-direction step
applied on the low output of the X-direction step (step 2001) and
the Y-direction step applied on the high output of the X-direction
step (step 2002) as described in FIGS. 18, 19 and 20.
[0107] The full 2D Wavelet transform is applied by using the 2D
Wavelet transform step iteratively in the classic Mallat
decomposition of the image (FIG. 21) ([M] 7.7). As mentioned
before, the Wavelet coefficients in our proposed transform are all
scaled, i.e. normalized in L.sub.2-norm as the Wavelet coefficients
computed in the accurate "mathematical" transform.
[0108] In order to achieve the third property (improved
approximation of the "mathematical" transform), we define an extra
matrix we call the "Half bit-matrix" which enables the
reversibility of the High Y-transform step (step 2002). The
elements that belong to this matrix are bits, such that each bit
corresponds to an HH-subband coefficient in the following
interpretation. Let us describe this by the following example.
[0109] Supposing
s(n)=7,d.sup.(l)(n)=9
[0110] are a coefficient pair results from a reversible
de-correlated 1D-wavelet step 36 { d ( 1 ) ( n ) = x ( 2 n + 1 ) -
x ( 2 n ) , s ( n ) = x ( 2 n ) + x ( 2 n + 1 ) 2 .
[0111] Now, d.sup.(1) (n) has to be multiplied by 37 1 2 ,
[0112] in order to be scaled.
[0113] The binary form of d.sup.(1) (n)=9 is
d.sup.(1)(n)=1001.sub.2.
[0114] If we now divide d.sup.(1) (n) by 2 in a floating-point
computation we get 38 d F P ( n ) = 1 2 d ( 1 ) ( n ) = 100.1 2
.
[0115] Let us call the bit, which located on the write-side of the
floating point the "Half Bit". Observe that the Half Bit of
d.sup.FP (n) is the LSB of d.sup.(1) (n). Therefore, an equivalent
way to do this in an integer computation without loosing the
Half-Bit is to calculate first the LSB of d.sup.(1) (n) by
[0116] HalfBit(n)=d.sup.(1)(n)mod 2=9mod 2=1, then to shift-write
d.sup.(1) (n) by
d(n)=d.sup.(1)(n)>>1=1001>>1=100.
[0117] By saving d (n) and HalfBit (n) we can restore back
d.sup.(1)(n).
[0118] In the proposed transform, this Half-bit is needed in the
HH-subband coefficient computation. Therefore in our wavelet
decomposition for every HH-subband coefficient (in all scales)
there is a corresponding bit, which is the coefficient's
Half-bit.
[0119] The Half bit matrix is hidden in the HH-subband in the
description of FIGS. 18, 19 and 20. It is described explicitly in
the specification of the transform as much in the coding
algorithm.
[0120] We now present our integer-to-integer versions of the Haar
transform and the CDF (1,3) transform for the 2-dimensional
case.
[0121] 3.1 Reversible Haar and (CDF) (1,3) Transforms
[0122] 3.1.1Haar Transform
[0123] With respect to FIG. 19:
[0124] 3.1.1.1 Step 1901: X-direction
[0125] Forward Step 39 Forward Step _ { s ( n ) = x ( 2 n ) + x ( 2
n + 1 ) 2 , d ( n ) = x ( 2 n + 1 ) - x ( 2 n ) . ( 3.20 ) Inverse
Step _ { x ( 2 n ) = s ( n ) - d ( n ) 2 , x ( 2 n + 1 ) = d ( n )
+ x ( 2 n ) . ( 3.21 )
[0126] Inverse Step
[0127] With respect to FIG. 20:
[0128] 3.1.1.2 Step 2001: Y-direction-Low Forward Step 40 { s ( n )
= x ( 2 n ) + x ( 2 n + 1 ) , d ( 1 ) ( n ) = x ( 2 n + 1 ) - x ( 2
n ) 2 , d ( n ) = 2 d ( 1 ) ( n ) . ( 3.22 )
[0129] Remarks:
[0130] 1. s(n) is a scaled LL-subband coefficient.
[0131] 2. s(n) and d.sup.(1) (n) are de-correlated and a reversible
couple (can be transformed back to x(2n) and x(2n+1)), but
d.sup.(1) (n) is not scaled (it is half its "real value"). Thus,
d.sup.(1) (n) is multiplied by 2. Nevertheless, the LSB of the
LH-subband coefficient d(n) is known to be 0 and not encoded.
[0132] Inverse Step 41 Inverse Step _ { x ( 2 n + 1 ) = 1 2 ( s ( n
) + d ( n ) + ( s ( n ) m o d 2 ) ) , x ( 2 n ) = s ( n ) - x ( 2 n
+ 1 ) . ( 3.23 )
[0133] With respect to FIG. 20:
[0134] 3.1.1.3 Step 2002: Y-direction-High Forward Step 42 { d ( 1
) ( n ) = x ( 2 n + 1 ) - x ( 2 n ) , HalfBit ( n ) = ( d ( 1 ) ( n
) ) m o d 2 , d ( n ) = d ( 1 ) ( n ) 2 , s ( n ) = x ( 2 n ) + d (
n ) . ( 3.24 )
[0135] Remark: d.sup.(1) (n) and s(n) are de-correlated and
reversible couples, but d.sup.(1) (n) is not scaled (It is twice
its "real value"). Therefore, d.sup.(1) (n) is divided by 2. By
doing that, we lose its least significant bit, which cannot be
restored. To solve this problem, as explained before, we save this
bit as the "Half-Bit". Giving this name to that coefficient means
that its weight is 43 1 2
[0136] in the "real mathematical scale", and it is the
[0137] least significant (from the approximation point of
view).
[0138] Inverse Step 44 Inverse Step _ { x ( 2 n ) = s ( n ) - d ( n
) , d ( 1 ) ( n ) = 2 d ( n ) + HalfBit ( n ) , x ( 2 n + 1 ) = d (
1 ) ( n ) + x ( 2 n ) . ( 3.25 )
[0139] 3.1.2 CDF (1,3) Transform
[0140] 3.1.2.1 Step 1901: X-direction
[0141] With respect to FIG. 19:
[0142] Forward Step 45 Forward Step _ { s ( n ) = x ( 2 n ) + x ( 2
n + 1 ) 2 , d ( 1 ) ( n ) = x ( 2 n + 1 ) - x ( 2 n ) , d ( n ) = d
( 1 ) ( n ) + s ( n - 1 ) - s ( n + 1 ) 4 . ( 3.26 ) Inverse Step _
{ d ( 1 ) ( n ) = d ( n ) - s ( n - 1 ) - s ( n + 1 ) 4 , x ( 2 n )
= s ( n ) - d ( 1 ) ( n ) 2 , x ( 2 n + 1 ) = x ( 2 n ) + d ( 1 ) (
n ) . ( 3.27 )
[0143] Inverse Step
[0144] With respect to FIG. 20:
[0145] 3.1.2.2 Step 2001: Y-direction-Low Forward Step 46 { s ( n )
= x ( 2 n ) + x ( 2 n + 1 ) , d ( 1 ) ( n ) = x ( 2 n + 1 ) - x ( 2
n ) + s ( n - 1 ) - s ( n + 1 ) 8 2 , d ( n ) = 2 d ( 1 ) ( n ) . (
3.28 )
[0146] Remark: See remarks for (3.22).
[0147] Inverse Step 47 { s ( 1 ) ( n ) = s ( n ) - s ( n - 1 ) - s
( n + 1 ) 8 , x ( 2 n + 1 ) = 1 2 ( s ( 1 ) ( n ) + d ( n ) + ( s (
1 ) ( n ) mod 2 ) ) , x ( 2 n ) = s ( n ) - x ( 2 n + 1 ) . ( 3.29
)
[0148] With respect to FIG. 20:
[0149] 3.1.2.3 Step 2002: Y-direction-High Forward Step 48 { s ( n
) = x ( 2 n ) + x ( 2 n + 1 ) 2 , d ( 1 ) ( n ) = x ( 2 n + 1 ) - x
( 2 n ) , d ( 2 ) ( n ) = d ( 1 ) ( n ) + s ( n - 1 ) - s ( n + 1 )
4 , d ( n ) = d ( 2 ) ( n ) 2 , HalfBit ( n ) = d ( 2 ) ( n ) mod
2. ( 3.30 )
[0150] Inverse Step 49 { d ( 1 ) ( n ) = 2 d ( n ) + HalfBit ( n )
- s ( n - 1 ) - s ( n + 1 ) 4 , x ( 2 n ) = s ( n ) - d ( 1 ) ( n )
2 , x ( 2 n + 1 ) = d ( 1 ) ( n ) + x ( 2 n ) . ( 3.31 )
[0151] We now compute the approximation error probabilities of our
method, and show that it is significantly smaller. We start with
the LL-subband error. Assuming e.sub.l is the approximation error
of the LL-subband in the i-th resolution (FIG. 21), the LL-subband
coefficient in the i-th resolution can be written as: 50 ll i ( new
) ( m , n ) = x i ( new ) ( 2 m + 1 , 2 n + 1 ) + x i ( new ) ( 2 m
, 2 n + 1 ) 2 + x i ( new ) ( 2 m + 1 , 2 n ) + x i ( new ) ( 2 m ,
2 n ) 2 = x i ( acc ) ( 2 m + 1 , 2 n + 1 ) + e i - 1 1 + x i ( acc
) ( 2 m , 2 n + 1 ) + e i - 1 2 2 + e ' + x i ( acc ) ( 2 m + 1 , 2
n ) + e i - 1 3 + x i ( acc ) ( 2 m , 2 n ) + e i - 1 4 2 + e " = x
i ( acc ) ( 2 m + 1 , 2 n + 1 ) + x i ( acc ) ( 2 m , 2 n + 1 ) + x
i ( acc ) ( 2 m + 1 , 2 n ) + x i ( acc ) ( 2 m , 2 n ) 2 + e i - 1
1 + e i - 1 2 + e i - 1 3 + e i - 1 4 2 + e ' + e " = = ll i ( acc
) ( m , n ) + e i , ( 3.32 )
[0152] where
[0153] ll.sub.1.sup.(new) (m,n) is the new transform LL-subband
coefficient (i-th resolution).
[0154] e.sub.l-1.sup.k for 1.ltoreq.k.ltoreq.4, are identical
random variable representing the error from the previous level.
[0155] e' and e" are random variables with P.D.F. defined in
(3.6).
[0156] x.sub.1.sup.(acc)(m,n) is the i-th resolution image
coefficient using (3.1) as the 1D Wavelet step.
[0157] ll.sub.1.sup.(acc) (m,n) is the normalized (L.sub.2-norm)
LL-subband coefficient resulting from the i-th 2D transform step
using (3.1) as the 1D Wavelet step (see FIG. 21). 51 e i = e i - 1
1 + e i - 1 2 + e i - 1 3 + e i - 1 4 2 + e ' + e " .
[0158] Consequently 52 E ( e i ) = 4 E ( e i - 1 ) 2 + ( - 1 4 ) +
( - 1 4 ) = 2 E ( e i - 1 ) - 1 2 , ( 3.33 ) Var ( e i ) = 4 Var (
e i - 1 ) 4 + 1 16 + 1 16 = Var ( e i - 1 ) + 1 8 . ( 3.34 )
[0159] By knowing that e.sub.-1=0 we get 53 { E ( e i ) = 1 2 - 2 '
, Var ( e i ) = i + 1 8 . ( 3.35 )
[0160] Now we can easily evaluate the approximation error of the 3
other subbands: 54 lh i ( new ) ( m , n ) = 2 x i ( new ) ( 2 m + 1
, 2 n + 1 ) + x i ( new ) ( 2 m , 2 n + 1 ) 2 - x i ( new ) ( 2 m +
1 , 2 n ) + x i ( new ) ( 2 m , 2 n ) 2 2 = ( x i ( acc ) ( 2 m + 1
, 2 n + 1 ) + x i ( acc ) ( 2 m , 2 n + 1 ) ) - ( x i ( acc ) ( 2 m
+ 1 , 2 n ) + x i ( acc ) ( 2 m , 2 n ) ) 2 + e i - 1 1 + e i - 1 2
- e i - 1 3 - e i - 1 4 2 + e ' - e " + 2 e ''' = lh i ( acc ) ( m
, n ) + e i LH , ( 3.36 )
[0161] where
[0162] lh.sub.1.sup.(new) (m,n) is the new transform LH-subband
coefficient (i-th resolution).
[0163] lh.sub.1.sup.(accurate) (m,n) is the normalized
(L.sub.2-norm) LH-subband coefficient resulting from the i-th 2D
transform step using (3.1) as the 1D Wavelet step (see FIG.
21).
[0164] e'" is a random variable with P.D.F. defined in (3.6). 55 e
i LH = e i - 1 1 + e i - 1 2 - e i - 1 3 - e i - 1 4 2 + e ' - e "
+ 2 e '''
[0165] all other symbols are defined like in (3.32).
[0166] Hence 56 E ( e i LH ) = 2 E ( e i - 1 ) - 2 E ( e i - 1 ) 2
+ ( - 1 4 ) - ( - 1 4 ) + 2 ( - 1 4 ) = - 1 2 , ( 3.37 ) Var ( e i
LH ) = 4 Var ( e i - 1 ) 4 + 1 16 + 1 16 + 4 1 16 = i + 3 8 . (
3.38 )
[0167] Similar estimation can be done for the HL and the HH
subbands.
[0168] The error estimation (for all subbands) are summarized in
the following table where the error is the difference between the
normalized (in L.sub.2-norm) coefficients according to our new
reversible transform and the "mathematical" transform (defined in
(3. 1)).
5TABLE 2 Normalized (in L.sub.2 - norm) approximation errors of the
Wavelet coefficients at resolution i .gtoreq. 0 (1 = 0 is the
highest resolution) using the proposed reversible Haar transform.
The result for the LL-subband is valid for the proposed reversible
(1,3) transform also. Expectation Variance Std LL.sub.i - error 57
1 2 - 2 i 58 i + 1 8 59 i + 1 8 LH.sub.i - error 60 - 1 2 61 i + 3
8 62 i + 3 8 HL.sub.i - error 63 - 1 4 64 i 8 + 1 16 65 i 8 + 1 16
HH.sub.i - error 66 - 1 4 67 i 8 + 1 16 68 i 8 + 1 16
[0169] The meaning of this result is that in a low bit-rate, where
only large coefficients are encoded, this error is negligible.
[0170] 4. Imaging Protocol and Distributed Database
[0171] Dividing the Data into Tiles and Bit-Planes
[0172] For the purpose of efficient rendering the coefficients may
be sub-divided into tiles. The tiles of this invention differ from
previous art as shown in FIG. 12. As in the lossy algorithm, here
also the subband tiles are further decomposed to subband data
blocks. Each data block of lossless subband tile (FIG. 12) will
have a 4D coordinate
(t_x, t_y, t_resolution,t_bitPlane)
[0173] where
0.ltoreq.t_bitPlane.ltoreq.maxBitPlane(t_resolution).
[0174] Each data block contains the following data in encoded
format:
[0175] 1. Fort_bitPlane .gtoreq.2:
[0176] a. A list of the indices of all subband coefficients whose
absolute value is in the range
[2.sup.1--.sup.bitplane-1,2.sup.t--.sup.bitplane).
[0177] b. The sign of all the coefficients of a.
[0178] c. For t_bitPlane>2, an additional precision bit for any
coefficient that belongs to the current bit plane or any higher bit
plane.
[0179] 2. For t_bitPlane=1, which we call the "least significant
bit plane":
[0180] a. A list of the indices of HL-subband and HH-subband
coefficients whose absolute value belongs to the set {-1, 0,1}.
[0181] b. A "zero flag" for each coefficient of a, which indicates
if the coefficient is equal to zero or not.
[0182] c. The sign of all the coefficients of a, whose "zero flag"
is false.
[0183] d. The LSB of the HL-subband and HH-subband coefficients
that belong to higher bit plane.
[0184] Remark:
[0185] Since the LH-subband contains only even coefficients, their
LSB must be zero and is not coded.
[0186] 3. For t_bitPlane=0, which we call the "half bit plane", a
matrix of 69 ( tileLength 2 ) 2
[0187] bits associated with HH-subband coefficients as their last
"half bit" (See (3.24) or (3.30)).
[0188] 5. The Progressive Subband Coding Algorithm
[0189] 5.1 The Encoding Algorithm
[0190] The encoding algorithm of the present invention is performed
at the server 120. In the present imaging system this rather time
consuming task is performed locally in near real-time for a ROI,
and not on the full image. The encoding algorithm is described for
images with a single color component, such as grayscale images, but
of course may also be applied to images with multiple color
components. The straightforward generalization for an arbitrary
number of components will be explained later.
[0191] The lossless algorithm receive as input the following
parameters:
6TABLE 4 Lossless Encoding Algorithm Input Parameters Variable
Meaning coef Matrix of subband coefficients, containing 70 3
.times. ( tileLength 2 ) 2 coefficients HalfBit 71 Matrix of bits
containing ( tileLength 2 ) 2 bits .
[0192] The coding strategy is similar in some sense to that
described in A. Said and W. Pearlman, "A new, fast and efficient
image codec based on set partitioning", IEEE Trans. Circuits and
Systems for video Tech., Vol. 6, No. 3, pp. 243-250, 1996, but the
preferred embodiment uses no "Zero Tree" data. For all the data
blocks with t_bitPlane .gtoreq.2, we use the lossy encoding
algorithm described in previous art with the parameters:
[0193] coef :=coef (The lossy parameter coef initialized with the
lossless parameter coef)
[0194] equalBinSize :=True
.epsilon..sub.c:=2
[0195] Remark: The lossy algorithm encodes all the bit-plane
information for t_bitPlane.gtoreq.2.
[0196] For t_bitPlane.ltoreq.1, i.e. the least significant bit
plane (of the lossless algorithm) and the half bit plane, we use a
different algorithm described in 5.1.3.
[0197] 5.1.1 Encoding Algorithm Initialization
[0198] The lossless encoding algorithm initialization is the same
as the lossy algorithm of .sctn.4.1.1 in the above-cited Ser. No.
09/386,264, which disclosure is incorporated herein by reference.
In order to initialize the encoding algorithm, the following
procedure is performed:
[0199] 1. Assign to each coefficient coef (x,y) its bit plane
b(x,y) such that:
.vertline.coef (x, y).vertline..epsilon.[.epsilon..sub.c2.sup.b,
.epsilon..sub.c2.sup.b+1)
[0200] 2. Compute the maximum bit plane over all such coefficients:
72 max BitPlane ( tile ) = max x , y ( b ( x , y ) )
[0201] 3. Write the value of maxBitPlane(tile) using one byte as
the header of the data block:
[0202] (t_x,t_y,t_resolution, maxBitPlane (t_resolution))
[0203] 4. Initialize all the coefficients as members of their
corresponding Type16 group.
[0204] 5. Initialize a list of significant coefficients to be
empty.
[0205] 6. Initialize a coefficient approximation matrix coef as
zero.
[0206] 5.1.2 The Outer Loop
[0207] The outer loop of the encoding algorithm scans the bit
planes from b=maxBitPlane (tile) to b=0. The output of each such
bit plane scan is the subband data block. Since the last stage of
the encoding algorithm is arithmetic encoding of given symbols, at
the beginning of each scan the arithmetic encoding output module is
redirected to the storage area allocated for the particular data
block. Once the bit plane scan is finished and the data block has
been encoded, the output stream is closed and the bit plane b is
decremented. After the outer loop is finished the following stages
are performed:
[0208] 1. Least significant bit plane is encoded
(t_bitPlane=1).
[0209] 2. Half bit plane is encoded (t_bitPlane=0).
[0210] The output of the least significant bit plane scan is the
data block (FIG. 14):
[0211] (t_x,t_y, t_resolution, t_bitPlane=1).
[0212] The half bit plane data block is:
[0213] (t_x, t_y, t_resolution, t_bitPlane=0).
[0214] 5.1.3 Bit-Plane Scan
[0215] For t_bitPlane.gtoreq.2, the framework of the bit plane scan
is described in FIG. 24, while the pseudo code is given in the
above-cited Ser. No. 09/386,264, which disclosure is incorporated
herein by reference. The scan, for a given level b (b .gtoreq.2),
encodes all of the coefficients data corresponding to the absolute
value interval [2.sup.b-1,2.sup.b).
[0216] Remark: The encoder method is LastBitPlane ( ) is associated
to the t_bitPlane=2.
[0217] For the least significant bit plane, a pseudo code is
described in FIG. 15, while a flow chart is described in FIG.
22.
[0218] Explanations to the least significant bit encoding
algorithm:
[0219] 1. The coefficients scanning procedure, i.e. moreCoef ( )
procedure in FIG. 15 or "More coefficients?" in FIG. 22 includes
all the coefficients belong to the HH and the HL-subband (FIG. 14).
The LH-subband is skipped, since the least significant bit of each
coefficient in it is zero (see remark 2 for (3.22)).
[0220] 2. The procedure is CoefReporeted ( ) ("Is coefficient
reported?" in the flow chart) returns false if the coefficient is
one of {-0,1,1}, i.e. in all higher bit plane scans it was
insignificant, otherwise it returns true.
[0221] 3. The procedure is CoetExactZero ( ) ("Coefficient is
zero?" in the flow chart) returns true iff the coefficient is
zero.
[0222] 4. The procedure getCoefSign ( ) returns the coefficient's
sign.
[0223] For the half bit plane, a pseudo code is described in FIG.
16.
[0224] 5.2 The Decoding Algorithm
[0225] Obviously, this algorithm is a reversed step of the encoding
algorithm of section 5. 1, performed in the server 120. The client
computer 110 during the progressive rendering operation performs
the decoding algorithm. Similar to the encoding algorithm, the
decoding algorithm is described for an image with one component
(such as a grayscale image), but of course could also be used with
an image with more than one component. The input parameters to the
lossless algorithm are given below:
7TABLE 5 Lossless decoding algorithm input parameters Variable
Meaning coef Empty matrix of subband coefficients to be filled by
the decoding algorithm. HalfBit 73 Matrix of bits containing (
tileLength 2 ) 2 bits .
[0226] For all the data blocks with t_bitPlane.gtoreq.2, a "lossy"
decoding algorithm is utilized. The input parameters for the lossy
algorithm are:
[0227] coef:=coef
[0228] equalBinSize:=True
[0229] .epsilon..sub.c:=2
[0230] 5.2.1 Decoding Algorithm Initialization
[0231] 1. Assign to each coefficient Coef (x, y) the value
zero.
[0232] 2. Assign to each bit belongs to the HalfBit matrix the
value zero.
[0233] 3. Initialize all the coefficients as members of their
corresponding Type16 group.
[0234] 4. Initialize the list of significant coefficients to be
empty.
[0235] 5. If the "first" data block
[0236] (t_x,t_y, t_resolution, maxBitPlane (t_resolution))
[0237] is available at the client, read the first byte, which is
the value of maxBitPlane (tile).
[0238] 5.2.2 The Outer Loop
[0239] Upon the completion of the outer loop in 5.1.2the following
stages are preformed:
[0240] 1. The decoding algorithm scans the least significant bit
plane. The input to this stage is encoded data block (t_x, t_y,
t_resolution, LeastSiginificant_bitPlane).
[0241] 2. The decoding algorithm scans the half bit plane. The
input to this stage is encoded data block (t_x, t_y, t_resolution,
Half_bitPlane).
[0242] 5.2.3 Bit Plane Scan
[0243] The preferred embodiment follows the lossy prior art of the
above-cited Ser. No. 09/386,264, which disclosure is incorporated
herein by reference, for t_bitPlane.gtoreq.2. The scan, for a given
level b, decodes all of the coefficients' data corresponding to the
absolute value interval [.epsilon.2.sup.b,.epsilon.2.sup.b+1).
Pseudo codes of the least significant bit plane scan and half bit
plane scan are described in FIG. 16.
[0244] Explanations to the Pseudo Code:
[0245] 1. The decoder's method moreCoef ( ) scans all the
coefficients in the HH, HL and LH subband. But, since the
LH-subband is skipped in the encoding algorithm, we don't call to
decodeSymbol ( ) for its coefficients. Instead of this, we update
their least significant bit as zero.
[0246] 2. Recall that LH-subband coefficients that have not been
reported until the least significant bit-plane must be zero since
they are known to be even.
[0247] 6. Client Workflow
[0248] With reference to FIG. 4, we describe the workflow at the
client unit 110. Any new ROI generated by the user's action such as
a zoom-in, a scroll, or a luminance tuning invokes in step 401 a
call from the GUI interface to the client imaging module with the
new ROI view parameters. The client imaging module then computes in
step 402 which data blocks are required for the rendering of the
ROI and checks if these data blocks are available in the client
cache. If not, their coordinate is added to a request list ordered
according to some progressive mode. The request list is then
encoded in step 403 and sent to the server. The server responds to
the request by sending back to the client a stream of data blocks,
in the order in which they were requested. In step 404 the client
inserts them to their appropriate location in the distributed
database. At various points in time step 405, a rendering of the
ROI, is invoked. Naturally, the rendering operation is progressive
and is able to use only the currently available data in the
client's database.
[0249] 6.1 Step 401:Receiving the ROI Parameters
[0250] The imaging module on the client computer 120 receives from
the GUI interface module view parameters detailed in Table 5. These
parameters are used to generate a request list for the ROI. The
same parameters are used to render the ROI.
8TABLE 5 Variable Meaning worldPolygon A 2D polygon in the original
image coordinate system scale The view resolution. 0 < scale
< 1 implies a low view resolution, scale = 1 original resolution
and scale > 1 a higher than original view resolution deviceDepth
A number in the set {8, 16, 24} representing the depth of the
output device (screen, printer) viewQuality A quality factor in the
range [1, 7] where 1 implies very low quality and 7 implies loss
less quality luminanceMap If active: a curve defining a mapping of
medical images with more than 8 bits (typically 10, 12, 16 bits)
per grayscale values to an 8 bit screen progressiveMo One of:
Progressive By Accuracy, Progressive By Resolution, Progressive by
Spatial Order
[0251] The basic parameters of a ROI are worldPolygon and scale
which determine uniquely the ROI view. If the ROI is to be rendered
onto a viewing device with limited resolution, then a worldPolygon
containing a large portion of the image will be coupled by a small
scale. In the case where the rendering is done by a printer, the
ROI could be a strip of a proof resolution of the original image
that has arrived from the server computer 120. This strip is
rendered in parallel to the transmission, such that the printing
process will terminate with the end of transmission. The other view
parameters determine the way in which the view will be rendered.
The parameters deviceDepth and viewQuality determine the quality of
the rendering operation. In cases the viewing device is of low
resolution or the user sets the quality parameter to a lower
quality, the transfer size can be reduced significantly.
[0252] The parameter luminanceMap is typically used in medical
imaging for grayscale images that are of higher resolution than the
viewing device. Typically, screens display grayscale images using 8
bits, while medical images sometimes represent each pixel using 16
bits. Thus, it is necessary to map the bigger pixel range to the
smaller range of [0,2559].
[0253] Lastly, the parameter progressiveMode determines the order
in which data blocks should be transmitted from the server 120. The
"Progressive By Accuracy" mode is the best mode for viewing in low
bandwidth environments. "Progressive By Resolution" mode is easier
to implement since it does not require the more sophisticated
accuracy (bit plane) management and therefore is commonly found in
other systems. The superiority of the "progressive by accuracy"
mode can be mathematically proven by showing the superiority of
"non-linear approximation" over "linear approximation" for the
class of real-life images. See, e.g., R. A. DeVore, "Nonlinear
approximation", ActaNumerica, pp. 51-150, 1998.
[0254] The "Progressive by Spatial Order" mode is designed, for
example, for a "print on demand" feature where the ROI is actually
a low resolution "proof print" of a high resolution graphic art
work. In this mode the image data is ordered and received in a top
to bottom order, such that printing can be done in parallel to the
transmission.
[0255] However, since lossless compression is mostly required in
medical images transmission, where typically more than 8 bits
images are used, we amplify our discussion on the curve
(luminanceMap hereinabove) which defines the mapping from the
original image gray scale range (typically 10,12,16 bits) to an
8-bit screen. Further more, in medical images viewing, regardless
of the original image depth, mapping is required in order to
control the brightness and contrast of the image.
[0256] 6.1.1 Luminance Mapping
[0257] Mapping from original image depth (e.g. 10,12,16 bits ) to
screen depth (typically 8-bits), is defined by a monotonic function
(FIG. 17):
f:[0,2.sup.original.sup..sub.--.sup.image.sup..sub.--.sup.depth-1].fwdarw.-
[0,2.sup.screen.sup..sub.--depth-1]. (6.1)
[0258] The curve influences not only the mapping, i.e. the drawing
to the screen, but also the request from the server. To understand
that, let us concentrate in the maximal gradient of the curve (FIG.
17). In a lossy mode, the request is created such that the image
approximation in the client side is close enough to the original
image, i.e., the RMS (Root Mean Square Error) is visually
negligible. When a curve (mapping function) is applied, the RMS can
be increased or reduced. The maximal RMS increasing factor depends
on the maximal gradient of the curve as follows: 74 RMS_increasing
_factor = RMS ( f ( I ) , f ( I ^ ) ) RMS ( I , I ^ ) max ( f ' ) ,
( 6.2 )
[0259] where
[0260] I is the original image
[0261] is the approximated image
[0262] f is the mapping function 75 RMS ( I 1 , I 2 ) = ; I 1 - I 2
r; L 2 Image_size
[0263] max (f') is the maximal gradient of the curve.
[0264] We consider the worst case of the RMS increasing factor
i.e.:
[0265] RMS_increasing_factor=Maximal_gradient=max (f')
[0266] If the RMS increasing factor is greater than 1, it means
that the "new RMS" may be greater than we consider as visually
negligible error. Thus, the request list should be increase (more
bit-planes should be requested from the server) in order to improve
the approximation accuracy. Conversely, if the RMS increasing
factor is smaller than 1, the request listing can be reduced. The
exact specification of this is given in the following section.
[0267] 6.2 Step 402: Creating the Request List
[0268] In step 402 using the ROI view parameters, the client
imaging module at the client computer 110 calculates data blocks
request list ordered according to the progressiveMode. Given the
parameters worldPolygon and Scale, it may be determined which
subband tiles in the "frequency domain" participate in the
reconstruction of the ROI in the "time domain". These tiles contain
all the coefficients that are required for an "Inverse
Subband/Wavelet Transform" (IWT) step that produces the ROI. First,
the parameter dyadicResolution(ROI) is computed, which is the
lowest possible dyadic resolution higher than the resolution of the
ROI. Any subband tiles of a higher resolution than
dyadicResolution(ROI) do not participate in the rendering
operation. Their associated data blocks are therefore not
requested, since they are visually insignificant for the rendering
of the ROI. If scale.gtoreq.1, then the highest resolution subband
tiles are required. If scale.ltoreq.2.sup.1-numberOfRevolutions
then only the lowest resolution tile is required. For any other
value of scale we perform the mapping described below in Table
6.
9TABLE 6 scale highestSubbandResolution scale .ltoreq.
2.sup.1-numberOfResolutions 1 2.sup.1-numberOfResolutions <
numberOfResolutions - .left brkt-bot.-log.sub.2(scale).right
brkt-bot. scale .ltoreq. 1 scale > 1 numberOfResolutions
[0269] Once it has been determined which subband tiles participate
in the rendering of the ROI, it is necessary to find which of their
data blocks are visually significant and in what order they should
be requested. Using well known rate/distortion rules from the field
of image coding (such as is described in S. Mallat and F. Falzon,
"Understanding image transform codes", Proc. SPIE Aerospace Conf.,
1997), it is not too difficult to determine an optimal order in
which the data blocks should be ordered by the client imaging
module (and thus delivered by the server 120). This optimal order
is described in steps 301-310 of FIG. 3 for the "Progressive By
Accuracy" mode. The underlying mathematical principal behind this
approach is "Non-Linear Approximation".
[0270] First, the subband coefficients with largest absolute values
are requested since they represent the most visually significant
data such as strong edges in the image. Notice that high resolution
coefficients with large absolute value are requested before low
resolution coefficients with smaller absolute value. Within each
given layer of precision (bit plane) the order of request is
according to resolution; low resolution coefficients are requested
first and the coefficients of highestSubbandResolution are
requested last.
[0271] The main difficulty of this step is this: Assume a subband
tile is required for the rendering of the ROI. This means that
t_resolution.ltoreq.dyadicResoltition(ROI) and the tile is required
in the IWT procedure that reconstructs the ROI. It must be
understood which of the data blocks associated with the subband
tile represent visually insignificant data and thus should not be
requested. Sending all of the associated data blocks will not
affect the quality of the progressive rendering. However, in many
cases transmitting the "tail" of data blocks associated with high
precision is unnecessary since it will be visually insignificant.
In such a case, the user will see that the transmission of the ROI
from the server 120 is still in progress, yet the progressive
rendering of the ROI seems to no longer to change the displayed
image.
[0272] Additionally, the influence of the luminance mapping on the
accuracy level of the requested data block as described below.
Supposing for some t_x,t_y and t_resolution, the set
[0273]
{(t_x,t_y,t_resolution,t_bitPlane).vertline.T.ltoreq.t_bitPlane.lto-
req.maxPlaneBit(t_resolution)} is requested where T is the minimal
bit plane required to the current view. Here, where the luminance
mapping is taken in account, the value of T might be increased or
decreased.
[0274] The number of bit planes reduced (added) from the request
list is 76 log 2 ( 1 Maximal_gradient ) .
[0275] I.e., for those t_x,t_y and t_resolution mentioned before,
the following set is requested:
[0276] {(t_x,t_y,t_resolution, t_bitPlane)
T'.ltoreq.t_bitPlane.ltoreq.max- PlaneBit (t_resolution)}
[0277] where 77 T ' = T + log 2 ( 1 Maximal_gradient ) .
EXAMPLES
[0278] 1. Given
[0279] Image depth of 12-bit
[0280] Screen depth of 8-bit
[0281] Linear luminance mapping, I.e., 78 Maximal gradient = 2 8 2
12 = 2 - 4 .
[0282] The number of bit planes reduced from the request list is:
79 log 2 ( 1 Maximal gradient ) = log 2 ( 1 2 - 4 ) = 4.
[0283] 2. Given a luminance mapping with Maximal gradient=2
[0284] The number of bit planes reduced from the request list is:
80 log 2 ( 1 Maximal gradient ) = log 2 ( 1 2 ) = - 1.
[0285] Thus one bit plane is added to the original set.
[0286] 6.3 Step 403: Encoding the Request List
[0287] The client imaging module in the client computer 110 encodes
the request list into a request stream that is sent to the server
computer 120 via the communication network 130 (FIG. 1). The
request list encoding algorithm is a simple rectangle-based
procedure. The heuristics of the algorithm is that the requested
data block usually can be grouped into data block rectangles. From
the request list of data blocks indexed by the encoding algorithm
computes structures of the type
{(t_x,t_y,t_resolution,t_bitPlane), n.sub.x,n.sub.y}, n.sub.x,
n.sub.y.gtoreq.1 (1.3)
[0288] Each such structure represents the n.sub.x.times.n.sub.y
data blocks
{(t.sub.--x+i,t.sub.--y+j,t_resolution, t_bitPlane)}, 0<i
<n.sub.t, 0.ltoreq.j<n.sub.y
[0289] The encoding algorithm attempts to create the shortest
possible list of structures, collecting the data blocks to the
largest possible rectangles can do this. It is important to note
that the algorithm ensures the order of data blocks in the request
list is not changed, since the server 120 will respond to the
request stream by transmitting data blocks in the order in which
they were requested. A good example of when this works well is when
a user zooms in into a ROI at a high resolution that was never
viewed before. In such a case the request list might be composed of
hundreds of requested data blocks, but they will be collected to
one (x,y) rectangle for each pair
(t_resolution, t_bitPlane).
[0290] 6.4 Step 404: Receiving the Data Blocks
[0291] The client computer 110 upon receiving from the server
computer 120 an encoded stream containing data blocks, decodes the
stream and inserts the data blocks into their appropriate location
in the distributed database using their ID as a key. The simple
decoding algorithm performed here is a reversed step of the
encoding infra. Since the client 110 is aware of the order of the
data blocks in the encoded stream, only the size of each data block
need be reported along with the actual data. In case the server 120
informs of an empty data block, the receiving module marks the
appropriate slot in the database as existing but empty.
[0292] Recall that the subband tile associated with each data block
is denoted by the first three coordinates of the four coordinates
of a data block (t_x,t _y,t_resolution). From the subband tile's
coordinates the dimensions are calculated of the area of visual
significance; that is, the portion of the ROI that is affected by
the subband tile. Assume that each subband tile is of length
tileLength and that the wavelet basis used has a maximal filter
size maxFilterSize, then defining hFilterSize:=[maxFilterSize/2]
and factor :=numberOfResolutions-t_resolut- ion+1,we have that the
dimensions of the affected region of the ROI (in the original
image's coordinate system) are
[t.sub.--x.times.tileLength.sup.factor-hFilterSize.sup.factor,(t.sub.--x+1-
).times.tileLength.sup.factor+hFilterSize.sup.factor].times.
[t.sub.--y.times.tileLength.sup.factor-hFilterSize.sup.factor,(t.sub.--y+1-
).times.tilelength.sup.factor+hFilterSize.sup.factor]
[0293] These dimensions are merged into the next rendering
operation's region. The rendering region is used to efficiently
render only the updated portion of the ROI.
[0294] 6.5 Progressive Rendering
[0295] During the transmission of ROI data from the server to the
client, the client performs rendering operations of the ROI. To
ensure that these rendering tasks do not interrupt the transfer,
the client runs two program threads: communications and rendering.
The rendering thread runs in the background and draws into a
pre-allocated "off-screen" buffer. Only then does the client use
device and system dependant tools to output the visual information
from the "off-screen" to the rendering device such as the screen or
printer.
[0296] The rendering algorithm performs reconstruction of the ROI
at the highest possible quality based on the available data at the
client. That is, data that was previously cached or data that
"just" arrived from the server. For efficiency, the progressive
rendering is performed only for the portion of the ROI that is
affected by newly arrived data. Specifically, data that arrived
after the previous rendering task began. This "updated region" is
obtained using the method of step 404 described in .sctn.6.4.
[0297] The parameters of the rendering algorithm are composed of
two sets:
[0298] 1. The ROI parameters described in Table 3.
[0299] 2. The parameters transmitted from the server explained in
Table 5, with the exception of the jumpSize parameter, which is a
"server only" parameter.
[0300] The rendering algorithm computes pixels at the dyadic
resolution dyadicRresoliltion(ROI). Recall that this is the lowest
possible dyadic resolution that is higher than the resolution of
the ROI. The obtained image is then resized to the correct
resolution. Using a tiling of the multiresolution representation of
the ROI, the steps of the algorithm are performed on a tile by tile
basis as described in FIG. 11. Since the tiles' length are
tileLength, which is typically chosen as 64, the rendering
algorithm is memory efficient.
[0301] 6.5.1 The Rendering Rate
[0302] As ROI data is transmitted to the client 110, the rendering
algorithm is performed at certain time intervals of a few seconds.
At each point in time, only one rendering task is performed for any
given displayed image. To ensure that progressive rendering does
not become a bottleneck, two rates are measured: the data block
transfer rate and the ROI rendering speed. If it predicted that the
transfer will be finished before a rendering task, a small delay is
inserted, such that rendering will be performed after all the data
arrives. Therefore, in a slow network scenario (as the Internet
often is), for almost the entire progressive rendering tasks, no
delay is inserted. With the arrival of every few kilobytes of data,
containing the information of a few data blocks, a rendering task
visualizes the ROI at the best possible quality. In such a case the
user is aware that the bottleneck of the ROI rendering is the slow
network and has the option to accept the current rendering as a
good enough approximation of the image and not wait for all the
data to arrive.
[0303] 6.5.2 Memory Constraint Subband Data Structure
[0304] This data-structure is required to efficiently store subband
coefficients, in memory, during the rendering algorithm. This is
required since the coefficients are represented in long integer
(lossless coding mode) or floating-point (lossy coding mode)
precision which typically require more memory than pixel
representation (1 byte). In lossy mode, the coefficients at the
client side 110 are represented using floating-point
representation, even if they were computed at the server side 120
using an integer implementation. This will minimize round-off
errors.
[0305] At the beginning of the rendering algorithm, coefficient and
pixel memory strips are initialized. dyadicWidth(ROI) may be
denoted as the width of the projection of the ROI onto the
resolution dyadicResolution(ROI). For each component and resolution
1<j.ltoreq.dyadic Re solution (ROI), four subband strips are
allocated for the four types of subband coefficients: hl,lh,hh and
HalfBit. The coefficient strips are allocated with dimensions 81 [
2 j - dyadic Revolution ( ROI ) - 1 .times. dyadicWidth ( ROI ) , 3
2 .times. tileLength + maxFilterSize 2 . ]
[0306] For each component and resolution
1.ltoreq.j<dyadicResolittion a pixel strip is allocated with
dimensions 82 [ 2 j - dyadic Revolution ( ROI ) .times. dyadicWidth
( ROI ) , tileLength + maxFilterSize 2 ]
[0307] Beginning with the lowest resolution 1, the algorithm
proceeds with a recursive multiresolution march from the top of the
ROI to bottom (y direction. Referring to FIGS. 10 and 11, in step
1101, the multiresolution strips are filled with sub-tiles of
coefficients 1050 decoded from the database or read from the memory
cache. From the coefficients we obtain multiresolution pixels 1051
using an inverse subband transform step 1102 (shown in further
detail in FIG. 10). Each tine a tile of pixels at resolutions
j<dyadicResolition(ROI) is reconstructed, it is written into the
pixel strip at the resolution j . Each time a tile of pixels at the
highest resolution dyadicResolution(ROI) is reconstructed, it is
fed into the inverse color transform and resizing steps 1103,
1104.
[0308] 6.5.3 Step 1101: Decoding and Memory Caching
[0309] The subband coefficients data structure described previously
in section 6.5.2 is filled on a tile basis. Each such subband tile
is obtained by decoding the corresponding data blocks stored in the
database or reading from the memory cache. The memory cache is used
to store coefficients in a simple encoded format. The motivation is
this: the decoding algorithm described previously in section 5.2 is
computationally intensive and thus should be avoided whenever
possible. To this end the rendering module uses a memory cache 111
where subband coefficient are stored in very simple encoded format
which decodes very fast. For each required subband tile, the
following extraction procedure is performed, described in FIG. 25,
beginning at step2501. In step2502, if no data blocks are available
in the database for the subband tile, its coefficients are set to
zero (step2503). In step2504, if the tile's memory cache storage is
updated, namely it stores coefficients in the same precision as in
the database, then the coefficients can be efficiently read from
there (step2505). In step2506, the last possibility is that the
database holds the subband tile in higher precision. Then, the tile
is decoded down to the lowest available bit plane using the
algorithm previously described in section 5.2 and the cached
representation is replaced with a representation of this higher
precision information.
[0310] 6.5.4 Step 1102: Inverse Lossless Wavelet Transform
[0311] This is an inverse step to step 603 performed in the server
(see 7.1.5). Following FIG. 21 we see that four "extended" subband
coefficient sub-tiles of length tileLength/2+maxFilterSize at the
resolution i are read from the coefficient strips data structure
and transformed to a tile of pixels at the next higher resolution
using losslessWaveletdTransformTy- pe (i). If
i+1<dyadicResolution (ROI), the tile of pixels obtained by this
step is inserted into the pixel memory strip at the resolution i+1.
If i+1=dyadicresolulion (ROI) the tile of pixels is processed by
the next step of color transform.
[0312] Remark: Recall from 5.1.1 that the "half bits" are
initialized as zeros, therefore the inverse step is well defined
even if their "real" value is not available in the client yet.
[0313] 6.5.5 Step 1103: Inverse Color Transform
[0314] This is an inverse step to step 603 performed at the server
120. It is performed only for tiles of pixels at the resolution
highestSubbandResolution. At this stage, all of the pixels of each
such tile are in the outputColorSpace and so need to be transformed
into a displaying or printing color space. For example, if the
original image at the server 120 is a color image in the color
space RGB, the pixels obtained by the previous step of inverse
subband transform are in the compression color space YUV. To
convert back from YUV to RGB, we use the inverse step described in
FIG. 13. If the ROI is at an exact dyadic resolution, then the tile
of pixels in the rendering color space is written into the
off-screen buffer. Else it is resized in the next step.
[0315] 6.5.6 Step 1104: image resize
[0316] In case the resolution of the ROI is not an exact dyadic
resolution, the image obtained by the previous step must be
re-sized to this resolution. This can be accomplished using
operating system imaging functionality. In most cases the operating
system's implementation is sub-sampling which produces in many
cases an aliasing effect which is visually not pleasing. To provide
higher visual quality, the imaging system of the present invention
may use the method of linear interpolation, for example described
in J. Proakis and D. Manolakis, "Digital signal processing",
Prentice Hall, 1996. The output of the interpolation is written to
the off-screen buffer. From there it is displayed on the screen
using system device dependant methods.
[0317] 6.5.7 Step 1105: Mapping to 8-Bit Screen
[0318] When luminanceMap is active mapping to 8-bit screen is
performed using the mapping function described in 6.1.1.
[0319] 7. Server Workflow
[0320] With reference to
[0321] FIG. 5, the operation of the server computer 120 (FIG. 1)
will now be described. Initially, an uncompressed digital image is
stored in, for example, storage 122 of the server computer 120.
This uncompressed digital image may be a two-dimensional image,
stored with a selected resolution and depth. For example, in the
medical field, the uncompressed digital image may be contained in a
DICOM file.
[0322] Once the client computer 110 requests to view or print a
certain image, the server performs the preprocessing step 501. This
step is a computation done on data read from the original digital
image. The results of the computation are stored in the server
cache device 121. After this fast computation a "ready to serve"
message is sent from the server to the client containing basic
information on the image.
[0323] In step 502, the server receives an encoded stream of
requests for data blocks associated with a ROI that needs to be
rendered at the client. The server then decodes the request stream
and extracts the request list.
[0324] In step 503, the server reads from cache or encodes data
block associated with low resolution portions of the ROI, using the
cached result of the preprocessing stage 501.
[0325] If the ROI is a high-resolution portion of the image, the
server, in step 504, reads from cache or performs a "local" and
efficient version of the preprocessing step 501. Specifically, a
local portion of the uncompressed image, associated with the ROI,
is read from the storage 122, processed and encoded. In step 505,
the data that was encoded in steps 503-504 is progressively sent to
the client in the order it was requested.
[0326] 7.1 Step 501: preprocessing
[0327] The preprocessing step is now described with respect to
[0328] FIG. 6. The preprocessing algorithm's goal is to provide the
fastest response to the user's request to interact with the image.
Once this fast computational step is performed, the server is able
to provide efficient "pixel-on-demand" transmission of any client
ROI requests that will follow. In most cases the first ROI is a
view of the full image at the highest resolution that "fits" the
viewing device. The preprocessing algorithm begins with a request
for an uncompressed image that has not been processed before or has
been processed but the result of this previous computation has been
deleted from the cache. As explained, this unique algorithm
replaces the possibly simpler procedure of encoding the full image
into some progressive format. This latter technique will provide a
much slower response to the user's initial request then the
technique described below. At the end of the algorithm a "ready to
serve ROI of the image" message is sent to the client containing
basic information on the image. While some of this information,
image dimensions, original color space, resolution etc., is
available to the user of the client computer, most of this
information is "internal" and required by the client to formulate
ROI request lists (.sctn.6.2) and progressively render (.sctn.6.5).
Next we describe in detail the preprocessing algorithm.
[0329] 7.1.1 Preprocessing Parameters
10TABLE 7 Variable Meaning losslessMode If true, preprocessing will
prepare data that can be used for lossless transmission.
subbandTransformType The framework allows the ability to select a
different subband transform for each resolution of each image. The
technical term is: non-stationary transforms. numberOfResolutions
The number of resolutions in the Multiresolution structure
calculated for the image. Typically, numberOfResolutions =
log.sub.2 ({square root over (ImageSize)}). jumpSize A number in
the range [0, numberOfResolutions - 1]. The preprocessing stage
computes only the top lower part of the image's multiresolution
pyramid of the size numberOfResolutions - jumpSize. tileLength
Typically = 64. Tradeoff between time and coding performance.
nTilesX (j) Number of subband tiles in the x direction at the
resolution j nTilesX (j) Number of subband tiles in the y direction
at the resolution j inputColorSpace Color space of uncompressed
original image. outputColorSpace Transmitted color space used as
part of the encoding technique. numberOfComponents Number of
components in OutputColorSpace. threshold (c, j) A matrix of values
used in lossy compression. The subband coefficients of the
component c at the resolution j with absolute value below threshold
(c, j) are considered as (visually) insignificant and set to
zero.
[0330] Given an input image, the parameters described in Table 7
are computed or chosen. These parameters are also written into a
header sent to the client 110 and are used during the progressive
rendering step 405 (see section 6.5, described previously). The
important parameters to select are:
[0331] 1. losslessMode: In this mode, progressive transmission of
images takes place until lossless quality is obtained. Choosing
this mode requires the preprocessing algorithm to use certain
reversible wavelet transforms, and can slow down the algorithm.
[0332] 2. subbandtransformType(j): The (dynamic) selection of
wavelet basis (as described, for example, in 1. Daubechies, "Ten
lectures on wavelets", Siam, 1992) is crucial to the performance of
the imaging system. The selection can be non-stationary, meaning a
different transform for each resolution j. The selection is derived
from the following considerations:
[0333] a. Coding performance (in a rate/distortion sense): This is
obviously required from any decent subband/wavelet transform.
[0334] b. Approximation of ideal low pass: It is favorable to
choose a transform such that low resolutions of the image will be
of high visual quality (some filters produce poor quality low
resolutions even before any compression takes place).
[0335] c. Fast transform implementation: Can the associated fast
transform be implemented using lifting steps (as described, for
example, by I. Daubechies and W. Sweldens, "Factoring wavelet
transforms into lifting steps", J. Fourier Anal. Appl., Vol. 4, No.
3, pp. 247-269, 1998), using only integer shifts and additions,
etc. Some good examples are the Haar and CDF transforms (1,3),
(2,2)*** described in 1. Daubechies, "Ten lectures on wavelets",
Siam, 1992.
[0336] d. Fast low pass implementation: A very important parameter,
since together with the parameter jumpSize, it determines almost
all of the complexity of the algorithm. For example, the CDF (1,3)
is in this respect the "optimal" transform with three vanishing
moments. Since the dual scaling function is the simple B-spline of
order 1, its low pass is simple averaging. Thus, the sequence of
CDF transforms, using the B-spline of order 1 as the dual scaling
function, but with wavelets with increasing number of vanishing
moments are in some sense optimal in the present system. They
provide a framework for both real time response and good coding
efficiency.
[0337] e. Lossless mode: If losslessMode is true we must choose the
filters from a subclass of reversible transforms (see, for example,
"Wavelet transforms that map integers to integers", A. Calderbank,
I. Daubechies, W. Sweldens, B. L. Yeo, J. Fourier Anal. Appl.,
1998).
[0338] f. Low system I/O: If the network 130 in FIG. 1 connecting
between the Image residing on the storage 122 and the imaging
server 120 is slow, the bottleneck of the preprocessing stage (and
the whole imaging system for that fact) might be simply the reading
of the original image. In such a case a transform may be chosen
with a lazy sub-sampling low pass filter that corresponds to
efficient selective reading of the input image. Many interpolating
subband transforms with increasing number of vanishing moments can
be selected to suit this requirement. However, this choice should
be avoided whenever possible, since it conflicts with (a) and
(b).
[0339] g. Image type: If the type of the image is known in advance,
an appropriate transform can be chosen to increase coding
efficiency. For example: Haar wavelet for graphic images, smoother
wavelet for real-life images, etc. In the graphic arts field, there
are numerous cases of documents composed of low resolution
real-life images and high resolution graphic content. In such a
case, a non-stationary sequence of transforms may be chosen: Haar
for the high resolutions and a smoother basis starting at the
highest resolution of a real-life image part. In case of low system
I/O (f), a non-stationary choice of interpolating transforms of
different orders is required.
[0340] 3. jumpSize: This parameter controls the tradeoff between
fast response to the user's initial request to interact with the
image and response times to subsequent ROI requests. When jumpSize
is large, the initial response is faster, but each computation of a
region of interest with higher resolution than the jump might
require processing of a large portion of the original image.
[0341] 4. InputColorSpace: The input color spaces supported in
lossless mode are:
[0342] a. Grayscale: For grayscale images
[0343] b. RGB
[0344] 5. outputColorSpace: The following are color spaces which
perform well in coding:
[0345] a. Grayscale: For grayscale images
[0346] b. YUV: for viewing color images
[0347] Referring to Table 7 [LP], losslessMode is set to true.
Threshold (c,j) is not in use, since in lossless mode, there is no
thresholding. The rest of the variables have the same meaning as in
the lossy algorithm.
[0348] 7.1.2 Memory Constraint Multiresolution Scan Data
Structure
[0349] Most wavelet coding algorithms have not addressed the
problem of memory complexity. Usually the authors have assumed
there is sufficient memory such that the image can be transformed
in memory from the time domain to a wavelet frequency domain
representation. It seems the upcoming JPEG2000 will address this
issue, as did its predecessor JPEG. The preprocessing algorithm
also requires performing subband transforms on large images,
although not always on the full image, and thus requires careful
memory management. This means that the memory usage of the
algorithm is not of the order of magnitude of the original image,
as described in J. M. Shapiro, "An embedded hierarchical image
coder using zero-trees of wavelet coefficients", IEEE Trans. Sig.
Proc., Vol. 41, No. 12, pp. 3445-3462, 1993.
[0350] Given an uncompressed image we allocate the following number
of memory strips
numberOfComponents.times.(numberOfResolutions-jumpSize)
[0351] of sizes
[2.sup.-(numberOfResolutions-1imageWidth,
3.times.tileLength/2+maxFilterSi- ze]
[0352] for 1.ltoreq.j.ltoreq.numberOfResolutions-jumpSize-1 and
[imageWidth, tileLength+2.times.maxFilterSize]
[0353] for j=numberOfResolutions-jumpSize
[0354] That is, the memory usage is proportional to
2.sup.-jumpSize.times.imageWidth. Each such strip stores low-pass
coefficients in the color space outputColorSpace at various
resolutions.
[0355] Referring to FIG. 6, during the preprocessing stage, the
resolutions are scanned simultaneously from start to end in the y
direction. For each color component and resolution, the
corresponding strip stores low-pass coefficients or pixels at that
resolution. The core of the preprocessing algorithm are steps
604-607, where tiles of pixels of length
tileLength+2.times.maxFilterSize are read from the memory strips
and handled one at a time. In step 604 the tile is transformed into
a tile of length tileLength containing two types of coefficient
data: subband coefficients and pixels at a lower resolution. The
subband coefficients are processed in steps 605-606 and are stored
in the cache. The lower resolution pixels are inserted in step 607
to a lower resolution memory strip. Whenever Such a new sub-tile of
lower resolution pixels is inserted into a strip, the algorithm's
memory management module performs the following check: If the part
of the sub-tile exceeds the current virtual boundaries of the
strip, the corresponding first lines of the strip are considered
unnecessary anymore. Their memory is (virtually) re-allocated for
the purpose of storing the sub-tile's new data.
[0356] 7.1.3 Step 601: Lossless Color-Transform
[0357] This step is uses the conversion formula described in FIG.
13. This step must be performed before step 602, because the
lossless color conversion is non-linear.
[0358] 7.1.4 Step 602: Lossless Wavelet Low Pass
[0359] The motivation for the low pass step is explained in
.sctn.6.1.4 in the above-cited Ser. No. 09/386,264, which
disclosure is incorporated herein by reference. In a lossless mode
there are a few emphasis that are represented here.
[0360] In step 602, the low pass filters of the transforms
subbandTransformType(j),
[0361]
numberOfResolutions-jumpSize<j.ltoreq.numberOfResolutions, are
used to obtain a low resolution strip at the resolution
numberOfResolutions-jumpSize (as can be seen in FIG. 26).
Typically, it is required to low pass about 2.sup.jumpSize lines of
the original image 1010 to produce one line of the low resolution
image. The low pass calculation is initiated by a read of tiles of
pixels from the memory strips performed in step 604. Whenever there
is an attempt to read missing low resolution lines, they are
computed by low passing the original image and inserted into the
memory strip. The insertion over-writes lines are no longer
required, such that the algorithm is memory constrained. In the
case where a non-linear color transform took place in the previous
step 601, the results of that transform are low-passed.
[0362] In the lossless mode of operation, the jumpSize parameter
defines the number of lossless wavelet low pass steps should be
done. A single low pass step is the same for Haar and CDF (1,3) and
defined by the following two stages (taken from (3.20) and (3.22):
83 X - d i r e c t i o n : s ( n ) = x ( 2 n ) + x ( 2 n + 1 )
2
[0363] Y-direction: s(n)=X(2n)+X(2n+1).
[0364] Namely, in a 2D representation, the low pass step is defined
by 84 l l ( m , n ) = x ( 2 m + 1 , 2 n + 1 ) + x ( 2 m , 2 n + 1 )
2 + x ( 2 m + 1 , 2 n ) + x ( 2 m , 2 n ) 2 . ( 7.1 )
[0365] ForjumpSize=1 and jumpSize=2 (other sizes practically are
not needed), the server performs these steps efficiently (almost
like the lossy algorithm) by a single operation that simulates
exactly jumpSize low pass steps defined in (7.1). As noticed from
(7.1), the simplicity of the formula makes filters such as Haar and
CDF (1,3) "optimal" in the respect of low pass efficiency.
[0366] 7.1.5 Step 603: Forward Lossless Wavelet Transform
[0367] In Step 603 we perform one step of an efficient local
lossless wavelet transform (.sctn.3), on a tile of pixels at the
resolution 1<j<numberOfResolutions-jumpSize . The type of
transform is determined by the parameter
losslessWaveletTransformType (j). As described in FIG. 1, the
transform is performed on an "extended" tile of pixels is of length
tileLength+2.times.maxFilterSize (unless we are at the boundaries),
read directly from a multi-resolution strip at the resolution j+1.
The output of the step is a lossless subband tile composed of
wavelet coefficients including Half bit coefficients and low
resolution coefficients/pixels. The transform step is efficiently
implemented in integers as described in .sctn.3.
[0368] The subband transform of step 603 outputs three types of
data: scaling function (low pass), wavelet (high pass) and
Halfbits. The wavelet coefficients are treated in step 604 while
the scaling function coefficients are treated in step 605.
[0369] Remark: Tiles of pixels which are located on the boundaries
sometimes need to be padded by extra rows and/or columns of pixels,
such that they will formulate a "full" tile of length
tileLength.
[0370] 7.1.6 Step 604: Variable Length encoding and storage In step
604, the subband coefficients that are calculated in step 603 are
variable length encoded and stored in the cache 121. If maxBitPlane
(tile)=0 we do not write any data. Else we loop on the coefficient
groups {coef(2.times.i+x,2.times.j+y)}.sub.x,y=0,1. For each such
group we first write the group's variable length length(i,j) using
log.sub.2 (maxBitPlane(tile)) bits. Then for each coefficient in
the group we write length (i,j)+1 bits representing the
coefficient's value. The least significant bit represents the
coefficient's sign: if it is 1 then the variable length encoded
coefficient is assumed to be negative. The HalfBit subband
coefficients are written in one-bit per coefficient.
[0371] 7.1.7 Step 605: Copying Low Pass Coefficients into the
Multiresolution Strip Structure
[0372] In step 503, unless losslessMode is true, the subband
coefficients calculated in step 604 are quantized. This procedure
is performed at this time for the following reason: It is required
that the coefficients computed in the previous step will be stored
in the cache 121. To avoid writing huge amounts of data to the
cache, some compression is required. Thus, the quantization step
serves as a preparation step for the next variable length encoding
step. It is important to point out that the quantization step has
no effect on compression results. Namely, the quantization step is
synchronized with the encoding algorithm such that the results of
the encoding algorithm of quantized and non-quantized coefficients
are identical.
[0373] A tile of an image component c at the resolution j is
quantized using the given threshold threshold(c,j): for each
coefficients x, the quantized value is .left brkt-bot.x/threshold
(c,j).right brkt-bot.. It is advantageous to choose the parameters
threshold (c,j) to be dyadic such that the quantization can be
implemented using integer shifts. The quantization procedure
performed on a subband tile is as follows:
[0374] 1. Initialize maxBitPlane(tile)=0.
[0375] 2. Loop over each group of four coefficients {coef
(2.times.i+x, 2.times.j+y)}.sub.x,y=0,1. For each such group
initialize a variable length parameter length(i,j)=0.
[0376] 3. Quantize each coefficient in the group coef (2.times.i+x,
2.times.j+y) using the appropriate threshold.
[0377] 4. For each coefficient, update length(i,j) by the bit plane
b of the coefficient, where the bit plane is defined by
.vertline.coef (2.times.i+x,2.times.j+y).vertline..epsilon..left
brkt-bot.2.sup.bthreshold(c,j),2.sup.b+1threshold(c,j))
[0378] 5. After processing the group of four coefficients, use the
final value of length (i,j) to update maxBitPlane (tile) by
maxBitPlane(tile)=max(maxBitPlane(tile),length (i,j))
[0379] 6. At the end of the quantization step, store the value
maxBitPlane(tile) in the cache 121.
[0380] Note that for subband tiles located at the boundaries we can
set to zero subband coefficients that are not associated with the
actual image, but only with a padded portion. To do this we take
into account the amount of padding and the parameter maxFilterSize.
The motivation for the "removal" of these coefficients is coding
efficiency.
[0381] 7.2 Step 502: Decoding the Request Stream
[0382] This is the inverse step of section 6.3. Once the request
stream arrives at the server 120, it is decoded block to a data
block request list. Each data structure the type representing a
group of requested data blocks is converted to the sub-list of
these data blocks.
[0383] 7.3 Step 503: Encoding low resolution part of ROI
[0384] Step 503 is described in
[0385] FIG. 7. It is only performed whenever the data blocks
associated with low-resolution subband tile are not available in
the server cache 121.
[0386] Step 701 is the inverse step of step 604 described in
.sctn.7.1.6. In the preprocessing algorithm subband tiles of lower
resolution, that is resolutions lower than
numberOfResolutions-jumpSize, were stored in the cache using a
variable length type algorithm. For such a tile we first need to
decode the variable length representation. The algorithm uses the
stored value maxBitPlane(tile).
[0387] 1. If maxBitPlane(tile)=0, then all the coefficients are set
to zero including the HalfBit subband.
[0388] 2. If maxBitPlane(tile)=1, then all the coefficients are set
to zero, and the HalfBit subband coefficient are read bit by bit
from cache.
[0389] 3. Else, as performed in 2, the HalfBit subband coefficient
are read bit by bit from cache, and we perform the following simple
decoding algorithm:
[0390] For each group of four coefficients {coef(2.times.i+x,
2.times.j+y)}.sub.x,y=0,1, we read log.sub.2 (maxBitPlane(tile))
bits representing the variable length of the group.
[0391] Assume the variable length is length(i,j). For each of the
four coefficients we then read length(i,j)+1 bits. The least
significant bit represents the sign. The reconstructed coefficient
takes the value: 85 ( readBits >> 1 ) .times. { - 1 readBits
& 1 = 1 1 readBits & 1 = 0
[0392] In step 702 we use the encoding algorithm described in
.sctn.5.1 to encode the requested data blocks associated with the
extracted subband tile.
[0393] 7.4 Step 504: Processing high resolution part of ROI
[0394] Step 504 is described in FIG. 8. In case we have used
jumpSize>0 in step 501 and the resolution of the
ROI>numberOfResolutions-jumpSize- , we are sometimes required to
perform a local variation of the preprocessing step described in
.sctn.7.1. Whenever the server receives a request list of data
blocks we check the following. If a data block has been previously
computed (present in the cache 121) or is associated with a low
resolution subband tile data block then it is either simply read
from the cache or handled in step 503. Else, the coordinates of the
data block are used to find the "minimal" portion of the ROI that
needs to be processed. Then, a local version of the preprocessing
algorithm is performed for this local portion. The difference here
is that step 804 replaces Variable Length coding step 604 of the
preprocessing algorithm by the encoding algorithm given in
.sctn.5.1.
[0395] 7.5 Step 505: Progressive Transmission of ROI
[0396] In the final step, the encoded data tiles are sent from the
server 120 to the client 110, in the order they were requested. In
many cases, data blocks will be empty. For example, for a region of
the original image with a constant pixel value, all of the
corresponding data blocks will be empty, except for the first one
that will contain only one byte with the value zero representing
maxBitPlane(tile)=0. For a low activity region of the image, only
the last data blocks representing higher accuracy will contain any
data. Therefore, to avoid the extra side information, rectangles of
empty data blocks are collected and reported to the client 110
under the restriction that they are reported in the order in which
they were requested. For blocks containing actual data, only the
data block's size in bytes need be reported, since the client 110
already knows which data blocks to expect and in which order.
[0397] The present invention has been described in only a few
embodiments, and with respect to only a few applications (e.g.,
commercial printing and medical imaging). Those of ordinary skill
in the art will recognize that the teachings of the present
invention may be used in a variety of other applications where
images are to be transmitted over a communication media
* * * * *