U.S. patent application number 17/052186 was filed with the patent office on 2021-02-25 for image processing system and method.
This patent application is currently assigned to HFIPIX, INC.. The applicant listed for this patent is HFIPIX, INC.. Invention is credited to Shawn L. KELLY.
Application Number | 20210058629 17/052186 |
Document ID | / |
Family ID | 1000005247134 |
Filed Date | 2021-02-25 |
![](/patent/app/20210058629/US20210058629A1-20210225-D00000.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00001.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00002.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00003.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00004.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00005.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00006.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00007.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00008.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00009.png)
![](/patent/app/20210058629/US20210058629A1-20210225-D00010.png)
View All Diagrams
United States Patent
Application |
20210058629 |
Kind Code |
A1 |
KELLY; Shawn L. |
February 25, 2021 |
IMAGE PROCESSING SYSTEM AND METHOD
Abstract
Initial low-quality images of a progressively-displayed
high-definition image are masked with corresponding
progressively-revealing mask filters or masking algorithms to
realistically obscure such low quality and therefore to provide a
realistically appearing progressive presentation of the
high-definition image.
Inventors: |
KELLY; Shawn L.; (Colorado
Springs, CO) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
HFIPIX, INC. |
Colorado Springs |
CO |
US |
|
|
Assignee: |
HFIPIX, INC.
Colorado Springs
CO
|
Family ID: |
1000005247134 |
Appl. No.: |
17/052186 |
Filed: |
May 9, 2019 |
PCT Filed: |
May 9, 2019 |
PCT NO: |
PCT/US2019/031625 |
371 Date: |
October 31, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62669296 |
May 9, 2018 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04N 19/80 20141101;
H04N 19/132 20141101; G06T 2207/30168 20130101; H04N 19/112
20141101; G06T 5/10 20130101; G06T 5/005 20130101; H04N 19/85
20141101; H04N 19/16 20141101 |
International
Class: |
H04N 19/16 20060101
H04N019/16; H04N 19/132 20060101 H04N019/132; G06T 5/00 20060101
G06T005/00; H04N 19/112 20060101 H04N019/112; H04N 19/80 20060101
H04N019/80; H04N 19/85 20060101 H04N019/85; G06T 5/10 20060101
G06T005/10 |
Claims
1. A method of processing a progressively-encoded image,
comprising: a. transmitting to a recipient a plurality of image
components of the progressively-encoded image; b. transmitting to
said recipient a corresponding plurality of sets of mask parameters
in correspondence with said plurality of image components, wherein
each set of mask parameters of said plurality of sets of mask
parameters, when used in cooperation with an associated mask
filter, provides for obscuring detailed features of an image
associated with a corresponding image component of said plurality
of image to components, while retaining a recognizable
representation of an unmasked version of said image, successive
image components of said plurality of image components are
associated with images of successively increasing image-quality,
and corresponding successive sets of mask parameters of said
plurality of sets of mask parameters provide for successively less
masking of said detailed features of said images associated with
said successive image components of said plurality of image
components.
2. (canceled)
3. A method of processing a progressively-encoded image,
comprising: a. receiving a plurality of image components of the
progressively-encoded image; b. receiving a corresponding plurality
of sets of mask parameters in correspondence with said plurality of
image components, wherein each set of mask parameters of said
plurality of sets of mask parameters, when used in cooperation with
an associated mask filter, provides for obscuring detailed features
of an image associated with a corresponding image component of said
plurality of image components, while retaining a recognizable
representation of an unmasked version of said image, successive
image components of said plurality of image components are
associated with images of successively increasing image-quality,
and corresponding successive sets of mask parameters of said
plurality of sets of mask parameters provide for successively less
masking of said detailed features of said images associated with
said successive image components of said plurality of image
components; c. masking each of said plurality of image components
of said progressively-encoded image with said mask filter using a
corresponding said set of mask parameters of said plurality of sets
of mask parameters so as to generate a corresponding masked image;
and d. displaying said masked image on a display device.
4. (canceled)
5. A method of processing a progressively-encoded image,
comprising: a. receiving a plurality of image components of the
progressively-encoded image, wherein successive image components of
said plurality of image components are associated with images of
successively increasing image-quality; b. masking each of a
plurality of images associated with said plurality of image
components with a mask filter using a corresponding predetermined
set of mask parameters so as to generate corresponding masked
image, wherein a degree to which each image of said plurality of
images is masked responsive to said predetermined set of mask
parameters is inversely related to said image-quality of said
image; and c. displaying each of said plurality of images in
succession on a display device.
6. (canceled)
7. A method of processing a progressively-encoded image as recited
in claim 1, further comprising: a. receiving a highest-definition
image; b. progressively encoding said highest-definition image so
as to generate said plurality of image components of said
progressively-encoded image; and c. storing said plurality of image
components for later transmission to said recipient, wherein said
later transmission is commenced upon demand from said
recipient.
8. A method of processing a progressively-encoded image as recited
in claim 7, wherein said highest-definition image is received from
a proprietor of a website on which said image is intended to be
displayed.
9. A method of processing a progressively-encoded image as recited
in claim 1, further comprising: a. providing for a user to
interactively adjust at least one set of mask parameters of said
plurality of sets of mask parameters responsive to a display of a
filtered version of a corresponding image component of said
plurality of image components filtered by said mask filter
responsive to said at least one set of mask parameters; and b.
storing said at least one set of mask parameters in association
with said corresponding image component.
10. A method of processing a progressively-encoded image as recited
in claim 1, further comprising receiving said plurality of image
components and said plurality of sets of mask parameters from a
separate image processing application.
11. A method of processing a progressively-encoded image as recited
in claim 1, wherein each set of mask parameters of said plurality
of sets of mask parameters comprises a value for each of at least
one mask parameter selected from a measure of contrast, a measure
of transparency, a measure of brightness, a measure of color, a
measure of a range of tonal values, a measure of a shift of a range
of tonal values, and a characterization of an associated image
histogram, wherein said value for each said at least one mask
parameter provides for obscuring artifacts in a corresponding said
image component to an extent that is inversely related to said
image-quality of said image component.
12. A method of processing a progressively-encoded image as recited
in claim 1, further comprising enhancing edge detail of at least
one relatively-lower-definition image component of said plurality
of image components prior to the transmission thereof to said
recipient.
13. A method of processing a progressively-encoded image as recited
in claim 1, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of sequential image
supplements, each sequential image supplement of said plurality of
sequential image supplements provides for reconstructing a next
progression of said progressively-encoded image from a previous
progression of said progressively-encoded image, beginning with
said lowest-definition base image, and said next progression of
said progressively-encoded image has a higher-definition than said
previous progression of said progressively-encoded image.
14. A method of processing a progressively-encoded image as recited
in claim 1, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of
progressively-relatively-higher-definition images culminating with
the image that was progressively encoded.
15-25. (canceled)
26. A method of processing a progressively-encoded image as recited
in claim 3, wherein each set of mask parameters of said plurality
of sets of mask parameters comprises a value for each of at least
one mask parameter selected from a measure of contrast, a measure
of transparency, a measure of brightness, a measure of color, a
measure of a range of tonal values, a measure of a shift of a range
of tonal values, and a characterization of an associated image
histogram; and said value for each said at least one mask parameter
provides for obscuring artifacts in a corresponding said image
component to an extent that is inversely related to said
image-quality of said image component.
27. A method of processing a progressively-encoded image as recited
in claim 3, further comprising enhancing edge detail of at least
one relatively-lower-definition unmasked image component of said
plurality of unmasked image components prior to said plurality of
image components being masked to obscure said detailed features of
said image.
28. A method of processing a progressively-encoded image as recited
in claim 3, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of sequential image
supplements, each sequential image supplement of said plurality of
sequential image supplements provides for reconstructing a next
progression of said progressively-encoded image from a previous
progression of said progressively-encoded image, beginning with
said lowest-definition base image, and said next progression of
said progressively-encoded image has a higher-definition than said
previous progression of said progressively-encoded image.
29. A method of processing a progressively-encoded image as recited
in claim 3, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of
progressively-relatively-higher-definition images culminating with
the image that was progressively encoded.
30-32. (canceled)
33. A method of processing a progressively-encoded image as recited
in claim 5, wherein each set of mask parameters comprises a value
for each of at least one mask parameter selected from a measure of
contrast, a measure of transparency, a measure of brightness, a
measure of color, a measure of a range of tonal values, a measure
of a shift of a range of tonal values, and a characterization of an
associated image histogram; and said value for each said at least
one mask parameter provides for obscuring artifacts in a
corresponding image component of said plurality of image components
to an extent that is inversely related to said image-quality of
said image component.
34. A method of processing a progressively-encoded image as recited
in claim 5, further comprising enhancing edge detail of at least
one relatively-lower-definition image component of said plurality
of image components prior to said plurality of image components
being masked to obscure detailed features of said
progressively-encoded image.
35. A method of processing a progressively-encoded image as recited
in claim 5, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of sequential image
supplements, each sequential image supplement of said plurality of
sequential image supplements provides for reconstructing a next
progression of said progressively-encoded image from a previous
progression of said progressively-encoded image, beginning with
said lowest-definition base image, and said next progression of
said progressively-encoded image has a higher-definition than said
previous progression of said progressively-encoded image.
36. A method of processing a progressively-encoded image as recited
in claim 5, wherein said plurality of image components comprise a
lowest-definition base image and a plurality of
progressively-relatively-higher-definition images culminating with
the image that was progressively encoded.
37-42. (canceled)
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] The instant application is the U.S. National Phase under 35
U.S.C. .sctn. 371 of International Application No.
PCT/US2019/031625 filed on 9 May 2019, with claims divided from
International Application No. PCT/US2019/031625, the latter of
which claims the benefit of prior U.S. Provisional Application Ser.
No. 62/669,296 filed on 9 May 2018. Each of the above-identified
applications is incorporated herein by reference in its
entirety.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] FIG. 1 illustrates a first aspect of an image processing
system that provides for progressively masking a
progressively-displayed high-definition image so as to provide for
obscuring artifacts in associated intermediate images, masked
versions of which are progressively displayed leading up to the
display of the final high-definition image;
[0003] FIG. 2 illustrates a process for creating the filter masks
for each of the intermediate images, masked versions of which are
progressively displayed when progressively displaying the
associated high-definition image in accordance with the image
processing system illustrated in FIG. 1;
[0004] FIG. 3 illustrates a process for determining a particular
associated filter-mask parameter for a corresponding particular
intermediate image, in support of the process illustrated in FIG.
2;
[0005] FIG. 4a illustrates a first-progression, first-quality
intermediate image associated with the progressive display of the
high-definition image illustrated in FIG. 4c, together with a
histogram of associated image pixel values for the entire
first-quality intermediate image;
[0006] FIG. 4b illustrates a masked version of the
first-progression, first-quality intermediate image illustrated in
FIG. 4a, together with a histogram of associated image pixel values
for the entire masked first-progression, first-quality intermediate
image;
[0007] FIG. 4c illustrates, for purposes of comparison, the
high-definition image from which the first-progression,
first-quality intermediate image illustrated in FIG. 4a is derived,
together with a histogram of associated image pixel values for the
entire high-definition image;
[0008] FIG. 4d illustrates a masked version of the high-definition
image illustrated in FIG. 4c, together with a histogram of
associated image pixel values for the entire masked high-definition
image, wherein the associated filter-mask parameter--associated
with the associated histogram--was selected so that the
corresponding images of FIGS. 4b and 4d had similar appearance;
[0009] FIG. 5a illustrates a second-progression, second-quality
intermediate image associated with the progressive display of the
high-definition image illustrated in FIGS. 4c and 5c, together with
a histogram of associated image pixel values for the entire
second-quality intermediate image, wherein the second-quality
intermediate image has a higher quality than the first-quality
intermediate image illustrated in FIG. 4a;
[0010] FIG. 5b illustrates a masked version of the
second-progression, second-quality intermediate image illustrated
in FIG. 5a, together with a histogram of associated image pixel
values for the entire masked second-progression, second-quality
intermediate image;
[0011] FIG. 5c illustrates, for purposes of comparison, the
high-definition image the same as illustrated in FIG. 4c--from
which the second-progression, second-quality intermediate image
illustrated in FIG. 5a is derived, together with a histogram of
associated image pixel values for the entire high-definition
image;
[0012] FIG. 5d illustrates a masked version of the high-definition
image illustrated in FIG. 5c together with a histogram of
associated image pixel values for the entire masked high-definition
image, wherein the associated filter-mask parameter--associated
with the associated histogram--was selected so that the
corresponding images of FIGS. 5b and 5d had similar appearance;
[0013] FIG. 6a illustrates a third-progression, third-quality
intermediate image associated with the progressive display of the
high-definition image illustrated in FIGS. 4c, 5c and 6c, together
with a histogram of associated image pixel values for the entire
third-quality intermediate image, wherein the third-quality
intermediate image has a higher quality than the second-quality
intermediate image illustrated in FIG. 5a;
[0014] FIG. 6b illustrates a masked version of the
third-progression, third-quality intermediate image illustrated in
FIG. 6a, together with a histogram of associated image pixel values
for the entire masked third-progression, third-quality intermediate
image;
[0015] FIG. 6c illustrates, for purposes of comparison, the
high-definition image the same as illustrated in FIGS. 4c and
5c--from which the third-progression, third-quality intermediate
image illustrated in FIG. 6a is derived, together with a histogram
of associated image pixel values for the entire high-definition
image;
[0016] FIG. 6d illustrates a masked version of the high-definition
image illustrated in FIG. 6c, together with a histogram of
associated image pixel values for the entire masked high-definition
image, wherein the associated filter-mask parameter--associated
with the associated histogram--was selected so that the
corresponding images of FIGS. 6b and 6d had similar appearance;
[0017] FIG. 7a is a copy of the first-progression, first-quality
intermediate image illustrated in FIG. 4a, for purposes of
comparison with the intermediate images illustrated in FIGS. 5a/7b
and 6a/7c, and for comparison with the high-definition image
illustrated in FIGS. 4c, 5c, 6c and 7d;
[0018] FIG. 7b is a copy of the second-progression, second-quality
intermediate image illustrated in FIG. 5a, for purposes of
comparison with the intermediate images illustrated in FIGS. 4a/7a
and 6a/7c, and for comparison with the high-definition image
illustrated in FIGS. 4c, 5c, 6c and 7d;
[0019] FIG. 7c is a copy of the third-progression, third-quality
intermediate image illustrated in FIG. 6a, for purposes of
comparison with the intermediate images illustrated in FIGS. 4a/7a
and 5a/7b, and for comparison with the high-definition image
illustrated in FIGS. 4c, 5c, 6c and 7d;
[0020] FIG. 7d a copy of the high-definition image the same as
illustrated in FIGS. 4c, 5c and 6c--from which the intermediate
images illustrated in FIGS. 7a-7c were derived, for purposes of
comparison therewith;
[0021] FIG. 8a is a copy of the masked first-progression,
first-quality intermediate image illustrated in FIG. 4b, for
purposes of comparison with the masked intermediate images
illustrated in FIGS. 5b/8b and 6b/8c, and for comparison with the
high-definition image illustrated in FIGS. 4c, 5c, 6c, 7d and
8d;
[0022] FIG. 8b is a copy of the masked second-progression,
second-quality intermediate image illustrated in FIG. 5b, for
purposes of comparison with the masked intermediate images
illustrated in FIGS. 4b/8a and 6b/8c, and for comparison with the
high-definition image illustrated in FIGS. 4c, 5c, 6c, 7d and
8d;
[0023] FIG. 8c is a copy of the masked third-progression,
third-quality intermediate image illustrated in FIG. 6b, for
purposes of comparison with the masked intermediate images
illustrated in FIGS. 4b/8a and 5b/8b, and for comparison with the
high-definition image illustrated in FIGS. 4c, 5c, 6c, 7d and
8d;
[0024] FIG. 8d a copy of the high-definition image--the same as
illustrated in FIGS. 4c, 5c, 6c, 7d and 8d--from which the masked
intermediate images illustrated in FIGS. 8a-8c were derived, for
purposes of comparison therewith;
[0025] FIG. 9 illustrates a first aspect of a process for
progressively receiving and forming intermediate and final images
of a progressively-displayed high-definition image, and for
applying associated mask filters to the intermediate images either
prior to, or during, the display thereof, in accordance with the
first aspect of the image processing system illustrated in FIG.
1;
[0026] FIG. 10 illustrates a second aspect of an image processing
system that provides for progressively generating and masking
associated intermediate images of a progressively-displayed
high-definition image so as to provide for obscuring artifacts in
the associated intermediate images that are progressively displayed
leading up to the display of the final high-definition image;
[0027] FIG. 11 illustrates a process for generating and masking
progressive image components, and for storing the masked
progressive image components for use by the second aspect of the
image processing system;
[0028] FIG. 12 illustrates a process for progressively receiving
and displaying masked intermediate images and a final image of a
progressively-displayed high-definition image, in accordance with
the second aspect of the image processing system illustrated in
FIG. 10;
[0029] FIG. 13a is a compressed and tonally-shifted version of the
first-progression, first-quality intermediate image illustrated in
FIG. 4a, together with a histogram of associated image pixel values
for the entire image illustrated in FIG. 13a, in accordance with a
second aspect of an associated masking process;
[0030] FIG. 13b is a compressed and tonally-shifted version of the
second-progression, second-quality intermediate image illustrated
in FIG. 5a, together with a histogram of associated image pixel
values for the entire image illustrated in FIG. 13b, in accordance
with the second aspect of the associated masking process;
[0031] FIG. 13c is a compressed and tonally-shifted version of the
third-progression, third-quality intermediate image illustrated in
FIG. 6a, together with a histogram of associated image pixel values
for the entire image illustrated in FIG. 13c, in accordance with
the second aspect of the associated masking process;
[0032] FIG. 13d a copy of the high-definition image--the same as
illustrated in FIGS. 4c, 5c, 6c, 7d and 8d--from which the
intermediate images illustrated in FIGS. 13a-13c were derived, for
purposes of comparison therewith, together with a histogram of
associated image pixel values for the entire image illustrated in
FIG. 13d;
[0033] FIG. 14 illustrates a flow chart of the first and second
aspects of the image processing system illustrated in FIGS. 1 and
10, respectively, from the point-of-view of an associated image
server device; and
[0034] FIG. 15 illustrates a flow chart of the first and second
aspects of the image processing system illustrated in FIGS. 1 and
10, respectively, from the point-of-view of an associated image
receiving and display device.
DESCRIPTION OF EMBODIMENT(S)
[0035] Referring to FIG. 1, a first aspect 10.1 of an image
processing system 10, 10.1 provides for uploading both a
high-definition image 12, IMG.sup.0 and an associated set of mask
parameters .alpha..sup.N, .alpha..sup.N-1, . . . , .alpha..sup.2,
.alpha..sup.1 from a website proprietor 14 to a webpage 16 of a
server device 18, acting as an internet webserver 18', for
distribution to a client internet-connected receiving device 20,
for example, operating an internet web browser 22 under control of
a user 24.
[0036] Due to transmission bandwidth limitations, the transmission,
receipt and display of a high-quality digital image is often
preceded by one or more relatively lower quality, and therefore
lower bandwidth, initial and/or intermediate image representations
so that viewer(s) thereof has/have a perceived lower delay before
being able to assimilate image content. Such a practice is inherent
in many progressive image delivery methods which either
incrementally reconstruct and display higher spatial image quality
from an initial low-quality image as additional image data is
received, or alternatively, which simply include the transmission,
receipt and display of initial, low-quality placeholder images
prior to the receipt and display of the separate higher quality
image. However, the low-quality initial or intermediate images of
such methods are typically of such low spatial quality that they
may appear heavily pixelated, and therefore artificial or heavily
blurred. These initial images therefore represent a sufficiently
substantial and unrealistic compromise of spatial quality that many
viewers prefer to simply wait for the final high-quality image
without having to view any prior intermediate image
representations.
[0037] Generally, the progressive transmission and display of
images provides for transforming the content of a
relatively-higher-quality image into a base image and one or more
image supplements, or supplemental images, wherein the base image
is transmitted and displayed first, and the one or more image
supplements or supplemental images are progressively transmitted
and progressively used to increase the quality of, or detail in,
the associated displayed image.
[0038] In accordance with a first aspect 26.1 of a progressive
imaging process 26, 26.1, successive image supplements .DELTA.(k-1,
k) are combined with the image content of a predecessor image to
generate its successor image, the latter being of relatively
greater quality than the former, wherein generally image supplement
.DELTA.(k-1, k) provides for transforming the k.sup.th image of the
progression into the corresponding (k-1).sup.st image of the
progression. For example, in accordance with what is known as
"progressive JPEG" (Joint Photographic Experts Group), each image
supplement .DELTA.(k-1, k) comprises an additional set of
coefficients for building detail of higher spatial frequency in the
JPEG-restored image. Alternatively, a high-quality image may be
progressively, and losslessly, transmitted, reconstructed and
displayed in accordance with the teachings of U.S. Pat. No.
8,798,136 or 8,855,195, each of which are incorporated herein by
reference, wherein the image supplements .DELTA.,(i,j) comprise the
associated extra data that are combined with data of a predecessor
image to generate a relatively-higher quality successor image. In
accordance with an interlaced imaging process, each image
supplement .DELTA.(k-1, k) may comprise values for additional
pixels that were missing from the predecessor image.
[0039] In accordance with a second aspect 26.2 of a
progressive-imaging process 26, 26.2, distinct supplemental images
IMG.sup.K-1 are successively displayed, wherein each successive
supplemental images IMG.sup.K-1 is a relatively-higher quality
image that replaces a corresponding relatively-lower quality, and
therefore, relatively-smaller bandwidth, predecessor image
IMG.sup.K.
[0040] More particularly, in accordance with a first embodiment
10.1' of the first-aspect image processing system 10, 10.1'
incorporating an associated first-aspect progressive imaging
process 26, 26.1, upon request from a user 24 seeking to display
the high-definition image 12, IMG.sup.0 on the client
internet-connected receiving device 20, the server device 18
initially transmits a base image IMG.sup.N, followed by image
supplements .DELTA.(N-1,N), .DELTA.(N-2,N-1), . . . , .DELTA.(2,3),
.DELTA.(1,2), .DELTA.(0-,1) thereto that provide for progressively
improving the resolution of the displayed image, culminating with
the display of the original high-definition image 12, IMG.sup.0 to
the extent possible, wherein, for example, a first image supplement
.DELTA.(N-1,N) provides for generating a first-improved image
IMG.sup.N-1 from the original base image IMG.sup.N, a second image
supplement .DELTA.(N-2,N-1) provides for generating a
second-improved image IMG.sup.N-2 from the first-improved image
IMG.sup.N-1, and so on until the last image supplement .DELTA.(0,
1) provides for generating the high-definition image 12, IMG.sup.0
from the next-to-last-improved image IMG.sup.1. The server device
18 further transmits the mask parameters .alpha..sup.N,
.alpha..sup.N-1, . . . , .alpha..sup.2, .alpha..sup.1 that provide
for masking the displayed images so as to obscure
progressive-display-related artifacts therein, in accordance with
an associated masking process.
[0041] Alternatively, in accordance with a second embodiment 10.1''
of the image processing system 10, 10.1'', the first-aspect image
processing system 10, 10.1'' may alternatively utilize an
underlying second-aspect progressive-imaging process 26.2, wherein
independent intermediate images IMG.sup.N-1, image IMG.sup.N-2, . .
. , image IMG.sup.2, image IMG.sup.1, and eventually the final
image IMG.sup.0, are each transmitted in succession, rather than
the above-described image supplements .DELTA.,(i,j), wherein the
separate independent intermediate images IMG.sup.N-1, image
IMG.sup.N-2, . . . , image IMG.sup.2, image IMG.sup.1 are each
accompanied by the mask parameters .alpha..sup.N, .alpha..sup.N-1,
. . . , .alpha..sup.2, .alpha..sup.1 that provide for masking the
displayed images so as to obscure progressive-display-related
artifacts therein, in accordance with an associated masking
process, the same as for the above-described first embodiment 10.1'
of the first-aspect image processing system 10, 10.1'.
[0042] The image processing system 10, 10.1 provides for masking
the above-described artifacts in the initial and/or intermediate
image representations so that the underlying relatively low quality
of these images is not perceived as such by the viewers, but
instead, the viewer perceives an image having an underlying
relatively high-quality content that emerges from a fog as the
quality of the underlying intermediate images improves.
[0043] This masking process can significantly improve the perceived
spatial quality of progressively-delivered initial and intermediate
images of progressive image delivery methods by restricting the
spatial detail visibility of such images so that low spatial
quality is not immediately apparent, while simultaneously providing
an impression or illusion of presumed high spatial quality being
seen through, or behind, a realistic obscuring medium, for example,
similar to a fog or haze, which obscures the presumed higher
quality of the image. Accordingly, the presentation of
progressively higher quality images with a concurrent progressive
reduction of this obscuration gives the appearance of a gradual but
realistic removal or clearing of the perceived obscuring medium to
reveal the final image quality which was reasonably presumed to
always exist but for the obscuration. This limitation of spatial
visibility, hereinafter referred to as "masking", may include, but
is not limited to, reductions in contrast, with or without changes
in transparency, brightness and/or color, as represented by
modifications of pixel values, for example, in one set of
embodiments, resulting in, at a minimum, a compression or
limitation of the range of tonal values of the image histogram
relative to that of the unmasked image. For example, in general,
such a compression or limitation may be applied by a transformation
of the image tonal values by an associated mask filter 28 as
follows:
OutputPixelValue(i,j,k)=T(.alpha..sup.k)BackgroundPixelValue(i,j)+(1-T(.-
alpha..sup.k))P(i,j,k) (1)
wherein:
P(i,j,k)=HalfMax+Bias(.alpha..sup.k)+.gamma.(.alpha..sup.k)(InputPixelVa-
lue(i,j,k)-HalfMax) (2)
and: [0044] i, j are pixel coordinates in the display; [0045] k is
the progressive image progression level, wherein k=0 for the
original high-quality image, and k=N for the lowest quality version
of that image; [0046] .alpha..sup.k is the mask parameter used for
masking the k.sup.th progression of the progressive image, which
controls the quality of the resulting masked image per the
preference of either the website proprietor or the client, with the
level of quality decreasing with increasing value of the
progression level k, wherein the mask parameters are in one-to-one
correspondence with the progression levels of the associated
progressively-encoded image; [0047] BackgroundPixelValue(i,j) the
value of the image pixel at location (i,j) that would otherwise be
displayed, absent a display of the progressive image. For example,
wherever an image is displayed, typically there is some default
value of each pixel (i,j) on the display in that location before
even the first, lowest quality image is displayed. For example, a
webpage may have a default background color of white before
anything is actually displayed from the progressive image.
Accordingly, BackgroundPixelValue(i,j) is the value that the pixel
would have had prior to displaying the corresponding pixel of the
first progression of the progressive image. [0048] T(.alpha..sup.k)
is a transparency level ranging in value between 0 and 1, with 0
providing for a fully opaque image over the corresponding
associated BackgroundPixelValue(i,j), and 1 providing for a fully
transparent image over the corresponding associated
BackgroundPixelValue(i,j), wherein the transparency level
T(.alpha..sup.k) is dependent upon the value of the mask parameter
.alpha..sup.k, i.e. dependent upon the level of quality of the
associated progressive image. For example, in one set of
embodiments, the transparency level T(.alpha..sup.k) decreases with
increasing image quality, i.e. with increasing value of mask
parameter .alpha..sup.k, the latter of which increases with
decreasing value of the progression level k, so that the initial
image of the progression of images is most transparent, and the
final image of the progression of images is least transparent;
[0049] InputPixelValue(i, j, k) is the tonal value of a given image
pixel at pixel location (i, j) of the kth progression of the
associated progressive image; [0050] OutputPixelValue(i, j, k) is
the transformed tonal value of the corresponding pixel at pixel
location (i, j) the kth progression of the associated masked
progressive image, wherein in all cases the OutputPixelValue(i, j,
k) is clamped between 0 and the maximum range of tonal value;
[0051] HalfMax is one half the maximum of the range of tonal
values, for example 255 as a maximum value for an 8 bit tonal
value; [0052] Bias(.alpha..sup.k) is an amount to shift the
resulting tonal values, thereby increasing or decreasing overall
image brightness, which may generally be dependent upon the value
of the mask parameter .alpha..sup.k, i.e. dependent upon the level
of quality of the associated progressive image; and [0053]
.gamma.(.alpha..sup.k) is a compression factor that ranges in value
between 0 and 1, wherein a value of 1 provides for no compression
and a value of 0 provides for complete compression, thereby
reducing image contrast, and the value of the compression factor
.gamma.(.alpha..sup.k) may generally be dependent upon the value of
the mask parameter .alpha..sup.k, i.e. dependent upon the level of
quality of the associated progressive image. For example, in one
set of embodiments, the value of the compression factor
.gamma.(.alpha..sup.k) increases with increasing image quality,
i.e. with increasing value of mask parameter .alpha..sup.k, the
latter of which increases with decreasing value of the progression
level k, so that the amount of compression decreases with
increasing image quality.
[0054] For example, the transparency level T(.alpha..sup.k),
bias(.alpha..sup.k), and compression factor .gamma.(.alpha..sup.k)
parameters of the mask filter 28 may determined by either the
website proprietor/owner or the client/user, as a function of, or
for discrete levels of, the mask parameter .alpha..sup.k measure of
image quality, so as to provide for subjectively optimizing the
presentation of progressive images. More particularly, these three
parameters T(.alpha..sup.k), bias(.alpha..sup.k),
.gamma.(.alpha..sup.k) ultimately impact the visibility of the
spatial detail of the image and therefore can be tuned to mask the
perceived low quality of initial and intermediate images of a
progressively-encoded image, so that the evolving image is
perceived as one of high quality notwithstanding the obscuration of
the initial and intermediate images by the mask filter 28.
[0055] It should be understood that the mask parameter
.alpha..sup.k can be either a singular value as described
hereinabove--with the other parameters T(.alpha..sup.k),
bias(.alpha..sup.k), .gamma.(.alpha..sup.k) dependent thereupon--or
may be an array or set of parameters, for example, individual
values of the transparency level T.sup.k, bias.sup.k, compression
factor .gamma..sup.k, or one or more other imaging parameters, for
example, color balance or value.
[0056] Referring to FIGS. 2 and 3, the mask parameters
.alpha..sup.N, .alpha..sup.N-1, . . . , .alpha..sup.2,
.alpha..sup.1 may be manually set by the user 24 by simulating the
progressive delivery of the high-definition image 12, IMG.sup.0,
and then, for each level of progression, determining the associated
mask parameters .alpha..sup.N, .alpha..sup.N-1, . . . ,
.alpha..sup.2, .alpha..sup.1 of an associated mask filter or
masking algorithm as necessary to sufficiently reduce or obscure
progression related granularity in the resulting masked image.
[0057] More particularly, referring to FIG. 2, the mask-filter
design process 200 commences in step (202) with receipt of a
high-definition image 12, IMG.sup.0 to be used as a basis for
establishing the associated mask parameters .alpha..sup.N,
.alpha..sup.N-1, . . . , .alpha..sup.2, .alpha..sup.1 of the mask
filter 28. Then, in step (204), the associated progressive imaging
process 26, 26.1, 26.2 corresponding to what will ultimately be
used by the webpage 16 to progressively transform the image
high-definition image 12, IMG.sup.0, and used for the display
thereof by the internet web browser 22 on the client
internet-connected receiving device 20--is simulated to provide for
generating the plurality of associated lower-quality progressive
images IMG.sup.N, IMG.sup.N-1, IMG.sup.N-2, . . . , image
IMG.sup.2, image IMG.sup.1 associated with the high-definition
image 12, IMG.sup.0. Then, in step (206), for each of the
lower-quality progressive images IMG.sup.N, IMG.sup.N-1,
IMG.sup.N-2, . . . , image IMG.sup.2, image IMG.sup.1, the
corresponding associated mask parameter .alpha..sup.N,
.alpha..sup.N-1, .alpha..sup.N-2, . . . , .alpha..sup.2,
.alpha..sup.1 is determined by an associated mask-parameter
determination process 300, which, referring to FIG. 3, commences in
step (302) with receipt of the particular lower-quality progressive
image IMG.sup.k for which the associated mask parameter
.alpha..sup.k is to be determined. Then, in step (304), the
associated mask parameter .alpha..sup.k--or mask parameters
.alpha..sup.k for the case of .alpha..sup.k being an array of mask
parameters--are initialized to an initial value, for example, a
nominal value or set of values for the lowest-quality progressive
images IMG.sup.N, or the previously-determined value(s)
.alpha..sup.k+1 for subsequent lower-quality progressive images
IMG.sup.k<N. Then, in step (306), the lower-quality progressive
image IMG.sup.k is masked by the mask filter 28, for example, in
accordance with equations 1 and 2, so as to generate an associated
masked lower-quality progressive image IMG.sup.k', which, in step
(308), is viewed--for example, by the website proprietor 14--and
subjectively judged in step (310) by the viewer to assess whether
the associated progression-related granularity that might be
visible in the masked lower-quality progressive image IMG.sup.k' is
objectionable, or, at the other extreme, the masking has been
excessive. If the resulting masked lower-quality progressive image
IMG.sup.k' has an acceptable appearance, then, in step (312), the
mask parameter .alpha..sup.k, or mask parameters .alpha..sup.k, are
returned to the mask-filter design process 200, which then proceeds
to the next lower-quality progressive image IMG.sup.k-1. Otherwise,
from step (310), then, in step (314), the mask parameter
.alpha..sup.k, or mask parameters .alpha..sup.k, is/are either
increased in value to further reduce progression-related
granularity in the masked lower-quality progressive image
IMG.sup.k', or reduced in value to decrease the amount of masking,
and therefor provide for displaying greater detail. Then, the
mask-parameter determination process 300 is repeated beginning with
step (306), until the acceptable mask parameter .alpha..sup.k, or
mask parameters .alpha..sup.k, is/are returned in step (312).
[0058] For example, in accordance with a first set of embodiments,
the mask parameters .alpha..sup.N, .alpha..sup.N-1, . . . ,
.alpha..sup.2, .alpha..sup.1 provide for setting the contrast and
transparency (similarly--opacity) of the masked image, for example,
via the above-described compression factor .gamma.(.alpha..sup.k),
and transparency level T(.alpha..sup.k), respectively. These
adjustments for each level of progressive image quality are
determined using example images of a particular progressive scheme
because the perceived quality of low-quality images is dependent on
that scheme. However, such determination may be done visually, with
masking to the extent that produces a result which appears as the
high-quality image masked by a realistic medium such as a fog or
low light level. In accordance with one set of embodiments, the
values of the associated the mask parameters .alpha..sup.N,
.alpha..sup.N-1, . . . , .alpha..sup.2, .alpha..sup.1 are currently
tailored for a particular progressive imaging process 26, 26.1,
26.2, regardless of the ultimate content of the sent images, and
these mask parameters .alpha..sup.N, .alpha..sup.N-1, . . . ,
.alpha..sup.2, .alpha..sup.1 are then later sent to the client
internet-connected receiving device 20 to be automatically applied
to the associated progressively-displayed images, for example,
through instructions in the HTML code which controls how the
webpage 16 is to be drawn/presented.
[0059] Referring again to FIG. 1, in accordance with one set of
embodiments, the mask-filter design process 200 is operated by the
website proprietor 14 on the server device 18/internet webserver
18' after transmitting the high-definition image 12, IMG.sup.0
provided thereto, whereby the associated mask parameters
.alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 are then configured on the server
device 18/internet webserver 18'. Alternatively, in accordance with
another set of embodiments, the mask-filter design process 200
could be operated by the website proprietor 14 as an image
processing application on a separate computer system, for example,
on an image server device 30 such as a secondary internet webserver
30', which then transmits the resulting mask parameters
.alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 to the primary internet webserver 18'
on the primary server device 18.
[0060] The effect of the mask filter 28 is illustrated in FIGS.
4a-c, 5a-c and 6a-c for three lower-quality images IMG.sup.3,
IMG.sup.2, IMG.sup.1 of progressively higher spatial quality, shown
in FIGS. 4a, 5a and 6a respectively, that are sequentially
displayed in advance of a final, relatively-higher spatial quality
image IMG.sup.0 shown in FIGS. 4c, 5c and 6c, all exemplifying the
associated progressive-imaging process 26, 26.1, 26.2. FIGS. 4b, 5b
and 6b illustrate corresponding masked lower-quality images
IMG.sup.3', IMG.sup.2', IMG.sup.1' resulting from the application
of mask filter 28 of the image processing system 10 to mask visible
spatial detail of artifacts in the lower-quality images IMG.sup.3,
IMG.sup.2, IMG.sup.1 of FIGS. 4a, 5a and 6a, respectively, through
a reduction in contrast resulting in a compression of the image
histogram shown as an inset in each of FIGS. 4a-b, 5a-b and 6a-b.
The amount of the associated image compression is determined so as
to decrease the visibility of spatial detail in the lower-quality
images IMG.sup.3, IMG.sup.2, IMG.sup.1 to a level which not only
obscures the appearance of low spatial quality in the lower-quality
images IMG.sup.3, IMG.sup.2, IMG.sup.1 but also provide for
corresponding resulting masked lower-quality images IMG.sup.3',
IMG.sup.2', IMG.sup.1' that each reasonably appear as a final
high-resolution image obscured by a similar, though not necessarily
identical, contrast reduction, or masking, thereof, as illustrated
in FIGS. 4d, 5d and 6d, respectively. Accordingly, in one set of
embodiments, the mask parameters .alpha..sup.N, .alpha..sup.N-1, .
. . , .alpha..sup.2, .alpha..sup.1 of the mask filter 28 are
determined so that the masked lower-quality images IMG.sup.3',
IMG.sup.2', IMG.sup.' have an appearance that is similar to the
similarly-masked high-definition image 12, IMG.sup.0, so that the
progression-related artifacts in the lower-quality images
IMG.sup.3, IMG.sup.2, IMG.sup.1 are obscured, whereby the masked
lower-quality images IMG.sup.3', IMG.sup.2', IMG.sup.1' are
substantially indistinguishable from a similarly masked
high-definition image 12, IMG.sup.0, leaving an impression of the
object in the high-definition image 12, IMG.sup.0 emerging from a
fog as the image progression of the associated
progressively-displayed high-definition image 12, IMG.sup.0
progresses. The associated amount of masking (e.g. contrast
reduction) and the actual algorithm for masking each progressive
initial image is determined in advance; stored as masking
parameters on the server device 18; and thereafter sent to the
client internet-connected receiving device 20 with instructions
regarding their application prior to, or simultaneous with, the
sending of each progressively-higher-quality image so that such
masking can be applied, and resulting intermediate images
thereafter displayed, by the client internet-connected receiving
device 20. For example, such instructions and parameters, including
the masking algorithms themselves, may be included as part of the
Hypertext Markup Language (HTML) of a webpage 16 provided by a
internet webserver 18', to be processed by an internet web browser
22 on a client internet-connected device 22 . Note that FIGS. 4d,
5d and 6d are not necessarily intended as the visual goal of the
masking result of FIGS. 4b, 5b and 6b, respectively, but simply to
illustrate that masking of the lower-quality images may be easily
interpreted as a realistic obscuration of the spatial detail
inherent in the high-quality, high-definition image 12,
IMG.sup.0.
[0061] Whereas FIGS. 4a-b, 5a-b and 6a-b illustrate the processing
and display of each progressively higher quality initial image, the
comparison between a progressive image display with and without
masking is more easily seen by rearranging the images of FIGS.
4a-c, 5a-c and 6a-c into FIGS. 7a-d and 8a-d, wherein the
progression from low to high image quality without masking is shown
in FIGS. 7a through 7d, and the progression from low to high image
quality with masking is shown in FIGS. 8a through 8d. Such
comparison illustrates that whereas the unmasked versions clearly
shows initial images of unrealistically low quality, the
progressive, masked images provide a visual suppression of the low
spatial quality of such images while providing an impression of an
inherently high-resolution image gradually being revealed in a more
natural, realistic way.
[0062] For example, in the embodiments illustrated in FIGS. 4b, 4d,
5b, 5d, 6b, 6d and 8a-c, the masking was originally performed using
Corel Photo-Paint version X6 with the menu drop down item "Adjust",
by selecting "Brightness/Contrast/Intensity" to bring up a tool
window with brightness, contrast and intensity, each having a
slider that provides for selecting a value in the range of -100 to
100, with 0 being the unmodified setting, using the following,
visually-determined settings: FIG. 4b: Contrast=-93; FIG. 5b:
Contrast=-88; and FIG. 6b: Contrast=-30. The histograms illustrated
in FIGS. 4a-d, 5a-d and 6a-d correspond to the correspond to the
original images presented in U.S. Provisional Application Ser. No.
62/669,296 filed on 9 May 2018. However, the corresponding masked
images of FIGS. 4b, 4d, 5b, 5d, 6b, 6d and 8a-c of the instant
application were generated using Adobe Photoshop Version 12.0x32
using respective Legacy Brightness/Contrast levels of -80, -60 and
-40, respectively, within an available range of +/-100, so as to
provide for printable images that better illustrates the masking
process.
[0063] Referring again to FIG. 1, if the image processing system
10, 10.1 utilizes the first-aspect progressive imaging process 26,
26.1 to progressively transform and transmit the high-definition
image 12, IMG.sup.0, then the base image IMG.sup.N is initially
transmitted to the client internet-connected receiving device 20,
followed by image supplements .DELTA.(N-1,N), .DELTA.(N-2,N-1), . .
. , .DELTA.(2,3), .DELTA.(1,2), .DELTA.(0-,1) interleaved with the
associated mask parameters .alpha..sup.N, .alpha..sup.N-1,
.alpha..sup.N-2, . . . , .alpha..sup.2, .alpha..sup.1, wherein the
image supplements .DELTA.(N-1,N), .DELTA.(N-2,N-1), . . . ,
.DELTA.(2,3), .DELTA.(1,2), .DELTA.(0-,1) provide for progressively
reconstructing progressively-higher-resolution images, beginning
with the base image IMG.sup.N, and culminating with the
high-definition image 12, IMG.sup.0. Each progressively-encoded
lower-quality progressive image IMG.sup.k leading up to the
high-definition image 12, IMG.sup.0 is filtered by a mask filter 28
on the client internet-connected receiving device 20 using the
associated mask parameters .alpha..sup.k to generate an associated
masked lower-quality progressive image IMG.sup.k' that is displayed
on the client internet-connected receiving device 20, either as
determined by the client internet-connected receiving device 20, or
responsive to instructions from the server device 18, for example
via software encoded in a webpage 16 to be processed by an internet
web browser 22, for the display of an intermediate image while the
actual, unmasked, lower-quality progressive images IMG.sup.k (or
the associated data thereof) are used for progressively
reconstructing the lower-quality progressive images IMG.sup.k and
eventually the high-definition image 12, IMG.sup.0. For example, a
progressively encoded JPEG (Joint Photographic Experts Group) image
provides progressively higher quality intermediate renderings of an
image as data is received and decoded prior to presentation of the
final high-definition image 12, IMG.sup.0. Therefore, in accordance
with the first aspect of the image processing system 10, 10.1, the
client internet-connected receiving device 20 uses the mask filters
18, or associated masking algorithms, only for the display of the
masked lower-quality progressive image IMG.sup.k', whereas the
unmasked lower-quality progressive images IMG.sup.k are
successively preserved in support of the progressive JPEG
reconstruction process.
[0064] In accordance with one set of embodiments of the first
aspect image processing system 10, 10.1, given the particular
progressive image display algorithm to be used, the number of
levels of image progression (and corresponding number of
intermediate images), and the selection of corresponding associated
values for the mask parameters .alpha..sup.N, .alpha..sup.N-1, . .
. , .alpha..sup.2, .alpha..sup.1, the progressively-improved images
are generated, masked and displayed on the client
internet-connected receiving device 20 in accordance with the
process illustrated in FIG. 9, culminating with an unmasked display
of the high-definition image 12, IMG.sup.0.
[0065] More particularly, referring to FIG. 9, in accordance with a
set of embodiments of the first-aspect image processing system 10,
10.1 incorporating a first-aspect progressive imaging process 26,
26.1, a progressive image-masking and display process 900 operative
on the client internet-connected receiving device 20 provides for
receiving, from the server device 18, a base image IMG.sup.N, a
plurality of associated image supplements .DELTA.(N-1,N),
.DELTA.(N-2,N-1), . . . , .DELTA.(2,3), .DELTA.(1,2),
.DELTA.(0-,1), and a plurality of associated mask parameters
.alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 interleaved therewith, and then
generating therefrom, and displaying, a corresponding plurality of
masked lower-quality progressive images IMG.sup.N', IMG.sup.N-2', .
. . , IMG.sup.2', IMG.sup.1'--with progressively increasing
quality--culminating with an unmasked display of the
high-definition image 12, IMG.sup.0. The progressive image-masking
process 900 commences in step (902) with receipt of the base image
IMG.sup.N and the associated mask parameter .alpha..sup.N, after
which, in step (904), the corresponding masked lowest-quality
progressive image IMG.sup.N' is generated by the associated mask
filter 28 from the lowest-quality progressive image IMG.sup.N (i.e.
the base image IMG.sup.N), with an associated level of masking
controlled by the value of the mask parameter .alpha..sup.N, after
which the resulting masked lowest-quality progressive image
IMG.sup.N' is displayed. In embodiments for which the total number
N of intermediate, lower-quality progressive images IMG.sup.N is
given a priori, in step (906), a counter k may be initialized to a
value of N-1 to track the number of remaining lower-quality
progressive images IMG.sup.k to be processed in subsequent steps
(908) through (912). Then, in step (908), the next image supplement
.DELTA.(k,k+1) and an associated mask parameter .alpha..sup.k are
received form the server device 18. The image supplement
.DELTA.(k,k+1) is an image transition component that provides for
generating the next lower-quality progressive image IMG.sup.k from
the previous lower-quality progressive image IMG.sup.k+1, with the
former being of higher quality than the latter. Then, in step
(910), the next progression lower-quality progressive image
IMG.sup.k is generated responsive to both the previous
lower-quality progressive image IMG.sup.k+1 and the image
supplement .DELTA.(k,k+1), in accordance with a counterpart to the
first-aspect progressive imaging process 26, 26.1 of the server
device 18. Then, in step (912), the corresponding masked
lower-quality progressive image IMG.sup.k' is generated by the
associated mask filter 28 from the lower-quality progressive image
IMG.sup.k using the associated mask parameter .alpha..sup.k, which
provides progressively less masking for the previous masked
lower-quality progressive image IMG.sup.k+1', after which the
resulting masked lower-quality progressive image IMG.sup.k' is
displayed. Then, steps (914) and (916) respectively provide for
testing and decrementing the counter k so as to provide for
repeating steps (908) through (912) for each of the
progressively-increasing-quality lower-quality progressive images
IMG.sup.k and associated masked lower-quality progressive images
IMG.sup.k'. Then, in step (918), a final image supplement
.DELTA.(0,1) is received, from which, in step (920), the final,
high-definition image 12, IMG.sup.0 is generated and displayed,
without masking. Alternatively, instead of using a counter k and
associated steps (906), (914) and (916), the termination of
repetitions of steps (908) through (912) could be controlled by a
signal from the server device 18 indicating when to transition to
step (918).
[0066] If, instead, the image processing system 10, 10.1 utilizes
the second-aspect progressive imaging process 26, 26.2 to
progressively transform and transmit the high-definition image 12,
IMG.sup.0, the lower-quality progressive images IMG.sup.k are each
independent of one another and are each transmitted along with an
associated mask parameter .alpha..sup.k, the latter of which are
used by an associated mask filter 28 on the client
internet-connected receiving device 20 to generate the
corresponding masked lower-quality progressive image IMG.sup.k'
from the corresponding lower-quality progressive images IMG.sup.k
for display on the client internet-connected receiving device
20.
[0067] Referring to FIG. 10, in accordance with a second aspect
10.2 of the image processing system 10, 10.2, the masking of the
lower-quality progressive images IMG.sup.N, IMG.sup.N-1,
IMG.sup.N-2, . . . , IMG.sup.2, IMG.sup.1 is done either on the
server device 18 or on an associated image server device 30, and
the components of the progressive image are each transmitted from
the server device 18, or the associated image server device 30, to
the client internet-connected receiving device 20 as masked
lower-quality progressive images IMG.sup.N', IMG.sup.N-1',
IMG.sup.N-2', . . . , IMG.sup.2', IMG.sup.1', culminating with the
high-definition image 12, IMG.sup.0, each of which are displayed
directly by the client internet-connected receiving device 20,
thereby freeing the client internet-connected receiving device 20
from further processing of the images. In accordance with one set
of embodiments, the mask parameters .alpha..sup.N, .alpha..sup.N-1,
.alpha..sup.N-2, . . . , .alpha..sup.2, .alpha..sup.1 are
determined in the same manner as described hereinabove for the
first-aspect image processing system 10,10.1, i.e. in accordance
with the mask-filter design process 200 operating on either the
server device 18 or an associated image server device 30, 30', for
the particular associated progressive imaging process 26, 26.1,
26.2, except that the associated masked lower-quality progressive
images IMG.sup.N', IMG.sup.N-1', IMG.sup.N-2', . . . , IMG.sup.2',
IMG.sup.1' generated in step (308) are saved for eventual
transmission to the client internet-connected receiving device 20.
Alternatively, the mask parameters .alpha..sup.N, .alpha..sup.N-1,
.alpha..sup.N-2, . . . , .alpha..sup.2, .alpha..sup.1 alone may be
determined by the mask-filter design process 200 on the image
server device 30 and then transmitted to the server device 18 along
with the high-definition image 12, IMG.sup.0, the latter of which
is then progressively encoded and then masked by an associated
progressive image-masking process 1100 on the server device 18 for
eventual transmission to the client internet-connected receiving
device 20.
[0068] More particularly, referring to FIG. 11, beginning with step
(1102) of the progressive image-masking process 1100, for each
lower-quality image component k of the associated progressively
generated image, generated from the high-definition image 12,
IMG.sup.0 received from the website proprietor 14, in step (1104),
the next progressive image component IMG.sup.k is generated by the
associated progressive imaging process 26, 26.1, 26.2 and the
associated mask parameter .alpha..sup.N is received from the
website proprietor 14, after which, in step (1106), the
corresponding masked lower-quality progressive image IMG.sup.k' is
generated from the corresponding lower-quality progressive image
IMG.sup.k by the mask filter 28 using the given value of the mask
parameter .alpha..sup.N, after which, in step (1108), the masked
lower-quality progressive image IMG.sup.k' is saved for eventual
transmission to the client internet-connected receiving device 20.
From step (1110), steps (1104) through (1108) are repeated for each
lower-quality image component k, after which, in step (1112), the
masked lower-quality progressive images IMG.sup.k' are returned for
eventual transmission to the client internet-connected receiving
device 20.
[0069] Alternatively, the lower-quality progressive images
IMG.sup.N, IMG.sup.N-1, IMG.sup.N-2, . . . , IMG.sup.2, IMG.sup.1
associated with the high-definition image 12, IMG.sup.0 may be
masked to lower quality place holder images before they are
transmitted to a client internet-connected receiving device 20,
such as a computer with an internet web browser 22, since such
images are typically discarded as higher quality images become
available. In such cases, due to a reduction in the image
complexity as a result of such masking, such image processing will
likely result in greater compression efficiency and therefore
faster transmission, without requiring subsequent masking by the
client internet-connected receiving device 20.
[0070] Referring to FIG. 12, in accordance with the second-aspect
image processing system 10, 10.2, an associated
masked-progressive-image display process 1200 provides for
displaying fully-formed masked lower-quality progressive images
IMG.sup.N', IMG.sup.N-1', IMG.sup.N-2', . . . , IMG.sup.2',
IMG.sup.1' of progressively-increasing quality from the server
device 18, culminating with receipt and display of the unmasked
high-definition image 12, IMG.sup.0. The masked base image
IMG.sup.N' is first received and displayed in step (1202). Then,
for an image component counter k--the value of which is initialized
in step (1204) to one less than the total number of image
components--for each of the masked lower-quality progressive images
IMG.sup.N', IMG.sup.N-1', IMG.sup.N-2', . . . , IMG.sup.2',
IMG.sup.1', the next masked lower-quality progressive images
IMG.sup.k' is received in step (1206) and displayed in step (1208),
after which, in step (1210), if all masked lower-quality
progressive images IMG.sup.k' have not been displayed, then, in
step (1212), the image component counter k is decremented, and the
process 1200 repeats beginning with step (1206). Otherwise, from
step (1210), after all the lower-quality image components have been
received and displayed, the unmasked high-definition image 12,
IMG.sup.0 is received in step (1214) and then displayed in step
(1216).
[0071] The amount of spatial detail masking for the best
progressive image presentation can be largely subjective. The mask
filter or filtering algorithm provides for sufficiently masking the
appearance of spatial detail so that the progressive process
appears as a gradual removal of high spatial detail masking rather
than a gradual increase in spatial quality itself. However, such
masking necessarily diminishes the detail in the lower resolution
images that may help the viewer more rapidly assimilate the content
the entire purpose of progressive imaging. So while a certain
degree of masking can be applied to initial images to minimize the
impression that the image is indeed one of lower quality, such
masking would be weighed against the benefit of additional albeit
low resolution detail. Accordingly, the relative subjectiveness of
the ideal amount of masking suggests that general masking
parameters, such as the amount of contrast modification, are
somewhat flexible. To make the application of such masking simpler,
such parameters may therefore be generally assigned based upon the
actual native resolution, or inherent relative quality between a
given low resolution image and the final image, rather than the
actual content of those images, making such parameters a simple
function of the given progressive imaging approach as modified by
the inclinations of those tasked with determining such best
parameters, rather than requiring an analysis of each image.
[0072] In accordance with another aspect of the image processing
system 10, one may further artificially enhance the lower
resolution image prior to the application of the masking process to
bring out the appearance of higher spatial quality detail, as long
as such enhancement does not simultaneously enhance the appearance
of low spatial quality. Such enhancement may include, but is not
limited to, artificial emphasis of edge structures in the image.
Whereas such enhancement may be taken to an excessive degree when
treating unmasked images of low spatial quality, thereby increasing
an artificial appearance, the associated masking process will also
decrease the visibility of such enhancement, allowing for more
aggressive application of the enhancement prior to masking.
[0073] The progressive reduction of applied masking with
progressive improvement in the inherent spatial image quality
provides the effect of revealing an image which seems like it had
always possessed high spatial quality, with the visibility of that
high spatial quality being obscured by the masking. Yet another
interpretation is that the high-quality image actually fades in, in
front of the mask, and ultimately obscures the mask, making the
mask gradually disappear while the high-quality image gradually
appears. Accordingly, the characteristics of the masking process
may include not only a reduction in the visibility of the
low-quality image spatial detail, but may also include
characteristics of a background image by blending the masked
lower-quality image with the background image. As a very simple
example, the masked low-quality image may include a degree of
transparency, whether global or as a pixel level image transparency
component, which is initially high and progressively decreases as
the image quality progressively increases and as the contrast
restriction progressively decreases, providing the effect of the
high-quality image gradually appearing in front of such a
background. As a similar example, the masked low-quality image may
include progressive changes in brightness, such as may naturally
and realistically occur when either gradually turning on a light,
or even turning it down to improve visual contrast. This is
particularly relevant when the background of a display is white,
such as is the case with many internet websites, and the desired
effect is to have an image progressively appear from the
background.
[0074] As a further example, in accordance with a second aspect of
a masking process, the intermediate images of FIGS. 4a, 5a and 6a
are masked to both compress and shift their tonal values to create
the respective masked lower-quality progressive images IMG.sup.3'',
IMG.sup.2'', IMG.sup.1'' of FIGS. 13a, 13b and 13c, respectively,
culminating with the final, unmasked high-definition image 12,
IMG.sup.0 illustrated in FIG. 13d, all such images including their
respective tonal histograms. The progressive images resulting from
this second aspect of the masking process therefore provide a
similar effect to the first aspect of the masking
process--affecting contrast alone--in that the image offers the
presumption or illusion that high spatial quality exists but, in
this case, is being masked by a generally white mask which
progressively fades to reveal the final image. However, since the
generally white mask may initially create the same appearance as
the white page background before the first, masked initial
low-quality image appears (such a completely masked, uniform white
image not shown), the high resolution image appears to be
progressively revealed from the page as if fading in from that
background, while also providing the illusion that presumably high
spatial detail is simply being revealed as part of that
process.
[0075] The utility of the masking process of the image processing
system 10 fundamentally relies on the fact that a reduction in
contrast makes spatial details in an image more difficult to see.
In its most basic form, such a reduction can be either a linear or
a non-linear compression, or even truncation of the tonal values of
the image histogram. In fact, while brightness changes simply shift
image tonal histogram rather than compress it, such shifting can
ultimately act to also limit the histogram once existing, but
shifted, tonal values reach either the maximum or minimum allowable
levels, which is why brightness can ultimately be increased until
the only remaining tonal level is the maximum (i.e. a uniformly
bright image), or decreased until the only remaining tonal level is
zero (i.e. a uniformly dark image), in both cases resulting in no
remaining visible spatial detail. A similar explanation applies to
changes in transparency, because the displayed pixel values are
often a simple weighted averaging of foreground and background
images. If the background image is completely white, then the tonal
histogram of the composite image will be both compressed and
shifted to white to a degree determined by the transparency. If the
background image is completely black, then the tonal histogram of
the composite image will be both compressed and shifted to black,
again to a degree determined by the transparency. In both cases,
since the tonal histogram of the displayed, composite image has
been compressed, the spatial detail of the foreground image will be
more difficult to see. Therefore the associated masking process of
the image processing system 10 may combine a number of methods
which effectively modify the visibility of spatial detail, so as to
appear as an obscuration of that detail, without changing spatial
detail itself, but rather, changing the visibility of that spatial
detail through adjustment of the tonal values of the displayed
image.
[0076] In general, the mask filters or masking algorithms may be
applied to progressive images either in accordance with
predetermined instructions given by a server device 18 to a client
internet-connected receiving device 20, such as through the
software encoding included in, or accessed by, a webpage 16, or may
be applied independently by a client internet-connected receiving
device 20 upon detection of a progressive image without such
instruction. For example, progressive JPEG images are inherently
encoded with, and detectable as, providing data for displaying
progressively higher quality images, and a server device 18 may
include instructions for the resultant masking of such progressive
JPEG images. However, in the absence of such instructions, and upon
such detection, a client internet-connected receiving device 20 may
independently apply masking in accordance with general masking
settings resident on the client internet-connected receiving device
20, such as settings within an internet browser application, which,
for example, show the initial images of progressive JPEG images
with higher transparency and then decreasing such transparency as
the quality improves. Progressive JPEG images, while often
requiring less bandwidth than non-progressive JPEG images, are at
present little used in websites, presumably due to the low and
artificial initial spatial quality as well as the difficulty in
visually determining when the final image is achieved. The
application of progressively-decreasing masking while image quality
progressively increases may serve to mitigate these issues.
[0077] Accordingly, the masking of intermediate images associated
with a progressive display of a high-definition image provides for
significantly diminishing the often objectionable, obvious low or
artificial visual spatial quality of the initial and intermediate
images, while still providing sufficient, progressively improving
image detail to accelerate the viewer's assimilation of the content
of that image.
[0078] Referring to FIG. 14, in accordance with either the first
10.1 or second 10.2 aspects of the image processing system 10,
10.1, 10.1', 10.1'', 10.2, a method 1400 of processing and
providing a progressively-encoded image--by a server device 18
acting as an image server--commences in step (1402) with receipt of
a high-definition image 12, IMG.sup.0, for example, from a website
proprietor 14. Then, in step (1404), the associated mask parameters
.alpha..sup.N, .alpha..sup.N-1, . . . , .alpha..sup.2,
.alpha..sup.1 are configured, for example, using the associated
mask-filter design process 200 that is run on either the server
device 18, or on a separate image server device 30.
[0079] Then, in step (1406), if each progressively-high-quality
image component IMG.sup.k-1 requires data from a
previous-lower-quality image component IMG.sup.k, i.e. as a result
of progressive-encoding by an associated first-aspect progressive
imaging process 26, 26.1, then, in step (1408), the server device
18 awaits a demand for an image from an associated client
internet-connected receiving device 20, and upon receipt thereof,
in step (1410), sends to the client internet-connected receiving
device 20 the lowest-quality, base image IMG.sup.N, following by
the associated image supplements .DELTA.(N-1,N), .DELTA.(N-2,N-1),
. . . , .DELTA.(2,3), .DELTA.(1,2), .DELTA.(0-,1) and, interleaved
therewith, the associated mask parameters .alpha..sup.N,
.alpha..sup.N-1, .alpha..sup.N-2, . . . , .alpha..sup.2,
.alpha..sup.1, together with instructions to apply the mask
parameters .alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 to the display of corresponding
progressively-high-quality image component IMG.sup.k-1 as data of
the image supplements .DELTA.(N-1,N), .DELTA.(N-2,N-1), . . . ,
.DELTA.(2,3), .DELTA.(1,2), .DELTA.(0-,1) and mask parameters
.alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 is received, in accordance with the
process 100 schematically illustrated in FIG. 1.
[0080] Otherwise, from step (1406), if the progressively-encoded
images IMG.sup.k are generated in accordance with the second-aspect
progressive-imaging process 26, 26.2, then, in step (1412), each
lower-quality progressive images IMG.sup.N, IMG.sup.N-1,
IMG.sup.N-2, . . . , IMG.sup.2, IMG.sup.1 is masked by a mask
filter 28 using a corresponding associated mask parameter
.alpha..sup.N, .alpha..sup.N-1, .alpha..sup.N-2, . . . ,
.alpha..sup.2, .alpha..sup.1 wherein the masking is done during the
mask-filter design process 200, or by a subsequent progressive
image-masking process 1100. Then, in step (1414), the server device
18 awaits a demand for an image from an associated client
internet-connected receiving device 20, and upon receipt thereof,
in step (1416), successively sends to the client internet-connected
receiving device 20 each masked lower-quality progressive image
IMG.sup.N', IMG.sup.N-1', IMG.sup.N-2', . . . , IMG.sup.2',
IMG.sup.1'--in a succession of progressively-increasing image
quality--following by the original high-definition image 12,
IMG.sup.0, for example, in accordance with the process 1000
schematically illustrated in FIG. 10.
[0081] Referring to FIG. 15, in accordance with either the first
10.1 or second 10.2 aspects of the image processing system 10,
10.1, 10.1', 10.1'', 10.2, a method 1500 of processing and
displaying a masked progressively-encoded image--by a client
internet-connected receiving device 20 commences with receipt of an
image in step (1502). If, in step (1504), the received image
includes masking and display instructions that provide for treating
the image as a series of progressively-improving-quality images,
then, in step (1506), if the image does not contain instructions to
replace each of the progressively-improving-quality images with a
progressively-higher-quality masked image as data is received, i.e.
in accordance with a first-aspect image processing system 10, 10.1,
10.1', 10.1'', also referred to as a first option, in step (1508),
a base image IMG.sup.N is received. Then, in step (1510), if the
current image is not the final high-definition image 12, IMG.sup.0,
then in step (1512), if the current, received image is not an image
supplements .DELTA.(k,k+1), then the current, received image is
masked by the mask filter 28 using the associated mask parameter
.alpha..sup.k and then displayed. If the current, received image is
an image supplements .DELTA.(k,k+1)--in accordance with a
first-aspect progressive imaging process 26, 26.1--then the
corresponding lower-quality progressive images IMG.sup.k-1 is first
formed therefrom based upon the previous lower-quality progressive
images IMG.sup.k before masking and display. Then, in step (1514),
depending upon the progressive imaging process 26, 26.1, 26.2, the
next image supplements .DELTA.(k,k+1) or image IMG.sup.k is
received, along with the associated mask parameter .alpha..sup.k,
after which, the process repeats beginning with step (1510).
Otherwise, from step (1510), if the final level of progress has
been reached, then, in step (1516), the high-definition image 12,
IMG.sup.0 is displayed, without masking.
[0082] Otherwise, from step (1506), if the image contains
instructions to replace each of the progressively-improving-quality
images with a progressively-higher-quality masked image as data is
received, i.e. in accordance with a second-aspect image processing
system 10, 10.2, also referred to as a second option, in step
(1518), the masked lower-quality progressive images IMG.sup.N',
IMG.sup.N-1', IMG.sup.N-2', . . . , IMG.sup.2', IMG.sup.1',
followed by the high-definition image 12, IMG.sup.0, are received
and progressively displayed in accordance with the
masked-progressive-image display process 1200 illustrated in FIG.
12.
[0083] Otherwise, from step (1504), if the received image does not
include masking and display instructions that provide for treating
the image as a series of progressively-improving-quality images,
then, in step (1520), if a progressively-encoded image has been
received, but without display instructions, then, in accordance
with a third aspect 10.3 of an image processing system 10, 10.3,
also referred to as a third option, in step (1522), the number of
progression levels is determined, and then successively decreasing
levels of masking, for example, successively decreasing levels of
transparency, are applied to each successive image of the
progression, per setting on the client internet-connected receiving
device 20, for example, predetermined settings, i.e. predetermined
values for the associated mask parameters .alpha..sup.N,
.alpha..sup.N-1, .alpha..sup.N-2, . . . , .alpha..sup.2,
.alpha..sup.1. Otherwise, from step (1520), in step (1524), the
image is displayed normally, without masking.
[0084] While specific embodiments have been described in detail in
the foregoing detailed description and illustrated in the
accompanying drawings, those with ordinary skill in the art will
appreciate that various modifications and alternatives to those
details could be developed in light of the overall teachings of the
disclosure. It should be understood, that any reference herein to
the term "or" is intended to mean an "inclusive or" or what is also
known as a "logical OR", wherein when used as a logic statement,
the expression "A or B" is true if either A or B is true, or if
both A and B are true, and when used as a list of elements, the
expression "A, B or C" is intended to include all combinations of
the elements recited in the expression, for example, any of the
elements selected from the group consisting of A, B, C, (A, B), (A,
C), (B, C), and (A, B, C); and so on if additional elements are
listed. Furthermore, it should also be understood that the
indefinite articles "a" or "an", and the corresponding associated
definite articles "the` or "said", are each intended to mean one or
more unless otherwise stated, implied, or physically impossible.
Yet further, it should be understood that the expressions "at least
one of A and B, etc.", "at least one of A or B, etc.", "selected
from A and B, etc." and "selected from A or B, etc." are each
intended to mean either any recited element individually or any
combination of two or more elements, for example, any of the
elements from the group consisting of "A", "B", and "A AND B
together", etc. Yet further, it should be understood that the
expressions "one of A and B, etc." and "one of A or B, etc." are
each intended to mean any of the recited elements individually
alone, for example, either A alone or B alone, etc., but not A AND
B together. Furthermore, it should also be understood that unless
indicated otherwise or unless physically impossible, that the
above-described embodiments and aspects can be used in combination
with one another and are not mutually exclusive. Accordingly, the
particular arrangements disclosed are meant to be illustrative only
and not limiting as to the scope of the invention, which is to be
given the full breadth of the appended claims, and any and all
equivalents thereof.
* * * * *