U.S. patent application number 10/530531 was filed with the patent office on 2006-06-29 for method and system for measuring video image degradations introduced by an encoding system with throughput reduction.
Invention is credited to Jamal Baina, Nathalie Montard.
Application Number | 20060140266 10/530531 |
Document ID | / |
Family ID | 32187643 |
Filed Date | 2006-06-29 |
United States Patent
Application |
20060140266 |
Kind Code |
A1 |
Montard; Nathalie ; et
al. |
June 29, 2006 |
Method and system for measuring video image degradations introduced
by an encoding system with throughput reduction
Abstract
A method of measuring the degradations in a digitized image,
which are introduced during the encoding of the image is provided.
The method consists in: retrieving the block decomposition of the
image used during the encoding thereof; offsetting the coding grid
in relation to the image such as to define an analysis block
decomposition of the image, whereby each analysis block covers a
boundary between two adjacent blocks; applying a transform
calculation to the pixel data of the image which is encoded using
the offset coding grid; extracting the transformed coefficients
obtained for each analysis block, coefficients which are likely to
be affected by a block effect; applying an inverse transform
calculation to the extracted transformed coefficients in order to
determine the pixel data of each analysis block; and, for each
analysis block, calculating a degradation indicator and,
subsequently, the degradation of the image from the degradation
indicators for each analysis block.
Inventors: |
Montard; Nathalie; (Metz,
FR) ; Baina; Jamal; (Champigneule, FR) |
Correspondence
Address: |
BACHMAN & LAPOINTE, P.C.
900 CHAPEL STREET
SUITE 1201
NEW HAVEN
CT
06510
US
|
Family ID: |
32187643 |
Appl. No.: |
10/530531 |
Filed: |
October 22, 2003 |
PCT Filed: |
October 22, 2003 |
PCT NO: |
PCT/FR03/03132 |
371 Date: |
November 17, 2005 |
Current U.S.
Class: |
375/240.2 ;
348/E17.003; 375/240.24; 375/E7.134; 375/E7.145; 375/E7.161;
375/E7.167; 375/E7.176; 375/E7.181; 375/E7.19; 375/E7.206;
375/E7.211; 375/E7.279 |
Current CPC
Class: |
H04N 19/61 20141101;
H04N 19/115 20141101; H04N 19/132 20141101; H04N 19/154 20141101;
H04N 19/172 20141101; H04N 17/004 20130101; H04N 19/86 20141101;
H04N 19/89 20141101; H04N 19/136 20141101; H04N 19/90 20141101;
H04N 19/176 20141101 |
Class at
Publication: |
375/240.2 ;
375/240.24 |
International
Class: |
H04N 11/04 20060101
H04N011/04; H04B 1/66 20060101 H04B001/66; H04N 7/12 20060101
H04N007/12; H04N 11/02 20060101 H04N011/02 |
Foreign Application Data
Date |
Code |
Application Number |
Nov 15, 2002 |
FR |
02/14329 |
Claims
1. A method for measuring degradations of a digitized image
introduced when coding the image, said method comprising the steps
of: dividing the image into coding blocks using a coding grid,
applying a coding processing on pixel data in each said block,
making use of a block transform calculation and an inverse block
transform calculation, determining the coding grid of the coded
image, in order to find an image division into coding blocks, used
when coding the image, shifting the coding grid with respect to the
coded image, so as to define an image division into analysis blocks
each covering a boundary between two adjacent coding blocks,
applying the block transform calculation to pixel data in the coded
image using the shifted coding grid to obtain transformed
coefficients for each analysis block defined by the shifted coding
grid, extracting coefficients that could be affected by a block
effect resulting from coding of the image, from the transformed
coefficients, applying the inverse block transform calculation to
the extracted transformed coefficients to determine the pixel data
for each analysis block, for each analysis block, estimating an
indicator of the degradation due to block effects, using pixel data
in the coded image and pixel data in each analysis block, obtained
by the inverse transform calculation, and determining an image
degradation measurement by summing the degradation indicators of
each analysis block.
2. The method according to claim 1, wherein the estimation of a
degradation indicator for each analysis block comprises the steps
of: calculating an average of inter pixel differences at the inter
block boundary of the coding grid, covered by the analysis block,
using pixel data obtained for the analysis block, calculating an
average and a standard deviation applicable to pixels in the two
adjacent blocks on the coding grid, partially covered by the
analysis block, calculating a weighting factor as a function of the
average and the standard deviation obtained for the analysis block,
and calculating a spatial activity of the analysis block using
spatial activities determined for each of the two adjacent blocks
in the coding grid partially covered by the analysis block, the
analysis block degradation indicator being determined as a function
of the calculated average of inter pixel differences, the weighting
factor and the spatial activity of the block.
3. The method according to claim 2, wherein the analysis block
degradation indicator .upsilon..sub.i,j is obtained using the
following formula: .upsilon. i , j = .DELTA. .times. .times. I i ,
j .times. w i , j 1 + .psi. .times. ACT i , j ##EQU12## in which
.DELTA.I.sub.i,j is the average of inter pixel differences at the
inter block boundary of the coding grid covered by the analysis
block, w.sub.i,j is the weighting factor, .PSI. is a predefined
constant, and ACT.sub.i,j is the spatial activity of the analysis
block.
4. The method according to either claim 2, wherein the transform
calculation is applied to coding blocks of the coded image, the
spatial activities determined for each of the two coding blocks
being obtained from the transformed coefficients for each of the
two coding blocks.
5. The method according to claim 4, wherein the spatial activities
determined for each of the two coding blocks are obtained from the
following formulas: ACT i , j G = 1 1 + DC i , j G .times. u , v =
0 u + v .noteq. 0 7 .times. [ AC i , j G .function. ( u , v )
.times. Nill .function. ( u , v ) ] 2 ##EQU13## ACT i , j D = 1 1 +
DC i , j D .times. u , v = 0 u + v .noteq. 0 7 .times. [ AC i , j D
.function. ( u , v ) .times. Nill .function. ( u , v ) ] 2
##EQU13.2## in which DC.sub.i,j.sup.G and AC.sub.i,j.sup.G(u, v)
and DC.sub.i,j.sup.D and AC.sub.i,j.sup.D (u, v) are the
transformed coefficients for each of the two adjacent coding blocks
partially covered by the analysis block, and Nill(u,v) is a masking
function modelling masking by neighbourhood.
6. The method according to claim 2, wherein the average and
standard deviation calculated for each analysis block are
determined from transformed coefficients for each of the two
adjacent coding blocks partially covered by the analysis block.
7. The method according to claim 2, wherein the weighting factor is
obtained by the following formula: w i , j .function. ( .mu. i , j
, .sigma. i , j , .zeta. ) = { .lamda. .times. .times. ln
.function. ( 1 + .mu. i , j 1 + .sigma. i , j ) .times. .times. if
.times. .times. .mu. i , j .ltoreq. .zeta. ln .function. ( 1 + 255
- .zeta. 1 + .sigma. i , j ) .times. .times. else , .times. .times.
in .times. .times. which .times. .times. .lamda. = ln .function. (
1 + 255 - .zeta. 1 + .sigma. i , j ) ln .function. ( 1 + .zeta. 1 +
.sigma. i , j ) ##EQU14## .mu..sub.i,j and .sigma..sub.i,j are the
average and standard deviation respectively calculated for each
analysis block and .zeta. is a parameter corresponding to the
maximum sensitivity of the human eye.
8. The method according to one of claim 1, further comprising a
prior selection step of selection of analysis blocks that could
contain a block effect before estimating a degradation indicator
for each analysis block.
9. The method according to claim 8, wherein the prior selection
step comprises a step consisting of separating analysis blocks for
which the extracted transformed coefficients are greater than a
predetermined threshold.
10. The method according to claim 8, wherein the prior selection
step comprises a step consisting of selecting analysis blocks with
pixels with an energy representing a significant proportion of the
energy of the block, at the inter block boundary of the coding grid
covered by the analysis block.
11. The method according to claim 1, wherein the coding grid is
shifted horizontally with respect to the coded image.
12. The method according to claim 1, wherein the coding grid is
shifted vertically with respect to the coded image.
13. The method according to claim 1, wherein the block transform
calculation is a discrete cosine transform calculation.
14. A system for measuring degradations of a digitized image
introduced when coding the image, said system comprising
calculation means for: dividing the image into coding blocks using
a coding grid, applying a coding processing on pixel data in each
block, making use of a block transform calculation and an inverse
block transform calculation, determining the coding grid of the
coded image, in order to find the image division into coding
blocks, used when coding the image, shifting the coding grid with
respect to the coded image, so as to define an image division into
analysis blocks each covering a boundary between two adjacent
coding blocks, applying the block transform calculation to pixel
data in the coded image using the shifted coding grid to obtain
transformed coefficients for each analysis block defined by the
shifted coding grid, extracting coefficients that could be affected
by a block effect resulting from coding of the image, from the
transformed coefficients, applying the inverse block transform
calculation to the extracted transformed coefficients to determine
the pixel data for each analysis block, for each analysis block,
estimating an indicator of the degradation due to block effects,
using pixel data in the coded image and pixel data in each analysis
block, obtained by the inverse transform calculation, and
determining an image degradation measurement by summing the
degradation indicators of each analysis block.
15. The system according to claim 14, wherein the calculation means
for estimating a degradation indicator for each analysis block
comprises means for: calculating an average of inter pixel
differences at the inter block boundary of the coding grid, covered
by the analysis block, using pixel data obtained for the analysis
block, calculating an average and a standard deviation applicable
to pixels in the two adjacent blocks on the coding grid, partially
covered by the analysis block, calculating a weighting factor as a
function of the average and the standard deviation obtained for the
analysis block, and calculating a spatial activity of the analysis
block using spatial activities determined for each of the two
adjacent blocks in the coding grid partially covered by the
analysis block, the analysis block degradation indicator being
determined as a function of the calculated average of inter pixel
differences, the weighting factor and the spatial activity of the
block.
16. The system according to claim 15, wherein the analysis block
degradation indicator .upsilon..sub.i,j is obtained using the
following formula: .upsilon. i , j = .DELTA. .times. .times. I i ,
j .times. w i , j 1 + .psi. .times. ACT i , j ##EQU15## in which
.DELTA.I.sub.i,j is the average of inter pixel differences at the
inter block boundary of the coding grid covered by the analysis
block, w.sub.i,j is the weighting factor, .PSI. is a predefined
constant, and ACT.sub.i,j is the spatial activity of the analysis
block.
17. The system according to claim 15, wherein the transform
calculation is applied to coding blocks of the coded image, the
spatial activities determined for each of the two coding blocks
being obtained from the transformed coefficients for each of the
two coding blocks.
18. The system according to claim 17, wherein the spatial
activities determined for each of the two coding blocks are
obtained from the following formulas: ACT i , j G = 1 1 + DC i , j
G .times. u , v = 0 u + v .noteq. 0 7 .times. [ AC i , j G
.function. ( u , v ) .times. Nill .function. ( u , v ) ] 2
##EQU16## ACT i , j D = 1 1 + DC i , j D .times. u , v = 0 u + v
.noteq. 0 7 .times. [ AC i , j D .function. ( u , v ) .times. Nill
.function. ( u , v ) ] 2 ##EQU16.2## in which DC.sub.i,j.sup.G and
AC.sub.i,j.sup.G (u,v) and DC.sub.i,j.sup.D and AC.sub.i,j.sup.D
(u, v) are the transformed coefficients for each of the two
adjacent coding blocks (partially covered by the analysis block,
and Nill(u,v) is a masking function modelling masking by
neighbourhood.
19. The system according to claim 15, wherein the average and
standard deviation calculated for each analysis block are
determined from transformed coefficients for each of the two
adjacent coding blocks partially covered by the analysis block.
20. The system according to claim 15, wherein the weighting factor
w.sub.i,j is obtained by the following formula: w i , j .function.
( .mu. i , j , .sigma. i , j , .zeta. ) = { .lamda. .times. .times.
ln .function. ( 1 + .mu. i , j 1 + .sigma. i , j ) .times. .times.
if .times. .times. .mu. i , j .ltoreq. .zeta. ln .function. ( 1 +
255 - .zeta. 1 + .sigma. i , j ) .times. .times. else , .times.
.times. in .times. .times. which .times. .times. .lamda. = ln
.function. ( 1 + 255 - .zeta. 1 + .sigma. i , j ) ln .function. ( 1
+ .zeta. 1 + .sigma. i , j ) ##EQU17## .mu..sub.i,j and
.sigma..sub.i,j are the average and standard deviation respectively
calculated for each analysis block and .zeta. is a parameter
corresponding to the maximum sensitivity of the human eye.
21. The system according to claim 14, further comprising prior
selection means for selecting analysis blocks that could contain a
block effect, before estimating a degradation indicator for each
analysis block.
22. The system according to claim 21, wherein the prior selection
means comprise means for separating analysis blocks for which the
extracted transformed coefficients are greater than a predetermined
threshold.
23. The system according to claim 21, wherein the prior selection
means comprise means for selecting analysis blocks with pixels with
an energy representing a significant proportion of the energy of
the block, at the inter block boundary of the coding grid covered
by the analysis block.
24. The system according to claim 14, wherein the coding grid is
shifted horizontally with respect to the coded image.
25. The system according to claim 14, wherein the coding grid is
shifted vertically with respect to the coded image.
26. The system according to claim 14, wherein the block transform
calculation is a discrete cosine transform calculation.
Description
BACKGROUND OF THE INVENTION
[0001] (1) Field of the Invention
[0002] This invention relates to a method and system for measuring
degradations of a video image introduced by a coding system with
reduction in throughput.
[0003] It is particularly but not exclusively applicable to the
domain of low throughput or very low throughput digital audiovisual
signal distribution networks, and to the domain of production of
such signals. It is particularly applicable to surveillance of the
service quality of a digital audiovisual signal broadcast
network.
[0004] (2) Prior Art
[0005] Digitizing of video signals provides a means for copying,
storing and transmitting this type of information while maintaining
a constant image quality. However, in practice, the large quantity
of information transferred by video images requires the use of
digital compression methods to reduce the binary throughput.
[0006] A compression method that is very widely used in video is
described in standard ISO/CEI 13918 MPEG2. This algorithm is said
to be of the "with losses" type since the restored image after
decoding is not identical to the original. This algorithm is based
on a division of the image into blocks and application of a
transform, for example of the discrete cosine transform type, to
the pixels in each block to obtain a frequency representation of
the luminance amplitude of pixels in the form of one coefficient
for each pixel in the block.
[0007] In order to maintain an acceptable quality for the final
television viewer, the compression algorithms take into account
perception properties of the human vision system. However,
throughput constraints imposed by transmission systems require the
application of compression ratios that have an influence on the
image quality perceived by the television viewer.
[0008] It is found that the importance of degradations caused by
coding depends both on the compression ratio and the complexity of
images. These degradations are particularly important when the
image is more complex, and particularly related to movement of
objects, brightness and texture.
[0009] The degradations that appear in the images following
application of the MPEG2 coding technique include granular errors,
deformations of contours, the appearance of so-called "exotic"
contours and block effects.
[0010] Therefore it would appear necessary to continuously evaluate
the quality of broadcast images. There are widely used subjective
evaluation methods for this purpose, that make use of human
evaluation. However, these methods are difficult to use and cannot
be used on a broadcasting network in operation in real time.
[0011] There are other so-called "with reference" methods based on
comparison of the image for which the quality is to be evaluated
with a reference image. The reference image is usually an image
that corresponds to the image to be analyzed before it is coded
and/or transmitted. This solution is not very practical, because it
requires access to one or several reference images. Furthermore if
the video image is transmitted, there is also the problem of
transporting the reference image to the place at which the image to
be analyzed is received.
[0012] Other so-called "without reference" solutions are used to
automatically analyze images without needing to make a comparison
with reference images. The efficiency and robustness of each of
these solutions lies in the method used to measure parameters
related to image quality.
[0013] Some of these solutions are based on detection of the block
effect made in the spatial domain by gradient calculations at block
boundaries. To avoid confusion between the boundary of an object in
the image with a block effect, the gradient is compared with
intra-block gradients. The block effect is detected by means of a
decision criterion applied to the behaviour of inter block and
intra block gradients.
[0014] Thus, the Rohde & Schwarz Company has developed a block
effect detection method consisting of calculating a horizontal
gradient vector for each macro block in the image, and calculating
an average of each component of the vector over the entire image.
Variations of components of this vector over time are used to bring
out components with marginal behaviour that represent block
boundaries degraded by the compression processing transform.
Detection of these marginal components provides a means for
determining a block effect detection criterion representative of
the image degradation.
[0015] This principle for calculating the gradient is also
described in patent FR 2 805 429 filed by the Applicant. This
patent application describes a method based on the combination of a
binary gradient image and a movement "pseudo vectors" image
calculated from at least two successive images. A combination of
these two images provides a means for estimating a ratio of false
contours in the image, and is then used to evaluate a quality
mark.
[0016] In patent FR 2 785 116 filed by the applicant, the gradients
calculated on the entire image to be analyzed are passed through
psychovisual filters that translate the contextual masking effect.
A ratio of boundaries of detected visible blocks is then calculated
searching for a pseudo periodicity among high value gradients, the
image quality being evaluated based on this ratio.
[0017] It is found that methods based on the calculation of
gradients apply filters to only consider a certain type of image
contents: boundaries or high frequencies. Therefore these methods
can only be used to analyze part of information contained in the
image. The result is that they have limited reliability in terms of
detection of image degradations. Furthermore, methods based on the
use of gradients to estimate the boundary or for the detection of
contours on the image are relatively sensitive to noise, which
affects the reliability of the estimate of the quality of the
intrinsic content of the image.
[0018] Furthermore, methods based on the calculation of the average
on the entire image drastically reduce the importance of
degradations located in a part of the image, which makes it
difficult to detect such local degradations and therefore affects
the reliability with which the image quality is evaluated.
[0019] Some methods allow an analysis on several successive images,
to reduce these disadvantages. Therefore these methods cannot be
used to analyze an isolated image outside the scope of the
video.
SUMMARY OF THE INVENTION
[0020] The purpose of the present invention is to eliminate these
disadvantages. This purpose is achieved by providing a method for
measuring degradations of a digitized image, introduced when coding
the image, the method consisting of dividing the image into coding
blocks using a coding grid and applying a coding processing on
pixel data in each block, making use of a block transform
calculation and an inverse block transform calculation. According
to the invention, this process includes steps of: [0021]
determining the coding grid of the coded image, in order to find
the image division into coding blocks, used when coding the image,
[0022] shifting the coding grid with respect to the coded image, so
as to define an image division into analysis blocks each covering a
boundary between two adjacent coding blocks, [0023] applying the
block transform calculation to pixel data in the coded image using
the shifted coding grid to obtain transformed coefficients for each
analysis block defined by the shifted coding grid, [0024]
extracting coefficients that could be affected by a block effect
resulting from coding of the image, from the transformed
coefficients, [0025] applying the inverse block transform
calculation to the extracted transformed coefficients to determine
the pixel data for each analysis block, [0026] for each analysis
block, estimating an indicator of the degradation due to block
effects, using pixel data in the coded image and pixel data in each
analysis block, obtained by the inverse transform calculation, and
[0027] determining an image degradation measurement by summing the
degradation indicators of each analysis block.
[0028] According to a special feature of the invention, the
estimation of a degradation indicator for each analysis block
comprises steps of: [0029] calculating an average of inter pixel
differences at the inter block boundary of the coding grid, covered
by the analysis block, using pixel data obtained for the analysis
block, [0030] calculating an average and a standard deviation
applicable to pixels in the two adjacent blocks on the coding grid,
partially covered by the analysis block, [0031] calculating a
weighting factor as a function of the average and the standard
deviation obtained for the analysis block, and [0032] calculating a
spatial activity of the analysis block using spatial activities
determined for each of the two adjacent blocks in the coding grid
partially covered by the analysis block, [0033] the analysis block
degradation indicator being determined as a function of the
calculated average of inter pixel differences, the weighting factor
and the spatial activity of the block.
[0034] Advantageously, the analysis block degradation indicator is
obtained using the following formula: .upsilon. i , j = .DELTA.
.times. .times. I i , j .times. w i , j 1 + .psi. .times. ACT i , j
##EQU1## in which .DELTA.I.sub.i,j is the average of inter pixel
differences at the inter block boundary of the coding grid covered
by the analysis block, w.sub.i,j is the weighting factor, .PSI. is
a predefined constant, and ACT.sub.i,j is the spatial activity of
the analysis block.
[0035] According to another special feature of the invention, the
transform calculation is applied to coding blocks of the coded
image, the spatial activities determined for each of the two coding
blocks being obtained from the transformed coefficients for each of
the two coding blocks.
[0036] Advantageously, the spatial activities determined for each
of the two coding blocks are obtained from the following formulas:
ACT i , j G = 1 1 + DC i , j G .times. u , v = 0 u + v .noteq. 0 7
.times. [ AC i , j G .function. ( u , v ) .times. Nill .function. (
u , v ) ] 2 ##EQU2## ACT i , j D = 1 1 + DC i , j D .times. u , v =
0 u + v .noteq. 0 7 .times. [ AC i , j D .function. ( u , v )
.times. Nill .function. ( u , v ) ] 2 ##EQU2.2## in which
DC.sub.i,j.sup.G and AC.sub.i,j.sup.G (u,v) and D.sub.i,j.sup.D and
AC.sub.i,j.sup.D(u,v) are the transformed coefficients for each of
the two adjacent coding blocks partially covered by the analysis
block, and Nill(u,v) is a masking function modelling masking by
neighbourhood.
[0037] Preferably, the average and standard deviation calculated
for each analysis block are determined from transformed
coefficients for each of the two adjacent coding blocks partially
covered by the analysis block.
[0038] According to another special feature of the invention, the
weighting factor is obtained by the following formula: w i , j
.function. ( .mu. i , j , .sigma. i , j , .zeta. ) = { .lamda.
.times. .times. ln .function. ( 1 + .mu. i , j 1 + .sigma. i , j )
.times. .times. if .times. .times. .mu. i , j .ltoreq. .zeta. ln
.function. ( 1 + 255 - .zeta. 1 + .sigma. i , j ) .times. .times.
else , .times. .times. in .times. .times. which .times. .times.
.lamda. = ln .function. ( 1 + 255 - .zeta. 1 + .sigma. i , j ) ln
.function. ( 1 + .zeta. 1 + .sigma. i , j ) ##EQU3## .mu..sub.i,j
and .sigma..sub.i,j being the average and standard deviation
respectively calculated for each analysis block and .zeta. being a
parameter corresponding to the maximum sensitivity of the human
eye.
[0039] According to another special feature of the invention,
analysis blocks that could contain a block effect are selected
before estimating a degradation indicator for each analysis
block.
[0040] Advantageously, the prior selection comprises a step
consisting of separating analysis blocks for which the extracted
transformed coefficients are greater than a predetermined
threshold.
[0041] Preferably, the prior selection comprises a step consisting
of selecting analysis blocks with pixels with an energy
representing a significant proportion of the energy of the block,
at the inter block boundary of the coding grid covered by the
analysis block.
[0042] According to another special feature of the invention, the
coding grid is shifted horizontally with respect to the coded
image.
[0043] Alternately, the coding grid is shifted vertically with
respect to the coded image.
[0044] Preferably, the block transform calculation is a discrete
cosine transform calculation.
[0045] The invention also relates to a system for measuring
degradations of a digitized image introduced when coding the image,
the system comprising calculation means for implementing the method
defined above.
BRIEF DESCRIPTION OF THE DRAWINGS
[0046] One preferred embodiment of the invention will be described
below as a non-limitative example with reference to the appended
figures, wherein:
[0047] FIG. 1 diagrammatically represents the measurement system
according to the invention, integrated into an image processing
system;
[0048] FIG. 2 shows the measurement system represented in FIG. 1 in
more detail;
[0049] FIG. 3 shows two image pixel blocks to illustrate the method
according to the invention,
[0050] FIG. 4 shows part of the system shown in FIG. 2 in more
detail;
[0051] FIG. 5 shows a curve representing a weighting function used
by the method according to the invention;
[0052] FIG. 6 shows other curves illustrating another weighting
function used by the method according to the invention;
[0053] FIG. 7 shows curves illustrating the variation in
degradation measurements obtained using the method according to the
invention applied to a sequence of images.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S)
[0054] FIG. 1 represents a degradation measurement system 1
according to the invention that is designed for use as a probe that
could be applied to any point in a digital image processing system
such as broadcasting system or a video signal production
system.
[0055] The measurement system 1 is designed particularly to measure
the quality of images output from a coding system 10 with reduced
throughput respecting the MPEG 2 standard. For this purpose, it is
based on a frequency and time filtering combination.
[0056] This system is applicable whenever there is a need to
identify coding defects in a digital video signal, particularly to
determine the appropriate throughput for a given image sequence as
a function of the expected quality.
[0057] The coding principle according to the MPEG 2 standard
consists of dividing the digitized image into blocks of N.times.N
pixels (for example where N is equal to 8) using a coding grid, and
applying a discrete cosine transform (DCT) to each block to change
from the spatial domain to the frequency domain, and then to cancel
some components of the transform corresponding to high frequencies
before applying the inverse DCT transformation to change back to
the spatial domain, in other words so as to recover the pixels from
the corresponding block.
[0058] The discrete cosine transform processing consists of making
a calculation of transform coefficients AC and DC obtained from the
following formulas, to each block B.sub.i,j of N.times.N and pixels
in the image: DC i , j = F i , j .function. ( 0 , 0 ) = 2 N 2
.times. x = 0 N - 1 .times. y = 0 N - 1 .times. f i , j .function.
( x , y ) .times. .times. and ( 1 ) AC i , j .function. ( u , v ) =
F i , j .function. ( u , v ) , where .times. .times. u + v .noteq.
0 .times. .times. in .times. .times. which ( 2 ) F i , j .function.
( u , v ) = 4 N 2 .times. x = 0 N - 1 .times. y = 0 N - 1 .times. f
i , j .function. ( x , y ) .times. cos .function. ( .pi. .times.
.times. u .function. ( 2 .times. x + 1 2 .times. N ) ) .times. cos
.function. ( .pi. .times. .times. v .function. ( 2 .times. y + 1 2
.times. N ) ) , ( 3 ) ##EQU4##
[0059] f.sub.i,j(x,y) represents the luminance of the pixel at
point (x,y) in block B.sub.i,j, x and y being the horizontal
position index and the vertical position index respectively of the
pixel in the block B.sub.i,j, and u and v are between 1 and N-1 and
represent the indexes of the horizontal and vertical spatial
frequencies respectively, and c(0)=1/ {square root over (2)} and
c(u)=1 if u.noteq.0.
[0060] The inverse processing of the discrete cosine transform
consists of applying the following formula to the coefficients
DC.sub.i,j and AC.sub.i,j(u,v) of each block B.sub.i,j: I i , j
.function. ( x , y ) = 4 N 2 .times. u = 0 N - 1 .times. v = 0 N -
1 .times. c .function. ( u ) .times. c .function. ( v ) .times. F i
, j .function. ( u , v ) .times. cos .function. ( .pi. .times.
.times. u .function. ( 2 .times. x + 1 2 .times. N ) ) .times. cos
.function. ( .pi. .times. .times. v .function. ( 2 .times. y + 1 2
.times. N ) ) ( 4 ) ##EQU5##
[0061] In FIG. 2, the measurement system 1 comprises a
synchronisation module 11 that synchronises each coded image to be
processed with its coding grid, in other words the module 11
divides the image into blocks of N.times.N pixels. One example of a
synchronisation processing that could be done by this module is
described in patent application FR 2 769 452 filed by the
Applicant. This synchronisation processing identifies the division
into blocks used by the previous coding processing done by the
system 10, within the image, and therefore determines the coding
block B.sub.i,j to which each pixel in the image belongs.
[0062] The coded image and its division into blocks are applied to
two processing branches 2, 3, the first branch 2 comprising a
discrete cosine transform module 16, and the second branch 3
comprising firstly a shift module 12 producing a horizontal shift
of the coding grid by half a block with respect to the image. FIG.
3 illustrates this horizontal shift by half a block, by showing a
block B.sub.i,j of 8.times.8 pixels horizontally shifted by half a
block with respect to the coding grid illustrated by the adjacent
blocks 21 and 22.
[0063] In the second processing branch 3, the coded image and the
shifted decoding grid at the output from the shift module 12 are
applied to the input of a discrete cosine transform module 13. A
horizontal frequencies extraction module 14 then processes
transform coefficients output from module 13. In fact, this module
consists of assuring that the frequency domain only retains
frequencies (coefficients AC.sub.i,j(u,v) and DC.sub.i,j) that
could make a vertical block effect visible. If N=8, it could thus
be estimated that it is only necessary to retain coefficients
DC.sub.i,j, AC.sub.i,j(u,v) such that: 0.ltoreq.u<8 if v=0,
2.ltoreq.u<8 if v=1, and 6.ltoreq.u<8 if v=2
[0064] The retained coefficients AC.sub.i,j(u,v) and DC.sub.i,j are
then processed by an inverse DCT transformation module 15 to
retrieve spatial information I.sub.i,j(x,y)=I(8i+x, 8j+y) related
to the retained frequencies obtained using formula (4). In this
respect, it is possible to consider only the fourth and fifth
columns 24, 25 concerned by the block effect due to coding, and
which correspond to the boundary 23 between the blocks 21 and 22 of
the coding grid in FIG. 3.
[0065] Therefore, the output from branch 3 provides pixel data
I(8i+x, 8j+3) and I(8i+x, 8j+4) for the complete image, where i and
j are the indexes of shifted blocks of the image using the shifted
coding grid obtained at the output from the shift module 12. The
output from branch 2 provides coefficients DC.sub.i,j and
AC.sub.i,j(u,v) for each image block related to the coding grid at
the output from the synchronisation module 11. For each shifted
block B.sub.i,j, the coefficients DC.sub.i,j.sup.G and
AC.sub.i,j.sup.G (u,v) in the left block 21 and the coefficients
DC.sub.i,j.sup.D and AC.sub.i,j.sup.D (u,v) of the right block 22
are available.
[0066] The data output from the two processing branches 2 and 3, in
other words at the output from modules 16 and 15, are applied to
the input of a block effect evaluation module 17.
[0067] The data at the input to module 17 shown in FIG. 4 are
applied to the following modules: [0068] a module 31 for
calculating the average of inter pixel differences at the vertical
boundary 23 between each pair of adjacent blocks 21, 22 in the
coded image; [0069] a module 32 for calculating the average and the
standard deviation of the luminance of each block; and [0070] a
local contrast calculation module 34.
[0071] The module 31 applies the following formula to pixel data
output from the second branch 3, to evaluate the average of inter
pixel differences on each shifted block B.sub.i,j: .DELTA. I , j =
1 8 .times. k = 0 7 .times. I .function. ( 8 .times. i + k , 8
.times. j + 4 ) - ( 8 .times. i + k , 8 .times. j + 3 ) ( 5 )
##EQU6##
[0072] The average thus calculated actually describes the behaviour
of the gradient operator at the vertical boundary 23 of two
adjacent blocks 21, 22 output from the image coding grid.
[0073] In module 32, an average .mu..sub.i,j and a standard
deviation .sigma..sub.i,j of the luminance are estimated for each
block B.sub.i,j on the two corresponding adjacent blocks 21, 22 in
the coding grid. For example, the AC and DC coefficients of the
coding blocks in the DCT domain output by the first branch 2 could
be used according to the following formulas: .mu. i , j = 1 2
.times. ( DC i , j G + DC i , j D ) .times. .times. and ( 6 )
.sigma. i , j = 1 2 .times. ( 1 63 .times. u , v = 0 u + v .noteq.
0 7 .times. [ AC i , j G .function. ( u , v ) ] 2 + 1 63 .times. u
, v = 0 u + v .noteq. 0 7 .times. [ AC i , j D .function. ( u , v )
] 2 ) ( 7 ) ##EQU7##
[0074] Obviously, the average .mu..sub.i,j and the standard
deviation .sigma..sub.i,j could also be obtained directly from the
pixels of blocks 21, 22 corresponding to each block B.sub.i,j using
classical formulas for calculating the average and the standard
deviation.
[0075] The averages and standard deviations calculated by the
module 32 are then applied to a module 33 for calculation of a
weighting factor w.sub.i,j(.mu..sub.i,j, .sigma..sub.i,j, .zeta.)
for each block B.sub.i,j. The module 33 applies the weighting
function defined by the following formula to calculate this
weighting factor: w i , j .function. ( .mu. i , j , .sigma. i , j ,
.zeta. ) = { .lamda. .times. .times. ln .function. ( 1 + .mu. i , j
1 + .sigma. i , j ) .times. .times. if .times. .times. .mu. i , j
.ltoreq. .zeta. ln .function. ( 1 + 255 - .zeta. 1 + .sigma. i , j
) .times. .times. else , ( 8 ) where .times. .times. .lamda. = ln
.function. ( 1 + 255 - .zeta. 1 + .sigma. i , j ) ln .function. ( 1
+ .zeta. 1 + .sigma. i , j ) ( 9 ) ##EQU8## and .zeta. is a
parameter corresponding to the maximum sensitivity of the eye.
[0076] This weighting function is described in the publication "A
Generalized Block-Edge Impairment Metric for Video Coding" by H. R
Wu and M. Yuen, IEEE Signal Processing Letters, Vol. 4, No. 11,
November 1997. However, the weighting function described in this
publication has a discontinuity at point .mu..sub.i,j=.zeta. when
the standard deviation .sigma..sub.i,j is not zero. Therefore, this
function is adapted in the method according to the invention to be
continuous at this point.
[0077] FIG. 5 illustrates this weighting function in the form of a
curve 41 as a function of the grey level or the average luminance
.mu..sub.i,j of pixels in the block, in the case in which .zeta. is
chosen to be equal to 81, corresponding to the maximum sensitivity
of the human eye, and where .sigma. is equal to 20.
[0078] The local contrast calculation performed by module 34
consists in estimating the spatial activity ACT.sub.i,j of each
block B.sub.i,j in the transform domain, starting from coefficients
DC.sub.i,j and AC.sub.i,j, the coefficients AC.sub.i,j being
weighted by a masking function to model masking by the
neighbourhood. The spatial activity ACT.sub.i,j of each block
B.sub.i,j is thus estimated by applying the following formula: ACT
i , j = 1 2 .times. ( ACT i , j G + ACT i , j D ) ( 10 ) where
.times. .times. ACT i , j G = 1 1 + DC i , j G .times. u , v = 0 u
+ v .noteq. 0 7 .times. [ AC i , j G .function. ( u , v ) .times.
Nill .function. ( u , v ) ] 2 ( 11 ) and .times. .times. ACT i , j
D = 1 1 + DC i , j D .times. u , v = 0 u + v .noteq. 0 7 .times. [
AC i , j D .function. ( u , v ) .times. Nill .function. ( u , v ) ]
2 ( 12 ) ##EQU9## in which Nill(u,v) is the masking function that
can for example be the function described in the document "A Visual
Model Weighted Cosine Transform for Image Compression and Quality
Assessment" by N. B. Nill, IEEE Transactions on Communications,
Vol. COM-33, No. 6, June 1985.
[0079] In this document, the masking function is given by the
following formula: Nill(u,v)=A(.omega.)H(.omega.) (13) where
.omega.= {square root over (u.sup.2+v.sup.2)} represents the radial
frequency expressed in cycles per degree (u and v.di-elect
cons.[0,N-1]), A .function. ( .omega. ) = 1 4 + 1 .pi. 2 .function.
[ ln .function. ( 2 .times. .pi..pi. .alpha. + 4 .times. .pi..pi. 2
.times. .omega. .alpha. 2 + 1 ) ] 2 , ( 14 ) H .function. ( .omega.
) = ( 0.31 + 0.69 .times. .omega. ) .times. e - 0.29 .times.
.omega. ( 15 ) ##EQU10## and .alpha. is commonly equal to 11.636
deg.sup.-1 for processing of luminance on 8 bits.
[0080] FIG. 6 illustrates this weighting function by representing
the curve 43 showing the variation of H(.omega.), the curve 44
showing the variation of A(.omega.) and the curve 45 showing the
variation of the function Nill(.omega.), as a function of the
spatial frequency .omega..
[0081] In formulas (11) and (12), normalisation by the DC
coefficient provides a means for considering the match to the local
contrast in accordance with the Weber's law, and one is added to
the denominator so that the function is continuous at zero.
[0082] Finally, the results of calculations carried out by modules
31, 33 and 34 are applied to a module 35 for evaluation of the
image degradation u due to the edge effect resulting from coding.
This module carries out the following calculation to estimate an
image degradation measurement: .upsilon. = i , j .times. .upsilon.
i , j ( 16 ) where .times. .times. .upsilon. i , j = .DELTA.
.times. .times. I i , j .times. w i , j .function. ( .mu. i , j ,
.sigma. i , j , .zeta. ) 1 + .psi. .times. ACT i , j ( 17 )
##EQU11## represents the impact of block effects on block B.sub.i,j
and .PSI. is an experimentally determined constant that adjusts the
importance assigned to the spatial activity compared with the
weighted average of inter pixel differences.
[0083] The measurement of the degradation U estimated by this
module corresponds to an indicator of the visibility by the human
eye of degradations due to the block effect.
[0084] FIG. 7 shows variations of this measurement in the shape of
curves 46, 47, 48, as a function of the image number in a sequence
of video images, for throughputs of 9, 6 and 4 Mbits/s
respectively.
[0085] These curves show that image degradations are amplified as
the throughput is reduced. Therefore, the coarse coding by
quantification of DCT coefficients results in the block effect. The
part of the image sequence in which the degradation measurement is
high corresponds to images for which the spatial and time contents
change suddenly.
[0086] Processing done by module 17 can be optimized by submitting
input block data to a selection using a module for selection of
blocks B.sub.i,j that could contain a block effect. To achieve
this, this module compares the energy of central columns 24 and 25
of each block B.sub.i,j with energy of the block. The block is
selected to estimate the degradation if the energy of columns 24
and 25 represents a significant proportion, for example more than
25%, of the total energy of the block. Before this selection is
made, blocks for which the DCT coefficients selected by the
extraction module 14 are below a predefined threshold, for example
close to zero, can be eliminated.
[0087] Note that the processing that has just been described is
designed to detect a vertical block effect. Obviously, this
processing could be modified in an easily understood manner to
detect a horizontal block effect. All that is necessary in module
12 is to make a vertical shift of a half block, and then in the
frequency extraction module 14 to select frequencies that could be
affected by a horizontal block effect, and finally in module 15 to
consider all of the reconstituted pixels and only retain pixels in
the fourth and fifth lines of the shifted block corresponding to
the horizontal boundary between the two adjacent non-shifted blocks
corresponding to the shifted block.
[0088] It would also be possible to perform vertical and horizontal
block effect detection processings in parallel, and to use the
results output by these two processings at the same time, for
example in module 35.
[0089] After comparing objective measurements obtained by the
method and the system according to the invention, with marks
assigned during subjective tests by panels of representative
persons, it is found that there is a strong correlation between
objective measurements and subjective measurements, confirming that
the method and system according to the invention are efficient.
* * * * *