U.S. patent application number 15/742274 was filed with the patent office on 2018-07-12 for hierarchical tiling method for identifying a type of surface in a digital image.
This patent application is currently assigned to Luxembourg Institute of Science and Technology (LIST). The applicant listed for this patent is Luxembourg Institute of Science and Technology (LI. Invention is credited to Marco Chini, Laura Guistarini, Renaud Hostache, Patrick Matgen.
Application Number | 20180197305 15/742274 |
Document ID | / |
Family ID | 53724422 |
Filed Date | 2018-07-12 |
United States Patent
Application |
20180197305 |
Kind Code |
A1 |
Hostache; Renaud ; et
al. |
July 12, 2018 |
Hierarchical Tiling Method for Identifying a Type of Surface in a
Digital Image
Abstract
The invention is directed to a method of identifying at least
one type of surface in a digital image, comprising the steps of:
(a) dividing (4) the image (2) in sub-images of the same size; (b)
analysing (6) the sub-images for identifying at least one type of
surface; (c) sub-dividing (10) into sub-images each of the
sub-images (8) of the preceding step where at least one type of
surface is not identified; (d) analysing (6) the sub-images of step
(c) for identifying at least one type of surface; and (e) iterating
steps (c) and (d). This is a hierarchical split based approach
(HSBA) that can be used for detecting water zones in a Synthetic
Aperture Radar (SAR) image.
Inventors: |
Hostache; Renaud;
(Thionville, FR) ; Chini; Marco; (Luxembourg,
LU) ; Matgen; Patrick; (Diekirch, LU) ;
Guistarini; Laura; (Luxembourg, LU) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Luxembourg Institute of Science and Technology (LI |
Esch/Alzette |
|
LU |
|
|
Assignee: |
Luxembourg Institute of Science and
Technology (LIST)
Esch/Alzette
LU
|
Family ID: |
53724422 |
Appl. No.: |
15/742274 |
Filed: |
July 5, 2016 |
PCT Filed: |
July 5, 2016 |
PCT NO: |
PCT/EP2016/065842 |
371 Date: |
January 5, 2018 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06K 9/0063 20130101;
G06T 2207/20021 20130101; G06K 2009/00644 20130101; G06T 7/41
20170101; G06T 2207/10044 20130101 |
International
Class: |
G06T 7/41 20060101
G06T007/41 |
Foreign Application Data
Date |
Code |
Application Number |
Jul 6, 2015 |
LU |
LU 92763 |
Claims
1.-18. (canceled)
19. Method of identifying at least one type of surface in a
synthetic aperture radar (SAR) image, comprising: (a) dividing the
image in sub-images; (b) analyzing, based on backscatter values of
the synthetic aperture radar (SAR) image, the sub-images for
identifying the at least one type of surface; (c) subdividing in
sub-images each of the sub-images of the preceding step where the
at least one type of surface is not identified; (d) analyzing the
sub-images of step (c) for identifying the at least one type of
surface; and (e) iterating steps (c) and (d); wherein step (b) is
based on the parametrization of two distributions of the number of
pixels of the sub-image as a function of their backscatter
values.
20. Method according to claim 19, wherein in step (c) only the
sub-image(s) of the preceding step where the at least one type of
surface is/are not identified are subdivided.
21. Method according to claim 19, wherein the analysis for
identifying the at least one type of surface is identical in steps
(b) and (d).
22. Method according to claim 19, wherein the analysis for
identifying the at least one type of surface is based on the
detection of several distributions in the total distribution of the
number of pixels of the sub-image as a function of their
backscatter values.
23. Method according to claim 19, wherein a first distribution of
the two distributions corresponds to the at least one type of
surface and a second distribution of the two distributions
corresponds to any other type of surface.
24. Method according to claim 19, wherein the two distributions
have an Ashman D coefficient that is higher than 2.
25. Method according to claim 19, wherein the number of pixels as a
function of the backscatter value is modeled by a theoretical
distribution, like a Gauss distribution, and fitted using the
Levenberg-Marquardt algorithm LMA.
26. Method according to claim 19, wherein the at least one type of
surface is identified when the number of pixels of the sub-image
that relate to one of the distributions corresponds to one of the
following: at least 10% of the total number of pixels of said
sub-image; or at least 20% of the total number of pixels of said
sub-image.
27. Method according to claim 19, wherein the at least one type of
surface is a water surface.
28. Method according to claim 19, wherein the mode of distribution
is expressed in sigma nought and has a value between -30 dB and -10
dB.
29. Method according to claim 19, wherein the mode of the second
distribution is expressed in sigma nought and has a value between
-12 dB and 5 dB.
30. Method according to claim 19, wherein dividing in step (a) and
subdividing in step (c) consist in dividing by four the image,
sub-image or each of the sub-images, respectively.
31. Method according to claim 19, wherein the iteration of step (e)
is stopped when in step (d) the at least one type of surface is
identified in each of the sub-images of step (c).
32. Method according to claim 19, further comprising: (f) providing
an identification of the at least one type of surface for the
entire image based on the identifications of at least one type of
surface at step (b) for each of the sub-images.
33. A device, comprising: a memory element; and computing means;
wherein the memory element and the computing means are configured
for carrying out a method of identifying at least one type of
surface in a synthetic aperture radar (SAR) image, comprising: (a)
dividing the image in sub-images; (b) analyzing, based on
backscatter values of the synthetic aperture radar (SAR) image, the
sub-images for identifying the at least one type of surface; (c)
subdividing in sub-images each of the sub-images of the preceding
step where the at least one type of surface is not identified; (d)
analyzing the sub-images of step (c) for identifying the at least
one type of surface; and (e) iterating steps (c) and (d); wherein
step (b) is based on the parametrization of two distributions of
the number of pixels of the sub-image as a function of their
backscatter values.
34. A computer program for improving the performance of a computer,
comprising: computer readable code means, which when run on a
computer, causes the computer to carry out a method of identifying
at least one type of surface in a synthetic aperture radar (SAR)
image, comprising: (a) dividing the image in sub-images; (b)
analyzing, based on backscatter values of the synthetic aperture
radar (SAR) image, the sub-images for identifying the at least one
type of surface; (c) subdividing in sub-images each of the
sub-images of the preceding step where the at least one type of
surface is not identified; (d) analyzing the sub-images of step (c)
for identifying the at least one type of surface; and (e) iterating
steps (c) and (d); wherein step (b) is based on the parametrization
of two distributions of the number of pixels of the sub-image as a
function of their backscatter values.
Description
TECHNICAL FIELD
[0001] The invention is directed to the field of digital image
processing, more particularly to the classification of surface(s)
in digital image, such as Synthetic Aperture Radar (SAR) images,
where a particular type of surface occupies only a small fraction
of the image.
BACKGROUND ART
[0002] The publication by F. Bovolo, L. Bruzzone, "A Split-Based
Approach to Unsupervised Change Detection in Large-Size
Multitemporal Images: Application to Tsunami-Damage Assessment",
IEEE Transactions on Geoscience and Remote Sensing, Vol. 45, No. 6,
pp. 1658-1670, 2007 discloses a split-based approach for
automatically detecting changes in a sequence of images. The method
consists essentially in (i) splitting the image into sub-images;
(ii) an analysis of each sub-image; and (iii) an automatic
threshold-selection procedure. In step (ii) changes are identified
by computing the histogram of difference values obtained from two
sub-images that are acquired on the same geographical area at two
different times. The sub-images are then sorted out according to
their probability to contain a significant amount of changed
pixels. The subset of the sub-images with a high probability to
contain changes is selected and analysed in step (iii) according to
a threshold-selection procedure applied separately to each
sub-image or to the joint distribution of pixels that is obtained
by merging all sub-images of the subset.
[0003] The publication S. Martinis, J. Kersten, A. Twele, "A fully
automated TerraSAR-X based flood service", ISPRS Journal of
Photogrammetry and Remote Sensing,
doi:10.1016/j.isprsjprs.2014.07.014, 2015 discloses an automatic
image processing to identify flooded surfaces from Synthetic
Aperture Radar (SAR) images. The processing of this teaching is
also a split-based approach and is based on the backscatter
statistics inferred from a single flood image to separate the
"water" class from the others.
[0004] Both above mentioned teachings apply a split-based approach
(SBA). This approach consists in tiling the image in sub-images of
equal sizes and defining a threshold based on the histograms
inferred from the different tiles. So far, SBA has been used to
generate tiles of fixed size. The size is defined in an arbitrary
way, using the SAR sensor resolution, the size of the scene and the
percentage of the image occupied by the targeted class/population
as indicators. However, this method is not efficient because i) the
maximum size of the tile enabling the robust parameterization of
the distribution function is unknown a priori and ii) the tiling
process is not linked to the parameterization process of the
distribution function.
SUMMARY OF INVENTION
Technical Problem
[0005] The invention has for technical problem to provide a more
efficient method for identifying particular types of surface(s) in
digital images, in particular SAR images, that occupy only a small
fraction of the image.
Technical solution
[0006] The invention is directed to a method of identifying at
least one type of surface in a digital image, comprising the steps
of: (a) dividing the image in sub-images; (b) analysing the
sub-images for identifying the at least one type of surface; with
the additional steps of (c) subdividing in sub-images each of the
sub-images of the preceding step where the at least one type of
surface is not identified; (d) analysing the sub-images of step (c)
for identifying the at least one type of surface; (e) iterating
steps (c) and (d).
[0007] The type(s) of surface in the digital image can correspond
to class(es) and/or population(s) of pixels of the image.
[0008] According to a preferred embodiment of the invention, in
step (c) only the sub-image(s) of the preceding step where the at
least one type of surface could not be identified are
subdivided.
[0009] In step (a) and/or in step (c), the divided or subdivided
sub-images are advantageously of the same size. They are preferably
non-overlapping. The number of sub-images resulting from the
division and/or each subdivision can be of four. They can be
square- or rectangle-shaped.
[0010] According to a preferred embodiment of the invention, the
analysis for identifying at least one type of surface is identical
in steps (b) and (d).
[0011] According to a preferred embodiment of the invention, the
digital image is a synthetic aperture radar SAR image. The digital
image can also be a change detection image, i.e. an image that is
the difference between two images of the same area acquired at
different time steps. In that case, the method can be used to
differentiate the pixels that changed from the pixels that did not
change.
[0012] According to a preferred embodiment of the invention, the
analysis for identifying the at least one type of surface is based
on backscatter values of the SAR image.
[0013] According to a preferred embodiment of the invention, the
analysis for identifying the at least one type of surface is based
on the detection of several distributions in the total distribution
of the number of pixels of the sub-image as a function of their
backscatter values.
[0014] According to a preferred embodiment of the invention, the
analysis for identifying the at least one type of surface is based
on the parameterization of two distributions of the number of
pixels of the sub-image as a function of their backscatter
values.
[0015] According to a preferred embodiment of the invention, the
first distribution corresponds to the at least one type of surface,
whereas the second one corresponds to any other type of
surface.
[0016] According to a preferred embodiment of the invention, the
two distributions have an Ashman D coefficient that is higher than
2.
[0017] According to a preferred embodiment of the invention, the
number of pixels as a function of the backscatter value is modeled
by a theoretical distribution (e.g. Gauss) and fitted using the
Levenberg-Marquardt algorithm (LMA).
[0018] According to a preferred embodiment of the invention, the at
least one type of surface is identified when the number of pixels
of the sub-image, related to one of the distributions, corresponds
to at least 10%, more preferably at least 20% of the total number
of pixels of said sub-image.
[0019] According to a preferred embodiment of the invention, the at
least one type of surface is a water surface.
[0020] According to a preferred embodiment of the invention, the
mode of the first distribution is expressed in sigma nought and has
a value between -30 dB and -10 dB.
[0021] According to a preferred embodiment of the invention, the
mode of the second distribution is expressed in sigma nought and
has a value comprised between -12 dB and 5 dB.
[0022] According to a preferred embodiment of the invention,
dividing in step (a) and subdividing in step (c) consist in
dividing by four the image, sub-image or each of the sub-images,
respectively.
[0023] According to a preferred embodiment of the invention, the
iteration of step (e) is stopped when in step (d) the at least one
type of surface is identified in each of the sub-images of step (c)
and/or when the size of the sub-images in step (c) is below a
predetermined minimum size.
[0024] According to a preferred embodiment of the invention, said
method comprises a further step (f) of providing an identification
of the at least one type of surface for the entire image based on
the identifications at step (b) for each of the sub-images.
[0025] The invention is also directed to a device comprising a
memory element and computing means, said element and means being
configured for carrying out the method according to the
invention.
[0026] The invention is also directed to a computer capable of
carrying out the method according to the invention.
[0027] The invention is also directed to a computer program
comprising computer readable code means, which, when it is run on a
computer, causes the computer to carry out the method according to
the invention.
[0028] The invention is also directed to a computer program product
comprising a computer-readable medium on which the computer program
according to the invention is stored.
Advantages of the Invention
[0029] The invention proposes a hierarchical split based approach
(HSBA) that, contrary to the split based approach (SBA) of the
prior art, does not fix the size of the tiles a priori but, rather,
searches for tiles of variable size that allow parameterizing the
statistical distribution function attributed to surface
water-related backscatter values. The tiling and the
parameterization processes are thus integrated. The HSBA
sequentially and selectively splits the image into sub-images of
decreasing size in order to identify tiles of variable size for
which a surface water-related distribution function can be
parameterized. This procedure thus renders the identification of
the at least one type of surface i) objective, ii) independent of
the different technical characteristics of the image scene (e.g.
spatial resolution or percentage of the extension of the at least
one type of surface with respect to extension of the entire image),
iii) robust and iv) efficient.
BRIEF DESCRIPTION OF THE DRAWINGS
[0030] FIG. 1 shows a SAR image where the darker areas correspond
to a river with flooded areas, the image being not yet processed
according to the invention.
[0031] FIG. 2 shows the SAR image of FIG. 1 after a first division
of the entire image into four sub-images of the same size.
[0032] FIG. 3 shows the SAR image of FIG. 2 where three of the
sub-images of FIG. 2, where no water surface could be identified,
are sub-divided each into four sub-images of the same size.
[0033] FIG. 4 shows the SAR image of FIG. 3 where ten of the
sub-images of FIG. 3, where no water surface could be identified,
are sub-divided each into four sub-images of the same size.
DESCRIPTION OF AN EMBODIMENT
[0034] The following embodiment is directed to a method to
delineate water bodies from a SAR image and will be described in
combination with FIGS. 1 to 5.
[0035] FIGS. 1 to 4 illustrate a SAR image at successive steps of
the method according to the invention and FIG. 5 is a flowchart
illustrating the principle of the invention.
[0036] FIG. 1 illustrates a Synthetic Aperture Radar (SAR) image
showing a geographical zone with a river and associated flooded
areas, visible as darker areas. A synthetic aperture radar, or SAR,
is a coherent radar system that generates high-resolution remote
sensing imagery. Signal processing uses magnitude and phase of the
received signals over successive pulses from elements of a
synthetic aperture to create an image. As the line of sight
direction changes along the radar platform trajectory, a synthetic
aperture is produced by signal processing that has the effect of
lengthening the antenna.
[0037] Backscatter is the portion of the outgoing radar signal that
the target redirects directly back towards the radar antenna. The
scattering cross section in the direction toward the radar is
called the backscattering cross section; the usual notation is the
symbol sigma. It is a measure of the reflective strength of a radar
target. The normalised measure of the radar return from a
distributed target is called the backscatter coefficient, or sigma
nought, and is defined as per unit area on the ground. Other
portions of the incident radar energy may be reflected and
scattered away from the radar or absorbed.
[0038] Generally speaking, the water zones generate a
backscattering that is rather different from the non-watered zones.
When the proportion of pixels of the image that correspond to water
zones is large enough the distribution of the pixels over the
backscattering values can be parameterized. This distribution can
be a mixture of several distributions, e.g. of the Gaussian-type.
The Gaussian-type distribution is illustrated next to the SAR image
in FIG. 1. The histogram shows then two identifiable distribution
functions, i.e. a first distribution of pixel values for the water
zones and a second one for the non-watered zones. This permits a
reliable identification of the watered zones.
[0039] Watered zones often represent however only a small fraction
of an entire SAR scene. In these circumstances it becomes
difficult, if not impossible, to accurately parameterize the
distribution function of backscatter values associated with watered
surface.
[0040] With reference to FIG. 2, the initial SAR image is divided
into four sub-images of the same size, thereby reducing the size of
each sub-image by four. As is visible in the figure, the lower left
sub-image comprises the highest proportion of watered zones,
thereby providing the bimodality in the distribution of the
backscattering values of the pixels. This means that for this
sub-image, the water zones can be identified in a reliable manner
based on the pixel distribution. For the remaining three
sub-images, i.e. the top left, top right and bottom right
sub-images, this reliable identification is not possible even
though the top left and bottom right sub-images contain water
zones.
[0041] In FIG. 3, we can observe that each of the above mentioned
remaining three sub-images have been subdivided, for instance in
four sub-images of the same size. As is apparent in FIG. 3, only
two of the twelve newly divided sub-images present bimodality.
[0042] In FIG. 4, the remaining recently divided sub-images showing
no bimodality are subdivided again, for instance into four
sub-images of the same size. As is apparent in FIG. 4, only two of
the forty newly divided sub-images show bimodality, so that the
remaining 38 sub-images are further subdivided as explained in
connection with FIGS. 2, 3 and 4, and so on.
[0043] The invention consists therefore in iterating by subdividing
the image or sub-images where no water zone can be identified in a
reliable manner. In the present embodiment, this identification of
the selected sub-images is based on the Ashman D coefficient that
is in reality a proxy of the bimodality in the distribution of the
backscattering values of the pixels of the image in question. It is
however understood that other criteria can be considered for the
identification of sub-images of interest. The division or
sub-division into four sub-images of the same size is a matter of
choice, being understood that other manners of dividing the image
and/or sub-images can be considered.
[0044] The iteration can be stopped based on different criteria.
For example, it can be stopped when reaching a given size of
sub-images. It can also be stopped when bimodality is observed for
each sub-image.
[0045] The identification of water zones by means of the above
described bimodality is based on the hypothesis that the image
histogram is composed of a mixture of two distributions
representing, respectively, the watered and non-watered classes. To
calibrate the parameters of the distribution function, a
Levenberg-Marquard algorithm (non-linear least squares) can be
used, while for evaluating if the two distributions are well
identified the Ashman D coefficient can be computed, this
coefficient having to be higher than 2. Typically, this processing
step assumes that the histogram can be separated into two Gaussian
distribution functions. A similar approach is detailed in the
publication of Giustarini, L.; Hostache, R.; Matgen, P.; Schumann,
G. J.-P.; Bates, P. D.; Mason, D. C., "A Change Detection Approach
to Flood Mapping in Urban Areas Using TerraSAR-X," Geoscience and
Remote Sensing, IEEE Transactions on, vol. 51, no. 4, pp. 2417,
2430, April 2013 doi: 10.1109/TGRS.2012.2210901. However, any other
type of distribution function can be considered.
[0046] The backscattering sigma nought values of the (Gaussian)
distribution that corresponds to water zones have a mode value
comprised between -30 dB and -10 dB. Similarly, backscattering
sigma nought values of the (Gaussian) distribution that corresponds
to non-watered zones have a mode value comprised between -12 dB and
5 dB.
[0047] FIG. 5 illustrates with a flowchart the method of the
invention. At step 2, an initial digital image is provided, being
understood that any type of digital image can be considered,
including a SAR image. The initial image can be large, e.g. more
than 10000 by 10000 pixels.
[0048] At step 4, the initial image is divided in sub-images. This
division is preferably made so that the sub-images are of the same
size. Their number can be four, whereas other ways of dividing the
image can be considered. In other words, the initial image is split
into a fixed number of sub-images. These sub-images do not
overlap.
[0049] At step 6, each of the sub-images resulting from the
division of the previous step 4 are analysed for potentially
identifying the at least one type of surface. At least one type of
surface can be a watered surface as in the embodiment described in
relation with FIGS. 1 to 4, being however understood that other
types of surfaces could be identified. The identification process
that is applied to the sub-images is preferably always the same.
This means that for some sub-images, the identification process can
provide no tangible result, i.e. no identification of at least one
type of surface. In other words, the result of step 6 can be of two
types for the different sub-images, i.e. either at least one type
of surface is detected or it is not. The identification process can
apply various approaches, including the bimodality approach
described above in relation with FIGS. 1 to 4.
[0050] For the sub-image(s) of step 6 where at least one type of
surface could not be identified, as mentioned in step 8, this or
each of these sub-image(s) is subdivided in step 10 into further
sub-images, preferably following the same division rule as in step
4. For instance, the sub-image or each of these sub-images can be
subdivided into four further non-overlapping sub-images of the same
size.
[0051] These sub-images of reduced size, resulting from step 10,
are then analysed at step 6 so as to potentially identify at least
one type of surface. With reference to the above discussion of step
6, this operation might result in the identification of at least
one type of surface for one or some of the sub-images, whereas it
can also result in the absence of identification of at least one
type of surface for the remaining sub-images. For these latter,
steps 8 and 10 apply in an iterative way. For the other sub-images,
i.e. those where at least one type of surface could be identified
and as identified in step 12, no further subdivision is proposed
and these zones where at least one type of surface has been
identified, are saved for constructing the identification of at
least one type of surface for the entire initial image.
[0052] The above discussed iteration from step 6 to steps 8 and 10
can be repeated until at least one type of surface is identified in
each sub-image. Since some portions of the image could be void of
at least one type of surface, the iteration can be stopped
automatically when reaching a certain minimum size of the
sub-images.
[0053] The above method can be operated as a computer program that
is executed on a computer.
* * * * *