U.S. patent application number 16/685609 was filed with the patent office on 2021-05-20 for cloud detection from satellite imagery.
This patent application is currently assigned to Maxar Intelligence Inc.. The applicant listed for this patent is Maxar Intelligence Inc.. Invention is credited to Brett W. Bader.
Application Number | 20210150182 16/685609 |
Document ID | / |
Family ID | 1000005565025 |
Filed Date | 2021-05-20 |
View All Diagrams
United States Patent
Application |
20210150182 |
Kind Code |
A1 |
Bader; Brett W. |
May 20, 2021 |
CLOUD DETECTION FROM SATELLITE IMAGERY
Abstract
Described herein are methods and systems for detecting clouds in
satellite imagery captured using first and second sensor arrays
that are carried by a satellite and physically offset from one
another on the satellite. Movement mask data is produced based
first image data and the second image data, obtained, respectively,
using the first and second sensor arrays carried by the satellite.
Cloud mask data is produced based on spectral information included
in one of the first and second image data. Cloud detection data is
produced based on the movement mask data and the cloud mask data,
the cloud detection data indicating where it is likely, based on
both the movement mask data and the cloud mask data, that one or
more clouds are represented within one of the first and second
image data. The cloud detection data can be used in various ways to
account for the clouds included within the satellite imagery.
Inventors: |
Bader; Brett W.; (Lyons,
CO) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Maxar Intelligence Inc. |
Westminster |
CO |
US |
|
|
Assignee: |
Maxar Intelligence Inc.
Westminster
CO
|
Family ID: |
1000005565025 |
Appl. No.: |
16/685609 |
Filed: |
November 15, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06K 9/6202 20130101;
G06K 9/00664 20130101; G06T 2207/30192 20130101; G06K 9/0063
20130101; G06T 7/254 20170101; G06T 2207/10032 20130101 |
International
Class: |
G06K 9/00 20060101
G06K009/00; G06K 9/62 20060101 G06K009/62; G06T 7/254 20060101
G06T007/254 |
Claims
1. A method for detecting clouds in satellite imagery captured
using first and second sensor arrays that are carried by a
satellite and physically offset from one another on the satellite,
the method comprising: receiving first and second image data
obtained, respectively, using the first and second sensor arrays
carried by the satellite; producing movement mask data based on
dissimilarities identified between the first image data and the
second image data; producing cloud mask data based on spectral
information included in one of the first and second image data;
intersecting the movement mask data with the cloud mask data to
thereby produce intersection data; and producing cloud detection
data based on the intersection data.
2. The method of claim 1, wherein: the first image data includes
first image information about a first geographic region; the second
image data includes second image information about the first
geographic region, which differs from the first image information
about the first geographic region included in the first image data;
the movement mask data indicates where it is likely that the
dissimilarities between the first image data and the second image
are due to movement of one or more objects within the first
geographic region; the cloud mask data indicates where it is
likely, based on the spectral information included in the one of
the first and second image data, that one or more clouds are
represented within the one of the first and second image data; and
the cloud detection data indicates where it is likely, based on the
dissimilarities between the first image data and the second image
data, as well as based on the spectral information included in the
one of the first and second image data, that one or more clouds are
represented within the one of the first and second image data.
3. The method of claim 1, wherein because the first sensor array
used to obtain the first image data is physically offset on the
satellite from the second sensor array used to obtain the second
image data: a second satellite viewing angle associated with the
second image data obtained using the second sensor array differs
from a first satellite viewing angle associated with the first
image data obtained using the first sensor array, with a difference
between the first and second satellite viewing angles being a
parallax angle; and a second time at which part of the second image
data corresponding to a piece of a first geographic region is
obtained using the second sensor array differs from a first time at
which part of the first image data corresponding to the piece of
the first geographic region is obtained using the first sensor
array, with a difference between the first and second times being a
temporal offset.
4. The method of claim 1, wherein: the first image data obtained
using the first sensor array includes first spectral band data
corresponding to a first spectral band, the second image data
obtained using the second sensor array includes second spectral
band data corresponding to a second spectral band that is adjacent
to and thereby proximate in wavelength to the first spectral band,
and the producing movement mask data comprises: performing edge
detection on the first spectral band data and the second spectral
band data to thereby produce first edge image data and second edge
image data; computing dissimilarity values between corresponding
image chips within the first and second edge image data; comparing
the dissimilarity values to a threshold; and producing the movement
mask data based on results of the comparing the dissimilarity
values to the threshold.
5. The method of claim 1, wherein the producing cloud mask data
based on spectral information included in one of the first and
second image data, comprises: computing chroma and lightness values
for each pixel of a plurality of pixels of the one of the first and
second image data; computing a cloud index value, for each pixel of
the plurality of pixels, based on the chroma and lightness values
computed for the pixel; comparing the computed cloud index value,
for each pixel of the plurality of pixels, to a threshold value;
and classifying each pixel of the plurality of pixels as either
corresponding to a cloud or not based on results of the comparisons
to the threshold value.
6. The method of claim 5, wherein the computing the cloud index
value, for each pixel of the plurality of pixels, is performing
using the following equation: index = L a C 2 + b ##EQU00001##
where index is the cloud index value for the pixel, L is the
lightness value calculated for the pixel, C is the chroma value
calculated for the pixel, and a and b are constants.
7. The method of claim 5, wherein the producing cloud mask data
further comprises, after the classifying each pixel of the
plurality of pixels as either corresponding to a cloud or not:
performing a binary opening morphological operation on results of
the classifying, to thereby reduce a number of false positives
included in the cloud mask data before the cloud mask data is
intersected with the movement mask data to produce the intersection
data.
8. The method of claim 1, wherein the producing cloud detection
data, based on the intersection data, comprises: identifying, based
on the intersection data, one or more groups of pixels that may
correspond to a cloud; for each group of pixels that may correspond
to a cloud, determining a magnitude of movement and a direction of
movement of the group; and clustering groups of pixels for which
the determined magnitude of movement and the determined direction
of movement are similar one another to thereby identify one or more
distinct banks of clouds that are included within the first and
second image data.
9. The method of claim 8, wherein, the producing the cloud
detection data further comprises: for each of the one or more
distinct cloud banks identified as a result of the clustering,
determining based on the first and second edge image data,
additional pixels that correspond to the cloud bank; classifying,
as corresponding to a cloud, saturated pixels that are adjacent to
another pixel that has already been classified as corresponding to
a cloud; performing a morphological fill operation to thereby
classify, as corresponding to a cloud, pixels that are surrounded
by other pixels that have already been classified as corresponding
to a cloud; up-sampling results of the morphological fill operation
to increase resolution to an original resolution of the first and
second image data; performing a morphological dilation operation on
results of the up-sampling to thereby produce dilated cloud
detection data, wherein each pixel included in the dilated cloud
detection data includes a respective cloud index value; comparing
the cloud index value, for each pixel of the dilated cloud
detection data, to a threshold value; and producing the cloud
detection data based on results of the comparing the cloud index
value, for each pixel of the dilated cloud detection data, to the
threshold value.
10. A non-transitory processor readable storage device having
processor readable code embodied on the processor read storage
device, the processor readable code for programming one or more
processors to perform a method comprising: receiving first and
second image data obtained, respectively, using the first and
second sensor arrays carried by the satellite; producing movement
mask data based on dissimilarities identified between the first
image data and the second image data; producing cloud mask data
based on spectral information included in one of the first and
second image data; intersecting the movement mask data with the
cloud mask data to thereby produce intersection data; and producing
cloud detection data based on the intersection data.
11. The non-transitory processor readable storage device of claim
10, wherein: the first image data includes first image information
about a first geographic region; the second image data includes
second image information about the first geographic region, which
differs from the first image information about the first geographic
region included in the first image data; the movement mask data
indicates where it is likely that the dissimilarities between the
first image data and the second image are due to movement of one or
more objects within the first geographic region; the cloud mask
data indicates where it is likely, based on the spectral
information included in the one of the first and second image data,
that one or more clouds are represented within the one of the first
and second image data; the cloud detection data indicates where it
is likely, based on the dissimilarities between the first image
data and the second image data, as well as based on the spectral
information included in the one of the first and second image data,
that one or more clouds are represented within the one of the first
and second image data; a second satellite viewing angle associated
with the second image data obtained using the second sensor array
differs from a first satellite viewing angle associated with the
first image data obtained using the first sensor array, with a
difference between the first and second satellite viewing angles
being a parallax angle; and a second time at which part of the
second image data corresponding to a piece of the first geographic
region is obtained using the second sensor array differs from a
first time at which part of the first image data corresponding to
the piece of the first geographic region is obtained using the
first sensor array, with a difference between the first and second
times being a temporal offset.
12. The non-transitory processor readable storage device of claim
10, wherein: the first image data obtained using the first sensor
array includes first spectral band data corresponding to a first
spectral band, the second image data obtained using the second
sensor array includes second spectral band data corresponding to a
second spectral band that is adjacent to and thereby proximate in
wavelength to the first spectral band, and the producing movement
mask data comprises: performing edge detection on the first
spectral band data and the second spectral band data to thereby
produce first edge image data and second edge image data; computing
dissimilarity values between corresponding image chips within the
first and second edge image data; comparing the dissimilarity
values to a threshold; and producing the movement mask data based
on results of the comparing the dissimilarity values to the
threshold.
13. The non-transitory processor readable storage device of claim
10, wherein the producing cloud mask data based on spectral
information included in one of the first and second image data,
comprises: computing chroma and lightness values for each pixel of
a plurality of pixels of the one of the first and second image
data; computing a cloud index value, for each pixel of the
plurality of pixels, based on the chroma and lightness values
computed for the pixel; comparing the computed cloud index value,
for each pixel of the plurality of pixels, to a threshold value;
and classifying each pixel of the plurality of pixels as either
corresponding to a cloud or not based on results of the comparisons
to the threshold value.
14. The non-transitory processor readable storage device of claim
13, wherein the computing the cloud index value, for each pixel of
the plurality of pixels, is performing using the following
equation: index = L a C 2 + b ##EQU00002## where index is the cloud
index value for the pixel, L is the lightness value calculated for
the pixel, C is the chroma value calculated for the pixel, and a
and b are constants.
15. The non-transitory processor readable storage device of claim
13, wherein the producing cloud mask data further comprises, after
the classifying each pixel of the plurality of pixels as either
corresponding to a cloud or not: performing a binary opening
morphological operation on results of the classifying, to thereby
reduce a number of false positives included in the cloud mask data
before the cloud mask data is intersected with the movement mask
data to produce the intersection data.
16. The non-transitory processor readable storage device of claim
10, wherein the producing cloud detection data, based on the
intersection data, comprises: identifying, based on the
intersection data, one or more groups of pixels that may correspond
to a cloud; for each group of pixels that may correspond to a
cloud, determining a magnitude of movement and a direction of
movement of the group; and clustering groups of pixels for which
the determined magnitude of movement and the determined direction
of movement are similar one another to thereby identify one or more
distinct banks of clouds that are included within the first and
second image data.
17. The non-transitory processor readable storage device of claim
16, wherein the producing the cloud detection data further
comprises: for each of the one or more distinct cloud banks
identified as a result of the clustering, determining based on the
first and second edge image data, additional pixels that correspond
to the cloud bank; classifying, as corresponding to a cloud,
saturated pixels that are adjacent to another pixel that has
already been classified as corresponding to a cloud; performing a
morphological fill operation to thereby classify, as corresponding
to a cloud, pixels that are surrounded by other pixels that have
already been classified as corresponding to a cloud; up-sampling
results of the morphological fill operation to increase resolution
to an original resolution of the first and second image data;
performing a morphological dilation operation on results of the
up-sampling to thereby produce dilated cloud detection data,
wherein each pixel included in the dilated cloud detection data
includes a respective cloud index value; comparing the cloud index
value, for each pixel of the dilated cloud detection data, to a
threshold value; and producing the cloud detection data based on
results of the comparing the cloud index value, for each pixel of
the dilated cloud detection data, to the threshold value.
18. A method for detecting clouds in satellite imagery captured
using first and second sensor arrays that are carried by a
satellite and physically offset from one another on the satellite,
the method comprising: producing movement mask data based on first
image data and the second image data, obtained, respectively, using
the first and second sensor arrays carried by the satellite, the
first image data including first image information about a first
geographic region, and the second image data including second image
information about the first geographic region, which differs from
the first image information about the first geographic region
included in the first image data; producing cloud mask data based
on spectral information included in one of the first and second
image data, the cloud mask data indicating where it is likely,
based on the spectral information included in the one of the first
and second image data, that one or more clouds are represented
within the one of the first and second image data; and producing
cloud detection data based on the movement mask data and the cloud
mask data, the cloud detection data indicating where it is likely,
based on both the movement mask data and the cloud mask data, that
one or more clouds are represented within the one of the first and
second image data.
19. The method of claim 18, wherein the producing movement mask
data comprises: performing edge detection on the first image data
and the second image data to thereby produce first edge image data
and second edge image data; computing dissimilarity values between
corresponding image chips within the first and second edge image
data; and producing the movement mask data based on results of
comparing the dissimilarity values to a threshold.
20. The method of claim 18, wherein the producing cloud mask data
based on spectral information included in one of the first and
second image data, comprises: computing color transformation values
for each pixel of a plurality of pixels of the one of the first and
second image data; computing a cloud index value, for each pixel of
the plurality of pixels, based on the color transformation values
computed for the pixel; comparing the computed cloud index value,
for each pixel of the plurality of pixels, to a threshold value;
and classifying each pixel of the plurality of pixels as either
corresponding to a cloud or not based on results of the comparisons
to the threshold value.
21. A system for detecting clouds in satellite imagery captured
using first and second sensor arrays that are carried by a
satellite and physically offset from one another on the satellite,
the system comprising: one or more processors configured to produce
movement mask data based on first image data and second image data,
obtained, respectively, using the first and second sensor arrays
carried by the satellite, the first image data including first
image information about a first geographic region, and the second
image data including second image information about the first
geographic region, which differs from the first image information
about the first geographic region included in the first image data;
produce cloud mask data based on spectral information included in
one of the first and second image data, the cloud mask data
indicating where it is likely, based on the spectral information
included in the one of the first and second image data, that one or
more clouds are represented within the one of the first and second
image data; and produce cloud detection data based on the movement
mask data and the cloud mask data, the cloud detection data
indicating where it is likely, based on both the movement mask data
and the cloud mask data, that one or more clouds are represented
within the one of the first and second image data.
Description
[0001] Still referring to FIG. 6, step 606 involves producing cloud
mask data based on spectral information included in one of the
first and second image data. Such cloud mask data indicates where
it is likely (based on the spectral information included in one of
the first and second image data) that one or more clouds are
represented within one of the first and second image data. For the
sake of discussion, it is assumed that it is the first image data
that is used to produce the cloud mask data at step 604. The
spectral information included in the first image data can include,
for example, B, G, R, and N1 band values for each pixel of
N.times.M pixels included in the first image (e.g., 504 in FIG.
5A). As will be discussed in further detail below with reference to
FIG. 8, in accordance with certain embodiments, step 606 can
involve producing a cloud index value for each of a plurality of
pixels (each of the N.times.M pixels included in the first image,
or a subset thereof) based on R, G, and B band values (also
referred to as RGB data) corresponding to those pixels, and
thresholding the cloud index values.
[0002] As noted above, embodiments of the present technology take
advantage of the fact that there is a physical offset between the
first and second sensor arrays that are used to obtain the first
and second image data respectively. More specifically, because the
first sensor array used to obtain the first image data is
physically offset on the satellite from the second sensor array
used to obtain the second image data, a second satellite viewing
angle associated with the second image data obtained using the
second sensor array differs from a first satellite viewing angle
associated with the first image data obtained using the first
sensor array, with a difference between the first and second
satellite viewing angles being a parallax angle. Further, because
the first sensor array used to obtain the first image data is
physically offset on the satellite from the second sensor array
used to obtain the second image data, a second time (e.g., T2) at
which part of the second image data corresponding to a piece (e.g.,
518 in FIG. 5) of the geographic region (e.g., 520 in FIG. 5) is
obtained using the second sensor array differs from a first time
(e.g., T1) at which part of the first image data corresponding to
the same piece (e.g., 518 in FIG. 5) of the geographic region
(e.g., 520 in FIG. 5) is obtained using the first sensor array,
with a difference between the first and second times being a
temporal offset (i.e., temporal offset=T2-T1).
[0003] Certain embodiments of the present technology, described
above, relate to a method for detecting clouds in satellite imagery
captured using first and second sensor arrays that are carried by a
satellite and physically offset from one another on the satellite.
Such a method can include receiving first and second image data
obtained, respectively, using the first and second sensor arrays
carried by the satellite; producing movement mask data based on
dissimilarities identified between the first image data and the
second image data; producing cloud mask data based on spectral
information included in one of the first and second image data;
intersecting the movement mask data with the cloud mask data to
thereby produce intersection data; and producing cloud detection
data based on the intersection data.
[0004] Certain embodiments of the present technology are directed
to a system for detecting clouds in satellite imagery captured
using first and second sensor arrays that are carried by a
satellite and physically offset from one another on the satellite,
the system comprising one or more processors configured to: reduce
movement mask data based first image data and the second image
data, obtained, respectively, using the first and second sensor
arrays carried by the satellite, the first image data including
first image information about a first geographic region, and the
second image data including second image information about the
first geographic region, which differs from the first image
information about the first geographic region included in the first
image data. The one or more processors of the system or further
configured to produce cloud mask data based on spectral information
included in one of the first and second image data, the cloud mask
data indicating where it is likely, based on the spectral
information included in the one of the first and second image data,
that one or more clouds are represented within the one of the first
and second image data; and produce cloud detection data based on
the movement mask data and the cloud mask data, the cloud detection
data indicating where it is likely, based on both the movement mask
data and the cloud mask data, that one or more clouds are
represented within the one of the first and second image data.
Additional details of such a system, which can include or be a
computer system, can be appreciated from the above description.
* * * * *