U.S. patent application number 17/419317 was filed with the patent office on 2022-05-12 for method and device for crop canopy chlorophyll fluorescence three-dimensional distribution information acquisition.
This patent application is currently assigned to JIANGSU UNIVERSITY. The applicant listed for this patent is JIANGSU UNIVERSITY. Invention is credited to Rongrong GU, Pingping LI, Jizhang WANG, Junjie YUAN, Yun ZHANG.
Application Number | 20220146428 17/419317 |
Document ID | / |
Family ID | |
Filed Date | 2022-05-12 |
United States Patent
Application |
20220146428 |
Kind Code |
A1 |
WANG; Jizhang ; et
al. |
May 12, 2022 |
METHOD AND DEVICE FOR CROP CANOPY CHLOROPHYLL FLUORESCENCE
THREE-DIMENSIONAL DISTRIBUTION INFORMATION ACQUISITION
Abstract
A method and a device for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition are
provided. The device includes a Cropobserver canopy chlorophyll
fluorescence detection device, a 3D camera and a computer system.
The 3D camera is connected to the computer system, Visual studio
2017 and MATLAB 2018 are run in the computer system, and the Visual
studio 2017 calls a point cloud library and a computer vision
library to realize three-dimensional visualization of chlorophyll
fluorescence information of crops to be tested. By means of the new
method and the new device, the problem of incompleteness of the
two-dimensional chlorophyll fluorescence information distribution
acquired is solved, overall 3D visual distribution of crop canopy
chlorophyll fluorescence distribution is realized, and important
technical support is provided for acquisition and research of
three-dimensional visual distribution information of chlorophyll
fluorescence of the whole crop canopy.
Inventors: |
WANG; Jizhang; (Zhenjiang,
CN) ; GU; Rongrong; (Zhenjiang, CN) ; ZHANG;
Yun; (Zhenjiang, CN) ; YUAN; Junjie;
(Zhenjiang, CN) ; LI; Pingping; (Zhenjiang,
CN) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
JIANGSU UNIVERSITY |
Zhenjiang |
|
CN |
|
|
Assignee: |
JIANGSU UNIVERSITY
Zhenjiang
CN
|
Appl. No.: |
17/419317 |
Filed: |
January 15, 2021 |
PCT Filed: |
January 15, 2021 |
PCT NO: |
PCT/CN2021/072032 |
371 Date: |
June 29, 2021 |
International
Class: |
G01N 21/64 20060101
G01N021/64; G06T 7/62 20060101 G06T007/62; G06T 7/12 20060101
G06T007/12; G01B 11/03 20060101 G01B011/03; G01B 11/00 20060101
G01B011/00; G06T 7/136 20060101 G06T007/136 |
Foreign Application Data
Date |
Code |
Application Number |
Apr 23, 2020 |
CN |
202010328726.3 |
Claims
1. A method for crop canopy chlorophyll fluorescence
three-dimensional (3D) distribution information acquisition,
comprising: respectively obtaining depth images and mapped color
images of laser dots emitted by a fluorescence-induced laser
emitter on a background plate before and after raising by using a
3D camera, and calibrating the depth images and the mapped color
images to obtain camera intrinsic matrices; obtaining spatial
coordinates of the laser dots based on pixel coordinates of edge
points in the color images and depth values of the edge points in
the depth images in combination with the camera intrinsic matrices;
obtaining a spatial linear equation according to the spatial
coordinates of the laser dots, and solving spatial coordinate
O.sub.2(a, b, c) of an aperture center of the fluorescence-induced
laser emitter relative to the 3D camera; acquiring chlorophyll
fluorescence information of a crop canopy to be tested, mapping dot
sequence number coordinates (g', h') of effective chlorophyll
fluorescence signals to pixel coordinates in the color images and
depth information (u''', v''', d''') of the depth images;
correspondingly characterizing (u''', v''', d''') to spatial
coordinates (x', y', z') using an aperture center of a depth sensor
in the 3D camera as a spatial coordinate origin, and
correspondingly characterizing a chlorophyll fluorescence
information signal sequence of the crop canopy to be tested to
spatial coordinates (x'+a, y'+b, z'+c) using the aperture center of
the fluorescence-induced laser emitter as a spatial coordinate
origin; performing three-dimensional visualization of the
chlorophyll fluorescence information of the crop canopy to be
tested; respectively generating, based on data in Text4-Text6 by
using a pointcloud function for point cloud generation, point
clouds pointcloud-Yield-Kinect, pointcloud-PAR-Kinect, and
pointcloud-rETR-Kinect that comprise spatial coordinates and
chlorophyll fluorescence information and use the aperture center of
the depth sensor as an origin; respectively generating, based on
data in Text7-Text9 by using the pointcloud function for point
cloud generation, point clouds pointcloud-Yield-CropObserver,
pointcloud-PAR-CropObserver, and pointcloud-rETR-CropObserver that
comprise spatial coordinates and chlorophyll fluorescence
information and use the aperture center of the fluorescence-induced
laser emitter as an origin; wherein the Text4 comprises data x ' ,
y ' , z ' , 100 .times. Fv Fm .times. ( 100 .times. F q ' F m ' ) ,
0. ##EQU00026## and 0, the Text5 comprises data x', y', z, 0,
PAR/10, and 0, the Text6 comprises data x', y', z', 0, 0, and rETR,
the Text7 comprises data x ' + a , y ' + b , z ' + c , 100 .times.
Fv Fm .times. ( 100 .times. F q ' F m ' ) , 0 , ##EQU00027## and 0,
the Text8 comprises data x'+a, y'+b, z'+c, 0, PAR/10, and 0, the
Text9 comprises data x'+a, y'+b, z'+c, 0, 0, and rETR; Fv F .times.
m ##EQU00028## is a maximum photochemical efficiency of leaves
under dark adaptation, F q ' F m ' ##EQU00029## is an actual
photochemical efficiency of the leaves under light adaptation, PAR
is a relative light intensity on a leaf surface, and rETR is a
relative electron transfer rate in the leaves; and characterizing
the point clouds comprising the spatial coordinates and the
chlorophyll fluorescence information to a separated green crop
point cloud by using a pcshowpair( ) function, to form a
three-dimensional visual distribution of the chlorophyll
fluorescence information of the crop canopy.
2. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 1, wherein the spatial coordinates of the laser dots are
expressed as M(x, y, z), and { x = ( u - u 0 ) .times. d sf x y = (
v - v 0 ) .times. d sf y x z = d s ##EQU00030## wherein u, v, and d
are pixel coordinates of the laser dots in the image, s is a ratio
of a depth value to an actual depth, f.sub.x and f.sub.y
respectively represent focal lengths of the 3D camera on x axis and
y axis, and (u.sub.0, v.sub.0) are pixel coordinates of an aperture
center of the 3D camera.
3. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 1, wherein the 100 .times. Fv Fm .times. ( 100 .times. F q '
F m ' ) , PAR / 10 , ##EQU00031## and rETR have a value range of
0-255.
4. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 1, wherein dot sequence numbers (g, h) corresponding to pixel
coordinates (u.sub.A1, v.sub.A1), (u.sub.B1, v.sub.B1), (u.sub.C1,
v.sub.C1), and (u.sub.D1, v.sub.D1) of edge points of the mapped
color images are respectively (1, 1), (e, 1), (e, f), and (1, f),
and pixel coordinates, which are corresponding to the dot sequence
numbers (g, h), in the depth images captured by the 3D camera are
recorded as points (u'', v''), wherein
u''=(g-1).DELTA..sub.x+u.sub.D1, and
v''=(h-1).DELTA..sub.y+v.sub.D1, wherein e is a number of dots
generated by a canopy chlorophyll fluorescence detection device in
a row direction, and f is a number of dots generated by the canopy
chlorophyll fluorescence detection device in a column
direction.
5. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 4, wherein
u'''=(g'-1).DELTA..sub.x+u.sub.D1,v'''=(h'-1).DELTA..sub.y+v.sub.D1,
wherein .DELTA..sub.x is a pixel distance between neighboring dots
generated by the fluorescence-induced laser emitter in the row
direction, and .DELTA..sub.y is a pixel distance between
neighboring dots generated by the fluorescence-induced laser
emitter in the column direction.
6. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 5, wherein .DELTA. x = p x e - 1 , and .times. .times.
.DELTA. y = p y f - 1 , ##EQU00032## wherein p.sub.x and p.sub.y
are respectively pixel pitches corresponding to head-to-tail
distances between dots generated by the canopy chlorophyll
fluorescence detection device in the row direction and the column
direction.
7. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 6, wherein p x = ( u A .times. .times. 1 - u B .times.
.times. 1 ) 2 + ( v A .times. .times. 1 - v B .times. .times. 1 ) 2
+ ( u C .times. .times. 1 - u D .times. .times. 1 ) 2 + ( v C
.times. .times. 1 - v D .times. .times. 1 ) 2 ) 2 .
##EQU00033##
8. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 6, wherein p y = ( ( u A .times. .times. 1 - u D .times.
.times. 1 ) 2 + ( v A .times. .times. 1 - v D .times. .times. 1 ) 2
+ ( u B .times. .times. 1 - u C .times. .times. 1 ) 2 + ( v B
.times. .times. 1 - v C .times. .times. 1 ) 2 2 . ##EQU00034##
9. The method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition according to
claim 1, further comprising: acquiring canopy chlorophyll
fluorescence three-dimensional distribution information of
different growth sequences of crops to be tested.
10. A device for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition for the
method according to claim 1, comprising: canopy chlorophyll
fluorescence detection device, a 3D camera and a computer system,
wherein the 3D camera is connected to the computer system, Visual
studio 2017 and MATLAB 2018 are run in the computer system, and the
Visual studio 2017 calls a point cloud library and a computer
vision library to realize three-dimensional visualization of
chlorophyll fluorescence information of crops to be tested.
11. The device according to claim 10, wherein the spatial
coordinates of the laser dots are expressed as M(x, y, z), and { x
= ( u - u 0 ) .times. d sf x y = ( v - v 0 ) .times. d sf y x z = d
s ##EQU00035## wherein u, v, and d are pixel coordinates of the
laser dots in the image, s is a ratio of a depth value to an actual
depth, f.sub.x and f.sub.y respectively represent focal lengths of
the 3D camera on x axis and y axis, and (u.sub.0, v.sub.0) are
pixel coordinates of an aperture center of the 3D camera.
12. The device according to claim 10, wherein the 100 .times. Fv Fm
.times. ( 100 .times. F q ' F m ' ) , PAR / 10 , ##EQU00036## and
rETR have a value range of 0-255.
13. The device according to claim 10, wherein dot sequence numbers
(g, h) corresponding to pixel coordinates (u.sub.A1, v.sub.A1),
(u.sub.B1, v.sub.B1), (u.sub.C1, v.sub.C1), and (u.sub.D1,
v.sub.D1) of edge points of the mapped color images are
respectively (1, 1), (e, 1), (e, f), and (1, f), and pixel
coordinates, which are corresponding to the dot sequence numbers
(g, h), in the depth images captured by the 3D camera are recorded
as points (u'', v''), wherein u''=(g-1).DELTA..sub.x+u.sub.D1, and
v''=(h-1).DELTA..sub.y+v.sub.D1, wherein e is a number of dots
generated by a canopy chlorophyll fluorescence detection device in
a row direction, and f is a number of dots generated by the canopy
chlorophyll fluorescence detection device in a column
direction.
14. The device according to claim 13, wherein
u'''=(g'-1).DELTA..sub.x+u.sub.D1,v'''=(h'-1).DELTA..sub.y+v.sub.D1,
wherein .DELTA..sub.x is a pixel distance between neighboring dots
generated by the fluorescence-induced laser emitter in the row
direction, and .DELTA..sub.y is a pixel distance between
neighboring dots generated by the fluorescence-induced laser
emitter in the column direction.
15. The device according to claim 14, wherein .DELTA. x = p x e - 1
, and .times. .times. .DELTA. y = p y f - 1 , ##EQU00037## wherein
p.sub.x and p.sub.y are respectively pixel pitches corresponding to
head-to-tail distances between dots generated by the canopy
chlorophyll fluorescence detection device in the row direction and
the column direction.
16. The device according to claim 15, wherein p x = ( u A .times.
.times. 1 - u B .times. .times. 1 ) 2 + ( v A .times. .times. 1 - v
B .times. .times. 1 ) 2 + ( u C .times. .times. 1 - u D .times.
.times. 1 ) 2 + ( v C .times. .times. 1 - v D .times. .times. 1 ) 2
) 2 . ##EQU00038##
17. The device according to claim 15, wherein p y = ( ( u A .times.
.times. 1 - u D .times. .times. 1 .times. ) 2 + ( v A .times.
.times. 1 - v D .times. .times. 1 ) 2 + ( u B .times. .times. 1 - u
C .times. .times. 1 ) 2 + ( v B .times. .times. 1 - v C .times.
.times. 1 ) 2 2 . ##EQU00039##
18. The device according to claim 10, further comprising: acquiring
canopy chlorophyll fluorescence three-dimensional distribution
information of different growth sequences of crops to be tested.
Description
CROSS REFERENCE TO THE RELATED APPLICATIONS
[0001] This application is the national phase entry of
International Application No. PCT/CN2021/072032, filed on Jan. 15,
2021, which is based upon and claims priority to Chinese Patent
Application No. 202010328726.3, filed on Apr. 23, 2020, the entire
contents of which are incorporated herein by reference.
TECHNICAL FIELD
[0002] The present invention belongs to the technical field of
chlorophyll fluorescence information acquisition, and in
particular, relates to a method and a device for crop canopy
chlorophyll fluorescence three-dimensional distribution information
acquisition.
BACKGROUND
[0003] The chlorophyll fluorescence analysis technology has the
characteristics of rapid and non-invasive measurement, is a novel
technology for studying the plant growth, and has been widely used
in the field of plant physiological information research. At
present, chlorophyll fluorescence monitoring can only obtain
chlorophyll fluorescence images, and there is an urgent need to
realize the three-dimensional characterization of chlorophyll
fluorescence distribution characteristics on the leaves and the
three-dimensional characterization of the whole canopy. The crop to
be tested is a single leaf or multiple leaves, and the acquired
image is two-dimensional. The health condition of the plant is
identified only from the perspective of leaves, and the
three-dimensional distribution of chlorophyll fluorescence
information of the whole crop canopy cannot be realized.
[0004] In order to acquire the three-dimensional chlorophyll
fluorescence of crops, Chinese Patent Application No. CN106546568A
discloses a method and device for obtaining three-dimensional
chlorophyll fluorescence image information of plants. In this
method, chlorophyll fluorescence image and grayscale image
information of the plant to be tested at different angles are
acquired, chlorophyll fluorescence images and grayscale images of
the plant to be tested are reconstructed using a three-dimensional
reconstruction method, and the three-dimensional fluorescence image
is corrected using a three-dimensional grayscale image, to obtain a
final three-dimensional chlorophyll fluorescence image. However,
this method requires the acquisition of chlorophyll fluorescence
images of the crops from different angles, and cannot realize the
characterization of the chlorophyll fluorescence three-dimensional
distribution of a crop canopy.
[0005] CropObserver, as a novel crop canopy chlorophyll
fluorescence detection device, can detect the chlorophyll
fluorescence of the crop canopy in real time, but can only acquire
two-dimensional data array, failing to localize the chlorophyll
distribution at specific canopy positions.
SUMMARY
[0006] To overcome the drawbacks in the prior art, the present
invention provides a method and a device for crop canopy
chlorophyll fluorescence three-dimensional distribution information
acquisition, to solve the problem that a three-dimensional visual
distribution in crop canopy cannot be realized for crop canopy
chlorophyll fluorescence distribution, providing important
technical support for acquisition and study of three-dimensional
visual distribution information of canopy chlorophyll fluorescence
of the whole crop canopy.
[0007] The above technical object of the present invention is
attained with the following technical means:
[0008] A method for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition,
specifically including:
[0009] respectively obtaining depth images and mapped color images
of laser dots emitted by a fluorescence-induced laser emitter on a
background plate before and after raising by using a 3D camera, and
calibrating the depth images and the mapped color images to obtain
camera intrinsic matrices;
[0010] obtaining spatial coordinates of the laser dots based on
pixel coordinates of edge points in the color images and depth
values of the edge points in the depth images in combination with
the camera intrinsic matrices; obtaining a spatial linear equation
according to the spatial coordinates of the laser dots, and solving
spatial coordinate O.sub.2(a, b, c) of an aperture center of the
fluorescence-induced laser emitter relative to the camera;
[0011] acquiring a chlorophyll fluorescence information of a crop
canopy to be tested, mapping dot sequence number coordinates (g',
h') of effective chlorophyll fluorescence signals to pixel
coordinates in the color images and depth information (u''', v''',
d''') of the depth images; correspondingly characterizing (u''',
v''', d''') to spatial coordinates (x', y', z') using an aperture
center of a depth sensor in the 3D camera as a spatial coordinate
origin, and correspondingly characterizing a chlorophyll
fluorescence information signal sequence of the crop to be tested
to spatial coordinates (x'+a, y'+b, z'+c) using the aperture center
of the fluorescence-induced laser emitter as a spatial coordinate
origin;
[0012] performing three-dimensional visualization of the
chlorophyll fluorescence information of the crop canopy to be
tested: respectively generating, based on data in Text4-Text6 by
using a pointcloud function for point cloud generation, point
clouds pointcloud-Yield-Kinect, pointcloud-PAR-Kinect, and
pointcloud-rETR-Kinect that include spatial coordinates and
chlorophyll fluorescence information and use the aperture center of
the depth sensor as an origin; respectively generating, based on
data in Text7-Text9 by using a pointcloud function for point cloud
generation, point clouds pointcloud-Yield-CropObserver,
pointcloud-PAR-CropObserver, and pointcloud-rETR-CropObserver that
include spatial coordinates and chlorophyll fluorescence
information and use the aperture center of the fluorescence-induced
laser emitter as an origin; where the Text4 includes data
x ' , y ' , z ' , 100 .times. Fv Fm .times. ( 100 .times. F q ' F m
' ) , 0 , ##EQU00001##
and 0, the Text5 includes data x', y', z', 0, PAR/10, and 0, the
Text6 includes data x', y', z', 0, 0, and rETR, the Text7 includes
data
x ' + a , y ' + b , z ' + c , 100 .times. Fv Fm .times. ( 100
.times. F q ' F m ' ) , 0 , ##EQU00002##
and 0, the Text8 includes data x'+a, y'+b, z'+c, 0, PAR/10, and 0,
the Text9 includes data x'+a, y'+b, z'+c, 0, 0, and rETR;
Fv Fm ##EQU00003##
is maximum photochemical efficiency of leaves under dark
adaptation,
F q ' F m ' ##EQU00004##
is actual photochemical efficiency of leaves under light
adaptation, PAR is a relative light intensity on the leaf surface,
and rETR is a relative electron transfer rate in leaves; and
[0013] characterizing the point clouds including the spatial
coordinates and the chlorophyll fluorescence information to a
separated green crop point cloud by using a pcshowpair( ) function,
to form a three-dimensional visual distribution of the chlorophyll
fluorescence information on the crop canopy.
[0014] Further, the spatial coordinates of the laser dots are
expressed as M(x, y, z), and
{ x = ( u - u 0 ) .times. d sf x y = ( v - v 0 ) .times. d sf y x z
= d s ##EQU00005##
[0015] where u, v, and d are pixel coordinates of the laser dots in
the image, s is a ratio of a depth value to an actual depth,
f.sub.x and f.sub.y respectively represent focal lengths of the
camera on x axis and y axis, and (u.sub.0, v.sub.0) are pixel
coordinates of an aperture center of the camera.
[0016] Further,
100 .times. Fv Fm .times. ( 100 .times. F q ' F m ' ) , PAR / 1
.times. 0 , ##EQU00006##
and rETR have a value range of 0-255.
[0017] Further, dot sequence numbers (g, h) corresponding to pixel
coordinates (u.sub.A1, v.sub.A1), (u.sub.B1, v.sub.B1), (u.sub.C1,
v.sub.C1), and (u.sub.D1, v.sub.D1) of edge points of the mapped
color images are respectively (1, 1), (e, 1), (e, f), and (1, f),
and pixel coordinates, which are corresponding to the dot sequence
numbers (g, h), in the depth images captured by the camera are
recorded as points (u'', v''), where
u''=(g-1).DELTA..sub.x+u.sub.D1, and
v''=(h-1).DELTA..sub.y+v.sub.D1, where e is the number of dots
generated by the canopy chlorophyll fluorescence detection device
in a row direction, and f is the number of dots generated by the
canopy chlorophyll fluorescence detection device in a column
direction.
[0018] Still further, u'''=(g'-1).DELTA..sub.x+u.sub.D1,
v'''=(h'-1).DELTA..sub.y+v.sub.D1, where .DELTA..sub.x is a pixel
distance between neighboring dots generated by the
fluorescence-induced laser emitter in the row direction, and
.DELTA..sub.y is a pixel distance between neighboring dots
generated by the fluorescence-induced laser emitter in the column
direction.
[0019] Still further,
.DELTA. x = p x e - 1 , and .times. .times. .DELTA. y = p y f - 1 ,
##EQU00007##
where p.sub.x and p.sub.y are respectively pixel pitches
corresponding to head-to-tail distances between dots generated by
the canopy chlorophyll fluorescence detection device in the row
direction and the column direction.
[0020] Still further,
p x = ( u A .times. .times. 1 - u B .times. .times. 1 ) 2 + ( v A
.times. 1 - v B .times. .times. 1 ) 2 + ( u C .times. 1 - u D
.times. .times. 1 ) + ( v C .times. .times. 1 - v D .times. .times.
1 ) 2 ) 2 . ##EQU00008##
[0021] Still further,
p y = ( ( u A .times. .times. 1 - u D .times. .times. 1 ) 2 + ( v A
.times. .times. 1 - v D .times. .times. 1 ) 2 + ( u B .times.
.times. 1 - u C .times. .times. 1 ) 2 .times. ( v B .times. .times.
1 - v C .times. .times. 1 ) 2 2 . ##EQU00009##
[0022] Further, the method further includes acquiring canopy
chlorophyll fluorescence three-dimensional distribution information
of different growth sequences of the crops to be tested.
[0023] A device for crop canopy chlorophyll fluorescence
three-dimensional distribution information acquisition, including a
canopy chlorophyll fluorescence detection device, a 3D camera and a
computer system, where the 3D camera is connected to the computer
system, Visual studio 2017 and MATLAB 2018 are run in the computer
system, and the Visual studio 2017 calls a point cloud library and
a computer vision library to realize three-dimensional
visualization of chlorophyll fluorescence information of crops to
be tested.
[0024] Compared with the prior art, the present invention has the
following beneficial effects.
[0025] (1) According to the present invention, crop canopy
chlorophyll fluorescence two-dimensional information of the crop
canopy chlorophyll fluorescence detection device CropObserver is
combined with the three-dimensional imaging technology of the 3D
camera, and the distribution of two-dimensional chlorophyll
fluorescence information obtained by the CropObserver in the crop
space is realized by establishing a relationship between relative
spatial coordinates of the 3D camera and the aperture center of the
chlorophyll fluorescence sensor of the CropObserver. This overcomes
the limitations of the method of measuring the distribution of
chlorophyll fluorescence information from a single leaf or multiple
leaves, and realizes the characterization of three-dimensional
distribution of chlorophyll fluorescence information of the crop
canopy.
[0026] (2) The present invention uses the pointcloud function for
point cloud generation to generate point clouds that include
spatial coordinates and chlorophyll fluorescence information and
use the aperture center of the depth sensor as an origin, and
generate point clouds that include spatial coordinates and
chlorophyll fluorescence information and use the aperture center of
the fluorescence-induced laser emitter as an origin, thereby
solving the problem of incompleteness of the two-dimensional
chlorophyll fluorescence information distribution acquired, and
realizing overall 3D visual distribution of crop canopy chlorophyll
fluorescence distribution.
[0027] (3) According to the present invention, chlorophyll
fluorescence information of different growth sequences of the crop
canopy is acquired, thereby solving the problem of missing of
information about the bottom of the crop canopy due to blocking by
the top leaves.
BRIEF DESCRIPTION OF THE DRAWINGS
[0028] FIG. 1 is a schematic diagram of a device for crop canopy
chlorophyll fluorescence three-dimensional distribution information
acquisition according to the present invention.
[0029] FIG. 2 is a flowchart of a method for crop canopy
chlorophyll fluorescence three-dimensional distribution information
acquisition according to the present invention.
[0030] FIG. 3A and FIG. 3B are schematic diagrams showing position
calibration and point selection of CropObserver and a 3D camera
according to the present invention, where FIG. 3A is a diagram
showing a position calibration and point selection process of
CropObserver and the 3D camera, and FIG. 3B is a diagram showing a
position calibration and point selection result of CropObserver and
the 3D camera.
[0031] FIG. 4 is a schematic diagram of a model structure of a 3D
camera according to the present invention.
[0032] In the drawings: 1--canopy chlorophyll fluorescence
detection device, 1-1--fluorescence-induced laser emitter,
1-2--LI-COR optical quantum sensor, 1-3--chlorophyll fluorescence
sensor, 1-4--HDMI port, 1-5--24V power input port, 1-6--USB3.0
port, 1-7--voltage converter, 1-8--iron chain, 2--3D camera,
3--triangular support, 4--computer system, 5--display, 6--mobile
storage device, 7--crop to be tested, 8--black background plate,
9--movable rack, 10--universal wheel.
DETAILED DESCRIPTION OF THE EMBODIMENTS
[0033] The present invention will be further described in detail
below with reference to drawings and embodiments, but the
protection scope of the present invention is not limited
thereto.
[0034] As shown in FIG. 1, a device for crop canopy chlorophyll
fluorescence three-dimensional distribution information acquisition
includes a canopy chlorophyll fluorescence detection device 1, a 3D
camera 2, a triangular support 3, a computer system 4, a display 5,
a mobile storage device 6, a crop to be tested 7, a black
background plate 8, a movable rack 9, and a universal wheel 10. The
canopy chlorophyll fluorescence detection device 1 is suspended
below the movable rack 9 by four iron chains 1-8, and the movable
rack 9 is capable of moving to right above the crop to be tested 7.
The crop to be tested 7 is placed on the black background plate 8.
The 3D camera 2 is located obliquely above the crop to be tested 7,
and is mounted on the triangular support 3, and is connected to the
computer system 4. The canopy chlorophyll fluorescence detection
device 1 includes: a LI-COR optical quantum sensor 1-2 disposed at
the top and configured to measure a light intensity, a chlorophyll
fluorescence sensor 1-3 disposed at the bottom and configured to
acquire chlorophyll fluorescence data reflected by the crop to be
tested, a fluorescence-induced laser emitter 1-1 disposed at the
bottom and configured to use a short pulse laser to excite
chlorophyll fluorescence of the crop to be tested 7, an HDMI port
1-4, a 24V power input port 1-5, a USB3.0 port 1-6, and a voltage
converter 1-7. The 24V power input port 1-5 is connected to the
mains through the voltage converter 1-7, and the HDMI port 1-4 is
connected to the display 5, to display a fluorescence parameter
monitoring interface and a fluorescence information acquisition
manner setting interface. When the canopy chlorophyll fluorescence
detection device 1 operates, the fluorescence-induced laser emitter
1-1 emits a short pulse laser to the crop to be tested 7. The
LI-COR optical quantum sensor 1-2 measures a light intensity of an
environment around the crop to be tested 7. The chlorophyll
fluorescence sensor 1-3 acquires chlorophyll fluorescence data
reflected by the crop canopy to be tested 7. The canopy chlorophyll
fluorescence detection device 1 saves the acquired light intensity
of the environment around the crop to be tested 7 and the acquired
chlorophyll fluorescence data, is connected to the mobile storage
device 6 through the USB3.0 port 1-6, and copies the saved data to
the computer system 4. Universal wheels 10 each having a locking
paddle are mounted below the movable rack 9 to allow the track to
move or lock the rack.
[0035] In this example, the crop to be tested 7 is cucumber, the
canopy chlorophyll fluorescence detection device 1 is the
Cropobserver canopy chlorophyll fluorescence detection device
manufactured by Phenotrait, the Netherlands, the 3D camera 2 is
Microsoft's Kinect V2 depth camera, and the computer system 4 is
Windows 10 system. The information acquisition control function of
the 3D camera 2 implements the acquisition of color images and
depth images in Visual studio 2017, the calibration function of the
3D camera 2 is implemented in MATLAB 2018 by a checkerboard
calibration kit based on the principle of Zhengyou Zhang's
calibration, and the point cloud acquisition function and the
chlorophyll fluorescence information characterization visualization
function of the 3D camera 2 are implemented by calling a point
cloud library (PCL) and a computer vision library (Open Source
Computer Vision Library, OpenCV) in Visual studio 2017. Visual
studio 2017 and MATLAB 2018 are software running in the computer
system 4.
[0036] As shown in FIG. 2, a method for crop canopy chlorophyll
fluorescence three-dimensional distribution information acquisition
specifically includes the following steps:
[0037] Step 1. The canopy chlorophyll fluorescence detection device
1 is disposed.
[0038] The movable rack 9 carries the canopy chlorophyll
fluorescence detection device 1 to move to the top of the crop
canopy to be tested 7. After initialization of the measurement
device, "Centre" and "Test Meas" buttons in the setting interface
are successively pressed, to cause laser dots emitted from the
fluorescence-induced laser emitter 1-1 to point to the center of
the crop to be tested 7. A measurement range is set, so that a
range of dots generated by the fluorescence-induced laser emitter
1-1 surrounds the crop to be tested 7. As shown in FIG. 3A and FIG.
3B, the fluorescence-induced laser emitter 1-1 projects four laser
dots at the edge. Because the full scanning range of the
fluorescence-induced laser emitter is 34.degree. in a row direction
and 40.degree. in a column direction, a scanning angle ratio of the
horizontal axis is set to m.sub.1, and a scanning angle ratio of
the longitudinal axis is set to m.sub.2, so that the four points at
the edge can surround the crop to be tested 7. A number of dots in
an array generated by the fluorescence-induced laser emitter 1-1 is
set, where a number of dots generated by the canopy chlorophyll
fluorescence detection device 1 in the row direction is set to e,
and a number of dots generated in the column direction is set to f.
"Start scan" is clicked to cause the canopy chlorophyll
fluorescence detection device 1 to start operating.
[0039] Step 2. Image acquisition and calibration of the 3D camera
2.
[0040] In this example, the 3D camera 2 has a color sensor
resolution of 1920.times.1080, and a depth sensor resolution of
512.times.424. Visual studio 2017 is run in the computer system 4,
and a computer vision library (Open Source Computer Vision Library,
OpenCV) and a camera SDK are called, to respectively acquire depth
frames data to arrays (DepthFrameDate) and color frames data to
arrays (ColorSpacePoint), and respectively saves them as a depth
image and a color image. A MapDepthFrameToColorSpace( ) function
based on the principle of bilinear interpolation is used to
calculate a mapping relationship between the depth image and the
color image according to depth frame information, and pixel
coordinates in the depth image are mapped to the color image, so
that coordinates of pixels in the depth image are mapped to
coordinates in the color image, to obtain a 512*424 array. Elements
of the array are coordinates in the color image that correspond to
the depth image, and contain color information. The array is saved
as a mapped color image.
[0041] Mapped color images corresponding to different positions of
a checkerboard calibration plate are acquired. The mapped color
images are input to Zhengyou Zhang's calibration toolkit in MATLAB
2018. A corner distance of the checkerboard is input. Valid
calibration pictures are screened to obtain n checkerboard images
with a calibration plot error of less than 0.2 pixels for
calibration, where n>20. Then a camera intrinsic matrix is
exported:
Intrinsic .times. .times. Matrix = [ f x 0 c x 0 f y c y 0 0 1 ] (
1 ) ##EQU00010##
where f.sub.x=f*sx,f.sub.y=f*sy,f is the focal length of the camera
(measured in mm), [sx, sy] represents the number of pixels per
millimeter in the (x, y) direction, f.sub.x and f.sub.y
respectively represent focal lengths of the camera on the x axis
and the y axis (measured in pixels), and [c.sub.x, c.sub.y] is the
aperture center of the camera.
[0042] Step 3. Calibration point information capture between the
canopy chlorophyll fluorescence detection device 1 and the 3D
camera.
[0043] As shown in FIG. 3A and FIG. 3B, the fluorescence-induced
laser emitter 1-1 emits four laser dots at the edge. Four red laser
dots A.sub.1, B.sub.1, C.sub.1, and D.sub.1 at the edge are shown
on the background plate 8. The camera 2 sequentially captures depth
images and mapped color images of the laser dots A.sub.1, B.sub.1,
C.sub.1, and D.sub.1. The background plate 8 is raised by h meters,
and red laser dots A.sub.2, B.sub.2, C.sub.2, and D.sub.2 at the
edge are shown on the background plate 8. The camera 2 sequentially
captures depth images and mapped color images of the laser dots
A.sub.2, B.sub.2, C.sub.2, and D.sub.2. Based on the calibration
method for the camera 2 in step 2, the camera 2 is respectively
calibrated before and after the background plate 8 is raised, to
obtain intrinsic matrices: Intrinsic Matrix-a and Intrinsic
Matrix-b.
[0044] Step 4. World coordinates of calibration edge points are
extracted.
[0045] A model structure of the 3D camera 2 is as shown in FIG. 4.
In FIG. 4, O.sub.0-uv is a pixel coordinate plane of the depth
image, and an origin O.sub.0 is at the upper left corner of the
imaging plane of the depth image, measured in pixels; O.sub.1-xy is
an image coordinate system, with an origin O.sub.1 being in the
center of the imaging plane of the depth image, measured in mm, and
coordinates of O.sub.1 in the pixel coordinate plane are (u.sub.0,
v.sub.0); O.sub.c-X.sub.cY.sub.cZ.sub.c is a coordinate system of
the 3D camera 2, with an origin O.sub.c being at an aperture center
of the depth sensor in the 3D camera; a distance between O.sub.c
and O.sub.1 is the focal length f. Because the pixel size of the
depth image captured by the camera 2 is 512.times.424, the camera 2
has different focal lengths on the x axis and the y axis, which are
respectively recorded as f.sub.x and f.sub.y. In this example, the
origin of coordinates of the camera coordinate system coincides
with the origin of coordinates of the world coordinate system.
O.sub.w-X.sub.wY.sub.wZ.sub.w is the world coordinate system of the
camera 2, and coincides with the camera coordinate system
O.sub.c-X.sub.cY.sub.cZ.sub.c. A point m is any point in the depth
image, and its coordinates in the pixel coordinate plane are (u,
v). The point m is mapped to three-dimensional coordinates
M(x.sub.c, y.sub.c, z.sub.c) in the camera coordinate system, where
z.sub.c represents a principal axis value of camera coordinates,
i.e., the distance from a target to the camera. The point m is
mapped to three-dimensional coordinate M(x.sub.w, y.sub.w, z.sub.w)
in the world coordinate system. Because the processing process is
separated performed on a single depth image, the origin of world
coordinates coincides with the origin of the camera, so the
coordinates of point M are recorded as M(x, y, z). According to the
geometrical relationship in FIG. 4, a correspondence between the
spatial point M(x, y, z) and the pixel coordinates m(u, v, d)
(where d refers to depth data in the depth image) of this point in
the image is:
{ u = xf x z + u 0 v = xf y z + v 0 d = z .times. s ( 2 )
##EQU00011##
[0046] where (u.sub.0, v.sub.0) is the pixel coordinates of the
aperture center of the camera; and s is a scaling factor, i.e., a
ratio of the depth value to an actual application, and s is
generally set to 1000.
[0047] A back calculation formula (2) may be written as follows:
when a point m(u, v, d) is known, a corresponding spatial
coordinate point M(x, y, z) is derived:
{ x = ( u - u 0 ) .times. d sf x y = ( v - v 0 ) .times. d sf y x z
= d s ( 3 ) ##EQU00012##
[0048] The depth images and the mapped color images of the four red
laser dots A.sub.1, B.sub.1, C.sub.1, and D.sub.1 at the edge and
the four red laser dots A.sub.2, B.sub.2, C.sub.2, and D.sub.2 at
the edge that are acquired in step 3 are imported into Matlab 2018.
For the color image, the color image is grayed using a super red
grayscale factor 2R-G-B (where R, G, and B are three color channel
components: red, green, and blue), to acquire red characteristics
of the laser dots at the edge, and obtain clear edge points. The
pixel coordinates of the edge points are extracted: (u.sub.A1,
v.sub.A1), (u.sub.B1, v.sub.B1), . . . , and (u.sub.D2, v.sub.D2).
(u.sub.A1, v.sub.A1), (u.sub.B1, v.sub.B1), . . . , and (u.sub.D2,
v.sub.D2) are mapped to the depth image, to obtain depths d.sub.A1,
d.sub.B1, . . . , and d.sub.D2. With reference to the intrinsic
matrices Intrinsic Matrix-a and Intrinsic Matrix-b in step 3,
spatial coordinate points of A.sub.1, B.sub.1, C.sub.1, D.sub.1,
A.sub.2, B.sub.2, C.sub.2, and D.sub.2 are acquired: (x.sub.A1,
y.sub.A1, z.sub.A1), (x.sub.B1, y.sub.B1, z.sub.B1), . . . , and
(x.sub.D2, y.sub.D2, z.sub.D2).
[0049] Step 5. A spatial position of an aperture center O.sub.2 of
the fluorescence-induced laser emitter relative to the 3D camera 2
is calibrated.
[0050] According to the coordinates of the spatial coordinate
points A.sub.1, B.sub.1, C.sub.1, D.sub.1, A.sub.2, B.sub.2,
C.sub.2, and D.sub.2 acquired in step 4, spatial linear equations
passing through A.sub.1 A.sub.2, B.sub.1 B.sub.2, C.sub.1 C.sub.2,
and D.sub.1 D.sub.2 are set up, which are respectively recorded as
straight lines l.sub.1, l.sub.2, l.sub.3, and l.sub.4. Assume that
the linear equations areas follows:
{ l 1 .times. : .times. ( x A .times. 1 - x A .times. .times. 2 )
.times. x + ( y A .times. .times. 1 - y A .times. .times. 2 )
.times. y + ( z A .times. .times. 1 - z A .times. .times. 2 )
.times. z + N 1 = 0 l 2 .times. : .times. ( x B .times. .times. 1 -
x B .times. .times. 2 ) .times. x + ( y B .times. .times. 1 - y B
.times. .times. 2 ) .times. y + ( z B .times. .times. 1 - z B
.times. .times. 2 ) .times. z + N 2 = 0 l 3 .times. : .times. ( x C
.times. .times. 1 - x C .times. .times. 2 ) .times. x + ( y C
.times. .times. 1 - y C .times. .times. 2 ) .times. y + ( z C
.times. .times. 1 - z C .times. .times. 2 ) .times. z + N 3 = 0 l 4
.times. : .times. ( x D .times. .times. 1 - x D .times. .times. 2 )
.times. x + ( y D .times. .times. 1 - y D .times. .times. 2 )
.times. y + ( z D .times. .times. 1 - z D .times. .times. 2 )
.times. z + N 4 = 0 ( 4 ) ##EQU00013##
[0051] In the above linear equations, N.sub.1, N.sub.2, N.sub.3,
and N.sub.4 are constants. By substituting points A.sub.1, B.sub.1,
C.sub.1, and D.sub.1 into the straight lines l.sub.1, l.sub.2,
l.sub.3, and l.sub.4 respectively, the constants N.sub.1, N.sub.2,
N.sub.3, and N.sub.4 can be calculated, and then the four linear
equations l.sub.1, l.sub.2, l.sub.3, and l.sub.4 can be solved.
[0052] Because the points A.sub.1, B.sub.1, C.sub.1, D.sub.1,
A.sub.2, B.sub.2, C.sub.2, and D.sub.2 are emitted from the
aperture center O.sub.2 of the fluorescence-induced laser emitter,
all the straight lines l.sub.1, l.sub.2, l.sub.3, and l.sub.4 pass
through the point O.sub.2. It is assumed that the spatial
coordinates of the aperture center of the fluorescence-induced
laser emitter relative to the camera are O.sub.2(a, b, c),
including three unknowns a, b, and c. By substitution into any
three of the above linear equations, the spatial coordinates
O.sub.2(a, b, c) of the aperture center of the fluorescence-induced
laser emitter relative to the camera can be calculated.
[0053] Step 6. The canopy chlorophyll fluorescence detection device
1 acquires chlorophyll fluorescence information of the crop to be
tested.
[0054] The number of dots generated by the Cropobserver in the
x-axis direction is set to e=50, and that in the y-axis direction
is set to f=50, a boundary for dots generated by the
fluorescence-induced laser emitter 1-1 is A.sub.1, B.sub.1,
C.sub.1, and D.sub.1, and a 50.times.50 dot array is formed, where
sequence numbers in the array are recorded as (g, h)
(1.ltoreq.g.ltoreq.50, 1.ltoreq.h.ltoreq.50). Neighboring dots are
spaced from each other in the row direction by the same distance,
which is recorded as a constant n.sub.1. Neighboring dots are
spaced from each other in the column direction by the same
distance, which is recorded as a constant n.sub.2. The value of
n 1 n 2 ##EQU00014##
equals to a ratio
m 1 m 2 ##EQU00015##
between scanning angle ratios in the row direction and the column
direction. "Start scan" is clicked to start measurement. The
fluorescence-induced laser emitter 1-1 generates dots in the
following order: first generating a dot at point A.sub.1, the
sequence number of the dot being recorded as (1, 1); generating 49
dots in a direction toward point D.sub.1 at equal intervals of
n.sub.1, the sequence number of the dot at point D.sub.1 being
recorded as (1, 50); then generating a dot at a position that is
distant from point A.sub.1 downward by a distance n.sub.2, the
sequence number of the dot being recorded as (2, 1); then
generating 49 dots in a direction toward point D.sub.1 downward by
a distance n.sub.2, the sequence number of the dot that is distant
from D.sub.1 downward by n.sub.2 being recorded as (2, 50); then
generating a dot at a position that is distant from point A.sub.1
downward by a distance 2n.sub.2, the sequence number of the dot
being recorded as (3, 1), and then generating 49 dots in a
direction toward point D.sub.1 downward by a distance 2n.sub.2 at
intervals of n.sub.1, the sequence number of the dot that is
distant from D.sub.1 downward by 2n.sub.2 being recorded as (3,
50); and so on. Dots are generated in sequence based on the above
rule, the sequence number of the dot at point B.sub.1 being
recorded as (50, 1). Finally, a dot is generated at point C.sub.1,
the sequence number of the dot at point C, being recorded as (50,
50). The position of the measurement point is changed every 5
seconds, and the chlorophyll fluorescence sensor 1-3 acquires and
stores a position at which the crop to be tested 7 reflects
chlorophyll fluorescence and fluorescence data of this
position.
[0055] The canopy chlorophyll fluorescence detection device 1
mainly measures the following parameters: (1) photochemical
efficiency: maximum photochemical efficiency
Fv Fm ##EQU00016##
of leaves under dark adaptation, and actual photochemical
efficiency
F q ' F m ' ##EQU00017##
of leaves under light adaptation; (2) PAR: relative light intensity
on the leaf surface; (3) rETR: relative electron transfer rate in
leaves. Fv=Fm-F.sub.0, where Fm is maximum chlorophyll fluorescence
measured under dark adaptation conditions, and F.sub.0 is an
initial value of the chlorophyll fluorescence parameter measured
under dark adaptation conditions; F.sub.q'=F.sub.m'-F.sub.t, where
F.sub.m' is maximum fluorescence under light adaptation, i.e., a
fluorescence intensity when all PSII reaction centers are closed
under light adaptation, and F.sub.t is real-time fluorescence of
the crop after receiving light for a period of time t; the relative
electron transfer rate
rETR=0.425.times.(F.sub.q'/F.sub.m').times.PAR. When the canopy
chlorophyll fluorescence detection device 1 operates, the computer
system 4 captures depth images and mapped color images of the crop
to be tested 7 using the 3D camera 2, where the depth images
including pixel and depth information are expressed as (u',v',d'),
and the color images including three color channels, red r', green
g', and blue b', are expressed as (u',v',r',g',b').
[0056] Step 7. The depth images and the mapped color images of the
crop to be tested are converted to point clouds for displaying.
[0057] A point cloud library (PCL) and a computer vision library
(Open Source Computer Vision Library, OpenCV) are called in Visual
studio 2017, and by traversing (u',v',d') acquired in step 6, the
crop depth images are converted based on formula (3) into spatial
coordinate points (X, Y, Z), which are saved in a matrix XYZ of
three columns, respectively named X, Y, and Z. The three color
channel components red, green, and blue of (r', g', b') acquired in
step 6 are separated to form three channel components r, g, and b,
which are respectively saved in a matrix RGB of three columns,
respectively named R, G, and B. Point cloud plots are generated
from the matrix components X, Y, Z, R, G, and B by using a
pointcloud function for point cloud generation.
[0058] Step 8. Segmentation is performed for the crop canopy to be
tested.
[0059] The point cloud plots in step 7 also contain background
point cloud information in addition to the crop to be tested. The
point cloud plots in step 6 are processed using a super green
grayscale operation (2R-G-B), to highlight the green crop point
cloud part. A binarization thresholding operator THRESH_OTSU in
OpenCV is used for thresholding, to separate the green crop point
cloud.
[0060] Step 9. Dot sequence number coordinates of effective
chlorophyll fluorescence signals are mapped to pixel coordinates in
the depth images and the mapped color images.
[0061] The sequence numbers (g, h) of the dots generated by the
canopy chlorophyll fluorescence detection device 1 in step 6
corresponding to the pixel coordinates (u.sub.A1, v.sub.A1),
(u.sub.B1, v.sub.B1), (u.sub.C1, v.sub.C1), and (u.sub.D1,
v.sub.D1) of the edge points in step 4 are respectively (1, 1), (e,
1), (e, f), and (1, f), and pixel pitches p.sub.x and p.sub.y
corresponding to head-to-tail distances between the dots generated
by the canopy chlorophyll fluorescence detection device 1 in the
row direction and the column direction are respectively:
p x = ( u A .times. .times. 1 - u B .times. .times. 1 ) 2 + ( v A
.times. 1 - v B .times. .times. 1 ) 2 + ( u C .times. 1 - u D
.times. .times. 1 ) + ( v C .times. .times. 1 - v D .times. .times.
1 ) 2 ) 2 ##EQU00018## and ##EQU00018.2## p y = ( ( u A .times.
.times. 1 - u D .times. .times. 1 ) 2 + ( v A .times. 1 - v D
.times. .times. 1 ) 2 + ( u B .times. .times. 1 - u D .times.
.times. 1 ) + ( v C .times. .times. 1 - v D .times. .times. 1 ) 2 2
, ##EQU00018.3##
where the dot sequence numbers (g, h) are evenly distributed in the
pixel coordinate plane. The pixel pitch between neighboring dots
generated by the fluorescence-induced laser emitter in the row
direction is recorded as .DELTA..sub.x, and the pixel pitch between
neighboring dots generated by the fluorescence-induced laser
emitter in the column direction is recorded as .DELTA..sub.y.
Assuming that a dot array generated by the fluorescence-induced
laser emitter is
e .times. f , .DELTA. x = p x e - 1 , .times. and ##EQU00019##
.DELTA. y = p y f - 1 . ##EQU00019.2##
The pixel coordinates in the depth images captured by the camera
corresponding to the dot sequence numbers (g, h) are recorded as
points (u'', v''), where u''=(g-1).DELTA..sub.x+u.sub.D1, and
v''=(h-1).DELTA..sub.y+V.sub.D1. A sequence number of a dot with a
chlorophyll fluorescence signal recorded in a cycle of the canopy
chlorophyll fluorescence detection device 1 is found. Coordinate
information and chlorophyll fluorescence information of this
sequence number are sequentially saved in a row in Text1 in the
following order:
g ' , h ' , F .times. v Fm .times. ( F q ' F m ' ) , PAR , rETR .
##EQU00020##
The depth images corresponding to (g', h') are (u''',v''',d'''),
u'''=(g'-1).DELTA..sub.x+u.sub.D1, and
v'''=(h'-1).DELTA..sub.y+v.sub.D1. The pixel coordinates, depth
information and chlorophyll fluorescence information of the depth
image are sequentially saved in a row in Text2 in the following
order:
u ''' , v ''' , d ''' , F .times. v Fm .times. ( F q ' F m ' ) ,
PAR , rETR . ##EQU00021##
[0062] Step 10. A chlorophyll fluorescence information signal
sequence of the crop canopy to be tested are correspondingly
characterized to spatial coordinates using the aperture center of
the depth sensor as a spatial coordinate origin.
[0063] Based on the coordinate conversion method in formula (3),
the first three columns of pixel and depth coordinates (u''', v''',
d''') in Text2 are converted into spatial coordinates (x', y', z')
using the aperture center of the depth sensor as a spatial
coordinate origin, which are sequentially saved, together with the
last three columns in Text2, in a row in Text3 in the following
order:
x ' , y ' , z ' , F .times. v Fm .times. ( F q ' F m ' ) , PAR ,
rETR . ##EQU00022##
[0064] Step 11. A chlorophyll fluorescence information signal
sequence of the crop canopy to be tested are correspondingly
characterized to spatial coordinates using the aperture center of
the fluorescence-induced laser emitter as a spatial coordinate
origin.
[0065] According to the spatial coordinates (x', y', z') using the
aperture center of the depth sensor as the spatial coordinate
origin in step 10, the spatial coordinates O.sub.2(a, b, c) of the
aperture center of the fluorescence-induced laser emitter relative
to the camera have been obtained in step 5, and thus spatial
coordinates of effective chlorophyll fluorescence signals using the
aperture center of the fluorescence-induced laser emitter as the
origin of space are (x'+a, y'+b, z'+c).
[0066] Step 12. Three-dimensional visualization of the canopy
chlorophyll fluorescence information of the crop to be tested is
performed.
[0067] The last three columns of data in Text3 are converted
into
100 .times. Fv Fm .times. ( 100 .times. F q ' F m ' ) , PAR / 10 ,
##EQU00023##
and rETR, so that their value ranges are 0 to 255, i.e., the three
columns of chlorophyll fluorescence information data fall within
value ranges of the red, green, and blue color channels. Data is
sequentially saved in a row in Text4 in the following order: x',
y', z',
100 .times. Fv Fm .times. ( 100 .times. F q ' F m ' ) , 0 , 0 .
##EQU00024##
Data is sequentially saved in a row in Text5 in the following
order: x', y', z', 0, PAR/10, 0. Data is sequentially saved in a
row in Text6 in the following order: x', y', z', 0, 0, rETR. Data
is sequentially saved in a row in Text7 in the following order:
x ' + a , y ' + b , z ' + c , 100 .times. Fv Fm .times. ( 100
.times. F q ' F m ' ) , 0 , 0. ##EQU00025##
Data is sequentially saved in a row in Text8 in the following
order: x'+a, y'+b, z'+c, 0, PAR/10, 0. Data is sequentially saved
in a row in Text9 in the following order: x'+a, y'+b, z'+c, 0, 0,
rETR. The PCL and the OpenCV are called in Visual studio 2017.
Point clouds pointcloud-Yield-Kinect, pointcloud-PAR-Kinect, and
pointcloud-rETR-Kinect that include spatial coordinates and
chlorophyll fluorescence information and use the aperture center of
the depth sensor as an origin are respectively generated based on
data in Text4-Text6 by using a pointcloud function for point cloud
generation. Point clouds pointcloud-Yield-CropObserver,
pointcloud-PAR-CropObserver, and pointcloud-rETR-CropObserver that
include spatial coordinates and chlorophyll fluorescence
information and use the aperture center of the fluorescence-induced
laser emitter as an origin are respectively generated based on data
in Text7-Text9 by using a pointcloud function for point cloud
generation.
[0068] The point clouds including the spatial coordinates and the
chlorophyll fluorescence information are characterized to the green
crop point cloud separated in step 8 by using a pcshowpair( )
function, to form a three-dimensional visual distribution of the
chlorophyll fluorescence information on the plant.
[0069] Step 13. Crop canopy chlorophyll fluorescence
three-dimensional point cloud distribution information is acquired
for different growth sequences of the crop to be tested. In this
example, the growth of the cucumber crop is divided into a
germination period, a seedling period, a flowering period and a
fruiting period, and three-dimensional point cloud distribution
information of the chlorophyll fluorescence of the crop canopy to
be tested is acquired by performing steps 1 to 12.
[0070] The embodiments are exemplary embodiments of the present
invention, but the present invention is not limited to the
above-mentioned embodiments. Any obvious improvement, replacement
or variation that can be made by one skilled in the art without
departing from the spirit of the present invention belongs to the
protection scope of the present invention.
* * * * *