U.S. patent application number 17/520136 was filed with the patent office on 2022-09-29 for hyper temporal lidar with controllable pulse bursts to resolve angle to target.
The applicant listed for this patent is AEYE, Inc.. Invention is credited to Joel Benscoter, Luis Dussan, Philippe Feru, Jordan Greene, Alex Liang, Igor Polishchuk, Allan Steinhardt.
Application Number | 20220308171 17/520136 |
Document ID | / |
Family ID | 1000005999433 |
Filed Date | 2022-09-29 |
United States Patent
Application |
20220308171 |
Kind Code |
A1 |
Greene; Jordan ; et
al. |
September 29, 2022 |
Hyper Temporal Lidar with Controllable Pulse Bursts to Resolve
Angle to Target
Abstract
A lidar system can include a lidar transmitter and a lidar
receiver, where the lidar transmitter controllably transmits a
pulse burst toward a target in a field of view and where the lidar
receiver resolves an angle to the target based on returns from the
pulse burst. The pulse burst can include a first pulse fired at a
first shot angle and a second pulse fired at a second shot
angle.
Inventors: |
Greene; Jordan; (Dublin,
CA) ; Benscoter; Joel; (Dublin, CA) ; Dussan;
Luis; (Dublin, CA) ; Steinhardt; Allan;
(Dublin, CA) ; Feru; Philippe; (Dublin, CA)
; Polishchuk; Igor; (Dublin, CA) ; Liang;
Alex; (Dublin, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
AEYE, Inc. |
Dublin |
CA |
US |
|
|
Family ID: |
1000005999433 |
Appl. No.: |
17/520136 |
Filed: |
November 5, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
63209179 |
Jun 10, 2021 |
|
|
|
63186661 |
May 10, 2021 |
|
|
|
63166475 |
Mar 26, 2021 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G01S 7/484 20130101;
G01S 17/42 20130101; G01S 7/4817 20130101 |
International
Class: |
G01S 7/484 20060101
G01S007/484; G01S 17/42 20060101 G01S017/42; G01S 7/481 20060101
G01S007/481 |
Claims
1. A lidar system comprising: a lidar transmitter that controllably
transmits a pulse burst toward a target in a field of view, wherein
the pulse burst includes a first pulse fired at a first shot angle
and a second pulse fired at a second shot angle; a control circuit
that schedules the pulse burst in response to a detection of the
target based on a return from a previous laser pulse shot, wherein
the previous laser pulse shot was fired at a third shot angle, and
wherein the control circuit defines the first and second shot
angles to surround the third shot angle; and a lidar receiver that
(1) computes a ratio of return energy in a return from the first
pulse of the pulse burst relative to return energy in a return from
the second pulse of the pulse burst, (2) retrieves an offset angle
from a lookup table based on the computed return energy ratio,
wherein the lookup table indexes offset angles by return energy
ratios that are derived from a curve that models expected return
energy as a function of an offset angle to the target, and (3)
resolves an angle to the target based on the third shot angle and
the retrieved offset angle.
2. (canceled)
3. (canceled)
4. The system of claim 1 wherein the first and second shot angles
are offset from the third shot angle by a value within a range
between 0.025 degrees and 0.1 degrees.
5. (canceled)
6. (canceled)
7. (canceled)
8. The system of claim 1 wherein the control circuit further
schedules the pulse burst according to a laser energy model that
models available energy for laser pulse shots by the lidar
transmitter over time.
9. The system of claim 8 wherein the lidar transmitter comprises a
mirror that is scannable over an axis with respect to the field of
view, wherein the lidar transmitter scans the mirror between a
plurality of shot angles with respect to the axis to define where
the lidar transmitter is aimed in the field of view with respect to
the axis, and wherein the control circuit schedules the pulse burst
according to the laser energy model and a mirror motion model that
models shot angles for the mirror over time.
10. The system of claim 9 wherein the control circuit (1) defines
shot times for the first and second shot angles based on the mirror
motion model and (2) evaluates the pulse burst for the defined shot
times with respect to the laser energy model.
11. The system of claim 10 wherein the control circuit repeats the
shot times definition and the pulse burst evaluation for one or
more different return scans until a return scan is found where the
laser energy model indicates sufficient energy is available for the
pulse burst.
12. The system of claim 9 wherein the lidar transmitter scans the
mirror in a resonant mode.
13. The system of claim 9 wherein the mirror is a first mirror,
wherein the axis is a first axis, wherein the lidar transmitter
further comprises a second mirror that is scannable over a second
axis with respect to the field of view, wherein the lidar
transmitter scans the second mirror between a plurality of shot
angles with respect to the second axis so that the combination of
the shot angles with respect to the first and second axes defines
where the lidar transmitter is aimed in the field of view.
14. The system of claim 13 wherein the lidar transmitter scans the
second mirror in a point-to-point mode that varies as a function of
a plurality of scheduled laser pulse shots for the lidar
transmitter.
15. The system of claim 9 wherein the lidar transmitter scans the
mirror at a scan frequency in a range between 100 Hz and 20
kHz.
16. The system of claim 9 wherein the lidar transmitter scans the
mirror at a scan frequency at a scan frequency in a range between
10 kHz and 15 kHz.
17. The system of claim 2 wherein the lidar transmitter comprises a
mirror that is scannable over an axis with respect to the field of
view, wherein the lidar transmitter scans the mirror between a
plurality of shot angles with respect to the axis to define where
the lidar transmitter is aimed in the field of view with respect to
the axis, wherein the target detection occurs during a first scan
of the mirror, and wherein the control circuit schedules the pulse
burst for a second scan of the mirror.
18. The system of claim 17 wherein the second scan is a next return
scan following the previous laser pulse shot.
19. The system of claim 2 wherein the lidar transmitter fires the
pulse burst sufficiently quickly after the previous laser pulse
shot for the target to exhibit a range that is deemed common for
the previous laser pulse shot and the pulse burst.
20. The system of claim 2 wherein the target has a known velocity
relative to the system, wherein the lidar transmitter fires the
pulse burst sufficiently slowly after the previous laser pulse shot
for the target to have changed range over the course of the
previous laser pulse shot and the pulse burst, and wherein the
lidar receiver compensates for the changed range based on the known
velocity.
21. The system of claim 20 wherein the target is moving.
22. The system of claim 20 wherein the lidar system is moving.
23. The system of claim 1 wherein the pulse burst exhibits a time
separation between a first pulse of the pulse burst and a second
pulse of the pulse burst in a range between 100 nsec and 10
.mu.sec.
24. The system of claim 23 wherein the time separation is in a
range between 200 nsec and 500 nsec.
25. The system of claim 1 wherein the resolved angle comprises an
azimuth angle.
26. The system of claim 1 further comprising: an optical
amplification laser source that generates laser pulses for the
previous laser pulse shot and the pulse burst, the optical
amplification laser source comprising a seed laser, a pump laser,
and an optical amplifier; and wherein the control circuit controls
the seed laser to adjust its seed energy to control energy levels
for a first pulse and a second pulse within the pulse burst.
27. The system of claim 26 wherein the control circuit adjusts the
seed energy to equalize the energy levels for the first and second
pulses within the pulse burst.
28. An article of manufacture for use by a lidar system to resolve
an angle to a target based on returns from a plurality of laser
pulse shots, the article comprising: machine-readable code that is
resident on a non-transitory machine-readable storage medium,
wherein the code defines processing operations to be performed by a
processor to cause the processor to: schedule a pulse burst in
response to a detection of the target based on a return from a
previous laser pulse shot, wherein the pulse burst comprises (1) a
first pulse to be fired at a first shot angle and (2) a second
pulse to be fired at the second shot angle, wherein the previous
laser pulse shot was fired at a third shot angle, and wherein the
first and second shot angles surround the third shot angle; trigger
a transmission of the scheduled pulse burst toward the target;
determine a first return energy amount for a return from the target
with respect to the first pulse within the pulse burst; determine a
second return energy amount for a return from the target with
respect to the second pulse within the pulse burst; and compute a
ratio of return energy with respect to the determined first and
second return energy amounts; retrieve an offset angle from a
lookup table based on the computed return energy ratio, wherein the
lookup table indexes offset angles by return energy ratios that are
derived from a curve that models expected return energy as a
function of an offset angle to the target; resolve an angle to the
target based on the third shot angle and the retrieved offset
angle.
29. A lidar method comprising: scheduling a pulse burst in response
to a detection of a target in a field of view based on a return
from a previous laser pulse shot, wherein the pulse burst comprises
(1) a first pulse to be fired at a first shot angle and (2) a
second pulse to be fired at the second shot angle, wherein the
previous laser pulse shot was fired at a third shot angle, and
wherein the first and second shot angles surround the third shot
angle; controllably transmitting the scheduled pulse burst toward
the target; computing a ratio of return energy in a return from the
first pulse of the pulse burst relative to return energy in a
return from the second pulse of the pulse burst; retrieving an
offset angle from a lookup table based on the computed return
energy ratio, wherein the lookup table indexes offset angles by
return energy ratios that are derived from a curve that models
expected return energy as a function of an offset angle to the
target; and resolving an angle to the target based on the third
shot angle and the retrieved offset angle.
30. A lidar system comprising: a lidar transmitter that
controllably transmits a pulse burst toward a target in a field of
view, wherein the pulse burst includes a first pulse fired at a
first shot angle and a second pulse fired at a second shot angle; a
control circuit that schedules the pulse burst in response to a
detection of the target based on a return from a previous laser
pulse shot, wherein the previous laser pulse shot was fired at a
third shot angle, and wherein the control circuit defines the first
and second shot angles to surround the third shot angle; a lidar
receiver that detects returns from the pulse burst; and a processor
that (1) computes a ratio of return energy in a return from the
first pulse of the pulse burst relative to return energy in a
return from the second pulse of the pulse burst, (2) retrieves an
offset angle from a lookup table based on the computed return
energy ratio, wherein the lookup table indexes offset angles by
return energy ratios that are derived from a curve that models
expected return energy as a function of an offset angle to the
target, and (3) resolves an angle to the target based on the third
shot angle and the retrieved offset angle.
31. The system of claim 30 wherein the first and second shot angles
are offset from the third shot angle by a value within a range
between 0.025 degrees and 0.1 degrees.
32. The system of claim 30 wherein the pulse burst exhibits a time
separation between a first pulse of the pulse burst and a second
pulse of the pulse burst in a range between 100 nsec and 500
nsec.
33. The system of claim 30 wherein the lidar receiver includes
signal processing circuitry that detects returns from the previous
laser pulse shot and the pulse burst, and wherein the processor is
part of the signal processing circuitry.
34. The system of claim 30 wherein the lidar transmitter comprises
a laser source that generates laser pulses for the previous laser
pulse shot and the pulse burst, wherein the control circuit
schedules the pulse burst based on a laser energy model as compared
to energy requirements for the first and second pulses of the pulse
burst, wherein the laser energy model (1) models a retention of
energy in the laser source after laser pulse shots and (2)
quantitatively predicts available energy amounts from the laser
source for laser pulse shots over time based on a history of prior
laser pulse shots.
35. The system of claim 1 wherein the lidar receiver comprises a
signal processing circuit, wherein the signal processing circuit
(1) computes the return energy ratio, (2) retrieves the offset
angle, and (3) resolves the angle to the target.
Description
CROSS-REFERENCE AND PRIORITY CLAIM TO RELATED PATENT
APPLICATIONS
[0001] This patent application claims priority to U.S. provisional
patent application 63/209,179, filed Jun. 10, 2021, and entitled
"Hyper Temporal Lidar with Controllable Pulse Bursts", the entire
disclosure of which is incorporated herein by reference.
[0002] This patent application also claims priority to U.S.
provisional patent application 63/186,661, filed May 10, 2021, and
entitled "Hyper Temporal Lidar with Controllable Detection
Intervals", the entire disclosure of which is incorporated herein
by reference.
[0003] This patent application also claims priority to U.S.
provisional patent application 63/166,475, filed Mar. 26, 2021, and
entitled "Hyper Temporal Lidar with Dynamic Laser Control", the
entire disclosure of which is incorporated herein by reference.
[0004] This patent application is related to (1) U.S. patent
application Ser. No. ______, filed this same day, and entitled
"Hyper Temporal Lidar with Pulse Burst Scheduling" (said patent
application being identified by Thompson Coburn Attorney Docket
Number 56976-214678), (2) U.S. patent application Ser. No. ______,
filed this same day, and entitled "Hyper Temporal Lidar with
Controllable Variable Laser Seed Energy" (said patent application
being identified by Thompson Coburn Attorney Docket Number
56976-214679), and (3) U.S. patent application Ser. No. ______,
filed this same day, and entitled "Hyper Temporal Lidar with
Switching Between a Baseline Scan Mode and a Pulse Burst Mode"
(said patent application being identified by Thompson Coburn
Attorney Docket Number 56976-214680), the entire disclosures of
each of which are incorporated herein by reference
INTRODUCTION
[0005] There is a need in the art for lidar systems that operate
with low latency and rapid adaptation to environmental changes.
This is particularly the case for automotive applications of lidar
as well as other applications where the lidar system may be moving
at a high rate of speed or where there is otherwise a need for
decision-making in short time intervals. For example, when an
object of interest is detected in the field of view for a lidar
transmitter, it is desirable for the lidar transmitter to rapidly
respond to this detection by firing high densities of laser pulses
at the detected object. However, as the firing rate for the lidar
transmitter increases, this places pressure on the operational
capabilities of the laser source employed by the lidar transmitter
because the laser source will need re-charging time.
[0006] This issue becomes particularly acute in situations where
the lidar transmitter has a variable firing rate. With a variable
firing rate, the laser source's operational capabilities are not
only impacted by periods of high density firing but also periods of
low density firing. As charge builds up in the laser source during
a period where the laser source is not fired, a need arises to
ensure that the laser source does not overheat or otherwise exceed
its maximum energy limits.
[0007] The lidar transmitter may employ a laser source that uses
optical amplification to support the generation of laser pulses.
Such laser sources have energy characteristics that are heavily
impacted by time and the firing rate of the laser source. These
energy characteristics of a laser source that uses optical
amplification have important operational impacts on the lidar
transmitter when the lidar transmitter is designed to operate with
fast scan times and laser pulses that are targeted on specific
range points in the field of view.
[0008] As a technical solution to these problems in the art, the
inventors disclose that a laser energy model can be used to model
the available energy in the laser source over time. The timing
schedule for laser pulses fired by the lidar transmitter can then
be determined using energies that are predicted for the different
scheduled laser pulse shots based on the laser energy model. This
permits the lidar transmitter to reliably ensure at a highly
granular level that each laser pulse shot has sufficient energy to
meet operational needs, including when operating during periods of
high density/high resolution laser pulse firing. The laser energy
model is capable of modeling the energy available for laser pulses
in the laser source over very short time intervals as discussed in
greater detail below. With such short interval time modeling, the
laser energy modeling can be referred to as a transient laser
energy model.
[0009] Furthermore, the inventors also disclose that mirror motion
can be modeled so that the system can also reliably predict where a
scanning mirror is aimed within a field of view over time. This
mirror motion model is also capable of predicting mirror motion
over short time intervals as discussed in greater detail below. In
this regard, the mirror motion model can also be referred to as a
transient mirror motion model. The model of mirror motion over time
can be linked with the model of laser energy over time to provide
still more granularity in the scheduling of laser pulses that are
targeted at specific range points in the field of view. Thus, a
control circuit can translate a list of arbitrarily ordered range
points to be targeted with laser pulses into a shot list of laser
pulses to be fired at such range points using the modeled laser
energy coupled with the modeled mirror motion. In this regard, the
"shot list" can refer to a list of the range points to be targeted
with laser pulses as combined with timing data that defines a
schedule or sequence by which laser pulses will be fired toward
such range points.
[0010] Through the use of such models, the lidar system can provide
hyper temporal processing where laser pulses can be scheduled and
fired at high rates with high timing precision and high spatial
targeting/pointing precision. This results in a lidar system that
can operate at low latency, high frame rates, and intelligent range
point targeting where regions of interest in the field of view can
be targeted with rapidly-fired and spatially dense laser pulse
shots.
[0011] According to additional example embodiments, the inventors
disclose that the detection intervals used by a lidar receiver to
detect returns of the fired laser pulse shots can be closely
controlled. Such control over the detection intervals used by the
lidar receiver allows for close coordination between the lidar
transmitter and the lidar receiver where the lidar receiver is able
to adapt to variable shot intervals of the lidar transmitter
(including periods of high rate firing as well as periods of low
rate firing).
[0012] Each detection interval can be associated with a different
laser pulse shot from which a return is to be collected during the
associated detection interval. Accordingly, each detection interval
is also associated with the return for its associated laser pulse
shot. The lidar receiver can control these detection intervals on a
shot-specific basis so that the lidar receiver will be able to use
the appropriate pixel sets for detecting the returns from the
detection interval's associated shots. The lidar receiver includes
a plurality of detector pixels arranged as a photodetector array,
and different sets of detector pixels can be selected for use to
detect the returns from different laser pulse shots. During a given
detection interval, the lidar receiver will collect sensed signal
data from the selected pixel set, and this collected signal data
can be processed to detect the associated return for that detection
interval. The choice of which pixel set to use for detecting a
return from a given laser pulse shot can be based on the location
in the field of the range point targeted by the given laser pulse
shot. In this fashion, the lidar receiver will readout from
different pixel sets during the detection intervals in a sequenced
pattern that follows the sequenced spatial pattern of the laser
pulse shots.
[0013] The lidar receiver can use any of a number of criteria for
deciding when to start and stop reading out from the different
pixel sets for detecting returns. For example, the lidar receiver
can use estimates of potential ranges to the targeted range points
to decide on when the collections should start and stop from
various pixel sets. As an example, if an object at range point X is
located 10 meters from the lidar system, it can be expected that
the return from the laser pulse shot fired at this object will
reach the photodetector array relatively quickly, while it would
take relatively longer for a return to reach the photodetector
array if the object at range point X is located 1,000 meters from
the lidar system. To control when the collections should start and
stop from the pixel sets in order to detect returns from the laser
pulse shots, the system can determine pairs of minimum and maximum
range values for the range points targeted by each laser pulse
shot, and these minimum and maximum range values can be translated
into on/off times for the pixel sets. Through intelligent control
of these on (start collection) and off (stop collection) times, the
risk of missing a return due to the return impacting a deactivated
pixel is reduced.
[0014] Moreover, the detection intervals can vary across different
shots (e.g., Detection Interval A (associated with Shot A to
support detection of the return from Shot A) can have a different
duration than Detection Interval B (associated with Shot B to
support detection of the return from Shot B)). Further still, at
least some of the detection intervals can be controlled to be of
different durations than the shot intervals that correspond to such
detection intervals. The shot interval that corresponds to a given
detection interval is the time between the shot that is associated
with that detection interval and the next shot in the shot
sequence. Counterintuitively, the inventors have found that it is
often not desirable for a detection interval to be of the same
duration as its corresponding shot interval due to factors such as
the amount of processing time that is needed to detect returns
within return signals. In many cases, it will be desirable for the
control process to define a detection interval so that it exhibits
a duration shorter than the duration of its corresponding shot
interval; while in some other cases it may be desirable for the
control process to define a detection interval so that it exhibits
a longer duration than the duration of its corresponding shot
interval. This characteristic can be referred to as a detection
interval that is asynchronous relative to its corresponding shot
interval duration.
[0015] Further still, the inventors also disclose the use of
multiple processors in a lidar receiver to distribute the workload
of processing returns. The activation/deactivation times of the
pixel sets can be used to define which samples in a return buffer
will be used for processing to detect each return, and multiple
processors can share the workload of processing these samples in an
effort to improve the latency of return detection.
[0016] The inventors also disclose the use of multiple readout
channels within a lidar receiver that are capable of simultaneously
reading out sensed signals from different pixel sets of the
photodetector array. In doing so, the lidar receiver can support
the use of overlapping detection intervals when collecting signal
data for detecting different returns.
[0017] Moreover, the inventors disclose a lidar system having a
lidar transmitter and lidar receiver that are in a bistatic
arrangement with each other. Such a bistatic lidar system can be
deployed in a climate-controlled compartment of a vehicle to reduce
the exposure of the lidar system to harsher elements so it can
operate in more advantageous environments with regards to factors
such as temperature, moisture, etc. In an example embodiment, the
bistatic lidar system can be connected to or incorporated within a
rear view mirror assembly of a vehicle.
[0018] Further still, the inventors disclose the use of pulse
bursts by a lidar system to improve the precision with which the
angle to a target in the field of view is resolved. These pulse
bursts can be scheduled in response to detection of the target in
the field of view, and the lidar system can employ the laser energy
model and mirror motion model to ensure that sufficient energy is
available for the scheduled pulses of the pulse burst.
[0019] The inventors also disclose the use of an optical
amplification laser source that employs a controllable variable
seed laser. The variable seed laser can be controlled to adjust the
seed energy levels for the laser in a manner that achieves a
desired regulation of the energy levels in the pulses of the pulse
burst (such as equalization of the energy levels in the pulses of
the pulse burst) despite the short time interval between such
pulses.
[0020] These and other features and advantages of the invention
will be described in greater detail below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0021] FIG. 1 depicts an example lidar transmitter that uses a
laser energy model to schedule laser pulses.
[0022] FIG. 2A depicts an example process flow the control circuit
of FIG. 1.
[0023] FIG. 2B-2D depict additional examples of lidar transmitters
that use a laser energy model to schedule laser pulses.
[0024] FIG. 3 depicts an example lidar transmitter that uses a
laser energy model and a mirror motion model to schedule laser
pulses.
[0025] FIGS. 4A-4D illustrate how mirror motion can be modeled for
a mirror that scans in a resonant mode.
[0026] FIG. 4E depicts an example process flow for controllably
adjusting an amplitude for mirror scanning.
[0027] FIG. 5 depicts an example process flow for the control
circuit of FIG. 3.
[0028] FIGS. 6A and 6B depict example process flows for shot
scheduling using the control circuit of FIG. 3.
[0029] FIG. 7A depicts an example process flow for simulating and
evaluating different shot ordering candidates based on the laser
energy model and the mirror motion model.
[0030] FIG. 7B depicts an example of how time slots in a mirror
scan can be related to the shot angles for the mirror using the
mirror motion model.
[0031] FIG. 7C depicts an example process flow for simulating
different shot ordering candidates based on the laser energy
model.
[0032] FIGS. 7D-7F depict different examples of laser energy
predictions produced by the laser energy model with respect to
different shot order candidates.
[0033] FIG. 8 depicts an example lidar transmitter that uses a
laser energy model and a mirror motion model to schedule laser
pulses, where the control circuit includes a system controller and
a beam scanner controller.
[0034] FIG. 9 depicts an example process flow for inserting marker
shots into a shot list.
[0035] FIG. 10 depicts an example process flow for using an eye
safety model to adjust a shot list.
[0036] FIG. 11 depicts an example lidar transmitter that uses a
laser energy model, a mirror motion model, and an eye safety model
to schedule laser pulses.
[0037] FIG. 12 depicts an example process flow for simulating
different shot ordering candidates based on the laser energy model
and eye safety model.
[0038] FIG. 13 depicts another example process for determining shot
schedules using the models.
[0039] FIG. 14 depicts an example lidar system where a lidar
transmitter and a lidar receiver coordinate their operations with
each other.
[0040] FIG. 15 depicts another example process for determining shot
schedules using the models.
[0041] FIG. 16 illustrates how the lidar transmitter can change its
firing rate to probe regions in a field of view with denser
groupings of laser pulses.
[0042] FIGS. 17A-17F depict example process flows for prioritized
selections of elevations with respect to shot scheduling.
[0043] FIG. 18A depicts an example lidar receiver in accordance
with an example embodiment.
[0044] FIG. 18B depicts an example process flow for use by the
lidar receiver of FIG. 18A to control the activation and
deactivation of detector pixels in a photodetector array.
[0045] FIG. 18C depicts an example lidar receiver in accordance
with another example embodiment.
[0046] FIG. 18D depicts an example process flow for use by the
lidar receiver of FIG. 18C to control the activation and
deactivation of detector pixels in a photodetector array.
[0047] FIGS. 19A and 19B show examples of detection timing for a
lidar receiver to detect returns from laser pulse shots.
[0048] FIG. 20 shows an example process flow for use by a signal
processing circuit of a lidar receiver to detect returns from laser
pulse shots.
[0049] FIGS. 21A and 21B show examples of a multi-processor
arrangement for distributing the workload of detecting returns
within a lidar receiver.
[0050] FIG. 22 shows an example process flow for assigning range
swaths to return detections for a shot list of laser pulse
shots.
[0051] FIGS. 23A and 23B show examples of mathematical operations
that can be used to assign range swath values to return
detections.
[0052] FIG. 24 shows another example process flow for assigning
range swaths to return detections for a shot list of laser pulse
shots.
[0053] FIG. 25 shows an example where a bistatic lidar system in
accordance with an example embodiment is deployed inside a
climate-controlled compartment of a vehicle.
[0054] FIG. 26 shows an example embodiment for a lidar receiver
which employs multiple readout channels to enable the use of
overlapping detection intervals for detecting the returns from
different shots.
[0055] FIG. 27 shows an example embodiment of a lidar receiver that
includes details showing how pixel activation can be controlled in
concert with selective pixel readout.
[0056] FIG. 28 shows an example pulse burst.
[0057] FIG. 29 shows an example of how mirror motion causes the
pulses of a pulse burst to be directed toward different shot
angles.
[0058] FIG. 30 shows an example of how a pulse burst can be used to
resolve an angle to a target detected by an initial laser pulse
shot.
[0059] FIG. 31 shows an example process flow for pulse burst
firing.
[0060] FIGS. 32A and 32B show example process flows for return
processing with respect to a pulse burst to more precisely resolve
an angle to target.
[0061] FIGS. 33A-33D show examples of shot return intensity as a
function of offset angle to the target.
[0062] FIG. 34 shows an example of how a plot of the derivative of
the shot return intensity shown by FIGS. 33A-33D versus offset
angle can be used to resolve the offset angle based on the return
intensities from the pulse burst.
[0063] FIG. 35 shows an example laser source where seed energy can
be controlled to equalize the energies within the pulses of a pulse
burst.
DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS
[0064] FIG. 1 shows an example embodiment of a lidar transmitter
100 that can be employed to support hyper temporal lidar. In an
example embodiment, the lidar transmitter 100 can be deployed in a
vehicle such as an automobile. However, it should be understood
that the lidar transmitter 100 described herein need not be
deployed in a vehicle. As used herein, "lidar", which can also be
referred to as "ladar", refers to and encompasses any of light
detection and ranging, laser radar, and laser detection and
ranging. In the example of FIG. 1, the lidar transmitter 100
includes a laser source 102, a mirror subsystem 104, and a control
circuit 106. Control circuit 106 uses a laser energy model 108 to
govern the firing of laser pulses 122 by the laser source 102.
Laser pulses 122 transmitted by the laser source 102 are sent into
the environment via mirror subsystem 104 to target various range
points in a field of view for the lidar transmitter 100. These
laser pulses 122 can be interchangeably referred to as laser pulse
shots (or more simply, as just "shots"). The field of view will
include different addressable coordinates (e.g., {azimuth,
elevation} pairs) which serve as range points that can be targeted
by the lidar transmitter 100 with the laser pulses 122.
[0065] In the example of FIG. 1, laser source 102 can use optical
amplification to generate the laser pulses 122 that are transmitted
into the lidar transmitter's field of view via the mirror subsystem
104. In this regard, a laser source 102 that includes an optical
amplifier can be referred to as an optical amplification laser
source 102. In the example of FIG. 1, the optical amplification
laser source 102 includes a seed laser 114, an optical amplifier
116, and a pump laser 118. In this laser architecture, the seed
laser 114 provides the input (signal) that is amplified to yield
the transmitted laser pulse 122, while the pump laser 118 provides
the power (in the form of the energy deposited by the pump laser
118 into the optical amplifier 116). So, the optical amplifier 116
is fed by two inputs--the pump laser 118 (which deposits energy
into the optical amplifier 116) and the seed laser 114 (which
provides the signal that stimulates the energy in the optical
amplifier 116 and induces pulse 122 to fire).
[0066] Thus, the pump laser 118, which can take the form of an
electrically-driven pump laser diode, continuously sends energy
into the optical amplifier 116. The seed laser 114, which can take
the form of an electrically-driven seed laser that includes a pulse
formation network circuit, controls when the energy deposited by
the pump laser 118 into the optical amplifier 116 is released by
the optical amplifier 116 as a laser pulse 122 for transmission.
The seed laser 114 can also control the shape of laser pulse 122
via the pulse formation network circuit (which can drive the pump
laser diode with the desired pulse shape). The seed laser 114 also
injects a small amount of (pulsed) optical energy into the optical
amplifier 116.
[0067] Given that the energy deposited in the optical amplifier 116
by the pump laser 118 and seed laser 114 serves to seed the optical
amplifier 116 with energy from which the laser pulses 122 are
generated, this deposited energy can be referred to as "seed
energy" for the laser source 102.
[0068] The optical amplifier 116 operates to generate laser pulse
122 from the energy deposited therein by the seed laser 114 and
pump laser 118 when the optical amplifier 116 is induced to fire
the laser pulse 122 in response to stimulation of the energy
therein by the seed laser 114. The optical amplifier 116 can take
the form of a fiber amplifier. In such an embodiment, the laser
source 102 can be referred to as a pulsed fiber laser source. With
a pulsed fiber laser source 102, the pump laser 118 essentially
places the dopant electrons in the fiber amplifier 116 into an
excited energy state. When it is time to fire laser pulse 122, the
seed laser 114 stimulates these electrons, causing them to emit
energy and collapse down to a lower (ground) state, which results
in the emission of pulse 122. An example of a fiber amplifier that
can be used for the optical amplifier 116 is a doped fiber
amplifier such as an Erbium-Doped Fiber Amplifier (EDFA).
[0069] It should be understood that other types of optical
amplifiers can be used for the optical amplifier 116 if desired by
a practitioner. For example, the optical amplifier 116 can take the
form of a semiconductor amplifier. In contrast to a laser source
that uses a fiber amplifier (where the fiber amplifier is optically
pumped by pump laser 118), a laser source that uses a semiconductor
amplifier can be electrically pumped. As another example, the
optical amplifier 116 can take the form of a gas amplifier (e.g., a
CO.sub.2 gas amplifier). Moreover, it should be understood that a
practitioner may choose to include a cascade of optical amplifiers
116 in laser source 102.
[0070] In an example embodiment, the pump laser 118 can exhibit a
fixed rate of energy buildup (where a constant amount of energy is
deposited in the optical amplifier 116 per unit time). However, it
should be understood that a practitioner may choose to employ a
pump laser 118 that exhibits a variable rate of energy buildup
(where the amount of energy deposited in the optical amplifier 116
varies per unit time).
[0071] The laser source 102 fires laser pulses 122 in response to
firing commands 120 received from the control circuit 106. In an
example where the laser source 102 is a pulsed fiber laser source,
the firing commands 120 can cause the seed laser 114 to induce
pulse emissions by the fiber amplifier 116. In an example
embodiment, the lidar transmitter 100 employs non-steady state
pulse transmissions, which means that there will be variable timing
between the commands 120 to fire the laser source 102. In this
fashion, the laser pulses 122 transmitted by the lidar transmitter
100 will be spaced in time at irregular intervals. There may be
periods of relatively high densities of laser pulses 122 and
periods of relatively low densities of laser pulses 122. Examples
of laser vendors that provide such variable charge time control
include Luminbird and ITF. As examples, lasers that have the
capacity to regulate pulse timing over timescales corresponding to
preferred embodiments discussed herein and which are suitable to
serve as laser source 102 in these preferred embodiments are
expected to exhibit laser wavelengths of 1.5 .mu.m and available
energies in a range of around hundreds of nano-Joules to around
tens of micro-Joules, with timing controllable from hundreds of
nanoseconds to tens of microseconds and with an average power range
from around 0.25 Watts to around 4 Watts.
[0072] The mirror subsystem 104 includes a mirror that is scannable
to control where the lidar transmitter 100 is aimed. In the example
embodiment of FIG. 1, the mirror subsystem 104 includes two
mirrors--mirror 110 and mirror 112. Mirrors 110 and 112 can take
the form of MEMS mirrors. However, it should be understood that a
practitioner may choose to employ different types of scannable
mirrors. Mirror 110 is positioned optically downstream from the
laser source 102 and optically upstream from mirror 112. In this
fashion, a laser pulse 122 generated by the laser source 102 will
impact mirror 110, whereupon mirror 110 will reflect the pulse 122
onto mirror 112, whereupon mirror 112 will reflect the pulse 122
for transmission into the environment. It should be understood that
the outgoing pulse 122 may pass through various transmission optics
during its propagation from mirror 112 into the environment.
[0073] In the example of FIG. 1, mirror 110 can scan through a
plurality of mirror scan angles to define where the lidar
transmitter 100 is targeted along a first axis. This first axis can
be an X-axis so that mirror 110 scans between azimuths. Mirror 112
can scan through a plurality of mirror scan angles to define where
the lidar transmitter 100 is targeted along a second axis. The
second axis can be orthogonal to the first axis, in which case the
second axis can be a Y-axis so that mirror 112 scans between
elevations. The combination of mirror scan angles for mirror 110
and mirror 112 will define a particular {azimuth, elevation}
coordinate to which the lidar transmitter 100 is targeted. These
azimuth, elevation pairs can be characterized as {azimuth angles,
elevation angles} and/or {rows, columns} that define range points
in the field of view which can be targeted with laser pulses 122 by
the lidar transmitter 100.
[0074] A practitioner may choose to control the scanning of mirrors
110 and 112 using any of a number of scanning techniques. In a
particularly powerful embodiment, mirror 110 can be driven in a
resonant mode according to a sinusoidal signal while mirror 112 is
driven in a point-to-point mode according to a step signal that
varies as a function of the range points to be targeted with laser
pulses 122 by the lidar transmitter 100. In this fashion, mirror
110 can be operated as a fast-axis mirror while mirror 112 is
operated as a slow-axis mirror. When operating in such a resonant
mode, mirror 110 scans through scan angles in a sinusoidal pattern.
In an example embodiment, mirror 110 can be scanned at a frequency
in a range between around 100 Hz and around 20 kHz. In a preferred
embodiment, mirror 110 can be scanned at a frequency in a range
between around 10 kHz and around 15 kHz (e.g., around 12 kHz). As
noted above, mirror 112 can be driven in a point-to-point mode
according to a step signal that varies as a function of the range
points to be targeted with laser pulses 122 by the lidar
transmitter 100. Thus, if the lidar transmitter 100 is to fire a
laser pulse 122 at a particular range point having an elevation of
X, then the step signal can drive mirror 112 to scan to the
elevation of X. When the lidar transmitter 100 is later to fire a
laser pulse 122 at a particular range point having an elevation of
Y, then the step signal can drive mirror 112 to scan to the
elevation of Y. In this fashion, the mirror subsystem 104 can
selectively target range points that are identified for targeting
with laser pulses 122. It is expected that mirror 112 will scan to
new elevations at a much slower rate than mirror 110 will scan to
new azimuths. As such, mirror 110 may scan back and forth at a
particular elevation (e.g., left-to-right, right-to-left, and so
on) several times before mirror 112 scans to a new elevation. Thus,
while the mirror 112 is targeting a particular elevation angle, the
lidar transmitter 100 may fire a number of laser pulses 122 that
target different azimuths at that elevation while mirror 110 is
scanning through different azimuth angles. U.S. Pat. Nos.
10,078,133 and 10,642,029, the entire disclosures of which are
incorporated herein by reference, describe examples of mirror scan
control using techniques and transmitter architectures such as
these (and others) which can be used in connection with the example
embodiments described herein.
[0075] Control circuit 106 is arranged to coordinate the operation
of the laser source 102 and mirror subsystem 104 so that laser
pulses 122 are transmitted in a desired fashion. In this regard,
the control circuit 106 coordinates the firing commands 120
provided to laser source 102 with the mirror control signal(s) 130
provided to the mirror subsystem 104. In the example of FIG. 1,
where the mirror subsystem 104 includes mirror 110 and mirror 112,
the mirror control signal(s) 130 can include a first control signal
that drives the scanning of mirror 110 and a second control signal
that drives the scanning of mirror 112. Any of the mirror scan
techniques discussed above can be used to control mirrors 110 and
112. For example, mirror 110 can be driven with a sinusoidal signal
to scan mirror 110 in a resonant mode, and mirror 112 can be driven
with a step signal that varies as a function of the range points to
be targeted with laser pulses 122 to scan mirror 112 in a
point-to-point mode.
[0076] As discussed in greater detail below, control circuit 106
can use a laser energy model 108 to determine a timing schedule for
the laser pulses 122 to be transmitted from the laser source 102.
This laser energy model 108 can model the available energy within
the laser source 102 for producing laser pulses 122 over time in
different shot schedule scenarios. By modeling laser energy in this
fashion, the laser energy model 108 helps the control circuit 106
make decisions on when the laser source 102 should be triggered to
fire laser pulses. Moreover, as discussed in greater detail below,
the laser energy model 108 can model the available energy within
the laser source 102 over short time intervals (such as over time
intervals in a range from 10-100 nanoseconds), and such a short
interval laser energy model 108 can be referred to as a transient
laser energy model 108.
[0077] Control circuit 106 can include a processor that provides
the decision-making functionality described herein. Such a
processor can take the form of a field programmable gate array
(FPGA) or application-specific integrated circuit (ASIC) which
provides parallelized hardware logic for implementing such
decision-making. The FPGA and/or ASIC (or other compute
resource(s)) can be included as part of a system on a chip (SoC).
However, it should be understood that other architectures for
control circuit 106 could be used, including software-based
decision-making and/or hybrid architectures which employ both
software-based and hardware-based decision-making. The processing
logic implemented by the control circuit 106 can be defined by
machine-readable code that is resident on a non-transitory
machine-readable storage medium such as memory within or available
to the control circuit 106. The code can take the form of software
or firmware that define the processing operations discussed herein
for the control circuit 106. This code can be downloaded onto the
control circuit 106 using any of a number of techniques, such as a
direct download via a wired connection as well as over-the-air
downloads via wireless networks, which may include secured wireless
networks. As such, it should be understood that the lidar
transmitter 100 can also include a network interface that is
configured to receive such over-the-air downloads and update the
control circuit 106 with new software and/or firmware. This can be
particularly advantageous for adjusting the lidar transmitter 100
to changing regulatory environments with respect to criteria such
as laser dosage and the like. When using code provisioned for
over-the-air updates, the control circuit 106 can operate with
unidirectional messaging to retain function safety.
[0078] Modeling Laser Energy Over Time:
[0079] FIG. 2A shows an example process flow for the control
circuit 106 with respect to using the laser energy model 108 to
govern the timing schedule for laser pulses 122. At step 200, the
control circuit 106 maintains the laser energy model 108. This step
can include reading the parameters and expressions that define the
laser energy model 108, discussed in greater detail below. Step 200
can also include updating the laser energy model 108 over time as
laser pulses 122 are triggered by the laser source 102 as discussed
below.
[0080] In an example embodiment where the laser source 102 is a
pulsed fiber laser source as discussed above, the laser energy
model 108 can model the energy behavior of the seed laser 114, pump
laser 118, and fiber amplifier 116 over time as laser pulses 122
are fired. As noted above, the fired laser pulses 122 can be
referred to as "shots". For example, the laser energy model 108 can
be based on the following parameters: [0081] CE(t), which
represents the combined amount of energy within the fiber amplifier
116 at the moment when the laser pulse 122 is fired at time t.
[0082] EF(t), which represents the amount of energy fired in laser
pulse 122 at time t; [0083] E.sub.P, which represents the amount of
energy deposited by the pump laser 118 into the fiber amplifier 116
per unit of time. [0084] S(t+.delta.), which represents the
cumulative amount of seed energy that has been deposited by the
pump laser 118 and seed laser 114 into the fiber amplifier 116 over
the time duration .delta., where .delta. represents the amount of
time between the most recent laser pulse 122 (for firing at time t)
and the next laser pulse 122 (to be fired at time t+.delta.).
[0085] F(t+.delta.), which represents the amount of energy left
behind in the fiber amplifier 116 when the pulse 122 is fired at
time t (and is thus available for use with the next pulse 122 to be
fired at time t+.delta.). [0086] CE(t+.delta.), which represents
the amount of combined energy within the fiber amplifier 116 at
time t+.delta. (which is the sum of S(t+.delta.) and F(t+.delta.))
[0087] EF(t+.delta.), which represents the amount of energy fired
in laser pulse 122 fired at time t+.delta. [0088] a and b, where
"a" represents a proportion of energy transferred from the fiber
amplifier 116 into the laser pulse 122 when the laser pulse 122 is
fired, where "b" represents a proportion of energy retained in the
fiber amplifier 116 after the laser pulse 122 is fired, where
a+b=1.
[0089] While the seed energy (S) includes both the energy deposited
in the fiber amplifier 116 by the pump laser 118 and the energy
deposited in the fiber amplifier 116 by the seed laser 114, it
should be understood that for most embodiments the energy from the
seed laser 114 will be very small relative to the energy from the
pump laser 118. As such, a practitioner can choose to model the
seed energy solely in terms of energy produced by the pump laser
118 over time. Thus, after the pulsed fiber laser source 102 fires
a laser pulse at time t, the pump laser 118 will begin re-supplying
the fiber amplifier 116 with energy over time (in accordance with
E.sub.P) until the seed laser 116 is triggered at time t+.delta. to
cause the fiber amplifier 116 to emit the next laser pulse 122
using the energy left over in the fiber amplifier 116 following the
previous shot plus the new energy that has been deposited in the
fiber amplifier 116 by pump laser 118 since the previous shot. As
noted above, the parameters a and b model how much of the energy in
the fiber amplifier 116 is transferred into the laser pulse 122 for
transmission and how much of the energy is retained by the fiber
amplifier 116 for use when generating the next laser pulse 122.
[0090] The energy behavior of pulsed fiber laser source 102 with
respect to the energy fired in laser pulses 122 in this regard can
be expressed as follows:
EF(t)=aCE(t)
F(t+.delta.)=bCE(t)
S(t+.delta.)=.delta.E.sub.P
CE(t+.delta.)=S(t+.delta.)+F(t+.delta.)
EF(t+.delta.)=aCE(t+.delta.)
[0091] With these relationships, the value for CE(t) can be
re-expressed in terms of EF(t) as follows:
CE .times. ( t ) = E .times. F .function. ( t ) a ##EQU00001##
[0092] Furthermore, the value for F(t+.delta.) can be re-expressed
in terms of EF(t) as follows:
F .function. ( t + .delta. ) = bEF .function. ( t ) a
##EQU00002##
[0093] This means that the values for CE(t+.delta.) and
EF(t+.delta.) can be re-expressed as follows:
CE .function. ( t + .delta. ) = .delta. .times. E P + b .times. E
.times. F .function. ( t ) a ##EQU00003## EF .function. ( t +
.delta. ) = a .function. ( .delta. .times. E P + b .times. E
.times. F .function. ( t ) a ) ##EQU00003.2##
[0094] And this expression for EF(t+.delta.) shortens to:
EF(t+.delta.)=a.delta.E.sub.P+bEF(t)
[0095] It can be seen, therefore, that the energy to be fired in a
laser pulse 122 at time t+.delta. in the future can be computed as
a function of how much energy was fired in the previous laser pulse
122 at time t. Given that a, b, E.sub.P, and EF(t) are known
values, and .delta. is a controllable variable, these expressions
can be used as the laser energy model 108 that predicts the amount
of energy fired in a laser pulse at select times in the future (as
well as how much energy is present in the fiber amplifier 116 at
select times in the future).
[0096] While this example models the energy behavior over time for
a pulsed fiber laser source 102, it should be understood that these
models could be adjusted to reflect the energy behavior over time
for other types of laser sources.
[0097] Thus, the control circuit 106 can use the laser energy model
108 to model how much energy is available in the laser source 102
over time and can be delivered in the laser pulses 122 for
different time schedules of laser pulse shots. With reference to
FIG. 2A, this allows the control circuit 106 to determine a timing
schedule for the laser pulses 122 (step 202). For example, at step
202, the control circuit 106 can compare the laser energy model 108
with various defined energy requirements to assess how the laser
pulse shots should be timed. As examples, the defined energy
requirements can take any of a number of forms, including but not
limited to (1) a minimum laser pulse energy, (2) a maximum laser
pulse energy, (3) a desired laser pulse energy (which can be per
targeted range point for a lidar transmitter 100 that selectively
targets range points with laser pulses 122), (4) eye safety energy
thresholds, and/or (5) camera safety energy thresholds. The control
circuit 106 can then, at step 204, generate and provide firing
commands 120 to the laser source 102 that trigger the laser source
102 to generate laser pulses 122 in accordance with the determined
timing schedule. Thus, if the control circuit 106 determines that
laser pulses should be generated at times t1, t2, t3, . . . , the
firing commands 120 can trigger the laser source to generate laser
pulses 122 at these times.
[0098] A control variable that the control circuit 106 can evaluate
when determining the timing schedule for the laser pulses is the
value of .delta., which controls the time interval between
successive laser pulse shots. The discussion below illustrates how
the choice of .delta. impacts the amount of energy in each laser
pulse 122 according to the laser energy model 108.
[0099] For example, during a period where the laser source 102 is
consistently fired every .delta. units of time, the laser energy
model 108 can be used to predict energy levels for the laser pulses
as shown in the following toy example. [0100] Toy Example 1, where
E.sub.P=1 unit of energy; .delta.=1 unit of time; the initial
amount of energy stored by the fiber laser 116 is 1 unit of energy;
a=0.5 and b=0.5:
TABLE-US-00001 [0100] Shot Number 1 2 3 4 5 Time t + 1 t + 2 t + 3
t + 4 t + 5 Seed Energy from Pump Laser (S) 1 1 1 1 1 Leftover
Fiber Energy (F) 1 1 1 1 1 Combined Energy (S + F) 2 2 2 2 2 Energy
Fired (EF) 1 1 1 1 1
[0101] If the rate of firing is increased, this will impact how
much energy is included in the laser pulses. For example, relative
to Toy Example 1, if the firing rate is doubled (.delta.=0.5 units
of time) (while the other parameters are the same), the laser
energy model 108 will predict the energy levels per laser pulse 122
as follows below with Toy Example 2. [0102] Toy Example 2, where
E.sub.P=1 unit of energy; .delta.=0.5 units of time; the initial
amount of energy stored by the fiber laser 116 is 1 unit of energy;
a=0.5 and b=0.5:
TABLE-US-00002 [0102] Shot Number 1 2 3 4 5 Time t + 0.5 t + 1 t +
1.5 t + 2 t + 3.5 Seed Energy from Pump Laser (S) 0.5 0.5 0.5 0.5
0.5 Leftover Fiber Energy (F) 1 0.75 0.625 0.5625 0.53125 Combined
Energy (S + F) 1.5 1.25 1.125 1.0625 1.03125 Energy Fired (EF) 0.75
0.625 0.5625 0.53125 0.515625
[0103] Thus, in comparing Toy Example 1 with Toy Example 2 it can
be seen that increasing the firing rate of the laser will decrease
the amount of energy in the laser pulses 122. As example
embodiments, the laser energy model 108 can be used to model a
minimum time interval in a range between around 10 nanoseconds to
around 100 nanoseconds. This timing can be affected by both the
accuracy of the clock for control circuit 106 (e.g., clock skew and
clock jitter) and the minimum required refresh time for the laser
source 102 after firing.
[0104] If the rate of firing is decreased relative to Toy Example
1, this will increase how much energy is included in the laser
pulses. For example, relative to Toy Example 1, if the firing rate
is halved (.delta.=2 units of time) (while the other parameters are
the same), the laser energy model 108 will predict the energy
levels per laser pulse 122 as follows below with Toy Example 3.
[0105] Toy Example 3, where E.sub.P=1 unit of energy; .delta.=2
units of time; the initial amount of energy stored by the fiber
laser 116 is 1 unit of energy; a=0.5 and b=0.5:
TABLE-US-00003 [0105] Shot Number 1 2 3 4 5 Time t + 2 t + 4 t + 6
t + 8 t + 10 Seed Energy from Pump Laser (S) 2 2 2 2 2 Leftover
Fiber Energy (F) 1 1.5 1.75 1.875 1.9375 Combined Energy (S + F) 3
3.5 3.75 3.875 3.9375 Energy Fired (EF) 1.5 1.75 1.875 1.9375
1.96875
[0106] If a practitioner wants to maintain a consistent amount of
energy per laser pulse, it can be seen that the control circuit 106
can use the laser energy model 108 to define a timing schedule for
laser pulses 122 that will achieve this goal (through appropriate
selection of values for .delta.).
[0107] For practitioners that want the lidar transmitter 100 to
transmit laser pulses at varying intervals, the control circuit 106
can use the laser energy model 108 to define a timing schedule for
laser pulses 122 that will maintain a sufficient amount of energy
per laser pulse 122 in view of defined energy requirements relating
to the laser pulses 122. For example, if the practitioner wants the
lidar transmitter 100 to have the ability to rapidly fire a
sequence of laser pulses (for example, to interrogate a target in
the field of view with high resolution) while ensuring that the
laser pulses in this sequence are each at or above some defined
energy minimum, the control circuit 106 can define a timing
schedule that permits such shot clustering by introducing a
sufficiently long value for .delta. just before firing the
clustered sequence. This long .delta. value will introduce a
"quiet" period for the laser source 102 that allows the energy in
seed laser 114 to build up so that there is sufficient available
energy in the laser source 102 for the subsequent rapid fire
sequence of laser pulses. As indicated by the decay pattern of
laser pulse energy reflected by Toy Example 2, increasing the
starting value for the seed energy (S) before entering the time
period of rapidly-fired laser pulses will make more energy
available for the laser pulses fired close in time with each
other.
[0108] Toy Example 4 below shows an example shot sequence in this
regard, where there is a desire to fire a sequence of 5 rapid laser
pulses separated by 0.25 units of time, where each laser pulse has
a minimum energy requirement of 1 unit of energy. If the laser
source has just concluded a shot sequence after which time there is
1 unit of energy retained in the fiber laser 116, the control
circuit can wait 25 units of time to allow sufficient energy to
build up in the seed laser 114 to achieve the desired rapid fire
sequence of 5 laser pulses 122, as reflected in the table below.
[0109] Toy Example 4, where E.sub.P=1 unit of energy;
.delta..sub.LONG=25 units of time; .delta..sub.SHORT=0.25 units of
time; the initial amount of energy stored by the fiber laser 116 is
1 unit of energy; a=0.5 and b=0.5; and the minimum pulse energy
requirement is 1 unit of energy:
TABLE-US-00004 [0109] Shot Number 1 2 3 4 5 Time t + 25 t + 25.25 t
+ 25.5 t + 25.75 t + 26 Seed Energy from Pump Laser (S) 25 0.25
0.25 0.25 0.25 Leftover Fiber Energy (F) 1 13 6.625 3.4375 1.84375
Combined Energy (S + F) 26 13.25 6.875 3.6875 2.09375 Energy Fired
(EF) 13 6.625 3.4375 1.84375 1.046875
[0110] This ability to leverage "quiet" periods to facilitate
"busy" periods of laser activity means that the control circuit 106
can provide highly agile and responsive adaptation to changing
circumstances in the field of view. For example, FIG. 16 shows an
example where, during a first scan 1600 across azimuths from left
to right at a given elevation, the laser source 102 fires 5 laser
pulses 122 that are relatively evenly spaced in time (where the
laser pulses are denoted by the "X" marks on the scan 1600). If a
determination is made that an object of interest is found at range
point 1602, the control circuit 106 can operate to interrogate the
region of interest 1604 around range point 1602 with a higher
density of laser pulses on second scan 1610 across the azimuths
from right to left. To facilitate this high density period of
rapidly fired laser pulses within the region of interest 1604, the
control circuit 106 can use the laser energy model 108 to determine
that such high density probing can be achieved by inserting a lower
density period 1606 of laser pulses during the time period
immediately prior to scanning through the region of interest 1604.
In the example of FIG. 16, this lower density period 1604 can be a
quiet period where no laser pulses are fired. Such timing schedules
of laser pulses can be defined for different elevations of the scan
pattern to permit high resolution probing of regions of interest
that are detected in the field of view.
[0111] The control circuit 106 can also use the energy model 108 to
ensure that the laser source 102 does not build up with too much
energy. For practitioners that expect the lidar transmitter 100 to
exhibit periods of relatively infrequent laser pulse firings, it
may be the case that the value for .delta. in some instances will
be sufficiently long that too much energy will build up in the
fiber amplifier 116, which can cause problems for the laser source
102 (either due to equilibrium overheating of the fiber amplifier
116 or non-equilibrium overheating of the fiber amplifier 116 when
the seed laser 114 induces a large amount of pulse energy to exit
the fiber amplifier 116). To address this problem, the control
circuit 106 can insert "marker" shots that serve to bleed off
energy from the laser source 102. Thus, even though the lidar
transmitter 100 may be primarily operating by transmitting laser
pulses 122 at specific, selected range points, these marker shots
can be fired regardless of the selected list of range points to be
targeted for the purpose of preventing damage to the laser source
102. For example, if there is a maximum energy threshold for the
laser source 102 of 25 units of energy, the control circuit 106 can
consult the laser energy model 108 to identify time periods where
this maximum energy threshold would be violated. When the control
circuit 106 predicts that the maximum energy threshold would be
violated because the laser pulses have been too infrequent, the
control circuit 106 can provide a firing command 120 to the laser
source 102 before the maximum energy threshold has been passed,
which triggers the laser source 102 to fire the marker shot that
bleeds energy out of the laser source 102 before the laser source's
energy has gotten too high. This maximum energy threshold can be
tracked and assessed in any of a number of ways depending on how
the laser energy model 108 models the various aspects of laser
operation. For example, it can be evaluated as a maximum energy
threshold for the fiber amplifier 116 if the energy model 108
tracks the energy in the fiber amplifier 116 (S+F) over time. As
another example, the maximum energy threshold can be evaluated as a
maximum value of the duration .delta. (which would be set to
prevent an amount of seed energy (S) from being deposited into the
fiber amplifier 116 that may cause damage when taking the values
for E.sub.P and a presumed value for F into consideration.
[0112] While the toy examples above use simplified values for the
model parameters (e.g. the values for E.sub.P and .delta.) for the
purpose of ease of explanation, it should be understood that
practitioners can select values for the model parameters or
otherwise adjust the model components to accurately reflect the
characteristics and capabilities of the laser source 102 being
used. For example, the values for E.sub.P, a, and b can be
empirically determined from testing of a pulsed fiber laser source
(or these values can be provided by a vendor of the pulsed fiber
laser source). Moreover, a minimum value for .delta. can also be a
function of the pulsed fiber laser source 102. That is, the pulsed
fiber laser sources available from different vendors may exhibit
different minimum values for .delta., and this minimum value for
.delta. (which reflects a maximum achievable number of shots per
second) can be included among the vendor's specifications for its
pulsed fiber laser source.
[0113] Furthermore, in situations where the pulsed fiber laser
source 102 is expected or observed to exhibit nonlinear behaviors,
such nonlinear behavior can be reflected in the model. As an
example, it can be expected that the pulsed fiber laser source 102
will exhibit energy inefficiencies at high power levels. In such a
case, the modeling of the seed energy (S) can make use of a
clipped, offset (affine) model for the energy that gets delivered
to the fiber amplifier 116 by pump laser 118 for pulse generation.
For example, in this case, the seed energy can be modeled in the
laser energy model 108 as:
S(t+.delta.)=E.sub.P max(a.sub.1.delta.+a.sub.0,offset)
[0114] The values for a.sub.1, a.sub.0, and offset can be
empirically measured for the pulsed fiber laser source 102 and
incorporated into the modeling of S(t+.delta.) used within the
laser energy model 108. It can be seen that for a linear regime,
the value for a.sub.1 would be 1, and the values for a.sub.0 and
offset would be 0. In this case, the model for the seed energy
S(t+.delta.) reduces to .delta.E.sub.P as discussed in the examples
above.
[0115] The control circuit 106 can also update the laser energy
model 108 based on feedback that reflects the energies within the
actual laser pulses 122. In this fashion, laser energy model 108
can better improve or maintain its accuracy over time. In an
example embodiment, the laser source 102 can monitor the energy
within laser pulses 122 at the time of firing. This energy amount
can then be reported by the laser source 102 to the control circuit
106 (see 250 in FIG. 2B) for use in updating the model 108. Thus,
if the control circuit 106 detects an error between the actual
laser pulse energy and the modeled pulse energy, then the control
circuit 106 can introduce an offset or other adjustment into model
108 to account for this error.
[0116] For example, it may be necessary to update the values for a
and b to reflect actual operational characteristics of the laser
source 102. As noted above, the values of a and b define how much
energy is transferred from the fiber amplifier 116 into the laser
pulse 122 when the laser source 102 is triggered and the seed laser
114 induces the pulse 122 to exit the fiber amplifier 116. An
updated value for a can be computed from the monitored energies in
transmitted pulses 122 (PE) as follows:
a=argmin.sub.a(.SIGMA..sub.k=1 . . .
N|PE(t.sub.k+.delta..sub.k)-aPE(t.sub.k)-(1-a).delta.t.sub.k|.sup.2)
[0117] In this expression, the values for PE represent the actual
pulse energies at the referenced times (t.sub.k or
t.sub.k+.delta..sub.k). This is a regression problem and can be
solved using commercial software tools such as those available from
MATLAB, Wolfram, PTC, ANSYS, and others. In an ideal world, the
respective values for PE(t) and PE(t+.delta.) will be the same as
the modeled values of EF(t) and EF(t+.delta.), However, for a
variety of reasons, the gain factors a and b may vary due to laser
efficiency considerations (such as heat or aging whereby back
reflections reduce the resonant efficiency in the laser cavity).
Accordingly, a practitioner may find it useful to update the model
108 over time to reflect the actual operational characteristics of
the laser source 102 by periodically computing updated values to
use for a and b.
[0118] In scenarios where the laser source 102 does not report its
own actual laser pulse energies, a practitioner can choose to
include a photodetector at or near an optical exit aperture of the
lidar transmitter 100 (e.g., see photodetector 252 in FIG. 2C). The
photodetector 252 can be used to measure the energy within the
transmitted laser pulses 122 (while allowing laser pulses 122 to
propagate into the environment toward their targets), and these
measured energy levels can be used to detect potential errors with
respect to the modeled energies for the laser pulses so model 108
can be adjusted as noted above. As another example for use in a
scenario where the laser source 102 does not report its own actual
laser pulse energies, a practitioner derives laser pulse energy
from return data 254 with respect to returns from known fiducial
objects in a field of view (such as road signs which are regulated
in terms of their intensity values for light returns) (see 254 in
FIG. 2D) as obtained from a point cloud 256 for the lidar system.
Additional details about such energy derivations are discussed
below. Thus, in such an example, the model 108 can be periodically
re-calibrated using point cloud data for returns from such
fiducials, whereby the control circuit 106 derives the laser pulse
energy that would have produced the pulse return data found in the
point cloud 256. This derived amount of laser pulse energy can then
be compared with the modeled laser pulse energy for adjustment of
the laser energy model 108 as noted above.
[0119] Modeling Mirror Motion Over Time:
[0120] In a particularly powerful example embodiment, the control
circuit 106 can also model mirror motion to predict where the
mirror subsystem 104 will be aimed at a given point in time. This
can be especially helpful for lidar transmitters 100 that
selectively target specific range points in the field of view with
laser pulses 122. By coupling the modeling of laser energy with a
model of mirror motion, the control circuit 106 can set the order
of specific laser pulse shots to be fired to targeted range points
with highly granular and optimized time scales. As discussed in
greater detail below, the mirror motion model can model mirror
motion over short time intervals (such as over time intervals in a
range from 5-50 nanoseconds). Such a short interval mirror motion
model can be referred to as a transient mirror motion model.
[0121] FIG. 3 shows an example lidar transmitter 100 where the
control circuit 106 uses both a laser energy model 108 and a mirror
motion model 308 to govern the timing schedule for laser pulses
122.
[0122] In an example embodiment, the mirror subsystem 104 can
operate as discussed above in connection with FIG. 1. For example,
the control circuit 106 can (1) drive mirror 110 in a resonant mode
using a sinusoidal signal to scan mirror 110 across different
azimuth angles and (2) drive mirror 112 in a point-to-point mode
using a step signal to scan mirror 112 across different elevations,
where the step signal will vary as a function of the elevations of
the range points to be targeted with laser pulses 122. Mirror 110
can be scanned as a fast-axis mirror, while mirror 112 is scanned
as a slow-axis mirror. In such an embodiment, a practitioner can
choose to use the mirror motion model 308 to model the motion of
mirror 110 as (comparatively) mirror 112 can be characterized as
effectively static for one or more scans across azimuth angles.
[0123] FIGS. 4A-4C illustrate how the motion of mirror 110 can be
modeled over time. In these examples, (1) the angle theta (.theta.)
represents the tilt angle of mirror 110, (2) the angle phi (.PHI.)
represents the angle at which a laser pulse 122 from the laser
source 102 will be incident on mirror 110 when mirror 110 is in a
horizontal position (where .theta. is zero degrees--see FIG. 4A),
and (3) the angle mu (.mu.) represents the angle of pulse 422 as
reflected by mirror 110 relative to the horizontal position of
mirror 110. In this example, the angle .mu. can represent the scan
angle of the mirror 110, where this scan angle can also be referred
to as a shot angle for mirror 110 as angle .mu. corresponds to the
angle at which reflected laser pulse 122' will be directed into the
field of view if fired at that time.
[0124] FIG. 4A shows mirror 110, where mirror 110 is at "rest" with
a tilt angle .theta. of zero degrees, which can be characterized as
the horizon of mirror 110. Laser source 102 is oriented in a fixed
position so that laser pulses 122 will impact mirror 110 at the
angle .PHI. relative to the horizontal position of mirror 110.
Given the property of reflections, it should be understood that the
value of the shot angle .mu. will be the same as the value of angle
.PHI. when the mirror 110 is horizontal (where .theta.=0).
[0125] FIG. 4B shows mirror 110 when it has been tilted about pivot
402 to a positive non-zero value of .theta.. It can be seen that
the tilting of mirror to angle .theta. will have the effect of
steering the reflected laser pulse 122' clockwise and to the right
relative to the angle of the reflected laser pulse 122' in FIG. 4A
(when mirror 110 was horizontal).
[0126] Mirror 110 will have a maximum tilt angle that can be
referred to as the amplitude A of mirror 110. Thus, it can be
understood that mirror 110 will scan through its tilt angles
between the values of -A (which corresponds to -.theta..sub.Max)
and +A (which corresponds to +.theta..sub.Max). It can be seen that
the angle of reflection for the reflected laser pulse 122' relative
to the actual position of mirror 110 is the sum of .theta.+.PHI. as
shown by FIG. 4B. In then follows that the value of the shot angle
.mu. will be equal to 2.theta.+.PHI., as can be seen from FIG.
4B.
[0127] When driven in a resonant mode according to sinusoidal
control signal, mirror 110 will change its tilt angle .theta.
according to a cosine oscillation, where its rate of change is
slowest at the ends of its scan (when it changes its direction of
tilt) and fastest at the mid-point of its scan. In an example where
the mirror 110 scans between maximum tilt angles of -A to +A, the
value of the angle .theta. as a function of time can be expressed
as:
.theta.=A cos(2.pi.ft)
where f represents the scan frequency of mirror 110 and t
represents time. Based on this model, it can be seen that the value
for .theta. can vary from A (when t=0) to 0 (when t is a value
corresponding to 90 degrees of phase (or 270 degrees of phase) to
-A (when t is a value corresponding to 180 degrees of phase).
[0128] This means that the value of the shot angle .mu. can be
expressed as a function of time by substituting the cosine
expression for .theta. into the expression for the shot angle of
.mu.=2.theta.+.PHI. as follows:
.mu.=2A cos(2.pi.ft)+.phi.
[0129] From this expression, one can then solve for t to produce an
expression as follows:
t = arccos .function. ( .mu. - .phi. 2 .times. A ) 2 .times. .pi.
.times. f ##EQU00004##
[0130] This expression thus identifies the time t at which the scan
of mirror 110 will target a given shot angle .mu.. Thus, when the
control circuit 106 wants to target a shot angle of .mu., the time
at which mirror 110 will scan to this shot angle can be readily
computed given that the values for .PHI., A, and f will be known.
In this fashion, the mirror motion model 308 can model that shot
angle as a function of time and predict the time at which the
mirror 110 will target a particular shot angle.
[0131] FIG. 4C shows mirror 110 when it has been tilted about pivot
402 to a negative non-zero value of -.theta.. It can be seen that
the tilting of mirror to angle -.theta. will have the effect of
steering the reflected laser pulse 122' counterclockwise and to the
left relative to the angle of the reflected laser pulse 122' in
FIG. 4A (when mirror 110 was horizontal). FIG. 4C also demonstrates
a constraint for a practitioner on the selection of the value for
the angle .PHI.. Laser source 102 will need to be positioned so
that the angle .PHI. is greater than the value of A to avoid a
situation where the underside of the tilted mirror 110 occludes the
laser pulse 122 when mirror is tilted to a value of .theta. that is
greater than .PHI.. Furthermore, the value of the angle .PHI.
should not be 90.degree. to avoid a situation where the mirror 110
will reflect the laser pulse 122 back into the laser source 102. A
practitioner can thus position the laser source 102 at a suitable
angle .PHI. accordingly.
[0132] FIG. 4D illustrates a translation of this relationship to
how the mirror 110 scans across a field of view 450. The mirror 110
will alternately scan in a left-to-right direction 452 and
right-to-left direction 454 as mirror 110 tilts between its range
of tilt angles (e.g., .theta.=-A through +A). For the example of
FIG. 4A where the value for .theta. is zero, this means that a
laser pulse fired at the untilted mirror 110 will be directed as
shown by 460 in FIG. 4D, where the laser pulse is directed toward a
range point at the mid-point of scan. The shot angle .mu. for this
"straight ahead" gaze is .PHI. as discussed above in connection
with FIG. 4A. As the angle .theta. increases from .theta.=0, this
will cause the laser pulses directed by mirror 110 to scan to the
right in the field of view until the mirror 110 tilts to the angle
.theta.=+A. When .theta.=+A, mirror 110 will be at the furthest
extent of its rightward scan 452, and it will direct a laser pulse
as shown by 462. The shot angle .mu. for this rightmost scan
position will be the value .mu.=2A+.PHI.. From that point, the
mirror 110 will begin scanning leftward in direction 454 by
reducing its tilt angle .theta.. The mirror 110 will once again
scan through the mid-point and eventually reach a tilt angle of
.theta.=-A. When .theta.=-A, mirror 110 will be at the furthest
extent of its leftward scan 452, and it will direct a laser pulse
as shown by 464. The shot angle .mu. for this leftmost scan
position will be the value .mu.=.PHI.-2A. From that point, the
mirror 110 will begin tilting in the rightward direction 450 again,
and the scan repeats. As noted above, due to the mirror motion
model 308, the control circuit 106 will know the time at which the
mirror 110 is targeting a shot angle of .mu..sub.i to direct a
laser pulse as shown by 466 of FIG. 4D.
[0133] In an example embodiment, the values for +A and -A can be
values in a range between +/-10 degrees and +/-20 degrees (e.g.,
+/-16 degrees) depending on the nature of mirror chosen as mirror
110. In an example where A is 16 degrees and mirror 110 scans as
discussed above in connection with FIGS. 4A-4D, it can be
understood that the angular extent of the scan for mirror 110 would
be 64 degrees (or 2A from the scan mid-point in both the right and
left directions for a total of 4A).
[0134] In some example embodiments, the value for A in the mirror
motion model 308 can be a constant value. However, some
practitioners may find it desirable to deploy a mirror 110 that
exhibits an adjustable value for A (e.g., a variable amplitude
mirror such as a variable amplitude MEMS mirror can serve as mirror
110). From the relationships discussed above, it can be seen that
the time required to move between two shot angles is reduced when
the value for amplitude A is reduced. The control circuit 106 can
leverage this relationship to determine whether it is desirable to
adjust the amplitude of the mirror 110 before firing a sequence of
laser pulses 122. FIG. 4E shows an example process flow in this
regard. At step 470, the control circuit 106 determines the settle
time (ts) for changing the amplitude from A to A' (where A'<A).
It should be understood that changing the mirror amplitude in this
fashion will introduce a time period where the mirror is relatively
unstable, and time will need to be provided to allow the mirror to
settle down to a stable position. This settling time can be
empirically determined or tracked for the mirror 110, and the
control circuit 106 can maintain this settle time value as a
control parameter. At step 472, the control circuit 106 determines
the time it will take to collect a shot list data set in a
circumstance where the amplitude of the mirror is unchanged
(amplitude remains A). This time can be referenced as collection
time tc. This value for tc can be computed through the use of the
laser energy model 108 and mirror motion model 308 with reference
to the shots included in a subject shot list. At step 474, the
control circuit 106 determines the time it will take to collect the
same shot list data set in a circumstance where the amplitude of
the mirror is changed to A'. This time can be referenced as
collection time tc'. This value for tc' can be computed through the
use of the laser energy model 108 and mirror motion model 308 (as
adjusted in view of the reduced amplitude of A') with reference to
the shots included in the subject shot list. At step 476, the
control circuit compares tc with the sum of tc' and ts. If the sum
(tc'+ts) is less than tc, this means that it will be time efficient
to change the mirror amplitude to A'. In this circumstance, the
process flow proceeds to step 478, and the control circuit 106
adjusts the amplitude of mirror 110 to A'. If the sum (tc'+ts) is
not less than tc, then the control circuit 106 leaves the amplitude
value unchanged (step 480).
[0135] Model-Based Shot Scheduling:
[0136] FIG. 5 shows an example process flow for the control circuit
106 to use both the laser energy model 108 and the mirror motion
model 308 to determine the timing schedule for laser pulses 122.
Step 200 can operate as described above with reference to FIG. 2A
to maintain the laser energy model 108. At step 500, the control
circuit 106 maintains the mirror motion model 308. As discussed
above, this model 308 can model the shot angle that the mirror will
target as a function of time. Accordingly, the mirror motion model
308 can predict the shot angle of mirror 110 at a given time t. To
maintain and update the model 308, the control circuit 108 can
establish the values for A, .PHI., and f to be used for the model
308. These values can be read from memory or determined from the
operating parameters for the system.
[0137] At step 502, the control circuit 106 determines a timing
schedule for laser pulses 122 using the laser energy model 108 and
the mirror motion model 308. By linking the laser energy model 108
and the mirror motion model 308 in this regard, the control circuit
106 can determine how much energy is available for laser pulses
targeted toward any of the range points in the scan pattern of
mirror subsystem 104. For purposes of discussion, we will consider
an example embodiment where mirror 110 scans in azimuth between a
plurality of shot angles at a high rate while mirror 112 scans in
elevation at a sufficiently slower rate so that the discussion
below will assume that the elevation is held steady while mirror
110 scans back and forth in azimuth. However, the techniques
described herein can be readily extended to modeling the motion of
both mirrors 110 and 112.
[0138] If there is a desire to target a range point at a Shot Angle
A with a laser pulse of at least X units of energy, the control
circuit 106, at step 502, can consult the laser energy model 108 to
determine whether there is sufficient laser energy for the laser
pulse when the mirror 110's scan angle points at Shot Angle A. If
there is sufficient energy, the laser pulse 122 can be fired when
the mirror 110 scans to Shot Angle A. If there is insufficient
energy, the control circuit 106 can wait to take the shot until
after mirror 110 has scanned through and back to pointing at Shot
Angle A (if the laser energy model 108 indicates there is
sufficient laser energy when the mirror returns to Shot Angle A).
The control circuit 106 can compare the shot energy requirements
for a set of shot angles to be targeted with laser pulses to
determine when the laser pulses 122 should be fired. Upon
determination of the timing schedule for the laser pulses 122, the
control circuit 106 can generate and provide firing commands 120 to
the laser source 102 based on this determined timing schedule (step
504).
[0139] FIGS. 6A and 6B show example process flows for implementing
steps 502 and 504 of FIG. 5 in a scenario where the mirror
subsystem 104 includes mirror 110 that scans through azimuth shot
angles in a resonant mode (fast-axis) and mirror 112 that scans
through elevation shot angles in a point-to-point mode (slow-axis).
Lidar transmitter 100 in these examples seeks to fire laser pulses
122 at intelligently selected range points in the field of view.
With the example of FIG. 6A, the control circuit 106 schedules
shots for batches of range points at a given elevation on whichever
scan direction of the mirror 110 is schedulable for those range
points according to the laser energy model 108. With the example of
FIG. 6B, the control circuit 106 seeks to schedule shots for as
many range points as it can at a given elevation for each scan
direction of the mirror 110 in view of the laser energy model 108.
For any shots at the subject elevation that cannot be scheduled for
a given scan direction due to energy model constraints, the control
circuit 106 then seeks to schedule those range points on the
reverse scan (and so on until all of the shots are scheduled).
[0140] The process flow of FIG. 6A begins with step 600. At step
600, the control circuit 106 receives a list of range points to be
targeted with laser pulses. These range points can be expressed as
(azimuth angle, elevation angle) pairs, and they may be ordered
arbitrarily.
[0141] At step 602, the control circuit 106 sorts the range points
by elevation to yield sets of azimuth shot angles sorted by
elevation. The elevation-sorted range points can also be sorted by
azimuth shot angle (e.g., where all of the shot angles at a given
elevation are sorted in order of increasing azimuth angle (smallest
azimuth shot angle to largest azimuth shot angle) or decreasing
azimuth angle (largest azimuth shot angle to smallest azimuth shot
angle). For the purposes of discussing the process flows of FIGS.
6A and 6B, these azimuth shot angles can be referred to as the shot
angles for the control circuit 106. Step 602 produces a pool 650 of
range points to be targeted with shots (sorted by elevation and
then by shot angle).
[0142] At step 604, the control circuit 106 selects a shot
elevation from among the shot elevations in the sorted list of
range points in pool 650. The control circuit 106 can make this
selection on the basis of any of a number of criteria. The order of
selection of the elevations will govern which elevations are
targeted with laser pulses 122 before others.
[0143] Accordingly, in an example embodiment, the control circuit
106 can prioritize the selection of elevations at step 604 that are
expected to encompass regions of interest in the field of view. As
an example, some practitioners may find the horizon in the field of
view (e.g., a road horizon) to be high priority for targeting with
laser pulses 122. In such a case, step 604 can operate as shown by
FIG. 17A to determine the elevation(s) which correspond to a
horizon in the field of view (e.g. identify the elevations at or
near the road horizon) (see step 1702) and then prioritize the
selection of those elevations from pool 650 (see step 1702). Step
1702 can be performed by analyzing lidar return point cloud data
and/or camera images of the field of view to identify regions in
the field of view that are believed to qualify as the horizon
(e.g., using contrast detection techniques, edge detection
techniques, and/or other pattern processing techniques applied to
lidar or image data).
[0144] As another example, the control circuit 106 can prioritize
the selection of elevations based on the range(s) to detected
object(s) in the field of view. Some practitioners may find it
desirable to prioritize the shooting of faraway objects in the
field of view. Other practitioners may find it desirable to
prioritize the shooting of nearby objects in the field of view.
Thus, in an example such as that shown by FIG. 17B, the range(s)
applicable to detected object(s) is determined (see step 1706).
This range information will be available from the lidar return
point cloud data. At step 1708, the control circuit sorts the
detected object(s) by their determined range(s). Then, at step
1708, the control circuit 106 prioritizes the selection of
elevations from pool 650 based on the determined range(s) for
object(s) included in those elevations. With step 1708,
prioritization can be given to larger range values than for smaller
range values if the practitioner wants to shoot faraway objects
before nearby objects. For practitioners that want to shoot nearby
objects before faraway objects, step 1708 can give priority to
smaller range values than for larger range values. Which objects
are deemed faraway and which are deemed nearby can be controlled
using any of a number of techniques. For example, a range threshold
can be defined, and the control circuit 106 can make the elevation
selections based on which elevations include sorted objects whose
range is above (or below as the case may be) the defined range
threshold. As another example, the relative ranges for the sorted
objects can be used to control the selection of elevations (where
the sort order of either farthest to nearest or nearest to farthest
governs the selection of elevations which include those
objects).
[0145] As yet another example, the control circuit 106 can
prioritize the selection of elevations based on the velocity(ies)
of detected object(s) in the field of view. Some practitioners may
find it desirable to prioritize the shooting of fast-moving objects
in the field of view. FIG. 17C shows an example process flow for
this. At step 1714, the velocity is determined for each detected
object in the field of view. This velocity information can be
derived from the lidar return point cloud data. At step 1716, the
control circuit 106 can sort the detected object(s) by the
determined velocity(ies). The control circuit 106 can then use
determined velocities for the sorted objects as a basis for
prioritizing the selection of elevations which contain those
detected objects (step 1718). This prioritization at step 1718 can
be carried out in any of a number of ways. For example, a velocity
threshold can be defined, and step 1718 can prioritize the
selection of elevation include an object moving at or above this
defined velocity threshold. As another example, the relative
velocities of the sorted objects can be used where an elevation
that includes an object moving faster than another object can be
selected before an elevation that includes the another (slower
moving) object.
[0146] As yet another example, the control circuit 106 can
prioritize the selection of elevations based on the directional
heading(s) of detected object(s) in the field of view. Some
practitioners may find it desirable to prioritize the shooting of
objects in the field of view that moving toward the lidar
transmitter 100. FIG. 17D shows an example process flow for this.
At step 1720, the directional heading is determined for each
detected object in the field of view. This directional heading can
be derived from the lidar return point cloud data. The control
circuit 1722 can then prioritize the selection of elevation(s) that
include object(s) that are determined to be heading toward the
lidar transmitter 100 (within some specified degree of tolerance
where the elevation that contains an object heading near the lidar
transmitter 100 would be selected before an elevation that contains
an object moving away from the lidar transmitter 100).
[0147] Further still, some practitioners may find it desirable to
combine the process flows of FIGS. 17C and 17D to prioritize the
selection of fast-moving objects that are heading toward the lidar
transmitter 100. An example for this is shown by FIG. 17E. With
FIG. 17E, steps 1714 and 1720 can be performed as discussed above.
At step 1724, the detected object(s) are sorted by their
directional headings (relative to the lidar transmitter 100) and
then by the determined velocities. At step 1726, the elevations
which contain objected deemed to be heading toward the lidar
transmitter 100 (and moving faster than other such objects) are
prioritized for selection.
[0148] In another example embodiment, the control circuit 106 can
select elevations at step 604 based on eye safety or camera safety
criteria. For example, eye safety requirements may specify that the
lidar transmitter 100 should not direct more than a specified
amount of energy in a specified spatial area over of a specified
time period. To reduce the risk of firing too much energy into the
specified spatial area, the control circuit 106 can select
elevations in a manner that avoids successive selections of
adjacent elevations (e.g., jumping from Elevation 1 to Elevation 3
rather than Elevation 2) to insert more elevation separation
between laser pulses that may be fired close in time. This manner
of elevation selection may optionally be implemented dynamically
(e.g., where elevation skips are introduced if the control circuit
106 determines that the energy in a defined spatial area has
exceeded some level that is below but approaching the eye safety
thresholds). Furthermore, it should be understood that the number
of elevations to skip (a skip interval) can be a value selected by
a practitioner or user to define how many elevations will be
skipped when progressing from elevation-to-elevation. As such, a
practitioner may choose to set the elevation skip interval to be a
value larger than 1 (e.g., a skip interval of 5, which would cause
the system to progress from Elevation 3 to Elevation 9).
Furthermore, similar measures can be taken to avoid hitting cameras
that may be located in the field of view with too much energy. FIG.
17F depicts an example process flow for this approach. At step
1730, the control circuit 106 selects Elevation X.sub.t (where this
selected elevation is larger (or smaller) than the preceding
selected elevation (Elevation X.sub.t-1) by the defined skip
interval. Then, the control circuit 106 schedules the shots for the
selected elevation (step 1732), and the process flow returns to
step 1730 where the next elevation (Elevation X.sub.t+1) is
selected (according to the skip interval relative to Elevation
X.sub.t).
[0149] Thus, it should be understood that step 604 can employ a
prioritized classification system that decides the order in which
elevations are to be targeted with laser pulses 122 based on the
criteria of FIGS. 17A-17F or any combinations of any of these
criteria.
[0150] At step 606, the control circuit 106 generates a mirror
control signal for mirror 112 to drive mirror 112 so that it
targets the angle of the selected elevation. As noted, this mirror
control signal can be a step signal that steps mirror 112 up (or
down) to the desired elevation angle. In this fashion, it can be
understood that the control circuit 106 will be driving mirror 112
in a point-to-point mode where the mirror control signal for mirror
112 will vary as a function of the range points to be targeted with
laser pulses (and more precisely, as a function of the order of
range points to be targeted with laser pulses).
[0151] At step 608, the control circuit 106 selects a window of
azimuth shot angles that are in the pool 650 at the selected
elevation. The size of this window governs how many shot angles
that the control circuit 106 will order for a given batch of laser
pulses 122 to be fired. This window size can be referred to as the
search depth for the shot scheduling. A practitioner can configure
the control circuit 106 to set this window size based on any of a
number of criteria. While the toy examples discussed below use a
window size of 3 for purposes of illustration, it should be
understood that practitioners may want to use a larger (or smaller)
window size in practice. For example, in an example embodiment, the
size of the window may be a value in a range between 2 shots and 12
shots. However, should the control circuit 106 have larger
capacities for parallel processing or should there be more lenient
time constraints on latency, a practitioner may find it desirable
to choose larger window sizes. Furthermore, the control circuit 106
can consider a scan direction for the mirror 110 when selecting the
shot angles to include in this window. Thus, if the control circuit
106 is scheduling shots for a scan direction corresponding to
increasing shot angles, the control circuit 106 can start from the
smallest shot angle in the sorted pool 650 and include
progressively larger shot angles in the shot angle sort order of
the pool 650. Similarly, if the control circuit 106 is scheduling
shots for a scan direction corresponding to decreasing shot angles,
the control circuit 106 can start from the largest shot angle in
the sorted pool 650 and include progressively smaller shot angles
in the shot angle sort order of the pool 650.
[0152] At step 610, the control circuit 106 determines an order for
the shot angles in the selected window using the laser energy model
108 and the mirror motion model 308. As discussed above, this
ordering operation can compare candidate orderings with criteria
such as energy requirements relating to the shots to find a
candidate ordering that satisfies the criteria. Once a valid
candidate ordering of shot angles is found, this can be used as
ordered shot angles that will define the timing schedule for the
selected window of laser pulses 122. Additional details about
example embodiments for implementing step 610 are discussed
below.
[0153] Once the shot angles in the selected window have been
ordered at step 610, the control circuit 106 can add these ordered
shot angles to the shot list 660. As discussed in greater detail
below, the shot list 660 can include an ordered listing of shot
angles and a scan direction corresponding to each shot angle.
[0154] At step 612, the control circuit 106 determines whether
there are any more shot angles in pool 650 to consider at the
selected elevation. In other words, if the window size does not
encompass all of the shot angles in the pool 650 at the selected
elevation, then the process flow can loop back to step 608 to grab
another window of shot angles from the pool 650 for the selected
elevation. If so, the process flow can then perform steps 610 and
612 for the shot angles in this next window.
[0155] Once all of the shots have been scheduled for the shot
angles at the selected elevation, the process flow can loop back
from step 612 to step 604 to select the next elevation from pool
650 for shot angle scheduling. As noted above, this selection can
proceed in accordance with a defined prioritization of elevations.
From there, the control circuit 106 can perform steps 606-614 for
the shot angles at the newly selected elevation.
[0156] Meanwhile, at step 614, the control circuit 106 generates
firing commands 120 for the laser source 102 in accordance with the
determined order of shot angles as reflected by shot list 660. By
providing these firing commands 120 to the laser source 102, the
control circuit 106 triggers the laser source 102 to transmit the
laser pulses 122 in synchronization with the mirrors 110 and 112 so
that each laser pulse 122 targets its desired range point in the
field of view. Thus, if the shot list includes Shot Angles A and C
to be fired at during a left-to-right scan of the mirror 110, the
control circuit 106 can use the mirror motion model 308 to identify
the times at which mirror 110 will be pointing at Shot Angles A and
C on a left-to-right scan and generate the firing commands 120
accordingly. The control circuit 106 can also update the pool 650
to mark the range points corresponding to the firing commands 120
as being "fired" to effectively remove those range points from the
pool 650.
[0157] In the example of FIG. 6B, as noted above, the control
circuit 106 seeks to schedule as many shots as possible on each
scan direction of mirror 110. Steps 600, 602, 604, and 606 can
proceed as described above for FIG. 6A.
[0158] At step 620, the control circuit 106 selects a scan
direction of mirror 110 to use for scheduling. A practitioner can
choose whether this scheduling is to start with a left-to-right
scan direction or a right-to-left scan direction. Then, step 608
can operate as discussed above in connection with FIG. 6A, but
where the control circuit 106 uses the scan direction selected at
step 620 to govern which shot angles are included in the selected
window. Thus, if the selected scan direction corresponds to
increasing shot angles, the control circuit 106 can start from the
smallest shot angle in the sorted pool 650 and include
progressively larger shot angles in the shot angle sort order of
the pool 650. Similarly, if the selected scan direction corresponds
to decreasing shot angles, the control circuit 106 can start from
the largest shot angle in the sorted pool 650 and include
progressively smaller shot angles in the shot angle sort order of
the pool 650.
[0159] At step 622, the control circuit 106 determines an order for
the shot angles based on the laser energy model 108 and the mirror
motion model 308 as discussed above for step 610, but where the
control circuit 106 will only schedule shot angles if the laser
energy model 108 indicates that those shot angles are schedulable
on the scan corresponding to the selected scan direction. Scheduled
shot angles are added to the shot list 660. But, if the laser
energy model 108 indicates that the system needs to wait until the
next return scan (or later) to take a shot at a shot angle in the
selected window, then the scheduling of that shot angle can be
deferred until the next scan direction for mirror 110 (see step
624). This effectively returns the unscheduled shot angle to pool
650 for scheduling on the next scan direction if possible.
[0160] At step 626, the control circuit 106 determines if there are
any more shot angles in pool 650 at the selected elevation that are
to be considered for scheduling on the scan corresponding to the
selected scan direction. If so, the process flow returns to step
608 to grab another window of shot angles at the selected elevation
(once again taking into consideration the sort order of shot angles
at the selected elevation in view of the selected scan
direction).
[0161] Once the control circuit 106 has considered all of the shot
angles at the selected elevation for scheduling on the selected
scan direction, the process flow proceeds to step 628 where a
determination is made as to whether there are any more unscheduled
shot angles from pool 650 at the scheduled elevation. If so, the
process flow loops back to step 620 to select the next scan
direction (i.e., the reverse scan direction). From there, the
process flow proceeds through steps 608, 622, 624, 626, and 628
until all of the unscheduled shot angles for the selected elevation
have been scheduled and added to shot list 660. Once step 628
results in a determination that all of the shot angles at the
selected elevation have been scheduled, the process flow can loop
back to step 604 to select the next elevation from pool 650 for
shot angle scheduling. As noted above, this selection can proceed
in accordance with a defined prioritization of elevations, and the
control circuit 106 can perform steps 606, 620, 608, 622, 624, 626,
628, and 614 for the shot angles at the newly selected
elevation.
[0162] Thus, it can be understood that the process flow of FIG. 6B
will seek to schedule all of the shot angles for a given elevation
during a single scan of mirror 110 (from left-to-right or
right-to-left as the case may be) if possible in view of the laser
energy model 108. However, should the laser energy model 108
indicate that more time is needed to fire shots at the desired shot
angles, then some of the shot angles may be scheduled for the
return scan (or subsequent scan) of mirror 110.
[0163] It should also be understood that the control circuit 106
will always be listening for new range points to be targeted with
new laser pulses 122. As such, steps 600 and 602 can be performed
while steps 604-614 are being performed (for FIG. 6A) or while
steps 604, 606, 620, 608, 622, 624, 626, 628, and 614 are being
performed (for FIG. 6B). Similarly, step 614 can be performed by
the control circuit 106 while the other steps of the FIGS. 6A and
6B process flows are being performed. Furthermore, it should be
understood that the process flows of FIGS. 6A and 6B can
accommodate high priority requests for range point targeting. For
example, as described in U.S. Pat. No. 10,495,757, the entire
disclosure of which is incorporated herein by reference, a request
may be received to target a set of range points in a high priority
manner. Thus, the control circuit 106 can also always be listening
for such high priority requests and then cause the process flow to
quickly begin scheduling the firing of laser pulses toward such
range points. In a circumstance where a high priority targeting
request causes the control circuit 106 to interrupt its previous
shot scheduling, the control circuit 106 can effectively pause the
current shot schedule, schedule the new high priority shots (using
the same scheduling techniques) and then return to the previous
shot schedule once laser pulses 122 have been fired at the high
priority targets.
[0164] Accordingly, as the process flows of FIGS. 6A and 6B work
their way through the list of range points in pool 650, the control
circuit 106 will provide improved scheduling of laser pulses 122
fired at those range points through use of the laser energy model
108 and mirror motion model 308 as compared to defined criteria
such as shot energy thresholds for those shots. Moreover, by
modeling laser energy and mirror motion over short time intervals
on the order of nanoseconds using transient models as discussed
above, these shot scheduling capabilities of the system can be
characterized as hyper temporal because highly precise shots with
highly precise energy amounts can be accurately scheduled over
short time intervals if necessary.
[0165] While FIGS. 6A and 6B show their process flows as an
iterated sequence of steps, it should be understood that if the
control circuit 106 has sufficient parallelized logic resources,
then many of the iterations can be unrolled and performed in
parallel without the need for return loops (or using a few number
of returns through the steps). For example, different windows of
shot angles at the selected elevation can be processed in parallel
with each other if the control circuit 106 has sufficient
parallelized logic capacity. Similarly, the control circuit 106 can
also work on scheduling for different elevations at the same time
if it has sufficient parallelized logic capacity.
[0166] FIG. 7A shows an example process flow for carrying out step
610 of FIG. 6A. At step 700, the control circuit 106 creates shot
angle order candidates from the shot angles that are within the
window selected at step 608. These candidates can be created based
on the mirror motion model 308.
[0167] For example, as shown by FIG. 7B, the times at which the
mirror 110 will target the different potential shot angles can be
predicted using the mirror motion model 308. Thus, each shot angle
can be assigned a time slot 710 with respect to the scan of mirror
110 across azimuth angles (and back). As shown by FIG. 7B, if
mirror 110 starts at Angle Zero at Time 1, it will then scan to
Angle A at Time 2, then scan to Angle B at Time 3, and so on
through its full range of angles (which in the example of FIG. 7B
reaches Angle J before the mirror 110 begins scanning back toward
Angle Zero). The time slots for these different angles can be
computed using the mirror motion model 308. Thus, if the window of
shot angles identifies Angle A, Angle C, and Angle I as the shot
angles, then the control circuit 106 will know which time slots of
the mirror scan for mirror 110 will target those shot angles. For
example, according to FIG. 7B, Time Slots 1, 3, and 9 will target
Angles A, C, and I. On the return scan, Time Slot 11 will also
target Angle I (as shown by FIG. 7B), while Time Slots 17 and 19
will also target Angles C and A respectively. As example
embodiments, the time slots 710 can correspond to time intervals in
a range between around 5 nanoseconds and around 50 nanoseconds,
which would correspond to angular intervals of around 0.01 to 0.1
degrees if mirror 110 is scanning at 12 kHz over an angular extent
of 64 degrees (where +/-A is +/-16 degrees).
[0168] To create the order candidates at step 700, the control
circuit 106 can generate different permutations of time slot
sequences for different orders of the shot angles in the selected
window. Continuing with an example where the shot angles are A, C,
and I, step 700 can produce the following set of example order
candidates (where each order candidate can be represented by a time
slot sequence):
TABLE-US-00005 Order Time Slot Candidate Sequence Comments
Candidate 1 1, 3, 9 This would correspond to firing laser pulses in
the shot angle order of ACI during the first scan for mirror 110
(which moves from left-to-right) Candidate 2 1, 9, 17 This would
correspond to firing laser pulses in the shot angle order of AIC,
where laser pulses are fired at Shot Angles A and I during the
first scan for mirror 110 and where the laser pulse is fired at
Shot Angle C during the second (return) scan for mirror 110 (where
this second scan moves from right-to-left). Candidate 3 3, 9, 19
This would correspond to firing laser pulses in the shot angle
order of CIA, where laser pulses are fired at Shot Angles C and I
during the first scan for mirror 110 and where the laser pulse is
fired at Shot Angle A during the second (return) scan for mirror
110. Candidate 4 3, 9, 21 This would correspond to firing laser
pulses in the shot angle order of CIA, where laser pulses are fired
at Shot Angles C and I during the first scan for mirror 110 and
where the laser pulse is fired at Shot Angle A during the third
scan for mirror 110 (which moves from left-to-right) . . . . . . .
. .
[0169] It should be understood that the control circuit 106 could
create additional candidate orderings from different permutations
of time slot sequences for Shot Angles A, C, and I. A practitioner
can choose to control how many of such candidates will be
considered by the control circuit 106.
[0170] At step 702, the control circuit 106 simulates the
performance of the different order candidates using the laser
energy model 108 and the defined shot requirements. As discussed
above, these shot requirements may include requirements such as
minimum energy thresholds for each laser pulse (which may be
different for each shot angle), maximum energy thresholds for each
laser pulse (or for the laser source), and/or desired energy levels
for each laser pulse (which may be different for each shot
angle).
[0171] To reduce computational latency, this simulation and
comparison with shot requirements can be performed in parallel for
a plurality of the different order candidates using parallelized
logic resources of the control circuit 106. An example of such
parallelized implementation of step 702 is shown by FIG. 7C. In the
example of FIG. 7C, steps 720, 722, and 724 are performed in
parallel with respect to a plurality of the different time slot
sequences that serve as the order candidates. Thus, steps 720a,
722a, and 724a are performed for Time Slot Sequence 1; steps 720b,
722b, and 724b are performed for Time Slot Sequence 2; and so on
through steps 720n, 722n, and 724n for Time Slot Sequence n.
[0172] At step 720, the control circuit 106 uses the laser energy
model 108 to predict the energy characteristics of the laser source
and resultant laser pulse if laser pulse shots are fired at the
time slots corresponding to the subject time slot sequence. These
modeled energies can then be compared to criteria such as a maximum
laser energy threshold and a minimum laser energy threshold to
determine if the time slot sequence would be a valid sequence in
view of the system requirements. At step 722, the control circuit
106 can label each tested time slot sequence as valid or invalid
based on this comparison between the modeled energy levels and the
defined energy requirements. At step 724, the control circuit 106
can compute the elapsed time that would be needed to fire all of
the laser pulses for each valid time slot sequence. For example,
Candidate 1 from the example above would have an elapsed time
duration of 9 units of time, while Candidate 2 from the example
above would have an elapsed time duration of 17 units of time.
[0173] FIGS. 7D, 7E, and 7F show examples of such simulations of
time slot sequences for our example where the shot angles to be
scheduled with laser pulses are Shot Angles A, C, and I. In this
scenario, we will assume that the laser energy model 108 will
employ (1) the value for E.sub.S as a constant value of 1 unit of
energy per unit of time and (2) the values for a and b as 0.5 each.
Furthermore, we will assume that there are 3 units of energy left
in the fiber laser 116 when the scan begins (and where the scan
begins at Angle Zero while moving from left-to-right). Moreover,
for the purposes of this example, the energy requirements for the
shots can be defined as (8,3,4) for minimum shot energies with
respect to shot angles A, C, and I respectively, and where the
maximum laser energy for the laser source can be defined as 20
units of combined seed and stored fiber energy (which would
translate to a maximum laser pulse energy of 10 units of
energy).
[0174] FIG. 7D shows an example result for simulating the time slot
sequence of laser pulses at time slots 1, 3, and 9. In this
example, it can be seen that this time slot sequence is invalid
because the shot energy for Time Slot 1 (targeting Shot Angle A) is
only 2 units of energy, which is below the minimum energy threshold
of 8 units for Shot Angle A. This time slot sequence also fails
because the shot energy for Time Slot 3 (targeting Shot Angle C) is
only 2 units of energy, which is below the minimum energy threshold
of 3 units for Shot Angle C.
[0175] FIG. 7E shows an example result for simulating the time slot
sequence of laser pulses at time slots 1, 9, and 17. In this
example, it can be seen that this time slot sequence is invalid
because the shot energy for Time Slot 1 (targeting Shot Angle A) is
too low.
[0176] FIG. 7F shows an example result for simulating the time slot
sequence of laser pulses at time slots 3, 9, and 21. In this
example, it can be seen that this time slot sequence is valid
because the shot energies for each time slot are at or above the
minimum energy thresholds for their corresponding shot angles (and
none of the time slots would violate the maximum energy threshold
for the laser source). It can be further surmised from FIG. 7F that
a simulation of a Time Slot Sequence of (3,9,19) also would have
failed because there is insufficient energy in a laser pulse that
would have been fired at Shot Angle A.
[0177] Accordingly, the simulation of these time slot sequences
would result in a determination that the time slot sequence of
(3,9,21) is a valid candidate, which means that this time slot
sequence can define the timing schedule for laser pulses fired
toward the shot angles in the selected window. The elapsed time for
this valid candidate is 21 units of time.
[0178] Returning to FIG. 7A, at step 704, the control circuit 106
selects the valid order candidate which has the lowest elapsed
time. Thus, in a scenario where the simulations at step 702 would
have produced two or more valid order candidates, the control
circuit 106 will select the order candidate that will complete its
firing of laser pulses the soonest which helps improve the latency
of the system.
[0179] For example embodiments, the latency with which the control
circuit 106 is able to determine the shot angle order and generate
appropriate firing commands is an important operational
characteristic for the lidar transmitter 100. To maintain high
frame rates, it is desirable for the control circuit 106 to carry
out the scheduling operations for all of the shot angles at a
selected elevation in the amount of time it takes to scan mirror
110 through a full left-to-right or right-to-left scan if feasible
in view of the laser energy model 108 (where this time amount is
around 40 microseconds for a 12 kHz scan frequency). Moreover, it
is also desirable for the control circuit 106 to be able to
schedule shots for a target that is detected based on returns from
shots on the current scan line during the next return scan (e.g.,
when a laser pulse 122 fired during the current scan detects
something of interest that is to be interrogated with additional
shots (see FIG. 16 discussed above)). In this circumstance, the
detection path for a pulse return through a lidar receiver and into
a lidar point cloud generator where the target of interest is
detected will also need to be taken into account. This portion of
the processing is expected to require around 0.4 to 10
microseconds, which leaves around 30 microseconds for the control
circuit 106 to schedule the new shots at the region of interest
during the next return scan if possible. For a processor of the
control circuit 106 which has 2Gflops of processing per second
(which is a value available from numerous FPGA and ASIC vendors),
this amounts to 50 operations per update, which is sufficient for
the operations described herein. For example, the control circuit
106 can maintain lookup tables (LUTs) that contain pre-computed
values of shot energies for different time slots within the scan.
Thus, the simulations of step 702 can be driven by looking up
precomputed shot energy values for the defined shot angles/time
slots. The use of parallelized logic by the control circuit 106 to
accelerate the simulations helps contribute to the achievement of
such low latency. Furthermore, practitioners can adjust operational
parameters such as the window size (search depth) in a manner to
achieve desired latency targets.
[0180] FIG. 8 shows an example embodiment for the lidar transmitter
100 where the control circuit 106 comprises a system controller 800
and a beam scanner controller 802. System controller 800 and beam
scanner controller 802 can each include a processor and memory for
use in carrying out its tasks. The mirror subsystem 104 can be part
of beam scanner 810 (which can also be referred to as a lidar
scanner). Beam scanner controller 802 can be embedded as part of
the beam scanner 810. In this example, the system controller 800
can carry out steps 600, 602, 604, 608, 610, and 612 of FIG. 6A if
the control circuit 106 employs the FIG. 6A process flow (or steps
600, 602, 604, 620, 608, 622, 624, 626, and 628 of FIG. 6B if the
control circuit 106 employs the FIG. 6B process flow), while beam
scanner controller 802 carries out steps 606 and 614 for the FIGS.
6A and 6B process flows. Accordingly, once the system controller
800 has selected the elevation and the order of shot angles, this
information can be communicated from the system controller 800 to
the beam scanner controller 802 as shot elevation 820 and ordered
shot angles 822.
[0181] The ordered shot angles 822 can also include flags that
indicate the scan direction for which the shot is to be taken at
each shot angle. This scan direction flag will also allow the
system to recognize scenarios where the energy model indicates
there is a need to pass by a time slot for a shot angle without
firing a shot and then firing the shot when the scan returns to
that shot angle in a subsequent time slot. For example, with
reference to the example above, the scan direction flag will permit
the system to distinguish between Candidate 3 (for the sequence of
shot angles CIA at time slots 3, 9, and 19) versus Candidate 4 (for
the same sequence of shot angles CIA but at time slots 3, 9, and
21). A practitioner can explicitly assign a scan direction to each
ordered shot angle by adding the scan direction flag to each
ordered shot angle if desired, or a practitioner indirectly assign
a scan direction to each ordered shot angle by adding the scan
direction flag to the ordered shot angles for which there is a
change in scan direction. Together, the shot elevations 802 and
order shot angles 822 serve as portions of the shot list 660 used
by the lidar transmitter 100 to target range points with laser
pulses 122.
[0182] The beam scanner controller 802 can generate control signal
806 for mirror 112 based on the defined shot elevation 820 to drive
mirror 112 to a scan angle that targets the elevation defined by
820. Meanwhile, the control signal 804 for mirror 110 will continue
to be the sinusoidal signal that drives mirror 110 in a resonant
mode. However, some practitioners may choose to also vary control
signal 804 as a function of the ordered shot angles 822 (e.g., by
varying amplitude A as discussed above).
[0183] In the example of FIG. 8, the mirror motion model 308 can
comprise a first mirror motion model 808a maintained and used by
the beam scanner controller 802 and a second mirror motion model
808b maintained and used by the system controller 800. With FIG. 8,
the task of generating the firing commands 120 can be performed by
the beam scanner controller 802. The beam scanner controller 810
can include a feedback system 850 that tracks the actual mirror
tilt angles .theta. for mirror 110. This feedback system 850
permits the beam scanner controller 802 to closely monitor the
actual tilt angles of mirror 110 over time which then translates to
the actual scan angles .mu. of mirror 110. This knowledge can then
be used to adjust and update mirror motion model 808a maintained by
the beam scanner controller 802. Because model 808a will closely
match the actual scan angles for mirror 110 due to the feedback
from 850, we can refer to model 808a as the "fine" mirror motion
model 808a. In this fashion, when the beam scanner controller 802
is notified of the ordered shot angles 822 to be targeted with
laser pulses 122, the beam scanner controller 802 can use this
"fine" mirror motion model 808a to determine when the mirror has
hit the time slots which target the ordered shot angles 822. When
these time slots are hit according to the "fine" mirror motion
model 808a, the beam scanner controller 802 can generate and
provide corresponding firing commands 120 to the laser source
102.
[0184] Examples of techniques that can be used for the scan
tracking feedback system 850 are described in the above-referenced
and incorporated U.S. Pat. No. 10,078,133. For example, the
feedback system 850 can employ optical feedback techniques or
capacitive feedback techniques to monitor and adjust the scanning
(and modeling) of mirror 110. Based on information from the
feedback system 850, the beam scanner controller 802 can determine
how the actual mirror scan angles may differ from the modeled
mirror scan angles in terms of frequency, phase, and/or maximum
amplitude. Accordingly, the beam scanner controller 802 can then
incorporate one or more offsets or other adjustments relating the
detected errors in frequency, phase, and/or maximum amplitude into
the mirror motion model 808a so that model 808a more closely
reflects reality. This allows the beam scanner controller 802 to
generate firing commands 120 for the laser source 102 that closely
match up with the actual shot angles to be targeted with the laser
pulses 122.
[0185] Errors in frequency and maximum amplitude within the mirror
motion model 808a can be readily derived from the tracked actual
values for the tilt angle .theta. as the maximum amplitude A should
be the maximum actual value for .theta., and the actual frequency
is measurable based on tracking the time it takes to progress from
actual values for A to -A and back.
[0186] Phased locked loops (or techniques such as PID control, both
available as software tools in MATLAB) can be used to track and
adjust the phase of the model 808a as appropriate. The expression
for the tilt angle .theta. that includes a phase component (p) can
be given as:
.theta.=A cos(2.pi.ft+p)
[0187] From this, we can recover the value for the phase p by the
relation:
.theta..apprxeq.A cos(2.pi.ft)-A sin(2.pi.ft)p
[0188] Solving for p, this yields the expression:
p = A .times. cos .function. ( 2 .times. .pi. .times. ft ) -
.theta. A .times. sin .function. ( 2 .times. .pi. .times. ft )
##EQU00005##
[0189] Given that the tracked values for A, f, t, and .theta. are
each known, the value for p can be readily computed. It should be
understood that this expression for p assumes that the value of the
p is small, which will be an accurate assumption if the actual
values for A, f, t, and .theta. are updated frequently and the
phase is also updated frequently. This computed value of p can then
be used by the "fine" mirror motion model 808a to closely track the
actual shot angles for mirror 110, and identify the time slots that
correspond to those shot angles according to the expression:
t = arccos .function. ( .mu. - .phi. 2 .times. A ) - p 2 .times.
.pi. .times. f ##EQU00006##
[0190] While a practitioner will find it desirable for the beam
scanner controller 802 to rely on the highly accurate "fine" mirror
motion model 808a when deciding when the firing commands 120 are to
be generated, the practitioner may also find that the shot
scheduling operations can suffice with less accurate mirror motion
modeling. Accordingly, the system controller 800 can maintain its
own model 808b, and this model 808b can be less accurate than model
808a as small inaccuracies in the model 808b will not materially
affect the energy modeling used to decide on the ordered shot
angles 822. In this regard, model 808b can be referred to as a
"coarse" mirror motion model 808b. If desired, a practitioner can
further communicate feedback from the beam scanner controller 802
to the system controller 800 so the system controller 800 can also
adjusts its model 808b to reflect the updates made to model 808a.
In such a circumstance, the practitioner can also decide on how
frequently the system will pass these updates from model 808a to
model 808b.
[0191] Marker Shots to Bleed Off and/or Regulate Shot Energy:
[0192] FIG. 9 depicts an example process flow for execution by the
control circuit 106 to insert marker shots into the shot list in
order to bleed off energy from the laser source 102 when needed. As
discussed above, the control circuit 106 can consult the laser
energy model 108 as applied to the range points to be targeted with
laser pulses 122 to determine whether a laser energy threshold
would be violated. If so, the control circuit 106 may insert a
marker shot into the shot list to bleed energy out of the laser
source 102 (step 902). In an example embodiment, this threshold can
be set to define a maximum or peak laser energy threshold so as to
avoid damage to the laser source 102. In another example
embodiment, this threshold can be set to achieve a desired
consistency, smoothness, and/or balance in the energies of the
laser pulse shots. For example, the marker shot can be fired to
bleed off energy so that a later targeted laser pulse shot (or set
of shots) exhibits a desired amount of energy.
[0193] For example, one or more marker shots can be fired to bleed
off energy so that a later targeted laser pulse shot (or set of
targeted shots) exhibits a desired amount of energy. As an example
embodiment, the marker shots can be used to bleed off energy so
that the targeted laser pulse shots exhibit consistent energy
levels despite a variable rate of firing for the targeted laser
pulse shots (e.g., so that the targeted laser pulse shots will
exhibit X units of energy (plus or minus some tolerance) even if
those targeted laser pulse shots are irregularly spaced in time).
The control circuit 106 can consult the laser energy model 108 to
determine when such marker shots should be fired to regulate the
targeted laser pulse shots in this manner.
[0194] Modeling Eye and Camera Safety Over Time:
[0195] FIG. 10 depicts an example process flow for execution by the
control circuit 106 where eye safety requirements are also used to
define or adjust the shot list. To support these operations, the
control circuit 106 can also, at step 1000, maintain an eye safety
model 1002. Eye safety requirements for a lidar transmitter 100 may
be established to define a maximum amount of energy that can be
delivered within a defined spatial area in the field of view over a
defined time period. Since the system is able to model per pulse
laser energy with respect to precisely targeted range points over
highly granular time periods, this allows the control circuit 106
to also monitor whether a shot list portion would violate eye
safety requirements. Thus, the eye safety model 1002 can model how
much aggregated laser energy is delivered to the defined spatial
area over the defined time period based on the modeling produced
from the laser energy model 108 and the mirror motion model 308. At
step 1010, the control circuit 106 uses the eye safety model 1002
to determine whether the modeled laser energy that would result
from a simulated sequence of shots would violate the eye safety
requirements. If so, the control circuit can adjust the shot list
to comply with the eye safety requirements (e.g., by inserting
longer delays between ordered shots delivered close in space, by
re-ordering the shots, etc.)
[0196] FIG. 11 shows an example lidar transmitter 100 that is
similar in nature to the example of FIG. 8, but where the system
controller 800 also considers the eye safety model 1002 when
deciding on how to order the shot angles. FIG. 12 shows how the
simulation step 702 from FIG. 7A can be performed in example
embodiments where the eye safety model 1002 is used. As shown by
FIG. 12, each parallel path can include steps 720, 722, and 724 as
discussed above. Each parallel path can also include a step 1200 to
be performed prior to step 722 where the control circuit 106 uses
the eye safety model 1002 to test whether the modeled laser energy
for the subject time slot sequence would violate eye safety
requirements. If the subject time slot sequence complies with the
criteria tested at steps 720 and 1200, then the subject time slot
sequence can be labeled as valid. If the subject time slot sequence
violates the criteria tested at steps 720 or 1200, then the subject
time slot sequence can be labeled as invalid.
[0197] Similar to the techniques described for eye safety in
connection with Figured 10, 11, and 12, it should be understood
that a practitioner can also use the control circuit to model and
evaluate whether time slot sequences would violate defined camera
safety requirements. To reduce the risk of laser pulses 122
impacting on and damaging cameras in the field of view, the control
circuit can also employ a camera safety model in a similar manner
and toward similar ends as the eye safety model 1002. In the camera
safety scenario, the control circuit 106 can respond to detections
of objects classified as cameras in the field of view by monitoring
how much aggregated laser energy will impact that camera object
over time. If the model indicates that the camera object would have
too much laser energy incident on it in too short of a time period,
the control circuit can adjust the shot list as appropriate.
[0198] Moreover, as noted above with respect to the laser energy
model 108 and the mirror motion model 308, the eye safety and
camera safety models can track aggregated energy delivered to
defined spatial areas over defined time periods over short time
intervals, and such short interval eye safety and camera safety
models can be referred to as transient eye safety and camera safety
models.
Additional Example Embodiments
[0199] FIG. 13 shows another example of a process flow for the
control circuit 106 with respect to using the models to dynamically
determine the shot list for the transmitter 100.
[0200] At step 1300, the laser energy model 108 and mirror motion
model 308 are established. This can include determining from
factory or calibration the values to be used in the models for
parameters such as E.sub.P, a, b, and A. Step 1300 can also include
establishing the eye safety model 1002 by defining values for
parameters that govern such a model (e.g. parameters indicative of
limits for aggregated energy for a defined spatial area over a
defined time period). At step 1302, the control law for the system
is connected to the models established at step 1300.
[0201] At step 1304, the seed energy model used by the laser energy
model 108 is adjusted to account for nonlinearities. This can
employ the clipped, offset (affine) model for seed energy as
discussed above.
[0202] At step 1306, the laser energy model 108 can be updated
based on lidar return data and other feedback from the system. For
example, as noted above in connection with FIG. 2D, the actual
energies in laser pulses 122 can be derived from the pulse return
data included in point cloud 256. For example, the pulse return
energy can be modeled as a function of the transmitted pulse energy
according to the following expression (for returns from objects
that are equal to or exceed the laser spot size and assuming modest
atmospheric attenuation):
Pulse .times. Return .times. Energy = ( PEApertur .times. e R
.times. e .times. c .times. e .times. iver .pi. .times. R 2 )
.times. Reflectivity ##EQU00007##
[0203] In this expression, Pulse Return Energy represents the
energy of the pulse return (which is known from the point cloud
256), PE represents the unknown energy of the transmitted laser
pulse 122, Aperture.sub.Receiver represents the known aperture of
the lidar receiver (see 1400 in FIG. 14), R represents the measured
range for the return (which is known from the point cloud 256), and
Reflectivity represents the percentage of reflectivity for the
object from which the return was received. Therefore, one can solve
for PE so long as the reflectivity is known. This will be the case
for objects like road signs whose reflectivity is governed by
regulatory agencies. Accordingly, by using returns from known
fiducials such as road signs, the control circuit 106 can derive
the actual energy of the transmitted laser pulse 122 and use this
value to facilitate determinations as to whether any adjustments to
the laser energy model 108 are needed (e.g., see discussions above
re updating the values for a and b based on PE values which
represent the actual energies of the transmitted laser pulses
122).
[0204] Also, at step 1308, the laser health can be assessed and
monitored as a background task. The information derived from the
feedback received for steps 1306 and 1308 can be used to update
model parameters as discussed above. For example, as noted above,
the values for the seed energy model parameters as well as the
values for a and b can be updated by measuring the energy produced
by the laser source 102 and fitting the data to the parameters.
Techniques which can be used for this process include least
squares, sample matrix inversion, regression, and multiple
exponential extensions. Further still, as noted above, the amount
of error can be reduced by using known targets with a given
reflectivity and using these to calibrate the system. This is
helpful because the reflectivity of a quantity that is known, i.e.
a fiducial, allows one to explicitly extract shot energy (after
backing out range dependencies and any obliquity). Examples of
fiducials that may be employed include road signs and license
plates.
[0205] At step 1310, the lidar return data and the coupled models
can be used to ensure that the laser pulse energy does not exceed
safety levels. These safety levels can include eye safety as well
as camera safety as discussed above. Without step 1310, the system
may need to employ a much more stringent energy requirement using
trial and error to establish laser settings to ensure safety. For
example if we only had a laser model where the shot energy is
accurate to only .dbd.3J per shot around the predicted shot, and
maximum shot energy is limited to 8, we could not use any shots
predicted to exceed 5. However, the hyper temporal modeling and
control that is available from the laser energy model 108 and
mirror motion model 308 as discussed herein allows us to obtain
accurate predictions within a few percent error, virtually erasing
the operational lidar impact of margin.
[0206] At step 1312, the coupled models are used with different
orderings of shots, thereby obtaining a predicted shot energy in
any chosen ordered sequence of shots drawn from the specified list
of range points. Step 1312 may employ simulations to predict shot
energies for different time slots of shots as discussed above.
[0207] At step 1314, the system inserts marker shots in the timing
schedule if the models predict that too much energy will build up
in the laser source 102 for a given shot sequence. This reduces the
risk of too much energy being transferred into the fiber laser 116
and causing damage to the fiber laser 116.
[0208] At step 1316, the system determines the shot energy that is
needed to detect targets with each shot. These values can be
specified as a minimum energy threshold for each shot. The value
for such threshold(s) can be determined from radiometric modeling
of the lidar, and the assumed range and reflectivity of a candidate
target. In general, this step can be a combination of modeling
assumptions as well as measurements. For example, we may have
already detected a target, so the system may already know the range
(within some tolerance). Since the energy required for detection is
expected to vary as the square of the range, this knowledge would
permit the system to establish the minimum pulse energy thresholds
so that there will be sufficient energy in the shots to detect the
targets.
[0209] Steps 1318 and 1320 operate to prune the candidate ordering
options based on the energy requirements (e.g., minimum energy
thresholds per shot) (for step 1318) and shot list firing
completion times (to favor valid candidate orderings with faster
completion times) (for step 1320).
[0210] At step 1322, candidate orderings are formed using elevation
movements on both scan directions. This allows the system to
consider taking shots on both a left-to-right scan and a
right-to-left scan. For example, suppose that the range point list
has been completed on a certain elevation, when the mirror is close
to the left hand side. Then it is faster to move the elevation
mirror at that point in time and begin the fresh window of range
points to be scheduled beginning on this same left hand side and
moving right. Conversely, if we deplete the range point list when
the mirror is closer to the right hand side it is faster to move
the mirror in elevation whilst it is on the right hand side.
Moreover, in choosing an order from among the order candidates, and
when moving from one elevation to another, movement on either side
of the mirror motion, the system may move to a new elevation when
mirror 110 is at one of its scan extremes (full left or full
right). However, in instances where a benefit may arise from
changing elevations when mirror 110 is not at one of its scan
extremes, the system may implement interline skipping as described
in the above-referenced and incorporated U.S. Pat. No. 10,078,133.
The mirror motion model 308 can also be adjusted to accommodate
potential elevation shift during a horizontal scan.
[0211] At step 1324, if processing time allows the control circuit
106 to implement auctioning (whereby multiple order candidates are
investigated, the lowest "cost" (e.g., fastest lidar execution
time) order candidate is selected by the control circuit 106
(acting as "auctioneer"). A practitioner may not want the control
circuit to consider all of the possible order candidates as this
may be too computationally expensive and introduce an undue amount
of latency. Thus, the control circuit 106 can enforce maximums or
other controls on how many order candidates are considered per
batch of shots to be ordered. Greedy algorithms can be used when
choosing ordering shots. Generally, the system can use a search
depth value (which defines how many shots ahead the control circuit
will evaluate) in this process in a manner that is consistent with
any real time consideration in shot list generation. At step 1326,
delays can be added in the shot sequence to suppress a set of shots
and thus increase available shot energy to enable a finer (denser)
grid as discussed above. The methodology for sorting through
different order candidates can be considered a special case of the
Viterbi algorithm which can be implemented using available software
packages such as Mathworks. This can also be inferred using
equivalence classes or group theoretic methods. Furthermore, if the
system detects that reduced latency is needed, the search depth can
be reduced (see step 1328).
[0212] FIG. 14 depicts an example embodiment for a lidar
transmitter 100 that shows how the system controller 800 can
interact with the lidar receiver 1400 to coordinate system
operations. The lidar receiver 1400 can receive and process pulse
returns 1402 to compute range information for objects in the field
of view impacted by the laser pulses 122. This range information
can then be included in the point cloud 1404 generated by the lidar
system. Examples of suitable technology for use as the lidar
receiver 1400 are described in U.S. Pat. Nos. 9,933,513 and
10,754,015, the entire disclosures of which are incorporated herein
by reference. In the example of FIG. 14, the system controller 800
can use the point cloud 1404 to intelligently select range points
for targeting with laser pulses, as discussed in the
above-referenced and incorporated patents. For example, the point
cloud data can be used to determine ranges for objects in the field
of view that are to be targeted with laser pulses 122. The control
circuit 106 can use this range information to determine desired
energy levels for the laser pulses 122 which will target range
points that are believed to correspond to those objects. In this
fashion, the control circuit 106 can controllably adjust the laser
pulse energy as a function of the estimated range of the object
being targeted so the object is illuminated with a sufficient
amount of light energy given its estimated range to facilitate
adequate detection by the lidar receiver 1400. Further still, the
beam scanner controller 802 can provide shot timing information
1410 to the receiver 1400 and the system controller 800 can provide
shot data 1412 (such as data identifying the targeting range
points) to the receiver 1400. The combination of this information
informs the receiver how to control which pixels of the receiver
1400 should be activated for detecting pulse returns 1402
(including when those pixels should be activated). As discussed in
the above-referenced and incorporated '513 and '015 patents, the
receiver can select pixels for activation to detect pulse returns
1402 based on the locations of the targeted range points in the
field of view. Accordingly, precise knowledge of which range points
were targeted and when those range points were targeted helps
improve the operations of receiver 1400. Although not shown in FIG.
14, it should also be understood that a practitioner may choose to
also include a camera that images the field of view, and this
camera can be optically co-axial (co-bore sighted) with the lidar
transmitter 100. Camera images can also be used to facilitate
intelligent range point selection among other tasks.
[0213] FIG. 15 shows another example of a process flow for the
control circuit 106 with respect to using the models to dynamically
determine the shot list for the transmitter 100. At step 1500, the
laser energy model 108 and mirror motion model 308 are established.
This can operate like step 1300 discussed above. At step 1502, the
model parameters are updated using pulse return statistics (which
may be derived from point cloud 1404 or other information provided
by the receiver 1400) and mirror scan position feedback (e.g., from
feedback system 850). At step 1504, the models are coupled so that
shot angles are assigned to time slots according to the mirror
motion model 308 for which shot energies can be predicted according
to the laser energy model 108. These coupled models can then be
embedded in the shot scheduling logic used by control circuit 106.
At step 1506, a list of range points to be targeted with laser
pulses 122 is received. At step 1508, a selection is made for the
search depth that governs how far ahead the system will schedule
shots.
[0214] Based on the listed range points and the defined search
depth, the order candidates for laser pulse shots are created (step
1510). The mirror motion model 308 can assign time slots to these
order candidates as discussed above. At step 1512, each candidate
is tested using the laser energy model 108. This testing may also
include testing based on the eye safety model 1002 and a camera
safety model. This testing can evaluate the order candidates for
compliance with criteria such as peak energy constraints, eye
safety constraints, camera safety constraints, minimum energy
thresholds, and completion times. If a valid order candidate is
found, the system can fire laser pulses in accordance with the
timing/sequencing defined by the fastest of the valid order
candidates. Otherwise, the process flow can return to step 1510 to
continue the search for a valid order candidate.
[0215] Controllable Detection Intervals for Return Processing:
[0216] In accordance with another example embodiment, the shot list
can be used to exercise control over how the lidar receiver 1400
detects returns from laser pulse shots 122. FIG. 18A shows an
example lidar receiver 1400 for use in a lidar system. The lidar
receiver 1400 comprises photodetector circuitry 1800 which includes
a photodetector array 1802. The photodetector array 1802 comprises
a plurality of detector pixels 1804 that sense incident light and
produce a signal representative of the sensed incident light. The
detector pixels 1804 can be organized in the photodetector array
1802 in any of a number of patterns. In some example embodiments,
the photodetector array 1802 can be a two-dimensional (2D) array of
detector pixels 1804. However, it should be understood that other
example embodiments may employ a one-dimensional (1D) array of
detector pixels 1804 if desired by a practitioner.
[0217] The photodetector circuitry 1800 generates a return signal
1806 in response to a pulse return 1402 that is incident on the
photodetector array 1802. The choice of which detector pixels 1804
to use for collecting a return signal 1806 corresponding to a given
return 1402 can be made based on where the laser pulse shot 122
corresponding to the return 1402 that was targeted. Thus, if a
laser pulse shot 122 is targeting a range point located as a
particular azimuth angle, elevation angle pair; then the lidar
receiver can map that azimuth, elevation angle pair to a set of
pixels 1804 within the array 1802 that will be used to detect the
return 1402 from that laser pulse shot 122. The mapped pixel set
can include one or more of the detector pixels 1804. This pixel set
can then be activated and read out from to support detection of the
subject return 1402 (while the pixels outside the pixel set are
deactivated so as to minimize potential obscuration of the return
1402 within the return signal 1806 by ambient or interfering light
that is not part of the return 1402 but would be part of the return
signal 1806 if unnecessary pixels 1804 were activated when return
1402 was incident on array 1802). In this fashion, the lidar
receiver 1400 will select different pixel sets of the array 1802
for readout in a sequenced pattern that follows the sequenced
spatial pattern of the laser pulse shots 122. Return signals 1806
can be read out from the selected pixel sets, and these return
signals 1806 can be processed to detect returns 1402
therewithin.
[0218] FIG. 18A shows an example where one of the pixels 1804 is
turned on to start collection of a sensed signal that represents
incident light on that pixel (to support detection of a return 1402
within the collected signal), while the other pixels 1804 are
turned off (or at least not selected for readout). While the
example of FIG. 18A shows a single pixel 1804 being included in the
pixel set selected for readout, it should be understood that a
practitioner may prefer that multiple pixels 1804 be included in
one or more of the selected pixel sets. For example, it may be
desirable to include in the selected pixel set one or more pixels
1804 that are adjacent to the pixel 1804 where the return 1402 is
expected to strike.
[0219] Examples of circuitry and control logic that can used for
this selective pixel set readout are described in U.S. Pat. Nos.
10,754,015 and 10,641,873, the entire disclosures of each of which
are incorporated herein by reference. These incorporated patents
also describe example embodiments for the photodetector circuitry
1800, including the use of a multiplexer to selectively read out
signals from desired pixel sets as well as an amplifier stage
positioned between the photodetector array 1802 and
multiplexer.
[0220] Signal processing circuit 1820 operates on the return signal
1806 to compute return information 1822 for the targeted range
points, where the return information 1822 is added to the lidar
point cloud 1404. The return information 1822 may include, for
example, data that represents a range to the targeted range point,
an intensity corresponding to the targeted range point, an angle to
the targeted range point, etc. As described in the above-referenced
and incorporated '015 and '873 patents, the signal processing
circuit 1820 can include an analog-to-digital converter (ADC) that
converts the return signal 1806 into a plurality of digital
samples. The signal processing circuit 1820 can process these
digital samples to detect the returns 1402 and compute the return
information 1822 corresponding to the returns 1402. In an example
embodiment, the signal processing circuit 1820 can perform time of
flight (TOF) measurement to compute range information for the
returns 1402. However, if desired by a practitioner, the signal
processing circuit 1820 could employ time-to-digital conversion
(TDC) to compute the range information. Additional details about
how the signal processing circuit 1820 can operate for an example
embodiment are discussed below.
[0221] The lidar receiver 1400 can also include circuitry that can
serve as part of the control circuit 106 of the lidar system. This
control circuitry is shown as a receiver controller 1810 in FIG.
18A. The receiver controller 1810 can process scheduled shot
information 1812 to generate the control data 1814 that defines
which pixel set to select (and when to use each pixel set) for
detecting returns 1402. The scheduled shot information 1812 can
include shot data information that identifies timing and target
coordinates for the laser pulse shots 122 to be fired by the lidar
transmitter 100. In an example embodiment, the scheduled shot
information 1812 can also include detection range values to use for
each scheduled shot to support the detection of returns 1412 from
those scheduled shots. These detection range values may include
minimum and maximum range values (Rmin and Rmax respectively) for
each shot. In this fashion, Rmin(i) would be the minimum detection
range associated with Shot(i) and Rmax(i) would be the maximum
detection range associated with Shot(i). These minimum and maximum
range values can be translated by the receiver controller 1810 into
times for starting and stopping collections from the selected
pixels 1804 of the array 1802 as discussed below.
[0222] The receiver controller 1810 and/or signal processing
circuit 1820 may include one or more processors. These one or more
processors may take any of a number of forms. For example, the
processor(s) may comprise one or more microprocessors. The
processor(s) may also comprise one or more multi-core processors.
As another example, the one or more processors can take the form of
a field programmable gate array (FPGA) or application-specific
integrated circuit (ASIC) which provide parallelized hardware logic
for implementing their respective operations. The FPGA and/or ASIC
(or other compute resource(s)) can be included as part of a system
on a chip (SoC). However, it should be understood that other
architectures for such processor(s) could be used, including
software-based decision-making and/or hybrid architectures which
employ both software-based and hardware-based decision-making. The
processing logic implemented by the receiver controller 1810 and/or
signal processing circuit 1820 can be defined by machine-readable
code that is resident on a non-transitory machine-readable storage
medium such as memory within or available to the receiver
controller 1810 and/or signal processing circuit 1820. The code can
take the form of software or firmware that define the processing
operations discussed herein. This code can be downloaded onto the
processor using any of a number of techniques, such as a direct
download via a wired connection as well as over-the-air downloads
via wireless networks, which may include secured wireless networks.
As such, it should be understood that the lidar receiver 1400 can
also include a network interface that is configured to receive such
over-the-air downloads and update the processor(s) with new
software and/or firmware. This can be particularly advantageous for
adjusting the lidar receiver 1400 to changing regulatory
environments. When using code provisioned for over-the-air updates,
the lidar receiver 1400 can operate with unidirectional messaging
to retain function safety.
[0223] FIGS. 19A and 19B illustrate time constraints involved in
detecting shot returns 1402 and how these time constraints relate
to the Rmin and Rmax values.
[0224] In FIGS. 19A and 19B, the horizontal axes correspond to
time. In FIG. 19A, the time at which a first laser pulse shot (Shot
1) is fired is denoted by T(1) (where the parenthetical (1) in T(1)
references the shot number). TT1(1) denotes when the receiver 1400
starts the collection from the pixel set used for detecting a
return from Shot 1. The time at which collection stops from this
pixel set (to end the readout of signal from the pixel set for
detecting a return from Shot 1) is denoted as TT2(2). It should be
understood that the parentheticals in these terms T, TT1, and TT2
reference the shot number for the detection. Thus, the time
duration from TT1(1) to TT2(1) represents the detection interval
for detecting a return from Shot 1 because it is during this time
interval that the receiver 1400 is able to collect signal from the
pixel set used for detecting a return from Shot 1.
[0225] As shown by FIG. 19A, the time duration from T(1) to TT1(1)
corresponds to the minimum range that must exist to the target
(relative to the receiver 1400) in order to detect a return from
Shot 1. This minimum range is denoted by Rmin(1) in FIG. 19A (where
the parenthetical references the shot number to which the minimum
range value is applicable). If the target were located less than
Rmin(1) from the receiver 1400, then the receiver 1400 would not be
able to detect the return from Shot 1 because the lidar receiver
would not yet have started collection from the pixel set used for
detecting that return.
[0226] FIG. 19A also shows that the time duration from T(1) to
TT2(1) corresponds to the maximum range to the target for detecting
a return from Shot 1. This maximum range is denoted by Rmax(1) in
FIG. 19A (where the parenthetical references the shot number to
which the maximum range value is applicable). If the target were
located greater than Rmax(1) from the receiver 1400, then the
receiver 1400 would not be able to detect the return from Shot 1
because the lidar receiver would have already stopped collection
from the pixel set used for detecting that return by the time that
the return strikes that pixel set.
[0227] Thus, so long as the target for Shot 1 is located at a range
between Rmin(1) and Rmax(1), the receiver 1400 is expected to be
capable of detecting the return if collection from the pixel set
starts at time TT1(1) and stops at time TT2(1). The range interval
encompassed by the detection interval of TT1(1) to TT2(1) can be
referred to as the range swath S(1) (where the parenthetical
references the shot number to which the range swath is applicable).
This range swath can also be referenced as a range buffer as it
represents a buffer of ranges for the target that make the target
detectable by the receiver 1400.
[0228] FIG. 19B further extends these time-range relationships by
adding a second shot (Shot 2). The time at which Shot 2 is fired by
the lidar transmitter 100 is denoted as T(2) in FIG. 19B. The start
collection time for the pixel set used to detect the return from
Shot 2 is denoted as TT1(2) in FIG. 19B, and the stop collection
time for the pixel set used with respect to detecting the return
from Shot 2 is denoted as TT2(2) in FIG. 19B. FIG. 19B further
shows that (1) the time duration from T(2) to TT1(2) corresponds to
the minimum range to the target for detecting a return from Shot 2
and (2) the time duration from T(2) to TT2(2) corresponds to the
maximum range to the target for detecting a return from Shot 2.
These minimum and maximum range values are denoted as Rmin(2) and
Rmax(2) respectively by FIG. 19B. The range swath S(2) defines the
range interval (or range buffer) between Rmin(2) and Rmax(2) for
the detection interval of TT1(2) to TT2(2).
[0229] In an example embodiment, the photodetector circuitry 1800
is capable of sensing returns from one pixel set at a time. For
such an example embodiment, the detection interval for detecting a
return for a given shot cannot overlap with the detection interval
for detecting a return from another shot. This means that TT1(2)
should be greater than or equal to TT2(1), which then serves as a
constraint on the choice of start and stop collection times for the
pixel clusters. However, it should be understood that this
constraint could be eliminated with other example embodiments
through the use of multiple readout channels for the lidar receiver
1400 as discussed below in connection with FIG. 26.
[0230] As noted above, each detection interval (D(i), which
corresponds to (TT1(i) to TT2(i)) will be associated with a
particular laser pulse shot (Shot(i)). The system can control these
shot-specific detection intervals so that they can vary across
different shots. As such, the detection interval of D(j) for
Shot(j) can have a different duration than the detection interval
of D(k) for Shot(k).
[0231] Moreover, as noted above, each detection interval D(i) has a
corresponding shot interval SI(i), where the shot interval SI(i)
corresponding to D(i) can be represented by the interval from shot
time T(i) to the shot time T(i+1). Thus, consider a shot sequence
of Shots 1-4 at times T(1), T(2), T(3), and T(4) respectively. For
this shot sequence, detection interval D(1) for detecting the
return from Shot(1) would have a corresponding shot interval SI(1)
represented by the time interval from T(1) to T(2). Similarly,
detection interval D(2) for detecting the return from Shot(2) would
have a corresponding shot interval SI(2) represented by the time
interval from T(2) to T(3); and the detection interval D(3) for
detecting the return from Shot(3) would have a corresponding shot
interval SI(3) represented by the time interval from T(3) to T(4).
Counterintuitively, the inventors have found that it is often not
desirable for a detection interval to be of the same duration as
its corresponding shot interval due to factors such as the amount
of processing time that is needed to detect returns within return
signals (as discussed in greater detail below). In many cases, it
will be desirable for the control process to define a detection
interval so that it exhibits a duration shorter than the duration
of its corresponding shot interval (D(i)<SI(i)). In this
fashion, processing resources in the signal processing circuit 1820
can be better utilized, as discussed below. Furthermore, in some
other cases, it may be desirable for the variability of the
detection intervals relative to their corresponding shot intervals
to operate where a detection interval exhibits a duration longer
than the duration of its corresponding shot interval
(D(i)>SI(i)). For example, if the next shot at T(i+1) has an
associated Rmin value greater than zero, and where the shot at T(i)
is targeting a range point expected to be at a long range while the
shot at T(i+1) is targeting a range point expected to be at medium
or long range, then it may be desirable for D(i) to be greater than
SI(i).
[0232] It can be appreciated that a laser pulse shot, Shot(i),
fired at time T(i) will be traveling at the speed of light. On this
basis, and using the minimum and maximum range values of Rmin(i)
and Rmax(i) for detecting the return from Shot(i), the minimum
roundtrip distance for Shot(i) and its return would be 2Rmin(i) and
the minimum roundtrip time for Shot(i) and its return would be
TT1(i)-T(i). The value for TT1(i) could be derived from Rmin(i)
according to these relationships as follows (where the term c
represents the speed of light):
( TT .times. 1 .times. ( i ) - T .function. ( i ) ) .times. c = 2
.times. R .times. min .function. ( i ) ##EQU00008## which .times.
can .times. be .times. re - expressed .times. as : ##EQU00008.2##
TT .times. 1 .times. ( i ) = 2 .times. R .times. min .function. ( i
) c + T .function. ( i ) ##EQU00008.3##
[0233] Thus, knowledge of when Shot(i) is fired and knowledge of
the value for Rmin(i) allows the receiver 1400 to define when
collection should start from the pixel set to be used for detecting
the return from Shot (i).
[0234] Similarly, the value for TT2(i) can be derived from Rmax(i)
according to these relationships as follows (where the term c
represents the speed of light):
( TT .times. 2 .times. ( i ) - T .function. ( i ) ) .times. c = 2
.times. R .times. max .function. ( i ) ##EQU00009## which .times.
can .times. be .times. re - expressed .times. as : ##EQU00009.2##
TT .times. 2 .times. ( i ) = 2 .times. R .times. max .function. ( i
) c + T .function. ( i ) ##EQU00009.3##
[0235] Thus, knowledge of when Shot(i) is fired and knowledge of
the value for Rmax(i) allows the receiver 1400 to define when
collection can stop from the pixel set to be used for detecting the
return from Shot(i).
[0236] A control process for the lidar system can then operate to
determine suitable Rmin(i) and Rmax(i) values for detecting the
returns from each Shot(i). These Rmin, Rmax pairs can then be
translated into appropriate start and stop collection times (the
on/off times of TT1 and TT2) for each shot. In an example
embodiment, if the lidar point cloud 1404 has range data and
location data about a plurality of objects of interest in a field
of view for the receiver 1400, this range data and location data
can be used to define current range estimates for the objects of
interest, and suitable Rmin, Rmax values for detecting returns from
laser pulse shots that target range points corresponding to where
these objects of interest are located can be derived from these
range estimates. In another example embodiment, the control process
for the lidar system can access map data based on the geographic
location of the receiver 1400. From this map data, the control
process can derive information about the environment of the
receiver 1400, and suitable Rmin, Rmax values can be derived from
this environmental information. Additional example embodiments for
determine the values for the Rmin, Rmax pairs are discussed
below.
[0237] FIG. 18B depicts an example process flow for execution by
the receiver controller 1810 for the lidar receiver 1400 of FIG.
18A to control the selection of detector pixels 1804 in the
photodetector array 1802 for readout. In this example, the lidar
system includes a buffer 1830 in which the scheduled shot
information 1812 is buffered by the control circuit 106. This
scheduled shot information 1812 can include, for each scheduled
laser pulse shot, data that identifies the time the shot is to be
fired, data that identifies range point to be targeted with the
subject shot (e.g., the azimuth and elevation angles at which the
subject shot will be fired), and data that defines the detection
interval for detecting the return from the subject shot (e.g., data
that identifies the Rmin and Rmax values to use for detecting the
return from the subject shot). The entries 1832 in buffer 1830 thus
correspond to the shot time, azimuth angle, elevation angle, Rmin,
and Rmax values for each shot. Moreover, the order in which these
entries are stored in the buffer 1832 can define the shot schedule.
For example, the buffer can be a first in/first out (FIFO) buffer
where entries 1832 are added into and read out of the buffer in
accordance with the order in which the shots are to be fired.
[0238] Steps 1850, 1852, and 1854 of FIG. 18B define a translation
process flow for the receiver controller 1810. At step 1850, the
receiver controller 1810 reads the entry 1832 corresponding to the
first shot to determine the shot time, shot angles, and range swath
information for that shot. At step 1852, the receiver controller
1852 determines which pixel set of the photodetector array 1800 to
select for detecting the return from the shot defined by the shot
angles of the entry 1832 read at step 1850. This can be
accomplished by mapping the azimuth and elevation angles for the
shot to a particular pixel set of the array 1802. Thus, step 1852
can operate to generate data that identifies one or more pixels
1804 of the array 1802 to include in the pixel set for detecting
the return from the subject shot. The above-referenced and
incorporated '015 and '873 patents describe how a practitioner can
implement this mapping of shot locations to pixels. At step 1854,
the receiver controller 1810 translates the shot time and Rmin,
Rmax values into the TT1, TT2 values. This translation can use the
expressions discussed above for computing TT1(i) and TT2(i) as a
function of T(i), Rmin(i), and Rmax(i) for a given Shot(i). The
values for the determined pixel set and the start/stop collection
times for the pixel set can then be added to buffer 1840 as a new
entry 1842. The process flow can then return to step 1850 to
iterate through steps 1850-1854 and generate the control data for
the next shot (as defined by the next entry 1832 in buffer 1830),
and so on for so long as there are new entries 1832 in buffer
1830.
[0239] Steps 1856, 1858, and 1860 of FIG. 18B define a readout
control process flow for the receiver controller 1810. Each entry
1842 in buffer 1840 corresponds to exercising control over
detecting the return from a different shot and identifies the pixel
set to use for the detection as well as the start and stop
collection times for that detection (TT1, TT2 values). At step
1856, the receiver controller reads the entry 1842 corresponding to
the first shot to identify the pixel set, TT1, and TT2 values for
detecting the return from that shot. Then, at the time
corresponding to TT1, the receiver controller 1810 starts
collecting the sensed signal from the identified pixel set (step
1860). In this fashion, the receiver controller 1810 can provide
control data 1814 to the photodetector circuitry 1800 at the time
corresponding to TT1 that instructs the photodetector circuitry
1800 to start the signal readout from the pixel(s) within the
identified pixel set. Next, at the time corresponding to TT2, the
receiver controller 1810 stops the collection from the identified
pixel set (step 1862). To accomplish this, the receiver controller
1810 can provide control data 1814 to the photodetector circuitry
1800 at the time corresponding to TT2 that instructs the
photodetector circuitry 1800 to stop the readout from the pixel(s)
within the identified pixel set. From there, the process flow
returns to step 1856 to continue the readout control process flow
for the next entry 1842 in buffer 1840. This iteration through
steps 1856-1860 continues for so long as there are new entries 1842
in buffer 1840 to process.
[0240] As discussed above in connection with FIGS. 8, 11, and 14, a
practitioner may want the lidar system to exercise highly precise
control over when the laser pulse shots are fired; and this can be
accomplished by having the beam scanner controller 802 provide
firing commands 120 to the laser source 102 precisely when the fine
mirror motion model 808a indicates the lidar transmitter 100 will
be pointing at a particular shot angle (e.g., an azimuth angle,
elevation angle pair). The beam scanner controller 802 can then
report these precise shot times to the receiver 1400 as shot timing
information 1412. Accordingly, as shown by FIG. 18C, the lidar
receiver 1400 can receive the scheduled shot information 1812 in
the form of (1) the shot timing information 1410 from the beam
scanner controller 802 (which in this example will identify the
precise shot times for each shot) and (2) the shot data 1312 from
the system controller 800 (which in this example will identify the
shot angles for each shot as well as the Rmin and Rmax values for
each shot).
[0241] FIG. 18D depicts an example process flow for the receiver
controller 1810 with respect to the example of FIG. 18C. The
process flow of FIG. 18D can operate in a similar fashion as the
process flow of FIG. 18D with a couple of exceptions. For example,
at step 1854 of FIG. 18D, the receiver controller 1810 can compute
the start and stop collection times as time offsets relative to the
fire time for the shot rather than as absolute values. That is,
rather than computing values for TT1(i) and TT2(i), the receiver
controller can compute values for (1) TT1(i)-T(i) (which would
identify the TT1(i) offset relative to fire time T(i)) and (2)
TT2(i)-T(i) (which would identify the TT2(i) offset relative to
fire time T(i)) as follows:
T .times. T .times. 1 .times. ( i ) .times. Offset = TT .times. 1
.times. ( i ) - T .function. ( i ) = 2 .times. R .times. min
.function. ( i ) c ##EQU00010## TT .times. 2 .times. ( i ) .times.
Offset = TT .times. 2 .times. ( i ) - T .function. ( i ) = 2
.times. R .times. max .function. ( i ) c ##EQU00010.2##
[0242] Then, after step 1856 is performed to read entry 1842 in
buffer 1840, the receiver controller 1870 can also determine the
fire time T(i) for the subject shot(i) based on the shot timing
information 1410 received from the beam scanner controller 802.
Using this shot time as the frame of reference for the TT1 and TT2
offset values, steps 1858 and 1860 can then operate to start and
stop collections from the pixel set at the appropriate times.
[0243] FIG. 20 shows an example process flow for use by the signal
processing circuit 1820 to detect returns 1402 and compute return
information 1822 for the returns 1402. At step 2000, the signal
processing circuit 1820 digitizes the sensed signal 1806 produced
by the photodetector circuitry 1800. This sensed signal 1806
represents the incident light on the activated pixels 1804 of the
array 1802 over time, and thus is expected to include signals
corresponding to the returns 1402. The signal processing circuit
1820 can perform step 2000 using an ADC to produce digital samples
2004 that are added to buffer 2002.
[0244] The signal processing circuit 1820 then needs to segment
these samples 2004 into groups corresponding to the detection
intervals for the returns from each shot. This aspect of the
process flow is identified by the detection loop 2020 of FIG. 20. A
processor 2202 within the signal processing circuit 1820 can
perform this detection loop 2020. To help accomplish this, the
processor can access the buffer 1840 to determine the start and
stop collection times for detecting the returns from each shot. As
discussed above, entries 1842 in buffer 1840 can include the
start/stop collection times as either absolute or offset values for
TT1 and TT2. At step 2006, the processor reads the next entry 1842
in buffer 1840 to determine the TT1 and TT2 values (which as noted
can be either absolute values or offset values). These TT1 and TT2
values can then be used to find which samples 2004 in the buffer
2002 correspond to the detection interval of TT1 to TT2 for
detecting the subject return from (step 2008). The processor thus
reads the digital samples 2004 corresponding to the subject
detection interval, and then processes the digital samples 2004 in
this group to detect whether the return is present (step 2010).
When a return is detected, step 2010 can also compute return
information 1822 based on these samples 2004. As noted above, the
samples 2004 can be processed to compute a range to target for the
shot. For example, according to a TOF flight technique, the
processor can compute the range for the return based on knowledge
of when the shot was fired, when the detected return was received,
and the value for the speed of light. The samples 2004 can also be
processed to compute an intensity for the shot return. For example,
the return intensity can be computed by multiplying the return
energy by the range squared, and then dividing by the transmitted
shot energy and again by the effective receiver pupil aperture.
From step 2010, the detection loop 2020 can return to steps 2006
and 2008 to read the next entry 1842 from buffer 1840 and grab the
next group of samples 2004 from buffer 2002 and then detect the
return and compute its return information at step 2010 (and so on
for additional returns).
[0245] Multi-Processor Return Detection:
[0246] The amount of time needed by processor 2022 to perform the
detection loop 2020 is an important metric that impacts the lidar
system. This amount of time can be characterized as Tproc, and it
defines the rate at which processor 2022 draws samples 2004 from
buffer 2002. This rate can be referenced as Rate 1. The rate at
which the receiver adds samples 2004 to buffer 2002 can be
referenced as Rate 2. It is highly desirable for the processor 2022
to operate in a manner where Rate 1 is greater than (or at least no
less than) Rate 2 so as to avoid throughput problems and potential
buffer overflows. To improve throughput for the lidar receiver 1400
in this regard, the signal processing circuit 1820 can include
multiple processors 2022 that distribute the detection workload so
that the multiple processors 2022 combine to make it possible for
the receiver 1400 to keep up with the shot rate of the lidar
transmitter 100 even if Rate 1 is less than the shot rate of the
lidar transmitter 100. For example, if there are N processors 2022,
then Rate 1 can be N times less than that shot rate of the lidar
transmitter 100 while still keeping pace with the shots. FIG. 21A
shows an example of a multi-processor architecture for the signal
processing circuit 1820 in this regard. As shown by FIG. 21A, the
processor 2022 comprises two or more processors 2022.sub.1, . . . ,
2022.sub.N. Each processor 2022.sub.i can access buffers 1840 and
2002 to perform the operations set forth by steps 2006-2010 of FIG.
20 for the returns corresponding to different shots. FIG. 21B shows
an example of control flow for the different processors 2022.sub.i.
At step 2100, each processor 2022.sub.i decides whether it is free
to process another return. In other words, has it finished
processing the previous return it was working on? If the subject
processor 2022.sub.i decides that it is free, it proceeds to step
2102 where it performs the detection processing loop 2020 for the
next return available from the buffers 1840 and 2002. In this
fashion, each processor 2022.sub.i can grab samples 2004 from
buffer 2002 to work on the next return on a first come first served
basis and thereby distribute the workload of processing the returns
across multiple processors to help reduce the processing latency of
the signal processing circuit 1820.
[0247] The processors 2022.sub.i can take any of a number of forms.
For example, each processor 2022.sub.i can be a different
microprocessor that shares access to the buffers 1840 and 2002. In
this fashion the different microprocessors can operate on samples
2004 corresponding to different returns if necessary. As another
example, each processor 2022.sub.i can be a different processing
core of a multi-core processor, in which case the different
processing cores can operate on samples 2004 corresponding to
different returns if necessary. As yet another example, each
processor 2022.sub.i can be a different set of parallelized
processing logic within a field programmable gate array (FPGA) or
application-specific integrated circuit (ASIC). In this fashion,
parallelized compute resources within the FPGA or ASIC can operate
on samples 2004 corresponding to different returns if
necessary.
[0248] It is expected that the use of two processors 2022 will be
sufficient to distribute the workload of processing the samples
2004 within buffer 2002. With this arrangement, the two processors
2022 can effectively alternate in terms of which returns they will
process (e.g., Processor 1 can work on the samples for
even-numbered returns while Processor 2 works on the samples for
the odd-numbered returns). However, this alternating pattern may
not necessarily hold up if, for example, the detection interval for
Return 1 is relatively long (in which case Processor 1 may need to
process a large number of samples 2004) while the detection
intervals for Returns 2 and 3 are relatively short. In this
example, it may be the case that Processor 1 is still processing
the samples from Return 1 when Processor 2 completes its processing
of the samples from Return 2 (and thus Processor 2 is free to begin
processing the samples from Return 3 while Processor 1 is still
working on the samples from Return 1).
[0249] Moreover, the return information 1822 computed by each
processor 2022.sub.i can be effectively joined or shuffled together
into their original time sequence of shots when adding the return
information 1822 to the point cloud 1404.
[0250] Choosing Rmin, Rmax Values:
[0251] The task of choosing suitable Rmin and Rmax values for each
shot can be technically challenging and involves a number of
tradeoffs. In an ideal world, the value of Rmin would be zero and
the value of Rmax would be infinite; but this is not feasible for
real world applications because there are a number of constraints
which impact the choice of values for Rmin and Rmax. Examples of
such constraints are discussed below, and these constraints
introduce a number of tradeoffs that a practitioner can resolve to
arrive at desirable Rmin and Rmax values for a given use case.
[0252] For an example embodiment as discussed above where the lidar
receiver 1400 is only capable of receiving/detecting one return at
a time, a first constraint is the shot timing. That is, the
receiver 1400 needs to quit listening for a return from Shot 1
before it can start listening for a return from Shot 2.
Accordingly, for a given fixed shot spacing, if a practitioner
wants to have fixed Rmin and Rmax values, their differences must be
equal to the intershot timing (after scaling by 2/c). For example,
for a 1 .mu.sec detection interval, the corresponding range buffer
would be a total of 150 m. This would permit Rmin to be set at 0 m
and Rmax to be set at 150 m (or Rmin=40 m, Rmax=190 m, etc.). Thus,
if Rmax is increased, we can avoid adding time to Tproc by also
increasing the value of Rmin by a corresponding amount so that the
Rmax-Rmin does not change.
[0253] A second constraint on Rmin, Rmax values is physics. For
example, the receiver 1400 can only detect up to a certain distance
for a given shot energy. For example, if the energy in a laser
pulse shot 122 is low, there would not be a need for a large Rmax
value. Moreover, the receiver 1400 can only see objects up to a
certain distance based on the elevation angle. As an example, the
receiver 1400 can only see a short distance if it is looking at a
steep downward elevation angle because the field of view would
quickly hit the ground at steep downward elevation angles. In this
regard, for a receiver 1400 at a height of 1 m and an elevation
angle of -45 degrees, Rmax would be about 1.4 m. The light
penetration structure of the air within the environment of the
lidar system can also affect the physics of detection. For example,
if the lidar receiver 1400 is operating in clear weather, at night
with dark or artificial lighting, and/or in a relatively open area
(e.g., on a highway), the potential value for Rmax could be very
large (e.g., 1 km or more) as the lidar receiver 1400 will be
capable of detecting targets at very long range. But, if the lidar
receiver 1400 is operating in bad weather or during the day (with
bright ambient light), the potential value for Rmax may be much
shorter (e.g., around 100 m) as the lidar receiver 1400 would
likely only need to be capable of detecting targets at relatively
shorter ranges.
[0254] A third constraint arises from geometry and a given use
case. Unlike the physics constraints in the second constraint
category discussed above (which are based on features of the air
surrounding the lidar system), geometry and use case can be
determined a priori (e.g., based on maps and uses cases such as a
given traffic environment that may indicate how congested the field
of view would be with other vehicles, buildings, etc.), with no
need to measure attributes in the return data. For example, if the
goal is to track objects on a road, and the road curves, then there
is no need to set Rmax beyond the curve. Thus, if the receiver 1400
is looking straight ahead and the road curves at a radius of
curvature of 1 km, roughly 100 m for Rmax would suffice. This would
be an example where accessing map data can help in the choice of
suitable Rmax values. As another example, if the lidar receiver
1400 is operating in a relatively congested environment (e.g., on a
city street), the potential value for Rmax may be relatively short
(e.g., around 100 m) as the lidar receiver 1400 would likely only
need to be capable of detecting targets at relatively short ranges.
Also, for use cases where there is some a priori knowledge of what
the range is to an object being targeted with a laser pulse shot,
this range knowledge can influence the selection of Rmin and Rmax.
This would be an example where accessing lidar point cloud data
1404 can help in the choice of suitable Rmin and Rmax values. Thus,
if a given laser pulse shot is targeting an object having a known
estimated range of 50 m, then this knowledge can drive the
selection of Rmin, Rmax values for that shot to be values that
encompass the 50 m range within a relatively tight tolerance (e.g.,
Rmin=25 m and Rmax=75 m).
[0255] A fourth constraint arises from the processing time needed
to detect a return and compute return information (Tproc, as
discussed above). If the receiver 1400 has N processors and all are
busy processing previous returns, then the receiver 1400 must wait
until one of the processors is free before processing the next
return. This Tproc constraint can make it undesirable to simply set
the detection intervals so that they coincide with their
corresponding shot intervals (e.g., TT1(i)=T(i) and TT2(i)=T(i+1),
where TT1(1)=T(1), TT2(1)=T(2), and so on). For example, imagine a
scenario where the receiver 1400 includes two processors for load
balancing purposes and where the shot spacing has a long delay
between Shots 1 and 2 (say 100 .mu.sec), and then a quick sequence
of Shots 2, 3, and 4 (say with intershot spacing of 5 .mu.sec). If
Tproc is 2.times. realtime, then Processor A would need 200 .mu.sec
to process the return from Shot 1, and Processor B would need 10
.mu.sec to process the return from Shot 2. This means that
Processor A would still be working on Shot 1 (and Processor B would
still be working on Shot 2) when the return from Shot 3 reaches the
receiver 1400. Accordingly, the system may want to tradeoff the
detection interval for detecting the return from Shot 1 by using a
smaller value for Rmax(1) so that there is a processor available to
work on the return from Shot 3. Thus, the variable shot intervals
that can be accommodated by the lidar system disclosed herein will
often make it desirable to control at least some of the detection
intervals so that they have durations that are different than the
durations of the corresponding shot intervals, as discussed
above.
[0256] Accommodating the Tproc constraint can be accomplished in
different ways depending on the needs and desires of a
practitioner. For example, under a first approach, the Rmax value
for the processor that would be closest to finishing can be
redefined to a lesser value so that processor is free exactly when
the new shot is fired. In this case, the Rmin for the new shot can
be set to zero. Under a second approach, we can keep Rmax the same
for the last shot, and then set Rmin for the new shot to be exactly
the time when the processor first frees up. Additional aspects of
this constraint will be discussed in greater detail below.
[0257] A fifth constraint arises from the amount of time that the
pixels 1804 of the array 1802 need to warm up when activated. This
can be referred to as a settle time (Tsettle) for the pixels 1804
of the array 1802. When a given pixel 1804 is activated, it will
not reliably measure incident light until the settle time passes,
which is typically around 1 .mu.sec. This settle time effectively
defines the average overall firing rate for a lidar system that
uses example embodiments of the lidar receiver 1400 described
herein. For example, if the firing rate of the lidar transmitter
100 is 5 million shots per second, the settle time would prevent
the receiver 1400 from detecting returns from all of these shots
because that would exceed the ability of the pixels 1804 to warmup
sufficiently quickly for detecting returns from all of those shots.
However, if the firing rate is only 100,000 shots per second, then
the settle time would not be a limiting factor.
[0258] FIG. 27 shows an example embodiment that illustrates how the
circuitry of the receiver 1400 can accommodate the settle time for
the pixels 1804. For ease of illustration, the array 1802 of FIG.
27 includes only four pixels 1804. However, it should be understood
that a practitioner would likely choose a much larger number of
pixels 1804 for the array 1802. FIG. 27 shows an example where the
photodetector circuit 1800 includes an amplifier network that
connects the pixels 1804 of array 1802 to a corresponding amplifier
that amplifies the signals from its corresponding pixel 1804. Each
amplified signal is then fed as an input line to multiplexer 2710.
Thus, as shown by FIG. 27, Pixel 1 feeds into amplifier A1, which
in turn feeds into multiplexer input line M1. Similarly, Pixel 2
feeds into amplifier A2, which in turn feeds into multiplexer input
line M2 (and so on for Pixels 3 and 4). As discussed in the
above-referenced and incorporated U.S. Pat. Nos. 9,933,513 and
10,754,015, the amplifiers in the amplifier network can be
maintained in a quiescent state when their corresponding pixels
1804 are not being used to detect returns. In doing so, the amount
of power consumed by the receiver 1400 during operation can be
greatly reduced. When it is time for a given pixel 1804 to be used
to detect a return, that pixel's corresponding amplifier is
awakened by powering it up. However, as discussed above, the pixel
will need to wait for the settle time before its corresponding
amplifier can pass an accurate signal to the multiplexer 2710. This
means that if the lidar receiver 1400 is to start collection from a
pixel at time TT1, then it must activate that pixel at least
Tsettle before TT1.
[0259] Multiplexer 2710 operates to read out a sensed signal from a
desired pixel 1804 in accordance with a readout control signal
2708, where the readout control signal 2708 controls which of the
multiplexer input lines are passed as output. Thus, by controlling
the readout control signal 2708, the receiver 1400 can control
which of the pixels 1804 are selected for passing its sensed signal
as the return signal 1806.
[0260] The receiver controller 1810 includes logic 2700 that
operates on the scheduled shot information 1812 to convert the
scheduled shot information into data for use in controlling the
photodetector circuit 1800. The scheduled shot information 1812 can
include, for each shot, identifications of (1) a shot time (T(i)),
(2) shot angles (e.g., an elevation angle, azimuth angle pair), (3)
a minimum detection range value (Rmin(i)), and (4) a maximum
detection range value (Rmax(i)). Logic 2700 converts this scheduled
shot information into the following values used for controlling the
photodetector circuit 1800: [0261] An identification of the pixel
set that will be used to detect the return from the subject Shot
(i). This identified pixel set is shown as P(i) by FIG. 27. [0262]
An identification of an activation time that will be used to define
the time at which the amplifier(s) corresponding to the identified
pixel set P(i) will be switched to a powered up state from a
quiescent state. This identified activation time is shown as Ta(i)
by FIG. 27. [0263] An identification of the start collection time
(TT1(i)) for the identified pixel set P(i) [0264] An identification
of the stop collection time (TT2(i)) for the identified pixel set
P(i) [0265] An identification of a deactivation time that will be
used to define the time at which the amplifier(s) corresponding to
the identified pixel set P(i) will be switched from the powered-up
state to the quiescent state.
[0266] The logic 2700 can also pass the shot times T(i) as shown by
FIG. 27.
[0267] The values for P(i) can be determined from the shot angles
in the scheduled shot information 1812 based on a mapping of shot
angles to pixel sets, as discussed in the above-referenced and
incorporated patents.
[0268] The values for Ta(i) can be determined so that the settle
time for the identified pixel set P(i) will have passed by the time
TT1(i) arrives so that P(i) will be ready to have collection
started from it at time TT1(i). A practitioner has some flexibility
in choosing how the logic 2700 will compute an appropriate value
for Ta(i). For example, the logic 2700 can activate the next pixel
set when the immediately previous shot is fired. That is, logic
2700 can set the value for Ta(i)=T(i-1), which is expected to give
P(i) enough time to power up so that collection from it can begin
at time TT1(i). However, in another example embodiment, the logic
2700 can set the value for Ta(i)=TT1(i)-Tsettle (or some time value
between these two options).
[0269] The values for TT1(i) and TT2(ii) can be computed from the
Rmin(i) and Rmax(i) values as discussed above.
[0270] The values for Td(i) can be determined so that Td(i) either
equals TT2(i) or falls after TT2(i), preferably sufficiently close
in time to TT2(i) so as to not unduly waste power. In choosing a
suitable value for Td(i), the logic 2700 can examine the upcoming
shots that are close in time to see if any of the pixels in P(i)
will be needed for such upcoming shots. In such a circumstance, the
logic 2700 may choose to leave the corresponding amplifier powered
up. But, in an example embodiment where a practitioner wants to
power down the amplifier(s) for a pixel set as soon as collection
from that pixel set stops, then TT2(i) can be used as the
deactivation time in place of a separate Td(i) value.
[0271] In the example of FIG. 27, the receiver controller 1810
includes pixel activation control logic 2702 and pixel readout
control logic 2704. Pixel activation control logic 2702 operates to
provide an activation control signal 2706 to the amplifier network
that (1) activates the amplifier(s) corresponding to each pixel set
P(i) at each time Ta(i) and (2) deactivates the amplifier(s)
corresponding to each pixel set P(i) at each time Td(i) (or TT2(i)
as the case may be). Pixel readout control logic 2704 operates to
provide a readout control signal 2708 to the multiplexer 1710 that
operates to (1) select each pixel set P(i) for readout at time
TT1(i) (to begin collection from that pixel set) and (2) de-select
each pixel set P(i) at time TT2(i) (to stop collection from that
pixel set).
[0272] Accordingly, FIG. 27 shows an example of how the receiver
controller 1810 can control which pixel sets of array 1802 will
pass their sensed signals as output in the return signal 1806 to be
processed by the signal processing circuit 1820. To facilitate this
timing control, the receiver controller 1810 can include a pipeline
of time slots that are populated with flags for the different
control signals as may be applicable, so that the logic 2702 and
2704 can adjust their respective control signals 2706 and 2708 as
may be appropriate as the flags come up as time marches on. By
activating pixel sets sufficiently prior to when collection from
them is to begin for return detection (in view of Tsettle), the
receiver 1400 is better able to support close range return
detections. For example, if the receiver 1400 were to wait until
TT1 to activate a given pixel set, this would mean that the pixel
set would not be ready to begin detection until the time
TT1+Tsettle. Given that a typical value of Tsettle is around 1
.mu.sec, this would translate to a minimum detection range of 150
m. By contrast, using the pixel activation technique of FIG. 27,
the receiver 1400 can support minimum detection ranges of 0 m.
[0273] With an example embodiment, the system begins with the shot
list and then chooses a suitable set of Rmin and Rmax values for
each shot. Of the five constraints discussed above, all but the
second and third constraints discussed above can be resolved based
simply on the shot list, knowledge of Tproc, and knowledge of the
number (N) of processors 2022 used for load balancing. For example,
the third constraint would need access to additional information
such as a map to be implemented; while the second constraint would
need either probing of the atmosphere or access to weather
information to ascertain how air quality might impact the physics
of light propagation.
[0274] In an example embodiment discussed below for computing
desired Rmin, Rmax values, the approach balances the first and
fourth constraints using a mathematical framework, but it should be
understood that this approach is also viable for balancing the
other constraints as well.
[0275] FIG. 22 shows an example process flow for assigning Rmin and
Rmax values to each shot that is scheduled by the control circuit
106. In this example, the control circuit 106 (e.g., system
controller 800) can perform the FIG. 22 process flow. However, it
should be understood that this need not necessarily be the case.
For example, a practitioner may choose to implement the FIG. 22
process flow within the receiver controller 1810 if desired.
[0276] The FIG. 22 process flow can operate on the shot list 2200
that is generated by the control circuit 106. This shot list 2200
defines a schedule of range points to be targeted with the laser
pulse shots 122, where each range point can be defined by a
particular angle pair {azimuth angle, elevation angle}. These shots
can also be associated with a scheduled fire time for each
shot.
[0277] As discussed above, a number of tradeoffs exist when
selecting Rmin and Rmax values to use for detecting each shot. This
is particularly the case when determining the detection interval in
situations where there is little a priori knowledge about the
target environment. Step 2202 of FIG. 22 can assign Rmin and Rmax
values to each shot based on an analysis that balances a number of
constraints that correspond to these tradeoffs. This analysis can
solve a cost function that optimizes the detection intervals based
on a number of constraints, including the first constraint
discussed above for an example embodiment where the receiver 1400
cannot listen to two returns from two different shots at the same
time. Thus it is desirable to optimize the maximum and minimum
ranges across a given shot list according to a preset cost
function, where inputs to the cost function can involve information
from the point cloud data 1404 of previous frames. In general, the
cost function can include a multiple range function (e.g., a
one-to-many or a many-to-many function) resulting in multiple
criteria optimization.
[0278] The shot list 2200 that step 2202 operates on can be defined
in any of a number of ways. For example, the shot list 2200 can be
a fixed list of shots that is solved as a batch to compute the
Rmin, Rmax values. In another example, the shot list 2200 can be
defined as a shot pattern selected from a library of shot patterns.
In this regard, the lidar system may maintain a library of
different shot patterns, and the control circuit 106 can select an
appropriate shot pattern based on defined criteria such as the
environment or operational setting of the lidar system. For
example, the library may include a desired default shot pattern for
when a lidar-equipped vehicle is traveling on a highway at high
speed, a desired default shot pattern for when a lidar-equipped
vehicle is traveling on a highway at low speed, a desired default
shot pattern for when a lidar-equipped vehicle is traveling in an
urban environment with significant traffic, etc. Other shot
patterns may include foviation patterns where shots are clustered
at a higher densities near an area such as a road horizon and at
lower densities elsewhere. Examples of using such shot pattern
libraries are described in the above-referenced and incorporated
U.S. Pat. App. Pub. 2020/0025887. Step 2202 can then operate to
solve for suitable Rmin, Rmax values for each of the shots in the
selected shot pattern.
[0279] With respect to step 2202, the plurality of criteria used
for optimization might include, for example, minimizing the range
offset from zero meters in front of the lidar receiver 1400, or
minimizing the range offset from no less than "x" meters in front
of the lidar receiver 1400 (where "x" is a selected preset value).
The cost function might also include minimizing the maximum number
of shots in the shot list that have a range beyond a certain preset
range "xx". In general, "x" and "xx" are adapted from point cloud
information in a data adaptive fashion, based on detection of
objects which the perception stack determines are worthy of further
investigation. While the perception stack may in some cases operate
at much slower time scales, the presets can be updated on a
shot-by-shot basis.
[0280] The value of optimization of the range buffers (specifically
controlling when to start and stop collection of each return) to
include multiple range buffers per scan row is that this allows
faster frame rates by minimizing dead time (namely, the time when
data is not being collected for return detection). The parameters
to be optimized, within constraints, include processing latency,
start time, swath (stop time minus start time), and row angle
offsets. Presets can include state space for the processor 2022,
state space for the laser source 102 (dynamic model), and state
space for the mirror 110.
[0281] Step 2202 solves equations for choosing range buffers (where
examples of these equations are detailed below), and then generates
the range buffer (Rmin and Rmax values) for each shot return. These
operations are pre-shot-firing.
[0282] The outer bounds for Rmin and Rmax for each shot return can
correspond to the pixel switching times TT1 and TT2, where TT1(k)
can be set equal to TT2(k-1) and where TT2(k) can be set equal to
TT1(k+1). It will often be the case where it is desirable for the
lidar receiver 1400 to turn off the old pixel set at exactly the
time the new pixel set is turned on.
[0283] A set of constraints used for a state space model can be
described as follows, for a use case where two processors 2022 are
employed to equally distribute the processing workload by handling
alternating returns.
[0284] We assume that the signal processing circuit 1820 begins
processing data the moment the initial data sample is available
(namely, at time TT1(k)). Processor A cannot ingest more data until
the processing for Return(k) is cleared, which we can define as
Tproc seconds after the previous return detection was terminated.
The same goes for Processor B. For ease of conception, we will
define Tproc as being one half of the realtime rate of return
detection (or faster). We will take TT1(k)=T(k) (where an Rmin of
zero is the starting point) to simplify the discussion, although it
should be understood that this need not be the case. With the TT1
values set equal to the fire times of their corresponding shots,
this means that the shot T(k+1) cannot be fired until the system
stops collecting samples from the last shot. In other words,
T(k+1)>TT2(k).
[0285] Collection for the shot fired at T(k+2) cannot be started
unless the previous shot processed by the same processor (e.g., the
same even or odd parity if we assume the two processors 2022
alternate return collections). This leads to the second of our two
inequalities:
T .function. ( k + 2 ) - T .function. ( k ) > 2 .times. ( T
.times. T .times. 2 .times. ( k ) - T .function. ( k ) ) .fwdarw. T
.function. ( k + 2 ) + T .function. ( k ) 2 > T .times. T
.times. 2 .times. ( k ) , .fwdarw. TT .times. 2 .times. ( k )
.ltoreq. min .function. ( T .function. ( k + 1 ) , T .function. ( k
+ 2 ) + T .function. ( k ) 2 ) ##EQU00011##
[0286] If we put together these equations, using
.gtoreq..apprxeq.>, adding relaxation constraints, and using S
as a shift operator (where ST(k)=T(k+1), we get:
TT .times. 2 .times. ( k ) + b k = S .times. T .function. ( k ) , b
k .gtoreq. 0 , S 2 .times. T .function. ( k ) + T .function. ( k )
2 = TT .times. 2 .times. ( k ) + b k ' , b k , b k ' .gtoreq. 0
##EQU00012##
[0287] These inequalities can be re-expressed using matrix notation
as shown by FIG. 23A. As noted, S is a shift operator where
ST(k)=T(k+1), and S.sup.2 denotes a shift from T(k) to T(k+2). TT2
and T are expressed as vectors of dimension B, and b is a positive
arbitrary 2N dimensional vector (of relaxation terms, built from
shuffled versions of b.sub.k, b.sub.k' of size twice that of TT2.
I.sub.n is an n-by-n identity matrix where the diagonal values are
all ones and the other values are all zeros; and O.sub.n is an
n-by-n matrix of all zeros. Inequality constraints can be replaced
with equality constraints using new entries as we have done here. B
can be referred to as a relaxation variable. For example, we can
replace x>0 with x+b=0 (where b>0). Note that T is known, and
TT2 is our free variable.
[0288] The equation of FIG. 23A is a state space equation since it
is expressed in terms of relations between past and current values
on an unknown value. This can be solved for real valued variables
using simultaneous linear inequality solvers such as quadratic
programming, which is available with software packages such as
MATLAB (available from Mathworks). An example quadratic programming
embodiment for the state space model of FIG. 23A is shown by FIG.
23B. In this fashion, at step 2202 of FIG. 22, the control circuit
106 can determine the detection interval data for each of a
plurality of laser pulse shots according to a state space equation
that is solved using multiple simultaneous inequality constraint
equations. This can yield the scheduled shot information 1812 where
the schedule of range points to be targeted with laser pulse shots
is augmented with associated detection interval data such as pixel
set activation/deactivation times and/or Rmin, Rmax values. While
these discussions are expressed in terms of TT2 values, it should
be understood that these solutions can also work from Rmax values
using the relationships discussed above where TT2 can be expressed
in terms of Rmax and the shot times.
[0289] FIG. 23A expresses all possible detection intervals
consistent with the shot list, and FIG. 23B represents one solution
amongst these possibilities (where this solution is one that,
loosely speaking, performs uniformly well). But, this solution is
not necessarily optimal. For example, at a shot elevation that is
low, the ground can be expected to be close, in which it case it
makes little sense to set TT2 in a fashion that enables long range
detection. A toy example can help illustrate this point.
[0290] Suppose our shot list has shot times in a sequence of 1
.mu.sec, 2 .mu.sec, 98 .mu.sec, 100 .mu.sec, 102 .mu.sec.
[0291] If we have two processors, each of which computes detections
at 2.times. realtime, we might have as a solution (where we will
assume in all cases that Rmin=0):
TABLE-US-00006 Processor A: Shot Time: 1 .mu.sec pulse 98 .mu.sec
pulse Range Interval: Rmax = 7.3 km Rmax = 150 m
TABLE-US-00007 Processor B: Shot Time: 2 .mu.sec pulse 100 .mu.sec
pulse Range Interval: Rmax = 7.3 km Rmax = 150 m
[0292] While this solution "works", it should be understood that
the two large Rmax values (>7 km) would "hog" the processors by
making them unavailable for release to work on another return for
awhile. This might not be ideal, and one might want to adjust the
solution for a smaller Rmax value. There are an almost endless set
of reasons why this is desirable because the processors are used
for a variety of functions such as: intensity computation, range
computation, velocity estimation, bounding box estimation etc.
[0293] Accordingly, the inventors also disclose an embodiment that
combines mathematical optimization functions with some measure of
value substitutions and updating in certain circumstances to arrive
a better solution (an example of which is discussed below in
connection with FIG. 24).
[0294] As another example where range substitutions and
optimization updates can improve the solution, suppose the shot
list obtained for a particular scenario fires at the following
times in units of microseconds, at elevation angle shown
respectively: [0295] Shot Time (.mu.sec)={12 40 70 86 101 121}
[0296] Elevation-Angle (degrees)={-10,-10,0,0,0,0}
[0297] Using the inequality for TT2 above and picking the largest
detection interval at each shot, the result for the first four
shots is:
TT2(1).ltoreq.40,TT2(2).ltoreq.63,TT2(3).ltoreq.85.5,TT2(4).ltoreq.101
[0298] This maps to detection intervals (in .mu.sec of time)
of:
{TT2(k)-T(k)}.sub.k=1,2,3,4={28,23,15.5,15}
[0299] The sub-optimal nature of this solution arises because it
yields large detection intervals at low elevations (where a long
detection interval is not needed) and a small detection intervals
at the horizon (where elevation angle is zero degrees, which is
where a long detection interval is more desirable).
[0300] As a solution to this issue, FIG. 24 discloses a process
flow where the control circuit 106 can swap out potentially
suboptimal detection intervals for more desirable detection
intervals. As shown by FIG. 24, step 2202 can operate as described
in connection with FIG. 22 to generate data corresponding to the
detection interval solutions computed in accordance with the models
of FIGS. 23A and 23B.
[0301] The control circuit 106 can also maintain a list 2402 of
range points with desired detection intervals. For example, the
list 2402 can identify various shot angles that will intersect with
the ground within some defined distance from the lidar system
(e.g., some nominally short distance). For a lidar-equipped
vehicle, examples of such shot angles would be for shots where the
elevation angle is low and expected to be pointing at the road
within some defined distance. For these shot angles, the detection
interval corresponding to Rmax need not be a large value because it
will be known that the shot will hit the ground within the defined
distance. Accordingly, for these low elevation angles, the list
2402 can define a desired Rmax or TT2 value that reflects the
expected distance to ground. As another example, the list 2402 can
identify shot angles that lie off the motion path of the lidar
system. For example, for a lidar-equipped vehicle, it can be
expected that azimuth angles that are large in absolute value will
be looking well off to the side of the vehicle. For such azimuth
angles, the system may not be concerned about potential targets
that are far away because they do not represent collision threats.
Accordingly, for these large absolute value azimuth angles, the
list 2402 can define a desired Rmax or TT2 value that reflects the
shorter range of potential targets that would be of interest. Range
segmentations that can be employed by list 2402 may include (1)
shot angles linked to desired Rmax or TT2 values corresponding to
0-50 m, (2) shot angles linked to desired Rmax or TT2 values
corresponding to 50-150 m, and (3) shot angles linked to desired
Rmax or TT2 values corresponding to 150-300 m.
[0302] Then, at step 2404, the control circuit 106 can compare the
assigned detection interval solutions produced by step 2202 with
the list 2402. If there are any shots with assigned detection
interval solutions that fall outside the desired detection
intervals from list 2402, the control circuit 106 can then swap out
the assigned detection interval for the desired detection interval
from list 2402 (for each such shot). Thus, step 2404 will replace
one or more of the assigned detection intervals for one or more
shots with the desired detection intervals from the list 2402.
[0303] The control circuit 106 can then proceed to step 2406 where
it re-assigns detection intervals to the shots that were not
altered by step 2404. That is, the shots that did not have their
detection intervals swapped out at step 2404 can have their
detection intervals re-computed using the models of FIGS. 23A and
23B. But, with step 2406, there will be fewer free variables
because one or more of the shots will already have defined
detection intervals. By re-solving the state space equation with
the smaller set of shots, more optical detection intervals for
those shots can be computed because there will be more space
available to assign to shots that may benefit from longer detection
intervals.
[0304] For example, we can re-consider the toy example from above
in the context of the FIG. 24 process flow. In this example, step
2404 will operate to impose a 50 m value for Rmax on the shots
targeting the elevation of -10 degrees. This 50 m value for Rmax
translates to around 0.3 .mu.sec. This relaxes the -10 degree cases
to: [0305] TT2(1)=T(1)+0.3=12.3 [0306] TT2(2)=40.3
[0307] This means that both processors 2022 are free when Shot 3 is
taken at time 70 (where Shot 3 is the first shot at the horizon
elevation, whose detection interval we wish to make long). The FIG.
24 process flow can make Shot 3 collect until time 85.5, which
frees up a processor 2022 (say Processor A) at time 101, just in
time to collect on Shot 5. The next shot begins at time 86,
whereupon Processor B is free, and Processor B can process 17.5
.mu.sec of data and still free up before the shot at time 121
arrives.
[0308] This yields the following for the toy example with respect
to FIG. 24: [0309] Shot Time (.mu.sec)={12 40 70 86 101 121} [0310]
Elevation-Angle (degrees)={-10,-10,0,0,0,0} [0311] Detection
Intervals: {TT2(k)-T(k)}.sub.k=1,2,3,4={0.3, 0.3, 15.5, 17.5}
[0312] We can see that the FIG. 24 process flow has increased the
detection interval for the zero degree elevation shots at the
expense of those at -10 degrees in elevation, which provides a
better set of detection intervals for the shot list.
[0313] Lidar System Deployment:
[0314] The inventors further note that, in an example embodiment,
the lidar receiver 1400 and the lidar transmitter 100 are deployed
in the lidar system in a bistatic architecture. With the bistatic
architecture, there is a spatial offset of the field of view for
the lidar transmitter 100 relative to the field of view for the
lidar receiver 1400. This spatial separation provides effective
immunity from flashes and first surface reflections that arise when
a laser pulse shot is fired. For example, an activated pixel
cluster of the array 1802 can be used to detect returns at the same
time that the lidar transmitter 100 fires a laser pulse shot 122
because the spatial separation prevents the flash from the newly
fired laser pulse shot 122 from blinding the activated pixel
cluster. Similarly, the spatial separation also prevents the
receiver 1400 from being blinded by reflections from surfaces
extremely close to the lidar system such as glass or other
transparent material that might be located at or extremely close to
the egress point for the fired laser pulse shot 122. An additional
benefit that arises from this immunity to shot flashes and nearby
first surface reflections is that it permits the bistatic lidar
system to be positioned in advantageous locations. For example, in
an automotive or other vehicle use case as shown by FIG. 25, the
bistatic lidar system 2500 can be deployed inside a
climate-controlled compartment 2502 of the vehicle 2504 (such as
the passenger compartment), which reduces operational risks to the
lidar system arising from extreme temperatures. For example, the
air-conditioning inside the compartment 2502 can reduce the risk of
the lidar system 2500 being exposed to excessive temperatures.
Accordingly, FIG. 25 shows an example where the bistatic lidar
system 2500 is deployed as part of or connected to a rear view
mirror assembly 2510 or similar location in compartment 2502 where
the lidar system 2500 can fire laser pulse shots 122 and detect
returns 1402 through the vehicle's windshield 2512. It should be
understood that the components of FIG. 27 are not shown to
scale.
[0315] Multi-Channel Readout for Returns:
[0316] For another example embodiment, it should be understood that
the detection timing constraint discussed above where the detection
intervals are non-overlapping can be removed if a practitioner
chooses to deploy multiple readout channels as part of the
photodetector circuitry 1800, where these multiple readout channels
are capable of separately reading the signals sensed by different
activated pixel clusters at the same time. FIG. 26 shows an example
receiver 1400 that includes multiple readout channels (e.g., M
readout channels). Each readout channel can include a multiplexer
2600 that reads the signals sensed by a given activated cluster of
pixels 1804, in which case the lidar receiver 1400 is capable of
detecting returns that impact different pixel clusters of the array
1802 at the same time. It should be understood that amplifier
circuitry can be placed between the array 1802 and the multiplexers
2600 as described above with reference to FIG. 27 and as described
in the above-referenced and incorporated U.S. Pat. Nos. 9,933,513
and 10,754,015. Through the use of multiple readout channels as
exemplified by FIG. 26, practitioners can relax the constraint that
the detection intervals for detecting returns from different shots
be non-overlapping. Among other benefits, this approach can open up
possibilities for longer range detections that might otherwise be
missed because collections from a first pixel cluster needed to
detect the long range return would have stopped so the receiver
1400 could start collection from a second pixel cluster needed to
detect a shorter range return. With the approach of FIG. 26,
collections from the first pixel cluster can continue for a longer
time period, even when collections are occurring from the second
pixel cluster through a different readout channel, thereby enabling
the detection of the longer range return.
[0317] Pulse Bursts to Resolve Angle to Target:
[0318] In accordance with another example embodiment, the lidar
system can fire pulse bursts. The use of pulse bursts can provide
greater precision in resolving an angle to a detected object in the
field of view, as explained in greater detail below. With a pulse
burst, the lidar transmitter 100 fires multiple laser pulse shots
with short time separations between the shots. FIG. 28 shows an
example pulse burst 2800 as plotted with respect to energy over
time. In the example of FIG. 28, the pulse burst includes two
pulses--a first pulse 2802 and a second pulse 2804. Pulses 2802 and
2804 can exhibit a Gaussian pulse shape. The time separation
between pulses 2802 and 2804 is expressed as t.sub.sep. For a pulse
burst, this time separation t.sub.sep will be short, namely a time
separation value in a range between 100 nsec and 10 .mu.sec. For
example, a preferred time separation can be a value in a range
between 200 nsec and 500 nsec. While FIG. 28 shows an example where
the first pulse 2802 and second pulse 2804 have roughly equal
energies, it should be understood that this need not be the case.
In some circumstances, the second pulse 2804 may exhibit a lower
energy than the first pulse 2802 due to the short charging time for
the laser source 102 during the time separation t.sub.sep between
the shots. However, some practitioners may employ techniques such
as those described below with respect to FIG. 35 to roughly
equalize the energies of pulses 2802 and 2804 via adjustable
control of seed energy for the laser source 102.
[0319] FIG. 29 shows how mirror motion can cause the pulses 2802
and 2804 of a pulse burst 2800 to be directed at different shot
angles. In the example of FIG. 29, mirror 110 can scan as discussed
above, and when laser source 102 fires the pulse burst 2800, the
first pulse 2802 will strike the mirror 110 when the mirror 110 is
at a tilt angle of .theta..sub.0. This results in the first pulse
2802 being directed at a shot angle of .mu..sub.0. The second pulse
2804 will strike the mirror 110 a short time later, when the mirror
110 is at a tilt angle of .theta..sub.1. This results in the second
pulse 2804 being directed at a shot angle of .mu..sub.1. Due to the
short time separation between pulses 2802 and 2804, the angular
separation between the pulses 2802 and 2804 will also be small.
[0320] The lidar system can leverage this small angular separation
between pulses 2802 and 2804 to more precisely resolve the shot
angle to an object detected in the field of view. When the lidar
system detects a return from a laser pulse shot fired at a given
shot angle, there will be some level of uncertainty about the
precise shot angle to the object from which the return was
received. This is because there will be a spreading of pulse energy
as the laser pulse shot propagates toward the object. If the object
is located precisely at the shot angle, the maximum amount of shot
energy will strike the object and reflect back to the lidar
receiver 1400. However, if the object is located slightly off the
shot angle, it may still receive some of the energy of the laser
pulse shot and reflect it back to the lidar receiver 1400 to
produce a return detection. Accordingly, there is some level of
angular imprecision when detecting a return from a single laser
pulse shot. This angular imprecision is due to the fact that the
intensity is unknown because, as is usually the case, we cannot
separate the contribution of the reduced energy from intensity
weakness versus angle offset. For example if we "expect" a 1 nJ
pulse return but instead get 0.8 nJ, we do not know if (1) our
object is 80% of the anticipated reflectivity or (2) the
reflectivity is as we anticipated, but the angle offset decremented
the received value by to 80% of the original value. The use of the
pulse burst as described herein can provide the system with the
additional measurements to reduce this uncertainty.
[0321] As noted, the use of pulse bursts can help resolve much of
this angular imprecision. FIG. 30 shows an example where the pulse
burst 2800 is used to help resolve a shot angle to an object. In
the example of FIG. 30, an initial laser pulse shot 3000 is fired
at an initial shot angle (which we will set at 0 degrees for ease
of explanation). The lidar receiver 1400 detects a return from shot
3000, which means that the initial shot angle can serve as a
candidate position 3004 for the detected object. In response to
this detection, the lidar system can then fire the pulse burst 2800
at shot angles which surround the initial shot angle, where one of
the pulses of the pulse burst 2800 is fired just to the left of the
initial shot angle and the other of the pulses of the pulse burst
2800 is fired just to the right of the initial shot angle. In the
example of FIG. 30, the first pulse 2802 of the pulse burst 2800
can be fired at a shot angle of -0.05 degrees while the second
pulse 2804 of the pulse burst 2800 can be fired at a shot angle of
+0.05 degrees. Returns from the shots for pulses 2802 and 2804 can
establish candidate positions 3002 and 3006 respectively for their
respective shot angles. While the example of FIG. 30 shows the shot
angles of pulses 2802 and 2804 as being offset from the initial
shot angle by 0.05 degrees, it should be understood that other
offsets from the initial shot angle could be employed. For example,
a practitioner may find it desirable to fire the pulses of the
pulse bursts at shot angles that are offset from the initial shot
angle by values in a range between 0.025 degrees and 0.1 degrees.
While the example angle resolution embodiments discussed herein are
focused on resolving the azimuth angle to the detected object, it
should be understood that similar techniques could be employed for
resolving elevation angle if desired. That said, in a lidar system
which has dual axis scan with one axis resonant, the resonant axis
will be the axis of choice for pulse bursts simply because the time
before revisit to an object would be much faster.
[0322] The lidar system can then use the returns from the initial
laser pulse shot 3000 and the pulse burst 2800 to more precisely
resolve the angle to the detected object. In this context, the
initial laser pulse shot 3000 and the pulses 2802 and 2804 can be
fired sufficiently quickly relative to the closing speed of the
object that any change in range to the object during the time from
the initial laser pulse shot to the shot for the second pulse 2804
will be negligible; in which case the range value for the returns
from all three shots can be deemed equal. Given the preferred
operational ranges for mirror scan frequency as discussed above and
typically expected velocities for objects in the field of view, it
can be expected that this assumption of negligible changes in range
for the object over the course of the three laser pulses shots will
be accurate. However, in situations where the time duration covered
by the three laser pulse shots is sufficiently long (relative to
the object's closing speed) that the change in range for the object
over this time is not negligible, the pulse burst can still help
resolve angular precision to the object if the object's closing
speed is known. With knowledge of object speed, range offsets can
be determined, and the effects of object motion can be removed from
the angular resolution process. Similarly, if the object is
relatively stationary, but the lidar system itself is moving at a
high rate of speed, knowledge of the lidar system's speed can be
used to offset range changes to allow for higher precision angular
resolution via pulse bursts. As an example, suppose a target has a
closing speed of 50 m/s (corresponding to two vehicles on approach
at 100 kph each). Suppose the time between pulses in a dual pulse
burst is 1 msec. In that time the object has moved 5 cm. Now
suppose in the first shot, at angle -0.05 degrees, we get two
returns, at 100 m and 101.15 m, and in the second shot at angle
+0.05 degrees, we get two returns at 102 m and 101.1 m. We proceed
by first imposing a range offset on the first shot, based in our
knowledge that closing speed imposes a shift of 5 cm, so now we are
comparing: {99.95 m, 101.1 m} to {101.95 m, 101.1 m}. In this
instance, clearly the second element in each shot corresponds to
the same target, while the first element corresponds to some other
object present in each beam solely.
[0323] FIG. 31 shows an example process flow for controlling the
firing of pulse bursts in response to object detections, and FIGS.
32A and 32B show example process flows for return signal processing
to more precisely resolve the angle to the detected object based on
the returns from the pulse burst.
[0324] At step 3100 of FIG. 31, the lidar receiver 1400 detects an
object (the target) with a return from a laser pulse shot fired at
a shot angle of .mu..sub.initial. This laser pulse shot can serve
as the initial pulse for angle resolution, and it can also be
referred to as the previous laser pulse shot with respect to the
subsequent pulse burst discussed below. As an example, the lidar
transmitter 100 can fire the initial laser pulse shot when the
lidar system is operating according to a baseline scan pattern,
such as a scan pattern where laser pulse shots are fired at defined
time intervals (such as a new laser pulse shot every 1 .mu.sec).
Another example of a baseline scan pattern can be shot pattern
corresponding to a software-defined frame such as any of those
described in U.S. Pat. No. 11,002,857, the entire disclosure of
which is incorporated herein by reference (e.g., a foviation shot
pattern, a region of interest shot pattern, etc.).
[0325] From the return detected at step 3100, the system will know
(1) a range to the detected target, (2) the initial shot angle for
the detected target, and (3) the return energy from the initial
pulse. This target can then be interrogated with a pulse burst 2800
to better resolve the angle to the target, in which case the lidar
transmitter 100 can switch from the baseline scan pattern to a
pulse burst mode where the pulse burst 2800 is fired at the
detected target. With the pulse burst, the time separation between
laser pulse shots will be significantly shorter than the time
separation between laser pulse shots when operating according to
the baseline scan pattern. This decrease in time separation between
pulse shots for the pulse burst mode may be an increase in a range
of 10.times. to 100.times. relative time separation between pulse
shots for the baseline scan mode. Thus, in an example where the
baseline scan pattern separates laser pulse shots by 10 .mu.sec,
the pulse burst mode can then separate the pulses 2802 and 2804 of
the pulse burst by 100 nsec for an example where a 100.times.
reduction in pulse separation is achieved. In another example,
where the baseline scan pattern separates laser pulse shots by 2
pec, a 10.times. factor reduction leads to a time separation
between pulses 2802 and 2804 of the pulse burst 2800 equal to 200
ns.
[0326] The decision to further interrogate the target can be made
by control circuit 106. For example, in an automotive use case, the
control circuit 106 can communicate with the vehicle's motion
planning control system, and the motion planning control system may
include control laws that decide which targets are worthy of
further interrogation. For example, if the motion planning control
system predicts that an incoming target has a sufficiently high
probability of posing a collision threat on a current motion
planning trajectory, the target can be slated for further
interrogation via a pulse burst. Examples of coordination between a
lidar system and a vehicle motion planning control system are
described in U.S. Pat. No. 10,495,757, the entire disclosure of
which is incorporated herein by reference.
[0327] At step 3102, the control circuit 106 specifies the shot
angles and energies to use for the pulses 2802 and 2804 of the
pulse burst 2800. The shot angle and energy for pulse 2802 can be
specified as .mu..sub.0 and E.sub.0 respectively. The shot angle
and energy for pulse 2804 can be specified as .mu..sub.1 and
E.sub.1 respectively. Shot angles .mu..sub.0 and .mu..sub.1 can be
centered around the initial shot angle .mu..sub.initial so that
they effectively surround the initial shot angle .mu..sub.initial.
A number of factors can impact the choices available for specifying
the shot angles for the pulses 2802 and 2804 of the pulse burst.
For example, the specified shot angles should be selected so that
the shot angles for pulses 2802 and 2804 are expected to stay on
the target. Accordingly, the specified shot angles should not be
too far off the initial shot angle to reduce the risk of missing
the target with pulses 2802 and 2804. Another factor that can
affect the choice of specified shot angles for pulses 2802 and 2804
would be the angular spacing of the shot list according to the
baseline scan pattern. In this regard, the specified shot angles
should be less than the angular spacing of the baseline scan
pattern because the goal of the pulse burst is to add angular
precision to the target than the baseline scan pattern would
otherwise provide. At the other end of the spectrum, the minimum
angular spacing can be impacted by the ability to discriminate
angular resolution above noise levels. In this regard, a likely
minimum for the specified shot angles relative to the initial shot
angle would be around plus or minus 0.025 degrees as smaller
angular spacing would erode the ability to rely on the conclusions
that can be drawn from the angle refinements due to noise. In
consideration of these factors, in an example embodiment, the shot
angles for pulses 2802 and 2804 can be a value that is a range of
around plus or minus 0.025 degrees to plus or minus 0.1 degrees
relative to the initial shot angle. The choice of shot energies
E.sub.0 and E.sub.1 for the pulses 2802 and 2804 of the pulse burst
2800 can be based on the expected range to the target (where the
energies are set sufficiently high to be able to detect returns
from the target at the expected range).
[0328] The control circuit 106 can then, at step 3104, schedule for
this pulse burst 2800 by inserting shots for .mu..sub.0 and
.mu..sub.1 for a subsequent scan line (e.g., the next return scan
from the scan that produced the detection at step 3100). If this
subsequent scan line previously had a shot for .mu..sub.initial
scheduled, step 3104 can also remove the shot for .mu..sub.initial
from the schedule. In a preferred embodiment, the pulse burst 2800
is scheduled to be fired between 30 .mu.sec and 100 .mu.sec after
the initial laser pulse shot is fired.
[0329] The control circuit 106 can then evaluate whether the
planned schedule for the specified pulse burst 2800 can be
accomplished during the subsequent scan line in view of the laser
energy model 108 and the mirror motion model 308. At step 3106, the
control circuit 106 uses the mirror motion model 308 to find the
time difference between pulses 2802 and 2804 of the specified pulse
burst 2800. It should be understood that this time difference would
correspond to the value t.sub.sep discussed above with respect to
FIG. 28. As noted above, the mirror motion model 308 can predict
the time slot at which the subject mirror 110 will direct shots at
a specified shot angle. Thus, by computing the time slots for shots
at shot angles of .mu..sub.0 and .mu..sub.1, the control circuit
106 can compute the time difference between these pulses 2802 and
2804 as the difference between the two time slots. Knowledge of
this time difference allows the control circuit 106 to also
determine how much energy is available in the laser source 102 for
the second pulse 2804.
[0330] Thus, at step 3108, the control circuit 106 uses the laser
energy model 108 and the determined time difference from step 3106
to simulate the energy levels for pulses 2802 and 2804 according to
the planned schedule. The control circuit 106 can use the computed
time difference from step 316 as the value of the time duration
.delta. that represents the time between the firing of pulse 2802
and the firing of pulse 2804. Accordingly, the laser energy model
108 can be used to determine the energy levels that the lidar
system can produce for pulses 2802 and 2804 according to the
planned schedule.
[0331] At step 3110, the control circuit 106 compares these
determined energy levels with the specified energy levels of
E.sub.0 and E.sub.1 for pulses 2802 and 2804 respectively. If the
determined energy levels for pulses 2802 and 2804 satisfy E.sub.0
and E.sub.1, then the process flow can proceed to step 3112. At
step 3112, the lidar transmitter 100 fires the specified pulse
burst 2800 according to the planned schedule. However, if the
determined energy level for pulse 2802 does not satisfy E.sub.0
and/or the determined energy level for pulse 2804 does not satisfy
E.sub.1, then the process flow would proceed from step 3110 to step
3114. At step 3114, the control circuit 106 updates the planned
scheduled for the specified pulse burst 2800 by deferring the pulse
burst 2800 to a subsequent scan line (e.g., the next scan line
after the scan line defined at step 3104 and evaluated at steps
3106-3110). It should be understood that the order of pulses 2802
and 2804 can be flipped as a result of the deferral in view of the
reverse scan direction exhibited by the next scan line. A
practitioner may choose to make this next scan line a scan line set
aside specially for the pulse burst to ensure that the laser source
102 will have sufficient energy available for E.sub.0 and E.sub.1.
From step 3114, the process flow can return to step 3108 where the
laser energy model 108 is consulted to simulate the energy levels
for the newly scheduled pulse burst 2800. In this fashion, the
control circuit 106 will eventually find the proper time to fire
the pulse burst at step 3112.
[0332] Once the pulse burst 2800 has been fired, the lidar receiver
1400 can then process the returns from this pulse burst 2800 to
more precisely resolve the angle to target. FIGS. 32A and 32B show
examples of different options for such return processing. The lidar
receiver 1400 can perform the process flows of FIGS. 32A and 32B
using signal processing circuit 1820 (e.g., a processor within the
signal processing circuit 1820).
[0333] The FIG. 32A process flow describes a first technique for
resolving the angle to the target. At step 3200 of FIG. 32A, the
lidar receiver 1400 processes the return signal 1806 corresponding
to the pulse burst 2800 to find a pulse pair within the return
signal 1806 that is separated by t.sub.sep. This pulse pair in the
return signal 1806 would correspond to the returns from pulses 2802
and 2804, and the lidar receiver 1400 can determine the energy
levels for the returns.
[0334] At step 3202, the lidar receiver 1400 then compares the
return energies for the initial pulse (known from step 3100), the
first pulse 2802 of the pulse burst 2800 (known from step 3200),
and the second pulse 2804 of the pulse burst (known from step
3200). If the return energy from the initial pulse is the largest,
this means that the initial shot angle serves as a good
approximation of the angle to the target and can be used to reflect
the angle to the target (step 3204). If the return energy from the
first pulse 2802 is the largest, this means that the lidar receiver
can resolve the target angle in the direction of the shot angle for
the first pulse 2802 (.mu..sub.0) (see step 3206). As an example,
the lidar receiver can use the first pulse shot angle .mu..sub.0 as
the target angle (or some value between the first pulse shot angle
.mu..sub.0 and the initial shot angle .mu..sub.initial). If the
return energy from the second pulse 2804 is the largest, this means
that the lidar receiver can resolve the target angle in the
direction of the shot angle for the second pulse 2804 (.mu..sub.1)
(see step 3208). As an example, the lidar receiver can use the
second pulse shot angle .mu..sub.1 as the target angle (or some
value between the first pulse shot angle .mu..sub.1 and the initial
shot angle .mu..sub.initial).
[0335] If still more precision is desired for resolving the angle
to the target, a technique such as the process of FIG. 32B can be
employed. Step 3200 of FIG. 32B can operate in the same manner as
step 3200 of FIG. 32A, to thereby detect the return energies for
the first and second pulses 2802 and 2804 of the pulse burst. At
this point, the lidar receiver 1400 can compute the target angle
using the technique discussed below.
[0336] FIG. 33A shows a plot of expected return energy from a laser
pulse shot that strikes a target as a function of an offset angle
to the target (for an example where the laser pulse shot exhibits a
Gaussian pulse shape). This plot will have the same basic shape as
the laser pulse shot itself. The offset angle in this context is
the difference between the shot angle and the true angle to the
target. Thus, if the shot angle happens to be the true angle to the
target, then the offset angle would be zero (and the return energy
would be at a maximum). However, as the offset angle increases in
value (in either the positive or negative direction), the return
energy will decrease as shown by the plot of FIG. 33A.
[0337] The lidar receiver 1400 can leverage the relationship shown
by the plot of FIG. 33A to better resolve the angle to the target
by determining a fit of the determined return energy levels for
pulses 2802 and 2804 to this curve. For example, FIG. 33B shows an
example where return 3302 from pulse 2802 falls just to the left of
the return energy peak and where return 3304 from pulse 2804 falls
just to the right of the return energy peak. In this scenario, the
lidar receiver 1400 can use the midpoint of these returns as the
offset angle for the object, in which case it should be understood
that the initial shot angle would serve as the true angle (given
that pulses 2802 and 2804 are equally offset from the initial shot
angle). FIG. 33C shows an example where the pulse burst returns
3302 and 3304 fall on the curve to the right of the peak return
energy. In this case, the true angle would fall to the right of the
initial shot angle (presuming the rightward direction corresponds
to a positive offset angle). FIG. 33D shows an example where the
pulse burst returns 3302 and 3304 fall on the curve to the left of
the peak return energy. In this case, the true angle would fall to
the left of the initial shot angle (presuming the leftward
direction corresponds to a negative offset angle).
[0338] However, while the shape of the FIG. 33A plot is known (due
to knowledge of the pulse shape), the scale of the FIG. 33A plot
will be unknown. Accordingly, curve fitting techniques can be used
to find out where given pulse returns fall on the FIG. 33A plot. To
facilitate such curve fitting, at step 3212 of FIG. 32B, the lidar
receiver 1400 can compute the energy ratio of the return energies
for the pulse pair. This ratio can be computed by taking the
difference of return intensities and dividing that difference by
the sum of the return intensities. Thus, if the return energy from
pulse 2802 is A, and the return energy from pulse 2804 is B, the
energy ratio can be computed as:
Energy .times. Ratio = A - B A + B ##EQU00013##
[0339] To translate the FIG. 33A plot to a scaleless relationship,
the derivative of the return energy can be plotted as a function of
offset angle as shown by FIG. 34. With this relationship, the
y-axis of the FIG. 34 plot corresponds to the computed energy
ratio. Accordingly, at step 3214, the lidar receiver 1400 can
determine the offset angle for the pulse burst based on where the
computed energy ratio from step 3212 falls on the FIG. 34 plot.
FIG. 34 shows an offset angle 3400 that corresponds to the FIG. 33B
example of pulse burst returns, an offset angle 3402 that
corresponds to the FIG. 33C example of pulse burst returns, and an
offset angle 3404 that corresponds to the FIG. 33D example of pulse
burst returns. Given that the pulse shape is known, this means that
the FIG. 34 plot can also be known in advance. Accordingly, the
plot of FIG. 34 can be represented as a lookup table (LUT) where
different offset angles are indexed by different energy ratio
values. Thus, at step 3214, the lidar receiver 1400 can determine
the subject offset angle for the pulse burst by looking up the
offset angle from the LUT that corresponds to the computed energy
ratio from step 3212. If there is not an entry in the LUT for the
precise energy ratio computed at step 3402, the lidar receiver 1400
can use interpolation to find the offset angle based on the offset
angles of the energy ratios immediately larger and smaller than the
computed energy ratio.
[0340] At step 3216, the lidar receiver 1400 can then resolve the
angle to the target based on the shot angle of the initial laser
pulse shot and the determined offset angle from step 3214. In this
regard, the refined angle to target can be computed as the sum of
the initial shot angle (.mu..sub.initial) and the determined offset
angle.
[0341] With FIGS. 32A and 32B, the refined angle to the target can
be reported to the point cloud along with a corresponding range
value applicable to the subject pulse shots. If desired, a
practitioner need not have the point cloud register all three
returns (from the initial laser pulse shot, pulse 2802, and pulse
2804). Instead, the point cloud may conflate the three returns to
the refined target angle, along with a corresponding range value
and some aggregation of the return energies from the shots (e.g., a
weighted average of the return intensities from the shots).
[0342] Furthermore, the three returns may be used by the system to
update velocity information. For example, if the ranges are offset
by an amount exceeding that anticipated variation from random noise
fluctuation, then one might attribute this difference to the
presence of a true range offset. The lidar receiver 1400 can then
apply this range offset, in reverse of the previously described
range alignment process, to derive velocity information for the
target.
[0343] Variable Laser Seed Energy to Control Pulse Burst
Energies:
[0344] The examples of FIGS. 32A and 32B presume that the energies
of pulses 2802 and 2804 have been roughly equalized in the
transmitted pulse burst 2800. However, this need not be the case.
In the event that there is a mismatch in energy between pulses 2802
and 2804 (e.g., where pulse 2804 may have lower energy), the lidar
receiver 1400 can employ scaling to normalize the return energy
from the second pulse 2804 to the energy level of the first pulse
2802. This scaling can proceed as follows. Suppose the system knows
the first pulse 2802 when fired had twice the energy of the second
pulse 2804. The lidar receiver can then divide the first pulse
return intensity measurement by 2 before computing the sum and
difference. Conversely, if pulse 2804 had 50% less energy than
pulse 2802, then when computing the energy ratio at step 3212, the
lidar receiver can double the value of B (or halve the value of A)
to compensate for the lesser energy in the second pulse 2804.
Similar scaling can be used when comparing the return energies at
step 3202. The system can thus isolate and remove all dependencies
except angle offsets so that it can attribute the angle offsets
alone to accounting for the change in intensity returns between
pulse burst shot returns.
[0345] FIG. 35 shows an example laser source 102 that uses
adjustable seed energy to regulate the energy levels in the pulses
of a pulse burst. The laser source 102 of FIG. 35 can be an optical
amplification laser source that comprises an optical amplifier 116,
pump laser 118, and variable seed laser 3500. As an example, the
optical amplification laser source can be a pulsed fiber laser
source where the optical amplifier 116 is a fiber amplifier. The
control circuit 106 can then provide a seed energy control signal
3502 to the variable seed laser 3500 to regulate the energy levels
in the pulses of a pulse burst.
[0346] While one can adjust the pump energy over time, this is not
a fast enough process for pulse bursts (where typical switching
times will be on the order of many microseconds). This is because
the pulse burst must be fired quickly to remain consistent with the
mirror speed. For example a typical mirror velocity with respect to
example embodiments discussed above is a degree per microsecond,
and a typical pulse burst angle offset is 0.1 degrees. This
translates into a need to fire the shots of the pulse burst at 100
nanosecond timescales.
[0347] To accommodate this need, the control circuit 106 can change
the energy in the variable seed laser 3500 for the shots of the
pulse burst.
[0348] Suppose we have a pump laser 118 that produces 1 W of energy
in the optical amplifier 116 (which might be a doped fiber laser
amplifier for example). If we fire two shots spaced by 100
nanoseconds, and we have a gain factor of a in the fiber amplifier
116; then the energy in each pulse shot of the pulse burst, after a
prior delay of T=1 .mu.sec with stored energy E=1 .mu.J is given in
units of .mu.J by:
E.sub.1=.alpha.E+(1-.alpha.)T=1
E.sub.2=(1-.alpha.)(0.1)+1=1.1-0.1.alpha.
[0349] It should be understood that without adjustable control over
the seed energy, the laser source 102 cannot make the two pulses in
a pulse burst have equal energy due to the time constraints of
charging the laser balanced against the short time interval between
pulses of the pulse burst. But, we can equalize the pulse energies
in the pulse burst if we can change the gain factor .alpha.. In
this simple toy example .alpha.=1.0.
[0350] Physically changing the gain for fixed pump power is
achieved by adjusting the seed energy. The reason such adjustment
of the seed energy adjusts the gain is that the role of the seed
laser 3500 is to stimulate the electrons in the excited energy
states in the fiber amplifier 116 to collapse to the ground state.
The more energy in the seed, then the stronger the electric field
and the more likely that ground state collapse-induced photon
emission occurs.
[0351] The expression below defines the relationship between pulse
energy and seed gain adjustments as follows:
E n = ( ( A n .times. e .times. w A o .times. l .times. d - k
.times. A n .times. e .times. w ) .times. E n - 1 + A n .times. e
.times. w .times. k .times. t n ) ##EQU00014##
[0352] In this equation: [0353] n is the index associated with the
n-th shot being fired. [0354] t.sub.n is the time difference
between shots at index n-1 and index n. [0355] E is the energy in
the shot corresponding to its index. [0356] k is a "gain factor",
and it corresponds to the gain term when the seed pulse energy is
set to the "normal" level. [0357] A.sub.new and A.sub.old are the
current and prior seed gain levels (corresponding to different seed
pulse energy levels for a fixed pump energy level). The values for
A can range from Ak being (nearly) zero to 1. The reason for the
[0,1] range is as follows. [0358] When the gain Ak is 1, the entire
energy in the optical amplifier 116 is expunged (E.sub.n=t.sub.n)
[0359] When Ak.about.0 the energy fired by the laser source 102 no
longer depends on the charge time and is roughly constant. Note
that the value of A can be bigger or smaller than unity depending
on need, as long as the value of Ak lies in these bounds.
[0360] Note that when A.sub.new and A.sub.old are both equal to 1
we get:
E.sub.n=E.sub.n-1(1-k)+kt.sub.n
[0361] Note that this is same as in laser energy model 108
discussed above for the expression EF(t+.delta.), but where t.sub.n
is used in place of .delta. and where k is used in place of a. But,
for the purpose of explaining the variable seed gain, it is useful
and makes algebra simpler to work with this new notation.
[0362] We can assure that the seed energy is varied to achieve the
two shots in the pulse burst having equal energy. We will describe
this process in mathematical detail below.
[0363] To start, let the energy in the initial pulse shot (the shot
prior to the pulse burst) be E. Next, let t.sub.sep be the time
between the pulse shots of the pulse burst, and let T be the time
before the first pulse shot of the pulse burst.
[0364] We will pursue the math using a toy example. To begin we
assume when we fire at time T, we had full pump level
(A.sub.old=1). Moreover, we can set values for T, t.sub.sep, E, and
k as follows: [0365] T=10 [0366] t.sub.sep=2 [0367] E=4 [0368]
k=1/2
[0369] Next, let x=A.sub.new and y=A'.sub.new, where A.sub.new and
A'.sub.new denote the seed gain at the first and second pulse shots
of the pulse burst respectively. We can also denote the
corresponding energies for the first and second pulse shots of the
pulse burst as E.sub.1st and E.sub.2nd respectively. Using the
expression for E.sub.n discussed above, this yields:
2 .times. x + 5 .times. x = E 1 .times. st ##EQU00015## y [ ( 1 x -
1 2 ) [ 2 .times. x + 5 ] + x ] = E 2 .times. n .times. d
##EQU00015.2##
[0370] We then seek to make the energies equal:
y [ ( 1 - x 2 ) [ 2 .times. x + 5 ] + x ] = 7 .times. x 2
##EQU00016##
[0371] This yields a quadratic formula that gives us the expression
for the seed gain:
0 = x 2 ( 7 + y ) + x .function. ( - y 2 ) - 5 .times. y .fwdarw.
##EQU00017## y 4 .times. ( 7 + y ) .+-. y 2 + 5 .times. y
.function. ( 7 + y ) 2 .times. ( 7 + y ) = x , E 1 .times. st = E 2
.times. n .times. d = E = 4 = 7 .times. x ##EQU00017.2##
[0372] Here, we assume we wish to keep the generated pulse energy
after optical amplification equal to the original (at 10.times.
less charge time), and the solution is:
x= 4/7=A.sub.new
A.sub.new'=y=.about.1.8
[0373] As expected the seed gain, in the second term, is larger
than the first. Notice that in both cases the seed gain is
physically realizable (with a 1 W laser, we can never get more than
t_n extra energy from charge time t.sub.n, and in our case we get
2/7 and 0.9 respectively).
[0374] In practice, it is often desirable to simply simulate the
laser energy model 108 and then find the desired seed gains that
result. The toy example discussed above simply shows the behavior
of this simulation for a particular case. The control circuit 106
can thus provide seed energy control signals 3502 to the variable
seed laser to control the values for A in a manner that achieves a
desired regulation of the energy levels in the pulses of the pulse
burst. As discussed above, this regulation can be equalized pulse
energy. However, it should be understood that this need not
necessarily be the case if desired by a practitioner.
[0375] While the invention has been described above in relation to
its example embodiments, various modifications may be made thereto
that still fall within the invention's scope.
[0376] For example, while the example embodiments discussed above
involve a mirror subsystem architecture where the resonant mirror
(mirror 110) is optically upstream from the point-to-point step
mirror (mirror 112), it should be understood that a practitioner
may choose to position the resonant mirror optically downstream
from the point-to-point step mirror.
[0377] As another example, while the example mirror subsystem 104
discussed above employs mirrors 110 and 112 that scan along
orthogonal axes, other architectures for the mirror subsystem 104
may be used. As an example, mirrors 110 and 112 can scan along the
same axis, which can then produce an expanded angular range for the
mirror subsystem 104 along that axis and/or expand the angular rate
of change for the mirror subsystem 104 along that axis. As yet
another example, the mirror subsystem 104 can include only a single
mirror (mirror 110) that scans along a first axis. If there is a
need for the lidar transmitter 100 to also scan along a second
axis, the lidar transmitter 100 could be mechanically adjusted to
change its orientation (e.g., mechanically adjusting the lidar
transmitter 100 as a whole to point at a new elevation while mirror
110 within the lidar transmitter 100 is scanning across
azimuths).
[0378] As yet another example, a practitioner may find it desirable
to drive mirror 110 with a time-varying signal other than a
sinusoidal control signal. In such a circumstance, the practitioner
can adjust the mirror motion model 308 to reflect the time-varying
motion of mirror 110.
[0379] As still another example, it should be understood that the
techniques described herein can be used in non-automotive
applications. For example, a lidar system in accordance with any of
the techniques described herein can be used in vehicles such as
airborne vehicles, whether manned or unmanned (e.g., airplanes,
drones, etc.). Further still, a lidar system in accordance with any
of the techniques described herein need not be deployed in a
vehicle and can be used in any lidar application where there is a
need or desire for hyper temporal control of laser pulses and
associated lidar processing.
[0380] As yet another example, while the example FIG. 31 process
flow shows the firing of the pulse burst 2800 in response to target
detection from an initial laser pulse shot (see step 3100), it
should be understood that the initial laser pulse shot can be
omitted in certain circumstances if desired by a practitioner.
Improvements in angle resolution can still be achieved through the
use of returns from just the pulse burst. While the additional use
of the initial pulse return helps further improve angle resolution
while also improving performance by averaging out more noise, some
practitioners may find it desirable to fire pulse bursts 2800 even
if an initial laser pulse shot was not fired.
[0381] As still another example, while the example embodiments for
angle resolution using pulse bursts discussed above employ compute
resources within the lidar receiver 1400 to resolve the angle to
the target, it should be understood that compute resources located
elsewhere in the lidar system could be employed for this purpose if
desired by a practitioner. While having the lidar receiver 1400
perform the angle resolution is advantageous because of the reduced
latency involved in having the relevant processing operations
performed by the system components that directly process the return
signals from the laser pulse shots, it should be understood that
some practitioners may choose to employ compute resources located
elsewhere (such as compute resources within the system controller
800 if a practitioner deems the increased latency arising from data
transfer across system components acceptable.
[0382] These and other modifications to the invention will be
recognizable upon review of the teachings herein.
* * * * *