U.S. patent application number 14/214637 was filed with the patent office on 2014-09-18 for apparatuses and methods for controlling a gimbal and other displacement systems.
This patent application is currently assigned to FREEFLY SYSTEMS, INC.. The applicant listed for this patent is FREEFLY SYSTEMS, INC.. Invention is credited to DAVID BLOOMFIELD, JOHN ELLISON, TABB FIRCHAU, STEVE WEBB.
Application Number | 20140267778 14/214637 |
Document ID | / |
Family ID | 50434165 |
Filed Date | 2014-09-18 |
United States Patent
Application |
20140267778 |
Kind Code |
A1 |
WEBB; STEVE ; et
al. |
September 18, 2014 |
APPARATUSES AND METHODS FOR CONTROLLING A GIMBAL AND OTHER
DISPLACEMENT SYSTEMS
Abstract
Apparatuses and methods for controlling a gimbal and other
displacement systems are disclosed herein. In accordance with one
or more embodiments of the invention, a pointing angle of a camera
attached to a gimbal may be controlled based, at least in part, on
one or more control signals provided by a controller. The control
signals may be used to compensate for displacement of the camera,
to add perceived displacement of the camera, to selectively align a
pointing angle of the camera and/or to allow a pointing angle to be
manually determined.
Inventors: |
WEBB; STEVE; (Gravesend,
GB) ; ELLISON; JOHN; (Ipswich, GB) ; FIRCHAU;
TABB; (Redmond, WA) ; BLOOMFIELD; DAVID;
(Redmond, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
FREEFLY SYSTEMS, INC. |
REDMOND |
WA |
US |
|
|
Assignee: |
FREEFLY SYSTEMS, INC.
REDMOND
WA
|
Family ID: |
50434165 |
Appl. No.: |
14/214637 |
Filed: |
March 14, 2014 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61792878 |
Mar 15, 2013 |
|
|
|
Current U.S.
Class: |
348/169 |
Current CPC
Class: |
H04N 5/2328 20130101;
H04N 5/23267 20130101; B66F 11/048 20130101; H04N 5/23258 20130101;
F16M 11/123 20130101; H04N 5/23261 20130101; F16M 11/18 20130101;
H04N 5/23248 20130101; G02B 27/646 20130101; H04N 5/2252 20130101;
H04N 5/2253 20130101 |
Class at
Publication: |
348/169 |
International
Class: |
H04N 5/232 20060101
H04N005/232 |
Claims
1-3. (canceled)
4. A method for correcting a pointing direction of a camera,
actively stabilized by an active stabilization system in accordance
with a commanded pointing angle, to compensate for translational
movements of the camera, the method comprising: determining a
distance from the camera to a filming target; deriving one or more
translational measurements associated with a translational movement
of the camera; calculating a correction update as a function of at
least the distance and the one or more translational measurements;
and adjusting the commanded pointing angle of the camera based on
the correction update to retain the filming target within a field
of view of the camera.
5. A method according to claim 4, further comprising: stabilizing
the pointing direction of camera based on the adjusted commanded
pointing angle.
6. A method according to claim 4, wherein the calculating a
correction update step comprises: calculating a correction update
as a function of at least the distance, the one or more
measurements, and a current pointing angle of the camera; wherein
the function for calculating the correction update rate provides
correction updates of lower values for the camera having a higher
elevation in relation to the filming target than for the camera
having a lower elevation in relation to the filming target for
translational movements of a same magnitude.
7. A method according to claim 4, wherein: the commanded pointing
angle is adjusted incrementally, using an incremental correction
update determined based on the correction update; and the distance,
the one or more translational measurements, and the correction
update are being updated with each incremental step.
8. A method according to claim 4, wherein: the one or more
translational measurements comprise a vertical height of the camera
in relation to the filming target; and the calculating a correction
update step comprises: calculating, based on the distance and the
vertical height, a desired pointing angle for retaining the filming
target within the field of view of the camera; and calculating the
correction update based on the desired pointing angle and a current
pointing angle of the camera.
9. A method for introducing controlled disturbance into an active
stabilization system executing a stabilization process to stabilize
a pointing angle of a camera housed by the active stabilization
system in accordance with a commanded angle, the method comprising:
acquiring a measurement associated with a movement of the active
stabilization system; determining a noise value based on the
acquired measurement; and injecting the noise value into the
stabilization process causing the process to adjust the pointing
angle of the camera in a direction away from the commanded angle of
the camera.
10. A method according to claim 9, wherein the injecting step
comprises: executing an angle-based control loop of the
stabilization process based on the commanded angle to calculate a
commanded rate; adjusting the commanded angle rate using the noise
value; and executing a rate-base loop of the stabilization process
based on the adjusted commanded angle.
11. A method according to claim 9, wherein the determining step
comprises: filtering the measurement to derive a transitory
component of the measurement; and scaling the transitory component
to determine the noise value.
12. A method according to claim 9, wherein: the method is performed
for one of a pan axis, a tilt axis, and a roll axis; and the
pointing angle of the camera is adjusted for the one axis.
13. A method according to claim 9, wherein the noise value is one
of an angle, an angular rate, a control torque, and a drive
current.
14. A method for adjusting a pointing angle of a camera housed by
an active stabilization system configured to stabilize the camera
in accordance with a commanded pointing angle, the system
comprising a steering member, the steering member rotatable around
one or more of a pan axis, tilt axis, and roll axis of the system,
the method comprising: deriving a joint angle measurement of the
steering member associated with a rotational movement of the
steering member; and adjusting the pointing angle of the camera,
based on the derived joint angle measurement, in a direction of the
rotational movement of the steering member, if the joint angle
measurement exceeds the threshold window.
15. A method according to claim 14, further comprising: actively
stabilizing the pointing angle of the camera in accordance with the
commanded pointing angle, if the joint angle measurement is within
the threshold window.
16. A method according to claim 14, further comprising: indicating,
by the active stabilization system, a pointing angle locked state,
if the joint angle measurement is within the threshold window.
17. A method according to claim 14, further comprising: updating
the joint angle measurement, wherein the updating step comprises:
reducing the joint angle measurement by a threshold value of the
threshold window, if the joint angle measurement exceeds the
threshold window, and setting the joint angle measurement to zero,
if the joint angle measurement is within the threshold window; and
deriving a control command for adjusting the pointing angle of the
camera based on the updated joint angle measurement.
18. A method according to claim 17, further comprising: executing
an angle-based control loop to derive a commanded angle rate; and
executing a stabilization control loop update based on the updated
joint angle measurement and a zero commanded angle to derive the
control command for adjusting the pointing angle.
19. A method for adjusting a pointing direction of a camera housed
by an active stabilization system, the active stabilization system
executing a stabilization process to stabilize the pointing
direction of the camera, the method comprising: detecting an
externally applied force; disabling the stabilization process upon
detecting a manual adjustment condition; adjusting the pointing
angle of the camera in a direction of the externally applied force;
measuring a pointing angle of the camera; and re-enabling the
stabilization process to stabilize the pointing direction of the
camera based on the measured pointing angle of the camera in
response to detecting that the detected manual adjustment condition
failed.
20. The method according to claim 19, wherein: the detecting the
manual adjustment condition comprises determining that the external
force exceeds a pre-set threshold; and the manual adjustment
condition fails when the external force falls below the
threshold.
21. The method according to claim 19, wherein the external force is
detected based on a control signal issued by the active
stabilization system for controlling movement of a motor of the
active stabilization system.
22. The method according to claim 19, wherein the disabling step
comprises: disabling an angle-based control loop of the
stabilization process; and resetting an integral accumulator of a
rate-based control loop of the stabilization process.
23. The method according to claim 19, executed for one or more of a
pan axis, a tilt axis, and a roll axis, wherein the externally
applied force is detected in relation to the one or more axes.
Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit of Provisional Patent
Application No. 61/792,878, filed on Mar. 15, 2013, which is hereby
incorporated by reference herein in its entirety.
TECHNICAL FIELD
[0002] Embodiments of the present invention relate generally to
control systems and in particular to control of a gimbal and other
displacement systems.
BACKGROUND
[0003] Generally, technology trends toward smaller and more
efficient devices and sensor technology is not an exception.
Sensors, such as those directed to measuring motion, temperature,
voltage, pressure, and other quantifiable metrics have become both
improved with respect to accuracy and size.
[0004] A number of industries, however, such as the motion picture
industry, have not yet appreciated the advantages provided by these
improvements. For example, gimbals used to orient cameras have
remained cumbersome, typically requiring multiple operators for use
and presenting difficulties in steady shooting, particularly when
the gimbal is carried and/or steered by an operator. Moreover,
difficulties are further introduced in circumstances when
displacement of a gimbal is intentional (e.g., during a chase scene
or an earthquake scene) but reliable control of the orientation of
the gimbal is still desired.
BRIEF DESCRIPTION OF THE DRAWINGS
[0005] FIG. 1 is a schematic block diagram of an apparatus
according to an embodiment of the present invention.
[0006] FIG. 2 is a schematic block diagram of an apparatus
according to an embodiment of the present invention.
[0007] FIG. 3 is a schematic block diagram of an apparatus
according to an embodiment of the present invention.
[0008] FIG. 4 is a schematic block diagram of an apparatus
according to an embodiment of the present invention.
[0009] FIG. 5 illustrates the effect of translation with constant
pointing angle.
[0010] FIG. 6 illustrates a vertical translation showing rate of
change of pointing direction.
[0011] FIG. 7 is a diagram of a gimbal tilt drive control with
motor and IMU mounted on a camera.
[0012] FIG. 8 is a diagram of a gimbal tilt drive control showing
translation compensation of gimbal tilt angle.
[0013] FIG. 9 illustrates how translation velocity is reduced for
an elevated view-point.
[0014] FIG. 10 illustrates direct inference of tilt angle from
height and distance.
[0015] FIGS. 11A and 11B illustrate normal gimbal stabilization and
an artificially implanted rotation to simulate camera operator
acceleration and rotation.
[0016] FIG. 12 is a diagram of a gimbal controller showing a scheme
for acceleration noise addition to tilt pointing angle.
[0017] FIG. 13 is a graph illustrating high-pass filtering of an
acceleration measurement.
[0018] FIG. 14 is a diagram of a gimbal controller showing an
introduction of noise optionally from: a pre-recording, remote
mounted IMU, artificial generator, or joint angle.
[0019] FIG. 15 illustrates two operators filming a scene showing
coordination complexity.
[0020] FIG. 16 is a diagram of a gimbal controller showing a joint
angle zeroing control loop.
[0021] FIG. 17 is a diagram of a gimbal controller showing steerage
of closed loop tilt angle by an external torque producing
force.
[0022] FIGS. 18A and 18B respectively illustrate a conventional
suspended camera with translation errors and a solution using a
feedback control loop.
[0023] FIG. 19 is a diagram of a camera height stabilization
controller suspended from a support beam.
[0024] FIG. 20 is a diagram of a camera height stabilization
controller suspended from a support beam showing movement of
command height by external force and force threshold detection.
[0025] FIG. 21 is a diagram of a camera height stabilization
controller suspended from a support beam showing a vario control
mode allowing forced height changes by operator pulling down or
lifting up the camera.
DETAILED DISCLOSURE
[0026] Apparatuses and methods for controlling a gimbal and other
displacement systems are disclosed herein. Certain details are set
forth below to provide a sufficient understanding of embodiments of
the invention. However, it will be clear to one having skill in the
art that embodiments of the invention may be practiced without
these particular details. Moreover, the particular embodiments of
the present invention described herein are provided by way of
example and should not be used to limit the scope of the invention
to these particular embodiments. In other instances, well-known
circuits, control signals, timing protocols, and software
operations have not been shown in detail in order to avoid
unnecessarily obscuring the invention.
[0027] FIG. 1 is a schematic block diagram of an apparatus 100
according to an embodiment of the invention. The apparatus 100 may
include an inertial measurement unit (IMU) 110, a controller 120, a
driver 130, and a gimbal 140. A camera CAM is shown in FIG. 1 as
being attached to the gimbal 140. The camera CAM may comprise any
camera known in the art, now or in the future.
[0028] The gimbal 140 may be used to control the orientation (e.g.,
tilt, pan, and/or roll) of the camera CAM. One or more components
of the apparatus 100 may be physically or electrically coupled to
the gimbal 140 or the camera CAM including, but not limited to, the
controller 120 and the IMU 110. In one embodiment for example, one
or more components of the apparatus 100 may be coupled to the
camera CAM such that the coupled components and the camera CAM have
fixed relative positions even when the camera CAM is displaced, for
instance, by a camera operator.
[0029] The gimbal 140 may include a motor 150 that may be
configured to control the gimbal 140 such that the camera CAM is
oriented in a particular manner. The motor 150 as shown in FIG. 1
may be representative of one or more motors of the gimbal 140. The
motor 150 may comprise a direct drive motor or may comprise a
geared or belt reduction drive. In some examples, the motor 150 may
comprise a plurality of servo motors configured to control the
gimbal 140 such that tilt, roll, and/or pan of the camera CAM may
be controlled. Moreover, in at least one embodiment, the motor 150
may be configured to provide one or more signals indicative of an
angle of displacement (displacement angle). Because the motor 150
may control the orientation of the gimbal 140, the motor 150 may
further be configured to determine a displacement angle for one or
more of the gimbal axes and provide signals indicative of the
displacement angles. By way of example, the motor 150 may determine
a vertical displacement angle, a horizontal displacement angle,
and/or a rotational displacement angle and provide signals
indicative of one or more of the determined displacement angles,
for instance, to the controller 120.
[0030] The IMU 110 may comprise any IMU known in the art, now or in
the future, and may be configured to detect changes in orientation
and/or displacement of the camera CAM. The IMU 110 may be attached
to the camera CAM and/or the gimbal 150. By detecting changes in
orientation and/or displacement of the camera CAM, the IMU 110 also
detects changes in orientation and/or displacement of the gimbal
140. For example, the IMU 110 may comprise one or more sensors
physically and/or electrically coupled to the camera CAM including,
but not limited to, a global positioning system (GPS) sensor, an
accelerometer (e.g., 3-axis accelerometer), a gyroscope (e.g.,
3-axis gyroscope), a compass (e.g., 3-axis compass), and a
barometer. Accordingly, the IMU 110 may be configured to measure
orientation of the camera CAM (and thereby the gimbal 140) in three
dimensions and/or velocity of displacement of the camera CAM on one
or more axes, and may further may be configured to provide one or
more measurement signals indicating the orientation, velocity of
the displacement, and/or direction of the displacement. In at least
one embodiment, the IMU 110 may provide measurement signals at a
rate of 160 Hz, though it will be appreciated by those having
ordinary skill in the art that any frequency may be used.
[0031] The controller 120 may be coupled to the IMU 110 and
configured to receive the measurement signals. Based, at least in
part, on the measurement signals, the controller 110 may provide
(e.g., generate) one or more control signals. The control signals
accordingly may be provided to the driver 130, which may in turn
provide the control signals to the motor 150 to control the
orientation of the gimbal, and thereby, the orientation of the
camera CAM that is attached to the gimbal 140.
[0032] As will be explained, the control signals provided by the
controller 120 may be used to adjust an orientation of the camera
CAM by causing the gimbal 140 to adjust a pointing angle of the
camera CAM. The controller 120 may, for instance, be configured to
adjust the pointing angle of the camera CAM based, at least in
part, on one or more measurement signals provided by the IMU 110.
For example, the pointing angle of the camera CAM may be measured
by the IMU 110 and the controller 120 may be configured to compare
the pointing angle (e.g., attitude) of the camera CAM with a
desired pointing angle (as indicated by a command attitude). The
controller 120 may determine the difference between the pointing
angle of the camera CAM and the desired pointing angle, and provide
control signals through the driver 130 to the gimbal 140 (e.g., the
motor 150) to adjust the orientation of the gimbal, and as a
result, to adjust the pointing angle of the camera CAM. By way of
example, the controller 120 may provide control signals to match
the pointing angle of the camera CAM to the desired pointing angle.
In another example, the velocity of displacement may determine how
quickly the pointing angle of the camera CAM may be adjusted.
[0033] In some examples, the controller 120 may be configured to
operate in a closed-loop control circuit to iteratively determine a
new pointing angle for the camera CAM using a current pointing
angle of the camera CAM. The controller 120 may, for example,
employ a proportional-integral-differential (PID) control loop, and
may update the current pointing angle at a fixed frequency, such as
400 Hz, although other frequencies may be used. In this manner, the
pointing angle of the camera CAM may be repeatedly updated (e.g.,
by repeatedly updating the orientation of the gimbal 140) using one
or more measurement signals provided by the IMU 110 in accordance
with examples described herein. As will be described in further
detail below, the pointing angle of the camera CAM may be adjusted
to compensate for vertical and/or horizontal displacement of the
camera CAM, to add perceived displacement of the camera CAM, to
selectively align a pointing angle to one or more displacement
angles of the gimbal 140 and/or to allow a pointing angle to be
manually determined.
[0034] In an example operation of the apparatus 100, the gimbal 140
(and thereby the camera CAM) may be displaced in one or more
directions. The IMU 110 may sense the displacement and provide
measurement signals to the controller 110 indicating the same.
Based, at least in part on, on the measurement signals, the
controller 120 may provide control signals to the gimbal (e.g., the
motor 150) to adjust a pointing angle of the camera CAM to
compensate for the displacement of the camera CAM.
[0035] In some instances, a camera operator may use the camera CAM
to film a particular subject (e.g., actor) and it may be desirable
to keep the camera CAM directed toward the subject despite
displacement of the camera CAM. Accordingly, the controller 120 may
be configured to adjust the orientation of the gimbal 140 in order
to adjust the pointing angle of the camera CAM such that the camera
CAM stays directed at the subject.
[0036] In at least one embodiment, adjusting the orientation of the
gimbal 140 in this manner may include determining a pointing
angular rate correction value. For example, in instances in which
the pointing angle is nominally horizontal, the angular rate for
adjusting the pointing angle may be modeled by the following
equation:
.omega. = - V X ##EQU00001##
[0037] Where .omega. represents an angular rate, V represents a
velocity of displacement, and X represents the horizontal distance
between the subject and the camera CAM. In at least one embodiment,
X may be determined using a range sensor (not shown in FIG. 1),
such as an ultrasonic, laser, or infrared sensor. The range sensor
may, for instance, be located on the front of the camera CAM and/or
aimed at the center of field of view of the camera CAM. The range
sensor may in some embodiments be mounted on the gimbal 140. The
range sensor may be configured to provide a signal (e.g., digital
or analog) to at least one of the IMU 110 or the controller 120
indicating the distance X between the camera CAM and a subject.
[0038] As can be seen, the angular rate is assigned a negative
value of the velocity V divided by the distance X. This may ensure,
for instance, that the pointing angle of the camera CAM is adjusted
in a direction opposite the displacement of the camera CAM. In at
least one embodiment, angular rates may be determined in both
horizontal and vertical directions.
[0039] In instances in which the pointing angle is not nominally
horizontal, the controller 110 may be configured to determine the
pointing angle using a different approach. For example, the
controller 110 may be configured to determine a tangential
translation velocity in accordance with the following equation:
Vtangential=cos .theta.*V
where Vtangential represents the tangential translation velocity,
.theta. represents the current pointing angle of the camera CAM,
and V represents the velocity of displacement. Once the tangential
translation velocity has been determined, the angular rate may be
determined with the previously described formula using the
tangential translation velocity in lieu of the velocity of the
displacement in accordance with the following equation:
.omega. = - Vtangential X ##EQU00002##
It will be appreciated by those having skill in the art that for
horizontal displacement of the camera CAM, Vtangential may be equal
to V.
[0040] As described, the controller 110 may update the orientation
of the gimbal 140, and thus the pointing angle of the camera CAM,
at a particular rate (e.g., 400 Hz) and may update the pointing
angle of the camera CAM according to a calculated angular velocity.
In at least one embodiment, the controller 110 may update the
pointing angle in accordance with the following equation:
.theta..sub.t+dt=.theta..sub.t+.omega..sub.t
where .theta..sub.t+dt represents an updated pointing angle,
.theta..sub.t represents the current pointing angle, and .sub.ffit
represents the current angular rate. By way of example, the
controller 110 may update the pointing angle of the camera CAM at a
rate of 400 Hz and may determine the angular rate to be
1.degree./s. The controller 120 may update the pointing angle of
the camera CAM using a rate 0.0025.degree. per cycle. Accordingly,
for a current angle of 25.degree., the updated angle may comprise
25.0025.degree..
[0041] In some cases, the velocity of displacement of the camera
CAM may not be linear. Thus, in some embodiments, the controller
110 may be configured to determine the pointing angle using
trapezoidal integration in accordance with the following
equation:
.theta. t + dt = .theta. t + ( .omega. t + .omega. t + dt ) 2
##EQU00003##
where .theta..sub.t+dt represents an updated pointing angle,
.theta..sub.t represents the current pointing angle, wt represents
the current angular rate and .omega..sub.t+dt represents the
previous angular rate.
[0042] In some instances, measurements by the IMU 110 may vary
slightly due to drift and/or noise. This may cause the controller
120 to improperly update the pointing angle of the camera CAM.
Thus, in at least one embodiment, only displacements above one or
more thresholds, such as a particular displacement threshold and/or
a velocity threshold, may cause the controller 110 to update the
pointing angle of the camera CAM. By way of example, the controller
110 may be configured to update the pointing angle only when
displacement of the camera CAM exceeds a rate of 5 cm/s.
[0043] While distances have been described herein as being
determined by a range sensor, m some embodiments, distances may be
determined in other manners as well. For example, distances may be
predetermined and/or stored in the controller 110. In another
example, distances may be provided to the controller 120, for
instance, via wired or wireless communications. Moreover, in at
least one embodiment, distance may be determined based, at least in
part, on lens focus adjustment performed by the camera CAM. For
example, the camera CAM may be configured to focus on a subject,
and focusing in this manner may allow one or more components of the
apparatus 100, such as the controller 120 to determine a distance
between the camera CAM and the subject. In yet another example,
horizontal displacement may be detected by the IMU 110, for
instance, based on GPS measurements and/or compass measurements,
and based, at least in part, on the displacement, a distance may be
determined and the pointing angle adjusted accordingly.
[0044] In some embodiments, a plurality of range sensors (e.g., 2
range sensors) may be used. While a first range sensor may be used
to determine distance between the camera CAM and a subject as
previously discussed, one or more additional range sensors may be
used to determine a height of the camera CAM relative to one or
more surfaces (e.g., a floor). Additionally or alternatively, the
IMU 110 may determine height, for instance, using a barometric
measurement. Once a height has been determined, a pointing angle of
the camera CAM may be determined. This may for instance, be used in
addition to or in lieu of other approaches for determining an angle
described herein and may be determined in accordance with the
following equation:
.theta. = sin - 1 D H ##EQU00004##
where .theta. represents the pointing angle, D represents the
distance between the camera CAM and the subject, and H represents
the measured height of the camera CAM.
[0045] In some examples, height measurements may be volatile due to
obstacles, inclines, declines, and/or sensor variation.
Accordingly, the controller 110 may be configured to employ an
offset correction, for instance, each time a command attitude is
provided to the controller 110. In at least one embodiment, an
offset correction may be implemented in accordance with the
following equation:
OC=H-D*sin TC
where OC represents an offset compensation, H represents a measured
height, D represents a distance between the camera CAM and a
subject, and TC represents the command attitude. Once the offset
correction has been determined, the pointing angle may be
determined in accordance with the following equation:
.theta. = sin - 1 D H - OC ##EQU00005##
[0046] While examples have been described herein with respect to
determining a pointing angle m a vertical direction, it will be
appreciated that described approaches for determining a pointing
angle may be employed in determining a pointing angle in a
horizontal direction as well.
[0047] A camera operator may desire to carry or otherwise displace
the camera CAM while filming. As a result, acceleration forces and
rotational movements may be applied to the camera CAM and other
elements of the apparatus 100. By way of example, a camera operator
may be walking and/or running while filming a subject. In
accordance with examples described herein, the image recorded by
the camera CAM may be smoothed due to adjustments made to the
pointing angle of the camera CAM by the controller 110 to
compensate for displacement. In some instances, however, it may be
desirable to replicate a perspective, such as that of a person
running or driving a car, while filming. Thus, in some examples,
relatively small adjustments of the pointing angle may be made to
mimic movement of a desired perspective, that is, to provide
perceived displacement. By way of example, perceived displacement
may be modeled in accordance with the following equation:
Displacement=tan d.theta.*D
where Displacement represents perceived displacement, d.theta.
represents an angular displacement and D represents the distance
between the camera CAM and the subject. By varying d.theta.,
different perceived displacements may be achieved. Displacement may
be used to adjust the pointing angle in other manners as well. For
example, as will be described in further detail below, acceleration
may be used to adjust the pointing angle.
[0048] FIG. 2 is a schematic block diagram of an apparatus 200
according to an embodiment of the invention. The apparatus 200
includes elements that have been previously described with respect
to the apparatus 100 of FIG. 1. Those elements have been identified
in FIG. 2 using the same reference numbers used in FIG. 1 and
operation of the common elements is as previously described.
Consequently, a detailed description of the operation of these
elements will not be repeated in the interest of brevity.
[0049] The apparatus 200 may include a controller 122 that may be
used to implement the controller 120 of FIG. 1. The controller 122
may include a scalar 170, a summer 175, and a control logic 180.
The scaler 170 may be configured to receive a filtered signal, for
instance, from a filter 160, and a scale signal. The scaler 170 may
further be configured to multiply the filtered signal by a factor
indicated by the scale signal to provide a multiplied signal. The
summer 175 may receive the multiplied signal and combine the
multiplied signal with a received command attitude. The summed
signal may be provided to the control logic 180, which may in turn
adjust the orientation of the gimbal 140 (and therefore the
pointing angle of the camera CAM) as described herein.
[0050] The filter 160 may be configured to receive at least one of
the supplemental adjustment signal and the measurement signal from
the multiplexer 190 and filter the received signals based, at least
in part, on a bandwidth signal. The bandwidth signal may, for
instance, indicate one or more frequencies by which the filter 160
should pass or block particular frequencies for the received
signals. For example, based, at least in part, on the bandwidth
signal, the filter 160 may operate as a high pass filter, a low
pass filter, a band pass filter, a band block filter, or a
combination thereof. In one embodiment and as will be explained in
further detail, based, at least in part, on the bandwidth signal,
the filter 160 may operate as a high pass filter such that elements
of the measurement signals corresponding to acceleration due to
gravity are filtered. While the filter 160 is shown independently
of the controller 122, it will be appreciated that in some
embodiments, the filter 160 may be included in the controller
122.
[0051] As described, the IMU 110 may include an accelerometer and
accordingly may be configured to provide a measurement signal
indicative of acceleration (e.g., vertical and/or horizontal
acceleration). In one embodiment, the IMU 110 may be configured to
measure acceleration during displacement of the camera CAM, and
provide an associated measurement signal that may be used to adjust
the pointing angle of the camera CAM. However, because acceleration
measured by the IMU 110 includes acceleration due to gravity, the
measurement signal may include acceleration other than transitory
acceleration applied to the camera CAM as a result of displacement.
Accordingly, the measurement signal may be provided to the filter
160, and the filter 160 may remove all acceleration except
transitory acceleration, for instance, by operating as a high pass
filter, and as described, the manner in which the measurement
signal is filtered by the filter 160 may be based, at least in
part, on the bandwidth signal. Subsequently, the filtered signal
may be scaled by the scaler 170, combined with a command attitude
at the summer 175, and provided to the control logic 180 for
adjustment of the gimbal orientation (and pointing angle). In at
least one embodiment, adjustments to a pointing angle of the camera
CAM may be done in a direction opposite a direction of
acceleration. By way of example, upward accelerations may cause a
downward tilt of the camera CAM.
[0052] In other embodiments, other signals may be used to adjust
the pointing angle. For example, as illustrated in FIG. 2,
supplemental adjustment signals may include, but are not limited
to, pre-recorded acceleration signals (e.g., acceleration of an
earthquake), artificially generated acceleration signals, for
instance, generated by an electronic device, acceleration signals
provided from a remotely mounted IMU (e.g., mounted to an animal or
to a remote chassis when a camera is mounted to a boom), or
displacement angle signals. It will be appreciated that these
acceleration signals are intended as examples, and any number
and/or type of acceleration signals may be provided to the
apparatus 200.
[0053] As shown, a number of supplemental adjustment signals may be
received and one or more of the signals may be selectively provided
to the filter 160. It will be appreciated, however, that in some
implementations, a plurality of acceleration signals may be
provided to the filter 160 simultaneously and/or sequentially such
that the pointing angle of the camera 140 may be adjusted in any
desired manner using any number of signals.
[0054] FIG. 3 is a schematic diagram of an apparatus 300 according
to an embodiment of the invention. The apparatus 300 includes
elements that have been previously described with respect to the
apparatus 100 of FIG. 1. Those elements have been identified in
FIG. 3 using the same reference numbers used in FIG. 1 and
operation of the common elements is as previously described.
Consequently, a detailed description of the operation of these
elements will not be repeated in the interest of brevity.
[0055] The apparatus 300 may include a controller 124 that may be
used to implement the controller 120 of FIG. 1. The controller 124
may receive a command attitude that may be used as a reference
angle as described herein. The apparatus 300 may further include an
LED 199 and a handle 155 that may be coupled to the gimbal 140.
[0056] As described, the motor 150 may be configured to determine a
displacement angle for each gimbal axis and further may provide a
signal indicating the same. In at least one embodiment, the handle
155 may comprise part of a frame to which the gimbal 140 is
attached. Accordingly, a camera operator may carry the gimbal 140
and steer the gimbal 140 to direct the camera CAM at a subject.
Because the handle comprises part of the frame to which the gimbal
140 is attached, in at least one embodiment, by determining the
displacement angle of the gimbal 140, the motor 150 may determine
an angle at which the handle has been rotated relative to the
camera CAM. In this manner, the controller 124 may be configured to
adjust the pointing angle of the camera CAM based, at least in
part, on an angle of the handle.
[0057] In operation, a camera operator may steer the gimbal 140 to
point the camera CAM in an intended direction. As described, in
steering the gimbal 140, the handle 155 may be rotated, and the
rotation may be detected by the controller 124 based, at least in
part, on displacement angle signals. If the displacement angle, or
angle indicated by the displacement angle signals, is different
than the reference angle, the controller 124 may adjust the
pointing angle of the camera CAM to match the angles. Once matched,
the updated pointing angle may be used as the reference angle. In
at least one embodiment, the controller 124 may be configured to
adjust the pointing angle at a rate based, at least in part, on the
magnitude of the difference between the angles. For example, the
further the pointing angle from the displacement angle, the more
quickly the pointing angle of the camera CAM may be adjusted. In
some instances, the controller 124 may further be configured to
wait a delay before adjusting the pointing angle. This may, for
instance, provide smoother operation of the camera CAM.
[0058] Moreover, the controller 124 may be configured to adjust the
pointing angle only when the difference between the reference angle
and the displacement angle exceeds a threshold. For a difference
not exceeding the threshold, the pointing angle may not be
adjusted. For a difference exceeding the threshold, the updated
pointing angle may be determined using the following pseudo
code:
TABLE-US-00001 if (angle_measured > angle_threshold) { angle_out
= angle_measured - angle_threshold } if (angle_measured <
angle_threshold) { angle_out = angle_measured + angle_threshold
}
where angle_measured represents the displacement angle,
angle_threshold represents the threshold angle, and angle_out
represents an updated pointing angle of the camera 140.
[0059] In at least one embodiment, during instances in which the
controller 124 determines that the angle does not exceed the
threshold, the controller 124 may enable the LED 199 to indicate
that the pointing angle of the camera CAM is in an "aligned" state
such that no adjustments are being made. Alternatively, in other
embodiments, the controller 124 may be configured to enable the LED
199 when the controller 124 is adjusting the pointing angle of the
camera CAM. The controller 124 may further be configured to receive
a joint lock signal. Responsive, at least in part, to assertion of
the joint lock signal, the controller 124 may lock the current
displacement angles, thereby assigning each displacement angle as a
respective reference angle, as described above. In this manner, an
operator may freely direct the camera without adjustment of the
pointing angle until the joint lock signal is no longer asserted.
Because the joint lock signal may be asserted using a switch, an
operator may selectively assert the joint lock signal to
selectively enable a "locked" state for the pointing angle of the
camera CAM. For example, a trigger switch positioned on the handle
may be pulled to enable the locked state and released to disable
the locked state. In some embodiments, the locked state may be
enabled by releasing the trigger and disabled by pulling the
trigger.
[0060] In some instances, a camera operator may desire to manually
adjust a pointing angle of a camera, for example, to direct a
camera toward a subject. In doing so, the camera operator may apply
a force to the camera CAM and/or the gimbal 140. Accordingly, the
external force may be measured, in direction and/or in magnitude,
and based, at least in part, on the measured force, a controller,
such as the controller 124 of FIG. 3 may allow the gimbal 150 to
adjust such that the camera CAM is adjusted as desired.
[0061] By way of example, the camera CAM may be maintained at a
first pointing angle, and a camera operator may wish to adjust the
pointing angle. In manually adjusting the camera CAM and/or the
gimbal 140, the camera operator may apply a force (e.g., torque)
that may be measured, for instance, using control signals provided
by a controller to control the gimbal 150. The control signals may,
for instance, be indicative of a reaction torque resulting from the
applied force. Additionally or alternatively, the force may be
measured by a pressure sensor that may be configured to provide a
signal to the controller 124 indicating a direction and/or
magnitude of an applied force. In response to the applied force,
the controller 124 may allow the camera CAM and/or the gimbal 140
to be adjusted manually. The controller 124 may, for instance,
cease any stabilization of the gimbal 140 while the force is
applied. When the force is no longer applied, the controller 124
may cease allowing the manual adjustment and begin to stabilize the
camera CAM at the new pointing angle in accordance with examples
described herein, e.g., new displacement angle may be used as a
reference angle. In this manner, an operator may adjust a pointing
angle any number of times and in each case, the controller 124 may
adjust the gimbal such that the camera CAM remains directed in
desired direction.
[0062] In some examples, the controller 124 may be configured to
allow the camera CAM and/or the gimbal 140 to be adjusted manually
only when an applied force exceeds a particular threshold. If a
force does not exceed a threshold, the controller 124 may stabilize
the gimbal 140 to compensate for the force as described herein. In
at least one embodiment, the threshold may be set by an
operator.
[0063] In some examples, the controller 124 may be configured to
operate as if the camera has inertia. That is, the controller 124
may decreasingly resist an adjustment of the pointing angle when an
operator initially performs a manual adjustment and to increasingly
resist an adjustment of the pointing angle when an operator is
completing the manual adjustment. The controller may, for instance,
resist adjustments in accordance with any differential equation
known in the art, including but not limited to one or more decay
equations (e.g., equations directed to viscosity). Dampening
changes in the rate of change of the pointing angle of the camera
CAM may provide a smoother change in pointing angle.
[0064] In some examples, the camera CAM may be adjusted using
approaches that do not include a gimbal. Instead, the camera CAM
may be adjusted using, for example, lifts, winches, balloon
systems, telescopic mast systems, booms, long rope systems,
pulleys, or any other type of support structure that may support a
camera.
[0065] FIG. 4 is a schematic block diagram of an apparatus 400
according to an embodiment of the present invention. The apparatus
400 includes elements that have been previously described with
respect to the apparatus 100 of FIG. 1. Those elements have been
identified in FIG. 4 using the same reference numbers used in FIG.
1 and operation of the common elements is as previously described.
Consequently, a detailed description of the operation of these
elements will not be repeated in the interest of brevity.
[0066] The apparatus 400 may include a controller 126 that may be
used to implement the controller 120 of FIG. 1, and may further
include a winch 202 and a motor 250. The winch 250 may be coupled
to the camera CAM and the motor 250 may be configured to adjust the
height of the camera CAM using the winch 202. In at least one
embodiment, the motor 250 may comprise the motor 150 of FIG. 1, or
may comprise a different motor.
[0067] The controller 126 may be configured to receive measurement
signals from the IMU 110 indicating a height of the camera CAM, a
velocity of displacement, acceleration of displacement, and/or
direction of displacement. In some examples, the height may be
determined in other manners as described herein. The controller 126
may further receive a commanded height, for instance, from a user,
and may compare the commanded height to the height of the camera
CAM. Once any difference between the commanded height and height of
the camera CAM is determined, the controller 126 may be configured
to provide control signals, using the driver 130, to the motor 250
to control the winch 202 such that the height of the camera CAM is
adjusted to the commanded height.
[0068] In some instances, a camera may be displaced over uneven
ground. To compensate, the controller 126 may further be configured
to receive a rate control signal indicating the rate at which the
height of the camera CAM may be adjusted. By way of example, a user
may use a remote control device, such as a joystick, to increase or
decrease the height of the camera CAM, for instance, by pushing or
pulling the joystick, respectively. In some instances, the
controller 126 may be configured to operate in a "vario mode"
wherein the controller may adjust the height of the camera CAM only
in response to receipt of a rate control signal. When operating in
the vario mode, the controller may measure displacement of the
camera CAM.
[0069] In some examples, the controller 126 may be configured to
receive one or more control signals (not shown in FIG. 4) from the
winch 202. For example, the controller 126 may receive one or more
control signals indicating rotations of one or more pulleys on the
winch 202. The controller 126 may set a maximum number of rotations
that pulleys may make in a particular direction to prevent jamming
or grounding of the camera CAM. In another example, the controller
126 may receive one more signals from limit switches indicating
that the winch is at risk of jamming or grounding the camera CAM,
and the controller 126 may prevent further height adjustment
accordingly.
[0070] The winch 202 may further be configured to include a
counterbalance weight. This may, for instance, reduce power
consumption by reducing power required to maintain a particular
height of the camera CAM. Additionally or alternatively, the winch
202 may include a gas spring. Both counterbalance weights and gas
springs used in this manner may be configured to maintain tension
in the cable of the winch.
[0071] While examples described herein are directed to a winch, in
some instances, the height of the camera CAM may be adjusted using
other devices. For example, a telescopic pole may be used to adjust
the height of the camera CAM, and may be hydraulic and/or
pneumatic.
[0072] As described with respect to FIG. 3, in some cases, a camera
operator may desire to adjust the pointing angle of the camera CAM.
Similarly, a camera operator may desire to adjust the height of the
camera CAM. Accordingly, the controller 126 may be configured to
measure a force applied to the camera CAM, allowing the height of
the camera to be adjusted, and maintain the camera CAM at the
adjusted height. As described, adjustment of the camera CAM in this
manner may include a threshold for the applied force and/or may be
dampened.
[0073] Examples have been described herein with respect to
adjustment of a pointing angle of a camera. It will be appreciated
that while several features have been described separately, that
some or all examples described herein may be implemented
simultaneously and/or may be selectively enabled such that any
combination of features may be used. Moreover, in many examples,
adjustment of a pointing angle is described with respect to a
single axis, however, it will be appreciated that examples
described herein may be applied to a plurality of axes such that a
pointing angle of a camera may be adjusted in any desirable
manner.
[0074] Moreover, particular embodiments of described examples may
further be described in the following examples. The following
examples are provided merely to highlight various aspects of
described examples and are not intended to be limiting in any
manner.
Example A
[0075] As described in Example A, the camera pointing angle may be
automatically adjusted to compensate for translational movements,
which otherwise uncorrected would take the subject outside the
camera field of view. By measuring subject distance using a simple
range sensor and combining with a high performance IMU that
accurately reports 3D velocity measurements, it is possible to
correct pointing pan and tilt angles to compensate for movements.
The correction rates are geometrically related displacement
velocity and pointing angles.
[0076] Referring to FIG. 5, an actively stabilized camera gimbal
will keep a constant pointing angle by means of gyroscopic
feedback. For close-in shots, and in particular where the gimbal is
a hand-held device, the effects of translation are noticeable with
the subject matter moving out of the camera FOV (field of view).
This may be corrected by changing pointing angle; however, thus far
this is a manual control process performed by the camera operator
or a remote tele-operator.
[0077] For fast translation movements and close-in shooting, it is
an impossibility to achieve a perfect in frame shot using manual
methods. Typically, the filming subject will be an actor, where it
is desirable to keep them in frame. This approach allows for an
automatic compensation of pointing angle based on a feedback
control loop using a combination of position/speed and distance
sensors.
[0078] The crux of the approach in Example A is to either actively
measure or pre-set an approximate distance from the filming target,
and to use this in combination with translational measurements to
correct a camera pointing angle.
[0079] Active distance measurement can be achieved by ultrasonic,
laser or infra-red (IR) sensors and would typically work over
distances where translation effects are noticeable. An example of
an Ultrasonic sensor is MB1200 marketed by MaxBotix Inc. An example
of an IR sensor is GP2Y0A02YK0F marketed by Sharp Electronics. The
type of sensor and detection profile can be tailored for specific
applications. Ultrasonic sensors have a good range and adjustable
beam profile suitable for a person target.
[0080] Modern developments on IMU's (inertial measurement units)
mean it is now possible to actively measure position and velocity
with centimeter (cm) and cm/s accuracy. Such IMU's typically
incorporate GPS, 3 axis accelerometer, 3 axis gyroscopes, 3 axis
compass and a barometer. By complex and novel proprietary
algorithms, it is possible to fuse the sensor data in order to
derive accurate readings for 3D position and velocity. Such a
device has been marketed by Webb Consultancy Ltd. and typically
gives cm and cm/s resolution readings at 160 Hz update rate.
[0081] In order to correct for a translational velocity, a pointing
angular rate correction value can be calculated. For example, if
the pointing angle is nominally horizontal then the two parameters
are related by a standard equation similar to that describing a
rotating body around an origin:
.omega.=-.nu..sub.up/.chi.
[0082] Where .omega. is angular rate, .nu..sub.up is velocity and
.chi. is the horizontal distance of the camera from the subject
(see FIG. 6). So here upward movement requires a downward
correction in pointing angle. The same equation applies to
horizontal displacements where the displacement velocity is that
measured tangentially for the radius connecting the camera lens and
the subject, by knowing the actual camera pointing direction and
local compass heading in combination with GPS measurements.
[0083] A gimbal tilt drive control can include a brushless motor
and an IMU mounted on the camera body. Referring to FIG. 7, the
gimbal controller uses a closed loop electro-mechanical feedback
control system to update its pointing angle at a regular rate.
Typically the control system is a digital controller based on a
software program using techniques such as PID
(proportional-integral-differential) control. The digital
controller normally has a fixed update rate whereby discrete
control decisions are made--typically at 400 Hz, but may be slower
of faster dependent on the specific design. The IMU measures the
tilt angle and the controller compares this with the commanded
angle and then this is used to update a power driver section, and
in turn operate an actuator in order that camera remains stabilized
at the desired setting. Preferably the IMU also measures tilt rate,
and an inner control loops performs stabilization of this motion.
The actuator may be some form of motor I gearbox I belt reduction
drive; however, it is preferable to use a direct drive brushless DC
motor (BLDC) to increase actuation bandwidth and precision. A BLDC
driver outputs 3 phase currents to the motor and usually requires
an angle resolver to form a local control loop in order to control
those currents accurately dependent on motor phase angle.
[0084] Referring to FIG. 8, range sensor information is also passed
to the tilt compensation controller, either as a digital or
analogue signal, and is used to process pointing angle corrections.
As an example, if a correction command of 1.degree./s is required,
then the controller will be required to make small integral steps
in pointing angle of 1/400=0.0025.degree. every 400 Hz update cycle
leading to the illusion of continuous motion. This motion would
approximately correct 5 cm/s tangential movement at 3 m.
.theta..sub.t+dt=.theta..sub.t+.omega..sub.t
[0085] The correction command is likely to be variable in speed
with time. The simple integration method above (Simpson's method)
would not perform well except for a constant speed. It is
preferable to use a more accurate numerical integration technique
in order to reduce integrated errors in the pointing angle. One
such technique is so called trapezoidal integration. Here the
current angular rate is averaged with the previous angular rate,
and that value is used for the angle step update:
.theta..sub.t+dt=.theta..sub.t+(.omega..sub.t+.omega..sub.t+dt)/2
where .theta. is the pointing angle.
[0086] Measurement of velocity using an IMU is not an exacting
capability; the IMU is subject to some drift and noise which could
cause a low random walk of pointing angle when nominally no
pointing direction change is required. Macroscopic changes in
velocity are very easy to detect. It is preferable to set some
threshold whereby only velocities above a certain level contribute
to a change in pointing angle. A typical threshold of 5 cm/s has
been found to work well for cinematography applications. It would
fall back to a manually operated pointing correction method for
displacement velocities slower than this threshold.
[0087] Further complication may be added where movement is not
normal to the pointing angle. For instance, if the camera is
already elevated with respect to the subject it would be in a
look-downward mode. Translation pointing correction may require
less adjustment the further this elevation--tending towards zero
for almost above the subject. This effect may be factored into the
equation by registering the current pointing angle in addition to
subject horizontal distance (in practice the displacement is likely
to be no greater than the horizontal distance).
[0088] Referring to FIG. 9, a simple modification to the pointing
rate equation may be made by calculating the tangential translation
velocity for a certain pointing angle. Here the translational
velocity is related to viewing angle by:
.nu..sub.tangential=cos(.theta.).nu..sub.up
[0089] In this case the tangential velocity may be used in
conjunction with the subject line of sight distance in order to
work out the angular correction rate. As expected the
.omega.=-v.sub.up/x for a horizontally displaced subject or zero
for a vertically displaced subject.
[0090] It is preferable that pointing distance is measured from the
camera head to the subject for best effect. Thus, the measurement
sensor would be mounted on the camera head and be aimed at the
center of the camera field of view.
[0091] If no range measurement sensor is used then the distance can
be pre-input into the gimbal controller to give an approximate
effect. This could be useable on a rehearsed or planned shoot.
[0092] Further to this direct or pre-input method of ranging, an
inference of subject range may be derived from lens focus
adjustment. By calibrating the focus control with real distance, a
measure of range may be deduced. The actual camera focus may be
automatically controlled or manually adjusted.
[0093] By similar means, horizontal translations may be compensated
for non-tangential movements. Here the pointing angle would be
related by compass and GPS derived Eastward and Northward
velocities.
[0094] An extension to tilt compensation by measuring actual height
also presented. Here a second range sensor can be placed to
interrogate the vertical height or the IMU may report measured
barometric altitude as shown in FIG. 8. Provided this height is
reset with a reference of zero for horizontal pointing then the
tilt angle is directly given by:
.theta.=.alpha. sin(distance/height)
[0095] This solution may be preferred for some situations, but
relies on an accurate height measurement without baseline wander
(see FIG. 10, which illustrates direct inference of tilt angle from
height and distance). Additionally, the camera operator commanded
tilt has to be correlated with the unobstructed height measurement.
For instance, if the height measurement is inaccurate due to an
obstacle in the path of the height sensor, or sloping ground, then
this height error would corrupt the tilt compensation. In practice,
an offset correction would be required to be updated every time the
camera operator finished changing the tilt command. The offset
correction would be the difference in the measured height and the
calculated height from tilt angle and distance:
Offset_correction=height_measureddistance.sin(tilt command)
Now:
.theta.=.alpha.
sin(distance/(height_measured-offset_correction))
[0096] A similar process can be applied to pointing direction;
however, in practice conventional GPS position measurements
typically have an accuracy of .about.2.5 m CEP over a long time
period. For close in shooting, this would be perceived as a
constant pointing angle wander and not give the desired effect. It
is possible to employ greater GPS positioning accuracy using DGPS
techniques (differential GPS).
Example B
[0097] Example B describes a method that introduces a specifiable
camera shake into a camera stabilization control system to make for
more realistic cinematography using an actively controlled gimbal.
Measurement of lateral acceleration or joint angles, in conjunction
with a high-pass filter provide a means to introduce a camera tilt
or pan disturbance giving the illusion of a vibration or
translational event. Pre-recorded, synthesized or remote mounted
IMU may be used to provide the seed noise signal.
[0098] An actively stabilized camera gimbal will keep a constant
pointing angle by means of gyroscopic feedback. Used in the movie
industry this can provide a useful means to aid a camera operator
keeping the subject in shot for an action scene.
[0099] However, the stabilization can be too good and lead to a
non-realistic result for a watching audience. Without stabilization
a scene could be difficult to film and most likely lead to unusable
results.
[0100] Embodiments of this invention describe a method of
introducing realistic disturbance on camera pointing angle whilst
maintaining a stabilized pointing direction.
[0101] According to Example B, an active electro-mechanical gimbal
controller for camera stabilization is based on 3 servo motors in
the tilt, roll and pan axis' operating in conjunction with an IMU
(inertial measurement unit) and integral control system for closed
loop feedback. Disturbance of pointing angle is corrected by the
control loop to maintain a constant pointing angle. FIG. 7 is a
diagram of a gimbal tilt control with a drive motor and IMU mounted
on a camera. Here, a simple diagram is provided for the tilt axis
and this control is normally replicated across the 3 degrees of
angular freedom. Typically, a PID
(proportional-integral-differential) controller normally
implemented by digital means such as a software program, is used to
output a driver control signal based on the error between the IMU
and commanded tilt angle. Preferably, an inner PID loop is used to
control tilt rate and by comparing to IMU derived tilt rate for
greater response bandwidth--in this case the outer angle loop would
be proportional only. The power driver output preferably controls
phase currents to a connected direct drive BLDC motor (brush less
DC motor). Some other form of actuator may be used, such as a
normal DC motor, pulley system or gearbox; however, the use of a
direct drive BLDC allows greater control precision and faster
response times.
[0102] The IMU sub-system typically consists of a 3-axis
accelerometer, 3-axis gyroscopes, barometer, compass and GPS. Such
a device has been marketed by Webb Consultancy Ltd. and
incorporates proprietary sensor fusion algorithms to accurately
derive pointing angles, velocities and acceleration both in body
frame and reference frame with a high update rate--typically 160
Hz.
[0103] Referring to FIG. 11A, during a filming sequence the gimbal
may be carried by a camera operator or a vehicle and be subject to
acceleration forces and also rotational movements of the support
structure. For instance, the camera operator may be walking or
running whilst tracking the subject. These acceleration forces will
be registered by the IMU and output as a combination of up/down,
left/right and forwards/backwards accelerations, and pan/tilt/roll
rates, all of which are resolved.
[0104] The control system will seek to eliminate all pan/tilt/roll
rate fluctuation and maintain a perfect constant attitude--within
the limits of the system bandwidth leading to a perfect steady
image at the camera. The crux of the approach in Example B is a
means to apply a controlled corruption to this stabilization to
provide the illusion it was filmed in first-person-view, immersing
the audience into the scene with more realism, whilst maintaining
accurate stabilization to allow the filming to take place.
[0105] The illusion of real movement can be mimicked by altering
the camera pointing angle by a small amount, such as illustrated in
FIG. 11B. For angular changes, the perceived displacement is
approximately
displacement=tan(d.theta.).distance
[0106] Where d.theta. is the angular displacement and distance is
the measurement to the filming subject. By scaling the value of
d.theta., different perceived displacements may be achieved
irrespective of real filming distance and indeed will be related to
the zoom set-up on the lens system.
[0107] Actual camera displacement is a transitory event returning
to the original position, it is like an acceleration phenomena. By
measuring IMU body accelerations, an approximation camera operator
noise may be achieved. Furthermore, this acceleration may be
resolved into up/down and left/right.
[0108] Using this acceleration information, an addition can be made
to the pointing angle. For instance up/down accelerations are
arranged to provide transitory up/down tilt by some adjustable
scaling factor. Referring to FIG. 12, raw acceleration measurements
are resolved by the IMU into the actual FOV (field of view)
irrespective of tilt angle. Up/down would be on a line from bottom
to top of the camera. Due to the Earth's gravitational field, there
will be a constant acceleration registered that will be resolved
depended on tilt angle. Since transitory acceleration is the aspect
of interest, this constant offset needs to be subtracted by an
adjustable high pass filter. FIG. 13 s a graph illustrating
high-pass filtering of an acceleration measurement. An adjustable
scale factor is then used to multiply this acceleration measurement
and add it into the commanded set angle. An adjustable filter
bandwidth can further alter the strength and nature of the effect.
Thus, a tilt motion is instructed in sympathy with transitory
accelerations and they may be dialed in for consistent behavior. It
would be normal to adjust the scale factor such that upward
acceleration provided a downward tilt and vice-versa. It is
possible to invert this factor and provide upward tilt for upward
acceleration for a particular illusion as desired.
[0109] Lateral accelerations would be treated with the same scheme,
but this time controlling pan axis.
[0110] By linking the real measured acceleration to the image a
realistic result may be achieved at the point of filming. Other
solutions may be use pre-recorded accelerations for a planned
sequence (see FIG. 14). Indeed the data could be synthesized by a
computer program, perhaps to simulate an earthquake. Accelerations
measured in other parts of the camera system might be preferable.
For instance, a camera gimbal supported on a boom from a camera car
is unlikely to measure appreciable high frequency acceleration
noise. As also illustrated in FIG. 14, by mounting a remote chassis
based accelerometer more realistic measurements may be made.
[0111] Camera shake can also be emulated by monitoring gimbal joint
angles. This would be akin to measuring rotational movements if the
camera was hard-mounted or held by an operator. Slight modification
to the show how this may be achieved using the BLDC resolver shown
in FIG. 14. By high pass filtering the measured joint angles a
symmetrical jitter or shake signal is derived. The signal may be
introduced at a controlled level in a similar way to the
acceleration derived shake.
Example C
[0112] In Example C, camera pointing angles may be adjusted by
moving the gimbal base in a pan, tilt or roll motion. By measuring
the angle of the gimbal base in relation to the camera, it is
possible to allow the user to control gimbal pan, tilt, and slew
rates. Examples of the invention allow seamless pointing and
translation control of a camera as a single operator.
[0113] An actively stabilized camera gimbal typically requires a
remote operator to control the pan and tilt slew rates. With a
handheld gimbal this requires two operators to translate and point
the gimbal simultaneously as illustrated in FIG. 15.
[0114] Successful filming requires careful collaboration between
the parties controlling the translation route and pointing plan of
the camera. There is a complexity of multiple radio transmitters
together with extra equipment and resources.
[0115] Examples of the invention allow a single operator
translating the camera to also intuitively control the camera
pointing angle without sacrificing the benefits of active
stabilization.
[0116] Stabilized gimbals typically require human operators to
control the pointing angle via a remote monitor and joystick.
Additionally, because of the lack of lightweight, handheld, and
easily moveable gimbals, it was impossible for the person operating
the gimbal to also carry the gimbal. With the advent of small,
lightweight, handheld gimbals, it is now possible for a human to
carry a stabilized gimbal. Accordingly, a way to intuitively
control the camera angles for the person carrying the gimbal is
desirable.
[0117] The crux of the approach in Example C is a method of using
the gimbal stabilization controller to also steer the camera
pointing direction by the operator using support base rotation and
tilting.
[0118] An active gimbal stabilization controller is detailed in
FIG. 7. An IMU attached to the camera senses camera tilt and a
digital controller based on PID techniques
(proportional-integral-differential) compares camera attitude with
desired attitude. Attitude error is amplified and fed to a BLDC
(brushless DC motor) via a power driver stage to form a closed loop
feedback control circuit. Preferably a direct drive BLDC is used
for increased control bandwidth and resolution however other
solutions may be used such as geared or belt reduction DC drives.
The digital controller typically runs at a fixed update rate of
some 400 Hz. The same set-up is repeated for pan and roll axis' and
a suitable mechanical arrangement.
[0119] The scheme as drawn in FIG. 7 requires a command attitude to
be fed into the controller as the "set-point". Conventionally this
may be via the second operator and a remote link or possible a
small thumb joystick that the camera operator may have access to.
Both are not an ideal solution and can compromise gimbal
maneuvering.
[0120] The subject of the approach of Example C uses a measurement
of the actual gimbal joint angles to provide a control signal
indicating the camera operators pointing intentions. The handheld
gimbal has a support base formed of lifting and steering handles
which the operator will always be holding. In order to control the
steerage, the support handles are rotated in the intended direction
and the gimbal will track this motion. For an example, in a
nominally horizontal stance the gimbal remains horizontal. If the
operator tilts the gimbal handles forwards then the gimbal will
start to tilt downwards at a rate proportional to the estimated
handle angle. Returning the handles to the level will stop further
movement, and indeed bringing them above the horizontal will cause
an upwards movement in tilt. In this way by tilting the handles one
way or another, the camera operator has direct control of the
pointing angle.
[0121] The same idea can be associated with the pan axis. The pan
process would stop once the gimbal has been steered to a point
where the handles are centralized. The simplest solution is to
set-up an outer control loop that effectively zeroes the joint
angle in both pan and tilt as shown in FIG. 16. In FIG. 16, the
desired tilt is down. When the camera angle is effectively the same
as the joint angle, the controller would be stable and converged.
The control loop would be arranged to be PI where the P
(proportional) control term allows for a stronger or faster
response for larger errors. The I (integral) term allows for a
time-constant which may be tuned to give a slow and fluid response
if a sufficiently large value is chosen.
[0122] For the operator there would be some issue that he would
exactly have to hold a zero joint angle to prevent the gimbal from
moving. This would be difficult to achieve in practice and there
would be small errors constantly being corrected. A solution to
this issue is to add some threshholding function to the joint angle
measurement. For angles within a certain limit the angle is
registered as zero, outside those angles the angle is registered
with this pseudo code:
if(angle_measured>angle_threshold)angle_out=angle_measured-angle
threshold
if(angle_measured<-angle_threshold)angle_out=angle_measured+angle
threshold
[0123] Effectively, a deadband is introduced that the camera
operator may not worry about accurate pointing within this region.
As soon as he exceeds it then slow movement would be achieved
proportional to the angle_out value.
[0124] To make this dead-banding more obvious, some form of visible
indicator such as an LED could be lit when inside the dead-band
giving a direct pointing lock indication.
[0125] A further optional enhancement to allow desirable pointing
control would be the use of a trigger button to lock the current
joint angles via a process of sampling the current joint angles.
The angle control loop then uses the sensed joint angle for
feedback rather than the attitude from the IMU. With this scheme,
the camera operator would be responsible for pointing direction
changes until the trigger was released. Thus, the operator may
easily switch between stabilized and non-stabilized pointing at the
press of a button.
Example D
[0126] Example D describes a method of active camera gimbal
steerage by sensing an externally applied torque to register a new
pointing direction. For external torques below some threshold the
gimbal reverts to a normal closed loop stabilization mode.
[0127] An actively stabilized camera gimbal typically requires a
remote operator to control the pan and tilt slew rates. With a
handheld gimbal this requires two operators to translate and point
the gimbal simultaneously as illustrated in FIG. 15.
[0128] Successful filming requires careful collaboration between
the parties controlling the translation route and pointing plan of
the camera. There is complexity of multiple radio transmitters
together with extra equipment and resources.
[0129] Examples of the invention allow a single operator
translating the camera to also directly control the camera pointing
angle without sacrificing the benefits of active stabilization. The
pointing control is intuitive and emulates a camera fixed to a
normal tripod.
[0130] Stabilized gimbals typically require human operators to
control the pointing angle via a remote monitor and joystick.
Additionally, because of the lack of lightweight, handheld, and
easily moveable gimbals, it was impossible for the person operating
the gimbal to also carry the gimbal. With the advent of small,
lightweight, handheld gimbals, it is now possible for a human to
carry a stabilized gimbal. Accordingly, a way to intuitively
control the camera angles for the person carrying the gimbal is
desirable.
[0131] The crux of the approach of Example D is a method of using
the gimbal stabilization controller to also allow an at will
steering capability of the camera pointing direction by the
operator applying an external disturbing force to the gimbal.
[0132] The force is in the direction of rotation required--for
instance down to tilt down and left to pan left. The force may be
applied to the gimbal structure or more naturally to the camera
lens body. If the disturbance force is removed, then the control
loop is designed to maintain the last pointing angle. The gimbal is
naturally balanced and there are nominally only low forces for the
stabilizer to react against.
[0133] A single axis active gimbal stabilization controller is
detailed in FIG. 7. An IMU attached to the camera senses camera
tilt and tilt rate. A digital controller based on PID techniques
(proportional-integral-differential) compares camera tilt with
desired tilt. The attitude error is amplified by a proportional
gain constant (Kp), termed a P loop, and fed into an inner angular
rate control loop. Here the IMU tilt rate is compared with the
output of the attitude P loop to give a control error. The control
error is amplified using proportional, integral and differential
constants (Kp, Ki & Kd) to form a PID loop. The output of this
loop is fed to a BLDC (brushless DC motor) via a power driver stage
to form an overall closed loop feedback control circuit.
Preferably, a direct drive BLDC is used for increased control
bandwidth and resolution; however, other solutions may be used such
as geared or belt reduction DC drives. The digital controller
typically runs at a fixed update rate of some 400 Hz. The same
set-up is repeated for pan and roll axis' using a suitable
mechanical arrangement.
[0134] Disturbances to the gimbal pointing direction are
automatically corrected using the above control loops. The scheme
as drawn in FIG. 7 requires a command attitude to be fed into the
controller as the "set-point". Conventionally, this may be via the
second operator and a remote link or possibly a small thumb
joystick that the camera operator may have access to. Both are not
an ideal solution and can compromise gimbal maneuvering.
[0135] For the approach of Example D, a method to register the
control force applied externally is used. One such method is to
measure the reaction torque or the control signal applied to the
driver motor. This torque is a direct result of the external force
and a threshold may be applied.
[0136] For forces below a threshold, the controller treats these as
normal stabilization requirements and simply keeps the control loop
in closed loop.
[0137] Forces above this threshold allow the gimbal to move in the
required direction.
[0138] If a force is applied to a control axis, there will be some
small displacement due to the natural control loop response. The
combination of registering the force above a certain level and this
displacement may be used to update a new holding angle. As soon as
the new holding angle is registered, the control loop will back off
and settle into the new position. Consequently, the torque will
also reduce and, unless the force is re-applied, motion will
stop.
[0139] In the stopped state, the gimbal will provide normal closed
loop attitude stabilization.
[0140] To continuously slew the gimbal, the operator has to
maintain a force in the required direction. Effectively, the
algorithm self-stops when this force falls below the threshold
point.
[0141] Diagrammatically the algorithm is illustrated in FIG. 17.
The sample and hold element register the current joint angle
whenever the output control signal exceeds a torque threshold. The
threshold measurement element also resets the integral accumulator
in the PID rate loop to effectively set the current control
position with a zero error. The net result is a smooth movement
with little discontinuity or pointing hysteresis. The setting of a
particular threshold value adjusts the amount of torque required
before movement is allowed. This may be tuned for operator
preference.
[0142] The algorithm may alternatively use a direct reading of
external force via some form of external force pressure sensor.
This would require more complexity.
[0143] The algorithm may be enhanced by mathematically treating the
external torque as a force acting on an inertia. The inertia would
be represented by an equation and allow some steady increase in
pointing direction like a heavy flywheel. On release of the torque,
the pointing direction would continue to change but slowly decay in
rate as if the flywheel was coming to rest. In this way, a smooth
pointed movement is possible, which is desirable for filming. The
decay equation can be represented by the physics of viscosity where
force is a function of velocity squared and could represent a real
oil damper. Thus, by the use of an electro-mechanical system and
some equations of motion, a large inertia camera and gimbal may be
emulated with a light weight system. The system could be tuned for
a particular response as required.
Example E
[0144] Example E describes a method of feedback control to
stabilize and control the height of a supported camera using an IMU
and PID control loops. Camera height is held constant when the
support structure may be moving up and down, for instance due to
terrain whilst a translation move is performed.
[0145] The solution of Example E eradicates vertical translation
errors and simplifies filming tasks for a camera operator.
[0146] Filming a subject often involves translation of a camera
from one position to another. It is desirable to perform such
translation with a stabilized gimbal in order to obtain good
results. It is further desirable to perform the translation at a
constant height without any vertical translation errors.
[0147] Often cameras are hand carried by operators or possibly
strung from a cable system. An example of a support system would be
"Easy-rig," illustrated in FIG. 18A, with which a camera operator
will find it difficult to not induce vertical translation
errors.
[0148] The subject of this Example E is to provide an active
control loop to remove all or some of the vertical translation
errors using an electromechanical system which may change the
camera suspension height as illustrated in FIG. 18B.
[0149] It is now possible to measure vertical position and velocity
in real time and to cm and cm/s accuracy. Such devices that report
this information are typically based on a sensor fusion of
accelerometer, gyroscopes and barometer, and may give update rates
typically at 160 Hz. One example has been marketed by Webb
Consultancy Ltd. It is an IMU (inertial measurement unit) intended
for autopilot control of VTOL flight systems and also has
integrated control capabilities.
[0150] By using this information, it is possible to construct a
digital control loop using PID (proportional, integral,
differential) techniques and to then feedback control errors to a
servo system that may correct the camera height and maintain a
constant value.
[0151] Referring to FIG. 19, the control preferably consists of
three input signals from the IMU-height, climb rate and climb
acceleration and a nested PID structure. An outer P loop registers
the actual height and compares to a commanded height to derive a
command climb rate. Errors in height will change the command
velocity +Ve thru -Ve. The next inner P loop compares real climb
rate with the commanded climb rate and produces and acceleration
command output. The inner-most PID loop does similar for the
measured acceleration and preferably includes differential and
integral terms to improve overall accuracy and speed. Deflection in
height, velocity or acceleration will produce a restoring control
signal at the control output.
[0152] The controller drives a winch system which preferably
consists of a sinusoidally commutated BLDC motor (brushless DC) and
using an angle resolver and driver circuit. Other systems may be
employed such as a DC motor, with or without gearbox, hydraulic or
compressed gas. The use of a BLDC motor provides convenience, low
torque ripple and high torque capabilities. The overall control
system has high bandwidth and resolution.
[0153] So far, the system has been described for a fixed height
command and the feedback ensures that effects of any support beam
translations are mitigated by the closed loop. In practice, there
would be a need to change the camera height such as when filming on
sloped ground. In this instance, the command height may be
manipulated using a `vario`-like control input. A joystick or thumb
wheel may be used to command a climb rate where central stick would
be stationary and +Ve or -Ve would be ascend or descend.
[0154] Further to this, it is possible to introduce a central
deadband on the control to force height hold in the approximately
central position. Alternatively, a joystick or knob may be sprung
loaded to force a return to the center and zero rate command.
[0155] It is possible for the height control command to be derived
from an assistant camera operator over a radio command link.
[0156] It is desirable to have extension limit clamps on the
control system to prevent grounding out or jamming. These may be
introduced by rotation counting or limit switches on the winch
cable.
[0157] As described, the weight of the camera system has to be
supported by the winch and, consequently, there is a constant
torque applied by the motor. This is wasteful of power and will
cause unnecessary motor heating. It is preferable to have some form
of counter-balance weight such as used in a passenger lift.
Alternatively, rather than a weight, a gas-spring may be used. Gas
springs have the advantage they may be designed to be used in both
compression and extension, and also the restoring force is
approximately constant versus displacement. Such a spring could be
matched to the camera weight in the same way a counter-weight would
be. The pressure in the spring may be manipulated to improve this
match and possibly settable by the camera operator to allow for
different camera equipment to be interchanged. By using a spring,
the control system is centralized and may provide both +Ve and -Ve
torques as required.
[0158] In this Example, it is required that the spring or
counter-balance weight is placed in a way to always keep the
support cable in tension.
[0159] It is possible to use other means to change the camera
height. For instance the support pole may be a telescoping design
with a geared rack and pinion drive mechanism. As previously
mentioned, it could also be hydraulic or pneumatic based.
[0160] A camera operator can find the work load of both pointing
and translating the camera to be quite high. The addition of the
command height vario control does not improve matters. It is
possible to introduce a control loop detection method that
recognizes they wish to change the height. By sensing the motor
torque command, if the operator tries to lift the camera up or pull
it down, there is knowledge they are wishing to perform this move.
The control may be crafted to allow this action by use of a
threshold detector and a sample and hold switch such as illustrated
in the diagram of FIG. 20. Here, once the force and resulting motor
control signal exceeds some threshold, a new height measurement is
captured and the PID integrator is reset. This allows a fluid-like
displacement of the camera and returns to a solid height hold
afterwards.
[0161] Alternatively, command climb rate may be forced to be zero
and operated in a purely vario mode. Here, the height may slowly
drift with time, especially if the weight is unbalanced; however,
the control system will still provide high bandwidth feedback
control for transient events, as shown in FIG. 21. The operator may
force the height to be changed and will feel a natural resistance
in doing so, but will not be prevented.
[0162] Thus-far, the Example E has been targeted at a single
operator carrying a camera on something like an Easy-rig support
system. The Example E doesn't preclude applications outside this
regime and may include: Crane systems, Balloon systems, Telescopic
mast systems, Extension boom systems that are commonly installed on
camera cars, Long rope systems for instance to descend into a deep
cave, or any other form of camera support structure.
[0163] From the foregoing it will be appreciated that, although
specific embodiments of the invention have been described herein
for purposes of illustration, various modifications may be made
without deviating from the spirit and scope of the invention.
Accordingly, the invention is not limited except as by the appended
claims.
* * * * *