U.S. patent application number 14/940909 was filed with the patent office on 2016-05-19 for system for improving safety in use of a machine of a kind comprising a body and an implement movable relative to the body.
This patent application is currently assigned to CATERPILLAR INC.. The applicant listed for this patent is Caterpillar Inc.. Invention is credited to Elie Abi-Karam, R. Neil Britten-Austin, Simon Conway, Sage Smith.
Application Number | 20160138249 14/940909 |
Document ID | / |
Family ID | 51900780 |
Filed Date | 2016-05-19 |
United States Patent
Application |
20160138249 |
Kind Code |
A1 |
Conway; Simon ; et
al. |
May 19, 2016 |
System for Improving Safety in Use of a Machine of a Kind
Comprising a Body and an Implement Movable Relative to the Body
Abstract
A system for improving safety in use of a machine of a kind
including a body and an implement movable relative to the body is
disclosed. The system may use sensed data to define a safety zone
around an implement to assist a user in avoiding an object of
interest. The system may provide functionality to change control of
the machine or implement in the event that an object of interest is
detected.
Inventors: |
Conway; Simon; (Leamington
Spa, GB) ; Britten-Austin; R. Neil; (Kirby Muxloe,
GB) ; Abi-Karam; Elie; (Narborough, GB) ;
Smith; Sage; (Apex, NC) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Caterpillar Inc. |
Peoria |
IL |
US |
|
|
Assignee: |
CATERPILLAR INC.
Peoria
IL
|
Family ID: |
51900780 |
Appl. No.: |
14/940909 |
Filed: |
November 13, 2015 |
Current U.S.
Class: |
701/50 |
Current CPC
Class: |
E02F 9/261 20130101;
E02F 9/264 20130101; G06T 11/00 20130101; E02F 9/2037 20130101;
G01S 13/867 20130101; E02F 9/24 20130101; E02F 9/2045 20130101;
G01S 13/88 20130101; E02F 3/96 20130101; E02F 9/265 20130101; E02F
9/2033 20130101 |
International
Class: |
E02F 9/26 20060101
E02F009/26; G01S 13/86 20060101 G01S013/86; G06T 11/00 20060101
G06T011/00 |
Foreign Application Data
Date |
Code |
Application Number |
Nov 14, 2014 |
EP |
14193345.7 |
Claims
1. A system for a machine of a kind comprising a body and an
implement, wherein the system is configured to define a safety zone
in the vicinity of the implement, the system comprising: a
processor configured to receive a plurality of system inputs and to
deliver a system output comprising criteria that define a position
and at least one dimension of the safety zone; and wherein the
plurality of system inputs comprises: (e) first system input data
relating to a type and/or a dimension of the implement; (f) second
system input data relating to a current position of the implement;
and one or both of: (g) third system input data relating to user
input control of a first type that governs ground propulsion of the
machine; and (h) fourth system input data relating to user input
control of a second type that governs movement of the implement
relative to the body and operation of the implement; wherein the
processor is configured to process the plurality of system inputs
in order to determine the system output.
2. The system of claim 1 further comprising at least one sensor
configured for obtaining at least some of the plurality of input
data.
3. The system of claim 2 wherein the at least one sensor includes
at least one camera.
4. The system of claim 3 wherein the at least one camera comprises
a camera located so as to provide an image of an implement that may
be connected to an implement coupling of the machine.
5. The system of claim 2 wherein at least one of the sensors is
located on an implement configured to be couplable to a
machine.
6. The system of claim 1 further comprising a data library
including data corresponding to values for each of the first,
second and third system input data and, for each combination of
input data values, a corresponding output data value, wherein the
step of processing the plurality of inputs in order to determine
the system output comprises searching the data library for an
output data value that corresponds with the first, second and third
system input data values.
7. The system of claim 1 wherein the processor is configured to
compute a function, the function having a plurality of function
inputs and at least one function output and wherein the processor
is configured to transfer the plurality of system inputs to the
function inputs and to deliver as the system output the at least
one function output.
8. The system of claim 1 and further comprising a display, wherein
the display is configured to receive and display the criteria that
define the position and at least one dimension of the safety
zone.
9. The system of claim 8 wherein the display is configured to show
the position and at least one dimension of the safety zone in the
context of a bird's eye view representation of the machine and
articles in the vicinity of the machine as determined from the data
received by the at least one system input.
10. The system of claim 9 wherein the at least one sensor includes
at least one sensor for detecting presence of at least one person
and wherein the processor is configured to represent a position of
the person thereby detected on the display.
11. The system of claim 10 wherein the at least one sensor for
detecting presence of at least one person includes an infra-red
sensor.
12. The system of claim 1 wherein the output represents guidance in
relation to a future trajectory of the safety zone, wherein the
future trajectory is calculated based on current system input
data.
13. The system of claim 3 wherein the first input data relating to
a type and/or a dimension of the implement are obtained by
configuring the processor to: obtain an image from one of the at
least one camera wherein said image includes an area of interest on
an implement connected to the machine, wherein the area of interest
is an area that is expected to show at least one alphanumeric
character; use an alphanumeric recognition technique, to recognise
the at least one alphanumeric character; search for the combination
of at least one alphanumeric character in a data library and
obtaining from a part of the data library associated with the
combination of at least one alphanumeric character data stored in
the data library in relation to the implement including at least
one of the following: implement type; implement dimension or
dimensions; degrees of freedom of implement or components of
implement; other data relating to the implement.
14. The system of claim 2 wherein the first input data relating to
a type and/or a dimension of the implement are obtained by
configuring the processor to: obtain information for identifying
the implement using at least one of the following technique: image
recognition; bar code recognition; QR code recognition; RFID
detection; search in a data library and obtain from a part of the
data library associated with the information obtained for data
stored in the data library in relation to the implement including
at least one of the following: implement type; implement dimension
or dimensions; degrees of freedom of implement or components of
implement; other data relating to the implement.
15. A machine comprising the system of claim 1.
16. The system of claim 2 further comprising a data library
including data corresponding to values for each of the first,
second and third system input data and, for each combination of
input data values, a corresponding output data value, wherein the
step of processing the plurality of inputs in order to determine
the system output comprises searching the data library for an
output data value that corresponds with the first, second and third
system input data values.
17. The system of claim 2 wherein the processor is configured to
compute a function, the function having a plurality of function
inputs and at least one function output and wherein the processor
is configured to transfer the plurality of system inputs to the
function inputs and to deliver as the system output the at least
one function output.
18. The system of claim 2 and further comprising a display, wherein
the display is configured to receive and display the criteria that
define the position and at least one dimension of the safety
zone.
19. The system of claim 18 wherein the display is configured to
show the position and at least one dimension of the safety zone in
the context of a bird's eye view representation of the machine and
articles in the vicinity of the machine as determined from the data
received by the at least one system input.
20. The system of claim 2 wherein the output represents guidance in
relation to a future trajectory of the safety zone, wherein the
future trajectory is calculated based on current system input data.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to European Patent
Application No. 14193345.7, filed Nov. 14, 2014, which is
incorporated by reference herein in its entirety for all
purposes.
TECHNICAL FIELD
[0002] The disclosure relates to the field of machines of a kind
comprising a body and an implement movable relative to the
body.
BACKGROUND
[0003] A user of a machine of the kind having a machine body and an
implement movable relative to the machine body can see directly
from only one perspective at any one time. As such, an implement
movable relative to the machine body may be visible to a user from
only one perspective, such as a rear or side of the implement
rather than, for example, from a front of the implement.
Accordingly, when precise control of the position of the implement
is necessary, a user may require additional information in order to
position the implement accurately, particularly in respect of a
part of the implement that is not directly visible to the user.
Such assistance may be provided, for example, by a camera or by a
colleague at a distance from the machine.
[0004] Even if a user has assistance of a colleague or from a
camera, the user still needs to be able to make judgements about a
future position of the implement in order to be able to adjust
their control of the ground propulsion of the machine and/or their
control of the position of the implement relative to the machine
body in order to ensure that the implement arrives at the desired
location relative, for example, to an article to be contacted by
the implement.
[0005] A user may, over time, develop sufficient experience and
familiarity to be able to infer a position of a part of an
implement that is not directly visible to them. With yet further
experience, a user may be able to make judgements regarding a
future position of the implement on the basis of various control
inputs and how to influence that future position by altering one or
more control inputs.
[0006] Against this background, there is provided a system for
improving safety in use of a machine of a kind comprising a body
and an implement movable relative to the body.
SUMMARY OF THE DISCLOSURE
[0007] A system for a machine of a kind comprising a body and an
implement, wherein the system is configured to define a safety zone
in the vicinity of the implement, the system comprising: [0008] a
processor configured to receive a plurality of system inputs and to
deliver a system output comprising criteria that define a position
and one or more dimensions of the safety zone; and [0009] wherein
the plurality of system inputs comprises: [0010] (a) first system
input data relating to a type and/or a dimension of the implement;
[0011] (b) second system input data relating to a current position
of the implement; and one or both of: [0012] (c) third system input
data relating to user input control of a first type that governs
ground propulsion of the machine; and [0013] (d) fourth system
input data relating to user input control of a second type that
governs movement of the implement relative to the body and
operation of the implement; [0014] wherein the processor is
configured to process the plurality of system inputs in order to
determine the system output.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Specific embodiments of the disclosure will now be
described, by way of example only, with reference to the
accompanying drawings in which:
[0016] FIG. 1 shows a schematic illustration of a machine having as
an implement a loader bucket, in which machine an embodiment of the
system of the present disclosure may be employed;
[0017] FIG. 2 shows a schematic illustration of a machine having as
an implement a fork attachment, in which machine an embodiment of
the system of the present disclosure may be employed;
[0018] FIG. 3 shows a schematic illustration of a machine,
specifically an excavator with a grapple attachment, in which an
embodiment of the system of the present disclosure may be
employed;
[0019] FIG. 4 shows a schematic illustration of a machine,
specifically an excavator with a grapple attachment, in which an
embodiment of the system of the present disclosure may be
employed;
[0020] FIG. 5 shows a schematic illustration of a machine,
specifically a track-type tractor having a front blade, in which an
embodiment of the system of the present disclosure may be
employed;
[0021] FIG. 6 shows a schematic view from above of the machine of
FIG. 5 showing the front blade in a first configuration;
[0022] FIG. 7 shows a schematic view from above of the machine of
FIG. 5 showing the front blade in a second configuration;
[0023] FIG. 8 shows a schematic view from above of the machine of
FIG. 5 showing the front blade in a third configuration;
[0024] FIG. 9 shows various implements, 9a to 9f, that may be
compatible with an embodiment of the system of the disclosure;
[0025] FIG. 10 shows a bird's eye schematic representation of a
view that may be presented to a user on a display on which are
superimposed various trajectories computed by a processor on the
basis of sensed inputs;
[0026] FIG. 11 shows a bird's eye schematic representation of a
view that may be presented to a user on a display on which is
superimposed an attachment safety zone;
[0027] FIG. 12 shows various implements, 12a to 12c, that may be
compatible with an embodiment of the system of the disclosure;
[0028] FIG. 13 shows two saw implements, 13a and 13b, that may be
compatible with an embodiment of the system of the disclosure;
[0029] FIG. 14 shows a skid steer loader having a brushcutter as
its implement;
[0030] FIG. 15 shows a skid steer loader having an auger as its
implement;
[0031] FIG. 16 shows a schematic representation of a view that may
be displayed to a user showing a trajectory of fork tips relative
to a surrounding environment;
[0032] FIG. 17 shows a schematic representation of a view that may
be displayed to a user showing information regarding a current
position of an implement;
[0033] FIG. 18 shows a view provided by a camera of an implement
including an alphanumeric indicator associated with the implement
for assisting in identification of the implement by an embodiment
of a system of the present disclosure;
[0034] FIG. 19 shows a view provided to a display from a camera
mounted on an implement, such as a grapple, having two opposing
jaws movable relative to one another; and
[0035] FIG. 20 shows a view of a machine showing examples of inputs
that may be used by a system in accordance with the disclosure.
DETAILED DESCRIPTION
[0036] In a first embodiment, the system of the present disclosure
may be employed in the context of a machine 100, a schematic
illustration of which is shown in FIG. 1. For ease of explanation,
some features of the machine 100 are not shown in the schematic
representation of FIG. 1.
[0037] The machine 100 may comprise a body 130 having a cab 140.
The machine 100 may also comprise, as one of many options for
implements, a loader bucket 120 at a front end of the machine that
is movable relative to the body 130. While the illustrated
embodiment shows a loader bucket 120, the implement may be
interchangeable with a number of alternative implements.
[0038] Movement of the machine 100 and of the implement 120 is
controllable by a user in the cab 140. A user may thereby control
both ground propulsion of the machine and movement of the implement
120 relative to the body 130.
[0039] Ground propulsion of the machine 100 may be governed by
devices and techniques that are well known in the art. In the case
of the machine 100 of FIG. 1, ground propulsion is effected by four
wheels. For example, movement of the machine forwards and backwards
may be through delivering power from an engine of the machine to
one or more of the four wheels of the machine via a gearbox. The
user may control such using a combination of devices located inside
the cab including accelerator pedal, brake pedal, clutch and gear
shift stick. Movement of the machine 100 left and right may be
governed by rotating the front wheels relative to a longitudinal
direction of the body of the machine whilst the machine is moving
forward or backwards. The user may control movement of the front
wheels left or right by moving a steering wheel located inside the
cab 140.
[0040] Movement of the implement 120 relative to the body 130 may,
for example, be actuated hydraulically and controlled by one or
more levers that may be located in the cab 140 such that a user can
operate them from the same position or a similar position as that
for operating the controls that govern ground propulsion. Depending
on the nature of the implement 120 and the mechanism of connection
to the body 130 of the machine 100, the implement 120 may be
controllable to move relative to the body 130 of the machine 100
with multiple degrees of freedom. The implement 120 of FIG. 1 may
be connected to the body 130 via a pair of arms 156, 158 that are
each pivotally connected at a proximal end thereof to the body of
the machine at a pair of pivots 152, 154. The pivots 152, 154 may
share a common axis. The implement 120 may be connected to a distal
end of each arm 156, 158 via a pair of further pivots (not shown in
FIG. 1).
[0041] In the FIG. 1 embodiment, the implement may be a loader
bucket 120. A height of the loader bucket 120 relative to the body
130 of the machine 100 (and hence, indirectly, relative to the
surrounding ground) may be governed by an angle of the pair or arms
156, 158 at the first pair of pivots 152, 154. An angle of the
loader bucket 120 may be governed both by (a) the angle of the pair
of arms 156, 158 at the first pair of pivots 152, 154 and (b) the
angle of the loader bucket 120 relative to the pair of arms at a
second pair of pivots (not shown). For the purposes of this
description, when discussing an angle of the pair of arms 156, 158,
unless otherwise stated, this refers to an angle relative to the
body 130. Also for the purposes of this description, the loader
bucket 120 may be described as horizontal when a bottom surface 126
of the loader bucket 120 is parallel to a plane defined by the
surrounding ground on which the wheels of the machine are situated.
The loader bucket 120 may be described as having a downward angle
when the bottom surface 126 of the loader bucket 120 is tipped
forwards relative to the machine 100 such that contents of the
loader bucket 120 may fall under gravity from a front opening of
the loader bucket 120. Conversely, the loader bucket 120 may be
described as having an upward angle when the bottom surface of the
loader bucket 120 is tipped rearwards relative to the machine 100
such that contents of the bucket may are prevented from falling
under gravity from the loader bucket 120.
[0042] It will be appreciated that, for many combinations of arm
angles and loader bucket angles, a front edge 122 of the loader
bucket 120 is not visible to a user sitting in the cab 140 of the
machine 100. For other combinations of arm angles and loader bucket
angles where a front edge 122 of the loader bucket 120 is visible
to a user sitting in the cab 140 of the machine 100, other features
of the loader bucket 120, such as a top edge of the loader bucket
120 may not be visible to the user. Furthermore, since for many
implements, including a loader bucket 120, there are further
degrees of freedom, including the possibility of changing an angle
between the bottom surface 126 of the bucket and a rear surface 128
of the bucket, there are further aspects of the implement and its
position that may not be visible to the user when at certain
angles.
[0043] The system may comprise one or more sensors, a processor and
a display visible to a user in the cab of the machine 100.
[0044] In the embodiment of FIG. 1, the sensors may comprise a
first camera (not shown) installed on the machine. The sensors may
comprise further cameras. The first camera may be configured to
obtain within its field of view an image of some or all of the
implement. (An example of such a view is given in FIG. 18.) The
processor may be equipped to run image processing software that may
be configured to recognise from the image a subset of the image
showing an alphanumeric characters 1890, or similar, displayed on
the implement 1820 in order to obtain, from a data library or
similar, data associated with an implement having that alphanumeric
characters 1890. For example, the data may include the type of
implement (e.g., bucket, forks, broom, blade or any other implement
known in the art) as well as its dimensions and other features of
the implement including, for example, degrees of freedom, potential
for opening and closing opposing jaws, or other movement of a part
of an implement relative to another part of the implement.
[0045] The data library may be any source of data, including a
source of data located within the processor or a source of data
located remotely from the processor and perhaps accessed wirelessly
over the internet or by other means.
[0046] The image processing software may be further configured to
detect a reference point or points of the implement 120 and a
reference point or points of the body 130 in order to determine a
position of an implement 120 reference point relative to a body 130
reference point.
[0047] Having determined a position of at least one reference point
on the implement 120 as well as details of the implement type and
size from the data library, these two sets of data may be used by
the image processing software to determine implement position data
in respect of a wider range of reference points, perhaps including
reference points that are not within the field of vision of the
camera.
[0048] The display (not shown in FIG. 1) may be any form of display
device. It may be, for example, a conventional emissive display
(such as, for example, an LCD, LED, OLED or any other display
device) mounted at a convenient place in the cab 140 of the machine
100. In an alternative, the display may comprise a projector device
configured to project onto, for example, a windscreen/windshield of
the cab 140. The display may be integrated with other instruments
in the cab 140, perhaps on manufacture of the machine 100, or may
be fitted subsequently and simply mounted at any convenient
location. Alternatively, the display may be a head-up display, an
optical head-mounted display or other wearable technology. Any
display that is viewable by a user may be suitable.
[0049] The display may be configured to display a live view of at
least a part of the implement in the context of its surroundings.
The live view may be that obtained by the first camera or another
camera. Alternatively, the live view may be schematic in nature and
may be generated from data derived from one or multiple cameras or
other data derived from one or more other sensors.
[0050] In some embodiments, the display may not provide either a
live view of an image or a schematic representation of a view.
Rather, it may simply superimpose information (such as guidance or
trajectory information) over a user's view. For example, in an
embodiment involving a head-up display, guidance or trajectory
information may be provided as an overlay of the actual view of the
environment as seen by the user.
[0051] It is possible that the live view may be from an angle
different to any and all of the cameras. For example, rather than
having a camera located at a distance above the machine in order to
obtain a direct bird's eye view, the live view may be a schematic
bird's eye representation of the machine and implement in its
surroundings assimilated from data obtained from a plurality of
cameras or sensors whose field of vision or field of sensing may
project outwardly from the machine 100. Such an assimilated
schematic bird's eye view may be particularly useful in providing
information to the user regarding how to position the machine
(e.g., how to control the ground propulsion of the machine)
relative to one or more articles in the surrounding environment,
possibly before moving the implement 120 relative to the body
130.
[0052] The image processing software may be configured to
superimpose onto the view a representation of one or more aspects
of the implement that may not be visible to a user of the machine
in the cab. For example, a front edge 122 of a loader bucket 120,
as in the embodiment of FIG. 1, may not be visible to a user in the
cab 140 of the machine 100 (see FIG. 18). The image processing
software may provide an indication of a current position of a front
edge of the implement 120 in the context of the implement and
machine, together with surrounding artefacts. This might be shown,
for example, in the bird's eye view.
[0053] The display may instead or in addition provide raw data
relating to a position of the implement rather than a view. For
example, the display may show a height of the front edge of the
bucket loader relative to the ground or relative to a reference
position on the body 130. It may show a tilt angle of the bucket
loader relative to a longitudinal horizontal direction and an angle
of the bucket loader relative to a transverse horizontal direction.
These might be displayed as one or more numerical values, perhaps
also with icons and perhaps also with a colour-coded representation
to signify appropriate (e.g., green) and inappropriate (e.g., red)
values. One example of a display showing such information is shown
in FIG. 17.
[0054] In addition to representing present machine and implement
position data, the system of the present disclosure may be used to
provide the user with predictive information regarding a future
position of the implement.
[0055] In one arrangement of such a predictive implementation, the
processor may predict a future position of the machine and the
implement on the basis of current sensor inputs. The processor may
predict a future position initially assuming current sensor inputs
are maintained at constant levels. Further functionality may react
to a change in sensor inputs to update a predicted future position
of the implement. The more input data that is provided, the more
variables there may be when predicting a future position.
[0056] FIG. 2 shows a schematic representation of a machine 200
known in the art as a backhoe loader. Such a machine 200 may
comprise a body 230, a cab 240, a first implement 220 at a front
end of the machine that is movable relative to the body 230 and a
second implement 270 at a back end of the machine 200 that is also
movable relative to the body 230. The machine 200 and its
implements 220, 270 may be operable by a user in the cab 240. A
user may thereby control both ground propulsion of the machine 200
and movement of the implements 220, 270 relative to the body
230.
[0057] The machine 200 may, as illustrated in FIG. 2, have as one
of many options for implements, a fork attachment 220 as its first
loading implement. While the illustrated embodiment shows a fork
attachment 220, the implement may be interchangeable with a number
of alternative implements.
[0058] The fork attachment 220 may itself include one or more
sensors 229. One of the one or more sensors 229 may be a camera
that may be configured to provide an image feed to the display
and/or from which data may be calculated in order to provide a user
with, for example, height information of the forks and/or width
information regarding the current distance between the forks. In
the case of an image feed, the user may therefore be able to view
the forks in the context of, for example, a pallet that the user
wishes to lift using the forks. Such functionality may be
particularly appropriate where the pallet is at a height
significantly above the user in the cab of the machine and where
otherwise a user may (a) require the assistance of another person
at a distance from the machine that allows that person to see the
pallet and forks together or (b) need to leave the cab of the
machine in order to view the position of the forks relative to the
pallet from an alternative perspective.
[0059] Another feature applicable to all embodiments but explained
in detail with respect to the FIG. 2 embodiment is implement
trajectory mapping. As explained above, an experienced user may be
able to make judgements regarding a future position of the
implement 220 on the basis of various control inputs and how to
influence that future position by altering one or more control
inputs. The system of the present disclosure may be able to
anticipate future positions on the basis of current inputs so as to
allow users without sufficient experience to make such judgements
to enjoy efficiencies.
[0060] In some embodiments, one of the one or more sensors may be a
camera that may be mounted on the body 230 of the machine 200 that
may provide an image feed via a processor to a display. A schematic
representation of such an image feed may be found in FIG. 16. The
image feed 1600 may show the forks 1610, 1620, or other implement,
in the context of a wider angle view, showing articles in the
environment surrounding the forks, perhaps including articles 1690
in the environment some distance ahead of the forks. Data relating
to a steering angle of the machine may be used by the processor to
calculate a trajectory of the tips in the event that the steering
angle remains unchanged. Such a trajectory 1630, 1640 may be
superimposed (see, for example, the dotted lines in FIG. 16) on the
displayed image provided by the camera (or a schematic version
thereof) in order to illustrate where the tips of the forks would
be located at a point in the future assuming that the steering
angle input remains unchanged. In the event that the steering angle
changes, the processor may update the trajectory prediction on the
basis of current steering angle and display the updated trajectory
in near to real time.
[0061] In some embodiments, the trajectory may be two-dimensional
while in other embodiments the trajectory may be
three-dimensional.
[0062] The data relating to a steering angle of the machine may be
provided by a position sensor on a steering rack that controls
angles of the wheels to determine an angle of the steering relative
to a longitudinal direction of the machine. Alternative techniques
for sensing wheel angle may also be appropriate.
[0063] In a further variation, sensor readings indicative of
changes in height in the forks as controlled by the user may also
be provided to the processor such that trajectory of fork position
may include also an indication of a future height. In this manner,
future height may be calculated and superimposed on the image
provided by the camera. Again, changes in the sensor reading
indicative of implement height control may be fed into the
processor and the trajectory may be updated in near real time to
take account of such changes.
[0064] In this way, an inexperienced user may be provided with
sufficient information to be able to change the steering control
and the implement height control simultaneously in order to arrive
at a trajectory that meets with an article of interest. In the case
of forks, a user may be in a position to change steering angle and
fork height simultaneously so that the forks arrive at an
appropriate position to pick up a desired pallet. The machine
position and fork height may be controlled by a user on the basis
of the feedback provided by the trajectory mapping element of the
system such that both the machine and the forks arrive at the
appropriate position in tandem. This may avoid an inexperienced
user having to perform various manoeuvres in series, such as, in a
first stage, positioning the machine in an appropriate position
through altering the ground propulsion control, including steering,
and, in a second stage started only after completion of the first
stage, positioning the forks of the implement relative to the
machine only after the machine is itself stationary. It also
reduces the likelihood that errors in the first stage machine
positioning are only identified by the user once the second stage
fork lifting stage has been completed, resulting in the user having
to return to the first stage of repositioning the machine
altogether.
[0065] In a further variation, in the case of an implement having
multiple degrees of freedom, these additional degrees of freedom
may be accommodated by the trajectory mapping element of the
system. Accordingly, for example, in the case of an implement
capable of movement relative to the machine body in terms of height
and angle, sensors in respect of the control of both of these
aspects of implement position relative to the machine body may be
provided to the processor for use in the trajectory mapping
functionality in order to provide a user with detailed predictions
of a future position of the implement based on current control
inputs, and may update in near real time in the case of changes to
any of those inputs.
[0066] As the skilled person would appreciate, in the case of the
backhoe loader exemplified by FIG. 2, an embodiment of the
disclosure may be used with respect to the second implement, at the
rear of the machine. In the illustration, this is shown as a bucket
attachment though other attachments are contemplated within the
scope of the disclosure. Indeed, one such alternative implement may
be a grapple, such as that described with reference to the
embodiment illustrated in FIG. 3.
[0067] FIG. 3 shows a machine, specifically an excavator 300,
having as its implement in the example figure a grapple 320. While
the illustrated embodiment shows a grapple 320, the implement may
be interchangeable with a number of alternative implements.
[0068] In the case of excavator 300, the degrees of freedom of the
implement relative to the machine may be different from those
associated with the loader bucket 120 shown in the context of the
machine 100 of FIG. 1 or with the forks 220 of shown in the context
of the machine 200 of FIG. 2.
[0069] The excavator may comprise a body 330 rotationally mounted
on a drive portion 335 that may comprise tracks for ground
propulsion. Rotational mounting may allow rotation about an axis
that projects normal to ground on which the drive portion rests.
The body 330 may comprise a cab 340 from which a user may control
both ground propulsion of the excavator 300 and movement of the
grapple 320 relative to the body 330.
[0070] The excavator 300 may further comprise a first arm 350
having a first end 351 and a second end 352. The first end 351 of
the first arm 350 may be pivotally connected to the body 330 via a
first pivot 355 (not shown). The excavator may further comprise a
second arm 360 having a first end 361 and a second end 362. The
first end 361 of the second arm 360 may be pivotally connected via
a second pivot 365 to the second end 352 of the first arm 350. The
second arm 360 may comprise an implement coupling portion 368 at
the second end 362 of the second arm 360. An axis of the first
pivot 355 may be parallel to an axis of the second pivot 365.
[0071] In the illustrated embodiment of FIG. 3, a grapple 320 is
attached to the implement coupling portion 368. The implement
coupling portion 368 may be configured to enable rotational
movement of the grapple 320 in a direction about an axis
perpendicular to the axis of the second arm 360.
[0072] The grapple 320 may comprise a first jaw 321 and a second
jaw 322. The first jaw 321 may be openable and closable relative to
the second jaw 322 and vice versa via a hinging arrangement
323.
[0073] The system may comprise one or more sensors, a processor and
a display visible to a user in the cab of the excavator 300. The
one or more sensors may comprise a grapple camera 324 located in
the grapple 320, perhaps between the two jaws 321, 322 adjacent the
hinging arrangement 323 such that when the grapple jaws 321, 322
are open the camera may provide an image to include opening teeth
325, 326 of each of the two jaws 321, 322 and any article that may
be in the vicinity of the teeth 325, 326. This may assist a user in
aligning the jaws 321, 322 relative to the article in order to grab
the article securely between the jaws 321, 322. For example, image
processing software may be configured to represent the grapple
(either schematically or as a live camera view) and to superimpose
on the representation a projection of a future position of the
grapple jaws based on current control inputs. This may be updated
by the image processing software in the event that inputs
change.
[0074] In addition, the embodiment of FIG. 3 may comprise one or
more further sensors for providing data relating to one or more of
the following: control of forward, rearward and rotational movement
of the tracks relative to the ground; control of the first and
second arms 350, 360 about their respective pivots; a distance
between the body 330 and the implement coupling portion 368: angle
of rotation of the grapple 320 about a longitudinal direction of
the second arm 360; grapple jaw angle representing an angle
subtended from the perspective of the hinging arrangement 323
between the opening teeth 325, 326 of first and second jaws 321,
322.
[0075] Data obtained from the camera and sensors may, for example,
be used to produce for display to the user a schematic
representation of the grapple 320 relative to an object within view
of the grapple camera 324 whose dimensions and position may be
obtained from the view provided by the grapple camera 324. The
schematic representation may show the grapple from an assimilated
position adjacent the grapple, even though there may not be a
camera at that location. Schematic representations of the implement
relative to the machine body may show its position relative to
other articles in the surrounding environment such as, but not
limited to, obstacles that the user may have reason to want to
avoid.
[0076] In addition or in the alternative, such data may be provided
also be provided to a user in a variety of formats including raw
distances and angles and with respect to relative scales.
[0077] A wide variety of grapple implements are known in the art.
FIG. 4 shows an excavator 400 that is the same as that of FIG. 3,
except that the grapple 320 of FIG. 3 (a grapple of a
clamshell-type) is substituted for a grapple 420 of a sorting
type.
[0078] Further grapples are contemplated within the scope of the
disclosure. For many types of grapples, such as the sorting grapple
of FIG. 4, it may be particularly useful to provide trajectory
information conceptually similar to that provided for the forks 220
of FIG. 2. A camera may be present within the grapple, between the
two jaws. The camera may provide a view of the environment directly
beneath the grapple. This view may be displayed to a user on the
system display. Furthermore, trajectory information obtained in a
similar manner as that for the forks of FIG. 2 (as shown in FIG.
16) may be provided with regard to future grapple jaw positions
based on current inputs.
[0079] An example of the kind of image that might be displayed is
shown in FIG. 19. This figure shows opposing jaws 1921, 1922 of a
grapple, such as the grapple of FIG. 4. Also shown is a pallet of
bricks 1925 as an example of an article intended to be collected by
the grapple. The system of the present disclosure superimposes on
the display an indication of the future trajectory of the grapple
1923, 1924 based on current inputs in order to provide the user
with guidance as to how to position the grapple jaws.
[0080] FIG. 5 shows a schematic representation of a front end of a
track-type tractor 500 including as its implement a blade 520. The
track-type tractor 500 may comprise a body cab 540 from which a
user may control ground propulsion of the track-type tractor 500 as
well as movement of the blade 520 relative to the machine body.
FIG. 6 shows a schematic representation of the front end of the
track-type tractor 500 of FIG. 5 from above. The blade 520 is shown
in a first, straight configuration. FIGS. 7 and 8 show the
schematic representation of the front end of the track-type tractor
500 of FIG. 6 with the blade 520 in second and third
configurations, respectively.
[0081] As can be seen from the first, second and third blade
configurations, the blade 520 may comprise a hinge. A first portion
521 of the blade 520 may be situated on a first side of the hinge
and a second portion 522 of the blade 520 may be situated on a
second side of the hinge. The hinge may enable the blade 520 to be
used in a single straight blade configuration or in a folded
configuration whereby the first portion 521 of the blade 520 is at
an angle other than 180.degree. with respect to the second portion
522 of the blade 520.
[0082] In addition, the blade 520 may be movable up and down
relative to the body 530 of the track-type tractor 500.
Furthermore, one side of the blade 520 may be movable up and down
independently of an opposite side of the blade 520 such that the
blade 520 may be lower at a first end than at a second end.
Furthermore, while in FIG. 5 the blade 520 is illustrated as being
substantially normal to a surface on which the track-type tractor
500 is resting, an angle of tilt of the blade 520 may be altered
such that the blade 520 is angled forward or backwards relative to
the body 530 of the track-type tractor 500.
[0083] In this embodiment, as in the previous embodiments, sensors
may be used to detect the implement type, angle, tilt, hinge
position (since the blade may be substituted for another
implement). Furthermore, sensors may be configured to provide data
regarding machine ground propulsion control. Such sensors may
include those known in the art. For example, sensors relating to
speed of a machine relative to the ground are known in machines for
providing data to a speedometer for display to a user. Furthermore,
the sensors may be configured to feedback changes to the sensed
parameters at frequent intervals. The data obtained from these
sensors may be processed in a processor and used to provide
information to the user via a display.
[0084] As referred to above in respect of the examples illustrated,
implements may be interchangeable. This may be the case for many
machines known in the art. For example, the track-type tractor of
FIG. 5 may, instead of a blade, have attached thereto any number of
alternative implements, such as those illustrated in FIGS. 9a to
9f. FIG. 9a shows a bucket 910, FIG. 9b shows a blade 920, FIG. 9c
shows a cold planer 930, FIG. 9d shows a compactor 940; FIG. 9e
shows a broom 950 and FIG. 9f shows a snow blower 960. Similarly,
as would be well understood by the skilled person, the backhoe
loader of FIG. 2 or the machine of FIG. 1 might be capable of
receiving any one of these implements.
[0085] For implements such as these (and others), in one
embodiment, the system of the present disclosure may provide a
schematic bird's eye view of the machine in its environment on
which are superimposed various representations of widths and areas
relative to the implement. An example of this is shown in FIG.
10.
[0086] First, the system may be configured to obtain information
regarding the implement type and size. This may be obtained in any
manner including that of alphanumeric recognition of a code on the
implement and visible to a camera on the machine, as described
above with reference to the FIG. 1 embodiment.
[0087] Based on information regarding machine type, there may be
superimposed onto the bird's eye view schematic representation
pairs of (potentially) parallel lines representing any or all of
the following: [0088] (a) the working width of the implement (e.g.,
in the case of a broom, the width that would benefit from the
broom); [0089] (b) the actual width of the implement (e.g., in the
case of a broom, the width of the implement including that extends
beyond that which would benefit from the broom); [0090] (c) a
safety zone representing width within which it is recommended that
people are avoided and that may widen with distance forward of the
implement and or may widen with speed of the machine); [0091] (d) a
snow trajectory zone that may, in the case of a snow blower,
indicated to a user an expected trajectory of snow affected by the
snow blower dependent upon direction of output nozzle, which may be
variable in three dimensions.
[0092] Other representations may also be shown, depending on the
implement selected.
[0093] The information required for the FIG. 10 embodiment may be
obtained by a combination of the following sensors: [0094] one or
more optical cameras mounted on the machine; [0095] one or more
infrared cameras mounted on the machine to allow detection of
people and animals that may be in the vicinity of the machine;
[0096] one or more sensors for detecting attachment type, such an
alphanumeric recognition via a camera, barcode recognition via a
camera, QR code recognition via a camera, RFID recognition via an
RDIF transceiver, or any other implement detecting strategy; [0097]
one or more sensors related to machine speed, which may or may not
be related to a speedometer of the machine; [0098] one or more
sensors related to machine steering control; [0099] one or more
sensors related to implement height; [0100] one or more sensors
related to implement angle; [0101] one or more sensors related to
implement tilt; [0102] one or more sensors related to other
implement factors such as extent to which jaws of an implement are
open, or angles of snow blowing nozzle relative to implement, or
any other implement specific variable; [0103] any other sensor that
may be used to provide data regarding machine location, machine
speed, machine steering, implement movement in any direction or any
other suitable sensor.
[0104] A schematic illustration of the various criteria that may be
detected for use by the system of any of the embodiments
illustrated herein is provided in FIG. 20. While FIG. 20 shows an
embodiment having forks, the radar functionality may be
particularly appropriate in implements such as saws.
[0105] The following is a list showing some of the variables to be
sensed and, in each case, a representative example of the kind of
sensor that may be used:
[0106] Type of attachment may be sensed by a camera with image
processing algorithm or non-contacting inductive (RFID) sensor.
Steering angle may be sensed by a non-contacting magnetic position
sensor. Snow blower nozzle direction may be sensed by a
non-contacting magnetic position sensor. Attachment angle may be
sensed by a non-contacting magnetic position sensor. Machine speed
may be sensed by a inductive or hall effect shaft rotation speed
sensor. Blade angle may be sensed by a non-contacting linear
position sensor. Front linkage height, tilt, and/or angle may be
sensed by a non-contacting rotary magnetic position sensor. Machine
level may be sensed by an accelerometer or gyroscopic inclination
sensor. Fork width may be sensed by a camera with image processing
algorithm. Forward radar may be sensed by a radar-based sensor with
object detection algorithms. Forward (work zone) camera may be
sensed by an optical camera. Downward (below ground) radar may be
sensed by a radar-based sensor with object detection algorithms.
`Birds eye` camera view may be sensed by a multiple cameras with
image processing/stitching functionality.
[0107] In some embodiments, the implement itself may comprise a
camera configured to provide data to a user via the system of the
disclosure. This may be particularly useful when aligning the
implement with an article to be contacted by the implement.
[0108] In a further embodiment, shown in FIG. 11, there may be
represented a safety zone around the implement that is superimposed
over a schematic view of the machine and implement in context. From
this and from other sensor data (perhaps including an infrared
camera to allow the image processing software to interpret the
presence of a person or people) it may be possible to see the
position of people in the environment relative to whether they fall
within or without that zone. In some variations, the system of the
present disclosure may automatically prevent implement use or
movement in the event that a person is detected within the safety
zone.
[0109] An embodiment including a safety zone may be particularly
applicable to attachments that are typically used while the machine
is moving relative to its surroundings. For example, a snow blower
(see FIG. 9f) or a broom (FIG. 9e) is, for the most part, generally
used while the machine is moving relative to the ground.
[0110] There may be potential hazards associated with this. In the
case of the broom 950 example, there may be a risk of debris being
propelled a significant distance from the broom 950. Similarly, in
the case of a snow blower 960, there may be the intention that snow
is diverted in a particular direction from the snow blower 960.
[0111] In order to reduce potential hazards, it may be advisable
for implements not to be used when in close proximity to people
and/or particular features in the surroundings. A safe distance
from an implement may depend on a number of factors including,
implement type, implement specific parameters (e.g., rotational
speed, in the case of a broom), position of implement relative to
machine body, forward or backward propulsion speed of machine,
steering position of machine, and potentially many other
factors.
[0112] For example, in the case of the broom, when forward or
rearward propulsion of the machine is fast, an appropriate distance
from the machine may be greater than when it is slow. Similarly,
when the broom is rotating fast, an appropriate distance from the
machine may be greater than when it is slow. The distance may be
different depending on the direction from the implement. For
example, the distance may be longer in front of the implement than
to the side of the implement. A safety zone may be said to be
defined by a perimeter around the implement inside which it is
inadvisable to enter. The size and shape of the safety zone may
depend on a wide number of variables. The size and shape of the
safety zone may be determined on the basis of a combination of the
current input data and may be obtained, for example, from a data
library or similar or calculated in the processor.
[0113] The safety zone may, in one embodiment, simply be
represented schematically to a user on a display. An example of
such an embodiment is shown in FIG. 11. In this embodiment, the
display may provide a schematic representation of a bird's eye view
of the machine 1100 showing the body 1130 and implement 1120. This
view may be assimilated from a plurality of sensors, as described,
for example, in relation to the FIG. 10 embodiment. Superimposed on
the schematic representation may be a representation of the safety
zone 1190. In this example, the safety zone 1190 extends a greater
distance forward of the implement than rearward of the implement.
It may be the case that the forward extent of the safety zone is
greater when the machine or implement is operating faster.
[0114] Articles present within or without the safety zone may be
represented in the bird's eye view. For example, a person might be
represented, for example by an image or perhaps by an icon on the
display at the position where the person is detected.
[0115] In a further embodiment, instead or in addition, the system
may automatically override control of the implement (for example,
rotation of a broom) in order to reduce risk in the event that an
obstacle (for example, a person) is detected within the safety
zone. The system may alternatively or in addition automatically
override control of ground propulsion of the machine, perhaps by
applying a brake to prevent ground propulsion of the machine.
[0116] FIG. 12a shows a saw 1210, FIG. 12b shows a brushcutter 1220
and FIG. 12c shows a mulcher 1230. Such implements, as well as some
implements already described, may benefit from further data sensing
regarding implement use and implement position. For example, a saw
1210 may benefit from sensing related to a particular article that
may be adjacent a saw blade in preparation for or during the
process of cutting.
[0117] FIG. 14 shows a brushcutter 1420 as an attachment to a
machine of a kind known as a skid-steer loader 1400.
[0118] In some embodiments, implements may benefit from further
sensing relative to the surrounding environment. For example, a saw
may benefit from sensing in relation to unseen features that may be
beneath or within a surface to be cut by the saw. For example,
where a saw might be used to cut into a surface, it may be
beneficial to detect pipes, cables, steel and other objects that
may be hidden within the surface to be cut.
[0119] Saw implements, together with radar detection zone and
optical camera focus zone are illustrated in FIGS. 13a and 13b.
[0120] To this end, a saw implement may be equipped with radar
functionality. The radar functionality may be configured to detect
objects that might include pipes, cables, steel and other objects,
including those that may be hidden behind or within other objects
and surfaces. The radar functionality may be provided by a radar
apparatus focusable on a position of interest ahead of the current
saw position. In tandem with this there may be provided an optical
camera focused on the same or a similar position of interest. In
this way, the processor may receive data from the radar apparatus
and the optical camera and process the information in order to
superimpose information obtained via those techniques onto a
schematic representation of the implement and its surroundings so
that a user may be provided with information to assist in
controlling ground propulsion of the machine and in controlling of
the implement relative to the machine body in order to determine a
preferred cutting path. The sensor data may be provided in near
real time and the processor may operate in near real time in order
to provide the information to the user in near real time and such
that the user may continuously adapt their controlling of the
machine and implement to plot a preferred path for the implement
relative to the surroundings, such as a surface to be cut.
[0121] In some embodiments, there may be more than one radar
apparatus. For example, there may be a first radar apparatus
configured to obtain information forward of the saw and relevant to
forward saw trajectory plotting and there may be a second radar
apparatus configured to obtain information closer to a current area
of impact of the saw. In this way, there may be information
provided to influence user control of forward movement of the saw
as well as information provided to influence user control of the
saw blade at the current position. This may be particularly
effective in preventing unintentional cutting of an object just
before said cutting may be due to occur.
[0122] FIG. 15 shows a machine, specifically of a kind known as a
skid steer loader 1500, having as its implement an auger 1520 for
drilling into a surface (instead of a loader bucket, for example).
In the case of some implements, including an auger, it may be
helpful to a user to obtain an indication of the depth of a distal
end of the auger relative to a reference point in order to be able
to drill a hole of the required depth, in addition to providing
information to a user regarding how to position the auger relative
to the surface prior to drilling. Such information may be presented
to as raw depth in formation relative to a surface or may be
represented schematically on a display.
[0123] In a further embodiment of the disclosure, sensor
information may be used to assist a user in positioning a machine
100 having a loading implement (such as a loader bucket 120)
adjacent a truck into which the user may wish to deposit contents
of the loader bucket 120. Such information may be presented to a
user on a display and/or may provide the user with information in
other ways such as, for example, providing an audible indication
when a part of the machine or implement comes within a particular
distance of the truck. In some embodiments, there may be active
braking that engages a brake of the machine in order to prevent a
collision. It may, alternatively or in addition, prevent a user
from further movement of an implement where such movement would
follow a trajectory that would result in a collision of the
implement with, for example, a side of the truck.
[0124] While various implements have been disclosed in relation to
various machine types, it will of course be appreciated by the
skilled person that the specific machine-implement combinations
described and illustrated herein are merely indicative. To the
extent that an implement is compatible for attachment to a
particular machine, the system of the present disclosure may be
equally applicable to that implement-machine combination.
[0125] For example, the disclosure may be equally applicable to
machines having any kind of ground propulsion known in the art. The
system may use input data regarding ground propulsion without
knowledge of how that ground propulsion may be achieved.
Accordingly, any particular embodiment of the disclosure is not
limited to whether the machine with which it operates is propelled
by wheels, tracks, a combination of the two or any other means.
Other means of ground propulsion than those explicitly recited
herein are known in the art.
[0126] The disclosure may be applicable to a machine having a wide
range of different implement possibilities. The implement may be
permanently attached to a particular machine or couplable to the
machine, and therefore substitutable for one or more of a further
range of implements.
[0127] While some particular combinations of sensors and inputs
have been disclosed in relation to specific embodiments, the
skilled person would appreciate that the different combinations of
sensors and inputs may be applicable to different embodiments of
machine and implement. The disclosure is not to be understood as
limited to the specific combination of sensors and inputs disclosed
in respect of the specific machines and implements. Any combination
of sensors and inputs may be equally applicable to any combination
of machine and implement.
INDUSTRIAL APPLICABILITY
[0128] The present disclosure finds application in improving the
efficient use of a machine, particularly though not exclusively by
an inexperienced user.
* * * * *