U.S. patent application number 17/632433 was filed with the patent office on 2022-08-25 for method and systems in a forest machine.
This patent application is currently assigned to Ponsse Oyj. The applicant listed for this patent is Ponsse Oyj. Invention is credited to Esko HAVIMAKI.
Application Number | 20220266868 17/632433 |
Document ID | / |
Family ID | 1000006378310 |
Filed Date | 2022-08-25 |
United States Patent
Application |
20220266868 |
Kind Code |
A1 |
HAVIMAKI; Esko |
August 25, 2022 |
METHOD AND SYSTEMS IN A FOREST MACHINE
Abstract
The invention relates to a method in a forest machine. In the
method the surroundings of the forest machine are observed using
one or more sensors. The observations are assembled to form a
database of the surroundings, and the database is utilized in the
operation of the forest machine. In the method observations of a
limited area are used. In addition, a database is formed in real
time in the forest machine. Further, a view is formed from the
database of selected observations, from which a track in the
surroundings is recognized. The forest machine is then oriented in
the direction of the track, once the forest machine's direction of
travel relative to the recognized track has been recognized. The
invention also relates to a system in a forest machine.
Inventors: |
HAVIMAKI; Esko; (VIEREMA,
FI) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Ponsse Oyj |
VIEREMA |
|
FI |
|
|
Assignee: |
Ponsse Oyj
VIEREMA
FI
|
Family ID: |
1000006378310 |
Appl. No.: |
17/632433 |
Filed: |
August 14, 2020 |
PCT Filed: |
August 14, 2020 |
PCT NO: |
PCT/FI2020/050529 |
371 Date: |
February 2, 2022 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
B60W 2420/403 20130101;
B60W 2420/52 20130101; B60W 60/0025 20200201; A01G 23/08 20130101;
G06V 20/58 20220101; B60W 2300/158 20130101 |
International
Class: |
B60W 60/00 20060101
B60W060/00; G06V 20/58 20060101 G06V020/58 |
Foreign Application Data
Date |
Code |
Application Number |
Aug 14, 2019 |
FI |
20195678 |
Claims
1. A method in a forest machine, the machine, the method including
observing surroundings of the forest machine using one or more
sensors, and the observations are assembled to form a database of
the surroundings, and the database is utilized in operation of the
forest machine, characterized in that the observations of a limited
area are used, and the database is formed in real time in the
forest machine, a view being formed from the database of selected
observations from which a track in the surroundings is recognized,
and the forest machine is oriented in a direction of the track,
once a travel direction of the forest machine relative to the
recognized track has been detected.
2. The method according to claim 1, characterized in that when
orienting, the forest machine is steered automatically,
semi-automatically, or with assistance, based on the view.
3. The Method according to claim 2, characterized in that, in
addition to steering, the forest machine is driven automatically,
semi-automatically, or with assistance.
4. The method according to claim 1, characterized in that the
surroundings are observed using one or more sensors, from whose
measurement data a 3D point cloud of the surroundings is
formed.
5. The method according to claim 4, characterized in that at least
of a camera, a radar, and a lidar is used as the one or more
sensor.
6. The method according to claim 1, characterized in that a ground
and trees are also recognized from the view.
7. The method according to claim 1, characterized in that the view
is recorded and the view is attached to a map program in the forest
machine.
8. The method according to claim 7, characterized in that, before
observing, a known starting point is sought, to position the forest
machine.
9. A method according to claim 8, characterized in that, after
seeking the known starting point, the view is attached to a map
program recorded in the forest machine, to position the forest
machine without a network connection.
10. The method according to claim 9, characterized in that, after
seeking the starting point, the view is updated by storing a part
of the view as known, to position the forest machine without a
network connection.
11. The method according to claim 1, characterized in that one or
more obstacles on the track, or in its vicinity, are also defined
from the view.
12. The method according to claim 11, characterized in that machine
learning is added to the observing step, which is arranged to
resolve whether to steer around the one or more defined
obstacle.
13. A system in a forest machine, the system including one or more
sensors arranged to observe surroundings of the forest machine and
a database to assemble observations of the surroundings, which
database is arranged to be utilized in operation of the forest
machine, characterized in that the system includes means, which are
arranged to form in real time in the forest machine the database of
the observations of a limited area, and to recognize a track in the
surroundings based on a view formed from the database, and to
orient the forest machine in a direction of the track, once a
travel direction of the forest machine relative to the recognized
track has been detected.
14. The system according to claim 13, characterized in that the
means include a data system containing the database, fitted to the
forest machine, which is arranged to operate a control means
belonging to the forest machine.
15. The system according to claim 14, characterized in that the
data system is arranged to operate driving means belonging to the
forest machine.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims benefit of PCT/FI2020/050529 filed
Aug. 14, 2020, which claims benefit of Finnish Patent Application
No. FI 20195678 filed Aug. 14, 2019, each of which is incorporated
by reference in its entirety.
FIELD OF THE INVENTION
[0002] The invention relates to a method in a forest machine, in
which method the surroundings of the forest machine are observed
using one or more sensors, and the observations are assembled to
form a database of the surroundings, and the database is utilized
in the operation of the forest machine. The invention also relates
to a system in a forest machine.
BACKGROUND OF THE INVENTION
[0003] In the prior art, a database formed of the surroundings is
used, for example, in measuring the quality characteristics of a
stand of trees, particularly from the viewpoint of the sawmill
industry. Using the developed technology, a detailed 3D point cloud
of the forest can be produced to form a database, from which even
the trunks and branches of individual trees can be measured. The
analysed data can then be used to plan felling. For example, a
harvester can be taken to a stand marked for felling in which there
is a tree it is wished to fell.
[0004] The creation of the database is a separate operation, which
requires its own equipment and own operator. In addition, the
creation of the database demands a great deal of computation and
storage capacity. From the database, a view that can be examined by
eye can be formed, which is transmitted to the harvester for final
utilization. Another problem then is the alignment of the view with
the ground, so that the driver can select the correct trees to
fell.
[0005] Despite all the collected and analysed information, the
driver of the forwarder following the harvester has only vague
location and quality data on the felled timber.
SUMMARY OF THE INVENTION
[0006] The invention is intended to create a new type of method for
a forest machine, by which a database of the surroundings can be
formed more simply than before and further that the database can be
utilized more concretely than before. In addition, the invention is
intended to create a new type of a system for a forest machine,
which is simple and can be utilized directly to operate the forest
machine. The characteristic features of the method according to the
present invention are stated in the accompanying claim 1.
Correspondingly, the characteristic features of the system
according to the present invention are stated in the accompanying
claim 13. In the method and system according to the invention, a
database formed of observations is utilized in a new and surprising
manner. Generally, the database is formed in the forest machine. In
addition, the database is preferably used in real time to operate
the forest machine. The collection of felled trees is then
facilitated and at the same time problem situations can be avoided.
In addition, data on a stand for felling remains in the system and
can be utilized later, for example, to verify the felling result or
the felling trace, or, for example, to verify the thinning density
or ground damage.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] In the following, the invention is described in detail with
reference to the accompanying drawings showing some embodiments of
the invention, in which
[0008] FIG. 1 shows the operation of a forest machine, in which the
method and system according to the invention are utilized,
[0009] FIG. 2a shows raw data detected from the ground,
[0010] FIG. 2b shows a view formed from the ground, for utilization
in the invention,
[0011] FIG. 3 shows a forest machine implementing the method
according to the invention and equipped with the system according
to the invention,
[0012] FIG. 4 shows a schematic diagram of the system according to
the invention.
DETAILED DESCRIPTION OF THE DRAWINGS
[0013] FIG. 1 shows a forest machine 10, which is used to collect
the trees previously felled and cut by a harvester. Generally, the
trees are cut into stacks 11 by timber grade, or at least into
separate grab piles. It is then as easy as possible for the forest
machine's driver to find the trees on the ground. At the same time,
the mixing of different timber grades with each other is avoided.
The forest machine can be driven in both directions, as the
driver's seat can also be rotated. To aid the driver, there is
nowadays usually also a camera in addition to mirrors. Accessories
warning of obstacles are also used. In some cases, in thinning the
driver has his back facing the driving direction when making a
load. The forest machine is then driven on the basis of the camera,
which is difficult and slow. At the same time, trees left standing
can easily be knocked. Alternatively, the driver must turn his seat
while moving the forest machine, which further slows work. FIG. 1
shows thinning felling, in which some trees are left to grow. The
forest machine's route then winds about, taking into account the
ground and the remaining trees. In this situation, the forest
machine's driver usually has his back to the driving direction and
is looking at the load space, to load as efficiently as
possible.
[0014] The invention relates to a method in a forest machine. In
the method, the forest machine's 10 surroundings are observed. In
addition, the observations are collected to form a database 12 of
the surroundings (FIG. 4) and the database is utilized in the
operation of the forest machine 10. In the invention, observations
of a limited area are used. The number of observations is then
limited, and thus little processing capacity is required. In
addition, the database 12 is formed in real time in the forest
machine 10. Firstly, the existing equipment in the forest machine
can be used for processing and secondly there is no delay in
forming the database. The method can then be applied without
investing in equipment and external apparatuses. Further, a view 19
is formed from the database 12 of selected observations, from which
a track 13 in the surroundings is recognized. A view can be formed
simply from the observation data, from which the track can be
recognized by software means. Though the apparatus observes the
entire surroundings, the number of observations remains limited by
selecting only observations from a limited area. In addition, the
forest machine 10 is oriented in the direction of the track 13,
once the direction of travel 14 of the forest machine 10 relative
to the recognized track 13 has been detected. Here the direction of
travel refers to the straight line along which the forest machine
would move, if the steering wheel was not moved. The forest machine
can move forward or back along this line, but information on the
direction of travel can be defined from the system in the forest
machine. In FIG. 1, the forest machine's 10 direction of travel 14
(dot-and-dash line) differs from the track's 13 direction (broken
line), which is detected by the method the forest machine 10 then
being steered in the direction of the track 13. In other words, the
system controls the forest machine on the basis of the
database.
[0015] The level of control can be selected in the system.
According to the invention, when steering, the forest machine 10 is
controlled automatically, semi-automatically, or with assistance,
based on the database. In automatic control, the driver trusts the
system, which controls the forest machine entirely on the basis of
the database. The driver can then concentrate on operating the
loader. In semi-automatic control, the system starts steering, but
the real steering movement is made by the driver. From even a small
steering movement the driver feels the direction in which the
system would have steered. In assisted control, the system proposes
a suitable steering direction, for example by an arrow in the
display of the data system belonging to the system, or by nudging
the controls, or some other haptic feedback through the controls.
After receiving assistance from the control, the driver decides on
the progress of the forest machine. In other words, the driver
decides when and at what speed the forest machine will move.
[0016] Even control by, or assistance by the system will
significantly facilitate the driver's work. The driver can
trustingly proceed along the track without actually steering
himself. Tree damage and other known drawbacks can then be avoided.
According to the invention, in addition to steering, the forest
machine 10 can be driven automatically, semi-automatically, or with
assistance. In automatic driving, the system determines the speed
of the forest machine according to the situation. In semi-automatic
driving, the system either slightly accelerates or slows the forest
machine to control driving. In assisted driving, for example if the
system detects the next stack being far away or the stack's
position being already in the system, the system proposes
acceleration. The driver can then accelerate the forest machine to
the next stack. As in steering, the system can guide the driver to
accelerate or slow by showing a suitable signal in the display. The
driver can preferably choose how the system controls steering and
driving. The driver can then select the level of automation
according to the situation. However, in every situation, collisions
with trees left to grow and other obstacles are avoided. In other
ways too possible problematic situations can be forecast.
Generally, when the last load is lifted from the stack and it is
wished to move the forest machine, the driver only has to open the
grab enough for the trees to be taken into the load in the load
space. This avoids the hoist and grab swaying when the forest
machine is driven to the next stack. Excess movements and swaying
stress the forest machine.
[0017] In the invention, the surroundings are observed using one or
more sensors 15, from whose measurement data a 3D point cloud of
the surroundings is formed. The system can utilize the 3D point
cloud directly, but the 3D point cloud can also be shown as a
visual view, which assists the forest machine's driver in his work.
One or more of the following devices are used as sensors: a camera
16, radar 17, or lidar 18. Information will then be sufficient to
form a precise and reliable database. FIG. 2a shows the raw data
collected by lidar. The raw data is an innumerable number of
measurements and measurement points, from which a point cloud is
formed. The point cloud obtained is processed according to the
intended purpose, such as, for example, to be visualized to form a
view (FIG. 2b), or then to be processed to form a set of tree maps.
For example, by pointing the devices downwards a view 19 is
obtained, from which the track 13 can be reliably recognized (FIG.
2b). In addition, the area being observed can be limited, keeping
the amount of information to be processed small. In other words,
the observations of the limited area are used in recognition. For
example, only a specific sector in front of the forest machine is
observed. The forest machine's data-processing capacity will then
be sufficient and there will be as little delay as possible in
forming a database. One example of a suitable lidar is the laser
observation device marketed by the manufacture Velodyne under the
product name VLP-16, which can measure up to 600 000 points a
second.
[0018] Not only the track, but also the ground and trees are
recognized from the view 19, more generally from the database.
Thus, a record of the stand to be cut remains for later use. The
view 19 is preferably recorded and attached to a map program in the
forest machine 10. The view can then be utilized the next time when
the previous track can be exploited.
[0019] The entire surroundings can be observed using other
equipment in the forest machine, so that the trees and ground can
also be observed, as stated above. The observation of the track,
which is essential for steering, can be made from a limited, i.e.
smaller amount of data, so that processing will be local and in
real time. In addition, a view is formed from the database, from
which primarily the track is recognized, as are various other
things. By using observations of a limited area, the amount of data
to be processed remains reasonable. Thus, processing can be done in
the forest machine and preferably also in real time. This avoids
delays in processing and data transmission when the steering of the
forest machine will be correctly timed and sufficiently precise for
the forest machine to be sure to stay on the track.
Non-positionality also prevents the accumulation of errors, as the
track is followed in real time on real ground. The view formed is
also simple, but the track can be recognized sufficiently reliably
and rapidly using present technology. During transfers the driver
drives the forest machine normally, but when the stand is reached
the driver guides the forest machine to the start of the track,
when the system detects the track and as the trip progresses it
steers the machine to remain on the track, while the driver
concentrates on operating the hoist.
[0020] Position is also important in recording. In other words, it
is necessary to know the real position of the recorded database or
the view formed from it. In the area of the stand a mobile or GPS
connection may not be available when positioning will fail. In the
invention, a known starting point 20 is thus sought before
observation, to position the forest machine 10 (FIG. 1). In
practice, a network connection can generally be still found on the
forest track, but farther into the forest connections are not
available. On the other hand, the starting point can be defined
from a known point included in the recorded map, the precise
co-ordinates of which are known. The known position is then defined
in the map program or the position is defined from some known
position, which is attached to the map program. The positioning of,
and/or at least a definition of the position of the starting point
of the map is always sought to be made primarily using GPS or some
other known satellite positioning, such as Glonass, Galileo,
Beidou, IRNSS, or QZSS.
[0021] Once the starting point 20 has been sought, the view 19 is
attached to the map program stored in the forest machine 10, so as
to position the forest machine 10 without a network connection. The
position of the forest machine will then be known even without a
positioning connection. In addition, after seeking the said
starting point 20 the view 19 is updated by recording part of the
view 19 as known to position the forest machine 10 without a
network connection. FIG. 1 shows the map program, to which the
starting point 20 is connected, instead of the view. Thus, the
position remains known, when the real view is attached as part of
the map program. Preferably the obstacles 21 on, or near the track
13 are also defined in the observation. Problems can then be
avoided beforehand. Large stones may have rolled onto the track
after the harvester, or a large tree may have fallen across the
track. Now the system can also be used to detect obstacles. Further
the system can be developed by connecting to observation machine
learning, which is arranged to decide whether to go around the
detected obstacle 21, or what should be done after detecting an
obstacle. Previous going around or over an obstacle remains in the
memory and the stored data can be utilized in future operations.
For example, there is reason to indicate to take care at a previous
very deep part of the track, or even to bypass the point in
question.
[0022] By means of the method and system according to the
invention, particularly the productivity of a forest machine can be
significantly raised, and better quality can be achieved in
thinning, when the trees left standing are not damaged. By
installing a laser scanner or a distance camera in the front or
rear of the forest machine, track data can be formed in the forest
machine, based on which the automation can be arranged to steer and
even drive the forest machine. Driving can be done by the driver or
the automation, after which the driver can concentrate on only
collecting trees while the automation takes care of steering and
driving.
[0023] As not only the ground but also the trees are observed, the
system can also be utilized in group felling and thinning. The
driving and steering automation can be programmed so that large
stones and stumps are not driven over, instead the system seeks the
best route from the track, for example, in terms of speed, length
of trip, or fuel consumption.
[0024] In the invention, the forest machine is steered along an
observed track. Observation and steering are preferably done
automatically, making steering movements by the driver unnecessary.
It is enough for the driver to define speed according to the
situation. For example, the forest machine can be driven between
small stacks without stopping and stopped only at larger stacks.
When starting again, acceleration is enough while the system steers
the forest machine along the track. A known camera view can still
support the driver to ensure staying on the track, but the system,
however, actively steers the forest machine.
[0025] The system can also be arranged to drive the forest machine
automatically. In other words, the system determines when the
forest machine will move and at what speed. For example, when the
hoist is moving the system slows down and when lifting a heavy
load, it stops the forest machine. At the same time, the system
observes the track and guides the forest machine to stay on the
track. At a large stack, where trees must be lifted more than once,
the system can be instructed to await starting. The control can be
the single press of a button in the hoist's controls, or a specific
function when operating the hoist, from which the system recognizes
the next lifting coming from the same stack. If point control is
being used, the system recognizes the situation when the grab is
being taken to the same stack by point control. The system then
keeps the forest machine stationary. After, or even during the last
lifting, the driver can press a specific button or carry out a
pre-defined operation, from which the system knows to move the
forest machine forward. The operation activating starting can be,
for example, an extended opening of the grab or the rotation of the
opened grab over a pre-set rotation-angle limit.
[0026] The system includes one or more sensors 15 arranged to
observe the forest machine's 10 surroundings. In addition, the
system includes a database 12 for collecting observations of the
surroundings. According to the method, the system includes means 34
for detecting a previous track 13 in the surroundings and the
forest machine's 10 direction of travel 14 on the basis of a view
formed from the database 12, as well as for steering the forest
machine 10 in the direction of the track 13, once the forest
machine's 10 direction of travel 14 relative to the detected track
13 has been recognized.
[0027] Nowadays in a forest machine there is a map program as part
of the data system 22, in which the database 12 is arranged. The
observation means, such as the sensor 15, are connected to the data
system and are set to control the necessary operating devices, such
as the control means 23 and/or driving means 24. In other ways too
the driver's commands are transmitted as electrical pre-control to
the hydraulic operating devices. For example, even one hydraulic
cylinder, by which the mutual angle of the forest machine's chassis
is altered, is enough to steer the forest machine. The adding of
driving to steering already demands more signals, but these too are
of a limited number. At the same time, for example economical
acceleration ramps can be defined in the system, by which the
forest machine's starting is as rapid as possible, but nevertheless
economical. Observation of the ground and surroundings can affect
the acceleration ramp. On an even and straight track acceleration
can even be sharp, but when approaching a curve, hill, or other
obstacle acceleration is more gentle. On the other hand, if the
driver is already preparing for the next lifting, speed is sought
to be kept even, to avoid stops. Repeated acceleration and slowing
are then avoided, so that progress is economical. At the same time,
strain on the driver and the hoist too is reduced.
[0028] The observation means, more specifically the sensors, can
include various devices by which the ground can be observed with
sufficient precision. For example, by combining the image material
of a 360-camera a 3D model can be formed, from which the track can
be observed. Correspondingly, it is possible to obtain a 3D point
cloud, from which the shapes of the ground such as tracks can be
observed, directly from radar and lidar imaging. Observation is
preferably combined with the map program of the forest machine's
data system, when the real position of the forest machine can be
defined on the basis of the map program and real-time observation,
even though a GPS signal is unavailable. Miniature flying devices,
such as drones, can also be utilized to observe the
surroundings.
[0029] FIG. 3 shows a forest machine 10, which includes a front
chassis 25 and a rear chassis 26. The front chassis 25 has an
engine and a cab 27. Correspondingly, the rear chassis 26 has a
load space 28 and a hoist 29. The front chassis 25 and the rear
chassis 26 are pivoted to each other and the chassis' mutual angle
of rotation is altered by means of one or more hydraulic cylinders
30, which are part of the steering means 23. One part of the system
is a display device 31, which is located in the cab 27. The display
31 is connected to the data system 22, which controls and monitors
the forest machine and its operating devices. The necessary
sensors, database, and accessories are connected to the data
system. In FIG. 3 a camera 16, radar 17, and lidar 18 are connected
to the system. Here the sensors are located in the front chassis,
but they can equally well also be located in the rear chassis. For
example, the angle between the forest machine's front and rear
chassis is detected by sensors, such as an inertia measurement
unit, based on which the control system detects the angle and
permits chassis-steering of the forest machine.
[0030] FIG. 4 shows a schematic example of the system. Here the
data system 22, including the display 31, contains all the means
needed for collecting and processing data and further for operating
the operating devices and thus for steering and driving the forest
machine, according to the method. Here the radar 17 is connected
directly to the data system 22, but it is also possible to utilize
observation means connected to the machine-control system, such as
a 360-camera. In the embodiment shown, the system controls a valve
32, which is linked to two cross-connected hydraulic cylinders 30.
In the forest-machine embodiment the hydraulic cylinders are fitted
on both sides of the chassis pivot. When steering the forest
machine, the working movements of both hydraulic cylinders are
equal in magnitude but opposite in direction. Thus, the forest
machine can be steered using a single valve. Correspondingly, for
driving a servo motor 33 is connected here to the system. The servo
motor can be arranged to operate the accelerator pedal, or directly
the feed-pump's regulator. Alternatively, the control data is
transmitted directly electrically to the machine-control system. In
a modem forest machine equipped with various observation means the
functionality of the method can be obtained by means of software.
The method then becomes available through a software update.
* * * * *