U.S. patent application number 17/425536 was filed with the patent office on 2022-06-16 for harvester with robotic gripping capabilities.
The applicant listed for this patent is Ceres Innovation, LLC. Invention is credited to Frank FAULRING, Jason FAULRING, Jacob GOELLNER, Mark KEYSER, Carlos RAMIREZ.
Application Number | 20220183230 17/425536 |
Document ID | / |
Family ID | |
Filed Date | 2022-06-16 |
United States Patent
Application |
20220183230 |
Kind Code |
A1 |
FAULRING; Frank ; et
al. |
June 16, 2022 |
HARVESTER WITH ROBOTIC GRIPPING CAPABILITIES
Abstract
Systems and methods here may include a vehicle with automated
subcomponents for harvesting delicate targets such as agriculture.
In some examples, the vehicle includes a targeting subcomponent and
a harvesting subcomponent. In some examples, the harvesting
subcomponent includes vacuum features which gently attach to target
agriculture to secure it. In some examples, the harvesting
subcomponent includes padded spoons to grasp and remove the target
agriculture from the foliage.
Inventors: |
FAULRING; Frank; (Salinas,
CA) ; RAMIREZ; Carlos; (Salinas, CA) ;
FAULRING; Jason; (Salinas, CA) ; KEYSER; Mark;
(Salinas, CA) ; GOELLNER; Jacob; (Salinas,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Ceres Innovation, LLC |
Salinas |
CA |
US |
|
|
Appl. No.: |
17/425536 |
Filed: |
January 23, 2020 |
PCT Filed: |
January 23, 2020 |
PCT NO: |
PCT/US20/14745 |
371 Date: |
July 23, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62796319 |
Jan 24, 2019 |
|
|
|
International
Class: |
A01D 46/00 20060101
A01D046/00; A01D 46/30 20060101 A01D046/30; B25J 15/06 20060101
B25J015/06; B25J 11/00 20060101 B25J011/00 |
Claims
1. A system for harvesting agriculture, comprising: a traversing
machine with a frame, an articulating robotic arm attached to the
frame, and a computer system attached to the frame with a processor
and memory, the computer system in communication with the
articulating robotic arm; wherein the articulating robotic arm
including at least two joints and at least one picker subassembly,
the picker subassembly including at least one actuator in
communication with the computer system; wherein the picker
subassembly includes a vacuum subassembly, the vacuum subassembly
in communication with the computer system, the vacuum subassembly
coupled to a nozzle with a terminal end, wherein the terminal
nozzle end includes a flexible baffle section; wherein the picker
subassembly further includes two grappler spoons, the grappler
spoons configured to pinch together toward the vacuum nozzle to
secure a target by the actuator, in response to commands from the
computer system.
2. The system of claim 1 wherein the picker subassembly vacuum
nozzle is mounted on an extender actuator, in communication with
the computer system, the extender actuator configured to extend
away from the picker subassembly and back toward the picker
subassembly, the grappler spoons configured to pinch toward the
nozzle terminal end when the vacuum nozzle is retracted.
3. The system of claim 1 wherein the flexible baffle section is
removable and friction fit to the picker subassembly and made of
silicone.
4. The system of claim 1 wherein the grappler spoons are each
attached to the picker subassembly by a flange and include at least
one rim; wherein the grappler spoons each include a resilient
membrane stretched over the at least one rim.
5. The system of claim 4 wherein the grappler spoon membrane is
removable and friction fit to the rim and made of silicone.
6. The system of claim 1 wherein the vacuum subassembly nozzle
terminal end is generally round in cross section and includes a
resilient end membrane with apertures to allow air to flow.
7. The system of claim 6 wherein the vacuum assembly nozzle
terminal end apertures include at least four radially extending
slots configured in a middle of the round resilient membrane.
8. The system of claim 1 wherein the picker subassembly is
configured to twist in relation to the robotic arm, such that the
twist might break a target stem when grappled by the terminal
nozzle end and spoons.
9. The system of claim 1 wherein the picker subassembly includes a
stem cutting saw configured around the picker subassembly, the stem
cutting saw configured to slide around the picker subassembly and
spin, relative to the picker subassembly.
10. The system of claim 1 further comprising at least two stereo
cameras mounted to the frame, in communication with the computer
system, configured to send image data regarding potential
agricultural targets to the computer system for analysis.
11. A system for harvesting agriculture, comprising: a traversing
machine with a frame; at least two wheels or tracks mounted to the
frame; a computer system with a processor and memory mounted to the
frame; a sensor mounted to the frame in communication with the
computer system; an articulating robotic arm in communication with
the computer system, the articulating robotic arm mounted to the
frame at a first end; a pair of pincher spoons mounted to a second
end of the articulating robotic arm, the pair of pincher spoons in
communication with a pinching actuator, the pinching actuator in
communication with the computer system and configured to pinch the
pair of pincher spoons toward one another; a vacuum system
including a pump and a hose, the pump in communication with the
computer system; the hose including a terminal end mounted at the
second end of the articulating robotic arm, between the pair of
pincher spoons.
12. The system of claim 1 wherein the terminal end of the vacuum
hose includes a pliable baffle configured to flex in use.
13. The system of claim 12 wherein the pliable baffle is removably
attached at a first end to the terminal end of the vacuum hose.
14. The system of claim 12 wherein the terminal end of the vacuum
hose includes an actuator and extendible section configured to
extend and retract between the pair of pincher spoons, the actuator
in communication with the computer system.
15. The system of claim 13 wherein the pliable baffle includes a
second end with slots configured to allow air to pass.
16. The system of claim 15 wherein the slots in the pliable baffle
second end are radially configured or concentrically
configured.
17. The system of claim 11 wherein the pincher spoons each include
an inner and outer rim of pliable material.
18. The system of claim 1 wherein the pair of pincher spoons
include a removable pliable membrane.
19. The system of claim 14 wherein the limit of retraction of the
terminal end of the vacuum hose is past and above the pair of
pincher spoons.
20. A method of harvesting agriculture, comprising: traversing a
harvester machine down a planter bed row of agriculture; receiving,
at a computer system on the harvester machine, image data of the
agriculture; determining, at the computer system, likely targets
for harvesting from the image data; sending coordinates of the
likely targets to a robotic arm attached at a first end to the
harvester machine; sending harvest commands to a picker assembly
mounted at a second end of the robotic arm, the harvest commands
including activating a vacuum pump to draw air through a vacuum
tube with a vacuum terminal end on the picker assembly, and
actuating pinching of two pincher spoons mounted around the vacuum
terminal end.
21. The method of claim 20 wherein the vacuum terminal end includes
a flexible baffle section with apertures to allow air to pass
through the vacuum tube.
Description
CROSS REFERENCE
[0001] This application relates to and claims priority to U.S.
Provisional application 62/796,319 filed Jan. 24, 2019 the entirety
of which is hereby incorporated by reference.
TECHNICAL FIELD
[0002] This application relates to the field of automated
agricultural harvesting equipment using robotic gripping
assemblies, mobile harvesting units, produce handling, and various
combinations of related technologies.
BACKGROUND
[0003] The agriculture industry is highly reliant on human pickers
to harvest a number of produce, including berries such as
strawberries. The reason human pickers are still used today,
despite the technological advancements available, is because of the
difficulty of identifying a target such as a berry in a field, that
is ready to be picked, reaching through the foliage of the plant to
grasp that berry, and then carefully removing that berry without
damaging it, to package and sell immediately. The harsh conditions
of the field make this a difficult task to automate and
roboticize.
[0004] Current automatic harvesting of such delicate and difficult
to grasp agricultural targets such as berries, while operating in a
harsh outdoor environment did not exist before this application,
let alone systems the capabilities described herein.
SUMMARY
[0005] Systems and methods here may include a vehicle having
various subcomponents for harvesting delicate agricultural items
such as berries. In some examples, the subcomponents may be
automated. In some examples, the vehicle may include a targeting
subcomponent and a harvesting subcomponent. In some examples, the
targeting subcomponent utilizes multiple cameras to create three
dimensional maps of the target and target areas sometimes including
the agricultural foliage. In some examples, the targeting
subcomponent may include any of various cameras, sensors, or other
targeting features to locate and map targets in an automated or
semi-automated manner. The system may then determine coordinates of
the mapped targets to be passed to the harvesting subcomponent. In
some examples, the harvesting subcomponent may include vacuum
features which help a nozzle attach to an agriculture target for
harvesting. In some examples, the harvesting subcomponent includes
padded spoons to aid in removal of the targeted agriculture from
the plant, including in some examples, a stem.
[0006] Systems and methods for harvesting agriculture describe here
may include a traversing machine with a frame, an articulating
robotic arm attached to the frame, and a computer system attached
on the frame with a processor and memory in communication with the
articulating robotic arm, the articulating robotic arm including at
least two joints and at least one picker subassembly, the picker
subassembly including at least one actuator in communication with
the computer system, in some examples, the picker subassembly
includes a vacuum subassembly, the vacuum subassembly in
communication with the computer system, the vacuum subassembly
coupled to a nozzle with a terminal end, wherein the terminal
nozzle end includes a flexible baffle section, in some examples,
the picker subassembly further includes two grappler spoons, the
grappler spoons configured to pinch together toward the vacuum
nozzle to secure a target by the actuator, in response to commands
from the computer system.
[0007] In some examples, additionally or alternatively, the picker
subassembly vacuum nozzle is mounted on an extender actuator, in
communication with the computer system, the extender actuator
configured to extend away from the picker subassembly and back
toward the picker subassembly, the grappler spoons configured to
pinch toward the nozzle terminal end when the vacuum nozzle is
retracted. In some examples, additionally or alternatively, the
flexible baffle section is removable and friction fit to the picker
subassembly and made of silicone. In some examples, additionally or
alternatively, the grappler spoons are each attached to the picker
subassembly by a flange and include at least one rim, and in some
examples, the grappler spoons each include a resilient membrane
stretched over the at least one rim. In some examples, additionally
or alternatively, the grappler spoon membrane is removable and
friction fit to the rim and made of silicone. In some examples,
additionally or alternatively, the vacuum subassembly nozzle
terminal end is generally round in cross section and includes a
resilient end membrane with apertures to allow air to flow. In some
examples, additionally or alternatively, the vacuum assembly nozzle
terminal end apertures include at least four radially extending
slots configured in a middle of the round resilient membrane. In
some examples, additionally or alternatively, the picker
subassembly is configured to twist in relation to the robotic arm,
such that the twist might break a target stem when grappled by the
terminal nozzle end and spoons. In some examples, additionally or
alternatively, the picker subassembly includes a stem cutting saw
configured around the picker subassembly, the stem cutting saw
configured to slide around the picker subassembly and spin,
relative to the picker subassembly. In some examples, additionally
or alternatively, at least two stereo cameras may be mounted to the
frame, in communication with the computer system, configured to
send image data regarding potential agricultural targets to the
computer system for analysis.
[0008] Systems and methods for harvesting agriculture, as described
here, may include a traversing machine with a frame, a computer
system with a processor and memory mounted to the frame, an
articulating robotic arm in communication with the computer system,
the articulating robotic arm mounted to the frame at a first end, a
pair of pincher spoons mounted to a second end of the articulating
robotic arm, the pair of pincher spoons in communication with a
pinching actuator, the pinching actuator in communication with the
computer system and configured to pinch the pair of pincher spoons
toward one another, a vacuum system including a pump and a hose,
the pump in communication with the computer system; the hose
including a terminal end mounted at the second end of the
articulating robotic arm, between the pair of pincher spoons. In
some examples, additionally or alternatively, the terminal end of
the vacuum hose includes a pliable baffle configured to flex in
use. In some examples, additionally or alternatively, the pliable
baffle is removably attached at a first end to the terminal end of
the vacuum hose. In some examples, additionally or alternatively,
the terminal end of the vacuum hose includes an actuator and
extendible section configured to extend and retract between the
pair of pincher spoons, the actuator in communication with the
computer system. In some examples, additionally or alternatively,
the pliable baffle includes a second end with slots configured to
allow air to pass. In some examples, additionally or alternatively,
the slots in the pliable baffle second end are radially configured
or concentrically configured. In some examples, additionally or
alternatively, the pincher spoons each include an inner and outer
rim of pliable material. In some examples, additionally or
alternatively, the pair of pincher spoons include a removable
pliable membrane. In some examples, additionally or alternatively,
the limit of retraction of the terminal end of the vacuum hose is
past and above the pair of pincher spoons.
[0009] Methods and systems of harvesting agriculture as described
here may include traversing a harvester machine down a planter bed
row of agriculture, receiving, at a computer system on the
harvester machine, image data of the agriculture, determining, at
the computer system, likely targets for harvesting from the image
data, sending coordinates of the likely targets to a robotic arm
attached at a first end to the harvester machine, sending harvest
commands to a picker assembly mounted at a second end of the
robotic arm, the harvest commands including activating a vacuum
pump to draw air through a vacuum tube with a vacuum terminal end
on the picker assembly, and actuating pinching of two pincher
spoons mounted around the vacuum terminal end. In some examples,
additionally or alternatively, the vacuum terminal end includes a
flexible baffle section with apertures to allow air to pass through
the vacuum tube.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] For a fuller understanding of the nature and objects of the
disclosure, reference should be made to the following detailed
description, taken in connection with the accompanying drawings, in
which:
[0011] FIG. 1 is a diagram showing example mobile vehicle examples
as described in the embodiments disclosed herein.
[0012] FIG. 2 are diagrams showing example robotic arm examples as
described in the embodiments disclosed herein.
[0013] FIG. 3 is a diagram showing example picker head example
details as described in the embodiments disclosed herein.
[0014] FIGS. 4A, 4B and 4C are diagrams showing example extraction
hardware example steps as described in the embodiments disclosed
herein.
[0015] FIG. 5 is a diagram showing an example bellows interacting
with a target as described in the embodiments disclosed herein.
[0016] FIGS. 6A, 6B and 6C are diagrams showing example cutaways of
bellows construction as described in the embodiments disclosed
herein.
[0017] FIGS. 7A, 7B, 7C, and 7D are diagrams showing example
bellows interacting with differently sized targets as described in
the embodiments disclosed herein.
[0018] FIGS. 8A, 8B and 8C are diagrams showing example bellows
construction as described in the embodiments disclosed herein.
[0019] FIG. 9 is a diagram showing example spoon grappler
construction as described in the embodiments disclosed herein.
[0020] FIG. 10 is a diagram showing more example spoon grappler
construction as described in the embodiments disclosed herein.
[0021] FIG. 11 is a diagram showing example spoon grappler cutaway
construction as described in the embodiments disclosed herein.
[0022] FIG. 12 is a diagram showing an example spoon grappler
interacting with a target as described in the embodiments disclosed
herein.
[0023] FIG. 13 is a diagram showing more example spoon grappler
construction as described in the embodiments disclosed herein.
[0024] FIG. 14 is a diagram showing an example picker head with a
stem trimmer as described in the embodiments disclosed herein.
[0025] FIG. 15 is an example computer system which may be used in
the embodiments disclosed herein.
DETAILED DESCRIPTION
[0026] Reference will now be made in detail to embodiments,
examples of which are illustrated in the accompanying drawings. In
the following detailed description, numerous specific details are
set forth in order to provide a sufficient understanding of the
subject matter presented herein. But it will be apparent to one of
ordinary skill in the art that the subject matter may be practiced
without these specific details. Moreover, the particular
embodiments described herein are provided by way of example and
should not be used to limit the scope of the invention to these
particular embodiments. In other instances, well-known data
structures, timing protocols, software operations, procedures, and
components have not been described in detail so as not to
unnecessarily obscure aspects of the embodiments of the
invention.
Overview
[0027] The systems and methods described here include an
automated/semi-automated system with machine(s) that is/are capable
of harvesting agricultural targets such as berries without human
hands touching the plants or targets themselves. Example overall
systems may include subcomponents such as a seeker or sensor
subsystem to find and locate the targets, that works with and
informs a picking subsystem to harvest the targets. The overall
system(s) may be mounted on wheels and/or tracks to advance down a
row of targets such as agricultural produce so that the seeker
subassembly may identify and map the targets while the picker
subsystem is used to harvest, gather, and move the targets.
[0028] In some example embodiments, additionally or alternatively,
the seeker subassembly includes a camera or multi-camera system
used by a remote operator to locate target berries and three
dimensionally map them. In such examples, these mapped coordinates
may then be queued for harvesting. Additionally or alternatively,
in some examples, the harvester subassembly is then able to follow
the seeker subassembly and harvest the berries whose mapped
locations are queued by the seeker subassembly. In some example
embodiments, the harvester subassembly includes at least one
robotic arm with multiple degrees of freedom capable of reaching
into the foliage of a plant and extracting a target such as a
berry. In some examples, the extraction is through a vacuum system.
In some examples, additionally or alternatively, the extractions is
augmented by a padded spoon grasper, capable of twisting and
snapping a berry stem.
[0029] It should be noted that the examples used here describing
berry harvesting, or even strawberry harvesting in the written
description and/or figures is not intended to be limiting and is
merely used as an example. The agricultural targets to which the
systems here may identify, map, and ultimately harvest may be any
sort including but not limited to berries such as strawberries,
blackberries, blueberries, and raspberries, other examples include
grapes, figs, kiwi, dragon fruit, or other fruits. Vegetables may
be harvested as well, such as Brussel sprouts, tomatoes, peppers,
beans, peas, broccoli, cauliflower, or other vegetable. Any type of
agricultural target may be harvested using the systems described
herein. Additionally or alternatively, the systems and methods here
may be used to target and gather non-agricultural items such as
garbage, or be used to take scientific samples such as rocks or
minerals in environments or situations where it may be advantageous
to avoid human contact or interaction.
[0030] Harvester Subassembly
[0031] In some example embodiments, a harvesting subassembly is
included as its own separate vehicle system from the seeker
subassembly. Additionally or alternatively, in some examples, the
harvesting subassembly may be in communication with or connected to
the seeker subassembly. In some examples, seeker subcomponents are
integrated into the harvesting assembly and one machine
incorporates all of the features described herein. The harvesting
subassembly may include any number of features that allow for
autonomous, semi-autonomous, or human operable harvesting of
delicate target agriculture such as berries, as described
herein.
[0032] In some examples, either harvesting or seeker subassembly
may be mounted on its own vehicle subassembly with wheels and/or
tracks or combination of both, to traverse down a row of
agriculture with the seeker subassembly identifying and mapping the
target berries and the harvester subassembly gathering targets.
[0033] FIG. 1 shows examples of the overall traversing machine to
which any of the various subassemblies may be attached and/or
coupled. In the example, the main traversing subassembly 152
includes various portions mounted to it including main driving
and/or steering wheels 154 and in some examples, guide wheels 156.
In some examples, guide wheels 156 may be canted outward in order
to support traversing a mound 101 should a planting bed mound be
configured for the plants. In some examples, tank treads or tracks
may be used instead of wheels, and/or a combination of wheels and
tracks may be used, the drawings depicting wheels are not intended
to be limiting.
[0034] In some examples, any number of robotic arms 160 may be
mounted to any of various frame portions 153 and/or chassis
portions 155 that comprise the overall traversing subassembly 152.
It should be noted that many variations of robotic arms 160 may be
used in the systems described here, including but not limited to
robotic wrists with link and joint combinations with linear and
rotational links, gantry robots with linear joints, cylindrical
robots connected to rotary base joints, polar robots for twisting,
and/or jointed-arm or articulating robots with twisting joints and
rotary joints. Any combination of these or other robotic assemblies
160 may be used on the systems described herein to manipulate a
picker head and/or sensors for harvesting agricultural targets as
described.
[0035] In some examples, the robotic arm(s) 160 may include at
least one picker head, at least one sensor, at least one light
system, and/or a combination of picker heads, sensors, and/or
lights. In some examples, the sensors are mounted to the frame 153
of the harvester.
[0036] For example purposes, the range 195 of the robotic arm 160
is shown in the FIG. 1 to show that the robotic arm 160 may reach
different sides of a particular row mound 101 where targets may be
found and an accumulator for processing targets, such as the
example traversing conveyor 178. It should be noted that the system
in FIG. 1, is shown straddling one plant bed row. In some examples,
one system may straddle two, three, or more plant bed rows. In some
examples, wider or narrower plant bed rows may be straddled and the
example in FIG. 1, is merely intended to be an example, and not
limiting. By making the system wider to straddle a second row, two
sets of arms 160 may be used to pick two rows, or three, or four,
or more number of arms, commensurate to the task of picking.
[0037] In some examples, multiple robotic arms 160 may be fit onto
one overall traversing vehicle 152. For example, systems may
include a primary picker assembly with a clean-up/redundant picker
assembly which operates behind the primary setup. In those
examples, up to eight picker arms may be employed, four on the
primary and four on the clean-up assembly, with one or two arms
operating on each side of two rows. The clean-up system may operate
in the same way that the primary system operates, to find targets
that the primary system did not harvest, and/or to operate as a
redundancy should one or more arms on the primary system
malfunction.
[0038] In some example embodiments, the harvesting subassembly may
include at least one robotic arm 160 with joints that allow for
multiple degrees of freedom. Such arms 160 may be configured to
maneuver around and in foliage of a target plant to extract the
target agriculture such as but not limited to berries of any
sort.
[0039] In examples where targets are fruit plants which are
harvested many multiple times during a single growing season, often
multiple times per week, leaving fruit on a fruit plant may curtail
the productivity of the plant. If the plant senses that it still
has fruit on it, even rotting or deformed fruit, it may not produce
more fruit. This would limit production, so one of the goals of the
systems and methods here is to remove all of the fruit when ripe,
or when it should be removed to make the plant replace rotting or
deformed fruit. As the bed rows may allow for some fruit to drape
over the side of the plastic, and become easily exposed to viewing,
other fruit may grow under the foliage, or on top of the bed row
tops and be obscured by foliage. Therefore, to find and harvest as
much fruit from each plant as possible, it may be necessary to
maneuver the foliage to better view and/or harvest fruit
targets.
[0040] In some examples, foliage moving arms may be included to
alter, move, displace, and or otherwise gently maneuver the foliage
of the plant to better expose the targets such as fruit berries to
be picked. In such examples, a bar, or arm, may be pulled across
the top of the foliage in order to temporarily move it out of the
way for the seeker cameras and/or the harvesting assembly to locate
and grapple the target. In some examples, this foliage moving arm
may be maneuvered parallel or substantially parallel to the top of
the row bed, and pull across the top of the foliage, bending the
plant, but not breaking the plant leaves. This may reveal targets
under the foliage, those laying on the top of the row bed, or those
caught up in the foliage.
[0041] In some examples, a flexible curtain may be dragged over the
foliage, to avoid damage to the foliage, but still pull it out of
the way for the seeker and/or harvester to operate. In some
examples, this flexible curtain may be a plastic skirt, in some
examples, it may be a fringed or sliced skirt. In some examples, it
may have fringes that drape over the foliage, and yet flex around
the foliage so as not to damage it. As the flexible skirt is pulled
over the plants, it helps the seeker subassembly find the targets
more easily by limiting the area to be targeted with a clean
backdrop. The flexible skirt may be dragged from one side in one
direction during a first harvest and the next time the other
direction, to avoid biasing or pulling the foliage in the same
direction each time.
[0042] In some examples, the overall traversing subassembly 152 may
include a transfer conveyor 178. Such a conveyor may include any
number of conveyor belts, chains, rope, or other mechanism that can
pull materials from one place to another. Such transfer conveyor
may be used to collect harvested targets and move them to a
packaging subassembly, or storage unit.
[0043] In some example embodiments, the robotic arms 160 may be
ruggedized in that the tolerances and durability of the arms are
developed for outside, dirty employment. In such examples, the
robotic arms are not to be operated in pristine factory settings.
The systems described here should be able to operate in bad
weather, precipitation, dirt, mud, heat, cold, and in jarring,
rough conditions. As such, the bearings, tolerances, and actuators
may be made of more durable materials than clean factory robotic
assemblies. In some examples, extra gaskets may be fit into the
various joints to keep dirt out of the more delicate metal
couplings and pivoting features of the robotic arms. In such
examples, gaskets may be made of rubber, plastic, or ceramic. The
robotic arms may be made with fewer joints to minimize the number
of potential problems that may occur. The robotic arms may be made
of thicker materials, may be heavier, and be rust-proofed,
waterproof, weatherized, and/or otherwise reinforced.
[0044] Robotic Arm Examples
[0045] In some examples, additionally or alternatively, the
harvester subassembly as shown in FIG. 1 or any other embodiment,
may include any number of robotic arms upon which the grappler,
picker heads may be mounted.
[0046] FIG. 2 shows an example robotic arm picker assembly which
may be configured to grapple targets from rows of plants as shown
in the top down view 201, and from the side view 203. In FIG. 2, a
picker head 202 is shown mounted to a robotic arm 260 which may
position the picker head 202 in any of various poses in order to
couple to a target 250. In the example, the targets are on a raised
planter bed mound 201 which the robot arm 260 traverses from above
in some examples mounted to a vehicle that traverses a row and
targets and then grapples targets. In use, the system as shown in
FIG. 2 may be positioned by the robotic arm 260, to harvest a
target 250 by securing it using the steps as described in FIG. 3 as
well as FIGS. 4A, 4B and 4C, then move the target 250, and/or
deposit the target in storage as described.
[0047] In example embodiments, the robotic arm 260 may be any of
the robotic assemblies described herein, and may include various
numbers of joints thereby allowing for various degrees of freedom
to move around and about the plants and rows, taking different
angles. In some examples, the robotic arm 260 may include six
degrees of freedom. In some examples, the robotic arm 260 may
include four degrees of freedom, five degrees of freedom, six
degrees of freedom, or seven degrees of freedom, or any other
number. In FIG. 2, the first joint connecting the arm 260 to the
harvester (not shown) may include a 360 degree rotating pivot joint
280, as well as a pivot hinge joint 282. The next joint is shown as
a pivot hinge joint 284 connecting arm portions 260. The last joint
on the example of FIG. 2 is the ones connecting the arm 260 to the
picker head 202 as a pivot hinge joint 286. It should be noted that
any kind of joints may be utilized in the robotic arm 260 including
but not limited to pivot hinge joints, rotating pivot joints, ball
and socket joints, condyloid joints, saddle joints, or any
combination of these or other joints may be used. For example, the
joint connecting the picker head 202 to robotic arm 260 may include
multiple hinged joints or a ball and socket joint to allow the
picker head 202 to turn, twist, and angle many multiple degrees in
order to locate and harvest targets 250.
[0048] In various example embodiments, the robotic arm 260 may be
any of various lengths, thereby affecting the range 295 of the arm,
which may be tailored to the needs of the particular field or mound
or target. In some examples, the robotic arm 260 may include one or
more telescoping portions 288, which may be elongated and/or
retracted, thereby affecting the length of that portion and the
overall reach 295 of the robotic arm 260. Any of the portions of
the robotic arm 260 could include such telescoping portions in any
combination.
[0049] Picker Head Examples--Vacuum Point of Contact
[0050] In some example embodiments, the harvesting subassembly may
include at least one picker head that first interacts with the
target in the field to remove or detach the target from the plant
it grows on. Such picker heads may be affixed to or be part of the
robotic arms as discussed in FIG. 1 and FIG. 2. In some example
embodiments, the at least one picker head may be mounted on or
partially mounted on a robotic harvesting arm, alone or in
combination with a camera and/or lighting system.
[0051] FIG. 3 shows an example picker head assembly, with a front
3A and side 3B view of the same assembly in detail. In the example
shown, the main picker head assembly 302 is mounted on two
actuators, one actuator for a pincers 304 and one actuator for an
extender 306. In some examples, no extender actuator 306 is
utilized. In examples where an extender is utilized, the extender
actuator 306 moves in and down to move the main nozzle 303 up and
down, relative to the robotic arm (not shown).
[0052] The main nozzle 303 may be a hollow tube which may be used
to secure a coupling suction portion 330 to a target 350 such as a
berry. In some examples, the coupling portion 330 may include one
or more bellows or bellow configurations that allow the coupling
portion to stay flexible and malleable to couple with the target
350. In some examples, the vacuum hose 303 may be connected with
the main nozzle 330 to impart a suction or lower than ambient
pressure within the nozzle tube 303, and thereby be able to attach
to and secure a target 350. In some examples, a vacuum pump
subsystem may be mounted on the harvesting subassembly and a vacuum
hose may run through or around each harvesting picker robotic arm.
In some examples, vacuum subassemblies may be mounted on the
robotic arm itself, along with a vacuum hose on the picker head
302.
[0053] In some examples, the compression coupling portion 330 may
be 1.250 inches in diameter, in some examples, the nozzle may be
between 1.00 and 0.75 inches in diameter, but in any case, the
nozzle could be customized to any size of intended target. In some
examples, the compression coupling or bellows portion 330 may be 2
inches in diameter. In some examples, the compression coupling or
bellows portion 330 may be between 1.75 and 2.25 inches in
diameter. In some examples, the nozzle may have an effective ported
area of 0.44 square inches. In some examples, the nozzle may have
an effective ported area between 0.4 and 0.5 square inches. In some
examples, the nozzle may have an effective ported area between 0.3
and 0.6 square inches. In some examples, the area of the
compression coupling portion 330 may expand or stretch when
attaching to a target berry and in such cases expand to an area of
1.56 square inches. In some examples, the area may expand to
between 1.5 and 1.6 square inches. In some examples, the area may
expand to between 1.4 and 1.7 square inches.
[0054] In some examples, the amount of suction power that the
vacuum system imparts, may be 35 inches of negative vacuum. In some
examples, 50 inches of negative vacuum may be used. In some
examples, between 40 and 70 inches of negative vacuum may be used.
Alternatively or additionally, in some examples, less than 80
inches negative of vacuum may be used so as to avoid damage to the
target 350. In some examples, less than 68 inches of negative
vacuum may be used. In some examples, the coupling or contact
portion 330 of the bellows may spread the contact area over 20% to
30% of the target or berry so as to mitigate localized contact
pressure to any one area and avoid damage, depending on the size of
the target. Additionally or alternatively, the vacuum system may be
able to reverse from suction to blowing air outward, to clear
debris from the bellows, before switching back to a suction mode
for harvesting.
[0055] The compression nozzle bellows portion 330 may include a
malleable hood or coupling section 332 which may include one or
more bellow sections, and a rim 334 around an opening 336 to aid in
coupling to a target. In some examples, the compression coupling
portion 330 is or made up of at least one of, or combination of a
neoprene sleeve, a silicone sleeve, a rubber sleeve, or other
natural or synthetic material that is soft and flexible including
embodiments with embedded fibers and/or wires. Such a malleable
coupling section 332 may be configured to deform or otherwise
compress when a target 350 is contacted and may include baffles or
other structure that allows for deformation and malleability. Such
a deformation or compression may allow for the rim 334 to more
easily conform to the target 350 and thereby form a better suction
fit for the opening 336. In some examples, the bellows 330 portion
may be removable and replaceable in the field using a friction fit
and rim as described herein.
[0056] In some examples, the compression nozzle portion 330 may
include an internal reverse conical mesh to help capture the target
350 yet be as gentle as possible on them as described herein. In
such examples, the mesh creates an environment where the negative
vacuum is acting on a broader surface of the target, thus
minimizing the chance of target damage from localized contact to
the grappler edges. This mesh forms a cradle for the target to lay
in even while being picked, handled, and moved. Such a mesh can be
made of silicone materials for durability and flexibility.
Alternate materials may be used such as a wire mesh, a plastic
mesh, or a combination of wire mesh with plastic coating. Silicon
coating may be used on a wire mesh in some example embodiments as
well. FIGS. 5, 6A-6C, 7A-7D, 8A-8C show more example embodiments
that may be used herein.
[0057] In some examples, the compression nozzle portion 330 and the
opening 334 may be sized for a most average target 350, big enough
for the biggest targets and flexible, but able to grasp and vacuum
even a smaller target.
[0058] Examples may also include an internal spring system, inside
or integrated into the coupling portion 330. Such a spring system
may be made of plastic or metal coil(s) that help return the
coupling portion 330 back to an extended shape after a target is
released by turning off the vacuum and thereby deposited.
Additionally or alternately, an iris or camera lens feature may be
integrated into the nozzle 330. In such examples, the system may be
able to adjust the size of the opening or nozzle end for different
sized targets, opening for larger targets, and constricting for
smaller targets. In such examples, a coil or spring could be wound
tighter for smaller targets and wound looser for larger
targets.
[0059] Another portion of the example embodiment of FIG. 3 is the
grappler spoons 312, 314. The grappler spoons 312, 314 may be
configured with the main nozzle 303 between them and be configured
to move in a pincer motion toward the nozzle 303 by a robotic
actuator 316 and a hinge 318 arrangement. In some example
embodiments, the grappler spoons include a cushion or pad 320, 322.
In some examples, the cushion 320, 322 may be made of or include
closed cell foam, neoprene, gel filled pads, liquid filled pads,
open cell foam, layers of foam of different densities, a foam
backing with a gel filled pad on top, and/or any combination of the
above or other material that may cushion a target 350 when the
grappler spoons 312, 314 pinch the target 350. In some examples,
the material contacting the target 350 is no more than 20-30
durometer in hardness. More discussion of grappler spoon examples
may be found in FIGS. 9, 10, 11, and FIG. 12.
[0060] In some examples, a picker head 302 may include multiple
grappler spoons. In some examples, three spoons may be employed in
a similar manner as those examples shown with two as in FIG. 3. In
some examples, four grappler spoons may be configured in two axes
around the picker head 302 assembly. In some examples,
alternatively or additionally, the grappler spoons include a hinged
and/or spring loaded portion at the end to better cushion the
target 350 when pinched. In some examples, the grappler spoons 312,
314 may pivot about the nozzle 303 to impart a twisting motion to
snap a berry or other stem as discussed herein using an actuator,
screw drive, motor, or other feature to twist the picker head
302.
[0061] In some examples, a pneumatic trash cleaning air jet 324 may
be mounted to the end of the grappler spoon 312, 314 in order to
help clear debris. In such examples, air holes may be configured on
the end lip of the spoons and face in various directions to direct
air toward foliage. In some examples, a line of holes may be
configured on the end lip of each grappler spoon 312, 314.
[0062] Example Picker Head and Picking Steps
[0063] As discussed in FIG. 3, the picker head may utilize features
to grasp, secure, move, and then subsequently deposit an
agricultural target such as a berry. Any combination of the
features described here, alone or in combination may be used for
this purpose.
[0064] FIGS. 4A, 4B and 4C show three snapshots in an example
multi-step process of target acquisition and grappling using the
picker head assembly 402 as described, where each step shows two
angles 413, 405 of the same picker head assembly 402, front 413 and
side 405. In an example target acquisition, first, 4A, the picker
head 402 is directed to a target 450 by a seeker subassembly as
discussed herein, using passed coordinates and/or manually steered.
Once directed and in place, a robotic arm (not shown) maneuvers the
picker head 402 into close proximity of the target 450 where the
compression nozzle portion end 430 may attach to the target 450 (as
described in FIG. 3) resting on the ground or surface 401 using low
pressure imparted by the vacuum nozzle 403. In some examples, this
vacuum nozzle 403 may be maneuvered in an extended configuration
444 using the extension actuator 406. In such a configuration, the
compression coupling portion 430 may attach, suction, or otherwise
temporarily hold the target 450 and thereby secure the target 450
with the vacuum suction through the vacuum tube 403.
[0065] Next, 4B, showing same picker head assembly 402, front 413
and side 405, the main nozzle tube 403 may be retracted using the
actuator for the extender 406. In some examples, this is a
generally upward motion 440 away from the ground 401 or surface and
toward the interior of the picker head assembly 402. Some examples
may forego the retraction step and not utilize the extension
actuator 406 and/or may not be configured with one. In examples
where retraction is utilized, with the target 450 attached to the
compression nozzle portion 430 which is now retracted into the
picker head assembly 402, the target 450 may be generally aligned
with however many grappler spoons 412, 414 are fit on the picker
head assembly 402. In some examples, the compression nozzle portion
430 and thereby the target 450 is retracted to align with the
respective spoon cushion portions 420, 422 of the grappler spoons
412, 414, no matter how many grappler spoons are utilized.
[0066] In some examples, as shown through the FIGS. 4A, 4B and 4C,
the spoon actuator 404 may be a piston (hydraulic or pneumatic)
assembly in communication with a computer to receive instructions
and send data and configured to raise and lower a bracket assembly
418 on the picker assembly 402 which may interact with a top end
472, 474 of each spoon arm 412, 414 to pivot each spoon arm 412,
414 about a pivot axis 410, 411, thereby opening and closing, or
pinching the two spoons 420, 422 together, and moving them apart.
In some examples, the spoon arms 412, 414 may include springs in
the pivot axis areas 410, 411, which may bias the spoons in the
closed or pinched position, and the bracket 418 may move in
relation to the top spoon ends 472, 474 to move down to work
against the spring tension to open the spoons, and up to allow the
springs to pinch the spoons 412, 414 together. This actuation may
take place due to the interaction between ramped or angled portions
of the top of the spoon arms 472, 474, and the bracket 418 moving
against the spring tension in the pivot axes 410, 411, pivoting
each arm 412, 414 about its respective axes 410, 411.
[0067] As targets 450 may vary in size and shape, the alignment
with the spoons 412, 414, may be obtained by retracting 440 the
compression nozzle portion 430 so that the rim 434 of the
compression nozzle portion 430 is at a place just above the
respective cushion portions 420, 422 of the grappler spoons 412,
414 thereby ensuring that the respective cushion portions 420, 422
of the grappler spoons 412, 414 are able to pinch together to grasp
442 the target 450 without touching the compression nozzle portion
430 when they are in the closed position. It should be noted that
more detail of example embodiments of the grappler spoon 412, 414,
cushion portions 420, 422 is found in FIGS. 9, 10, 11, 12 and
13.
[0068] In some examples, sensors may be placed on or near the
grappler spoons 412, 414 and/or cushions 420, 422, to sense the
size and/or shape of the target 450. In such examples, a feedback
loop may be used from the sensor data to adjust the distance the
compression nozzle portion 430 is retracted to align the target 50
with the grappler spoons 412, 414. In some examples, such a sensor
may be a light sensor, a laser sensor, a proximity sensor,
piezoelectric pressure sensors, and/or a camera to align the target
450 with the grappler spoons 412, 414.
[0069] In some examples, a pneumatic trash cleaning air jet 424 may
be mounted to the end of the grappler spoon 412, 414 in order to
help clear debris. Such an air jet 424 may include one or more
nozzles that are able to blast jets of air in various directions,
thereby moving, flapping, or otherwise disturbing plants, leaves,
dirt, sticks, stems, or other debris that the system is not trying
to target, but might be in the way of a target. In some examples,
the ends of the grappler spoons 412, 414 themselves may include one
or more nozzles, ports, or holes for air jets to blast debris. In
some examples, the outsides of the grappler spoons 412, 414,
opposite the respective cushion portions 420, 422 may include one
or more nozzles or ports, or holes for air jets to blast
debris.
[0070] In the example of the third step, 4C showing the same picker
head assembly 402, front 413 and side 405, when the compression
nozzle portion 430 and thereby the suctioned target 450 is
retracted 440 off the ground or surface 401 and aligned with the
grappler spoon cushions 420, 422, the gripper spoons 412, 414 may
be actuated by the grappler spoon actuator 404 and squeeze together
442 to grasp the target 450. In some examples, the retractable end
of the baffler 432 may retract above or past the point where the
pincher spoons 412, 414 may pinch together 442 so as to be able to
hold a target 450 with the suction through the pliable baffles
section 432 and the spoons 420, 422 at the same time. Only by
retracting 440 far enough could a target 450 be secured by both
systems at the same time. Such a configuration also allows for a
handoff, for example, the suction may be turned off once the
retracted portion 432 is secured by the pincher spoons 420, 422.
More example of cushioned spoons may be utilized as described
herein in FIGS. 9, 10, 12, and 13.
[0071] Thus, to secure a target 450, the vacuum suction nozzle 403
may include a pliable bellows 430 material that can conform to a
target 450 and secure suction to it. The shape of the holes in the
pliable bellows material, to apply the suction force may include
various designs as discussed in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C,
7D, 8A, 8B, and 8C.
[0072] In this third configuration, the target 450 may be grasped
by the grappler spoons 412, 414, and may still be held by the
compression nozzle portion 430 and the vacuum pressure from the
main nozzle 403. In such examples, a feedback loop may be used from
the sensor data to adjust the pressure used to squeeze the target
450. In some examples, additionally or alternatively, a single
expansion spring (not shown) that may be connected between spoons
412 and 414 may be used. The spring may set the tension that the
grappling spoons 412 and 414 may exert onto the target 450. In some
examples, additionally or alternatively, the sensors may be
piezoelectric pressure sensors on or under the grappler spoon
cushions 420, 422. In some examples, additionally or alternatively,
the sensors may include cameras to visually detect securing the
target 450. In some examples, additionally or alternatively, the
sensors may be tension sensors on the grappler spoon actuator 404
to sense the pressure exerted on the closure of the grappler spoons
412, 414. In some examples, additionally or alternatively, the
sensors may be tension sensors on the gripper spoons 412, 414
and/or in a hinged portion of the gripper spoons 412, 414 to sense
the pressure exerted on the closure of the grappler spoons 412,
414. In some examples, feedback loops may be analyzed by computer
systems in communication with the grappler spoon sensors 412, 414,
and also in communication with the actuators for the grappler
spoons 412, 414 to adjust the pressure on the target 450, which may
be used to secure differently sized targets 450 while minimizing
damage to larger targets 450 and/or making sure smaller targets 450
are secured. It should be noted that further embodiments in
addition to and/or in the alternative of the compression nozzle
portion 430 are discussed in more detail in FIGS. 5, 6A, 6B, 6C,
7A, 7B, 7C, 7D, 8A, 8B, and 8C.
[0073] In some example embodiments, once the gripper spoons 412,
414 have secured the target 450, the vacuum suction may be turned
off by the computer, reduced, or otherwise cut off from the
compression nozzle portion 430 which would in turn release the
pressure holding the target 450 to the compression nozzle portion
430 but leaving the target 450 in control of the grappler spoons
412, 414. In some examples, the padded grappler spoons 412, 414 may
be configured to flick and turn the target 450 to remove them from
their stems and thereby avoid having to cut a stem or plant in any
way. This removal process of a target 450 from a stem may be
advantageous in the shelf life of the target after harvesting and
may be cheaper and easier to accomplish in the field.
[0074] In some examples, this twisting motion may be a 90 degree
twist of the grappler picker head assembly 402. In some examples,
this may be a 180 degree twist. In some examples, this may be
between an 80 degree and 100 degree twist to snap a target 450
stem. In some examples, a snipping element may be used in lieu of
or in addition to the snapping, twisting motion of the grappling
spoons 412, 414. In such examples, a longer target stem may be
desired, and snapping or twisting may remove the stem close to the
target 450. Some example stem cutters are detailed in FIG. 14.
[0075] In some examples, the snapping of the stem by twisting may
benefit if the stem of the target 450 plant is pulled out and away
from the plant in order to impart a strain on the stem. In such
examples, the pulling of the stem first, and then twisting the stem
may result in cleaner and/or more accurate stem snaps. In some
example embodiments, the extension/retraction actuator 406 may
include a sensor that may sense resistance as the target 450 is
retracted. In some examples, tension sensors may be placed in
joints of the robotic arm(s) in communication with the computer
systems, to make such a determination.
[0076] In some examples, the twisting motion may be imparted only
when the resistance of the retraction of the target 450 meets a
particular threshold, as determined by computer systems in
communication with such sensors, thereby indicating that the stem
of the target 450 is under strain or is otherwise stretched. Such
resistance sensors may include a piezoelectric sensor, a strain
gage, or other sensor mounted in or on the retraction actuator 406.
In some examples, the target may be a berry that includes a calyx
portion where a few leaves and the stem attach to the target berry.
In some examples, this calyx portion may be identified by the
seeker subassembly to help determine which direction to pull the
target, normal to the calyx portion. In such examples, the camera
and computer system may be able to identify a color variation
between the berry itself and the calyx leaves and thereby the
stem.
[0077] In snipping examples, a scissors, saw, clipper, or other
sharp pincher may be secured to the picker head assembly 4C to cut
the stem of the target 450 at a desired length. In some examples,
one inch stems may be cut. Further additional or alternative
examples may be found in FIG. 14. After securing the target by the
steps described herein, the robotic arm assembly may then move the
target, and deposit it by releasing the vacuum suction (if used)
and opening the grappling spoons 412, 414, to release the target
into a collection bin, conveyor belt, packing container, or other
device to store, or transport the harvested targets.
[0078] Bellows Pliable Target Contact Examples
[0079] In some examples described herein, a pneumatic vacuum may be
utilized to first secure the picker head assembly 402 to a target
450 such as a strawberry. This vacuum attachment 403 to a target
450 may allow for the picker assembly 402 to extract a target 450
from foliage, pick it off a stem and/or off a resting surface 401.
Further vacuum attachment pliable bellows portions may be found in
FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C below. By
securing a vacuum attachment 430 to a target 450 first, further
even more grappling, twisting, and/or handling, may be accomplished
with spoons 412, 414, as described herein to aid in harvesting and
moving targets for example further spoon examples found in, FIGS.
9, 10, 11, and 12 below.
[0080] In such examples, pliable bellows 430 may be positioned on
the robotic assembly to within range of a specified target 450 and
the bellows section 430 extended 444 to within vacuum range.
Through the bellows tube 403, a negative pressure may be generated
by a pneumatic vacuum, pump, or other air or pneumatic suction
device (not shown) thereby sucking air through the nozzle end 436,
through the bellows tube 403 and thereby through the various ports,
holes, slots, and/or other shaped voids in the end of the bellows
as described in FIGS. 5, 6A, 6B, 6C, 7A, 7B, 7C, 7D, 8A, 8B, and 8C
below. As such bellows 430 material may be made of pliable material
to be able to better conform to the shape of a target more easily
to thereby secure a better vacuum hold on the target.
[0081] Such a task is made more difficult by the variety of shapes,
sizes, and orientation of targets needed to be grappled. Such a
task is also made more difficult by the potential of nearby
foliage, other targets, stems, dirt, sticks, etc. which could
interfere with the vacuum suction, and/or reduce the vacuum
pressure that may be applied to a target by the bellows. The
purpose of the bellows vacuum system is to create enough of a
vacuum attachment to the target such as a strawberry to secure the
target from all or as many orientations as possible, including the
apex point in examples of a strawberry. FIG. 5 shows an example of
what such an interaction with a target X12 might look like from the
inside or behind the bellows contact membrane 510. As shown, the
target 550 is deforming the bellows contact membrane 510 due to the
vacuum pressure being applied in the direction indicated 503 by a
vacuum and tube (not shown). The various holes 508 shown cut or
formed in the membrane 510 allow for the air to be sucked in the
direction shown 503 by the vacuum pump, thereby pulling the target
550 toward the vacuum pressure. FIG. 5 shows the ability of the
bellows membrane 510 to flex, stretch, bend, and conform its shape
to the target 550. This is made possible by using a material that
is both resilient to repeated use and stretching, yet pliable and
soft enough so as not to damage or minimize the damage done to a
target when the vacuum pressure is applied. Such materials may
include but are not limited to silicone, plastic, rubber,
polyurethane, polycarbonate, polyethylene, thermoplastic elastomers
(TPE) such as but not limited to Thermolast K. The material used
for bellows construction may pliable, stretchable, bendable, Food
and Drug Administration compliant, heat resistant, sterilizable,
and able to resiliently return to shape after being stretched. In
some examples, the material used for the bellows may be of a
durometer of 30-40 C with a finished thickness of 0.03 inches at
the contact membrane 508 and an expansion rate of 500%. In some
examples, the thickness at the contact membrane 508 may be between
0.02 and 0.04 inches. In some examples, the thickness at the
contact membrane 508 may be between 0.01 and 0.05 inches. In some
examples, the thickness at the contact membrane 508 may be between
0.025 and 0.035 inches thick. Any variation or range near these
limits may be used. In addition, the base material may be food
grade and/or Food and Drug Administration (FDA) approved.
[0082] As shown, once the target 550 is grappled, the vacuum flow
503 is obstructed or partially obstructed, and the vacuum negative
pressure begins to rise, thereby causing the holes and/or slots in
the contact membrane 508 to stretch open releasing more air flow to
the outer diameter of the bellows 510. Depending on the size of the
target, this may result in a cascading effect whereby the larger
targets 550 stretch open the holes/slots in the contact membrane
508 to allow more attraction area onto the big targets 550 to
offset their increased weight and mass over the small targets. Such
a cascading effect may be most pronounced when the apex of the
berry is engaged due to the pointy nature of a target apex (as
shown in FIGS. 7B and 7D). This self-regulating suction port area
adapts to the irregular shape of targets such as strawberries.
[0083] As can be seen from FIG. 5, the positioning and shape of the
holes formed or cut into the bellows contact membrane may allow for
gentle handling of the target 550 with maximum vacuum pressure
applied to the target. Such a balance of enough pressure to grapple
and handle the target, yet not enough to extract juice, bruise,
damage, or otherwise harm the target may be made by configuring the
holes in the membrane in a particular way so as to distribute the
pressure evenly, allowing the membrane to flex, and allowing the
bellows to grapple targets of various shapes, sizes, and
orientations. In some examples, the bellows may spread the contact
area over 20% to 30% of the target so as to mitigate localized
contact pressure to any one area. Such a contact would thereby
minimize damage to the target, yet maintain a sufficient suction
hold on it. FIGS. 6A, 6B and 6C show example bellows construction
cutaway views and FIGS. 8A, 8B and 8C are diagrams showing example
bellows with different hole shapes and patterns to address these
challenges.
[0084] It should be known that the material used to make the
bellows section may include any of the below or other malleable,
stretchable, and deformable yet resilient materials alone or in
combination: silicone, plastic, rubber, polyurethane,
polycarbonate, polyethylene, thermoplastic elastomers (TPE) such as
but not limited to Thermolast K. Such material may be imparted with
fibers to make them stronger and/or more resilient. Such materials
may be doped with chemicals, vulcanized, sprayed or treated with
chemicals to help with pliability and to prevent cracking or
breakdown. Such material may be food grade material as approved by
the Food and Drug Administration to be able to contact food and for
food handling. Any of the descriptions of the bellows, contact
coupling sections, or other target contact elements may include any
or all, and any combination of these materials, in the shapes,
sizes, and features described here and are not intended to be
limited.
[0085] Turning now to the size, shape, and configuration of the
bellows contact membrane, FIGS. 6A, 6B and 6C are diagrams showing
example cutaways of bellows 612 construction as described in the
embodiments disclosed herein. In FIG. 6A, a cutaway view of the
bellows 612 including the contact membrane 602 are shown. The
vacuum tube 610 is shown with the bellows section 612 secured to it
and/or around the end of it. In the example, the bellows section
612 is friction fit and secured by a rim or matting flare 614 on
the more rigid vacuum tube 610 which interacts with the more
malleable bellows material 612. The bellows section 612 may be
configured to attach to or around the vacuum tube 610 and be
removable, such that it could be quickly and easily replaced in the
field if it is damaged or worn out. Any of the bellows, contact
membranes, etc., described herein may be configured with such a
removable feature set allowing them to be replaced, removed,
cleaned, etc. in the field or maintenance depot.
[0086] In use, the vacuum pump (not shown) may be configured to
impart a suction in the direction shown 603 which would thereby
pull air through the holes in the contact membrane 602 as described
herein.
[0087] The thickness of the material making the bellows section 612
may vary in different sections of the bellows 612. The thickness of
this material may influence how much the material may stretch,
deform, or otherwise shape around a target in use as shown in FIG.
5. For example, the material into which the holes are cut or formed
in the contact membrane 602 may be between 0.02 and 0.04 inches. In
some examples, it may be between 0.01 and 0.05 inches thick. In
some examples, it may be between 0.02 and 0.06 inches thick. The
rim 620 around the contact section 602 may be between 0.03 and 0.04
inches thick. In some examples, the thickness around the contact
section 602 may be between 0.02 and 0.05 inches thick. In some
examples, around the contact section 602 may be between 0.03 inches
and 0.06 inches thick. In some examples, the rim 622 around the
vacuum tube 610 may be between 0.125 and 0.187 inches thick. In
some examples, the rim 622 around the vacuum tube 610 may be
between 0.1 and 0.2 inches thick. In some examples, the rim 622
around the vacuum tube 610 may be between 0.1 and 0.3 inches
thick.
[0088] FIG. 6B shows another cutaway example of a bellows section
632 with contact surface section 634. This is example, the
thickness of the sidewalls of the bellows section 636 is thicker
than that shown in FIG. 6A. In FIG. 6B, the sidewalls 636 are
between 0.125 inches and 0.187 inches thick near where it encircles
the vacuum tube 611. In some examples, the sidewalls 636 are
between 0.1 inches and 0.2 inches thick near where it encircles the
vacuum tube 611. In some examples, the sidewalls 636 tapers to a
thickness of between 0.05 inches and 0.06 inches at the rim 638 of
the ring around the contact membrane 634 with holes to provide the
suction pressure to a target (not shown). In some examples, the
sidewalls 636 tapers to a thickness of between 0.04 inches and 0.07
inches at the rim 638 of the ring around the contact membrane 634.
In some examples, the sidewalls 636 tapers to a thickness of
between 0.03 inches and 0.07 inches at the rim 638 of the ring
around the contact membrane 634. In the example of FIG. 6B, the
thickness of the outer rim 638 becomes thinner near the front of
the rim 639 closer to the contact membrane 634. Such examples could
include thicknesses of between 0.04 inches and 0.05 inches at the
outer portion of the rim 638 and between 0.03 inches and 0.04
inches at the front portion of the rim 639. In some examples, the
thicknesses may be between 0.03 inches and 0.06 inches at the outer
portion of the rim 638 and between 0.02 inches and 0.05 inches at
the front portion of the rim 639. In some examples, the thicknesses
may be between 0.02 inches and 0.07 inches at the outer portion of
the rim 638 and between 0.01 inches and 0.07 inches at the front
portion of the rim 639. Such thinner material at the front portion
639 may allow for grappling more delicate targets, smaller targets,
or lighter targets.
[0089] FIG. 6C shows another cutaway view of a bellows 642 but in
this example, the outer rim 648 and front portion 649 of the rim
around the contact membrane 644 is thicker, between 0.03 and 0.04
inches thick. In some examples, the front portion 649 of the rim
around the contact membrane 644 may be between 0.02 and 0.07 inches
thick. In some examples, the front portion 649 of the rim around
the contact membrane 644 may be between 0.02 and 0.1 inches thick.
The different thickness in the material for the bellows 632 may
allow for a more robust and/or longer work life for the bellows
section which may be replaced if damaged or worn out, as described
herein.
[0090] FIGS. 7A, 7B, 7C, and 7D are diagrams showing example
bellows interacting with differently sized targets as described in
the embodiments disclosed herein. In FIG. 7A, both a cutaway view
712 and a perspective view 722 of a medium sized target 750
interacting with the bellows with the vacuum pressure 703 pulling
the target 750 toward the bellows 712, 722. In the example, the
target 750 is medium sized and fits within the overall main section
of the contact membrane 702 when grappled from the side. The outer
rim of the bellows 704 does not come into contact with the target
750 just because of the size of the target 750 compared to the
bellows 712 width.
[0091] In FIG. 7B, the cutaway view 732 of the bellows shows the
target 751 contacted at the apex 752 point portion by the contact
membrane 732. As in FIG. 5, the contact membrane 732 is shown
deformed or stretched to secure the target 751 and hold it due to
the vacuum pressure 703. In the example, the rim 734 does not
contact the target 751 but surrounds it because of the size and
shape. The perspective view 742 shows the same target 751 secured
in the contact membrane 732 surrounded by the rim 724.
[0092] FIG. 7C shows a cutaway of the bellows 756 and a perspective
of the bellows 758 grappling a larger target 752. Because the
example target is large, both the contact membrane 760 and rim 762
contact the side of the target 752 and draw it toward the bellows
756, 758 toward the direction 703 of the vacuum pressure through
holes in the contact membrane 760. This figure demonstrates how
even larger targets may be grappled by the bellows 756, 758 and
still maintain a good vacuum seal on the target 752 to harvest and
move it, as described herein.
[0093] In FIG. 7D, the cutaway view 772 of the bellows shows the
target 753 contacted at the apex 754 point portion by the contact
membrane 774. As in FIG. 5, the contact membrane 774 is shown
deformed or stretched to secure the target 753 and hold it due to
the vacuum pressure 703. In the example, the rim 776 contacts the
edges of target 753 and surrounds it because of the size and shape.
The perspective view 782 shows the same target 753 secured in the
contact membrane 774 surrounded by the rim 776. In this figure, it
can be seen how even larger targets 753 may be grappled, harvested
and moved, from various orientations including the apex 754.
[0094] FIG. 8A shows two examples 802, 820 of the end of a bellows
unit, the contact membrane where the holes are made in the pliable
material, allowing the air to flow through and thereby suction
pressure to be generated on a target as shown in FIG. 5. The end-on
examples of the contact membrane 804, show various holes and slots
in the pliable material which may effectuate different pressures on
a target when applied with a vacuum pressure.
[0095] The first example 802 in FIG. 8A shows the main bellows
contact membrane circular shape 804 along with the interior rim 806
of the bellows section. Between the outside 804 and the interior
rim 806, the bellows contact membrane have no holes or other air
ports. This section between outside 804 and the interior rim 806
may be configured as an annular rim which may cushion and secure to
the target when applied in use, and may be any of various materials
and/or thicknesses as described in more detail in FIGS. 6A, 6B and
6C. Inside this rim between 804 and 806 is the section with the
patterned holes allowing air to flow in use. In the first example
of 802, the membrane includes annular positioned slots 812 ringing
the interior of the membrane in generally concentric circle shapes.
Regarding all of the contact membranes of the drawings here, in
use, the initial contact area of the target or berry covers over
the innermost holes or parts. At that time, the vacuum flow is at
least partially obstructed and the vacuum negative pressure begins
to rise, thereby causing the circular slots to stretch open
allowing more air flow to the outer diameter of the bellow rings.
Such a situation may have a cascading effect on larger targets
whereby the larger target berries stretch open the circular slots
more and thereby allow more attraction area and airflow which may
allow for harvesting targets with increased weight and mass. In
such examples, smaller target berries may not stretch the contact
surface as much as the larger targets, and thereby not increase the
air flow as much or the contact surface, which is unnecessary due
to smaller targets having smaller mass and weight.
[0096] In some examples, the cascading effect may be most
pronounced when an apex, point, or tip of a target berry is engaged
as shown below in FIGS. 7B and 7D. This self-regulating suction
port area lends itself well to adapting to the irregular shape of
target berries. In the various examples of contact membranes and
shapes of holes and slots therein, any of various arrangements may
be made to aid in contacting, attracting by negative suction power,
and holding, yet not damaging the target. The examples listed here
may be combined in any way and may be configured in any combination
of the embodiments listed or with other combination elements.
[0097] The slots 812 are elongated between 0.12 inches and 0.14
inches in length and between 0.04 inches and 0.05 inches wide. In
some examples, the slots 812 are elongated between 0.1 inches and
0.2 inches in length and between 0.02 inches and 0.08 inches wide.
The example shows multiple annular rings of slots, four in this
example, but could be any number with the most interior ring 814
shown toward the middle of the membrane 802. In the example, the
rings of increasingly smaller slots between 812 and 814 include
approximately the same sized and shaped slots, with generally
rounded ends and a generally consistent thickness, except for the
decreasing diameter of the overall ring shape they make, and are
overlapping in their positioning. That is to say, the most outside
ring of slots 812 each ends in an approximate mid-point for the
next interior ring of slots, and then again overlapping each ring
until the most interior ring 812. The number of slots on each ring
812, 814 may depend on the size and shape of each slot and the
spacing between rings may vary between 0.04 and 0.06 inches. In
some examples, the spacing between rings may vary between 0.02 and
0.09 inches. In some examples, the spacing between rings may vary
between 0.02 and 0.125 inches.
[0098] In the example of FIG. 8A, inside the rings of slots 812 to
814, a center most section includes slots that are not annularly
placed around the middle as the rings 812 are, but slots 817, 816
angled toward the center of the membrane 804 with a central hole
822. The sizes and shapes of these most central slots 817, 816 may
vary between 0.03 and 0.06 inches in width. In some examples, the
sizes and shapes of these most central slots 817, 816 may vary
between 0.02 and 0.08 inches in width. In some examples,
alternating centrally points slots 817 are longer than others 816.
In some examples, the longer slots 817 may be between 0.2 and 0.21
inches long, and 0.03 and 0.06 inches wide. In some examples, the
longer slots 817 may be between 0.1 and 0.3 inches long, and 0.02
and 0.09 inches wide.
[0099] In some examples, the centrally pointing slots 817, 816 may
be pie shaped or wedge shaped with the most exterior portions wider
than the more interior portions. In some examples, the centrally
pointing slots may be between 0.08 and 0.09 inches wide at the
exterior portion and between 0.04 and 0.05 inches wide at the end
closest to the center. In some examples, the centrally pointing
slots may be between 0.04 and 0.125 inches wide at the exterior
portion and between 0.02 and 0.09 inches wide at the end closest to
the center. In some examples, the centermost hole 822 is circularly
shaped and 0.12 inches in diameter. In some examples, the
centermost hole 822 is circularly shaped and between 0.1 inches and
0.2 inches in diameter.
[0100] The next example of hole patterns in the bellows contact
membrane 820 includes similar annularly placed slots 824 in
concentric circle shapes, similar to the design in 802 but the
interior holes are shown as circularly shaped 826 instead of
elongated slots. In the example of 820 the interior of the contact
membrane includes two concentric rings of circularly shaped holes
826 and one central hole 828. In some examples, the diameters of
the concentric ring circles 826 may be between 0.09 and 0.1 inches.
In some examples, the diameters of the concentric ring circles 826
may be between 0.05 and 0.2 inches. In some examples, the central
hole 828 may be between 0.12 and 0.13 inches in diameter. In some
examples, the central hole 828 may be between 0.07 and 0.2 inches
in diameter.
[0101] FIG. 8B shows two more examples of contact membranes 840 and
860. In 840 example the concentric ring of slots 842 may be
overlapping and include dimensions of between 0.13 and 0.14 inches
long, between 0.03 and 0.05 inches wide and have a spacing between
rings or between 0.06 and 0.075 inches. In some examples, 840
example concentric ring of slots 842 may be overlapping and include
dimensions of between 0.08 and 0.2 inches long, between 0.1 and
0.09 inches wide and have a spacing between rings or between 0.02
and 0.125 inches. The example shows the overlapping pattern shown
above where the midpoint of one slot approximately lines up with
the end or the spacing between slots in the next ring inward or
outward, whichever pertains. In the example of 840, the interior
concentric ring holes are oval shaped 844 with the apex pointing
toward the central hole 846. In some examples, the outer ring 848
of the inner oval shapes is thicker with a width between 0.1 inches
and 0.12 inches. In some examples, the outer ring 848 of the inner
oval shapes may have a width between 0.05 inches and 0.2 inches. In
some examples, the inner of the two oval rings 848 includes ovals
with thinner widths between 0.08 and 0.1 inches. In some examples,
the inner of the two oval rings 848 includes ovals have widths
between 0.05 and 0.25 inches. In some examples, the oval rings may
have their apexes pointed around the ring instead of toward the
center hole 846 as shown. In some examples, the thicker ovals may
be positioned in the interior of the two oval rings and the thinner
width ovals may be positioned on the outer of the two inner oval
rings. In some examples, the number of rings may be one, two,
three, four, five, or more rings of holes.
[0102] In the example 860, the concentric ring of slots 862
includes slots with wider widths than 840. In some examples, the
widths of slots 862 is between 0.05 and 0.07 inches wide. In some
examples, the widths of slots 862 may be between 0.02 and 0.09
inches wide. In some examples, the number of concentric rings of
slots is three as shown. In some examples, the number of concentric
rings slots is two, one, four, or more (not shown). In some
examples, the concentric rings of ovals 864, 868 may be between
0.02 and 0.09 inches wide. In some examples, the central hole 866
is between 0.125 inches and 0.014 inches in diameter.
[0103] FIG. 8C shows another embodiment of a contact membrane 870
of the bellows with the outer rim 872, but the example shows the
concentric rings of slots 874, 876, etc. are not shaped as those in
FIGS. 8A and 8B with generally rounded ends and a generally
consistent thickness, instead, the thickness varies to make the
slots more thick on one side than the other as in 874 and in some
examples, even include a side with a pointy, triangular shape, bean
shape, or other 877. In some examples, the ring of concentric slots
may be similarly shaped to those in FIGS. 8A and 8B with a
generally consistent thickness 878. Any combination of these shapes
or other shapes may be configured as described here.
[0104] Spoon Examples
[0105] As shown in FIG. 3, the cushion or spoon portions 320, 322,
of the gripper pincher arms 312, 314 may include different
embodiments and combinations in order to help secure yet cushion
the agricultural target for harvesting. Therefore, such cushions
may be firm yet flexible, made of FDA compliant material for food
handling, yet rugged for outdoor use.
[0106] And because the cushions or spoons may need to be flexible
to be soft, they may wear out or get damaged. They may become dirty
or spoiled. In some examples below, the spoons may include pads or
membranes that may allow for removal of the pads for cleaning or
fixing, or replacement of pad material for different targets. Below
are some examples, alone or in combination, which may be used as
cushions, spoons, pads, or otherwise agricultural target grapplers
for the harvester.
[0107] FIG. 9 is a diagram showing example spoon grappler
construction as described in the embodiments disclosed herein. As
shown in FIG. 9, one of potentially multiple spoon portions is
shown attached to a flange 910, which in turn would be attached to
the grapplers and actuators as shown in FIG. 3 etc. The spoon
portion may include a rigid outer rim 902 and a rigid inner rim 904
attached to the flange 910. In some examples, the two rims 902, 904
are separated from one another 906 with a gap or space 920 between
the two. The target to be grappled would be captured within this
gap 920, cushioned by the material covering the outer rim 902 and
surrounding the padding 922 of the inner rim 904.
[0108] The two rims 902, 904 may each be wrapped in or otherwise
covered include soft, pliable material that would not damage the
targets. In some examples, the outer rim 902 may be completely
covered with a plastic, neoprene, or other soft and deformable yet
resilient and FDA compliant material that would cushion the target.
The inner rim 904 may include a soft and pliable rim section 922
similar to an earphone with padding and soft yet resilient
cushioning. Together, the material covering the outer rim 902 and
the padding around the inner rim 904 may work together to gently
yet firmly attach to a target, and squeeze against another spoon
arrangement, configured as a mirror image to that shown in FIG. 9,
to hold a target, along with or in conjunction with the vacuum and
bellows assembly as described herein.
[0109] FIG. 10 is a diagram showing more example spoon grappler
construction as described in the embodiments disclosed herein
including a top down view 1030 and a front view 1040. In the
example, the outer rim 1002 and inner rim 1004 of the spoon section
are shown attached to the flange 1010. The front view shows the
inner rim padding 1022 and the interior space 1020 formed inside
the padding of the inner rim 1022 portion like an earphone
configuration. The target would be captured between the inner rim
1004 padding 1022 cushioned by material covering the outer rim
1002, and a second spoon assembly (not shown) which could squeeze
together gently yet firmly enough to secure a target.
[0110] FIG. 11 is a diagram showing example spoon grappler cutaway
construction as described in the embodiments disclosed herein. In
the example of FIG. 11, the flange 1110 is attached to only one rim
1102 upon which a resilient yet pliable membrane 1104 is attached.
The membrane 1104 includes a thin section 1122 which may contact
the target when assembled. The rear of the assembly 1130 may be
made of a firmer or harder material than the membrane 1104 and
would serve as a structural support for the rim 1102. In some
examples, the membrane 1104 would be able to be replaced by a
friction fit around the rim 1102 would be able to be replaced in
the field in order to repair damaged membranes 1104 or change them
due to weather conditions, or targets. In some examples, the
membrane material 1104 may be any of those described herein for the
bellows, or spoon membranes in any combination.
[0111] FIG. 12 is a diagram showing an example spoon grappler
interacting with a target as described in the embodiments disclosed
herein. FIG. 12 shows an end on view 1230, a frontal view 1240 and
a perspective view 1242 of a grappler spoon with a flat membrane
embodiment. The flange 1210 attaching the spoon to the picker head
assembly (not shown) is shown attached to a rim section 1204
covered by a membrane 1222. The spoon is shown grappling or
interacting with a target 1250. The spoon in use, would pinch the
target 1250 between itself and at least a second spoon (not shown)
and/or the vacuum bellows arrangement to hold and harvest a target.
As the membrane 1222 is made of soft, pliable, yet resilient
material, it may deflect when grappling the target 1250 to cushion
it softly, yet firmly enough to hold it while snapping the stem or
otherwise removing it from the plant (not shown).
[0112] FIG. 13 also shows a perspective 1342, end on 1330 and
frontal 1340 view of a spoon embodiment. FIG. 13 shows a similar
arrangement as FIG. 12 but with a larger target 1350 interacting
with the membrane 1322 of the spoon on the rim 1304 and flange
1310. As can be seen from FIG. 13, even a larger target 1350 is
able to be grasped by the spoon rim 1304 and membrane 1322 to
secure it.
[0113] The material used for membranes for the spoons in FIGS. 9,
10, 11, 12, and/or 13 may be similar to, or the same as those for
the bellows, for example, they may be pliable, stretchable,
bendable, Food and Drug Administration compliant, heat resistant,
sterilizable, and able to resiliently return to shape after being
stretched. In some examples, the material used for the membranes of
the spoons may be of a durometer of 30-40 C with a finished
thickness of 0.03 inches at the contact membrane and an expansion
rate of 500%. In some examples, the thickness at the contact
membrane may be between 0.02 and 0.04 inches. In some examples, the
thickness at the contact membrane may be between 0.01 and 0.05
inches. In some examples, the thickness at the contact membrane may
be between 0.025 and 0.035 inches thick. Any variation or range
near these limits may be used. In addition, the base material may
be food grade and/or Food and Drug Administration (FDA)
approved.
[0114] Stem Cutter Examples
[0115] FIG. 14 shows an example of a stem cutter embodiment 1402
which may be used in conjunction with a picker head assembly 1420.
In the example, the cutter assembly 1402 may slide up and down 1430
relative to and outside the picker head assembly 1420 as a sleeve,
wherein the vacuum bellows portion may extend through 1440 the stem
cutter assembly 1420 to attach to a target, then retract 1442 until
the stem cutter 1402 contacts the stem to cut it.
[0116] The example stem cutter shows a rotating 1412 motion for the
stem cutter 1402 to cut or shear the stem with saw teeth 1422. In
the example, the stem cutter 1402 may include a nested or second
cutter 1404 which may rotate counter to one another to impart a
shearing force on a stem (not shown).
[0117] The spinning or turning of the stem cutter assembly 1402
and/or 1404 may be imparted using a rotating motor assembly (not
shown) that may attach to the stem cutter sleeves 1402, 1404 to
spin them 1412. Such spinning may not need to be a full 360 degree
rotation, but in some examples, may be 10 degrees, 30 degrees, 60
degrees, or 180 degrees. In some examples, a spinning of the inner
cutting sleeve 1404 and the outer cutting sleeve 1402 in opposite
directions would be enough to slice, cut or otherwise sever a stem
from a target attached to the vacuum bellows 1440 when placed in
contact with one another. Such spinning may be accomplished and
controlled by a computer system programmed to slice stems when the
picker head assembly pulls the target from its plant. In some
examples, as described herein, a tension sensor may be utilized on
the robotic arm assembly, in a joint, to sense that the target is
being pulled and resisting being pulled from the plant, and at a
predetermined tension threshold, activate the spinning stem cutters
1402, and./or 1404 to cut the stem.
[0118] Seeker/Sensor Subassemblies
[0119] In some examples, the harvesting described herein is
directed by a seeker subassembly that is able to identify targets
for harvesting, pass coordinates for the targets to the picker
subassembly for extraction. Such seeker subassemblies may include
any number of cameras (visible light, thermal, UV or other),
radars, lidars, lasers, acoustic location finders, GPS, inertial
navigation systems, piezoelectric sensors, and/or any combination
of these or other sensors to locate and identify targets.
[0120] In some examples, a seeker subassembly vehicle works
independently from the harvester subassembly, and in some examples,
the two subassemblies are on the same traversing machine. In some
examples, the harvesting subassembly has its own wheels and/or
tracks or combination of both, to traverse down a row of
agriculture and harvest the mapped targets it receives from the
seeker subassembly. In some examples, the seeker subassembly
vehicle and harvesting subassembly vehicle are able to mate,
connect, and/or otherwise work in concert by connection. In some
examples, this connection includes a wired connection to allow for
target information to be passed from the seeker subassembly to the
harvesting subassembly. In some examples, the two subassemblies
communicate wirelessly. In some examples, a combination of wired
and wireless communication may be arranged.
[0121] In some examples, either or both the subassemblies may
include location sensing and determining devices. In some examples,
GPS location sensors may be configured on both or either
subassemblies for the computer systems to determine locations
and/or steer. In some examples, cellular towers and signals may be
used for location sensing by the subassemblies. In some examples,
inertial navigations systems may be used such as a ring laser gyro,
a magnetic gyro and/or any other combination of such with a
computer system. In some examples, additionally or alternatively,
the cameras in the seeker subassembly and/or other cameras on the
harvesting subassembly may be used to identify and track an
agricultural row down which the vehicle may be steered. The
location sensing and/or steering may be fed into any computer
system, either located on the harvesting/seeking systems or
remotely, in order to autonomously, semi-autonomously and/or allow
for human activated remote steering. Any combination of these or
other systems may be used to locate and/or steer the systems
here.
[0122] The initial configuration is intended to utilize
self-steering on row with the intention to utilize a droid tender
(person) to steer the droid off-row for unloading accumulated berry
containers and reloading empty containers, then finally steering
the droid back onto a new row to be picked. The droid will have the
ability to be converted to full autonomous mode for turnaround at
the head lands as well as unloading and loading berry
containers.
[0123] In some examples, the seeker subassembly may include one or
more robotic arms with sensor(s) attached. In such examples, the
robotic arms may include joints which provide multiple degrees of
freedom of movement. Such multiple degrees of freedom may be useful
to move in, on, around, and through foliage of target agriculture
to find the target to be harvested, for example, a berry within the
foliage of a shrub or plant.
[0124] In some example embodiments, additionally or alternatively,
the seeker subassembly robotic arms may include at least one camera
as described herein. In some examples, the seeker subassembly
robotic arms may include at least one light system as described
herein. In some example embodiments, a single robotic arm may
include a multitude of cameras and light systems. In some example
embodiments, additionally or alternatively, the cameras and/or
lights may be integrate into the harvesting robotic arms. For
example, referring to FIG. 1, in some examples, the picker 102 on
the end of the robotic arm 160 could include a camera and light
system. In some examples, the sensors and/or light systems may be
attached to the frame 153 of the system.
[0125] In some examples, a cascade of hierarchal cameras may be
employed on the systems. In such examples, larger scope or angled
cameras may be used to identify one or more targets from a wide
angle. In such examples, a first coordinate mapping may be
calculated using the wide angle lens cameras. In such systems, the
back end computers may receive the first coordinate or mapped
information and use that to focus a second camera system on the
selected targets for a more refined targeting. The second set of
narrower angle cameras may be configured to hone in on the targets
that the wide angle system first mapped, and refine or detail a
tighter set of coordinates for each target. This arrangement of
passing from wide angle camera systems to a second set of narrower
camera systems may allow for a tight control loop for the picker
assemblies.
[0126] In some examples, one or more laser sensors may be
configured on the systems to find, locate, and map targets. In some
examples, lasers may be employed to augment other camera assemblies
to illuminate targets for cameras with matched wavelength receptors
to capture images. In some examples, lasers may be used
exclusively. In some examples, each picker head may include its own
laser system to be used as a range finder, a color differentiator,
and/or other sensor for the final picking action at the target
itself.
[0127] In some example embodiments, additionally or alternatively,
the seeker subassembly and/or harvesting subassembly robotic arms
include at least one foliage moving flipper or pneumatic air jets
configured to alter, move, displace, and or otherwise gently
maneuver the foliage of the plant to better expose the berries.
[0128] In some example embodiments, the seeker subassembly may
include a camera and/or multiple cameras arranged so as to be able
to view the target foliage and thereby the target agriculture to be
harvested. In some examples, multiple cameras may be arranged on
the seeker subassembly such that images taken from the multiple
cameras may be processed by a computer system to create three
dimensional (3-D) images using machine vision. In some examples,
these images are made of pixels and the computer systems is able to
identify targets represented by pixels to be harvested and map the
targets in three dimensions. In some examples, the cameras may be
configured to acquire multi-spectral or hyper-spectral imagery to
enable the use of advanced analysis algorithms for evaluating fruit
health, quality and ripeness. In some examples, the images gathered
may include those of a thermal imaging system for evaluating the
temperature of the berry to be harvested. These cameras may
comprise of cooled or uncooled sensors generating area-scanned
images of at least 640.times.480 pixels. Some embodiments may
utilize a single thermopile based sensor to provide an integrated
temperature measurement of the mean temperature of the target such
as a berry.
[0129] To deal with leaves, stems and trash obstructing access to
the targeted berry, the berry grappler may include the ability to
recognize obstructions and perform a second operation to clear
access to the targeted berry. The berry grappler may be equipped
with a trash diverter located at its most forward tip of the
grappler spoons. The trash diverter may be equipped with air jets,
or foliage hook device, or rotating paddle device and other
appropriate methods to displace the obstruction. In operation, the
berry grappler trash diverter may move in a progressive diameter
rotation around the targeted berry location, clearing the
obstructions. This second operation may only take place when
obstructions are viewed by the stereo cameras.
[0130] In some examples, the three dimensional image data processed
by and sent from the camera(s) may allow for a virtual reality
environment to be created for a human user. In such examples, a
virtual reality headset or display may be utilized by a user,
remote or close to the harvester, to identify target agriculture
and thereby send the target mapping coordinates to the harvesting
machine for harvesting.
[0131] In some examples, data created by the cameras and data
created by the human selection of agriculture may be stored by a
computer device. In such examples, the identification data may be
amassed in order to analyze and later create algorithms for neural
network engines to process. In such examples, after much data of
targeting, identification, and harvesting information is gathered,
an neural network engine can be trained may be able to replicate
some or all of the human targeting using the three dimensional
maps.
[0132] Examples of cameras which may be used in the described
systems include stereo vision with resolution of 1920.times.1080
and frame rates of 30 per second. Some examples include stereo
vision with resolution of between 1500-2000.times. between
1000-1200 and frame rates between 10 and 50 per second.
[0133] Lighting Examples
[0134] In some example embodiments, the seeker subassembly includes
various specialized lighting which may be used to find and identify
targets. Such lights may be configured on the ends of robotic arms,
integrated into robotic arms that include picker heads, or cameras.
Examples are shown in FIG. 1. Such lights may be fixed onto other
sub-assemblies on the seeker assembly and/or harvesting
sub-assembly.
[0135] In some examples, such specialized lighting may be
configured to emit a certain wavelength or spectrum of wavelengths
such as but not limited to visible light, infra-red light, and/or
ultra-violet light. In some examples, the lighting may be at a
wavelength that excites items to fluoresce. In some example
embodiments, light spectrum filters may be used by the cameras
described herein to filter out or delete wave lengths of light that
would otherwise block out any fluorescent properties reflected or
emitted by targets such as berries.
[0136] In some examples, the specialized lighting may be light
emitting diodes which are tuned to emit light at a specific
frequency. In some examples, that frequency may be a combination of
470 nm (blue) and 635 nm (red). In some examples, the lights may be
LED lights. In some examples, the lights may be incandescent
lights. In some examples, the lights may be halogen lights,
fluorescent lights, metal-halide, neon, high-intensity discharge
lamps, or any permutation or combination of any of the above.
[0137] Mapping and Passing Target Coordinates
[0138] In some example embodiments, additionally or alternatively,
sensors onboard the harvesting systems such as machine vision
camera and computer systems may be used to map target agriculture
in three dimensions and pass the coordinates to the harvester
subassembly for harvesting. These mapping coordinates may be
described in a global coordinate system such as Universal
Transverse Mercader (UTM), or a local coordinate system frame
relative to the coordinate system defined by the three dimensional
imaging system on the harvester. In some examples, an X,Y,Z
coordinate system may be employed using an anchor point in the
camera view and/or on the traversing machine itself.
[0139] The various sensors described herein including but not
limited to visible light cameras, infrared cameras, ultraviolet
cameras, lidars, radars, lasers, or other sensors may be used to
scan the produce plants and identify targets. Using the automated,
semi-automated, or manual selection processes and systems described
herein, the systems could generate coordinates for selected
targets. These mapped target coordinates may then be queued in a
buffer or database, for the harvester subassembly to harvest. In
some examples, after one coordinate is added to the harvesting
coordinate queue, more targets may be added to the queue to be
harvested in turn. In such examples, the targeting subassembly,
machine vision, and target mapping may occur without lag or delay
in the handoff from targeting to harvesting, and not be hampered by
the limitations of the harvesting subassembly itself. In such a
way, in some examples additionally or alternatively, the targeting
subassembly may be mounted on a separate vehicle to travel at its
own speed and send targeting mapped data to the harvesting
subassembly by wireless communications. In some examples, the
targeting subassembly may be a part of the overall machine and
connected to or in communication with the harvesting subassembly to
pass the targeting mapped coordinate queue by wired communications
to the harvesting subassembly. In some examples, a cloud or
distributed computing resource may be utilized so that the
targeting queue may be relayed or sent to the harvesting
subassembly wirelessly.
[0140] In some examples, the mapping may be done early or before a
harvester machine may come down a row. Additionally or
alternatively, in some examples, mapping may be done just before
harvesting, on the same machine in some examples to minimize the
variables of the berries and/or foliage moving. Any time between
target mapping and harvesting may be utilized, depending on the
circumstances of the harvest.
[0141] In some examples, mapping information may be stored in a
remote server, cloud server, or distributed system, for the purpose
of future analysis (post processing) of the imagery to evaluate the
condition of the plant. Post processing operations may include an
evaluation of the plant for disease, nutrient deficiency, unripe
berry inventory, and/or other plant damage. Data gathering and
analysis on all types of agricultural specifics may be accomplished
using the suite of cameras and/or sensors on the systems described
herein. For example, outputs of post processing operations may be
utilized to selectively address in-field issues at a plant-local
scale that may otherwise require broad remedies using traditional
methods. Other outputs of post processing operations may generate
statistical data related to observations and measurements that are
made while the harvester is operating in the field that can be
advantageous to the growers business efforts.
[0142] Automation and Remote Examples
[0143] Additionally or alternatively, the systems described here
may be used to harvest agricultural targets in an automated,
semi-automated, or even manually controlled manner. In some
examples, the semi-automated manner may be arranged in a remote
setting, allowing for a human to interact with camera views from
the harvester to help target the produce.
[0144] The variations on these options depend on how much a remote
or local computing system may be programmed to identify and harvest
a target. For example, in a fully manually controlled system, a
human operator may control the movements of both the seeker system
and the harvesting system. In such examples, by remote control
using a joystick or other computer driven operating device(s) a
human could scan the rows of plants for a target using the camera
systems, and even maneuver the robotic arms that the camera systems
are connected to, to identify targets, and then use a control
system such as a joystick to maneuver the picker head assembly to
the target, and then harvest the target as described herein. Such
examples would allow for remote operation of the systems such as by
wireless control to allow for human controllers to be stationed
anywhere in the world, through some kind of wireless uplink.
[0145] The other extreme of control systems would be a fully
automated system. In such a system, the traversing machines would
move down a row of agricultural targets and the seeker subassembly
would use machine learning/artificial intelligence/neural
networks/and/or other programming to seek out and identify targets
with the seeker subassemblies and then harvest them as described
using the picker heads. Such examples would depend on computer
algorithms and programs to determine using the inputs from the
cameras and sensors, what a target may be and where they are
located. For example, a color camera may be used by the computer
system to detect a red strawberry amongst the green foliage of the
plant it is growing on. Then a laser system could be used to
determine a proximate location and range from the system and the
computers could use that information to triangulate a three
dimensional coordinate system and identify where the target is
located in space, relative to the traversing machine. Next, the
coordinates could be passed to the harvesting subassembly where the
picker heads may attach to and harvest the target strawberry, in
some examples using its own sensors such as cameras and lasers.
[0146] The middle-ground option between the fully automated and the
manually controlled system would be some variant of semi-automated
seeking and harvesting. The degree of semi-autonomy and which
portions were automated and which manually controlled could vary
from separate subassemblies. For example, the seeker subassembly
may be more manually controlled with a human interacting with the
cameras and sensors to help identify targets. In some examples,
that may include a human interacting with a graphical user
interface "GUI" such as a touchscreen to identify a target
displayed on the screen
[0147] In any of the above examples of automation, the sensors
onboard the harvesting system may be used to create, track and pass
coordinates of the targets for harvesting.
[0148] Example Computer Device(s)
[0149] In example systems described herein, various computer
components may be utilized to operate the systems. For example, a
communication computer system may allow for remote operation of the
machines, sensors may send information to a computer system to help
differentiate targets from non-targets, target location and mapping
information may be calculated, stored, sent, and utilized between
the seeker systems and harvesting systems, steering and driving
instructions may be calculated and utilized, machine
learning/artificial intelligence/and/or neural networks may be
employed by computer systems to find and harvest targets, and any
of the other computer operations as described herein.
[0150] In some examples, alternatively or additionally, a WiFi
system/cellular system/Bluetooth system, or any other communication
system, with the appropriate antenna system and a processor and
memory as described herein, may be used on a subassembly. In some
embodiments, alternatively or additionally, the hardware may
include a single integrated circuit containing a processor core,
memory, and programmable input/output peripherals.
[0151] In some examples, various computer components may be used in
the seeker and/or harvesting subassemblies, as well as the
communication systems, control systems, and/or any other portion of
the systems described herein.
[0152] FIG. 15 shows an example computer device 1500 that may be
used in practicing example embodiments described herein. Such
computer device 1500 may be the back end server systems use to
interface with the network, receive and analyzed data, as well as
generate test result GUIs. Such computer 1500 may be a mobile
device used to create and send data, as well as receive and cause
display of GUIs representing data. In FIG. 15, the computer device
could be a smartphone, a laptop, tablet computer, server computer,
or any other kind of computer device. The example shows a processor
CPU 1510 which could be any number of processors in communication
via a bus 1512 or other communication with a user interface 1514.
The user interface 1514 could include any number of display devices
1518 such as a screen. The user interface also includes an input
such as a touchscreen, keyboard, mouse, pointer, buttons, joystick
or other input devices. Also included is a network interface 1520
which may be used to interface with any wireless or wired network
in order to transmit and receive data. Such an interface may allow
for a smartphone, for example, to interface a cellular network
and/or WiFi network and thereby the Internet. The example computer
device 1500 also shows peripherals 1524 which could include any
number of other additional features such as but not limited to
cameras, sensors 1525, and/or antennae 1526 for communicating
wirelessly such as over cellular, WiFi, NFC, Bluetooth, infrared,
or any combination of these or other wireless communications. The
computer device 1500 also includes a memory 1522 which includes any
number of operations executable by the processor 1510. The memory
in FIG. 15 shows an operating system 1532, network communication
module 1534, instructions for other tasks 1538 and applications
1538 such as send/receive message data 1540 and/or SMS text message
applications 1542. Also included in the example is for data storage
1558. Such data storage may include data tables 1560, transaction
logs 1562, user data 1564 and/or encryption data 1570. The computer
device 1500 also include one or more graphical processing units
(GPUs) for the purposes of accelerating in hardware computationally
intensive tasks such as execution and or evaluation of the neural
network engine and enhanced image exploitation algorithms operating
on the multi-modal imagery collected. The computer device 1500 may
also include one or more reconfigurable hardware elements such as a
field programmable gate array (FPGA) for the purposes of hardware
acceleration of computationally intensive tasks.
[0153] The computer architecture for the harvester can be described
as a distributed computer system comprising of elements or
processing centers that exist on the harvester, a central server
system which may or may not be a cloud based resource and an
operator processing system. Each of these processing centers are
interconnected through an IP network which may include local
private wireless networks, private wide area networks and/or public
networks such as the Internet. Computational tasks are divided such
that real-time tasks are executed on the local harvester processor,
post-processing operations and non-real time computation are
executed on the central server and user-interface computation are
performed on the operator processing center.
CONCLUSION
[0154] As disclosed herein, features consistent with the present
inventions may be implemented by computer-hardware, software and/or
firmware. For example, the systems and methods disclosed herein may
be embodied in various forms including, for example, a data
processor, such as a computer that also includes a database,
digital electronic circuitry, firmware, software, computer
networks, servers, or in combinations of them. Further, while some
of the disclosed implementations describe specific hardware
components, systems and methods consistent with the innovations
herein may be implemented with any combination of hardware,
software and/or firmware. Moreover, the above-noted features and
other aspects and principles of the innovations herein may be
implemented in various environments. Such environments and related
applications may be specially constructed for performing the
various routines, processes and/or operations according to the
invention or they may include a computer or computing platform
selectively activated or reconfigured by code to provide the
necessary functionality. The processes disclosed herein are not
inherently related to any particular computer, network,
architecture, environment, or other apparatus, and may be
implemented by a suitable combination of hardware, software, and/or
firmware. For example, various machines may be used with programs
written in accordance with teachings of the invention, or it may be
more convenient to construct a specialized apparatus or system to
perform the required methods and techniques.
[0155] Aspects of the method and system described herein, such as
the logic, may be implemented as functionality programmed into any
of a variety of circuitry, including programmable logic devices
("PLDs"), such as field programmable gate arrays ("FPGAs"),
programmable array logic ("PAL") devices, electrically programmable
logic and memory devices and standard cell-based devices, as well
as application specific integrated circuits. Some other
possibilities for implementing aspects include: memory devices,
microcontrollers with memory (such as 1PROM), embedded
microprocessors, Graphics Processing Units (GPUs), firmware,
software, etc. Furthermore, aspects may be embodied in
microprocessors having software-based circuit emulation, discrete
logic (sequential and combinatorial), custom devices, fuzzy
(neural) logic, quantum devices, and hybrids of any of the above
device types. The underlying device technologies may be provided in
a variety of component types, e.g., metal-oxide semiconductor
field-effect transistor ("MOSFET") technologies like complementary
metal-oxide semiconductor ("CMOS"), bipolar technologies like
emitter-coupled logic ("ECL"), polymer technologies (e.g.,
silicon-conjugated polymer and metal-conjugated polymer-metal
structures), mixed analog and digital, and so on.
[0156] It should also be noted that the various logic and/or
functions disclosed herein may be enabled using any number of
combinations of hardware, firmware, and/or as data and/or
instructions embodied in various machine-readable or
computer-readable media, in terms of their behavioral, register
transfer, logic component, and/or other characteristics.
Computer-readable media in which such formatted data and/or
instructions may be embodied include, but are not limited to,
non-volatile storage media in various forms (e.g., optical,
magnetic or semiconductor storage media) and carrier waves that may
be used to transfer such formatted data and/or instructions through
wireless, optical, or wired signaling media or any combination
thereof. Examples of transfers of such formatted data and/or
instructions by carrier waves include, but are not limited to,
transfers (uploads, downloads, e-mail, etc.) over the Internet
and/or other computer networks by one or more data transfer
protocols (e.g., HTTP, FTP, SMTP, and so on).
[0157] Unless the context clearly requires otherwise, throughout
the description and the claims, the words "comprise," "comprising,"
and the like are to be construed in an inclusive sense as opposed
to an exclusive or exhaustive sense; that is to say, in a sense of
"including, but not limited to." Words using the singular or plural
number also include the plural or singular number respectively.
Additionally, the words "herein," "hereunder," "above," "below,"
and words of similar import refer to this application as a whole
and not to any particular portions of this application. When the
word "or" is used in reference to a list of two or more items, that
word covers all of the following interpretations of the word: any
of the items in the list, all of the items in the list and any
combination of the items in the list.
[0158] Although certain presently preferred implementations of the
invention have been specifically described herein, it will be
apparent to those skilled in the art to which the invention
pertains that variations and modifications of the various
implementations shown and described herein may be made without
departing from the spirit and scope of the invention. Accordingly,
it is intended that the invention be limited only to the extent
required by the applicable rules of law.
[0159] The foregoing description, for purpose of explanation, has
been described with reference to specific embodiments. However, the
illustrative discussions above are not intended to be exhaustive or
to limit the invention to the precise forms disclosed. Many
modifications and variations are possible in view of the above
teachings. The embodiments were chosen and described in order to
best explain the principles of the invention and its practical
applications, to thereby enable others skilled in the art to best
utilize the invention and various embodiments with various
modifications as are suited to the particular use contemplated.
[0160] The foregoing description, for purpose of explanation, has
been described with reference to specific embodiments. However, the
illustrative discussions above are not intended to be exhaustive or
to limit the invention to the precise forms disclosed. Many
modifications and variations are possible in view of the above
teachings. The embodiments were chosen and described in order to
best explain the principles of the invention and its practical
applications, to thereby enable others skilled in the art to best
utilize the invention and various embodiments with various
modifications as are suited to the particular use contemplated.
Etc.
* * * * *