U.S. patent application number 15/512058 was filed with the patent office on 2017-11-23 for system and method using robots to assist humans in order fulfillment.
This patent application is currently assigned to Feich Roboltics, Inc.. The applicant listed for this patent is Michael Ferguson, Melonee Wise. Invention is credited to Michael Ferguson, Melonee Wise.
Application Number | 20170337506 15/512058 |
Document ID | / |
Family ID | 56614935 |
Filed Date | 2017-11-23 |
United States Patent
Application |
20170337506 |
Kind Code |
A1 |
Wise; Melonee ; et
al. |
November 23, 2017 |
System and Method Using Robots to Assist Humans in Order
Fulfillment
Abstract
A system using one or more robots to assist a human in order
fulfillment includes: a server configured to receive an order
comprising an order item; inventory storage operationally connected
to the server, the inventory storage comprising inventory items; an
order robot operationally connected to the server, the order robot
configured to assist a human to pick an order item from the
inventory items; and a human-operated device operably connected to
one or more of the server, the inventory storage, and the order
robot, the human-operated device configured to assist the human to
pick the order item.
Inventors: |
Wise; Melonee; (San Jose,
CA) ; Ferguson; Michael; (San Jose, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Wise; Melonee
Ferguson; Michael |
San Jose
San Jose |
CA
CA |
US
US |
|
|
Assignee: |
Feich Roboltics, Inc.
San Jose
CA
|
Family ID: |
56614935 |
Appl. No.: |
15/512058 |
Filed: |
February 11, 2016 |
PCT Filed: |
February 11, 2016 |
PCT NO: |
PCT/US16/17628 |
371 Date: |
March 16, 2017 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62115440 |
Feb 12, 2015 |
|
|
|
62117457 |
Feb 18, 2015 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G05B 19/00 20130101;
G06F 7/00 20130101; G06Q 10/087 20130101; G05B 19/418 20130101 |
International
Class: |
G06Q 10/08 20120101
G06Q010/08; G05B 19/418 20060101 G05B019/418 |
Claims
1. A system using one or more robots to assist a human in order
fulfillment, comprising: a server configured to receive an order
comprising an order item; inventory storage operably connected to
the server, the inventory storage comprising inventory items; an
order robot operably connected to the server, the order robot
configured to assist a human to pick an order item from the
inventory items; and a human-operated device operably connected to
one or more of the server, the inventory storage, and the order
robot, the human-operated device configured to assist the human to
pick the order item.
2. The system of claim 1, wherein the human-operated device is
configured to assist the human to perform one or more of locating
the order item, grasping the order item, and placing the order item
so as to be accessible to the order robot.
3. The system of claim 1, wherein the order robot is mobile.
4. The system of claim 1, wherein assisting comprises leading the
human.
5. The system of claim 1, wherein assisting comprises following the
human.
6. The system of claim 1, wherein the order robot comprises an
information capture device configured to make an information
capture.
7. The system of claim 6, wherein the information capture is usable
to perform one or more of detecting a human location and tracking
motion of the human.
8. The system of claim 6, wherein the information capture comprises
a pick list comprising one or more order items to be picked.
9. The system of claim 6, wherein the information capture device
comprises one or more of a depth camera, a Red Green Blue (RGB)
camera, another photographic camera, a videocamera, a laser, a
wireless receiver, a fiducial, a scanner, and another information
capture device.
10. The system of claim 6, wherein the information capture device
comprises a laser scanner.
11. The system of claim 1, wherein the order robot is configured,
using the information capture, to position itself at an approximate
robot position.
12. The system of claim 11, wherein the robot position comprises an
approximate robot position relative to the human.
13. The system of claim 11, wherein the robot position comprises a
point approximately located on the circumference of a circle
centered at a human location of the human.
14. The system of claim 13, wherein the robot position comprises a
point that is approximately nearest to the human of points located
on the circumference of a circle centered at the human location of
the human.
15. The system of claim 13, wherein the circle has a radius that is
predetermined by a user.
16. The system of claim 1, wherein the order robot moves so as to
maintain one or more of a substantially constant distance from the
human and a substantially constant orientation to the human.
17. The system of claim 16, wherein the robot position comprises an
approximate distance from the human.
18. The system of claim 17, wherein the distance is predetermined
by a user.
19. The system of claim 17, wherein the distance is calculated by
the server.
20. The system of claim 11, wherein the robot position is
predetermined by a user.
21. The system of claim 11, wherein the robot position is
calculated by the server.
22. The system of claim 11, wherein the robot position comprises a
desired orientation of the order robot.
23. The system of claim 22, wherein the orientation is
predetermined by a user.
24. The system of claim 22, wherein the orientation is calculated
by the server.
25. The system of claim 1, wherein picking comprises one or more of
locating the order item, selecting the order item, and placing the
order item so as to be accessible to the order robot.
26. The system of claim 1, wherein the order robot is configured to
provide data to the server.
27. The system of claim 26, wherein the data comprises one or more
of an order item pick time, an order item pick location, an image,
a point cloud, two-dimensional data, three-dimensional data, a
metric, and other data.
28. The system of claim 27, wherein the metric comprises one or
more of a human moving speed, a human picking speed, and human
health data.
29. The system of claim 28, wherein the human health data comprises
one or more of human exhaustion, elevated human body temperature,
and another human health datum.
30. The system of claim 1, wherein the order robot comprises a hit
detection device configured to detect when the order robot is
contacted.
31. The system of claim 30, wherein the hit detection device
comprises one or more of a designated panel, a gyro, an
accelerometer, an inertial measurement unit (IMU), and another hit
detection device.
32. The system of claim 1, further comprising a cuing device
configured to cue the human regarding one or more of an order item
to pick, a pick location, and another pick characteristic.
33. The system of claim 32, wherein the cuing device comprises one
or more of an audio cuing device, a visual cuing device, and
another cuing device.
34. The system of claim 32, wherein the cuing device comprises
lights comprised in the order robot, the lights configured to show
in which direction the order robot will be heading.
35. The system of claim 1, wherein the human-operated device is
controlled by the human.
36. The system of claim 1, wherein the human-operated device
comprises one or more of a heads-up display, a smart watch, a
tablet, a scanner, a mobile computer, a wireless human-operated
device, a headset, and another human-operated device.
37. The system of claim 1, wherein the human-operated device
comprises one or more of a radio frequency (RF) scanner and a radio
frequency identification (RFID) reader. Preferably, although not
necessarily, the human-operated device 127 is controlled by the
human 120.
38. A method using one or more robots to assist a human in order
fulfillment, comprising: receiving, by an order robot, a selection
of the order robot by the human; receiving, by the order robot,
from a server, an order comprising an order item; receiving, by the
order robot, from the server, a direction to assist the human to
pick the order; receiving, by the order robot, an assist trigger
configured to trigger the order robot to assist the human to pick
the order item; assisting the human, by the order robot, in
response to the assist trigger, to pick the order item; receiving,
by the order robot, from the server, an instruction to carry the
order item to a final destination; and carrying the order item to
the final destination.
39. The method of claim 38, wherein the step of receiving the
direction comprises receiving a direction to assist the human to
pick the order item from inventory items comprised in inventory
storage.
40. The method of claim 38, further comprising a step, performed
after the step of receiving the selection and before the step of
receiving the order, of transmitting the selection, by the order
robot, to a server.
41. The method of claim 38, further comprising a step, performed
after the step of receiving the order, and before the step of
receiving the assist trigger, of: capturing, by the order robot
using an information capture device, a pick list comprising one or
more items to be picked.
42. The method of claim 41, further comprising a step, performed
after the step of capturing the pick list, and before the step of
receiving the assist trigger, of: sending, by the order robot, to
the server, the pick list comprising the one or more order items to
be picked.
43. The method of claim 38, wherein the step of receiving the
selection comprises receiving the selection through an information
capture device comprised in the order robot.
44. The method of claim 38, wherein the step of receiving the
selection comprises receiving the selection via a button comprised
in the order robot that is pressed by the human.
45. The method of claim 38, wherein the step of receiving the
selection comprises receiving the selection via a transmission from
a human-operated device available to the human.
46. The method of claim 45, wherein the step of receiving the
selection comprises receiving the selection using a user interface
comprised in the human-operated device.
47. The method of claim 46, wherein the step of receiving comprises
receiving the selection via a map location designated by the human
using the user interface.
48. The method of claim 38, wherein the step of receiving the
selection comprises receiving the selection from the server.
49. The method of claim 38, wherein the step of transmitting
comprises transmitting through an information capture device
comprised in the order robot.
50. The method of claim 38, further comprising a step, performed
after the step of receiving the order, and before the step of
receiving the assist trigger, of: receiving, by the order robot,
from the human, a pick list comprising one or more order items to
be picked.
51. The method of claim 50, further comprising a step, performed
after the step of receiving the pick list, and before the step of
receiving the assist trigger, of: decoding, by the order robot, an
item comprised in the pick list.
52. The method of claim 38, wherein assisting comprises leading the
human.
53. The method of claim 38, wherein assisting comprises following
the human.
54. The method of claim 38, wherein the assist trigger comprises
one or more of a hit, a command, a microphone, a speech recognition
system, a command from the server, a command from a human-operated
device, and another assist trigger.
55. The method of claim 38, further comprising a step, performed
after the step of receiving the order, and before the step of
receiving the assist trigger, of: capturing, by the order robot,
information usable to perform one or more of detecting a human
location and tracking motion of the human.
56. The method of claim 55, wherein the step of capturing comprises
monitoring health of the human.
57. The method of claim 56, wherein the step of capturing comprises
monitoring one or more of a human moving speed, a human picking
speed, and human health data.
58. The method of claim 57, wherein the human health data comprises
one or more of human exhaustion, elevated human body temperature,
and another human health datum.
59. The method of claim 38, wherein picking comprises one or more
of locating the order item, selecting the order item, and placing
the order item so as to be accessible to the order robot.
60. The method of claim 38, further comprising a step, performed
after the step of assisting, and before the step of receiving the
instruction, of: providing, by the order robot, to the server,
data.
61. The method of claim 60, wherein the data comprises one or more
of an order item pick time, an order item pick location, an image,
a point cloud, two-dimensional data, three-dimensional data, a
metric, and other data.
62. The method of claim 61, wherein the metric comprises one or
more of a human moving speed, a human picking speed, and human
health data.
63. The method of claim 62, wherein the human health data comprises
one or more of human exhaustion, elevated human body temperature,
and another human health datum.
64. The method of claim 56, further comprising a step performed
after the step of receiving the order, and before the step of
receiving the assist trigger, of: alerting a supervisor, by the
order robot, if a human is having a medical issue.
65. The method of claim 38, wherein the assisting begins when the
order robot receives an assist trigger from a human.
66. The method of claim 65, wherein the assist trigger comprises a
light kick of a hit detection device comprised in the order
robot.
67. The method of claim 65, wherein the assist trigger comprises
one or more of a hit, a heads-up display, a smart watch display, a
tablet display, a command, a wireless device display, a headset
instruction, a microphone instruction, a speech recognition system
instruction, and another assist trigger.
68. The method of claim 38, wherein assisting comprises cuing the
human regarding the order item to be picked.
69. The method of claim 68, wherein cuing comprises one or more of
cuing using an audio cuing device, cuing using a visual cuing
device, and another form of cuing.
70. The method of claim 68, wherein cuing comprises cuing the human
regarding one or more of an order item to pick, a pick location,
and another pick characteristic.
71. The method of claim 68, wherein the visual cuing device
comprises a projector configured to project a cue onto one or more
of a floor and another surface.
72. The method of claim 71, wherein the projector comprises a
pick-to-light illumination device.
73. The method of claim 68, wherein cuing comprises using lights
comprised in the order robot to show in which direction the order
robot will be heading.
74. The method of claim 38, wherein assisting comprises
maintaining, by the order robot, one or more of a substantially
constant distance from the human and a substantially constant
orientation to the human.
75. The method of claim 38, where assisting comprises adjusting, by
the order robot, its pace to approximately match the pace of the
human.
76. The method of claim 75, wherein the adjusting is performed
automatically.
77. The method of claim 38, wherein assisting comprises navigating,
by the order robot, to a desired pick location.
78. The method of claim 77, wherein the navigating is performed
automatically.
79. The method of claim 38, wherein assisting comprises providing,
by the order robot, one or more shelves located on top of the order
robot and comprising a pick-to-light illumination device.
80. The method of claim 38, further comprising a step, performed
after the step of assisting, and before the step of receiving the
instruction, of sending, by the order robot, to the server, an
order fulfillment confirmation.
81. A method using one or more robots to assist a human in order
fulfillment, comprising: receiving, by a server, an order
comprising an order item; receiving, by the server, a selection by
the human of the order robot; sending the order, by the server, to
one or more of an order robot and a human-operated device available
to the human; sending, by the server, to the order robot, a
direction to assist the human to pick the order item; determining,
by the server, that the order is nearly complete; determining, by
the server, that an order robot replacement suitable for fulfilling
a next order has not yet been sent; sending a dispatch instruction,
by the server, to the order robot replacement instructing its
dispatch; determining, by the server, that the order is complete;
and instructing the order robot, by the server, to carry the order
item to a final destination.
82. The method of claim 81, wherein the step of sending the
direction comprises sending a direction to assist the human to pick
the order item from inventory items comprised in inventory
storage.
83. The method of claim 81, wherein the step of receiving the
selection comprises receiving the selection from the order
robot.
84. The method of claim 81, wherein the step of receiving the
selection comprises receiving the selection from the human-operated
device.
85. The method of claim 84, further comprising a step, performed
after the step of receiving the selection and before the step of
sending the order, of sending the selection, by the server, to the
order robot.
86. The method of claim 81, further comprising a step, performed
after the step of sending the order and before the step of
determining that the order is nearly complete, of: receiving, by
the server, from the order robot, a pick list comprising one or
more order items to be picked.
87. The method of claim 81, wherein the step of instructing
comprises a sub-step of: determining one or more of the order item
and a location of the order item.
88. The method of claim 81, further comprising a step, performed
after the step of sending the order and before the step of
determining that the order is complete, of: sending, by the server,
to the order robot, an assist trigger configured to trigger the
order robot to assist the human to pick the order item.
Description
PRIORITY CLAIM
[0001] The present application claims the priority benefit of U.S.
provisional patent application No. 62/115,440 filed Feb. 12, 2015
and entitled "System and Method for Order Fulfillment and Inventory
Management Using Robots," and of U.S. provisional patent
application No. 62/117,457 filed Feb. 18, 2015 and entitled "System
and Method for Order Fulfillment Using Robots to Assist in
Selecting Stock Items," the disclosures of which are incorporated
herein by reference.
CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application contains subject matter that is related to
the subject matter of the following applications, which are
assigned to the same assignee as this application. The below-listed
applications are hereby incorporated herein by reference in its
entirety:
[0003] "SYSTEM AND METHOD FOR ORDER FULFILLMENT USING ROBOTS," by
Wise, et al., co-filed herewith.
SUMMARY
[0004] Embodiments of this invention relate in general to inventory
management and order fulfillment systems, and more particularly to
a system and method using robots to assist humans in order
fulfillment.
[0005] A system using one or more robots to assist a human in order
fulfillment includes: a server configured to receive an order
comprising an order item; inventory storage operably connected to
the server, the inventory storage comprising inventory items; an
order robot operably connected to the server, the order robot
configured to assist a human to pick an order item from the
inventory items; and a human-operated device operably connected to
one or more of the server, the inventory storage, and the order
robot, the human-operated device configured to assist the human to
pick the order item.
[0006] A method using one or more robots to assist a human in order
fulfillment includes: receiving, by an order robot, a selection of
the order robot by the human; receiving, by the order robot, from a
server, an order comprising an order item; receiving, by the order
robot, from the server, a direction to assist the human to pick the
order item; receiving, by the order robot, an assist trigger
configured to trigger the order robot to assist the human to pick
the order item; assisting the human, by the order robot, to pick
the order item; receiving, by the order robot, from the server, an
instruction to carry the order item to a final destination; and
carrying the order item to the final destination.
[0007] A method using one or more robots to assist a human in order
fulfillment includes: receiving, by a server, an order comprising
an order item; receiving, by the server, a selection by the human
of the order robot; sending the order, by the server, to one or
more of the order robot and a human-operated device available to
the human; sending, by the server, to the order robot, a direction
to assist the human to pick the order item; determining, by the
server, that the order is nearly complete; determining, by the
server, that an order robot replacement suitable for fulfilling a
next order has not yet been sent; sending a dispatch instruction,
by the server, to the order robot replacement instructing its
dispatch; determining, by the server, that the order is complete;
and instructing the order robot, by the server, to carry the order
item to a final destination.
DESCRIPTION OF THE DRAWINGS
[0008] FIGS. 1A-1K are a set of drawings that depict a sequence of
events in a system using robots to assist humans in order
fulfillment in which an order robot follows one or more humans as
they pick one or more order items.
[0009] FIGS. 2A-2D are a set of drawings that depict a sequence of
events in a system using robots to assist humans in order
fulfillment which an order robot leads one or more humans as they
pick one or more order items.
[0010] FIG. 3 is a drawing of a system using robots to assist
humans in order fulfillment showing that the following behavior of
the order robot may be initiated by a hit such as by lightly
kicking a hit detection device comprised in the order robot.
[0011] FIG. 4 is a drawing of a system using robots to assist
humans in order fulfillment illustrating that the order robot can
autonomously give the human cues as to one or more of what to pick,
where to pick, and the like.
[0012] FIG. 5 is a flowchart of a method 500 for order fulfillment
using robots to assist humans.
[0013] FIG. 6 is a flowchart of a method 600 for order fulfillment
using robots to assist humans.
DETAILED DESCRIPTION
[0014] Embodiments of the invention relate in general to inventory
management and order fulfillment systems, and more particularly to
a system and method using robots to assist humans in order
fulfillment to assist the selection of stock items to fulfill
orders. More specifically, embodiments of the invention relate to a
system and method using robots to assist humans in order
fulfillment to assist humans picking stock items. More
specifically, embodiments of the invention relate to a system and
method using robots to assist humans in order fulfillment
comprising a mobile base to assist humans picking stock items. More
specifically, embodiments of the invention relate to a system and
method using robots to assist humans in order fulfillment
comprising an order robot to assist humans picking stock items.
[0015] According to embodiments of the invention, an order robot is
used to transport items that a human is picking from inventory
storage. For example, the order robot is used to transport stock
items that a human is picking from inventory storage to fulfill an
order. Inventory storage may comprise one or more of a fixed shelf,
a non-fixed shelf, a rack, and another inventory storage.
[0016] For example, the order robot makes one or more information
captures using one or more information capture devices, with one or
more of the one or more information captures being used to perform
one or more of detecting a human location and tracking motion of
one or more humans. For example, the information capture device can
comprise one or more of a depth camera, a Red Green Blue (RGB)
camera, another type of photographic camera, a videocamera, a
laser, a wireless receiver, a fiducial, a scanner, and another
information capture device. For example, the information capture
device comprises a laser scanner. For example, the information
capture device comprises a scanning laser rangefinder. For example,
the information capture can comprise one or more of a camera image,
a video, a depth image, a laser measurement, and another
information capture.
[0017] According to other embodiments of the invention, the system
comprises a human-operated device. The human-operated device may be
one or more of carried by the human, carried in clothing of the
human, and placed near the human. For example, the human-operated
device may be carried in a pocket comprised in one or more of a
shirt worn by the human and pants worn by the human. The
human-operated device comprises one or more of a heads-up display,
a smart watch, a tablet, a scanner, a mobile computer, a wireless
human-operated device, a headset, and another human-operated
device. Preferably, although not necessarily, the human controls
the human-operated device. The server may send the order to the
human-operated device, and the human-operated device may receive
the order, so that the order is made available to the human.
[0018] According to further embodiments of the invention, following
the information capture, the order robot positions itself at an
approximate distance from a human. For example, the approximate
distance may be predetermined by a user. Alternatively, or
additionally, following the information capture, the order robot
positions itself at an approximate position. For example, following
the information capture, the order robot positions itself at an
approximate robot position relative to a human. For example, the
robot position may be predetermined by a user. For example, the
robot position comprises a point approximately located on the
circumference of a circle centered at a human location of a human.
For example, the robot position comprises a point approximately
located on the circumference of a circle centered at a human
location of a human and with a radius that is predetermined by a
user. For example, the robot position comprises a point that is
approximately nearest to a human of points located on the
circumference of a circle centered at the human location of the
human and with a radius that is predetermined by a user.
[0019] In the case where the order robot is following a human, the
server can dispatch robots to one or more previously designated
waiting spots. The one or more previously designated waiting spots
can be near the inventory to be picked. Alternatively, or
additionally, the one or more previously designated waiting spots
may be located near a central staging area for robots. The order
robots are configured to engage in following behavior. The order
robots are configured to move from the previously designated
waiting stop to a stop made by a human to pick inventory. The order
robots are further configured to move from a stop made by a human
to pick inventory to another stop made by a human to pick
inventory.
[0020] According to an alternative set of embodiments, the mobile
base may not be constrained to the plane and may fly using one or
more propellers. According to another alternative set of
embodiments, the order robot may move using actuated legs.
According to yet another set of embodiments, the order robot may
move itself by balancing on a ball.
[0021] According to a further set of embodiments, an order robot
waiting spot is located near a charging station, thereby allowing
the order robot to recharge its battery while waiting for the human
to arrive. For example, the charging station may be a wireless
charging station. For example, the charging station may be a wired
charging station.
[0022] According to further embodiments of the invention, the order
robot may be configured to provide data to a server. For example,
the order robot may be configured to provide logging data to the
server. For example, the order robot may be configured to provide
logging data to a central server. For example, the order robot may
be configured to provide logging data to the central server located
within an inventory storage. For example, the order robot may be
configured to provide the logging data to the central server that
is connected to the warehouse via a network connection. The data
may comprise an order item pick time, an order item pick location,
an image, a point cloud, two-dimensional data, three-dimensional
data, a metric, and other data.
[0023] According to another set of embodiments, the system may
comprise an information capture device. For example, the order
robot may comprise the information capture device. As another
example, alternatively, or additionally, the server may comprise
the information capture device. As yet another example,
alternatively or additionally, the information capture device may
be a fixed installation operably connected to the server.
[0024] For example, the information capture device may capture an
image of one or more pick lists. For example, the one or more pick
lists comprise one or more order items to be picked. For example,
the pick lists may comprise one or more of paper pick lists,
electronic pick lists and other pick lists. For example, the human
presents the one or more pick lists to the order robot, which then
decodes one or more order items to be picked comprised in the one
or more pick lists. For example, the order robot then correlates
one or more of the items to be picked with a database of product
locations.
[0025] For example, the order robot leads the human through the
warehouse, using one or more of the aforementioned methods of
communication to tell the human what to pick. In another set of
embodiments, the order robot sends to the server one or more of a
coded pick list and a decoded pick list. For example, the server is
connected wirelessly to a network. For example, the server is
connected via a wired connection to the network. For example, the
server comprises a wired server. For example, the server comprises
the database of product locations. For example, the server then
returns to the order robot one or more of a location and a quantity
of items to be picked. For example, the server spreads the pick
list among more than one order robot, thereby allowing a pick list
to be spread across several humans.
[0026] FIGS. 1A-1K are a set of drawings that depict a sequence of
events in a system 100 using robots 110 to assist humans 120 in
order fulfillment in which an order robot 110 follows a human 120
as the human picks one or more order items. For example, the system
100 comprises a warehouse 100. Preferably, although not
necessarily, the order robot 110 comprises a mobile order robot
110. Preferably, although not necessarily, the order robot 110
comprises a mobile order robot base 110.
[0027] A server 125 operably connected with the order robot 110
receives an order. The server 125 sends the order to one or more of
a human-operated device 127 and the order robot 110. As shown in
FIG. 1A, the server 125 sends the new order to the human-operated
device 127 so as to be available to the human 120. The
human-operated device 127 receives the order from the server 125.
As depicted in FIG. 1A, the human-operated device 127 is held by
the human 120.
[0028] The human-operated device 127 comprises one or more of a
heads-up display 127, a smart watch 127, a tablet 127, a scanner
127, a mobile computer 127, a wireless human-operated device 127, a
headset 127, and another human-operated device 127. For example,
the scanner 127 may comprise one or more of a radio frequency (RF)
scanner 127 and a radio frequency identification (RFID) reader 127.
Preferably, although not necessarily, the human-operated device 127
is controlled by the human 120.
[0029] For example, the human 120 may be equipped with a headset
127 configured to communicate with the order robot 110 without
picking up one or more of ambient noise and robot noise. For
example, the headset 127 may be operably connected to the order
robot 110. For example, the headset 127 may be operably connected
to the server 125. For example, the headset 127 may be wirelessly
connected to the order robot 110. For example, the headset 127 may
be wirelessly connected to the server.
[0030] The order robot 110 follows the human 120 as the human 120,
guided by the human-operated device 127, picks from an inventory
storage 130 one or more order items 140A-140H to fulfill an order.
Picking the order item 140A-140H comprises one or more of locating
the order item 140A-140H, grasping the order item 140A-140H, and
placing the order item 140A-140H so as to be accessible to the
order robot 110. It is not intended that the terms locating,
grasping, and placing be limiting. For example, grasping can occur
without the human 120 exerting any effort to retain the order item
140A-140H. For example, grasping can comprise allowing the order
item 140A-140H to rest on the human 120. For example, the
human-operated device 127 displays one or more of the order items
140A-140H to be picked.
[0031] Those of skill in the art will recognize that any number of
order robots 110 may be present according to embodiments of the
invention. Similarly, those of skill in the art will recognize that
any number of humans 120 may be present.
[0032] As depicted, the order robot 110 comprises order storage
150. Order storage 150 comprises one or more of an order bin, an
order shelf, and an order box. As depicted, the order storage 150
comprises three order shelves, a first order shelf 160A, a second
order shelf 160B, and a third order shelf 160C. As depicted, the
first order shelf 160A comprises a first order shelf order item
170. As depicted, the third order shelf 160C comprises a third
order shelf order item 175.
[0033] Those of skill in the art will further recognize that the
first order shelf 160A may hold an arbitrary number of first order
shelf order items 170, the second order shelf 160B may hold an
arbitrary number of second order shelf order items (not pictured),
and the third order shelf 160C may hold an arbitrary number of
third order shelf order items 175.
[0034] In FIG. 1B, the order robot 110 receives a selection of the
order robot 110 by the human 120. As depicted, the order robot 110
comprises an information capture device 180 that receives the
selection of the order robot 110 by the human 120. The information
capture device 180 can comprise one or more of a depth camera, a
Red Green Blue (RGB) camera, another type of photographic camera, a
videocamera, a laser, a wireless receiver, a fiducial, a scanner,
and another information capture device. For example, the
information capture device 180 comprises one or more of a radio
frequency (RF) scanner 180 and a radio frequency identification
(RFID) reader 180. As depicted, the information capture device 180
comprises a laser scanner 180. For example, the laser scanner 180
comprises a scanning laser rangefinder 180. The human 120 presses a
button on the human-operated device 127 to select the order robot
110 and the human-operated device 127 transmits the selection to
the order robot 110.
[0035] Alternatively, the order robot 110 receives the selection
via a button comprised in the order robot 110 that is pressed by
the human 120. Alternatively, the human 120 designates a map
location using a user interface comprised in the human-operated
device 127 and the device transmits the selection to the order
robot 110. Alternatively, the server 125 receives the selection
from the human-operated device 127 and then the server 125
transmits the selection to the order robot 110.
[0036] In FIG. 1C, as depicted, the selected order robot 110
transmits the selection to the server 125 using the information
capture device 180. Alternatively, or additionally, the
human-operated device 127 transmits the selection of the order
robot 110 to the server 125.
[0037] In FIG. 1D, the server 125 transmits the task to the
selected order robot 110 through the information capture device
180.
[0038] In FIG. 1E, the order robot 110 assists the human 120 in
fulfilling the order. As depicted, the order robot 110 follows the
human 120.
[0039] The assisting behavior of the order robot 110 may be
initiated by one or more of a hit, a command, a microphone, a
speech recognition system, a command from the server 125, a command
from the human-operated device 127, and another assist trigger
configured to trigger the order robot 110 to assist the human 120
to pick the order item.
[0040] For example, the command may comprise a spoken command. For
example, the command may comprise a spoken command from the human
120. For example, the order robot 110 may comprise one or more of
the microphone (not pictured) and the speech recognition system
(not pictured).
[0041] As shown in FIG. 1E, the order robot 110 may use the
information capture device 180 to make an information capture to
perform one or more of detecting a human location and tracking a
motion of the human 120. The information capture device 180
generates a sensor plane 181. As depicted, the sensor plane 181
illuminates legs 182 of the human 120 at a human location 183.
Accordingly, the order robot 110 thereby performs an information
capture that detects the human location 183 of the human 120.
[0042] For example, the information capture enables the order robot
110 to avoid one or more of colliding with the human 120 and losing
track of the human 120. The information capture comprises one or
more of a camera image, a video, a depth image, a laser
measurement, and another information capture.
[0043] As shown in FIG. 1F, the order robot 110, using the
information obtained by the information capture device 180 in the
information capture, adjusts its robot position relative to the
human 120. As depicted in FIG. 1F, using the information capture,
the order robot 110 moves toward the human 120.
[0044] Using the information capture, the order robot 110 positions
itself at an approximate robot position 186 relative to the human
120. For example, the robot position 186 may be predetermined by a
user. For example, the robot position 186 comprises a point 186
that is approximately located on the circumference of a circle 187
having a radius 190 and centered at the human location 183 of the
human 120. For example, the circle 187 has a radius 190 that is
predetermined by a user. That is, the radius 190 comprises an
approximate desired distance 190 of the order robot 110 from the
human 120. As depicted, the robot position 186 comprises a point
186 that is approximately nearest to the human 120 of points
located on the circumference of the circle 187 centered at the
human location 183 of the human 120 and with a radius 190 that is
predetermined by a user.
[0045] As shown in FIG. 1F, the order robot 110 positions itself at
the approximate desired distance 190 from the human 120. Once a
human 120 has been detected, a control program can send commands to
an actuator of the order robot 110. The actuator may comprise one
or more of one or more wheels and a non-wheeled actuator. For
example, the system may be configured so that the order robot 110
moves in the correct direction so as to maintain one or more of a
substantially constant distance 190 from the human and a
substantially constant orientation to the human.
[0046] For example, the desired distance 190 may be predetermined
by a user. For example, the desired distance 190 may be calculated
by the server 125. Alternatively, or additionally, following the
information capture, the order robot 110 positions itself at an
approximate robot position 186. For example, the robot position 186
is predetermined by a user. For example, the robot position 186 is
calculated by the server 125.
[0047] The order robot 110 is configured to provide data to the
server 125. For example, the order robot 110 is configured to
provide logging data to the server 125. The data may comprise one
or more of an order item pick time, an order item pick location, an
image, a point cloud, two-dimensional data, three-dimensional data,
a metric, and other data.
[0048] A metric may comprise one or more of a human moving speed, a
human picking speed, and human health data. The order robot 110 may
use the information capture device 180 to monitor the health of the
human. For example, the order robot 110 may use the information
capture device 180 to monitor one or more of human exhaustion,
elevated human body temperature, and another human health
indicator. If a human is having a medical issue, the order robot
110 may alert a supervisor through one or more of wireless
communications and wired communications.
[0049] In FIG. 1G, the human 120 picks up the fourth order item
140D from the inventory storage 130. As depicted in FIG. 1G, the
human-operated device 127 is carried in clothing of the human 120.
As depicted, the human-operated device 127 is carried in a pocket
comprised in pants worn by the human 120.
[0050] In FIG. 1H, the human 120 carries the fourth order item 140D
toward the second order shelf 160B.
[0051] In FIG. 1I, the human 120 positions the fourth order item
140D on the second order shelf 160B. That is, the human 120 places
the fourth order item 140D directly onto the order robot 110 so as
to be accessible to the order robot 110.
[0052] In FIG. 1J, the order robot 110 sends an order fulfillment
confirmation to the server 125. Alternatively, the human-operated
device 127 sends the order fulfillment confirmation to the server
125.
[0053] In FIG. 1K, the server 125 determines whether the order is
nearly complete. Nearly complete may be defined based on a fixed
number of order items 140A-140H remaining to be picked. For
example, an order may be defined as nearly complete if three or
fewer items 140A-140H remain to be picked in the order.
Alternatively, or additionally, nearly complete may be defined
based on a typical time required to pick remaining order items
140A-140H. For example, nearly complete may be defined based on the
typical time required to pick the remaining order items 140A-140H
using a pick metric. For example, the pick metric comprises an
average transit time between the current inventory storage 130 and
an anticipated next inventory storage 130. For example, the pick
metric comprises a distance remaining to be traveled computed using
a location of one or more order items 140A-140H to be picked.
[0054] If the server 125 determines that the order is not nearly
complete, the server 125 transmits an order to one or more of the
order robot 110 and the human-operated device 127 to order the
picking of another order item 140A-140H.
[0055] If the server 125 determines that the order is nearly
complete, the server queries if an order robot replacement 110A
that is suitable for use for fulfilling a next order has yet been
sent. Sending the order robot replacement 110A may be desirable to
minimize delays in fulfilling the next order. If no order robot
replacement 110A has been sent yet, the server 125 sends an order
to the order robot replacement 110A ordering its dispatch. The
order robot replacement 110A moves per the order it received from
the server 125.
[0056] Then the server 125 determines whether the order is
complete. If an order robot replacement 110A has already been sent,
the server 125 determines whether the order is complete.
[0057] If the server 125 determines that the order is complete, as
depicted, the server 125 instructs the order robot 110 to carry the
order items 140A-140H to a final destination 195. The order robot
110 then heads off to the final destination 195 for offloading of
the order items 140A-140H.
[0058] Process terminates or restarts using replacing robot 1106.
Replacing robot may navigate to and follow current robot 110 that
is following human 120.
[0059] FIGS. 2A-2D are a set of drawings that depict a sequence of
events in a system using robots to assist humans in order
fulfillment in which an order robot 110 leads one or more humans
120 as they pick one or more order items 140A-140H. As shown in
FIGS. 2A-2D, according to alternative embodiments, the order robot
110 may lead the human, rather than following the human as shown in
FIGS. 1A-1K. The control device is configured to determine one or
more of the order item 140A-140H to pick and a location of the
order item 140A-140H. The order robot 110 can navigate its way
through the inventory storage 130 to the desired position. For
example, the order robot 110 can automatically navigate its way
through the inventory storage 130 to the desired position.
[0060] FIGS. 2A-2D depict a system 200 using robots to assist
humans in order fulfillment in which the order robot 110 leads the
human 120 as the human 120 picks one or more order items to fulfill
an order. Other than the fact that the order robots 110 leads the
human 120 in FIGS. 2A-2D, rather than following the human 120 as in
FIGS. 1A-1K, the system 200 functions in a similar way to the
system 100. For example, the system 200 comprises a warehouse 200.
The order robot 110 may lead the human 120 as the human 120 picks
one or more order items.
[0061] FIGS. 2A-2D also depict the inventory storage 130 comprising
the first through eighth inventory items 140A-140H. Preferably,
although not necessarily, the order robot 110 comprises a mobile
order robot 110. Those of skill in the art will recognize that any
number of order robots 110 may be present. Similarly, those of
skill in the art will recognize that any number of humans 120 may
be present.
[0062] For example, as depicted in FIG. 2A, the order robot 110
comprises order storage 150. Order storage 150 comprises one or
more of an order bin, an order shelf, and an order box. As
depicted, the order storage 150 comprises three order shelves, a
first order shelf 160A, a second order shelf 160B, and a third
order shelf 160C. As depicted, the first order shelf 160A comprises
a first order shelf order item 170. As depicted, the third order
shelf 160C comprises a third order shelf order item 175.
[0063] Those of skill in the art will further recognize that the
first order shelf 160A may hold an arbitrary number of first order
shelf order items 170, the second order shelf 160B may hold an
arbitrary number of second order shelf order items (not pictured),
and the third order shelf 160C may hold an arbitrary number of
third order shelf order items 175.
[0064] As depicted in FIG. 2A, the order robot 110 further
comprises an information capture device 180.
[0065] In FIG. 2B, the human 120 picks up the fifth order item 140E
from the inventory storage 130.
[0066] In FIG. 2C, the human 120 carries the fifth order item 140E
toward the second order shelf 160B.
[0067] In FIG. 2D, the human positions the fifth order item 140E on
the second order shelf 160B.
[0068] FIG. 3 is a drawing of a system 300 using robots to assist
humans in order fulfillment showing that the following behavior of
the order robot 110 may be initiated by a hit such as by lightly
kicking a hit detection device comprised in the order robot 110.
For example, the following behavior of the order robot may be
initiated by a human 120 lightly kicking the order robot 110 on a
designated panel. In this set of embodiments, the order robot 110
comprises a hit detection device. For example the hit detection
device may comprise one or more of a designated panel, a gyro, an
accelerometer, an inertial measurement unit (IMU), and another hit
detection device. For example, sensor fusion between the
information capture device 180 and the IMU can facilitate hit
detection.
[0069] The assisting behavior of the order robot 110 may be
initiated by one or more of a hit, a command, a microphone, a
speech recognition system, a command from the server 125, a command
from the human-operated device 127, and another assist trigger
configured to trigger the order robot 110 to assist the human 120
to pick the order item.
[0070] The following behavior of the order robot 110 may be
initiated by one or more of a hit, a heads-up display, a smart
watch, a tablet, a command, a wireless device, a headset, a
microphone, a speech recognition system, and another following
behavior initiator. For example, one or more of a smart watch, a
tablet, and the like may also be used to display one or more of the
items to be picked.
[0071] For example, the command may comprise a spoken command. For
example, the command may comprise a spoken command from a human.
For example, the order robot 110 may comprise one or more of a
microphone and a speech recognition system. For example, the human
may be equipped with a headset configured to communicate with the
order robot 110 without picking up one or more of ambient noise and
robot noise. For example, the headset may be operably connected to
the order robot 110. For example, the headset may be operably
connected to the server 125. For example, the headset may be
wirelessly connected to the order robot 110. For example, the
headset may be wirelessly connected to the server 125.
[0072] As another example, the following behavior of the order
robot 110 may be initiated by a heads-up display 127.
[0073] FIG. 4 is a drawing of a system 400 using robots to assist
humans in order fulfillment illustrating that the order robot 110
can autonomously give the human 120 cues as to one or more of what
to pick, where to pick, and the like. The human 120, guided by the
human-operated device 127, picks from an inventory storage 130 one
or more order items 140A-140H to fulfill an order.
[0074] The order robot 110 may use a projector 410 to project the
cues onto one or more of the floor and another surface. As
depicted, the projector 410 is indicating item 140B to be picked
from the inventory storage 130 by the human 120, guided by the
human-operated device 127. For example, the projector 410 comprises
a pick-to-light illumination device. For example, the projector 410
comprises another projector 410 other than a pick-to-light
illumination device.
[0075] According to these embodiments, the order robot 110 is the
leader of the human 120, as depicted in FIGS. 2A-2D. The order
robot 110 may suggest where the human 120 should go. For example,
the order robot 110 may comprise lights (not shown) configured to
show in which direction the order robot 110 will be heading.
[0076] The order robot 110 may adjust its pace to approximately
match the pace of the human 120. For example, the order robot 110
may automatically adjust its pace to approximately match the pace
of the human 120. For example, the order robot 110 may adjust its
pace to approximately match the pace of the human 120 if the human
120 falls behind the order robot 110. For example, the order robot
110 may adjust its pace to approximately match the pace of the
human 120 if the human 120 keeps up with the order robot 110.
[0077] According to further embodiments of the invention, the order
robot 110 may comprise a cuing device. The cuing device may
comprise one or more of a visual cuing device and an audio cuing
device. The visual cuing device may comprise one or more of an
onboard projector, a camera system, an augmented reality system, a
laser pointer, a light-emitting diode (LED) light, and another
visual cuing device that can show the human what to pick. The
visual cuing device is configured to display one or more visual
cues to the human regarding one or more order items to pick. The
visual cuing device may also be configured to record evidence of
one or more of the picking of the one or more order items and the
placement of the one or more order items so as to be accessible to
the order robot 110. The audio cuing device is configured to
specify the task to the human. The audio cuing device may comprise
one or more of an on-robot speaker and a headset. The headset may,
for example, be wirelessly connected to the order robot 110.
Alternatively, a wired connection may connect the headset and the
order robot 110. As another alternative, the order robot 110 can
wirelessly communicate with the server, and the server can
wirelessly communicate with the headset.
[0078] FIG. 5 is a flowchart of a method 500 for order fulfillment
using robots to assist humans. The order of the steps in the method
500 is not constrained to that shown in FIG. 5 or described in the
following discussion. Several of the steps could occur in a
different order without affecting the final result.
[0079] In step 510, an order robot receives a selection of the
order robot by a human. Block 510 then transfers control to block
520.
[0080] In step 520, the order robot receives, from a server, an
order comprising an order item. Block 520 then transfers control to
block 530.
[0081] In step 530, the order robot receives from the server a
direction to assist the human to pick the order item. Block 530
then transfers control to block 540.
[0082] In step 540, the order robot receives an assist trigger
configured to trigger the order robot to assist the human to pick
the order item. Block 540 then transfers control to block 550.
[0083] In step 550, the order robot receives from the server an
instruction to carry the order item to a final destination. Block
550 then transfers control to block 560.
[0084] In step 560, the order robot assists the human to pick the
order item. Block 560 then transfers control to block 570.
[0085] In step 570, the order robot carries the order item to the
final destination. Block 570 then terminates the process.
[0086] FIG. 6 is a flowchart of a method 600 for order fulfillment
using robots to assist humans. The order of the steps in the method
600 is not constrained to that shown in FIG. 6 or described in the
following discussion. Several of the steps could occur in a
different order without affecting the final result.
[0087] In step 610, a server receives an order comprising an order
item. Block 610 then transfers control to block 620.
[0088] In step 620, the server receives a selection by the human of
the order robot. Block 620 then transfers control to block 630.
[0089] In step 630, the server sends the order to one or more of
the order robot and a human-operated device available to the human.
Block 630 then transfers control to block 640.
[0090] In step 640, the server sends to the order robot a direction
to assist the human to pick the order item. Block 640 then
transfers control to block 650.
[0091] In step 650, the server determines if the order is nearly
complete. If the order is determined to be nearly complete, block
650 then transfers control to block 660. If the order is not nearly
complete, the process reverts to block 640.
[0092] In step 660, the server determines if an order robot
replacement suitable for fulfilling a next order has yet been sent.
If the order robot replacement has not yet been sent, block 660
then transfers control to block 670. If the order robot replacement
has already been sent, block 660 then transfers control to block
680.
[0093] In step 670, the server sends a dispatch instruction to the
order robot replacement instructing its dispatch. Block 670 then
transfers control to block 680.
[0094] In step 680, the server determines if the order is complete.
If the order is complete, block 680 then transfers control to block
690. If the order is not complete, the process reverts to block
640.
[0095] In step 690, the server instructs the order robot to carry
the order item to a final destination. Block 690 then terminates
the process.
[0096] For example, it will be understood by those skilled in the
art that software used by the system and method for order
fulfillment and inventory management using robots may be located in
any location in which it may be accessed by the system. It will be
further understood by those of skill in the art that the number of
variations of the network, location of the software, interactions
of robots and humans, and the like are virtually limitless. It is
intended, therefore, that the subject matter in the above
description shall be interpreted as illustrative and shall not be
interpreted in a limiting sense.
[0097] While the above representative embodiments have been
described with certain components in exemplary configurations, it
will be understood by one of ordinary skill in the art that other
representative embodiments can be implemented using different
configurations and/or different components. For example, it will be
understood by one of ordinary skill in the art that the order of
certain steps and certain components can be altered without
substantially impairing the functioning of the invention. For
example, the one or more shelves may be located within the order
robot rather than on top of the order robot.
[0098] The representative embodiments and disclosed subject matter,
which have been described in detail herein, have been presented by
way of example and illustration and not by way of limitation. It
will be understood by those skilled in the art that various changes
may be made in the form and details of the described embodiments
resulting in equivalent embodiments that remain within the scope of
the invention. It is intended, therefore, that the subject matter
in the above description shall be interpreted as illustrative and
shall not be interpreted in a limiting sense.
* * * * *