U.S. patent application number 14/675206 was filed with the patent office on 2016-04-21 for method, computer program product, and system for providing a sensor-based environment.
The applicant listed for this patent is Toshiba Global Commerce Solutions Holdings Corporation. Invention is credited to Devin Sherrone Brown, Dean Frederick Herring, Ankit Singh.
Application Number | 20160110751 14/675206 |
Document ID | / |
Family ID | 55748790 |
Filed Date | 2016-04-21 |
United States Patent
Application |
20160110751 |
Kind Code |
A1 |
Brown; Devin Sherrone ; et
al. |
April 21, 2016 |
METHOD, COMPUTER PROGRAM PRODUCT, AND SYSTEM FOR PROVIDING A
SENSOR-BASED ENVIRONMENT
Abstract
Method, computer program product, and system of providing
rewards to a person based on item interactions during a transaction
in an environment, where the person is associated with a personal
profile. The method includes generating, based on information in
the personal profile, one or more tasks for the person to complete
during the transaction, each task including at least one item
interaction. The method further includes presenting, using an
output device within the environment, the one or more tasks to the
person, analyzing image information acquired by one or more visual
sensors within the environment to evaluate the person's performance
of the one or more tasks, and awarding, based on the evaluation, an
amount of rewards to the person.
Inventors: |
Brown; Devin Sherrone;
(Durham, NC) ; Herring; Dean Frederick;
(Youngsville, NC) ; Singh; Ankit; (Morrisville,
NC) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Toshiba Global Commerce Solutions Holdings Corporation |
Tokyo |
|
JP |
|
|
Family ID: |
55748790 |
Appl. No.: |
14/675206 |
Filed: |
March 31, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62064323 |
Oct 15, 2014 |
|
|
|
Current U.S.
Class: |
705/14.25 |
Current CPC
Class: |
G06F 3/013 20130101;
G01G 19/40 20130101; G06Q 30/0609 20130101; G07G 1/0045 20130101;
G06K 2209/17 20130101; G06Q 30/0235 20130101; G06K 9/00362
20130101; G06K 9/46 20130101; G06Q 30/0631 20130101; G07G 1/0063
20130101; G06K 9/00255 20130101; G06K 2009/00328 20130101; G06Q
30/016 20130101; G06Q 30/0639 20130101; G06F 1/163 20130101; G06K
9/209 20130101; G06Q 20/322 20130101; G06T 11/60 20130101; G01S
19/13 20130101; G06Q 30/0224 20130101; G01B 11/02 20130101; G06K
9/00288 20130101; G06Q 30/0629 20130101; G06Q 10/087 20130101; G06Q
20/209 20130101; H04N 5/232 20130101; G06K 9/78 20130101; H04N
5/225 20130101; G06Q 30/0633 20130101; G06K 9/6201 20130101; G06T
2207/30232 20130101; G07G 1/0054 20130101; G06K 9/00221 20130101;
G06Q 30/0269 20130101; G06K 9/6267 20130101; G06Q 30/0635 20130101;
G06Q 20/204 20130101; G06F 3/012 20130101; G07G 1/0072 20130101;
G01G 19/4144 20130101; G06K 9/00624 20130101; G06Q 20/18 20130101;
G06Q 30/0251 20130101; H04N 7/181 20130101; G01B 11/00 20130101;
G06K 9/00577 20130101; G06Q 30/0623 20130101; G06K 9/00536
20130101; G06T 7/66 20170101; G06Q 20/208 20130101 |
International
Class: |
G06Q 30/02 20060101
G06Q030/02 |
Claims
1. A computer-implemented method of providing rewards to a person
based on item interactions during a transaction in an environment,
the person associated with a personal profile, the method
comprising: generating, based on information in the personal
profile, one or more tasks for the person to complete during the
transaction, each task including at least one item interaction;
presenting, using an output device within the environment, the one
or more tasks to the person; analyzing image information acquired
by one or more visual sensors within the environment to evaluate
the person's performance of the one or more tasks; and awarding,
based on the evaluation, an amount of rewards to the person.
2. The method of claim 1, wherein the amount of rewards are applied
to one of the personal profile and a virtual transaction record
reflecting the person's item interactions during the
transaction.
3. The method of claim 1, wherein the information in the personal
profile includes one or more of an item interaction history, item
preferences, health information, prospective item information for
the transaction, and participation information for one or more
initiatives.
4. The method of claim 1, wherein generating the one or more tasks
is further based on one or more of item incentive information,
initiative information, and item inventory information.
5. The method of claim 4, wherein the item inventory information
indicates a first item nearing a corresponding expiration date, and
wherein at least one task includes the person locating the first
item.
6. The method of claim 1, wherein the one or more tasks includes
locating a selected item, wherein presenting the one or more tasks
includes providing descriptive information about the item without
explicitly identifying the item.
7. The method of claim 1, wherein generating the one or more tasks
includes: determining an entry of a shopping list included in the
personal profile; and determining a categorical grouping that
corresponds to the entry and that includes a plurality of items,
wherein the one or more tasks are presented with respect to the
determined grouping, and wherein evaluating the performance of the
one or more tasks is based on which of the plurality of items is
involved in the corresponding item interaction.
8. A computer program product of providing rewards to a person
based on item interactions during a transaction in an environment,
the person associated with a personal profile, the computer program
product comprising: a computer-readable storage medium having
computer-readable program code embodied therewith, the
computer-readable program code executable by one or more computer
processors to perform an operation that includes: generating, based
on information in the personal profile, one or more tasks for the
person to complete during the transaction, each task including at
least one item interaction; presenting, using an output device
within the environment, the one or more tasks to the person;
analyzing image information acquired by one or more visual sensors
within the environment to evaluate the person's performance of the
one or more tasks; and awarding, based on the evaluation, an amount
of rewards to the person.
9. The computer program product of claim 8, wherein the amount of
rewards are applied to one of the personal profile and a virtual
transaction record reflecting the person's item interactions during
the transaction.
10. The computer program product of claim 8, wherein the
information in the personal profile includes one or more of an item
interaction history, item preferences, health information,
prospective item information for the transaction, and participation
information for one or more initiatives.
11. The computer program product of claim 8, wherein generating the
one or more tasks is further based on one or more of item incentive
information, initiative information, and item inventory
information.
12. The computer program product of claim 11, wherein the item
inventory information indicates a first item nearing a
corresponding expiration date, and wherein at least one task
includes the person locating the first item.
13. The computer program product of claim 8, wherein the one or
more tasks includes locating a selected item, wherein presenting
the one or more tasks includes providing descriptive information
about the item without explicitly identifying the item.
14. The computer program product of claim 8, wherein generating the
one or more tasks includes: determining an entry of a shopping list
included in the personal profile; and determining a categorical
grouping that corresponds to the entry and that includes a
plurality of items, wherein the one or more tasks are presented
with respect to the determined grouping, and wherein evaluating the
performance of the one or more tasks is based on which of the
plurality of items is involved in the corresponding item
interaction.
15. A system of providing rewards to a person based on item
interactions during a transaction in an environment, the person
associated with a personal profile, the method comprising: one or
more computer processors; at least a first visual sensor disposed
within the environment and communicatively coupled with the one or
more computer processors; and a memory containing program code
which, when executed by the one or more computer processors,
performs an operation that includes: generating, based on
information in the personal profile, one or more tasks for the
person to complete during the transaction, each task including at
least one item interaction; presenting, using an output device
within the environment, the one or more tasks to the person;
analyzing image information acquired by one or more visual sensors
within the environment to evaluate the person's performance of the
one or more tasks; and awarding, based on the evaluation, an amount
of rewards to the person.
16. The system of claim 15, wherein the amount of rewards are
applied to one of the personal profile and a virtual transaction
record reflecting the person's item interactions during the
transaction.
17. The system of claim 15, wherein the information in the personal
profile includes one or more of an item interaction history, item
preferences, health information, prospective item information for
the transaction, and participation information for one or more
initiatives.
18. The system of claim 15, wherein generating the one or more
tasks is further based on one or more of item incentive
information, initiative information, and item inventory
information.
19. The system of claim 18, wherein the item inventory information
indicates a first item nearing a corresponding expiration date, and
wherein at least one task includes the person locating the first
item.
20. The system of claim 15, wherein generating the one or more
tasks includes: determining an entry of a shopping list included in
the personal profile; and determining a categorical grouping that
corresponds to the entry and that includes a plurality of items,
wherein the one or more tasks are presented with respect to the
determined grouping, and wherein evaluating the performance of the
one or more tasks is based on which of the plurality of items is
involved in the corresponding item interaction.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims benefit of United States provisional
patent application Ser. No. 62/064,323, filed Oct. 15, 2014,
entitled "Integrated Shopping Environment," which is herein
incorporated by reference.
BACKGROUND
[0002] The present disclosure relates to a sensor-based
environment, and more specifically, to techniques for
identification of items selected by a person in the environment
using visual identification and using personal profile information
associated with the person.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0003] FIG. 1 illustrates an exemplary environment including a
plurality of items, according to one embodiment.
[0004] FIG. 2 illustrates an exemplary system of providing rewards
to a person based on item interactions during a transaction in an
environment, according to one embodiment.
[0005] FIG. 3 is a block diagram illustrating operation of a system
of providing rewards to a person based on item interactions during
a transaction in an environment, according to one embodiment.
[0006] FIG. 4 illustrates a method 400 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment.
[0007] FIG. 5 illustrates a method 500 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment.
[0008] FIG. 6 illustrates a method 600 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment.
[0009] FIG. 7 illustrates a method 700 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment.
[0010] FIG. 8 illustrates a method 800 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment.
[0011] To facilitate understanding, identical reference numerals
have been used, where possible, to designate identical elements
that are common to the figures. It is contemplated that elements
disclosed in one embodiment may be beneficially utilized on other
embodiments without specific recitation. The illustrations referred
to here should not be understood as being drawn to scale unless
specifically noted. Also, the drawings are often simplified and
details or components omitted for clarity of presentation and
explanation. The drawings and discussion serve to explain
principles discussed below, where like designations denote like
elements.
DETAILED DESCRIPTION
[0012] Aspects of the current disclosure relate to an integrated
environment capable of providing a personalized, automated, and
adaptive experience for a person within the environment. A number
of different sensor devices may be employed within the environment,
and networked with various computing devices such as point-of-sale
(POS) terminals, digital signage, servers, and mobile or handheld
computing devices to provide a seamless integration of mobile
technologies and e-commerce into traditional experiences.
[0013] Using a system having one or more visual sensors within the
environment, a retailer or other provider may compile and process
environmental data related to a person's transaction within the
environment. The environmental data may be used to identify
interactions of the person with various items in the environment,
which may include selecting items for presentation during a
subsequent checkout transaction. The identification of the
interactions and items may be based on an analysis of static
imagery--such as an image of contents of the person's shopping cart
at a particular time--and/or dynamic imagery (e.g., using a
continuous video feed). The identification may occur at different
times and places during the transaction within the environment,
such as while the person browses items, while scanning items during
a checkout transaction, and so forth.
[0014] A rewards program may be administered within an environment,
incentivizing certain tasks performed by persons such as customers.
Rewards programs may be implemented in various forms, such as games
or other tasks for the person to perform during the transaction,
which are incentivized as appropriate. Generally, the benefits
provided through rewards programs may keep customers more engaged
during their transactions, and therefore more likely to return for
subsequent transactions. Performing the rewards tasks may benefit
the individual customers--offering monetary discounts or other
incentives--but may also benefit the environment administrators
(e.g., retailers) and/or item manufacturers due to the increased
visibility of items and possibility of increased sales. In some
cases, the rewards tasks may further benefit the environment
administrators through customers' willing participation in
otherwise menial tasks, which may allow for a better distribution
of employees to provide service to customers. All in all, rewards
programs may provide a better experience for customers,
individually as well as in the aggregate.
[0015] However, not all tasks are equally appealing or interesting
to all customers. In various embodiments, personal profile
information that is compiled for an individual person may be used
to adapt the tasks to the personal interests of the person. The
personal profile information may include an item history,
preferences, as well as other information. By using the personal
profile information to make the tasks more relevant and interesting
to the person, the chances of willing participation by the person
in the rewards program are increased.
[0016] While generally discussed within the context of a shopping
environment, such as a retail store or commercial environment, it
is contemplated that the techniques disclosed herein may be applied
to other environments (some non-limiting examples include
libraries, museums, classrooms, hospitals, etc.) to provide a
similar experience for persons included therein.
[0017] FIG. 1 illustrates an exemplary environment including a
plurality of items, according to one embodiment. The environment
100 includes a plurality of sensor modules 105 disposed in the
ceiling 110 of the environment. The sensor modules 105 may each
include one or more types of sensors, such as video sensors (e.g.,
cameras), audio sensors (e.g., microphones), and so forth. Sensor
modules 105 may also include actuating devices for providing a
desired position and/or orientation of the included sensor(s).
Generally, the sensor modules or individual sensors may be disposed
at any suitable location within the environment 100. Some
non-limiting examples of alternative locations include below,
within, or above a floor 115 of the environment, within other
structural components of the environment 100 such as a shelving
unit 120 or walls, and so forth. In some embodiments, sensors may
be disposed on, within, or near item display areas such as the
shelving unit 120. The sensors may be oriented toward expected
locations of personal interactions with items in order to acquire
better data about the person's interactions, such as determining
the person's field of view relative to certain items, updating a
virtual cart or transaction record for the person's transaction in
the environment, and so forth.
[0018] Environment 100 also includes a number of computer-based
terminals (or kiosks) 125. Generally, terminals 125 may be
configured for performing customer checkout and/or other functions,
such as providing information to a customer or employee. Each
terminal 125 may each include a discrete computing device or
portions of a computing system, and may include various I/O
devices, such as visual displays, audio speakers, cameras,
microphones, etc. for interacting with various persons such as
customers and/or employees. In some embodiments, a person 130 in
the environment may have a mobile computing device, such as a
smartphone 135, that communicatively couples with the terminal 125
for completing a checkout transaction. For example, the person's
smartphone 135 may include payment information, identification
information, etc. that facilitate completion of the checkout
transaction. In one embodiment, the mobile computing device may
execute a store application that connects with the computing system
of the environment (e.g., to store servers or other computing
devices through the Internet). In one embodiment, the mobile
computing device may be directly connected with kiosk 125 through
wireless networks established within the environment (e.g., over
Wi-Fi or Bluetooth). In one embodiment, the mobile computing device
may couple with the kiosk 125 when brought within range, e.g.,
using Bluetooth or near-field communication (NFC).
[0019] Environment 100 also includes one or more shelving units 120
having shelves 140 that support various store items 145. Though not
shown, multiple shelving units 120 may be disposed in a particular
arrangement in the environment 100, with the space between adjacent
shelving units forming aisles through which customers and employees
may travel. For example, customers may navigate the aisles and/or
approach the shelving units 120 to view items 145 included therein,
to handle the items, to select the items, etc. In another example,
employees may navigate the aisles and/or approach the shelving
units 120 to view stock levels of the items 145, to determine
out-of-place items, etc. In some embodiments, shelving units 120
may include visual sensors or other sensor devices or I/O devices.
The sensors or devices may couple with the person's smartphone 135
and/or other networked computing devices (including terminals 125
and/or servers) that are associated with the environment 100. For
example, the front portions 150 of shelves 140 may include video
sensors oriented outward from the shelving unit 120 (i.e., toward
the aisle) to acquire image information for a person's interactions
with items 145 on the shelving unit 120, with the image information
provided to back-end servers for storage and/or analysis. In some
cases, some or all of the image information may also be accessible
by a person's mobile computing device. In some embodiments,
portions of the shelving unit 120 (such as the front portions 150
of shelves 140) may include indicator lights or other visual
display devices or audio output devices that are able to
communicate with a person.
[0020] During an exemplary transaction in the environment, the
person 130 may have a shopping receptacle in which the person
places items after they are selected for purchase. Examples of
shopping receptacles include shopping carts, baskets, or other
containers that may be carried or otherwise transported by the
person during the transaction. Upon completion of the
transaction--for example, the person has selected all of the
desired items--the person may approach one of the terminals 125 or
a designated checkout area to perform a checkout transaction.
[0021] In some cases, the checkout transaction may have "touchless"
aspects or may be entirely touchless. For example, visual sensors
included in the environment and/or within the approached terminal
125 may acquire image information that is usable to identify the
person, items included within the shopping receptacle, etc. and
that streamlines or otherwise facilitates the checkout transaction.
As will be discussed further herein, logic may be applied to
enhance the analysis of the image information using personal
profile information associated with the person conducting the
transaction. Application of the personal profile information may
result in item identification processes having greater accuracy
and/or confidence levels, as well as being performed more quickly.
The improved accuracy and speed may help the person to spend a
shorter time completing their checkout transaction, which can
improve their overall transaction experience. Reducing time for
checkout transactions also increases the collective throughput at
the checkout area. In some cases, a person may be able to complete
a checkout transaction simply as part of departing the environment,
without requiring the person to stop at a checkout terminal or in
the checkout area. In some cases, the person's time in the checkout
area may be significantly reduced, such as only a momentary pause
at a checkout terminal.
[0022] FIG. 2 illustrates an exemplary system of providing rewards
to a person based on item interactions during a transaction in an
environment, according to one embodiment. The system 200 includes a
number of components that are disposed within the environment 100.
The system may also include components that are outside the
environment--for example, a server 265 may be located remotely or
proximately disposed to the environment (such as within a back room
in the same building that is not accessible by customers).
[0023] Components within the environment include one or more
sensors 205 of various types, such as visual sensors 210, audio
sensors 215, and weight sensors 220. The sensors 205 may also
include other sensors 225 capable of providing meaningful
information about personal interactions within the environment,
e.g., location sensors. The sensors 205 may be discrete sensor
devices deployed throughout the environment 100 in fixed and/or
movable locations. Sensors 205 may be statically included in walls,
floors, ceilings, displays, or other non-sensor devices, or may be
included in shopping receptacles capable of being transported
through the environment. For example, weight sensors 220 may be
disposed in fixed locations within the environment, such as within
the floor or within a surface of a checkout terminal, and may also
include load cells or other sensors disposed in a basket portion of
a shopping receptacle. In one embodiment, sensors 205 may include
adjustable-position sensor devices, such as motorized cameras
(i.e., an example of visual sensors 210) attached to a rail, wire,
or frame. In one embodiment, sensors 205 may be included on one or
more unmanned vehicles configured to travel through some or all of
the environment 100, such as unmanned ground vehicles (UGVs) or
unmanned aerial vehicles (UAVs or "drones"). Sensors 205 may also
include sensor devices that are included in computing devices
associated with the environment 100, such as personal devices 230,
employee devices 235, and terminals 240. In some cases, the
computing devices (or the component sensor devices) may be
implemented as body-worn or carried devices.
[0024] Personal devices 230 and employee devices 235 may each
include passive or actively-powered devices capable of
communicating with at least one of the networked devices of system
200. One example of a passive device (which may be worn or carried)
is a NFC tag. Active devices may include mobile computing devices,
such as smartphones or tablets, or wearable devices such as a
Google Glass.TM. interactive eyepiece (Glass is a trademark of
Google Inc.). The personal devices 230 generally denotes ownership
or possession of the devices by customers within the environment
100, while the employee devices 235 denotes ownership or possession
by the retailer or other administrator of the environment 100. In
some cases, employee devices 235 may be carried by employees and
used in the course of their employment. Personal devices 230 and
employee devices 235 may execute applications or other program code
that generally enables various functions and features accessible
using server 265 and/or other networked computing devices. In some
embodiments, sensor devices that are included with the personal
devices 230 or employee devices 235 may be included in the sensors
205.
[0025] System 200 includes a plurality of terminals 240 within the
environment 100. Terminals 240 generally include any structure that
is capable of receiving input from and/or producing output to
people (e.g., customers, employees) within the environment 100. The
terminals 240 may include computing systems, portions of computing
systems, or devices controllable by computing systems. In one
example, a terminal 240 may include a computing device that is
communicatively coupled with a visual display and audio speaker(s),
as well as being communicatively coupled with one or more input
devices. In another example, a terminal 240 may include a visual
display and associated driver hardware, but a computing device
coupled to the terminal and providing data for display is disposed
separately from the terminal. In some embodiments, terminals 240
may be implemented as standalone devices, such as a kiosk disposed
on the store floor or monolithic device disposed on a shelf or
platform. In some embodiments, terminals 240 may be integrated
partially or wholly with other components of the environment 100,
such as input or output devices included with shelving or other
structural components in the environment (e.g., components used for
product display or storage). In some embodiments, terminals 240 may
be modular and may be easily attachable and detachable to elements
of the environment 100, such as the structural components.
[0026] Generally, terminals 240 may be distributed throughout the
environment 100 and may enhance various phases of the person's
transactions within the environment. For example, terminals 240 may
include digital signage (i.e., included as an example of other
terminals 255) disposed throughout the environment, such as
included in or near aisles, endcaps, displays, and/or shelving in
the environment. A person during a transaction may view and/or
interact with the digital signage as he or she moves throughout the
environment. The digital signage may be included in a static
display or may be movable, such as including digital signage within
a shopping receptacle. Terminals 240 may also include one or more
types of terminals usable for completing checkout transactions,
such as employee-manned POS terminals 245 and self-checkout
terminals 250. In some cases, the terminals 240 that provide
checkout functionality may be disposed within a designated checkout
area within the environment 100.
[0027] In some embodiments, terminals 240 may provide an integrated
functionality. For example, terminals 240 may function in a first
mode as digital signage, and when engaged by a person (i.e.,
receiving input from the person), the terminals function in a
second mode as a self-checkout terminal or other type of
terminal.
[0028] Server 265 generally includes processor(s), memory, and
communications capabilities and may perform various computing tasks
to support the operation of the environment 100. Server 265 may
communicate using various wired and/or wireless communications
methods with terminals 240, sensors 205, and with other networked
devices such as personal devices 230 and employee devices 235.
Server 265 generally executes computer program code in which input
data is received from networked devices, the input data is
processed and/or stored by the servers, and output data is provided
to networked devices for operation of the environment 100.
[0029] Network 260 may include one or more networks of various
types, including a local area or local access network (LAN), a
general wide area network (WAN), and/or a public network (e.g., the
Internet). In one embodiment, various networked computing devices
of the system 200 are interconnected using a LAN, and one or more
computing devices (e.g., server 265, personal devices 230) include
connections to the Internet.
[0030] FIG. 3 is a block diagram illustrating operation of a system
of providing rewards to a person based on item interactions during
a transaction in an environment, according to one embodiment.
Specifically, the arrangement 300 illustrates operation of the
system 200. Arrangement 300 includes a number of processors 305,
memory 310, and input/output 315, which are interconnected using
one or more connections 320. In one embodiment, the arrangement 300
may be implemented as a singular computing device and connection
320 may represent a common bus. In other embodiments, arrangement
300 is distributed and includes a plurality of discrete computing
devices that are connected through wired or wireless networking.
The processors 305 may include any processing element suitable for
performing functions described herein, and may include single or
multiple core processors, as well as combinations thereof.
Processors 305 may be included in a single computing device, or may
represent an aggregation of processing elements included across a
number of networked devices such as the personal devices 230,
terminals 240, etc. discussed above.
[0031] Input/output (I/O) 315 includes one or more output devices
370 and one or more sensors 205. Output devices 370 include one or
more devices for presenting information to customers and generally
include audio output devices 371 and/or visual output devices 372.
The audio output devices 371 may include conventional audio
speakers having any suitable form factor (e.g., standalone,
integrated in a stereo, headphones, etc.), as well as devices using
alternative methods of producing sound perceptible by a person such
as a customer or employee, such as bone conduction transducers in a
worn device. Visual output devices 372 may include visual displays
and various visual indicators such as light emitting diodes (LEDs).
In some embodiments, the terminals 240, personal devices 230, and
employee devices 235 of FIG. 2 may include the output devices 370,
such as visual devices 372 (e.g., visual displays, indicators)
and/or audio devices 371 (e.g., speakers) for communicating with
persons during their transactions. Other output devices 373 may
provide information to customers through tactile feedback (e.g.,
haptic devices) or using other sensory stimuli. Sensors 305 may
include visual sensors 210 which may be carried or worn sensors
374, and distributed sensors 376 that are disposed throughout the
environment. In one embodiment, the distributed sensors 376 are
disposed in a static arrangement in the environment. In one
embodiment, at least some of the distributed sensors 376 are
movable. For example, the distributed sensors 376 may be included
on movable product displays or structures, and/or unmanned vehicles
(e.g., aerial or ground-based vehicles). Other sensors 378 may also
be included that are suitable for collecting information about a
person and his/her interactions within the environment. Examples of
other sensors 378 include without limitation infrared (IR) sensors,
thermal sensors, weight sensors, capacitive sensors, magnetic
sensors, sonar sensors, radar sensors, lidar sensors, and so
forth.
[0032] I/O 315 may also include input devices 368 suitable for
receiving input from persons, such as cameras, keyboards or
keypads, touchscreens, buttons, inertial sensors, etc. I/O 315 may
further include wired or wireless connections to an external
network (e.g., network 260) using I/O adapter circuitry.
[0033] The visual sensors 210 may be used to capture image
information 346 including the person and/or the environment, which
may include views from various perspectives (e.g., a customer-worn
visual sensor, static or movable visual sensors at various
locations in the environment). The image information 346 may be
stored in memory 310, and may be individually or collectively
processed by processors 305 to determine information about persons
within the environment and their respective interactions with items
in the environment.
[0034] Memory 310 may include a variety of computer-readable media
selected for their size, relative performance, or other
capabilities: volatile and/or non-volatile media, removable and/or
non-removable media, etc. Memory 310 may include cache, random
access memory (RAM), storage, etc. Storage included as part of
memory 310 may typically provide a non-volatile memory for the
networked computing devices (e.g., server 265), and may include one
or more different storage elements such as Flash memory, a hard
disk drive, a solid state drive, an optical storage device, and/or
a magnetic storage device. Memory 310 may be included in a single
computing device or may represent an aggregation of memory included
in networked devices. Memory 310 may include a plurality of modules
for performing various functions described herein. The modules
generally include program code that is executable by one or more of
the processors 305. As shown, modules include an image analysis
module 348, a rewards module 350, and an inventory management
module 358. The modules may also interact to perform certain
functions. For example, rewards module 350 may make calls to the
image analysis module 348 to evaluate using image information 346 a
person's performance of a presented task. The person of ordinary
skill will recognize that the modules provided here are merely
non-exclusive examples; different functions and/or groupings of
functions may be included as desired to suitably operate the
environment.
[0035] Memory 310 includes a plurality of personal profiles 322
corresponding to the different persons. In some cases, the personal
profiles 322 may be associated with a loyalty program administered
in the environment. During operation of the system 200, the
personal profiles 322 may be associated with a current transaction
for a person after identifying the person using the person's
portable computing device (e.g., a login using a mobile computing
app), a visual scan of the person's loyalty card or the person's
face or other identifying features, etc. In one embodiment, the
personal profiles 322 and at least some of the image information
346 may be stored on the server 265 or on a separate database.
[0036] The personal profiles 322 include a number of different
types of information that may be useful to augment visual
identification of items. An item interaction history 324 may
reflect the person's interactions with various items in the
environment, which may have occurred during previous transactions
and/or earlier during the current transaction. The item interaction
history 324 may be adaptively updated as image information is
acquired and the person's identified interactions are added into
the virtual transaction record 349. Some non-limiting examples of
item interactions include viewing items 325, handling items 326,
and purchasing items 327. The item interactions may be directly
observed and/or deduced using visual sensors 210 in the
environment. In one example of direct observation, a person wears a
forward-looking visual sensor that generally represents their field
of view (i.e., including viewed items) at a given time. In one
example of a deduced observation, a gaze-tracking camera tracks the
eye movement of the person, but its information alone may be unable
to discern what items the person is viewing. However, using image
information to determine the person's position within the
environment and the relative orientation of their head, and perhaps
referenced with item location information, the system may be able
to determine what items are being viewed by the person and add this
information into the viewing items 325 portion of the item
interaction history 324.
[0037] Personal profiles 322 also include personal preferences 328
reflecting preferences that are explicitly specified by the person
(explicit preferences 329), or that may be deduced by the system
based on other information included in the personal profile 322
(deduced preferences 330), such as the person's item interaction
history 324. The person may specify preferences using a personal
computing device (e.g., through a mobile computing device app) or
through terminals or other computing devices accessible in the
environment. Personal preferences 328 may reflect preferences of
one item in relation to another (e.g., apples tend to be purchased
at the same time as oranges), one grouping of items in relation to
another (e.g., junk food preferred over vegetables), transaction
preferences, and so forth. In one example, a person may prefer to
purchase their fresh produce during a certain transaction that
occurs with some regularity, say every couple of weeks. Therefore,
when a first item is selected that is positively identified as a
fresh vegetable, it is likely that a significant number of other
items in the same transaction will also be fruits and/or
vegetables. This information may be used to help identify items
detected by the image analysis.
[0038] Personal profiles 322 may also include health information
332 for the person. For example, a person could specify that they
have certain allergies or other medical conditions 333. For
example, a person may have allergic reactions to certain items such
as nuts, shellfish, eggs, etc. The other medical conditions, while
perhaps not as serious as an allergic reaction, may still cause
discomfort to the person. For example, oils in mango skin may cause
dermatitis for persons who are sensitive to poison ivy. Items that
are related to the allergies and other conditions 333 are generally
less likely to be selected by the person during a transaction. In
some embodiments, the allergies and other conditions 333 may be
used to remove certain possible items from consideration while
performing item identification, as it would be highly unlikely that
the person would be selecting an item to which they are allergic.
Health information 332 may also include one or more nutrition goals
334 and fitness goals 335 for the person.
[0039] For example, nutrition goals 334 may include a set of values
representing various nutrients (calories, vitamins, minerals,
etc.), which may be provided as daily recommendations or perhaps
indicated as part of a specific diet plan for the person. The
nutrition goals 334 may interact with or may partially overlap with
the fitness goals 335 (e.g., caloric content related to weight
loss).
[0040] Personal profiles 322 may also include prospective item
information 336 that suggests that a person may select a particular
type of item during the transaction. The prospective item
information 336 may be indicated by the person or deduced. For
example, the person may enter a shopping list or wish list 337
using a mobile computing device app, and the items listed therein
are very likely to be selected by the person during the
transaction. In another example, a recipe 338 is entered by the
person in the mobile computing device app (e.g., the person wishes
to purchase the component ingredients to prepare the recipe).
Alternatively, recipes may be accessed by the system 200 responsive
to a number of items previous identified during the transaction.
For example, say a person selects containers of peanut butter and
of jelly during a particular transaction. The system may search for
recipes that include both of those ingredients--one of which is a
peanut butter and jelly sandwich. Based on the information in this
recipe 338, it is more likely that the person will also select
bread during the transaction. Accordingly, a visually detected item
with physical attributes roughly corresponding to a loaf of bread
(e.g., having an approximate size and shape as the loaf) may be
more easily identified as the loaf of bread, without the need for
full image processing.
[0041] Memory 310 includes a virtual transaction record 349
reflecting the person's item interactions, which may be determined
using acquired image information 346. In some embodiments, the
virtual transaction record 349 is included in the personal profile
322; in some embodiments, the two may be separately maintained in
memory 310. The person's interactions with different items may be
identified using image information, and those interactions stored
in the virtual transaction record 349. When items are identified as
being selected by the person, the selected items may be added to
item lists (e.g., a virtual cart) within the virtual transaction
record 349. The item lists are then used as part of completing the
checkout transaction. In some cases, as the virtual transaction
record 349 is updated, corresponding portions of the item
interaction history 324 are also updated.
[0042] Image information 346 is generally acquired using the visual
sensors 210. The image information 346 may be maintained in any
form suitable for performing image analysis--one or more still
images, video, and so forth. In some cases, the image information
346 does not include labeling (such as a universal product code
(UPC) or quick response (QR) code) or other encoded information
that is sufficient to uniquely identify the item. In some cases,
the image information 346 includes some encoded information, but
not enough to positively identify the corresponding item (e.g., the
barcode is partially obscured). In some embodiments, partial
encoded identification information may be used to narrow a set of
possible items for item identification.
[0043] Image analysis module 348 is configured to receive image
information 346 as inputs, and to perform analysis according to any
suitable techniques. Generally, the image analysis module 348 may
perform edge detection or comparable techniques to define areas
within the image information 346 that correspond to different
items. The image analysis module 348 may further compare the image
information 346 with reference image information 343 to attempt to
identify the detected items. In some cases, reference image
information 343 may be intentionally simplified, such as acquiring
the reference image information under controlled conditions,
including only a singular item within the reference image
information, and so forth.
[0044] The image analysis module 348 may further be able to detect
predefined gestures or other behaviors by a person relative to
particular items. In some embodiments, an item interaction
represents a determined behavior of the person relative to an
identified item. Some non-limiting examples of item interactions
include viewing items 325, handling items 326, and purchasing items
327. In some embodiments, the image analysis module 348 is
configured to output item interactions to the virtual transaction
record 349 and/or to the rewards module 350. In some embodiments,
the identified items and behaviors are output to the virtual
transaction record 349, where the item interactions are determined
(i.e., association of items with behaviors). The virtual
transaction record 349 may then provide the determined item
interactions to the rewards module 350.
[0045] The item interactions (including identified items and
behaviors) from the image analysis module 348 may be provided as
inputs to the rewards module 350. Rewards module 350 is generally
configured to administer the rewards component of the system 200,
which incentivizes a person for performing certain tasks within the
environment. The incentives may be provided to the person in any
suitable form, such as money, points, discounts, coupons,
promotions, giveaways, etc. In some cases, a rewards component of
an environment may be completely responsive, with the incentives
determined based on actions the person has already taken within the
environment. In various embodiments described herein, the rewards
component may include proactive aspects, providing specific tasks
for the person to perform during their transaction within the
environment. While a person's participation in the provided rewards
tasks is not guaranteed, determining suitable tasks and appropriate
incentives for the tasks increases the chance that the person will
participate. The rewards module 350 includes a number of functional
sub-modules 351, which include task generation 352, evaluation 354,
and award 356. The rewards module 350 may further include one or
more predefined games 353 that provide a context or cohesive theme
for the individually generated tasks. The addition of context, such
as animated characters, a story or narrative, or other audiovisual
elements, may increase a level of interest or engagement for the
person, further encouraging the person's participation in the
rewards tasks.
[0046] The task generation sub-module 352 determines one or more
tasks for the person to perform within the environment during their
transaction. Each task may include one or more specific item
interactions for the person to perform. Generally, the task
generation sub-module 352 may use information associated with the
person, such as the personal profile 322 and the virtual
transaction record 349, to generate tasks that are tailored to the
person's interests or that are otherwise intended to influence the
person's actions during the transaction.
[0047] The task generation sub-module 352 may also use item
information 342 to determine the one or more tasks to be performed.
Item information 342 includes location information 344, which may
be used to select appropriate items to be included in the tasks.
For example, based on the person's location in the environment,
remaining items on his or her shopping list 337, etc., the task
generation sub-module 352 may select items that are relatively
close (e.g., within several feet, along the same aisle, etc.) to
the location or to anticipated locations of the person. In this
way, the tasks may be perceived as less inconvenient for the
person. Generally, less inconvenience in the tasks may correlate to
greater participation of persons in the rewards tasks.
[0048] In additional to personal information and item information,
the tasks generated by the task generation sub-module 352 may also
reflect other competing considerations, such as inventory
information, incentive information, and initiatives. For example,
an inventory management module 358 may operate to track inventory
levels for the various items on display in the environment, as well
as additional item inventory (say, located in nearby store rooms,
warehouses, etc.) for restocking the environment, and ordering
information to timely acquire additional items. The inventory
management module 358 may include item expiration information 359
that may be used for item restocking, rotation, replacement, etc.
as necessary. For example, the item expiration information 359 may
be used to generate prompts for employees to locate and sort out
expired or near expiring items from the rest of the items on
display.
[0049] One or more incentives 360 may be available for promoting
various items. Incentives 360 may have any suitable form, such as
discounts, offers, rebates, etc. for the associated items.
Incentives may originate from different sources, such as from the
item manufacturer 361 and from the environment 362 (e.g.,
store-specific). In some cases, incentives may be available from
different sources corresponding to the same item. To resolve any
incompatibility between the different incentives, the incentives
may be prioritized according to the source, the amount, etc. For
example, incentives provided at the environment level may override
any conflicting incentives from the item manufacturer. In another
example, the "better" (i.e., greater amount) of the incentives may
be given priority regardless of source.
[0050] One or more initiatives 364 may be used to promote various
features associated with the environment. In one embodiment, a
website initiative 365 may encourage a person to access a website
related to the environment using their personal computing device.
An example operation of a website initiative 365 is discussed below
with respect to FIG. 7. The initiatives 364 may include one or more
other features 366 that promote any desired aspects (virtual or
physical) associated with the environment. For one example
initiative, a corresponding task may require that the person access
item information using a mobile computing app associated with the
environment. In another example initiative, corresponding tasks may
direct shoppers to visit a newly-remodeled section of the
environment.
[0051] The generated tasks may be presented to the person using a
suitable output device 370 located within the environment. For
example, the tasks may be transmitted to a personal computing
device that belongs to the person, or to a computer-based terminal
disposed within the environment and suitably close to the person's
location or along the person's anticipated route within the
environment. The generated tasks may be presented to the person in
a direct or an indirect manner. A direct presentation of the task
could ask the person to interact with the specific item--for
example, locate item x. However, an indirect presentation of the
task may use descriptive information about the particular item,
requiring the person to consider the descriptive information or to
search a plurality of different items to find the particular item.
For example, a task may require a person to locate an item having
particular nutrition information 345. In some cases, the
descriptive information may be in the form of a riddle or puzzle to
be solved by the person prior to completion of the task. In some
cases, using an indirect approach may increase the person's
engagement with the rewards tasks and increase their overall
willingness to participate.
[0052] In addition to associated items and behaviors, the generated
tasks may also specify one or more criteria for task completion or
for rewards scoring. Criteria may include a time element (e.g.,
perform this task within three minutes) and/or specific aspects of
task execution (e.g., locate the item and capture an image of the
associated labeling, barcode, etc.).
[0053] The evaluation sub-module 354 receives as input one or more
of the generated tasks, as well as performance information based on
acquired image information 346, and scores the performance of the
tasks based on the performance information. In some embodiments,
the evaluation sub-module 354 associates item and behavior
information received from the image analysis module 348 to
determine the item interaction. In other embodiments, the image
analysis module 348 or the virtual transaction record 349 provides
the determined item interaction to the evaluation sub-module 354.
The determined item interaction is compared with the generated
tasks to determine whether a particular task was completed, as well
as whether it was performed according to prescribed criteria. The
evaluation sub-module 354 may use any suitable algorithm(s) for
scoring performance of the tasks.
[0054] The award sub-module 356 receives scoring information from
the evaluation sub-module 354 assessing the person's performance of
the task. The scoring information may be provided in any suitable
fashion--an overall score of the task performance, component scores
related to the prescribed performance criteria, and so forth. The
award sub-module 356 receives the scoring information and
determines a corresponding amount of awards to reward to the
person's account. The particular task and the performance score may
be combined in a predefined relationship to determine the amount of
rewards, for example, according to any suitable equation. In some
cases, the tasks may be given different weights depending on their
relative importance, which may be determined using inventory
information (e.g., a need to quickly sell the associated item),
incentive information, initiative information, and so forth.
[0055] The award sub-module 356 may associate the determined amount
of rewards with the person using the personal profile 322 and/or
the virtual transaction record 349. The rewards may be applied by
the person during the current transaction, or may accumulate for
application during a later transaction. In some cases, the rewards
may be tied to promotions outside the transaction and/or
environment, such as a mailed rebate or an electronic promotion to
an event.
[0056] FIG. 4 illustrates a method 400 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment. The method 400 may
generally be used in conjunction with the arrangement 300 described
above.
[0057] Method 400 begins at block 405, where one or more tasks are
generated for the person to complete during the transaction. The
tasks are generated based on information included in the personal
profile associated with the person, such as an item interaction
history, personal preference information, health information, and
prospective item information. Each task may relate to one or more
item interactions, which associate items in the environment with
certain predetermined behaviors.
[0058] At block 415, the one or more tasks are presented to the
person using an output device within the environment. In some
embodiments, the presentation may be provided to a personal
computing device associated with the person (such as a smartphone
or tablet). In some embodiments, the presentation may be provided
to a computing device disposed within the environment, such as a
terminal in the proximity of the person or along an anticipated
path of the person through the environment.
[0059] At block 425, acquired image information is analyzed to
evaluate the person's performance of the one or more presented
tasks. The image information is acquired using one or more visual
sensors disposed within the environment. In one or more
embodiments, the analysis may be further based on input provided by
other sensor devices or by the person through a user interface. In
some embodiments, the evaluation may be binary--that is, does the
image information (as represented in the determined item
interaction) indicate that the person completed the task? In other
embodiments, the evaluation may be based on one or more
criteria--for example, performing the task correctly, performing
the task within a prescribed window of time, within a predetermined
time limit, and so forth.
[0060] At block 435, an amount of rewards is awarded to the person
based on the evaluation. In some embodiments, the rewards are
associated with the person's personal profile and/or the virtual
transaction record (associated with the person's current
transaction). The rewards may be redeemed during the current
transaction or during a later transaction, consistent with the
overall rewards system and the person's preferences (e.g.,
accumulating rewards for a later transaction).
[0061] FIG. 5 illustrates a method 500 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment. The method 500 represents
one possible implementation of the method 400, and may generally be
used in conjunction with the arrangement 300 described above. In
one embodiment, the method 500 represents an item-matching game
available within the environment.
[0062] At block 505, at least one item is selected for a
prospective task. In some embodiments, selection of the item is
based on location information 510, such as person location
information 512 and/or item location information 344. The person
location information 512 may be provided directly or determined
through image analysis of image information 346. Selecting an item
closer to the person's location may cause the associated task to be
perceived as less inconvenient by the person. In some embodiments,
selection of the item may also be based on personal profile 322
information, such as the person's preferences 328 or prospective
item information 336. For example, the system may more frequently
select items with which the person has interacted or has a relative
preference. In some embodiments, the prospective item information
336 may be used to suggest alternative items (e.g., a different
brand of an item included on a shopping list, or a suitable
substitute item) to the person.
[0063] At block 515, descriptive information for the item is
selected. In some cases, the descriptive information may include
merely identification information (i.e., a unique name, size, etc.)
for the item. In some cases, however, descriptive information may
include attributes or other qualities of the item, such as
nutrition information, without providing unique identification
information. In some embodiments, the amount of descriptive
information included with a task is adaptive, based on the person's
preferences and/or other task generation considerations such as
incentives, inventory, initiatives, and so forth. Blocks 505 and
515 may be repeated as desired--e.g., as part of determining a
plurality of tasks for the person to perform.
[0064] At block 525, a task including the selected descriptive
information is presented to the person. In some embodiments, a
single task is presented at block 525. In other embodiments, more
than one task is presented (e.g., a checklist). The presented task
may be selected from a plurality of determined tasks for the
person. In some embodiments, the task is presented to the person
using a personal computing device. In some embodiments, the task is
presented using a computing device disposed within the environment,
such as a terminal located near the person's location.
[0065] At block 535, acquired image information is analyzed to
determine one or more item interactions performed by the person. At
block 540, the item interactions are evaluated to determine whether
any match the task. If at least one match is determined ("YES") the
method proceeds to block 545, where performance of the task is
evaluated according to one or more criteria. However, if no matches
are determined ("NO"), the method returns to block 535 to continue
analysis of image information.
[0066] At block 555, the rewards for the person are updated based
on the evaluation of task performance. The rewards may be
maintained as part of the personal profile and/or a virtual
transaction record for the current transaction. The grouping of
blocks 535, 540, 545, and 555 may be generally referred to as a
subroutine 565.
[0067] At block 560, the method determines whether all generated
tasks have been completed. If complete ("YES"), the method ends.
However, if not complete ("NO"), the method returns to one of block
525 and 535, depending generally on whether multiple tasks were
presented to the person. If multiple tasks were presented at once
to the person (e.g., as a checklist), the method may return to
block 535 to continue analysis of image information. However, if
only a single task was presented to the person at the first
instance, a next task is presented to the user when the method
returns to block 525.
[0068] FIG. 6 illustrates a method 600 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment. The method 600 represents
one possible implementation of the method 400, and may generally be
used in conjunction with the arrangement 300 described above. In
one embodiment, the method 600 represents an expired items game
available within the environment.
[0069] Method 600 begins at block 605, where the system determines
that one or more expiring items have either reached their
expiration date or are nearing their expiration date. The
determination may be based on information available through
inventory management module 358, such as the item expiration
information 359. A retailer or other provider (e.g., an item
manufacturer) may generally wish to remove expiring items from
display as customers or shoppers may perceive
[0070] At block 615, a task involving at least one of the expiring
items of block 605 is presented to the person during a transaction.
The task (i.e., the selected item and/or associated behavior) may
be selected for the person based on personal profile 322
information, such as preferences 328 and/or prospective item
information 336. For example, given two different types of expiring
items, one type of which the person does not prefer (i.e., a
negative preference), the system may tend to select the other type
to present as part of the task. The task may further be based on
location information 510 to reduce any inconvenience for the person
who is to perform the task.
[0071] Generally, the task may include locating an expiring item
and/or sorting the expiring item from other non-expiring items. In
some cases, the task may be associated with a discount on
purchasing the expiring item. By incentivizing customers or
shoppers to locate the expiring items, the retailer or environment
administrator may not need to send an employee to perform the same
task.
[0072] At block 625, subroutine 565 is performed. As discussed
above, the subroutine 565 generally includes analyzing image
information, determining a match with the presented task,
evaluating performance of the task, and updating rewards based on
the evaluation. At an optional block 630, the system determines
whether the presented task item had been sorted from other items,
but not selected by the person. For example, the person may have
located the expiring item consistent with the presented task, but
did not remove the item as he or she did not intend to purchase the
item. In this case, the retailer or environment administrator may
not wish to leave the expiring item in place, but may remove the
item to avoid selection by other shoppers. Accordingly, if the item
was sorted but not selected ("YES"), at block 635 the system may
notify an employee to collect the sorted item.
[0073] In an alternative embodiment, the presented task may
prescribe a placement of the expiring (but not yet expired) item,
to improve the likelihood that the expiring item is selected prior
to its expiration date. For example, an expiring item located near
the bottom of a stack of items may be located by the person and
placed near the top of the stack, so that the item is more visible
and may be more likely to be selected by other shoppers.
[0074] FIG. 7 illustrates a method 700 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment. The method 700 represents
one possible implementation of the method 400, and may generally be
used in conjunction with the arrangement 300 described above. In
one embodiment, the method 700 represents an online services game
available within the environment.
[0075] Method begins at block 705, where a task is presented
involving one or more initiatives and one or more items. The
initiatives 364 may specify one or more physical or virtual
features associated with the environment, such as website features,
mobile app features, etc.
[0076] The associated item for the task may be selected based on
personal profile 322 information, such as prospective item
information 336 and other information 340. In some cases, the
associated item for the task may be selected from prospective items
included in the prospective item information 336. For example, say
the initiative relates to using a mobile computing app to research
an item or view a comparison of items. One associated task may be
acquiring image information of an item using a mobile computing
device; the app then loads the information related to the item for
viewing by the person (i.e., the feature highlighted in the
initiative). Here, the system may select an item for the task based
on items that the person has included in a shopping list but has
not yet selected for purchase (e.g., by removing from display and
placing in a shopping receptacle). This information may be
reflected in the virtual transaction record 349. Another example of
a task is accessing a web page associated with an item that is
already selected or expected to be selected by the person (say,
based on prospective item information 336, purchase history, etc.).
The task may require that the person transmit a link to the page
(or other content from the page) to a computing device associated
with the environment (such as server 265).
[0077] By selecting items for a task in which the person may
already be interested, and by integrating the task into an action
that the person is expected to perform (e.g., selecting an item on
a shopping list), the level of participation in initiatives and
other rewards tasks may generally be increased.
[0078] The other information 340 may include initiatives
participation 710 information, which reflects the person's
participation in various initiatives. For example, the system may
determine not to present a task relating to an initiative which the
person has previously participated in. In one embodiment, the
initiatives participation 710 may be as simple as single data bits
indicating whether the person has participated or not. Of course,
the initiatives participation may include more in-depth information
about the nature of the participation, items involved, etc.
[0079] At block 715, subroutine 565 is performed. As discussed
above, the subroutine 565 generally includes analyzing image
information, determining a match with the presented task,
evaluating performance of the task, and updating rewards based on
the evaluation. Of note, the acquired image information may be
acquired as part of the person's performance of the task--such as
capturing an image of the item included in the presented task--or
perhaps reflected in image information that includes the person
manipulating the computing device to perform the task.
[0080] FIG. 8 illustrates a method 800 of providing rewards to a
person based on item interactions during a transaction in an
environment, according to one embodiment. The method 800 represents
one possible implementation of the method 400, and may generally be
used in conjunction with the arrangement 300 described above. In
one embodiment, the method 800 represents a profile matching game
available within the environment.
[0081] Method 800 begins at block 805, where an entry is determined
from a shopping list 337 of a person's personal profile 322. In an
alternative embodiment, other prospective item information
associated with the personal profile 322 may be substituted for the
shopping list. In some embodiments, the shopping list entry may
correspond directly to an item in the environment--for example,
exactly matching an item brand, name, size, etc. In other
embodiments, the shopping list may describe one or more items
generically without including the unique identification
information--for example, an entry including "soda" instead of a
more specific intended item, such as a six-pack of 12-ounce cans of
Coca-Cola.RTM.. Alternatively, an item may be specifically included
in the shopping list entry, but is out of stock in the
environment.
[0082] At block 815, the system determines a categorical grouping
corresponding to the entry. For example, for an entry of "lettuce,"
some possible categorical groupings include .sup."green
vegetables," "leafy greens," "vegetables," etc. The most pertinent
categorical grouping may be determined using profile information
and/or the virtual transaction record--for example, the selection
of other vegetables (e.g., carrots, celery, cucumbers, tomatoes)
during the transaction may indicate that the person is purchasing
items to assemble a salad. As a number of different vegetables have
been selected in this example, an appropriate grouping for the
lettuce entry may be narrower than merely "vegetables" or "green
vegetables" categories. Assume for the purposes of this example
that the system determines "leafy greens" to be the best
categorical grouping. The example leafy greens category may
encompass a plurality of different items--different varieties of
lettuce, spinach, kale, collards, Swiss chard, cabbage, Brussels
sprouts, etc. Each of the plurality of items may have different
qualities or characteristics, such as nutrition information, which
may be comparatively ranked relative to one or more criteria.
[0083] At block 825, a task involving the grouping is presented to
the person. The task may be presented relative to the one or more
criteria, which may be explicitly included or deduced from the
personal profile 322. The criteria may be based on health
information 332, such as nutrition goals 334 and/or fitness goals
335. The task may further be based on inventory and/or incentive
information, such as a need to sell certain items.
[0084] Continuing the earlier example, say one of the person's
nutrition goals 334 is increasing daily potassium intake. Each of
the plurality of items in the "leafy greens" category may have a
differing amount of potassium content. Thus, the items may be given
a relative ranking according to their potassium content--e.g.,
spinach as having the most potassium, Swiss chard second-most, kale
third-most, and so forth. Of course, a combination of multiple
criteria may be used to rank the categorical items; the criteria
may be weighted as part of determining scoring for each of the
items.
[0085] Presenting the task to the person may include directing the
person to find the item within the categorical grouping that best
matches one or more criteria. In the example, the task may prompt
the person to locate the leafy green vegetable having the highest
potassium content.
[0086] At block 835, subroutine 565 is performed. As discussed
above, the subroutine 565 generally includes analyzing image
information, determining a match with the presented task,
evaluating performance of the task, and updating rewards based on
the evaluation. In this case, evaluating the person's performance
is based on which item the person selects from the categorical
grouping. For example, the person might receive a largest score for
selecting the spinach, and lesser scores for selecting one of the
other types of leafy greens. In some embodiments, the system may
present the "best" choice to the person after the person's item
selection has been made, in order to improve their understanding
and item selection abilities.
[0087] At block 840, the system determines whether additional
entries remain on the shopping list. If so ("YES"), the method
returns to block 805. Accordingly, the profile matching game may be
implemented as an incremental game, adapting the task and/or
scoring based on the person's selection of the previous item. Based
on the item selected from a first grouping, the system assess how
the selected item affects nutrition goals, fitness goals, etc.
included in the personal profile, and may adapt the next task
accordingly. For example, if the person selects the leafy greens
item corresponding to the highest potassium content, evaluation of
subsequent shopping list entries may lessen the relative importance
of the items' potassium content (e.g., adjusting relative
weighting) in favor of other criteria. The system may further
suggest one or more items (not included on the shopping list) based
on the person's selections, whether to complement the selected
items or to compensate for deficiencies of the selected items.
[0088] The descriptions of the various embodiments of the present
disclosure have been presented for purposes of illustration, but
are not intended to be exhaustive or limited to the embodiments
disclosed. Many modifications and variations will be apparent to
those of ordinary skill in the art without departing from the scope
and spirit of the described embodiments. The terminology used
herein was chosen to best explain the principles of the
embodiments, the practical application or technical improvement
over technologies found in the marketplace, or to enable others of
ordinary skill in the art to understand the embodiments disclosed
herein.
[0089] In the preceding, reference is made to embodiments presented
in this disclosure. However, the scope of the present disclosure is
not limited to specific described embodiments. Instead, any
combination of the following features and elements, whether related
to different embodiments or not, is contemplated to implement and
practice contemplated embodiments. Furthermore, although
embodiments disclosed herein may achieve advantages over other
possible solutions or over the prior art, whether or not a
particular advantage is achieved by a given embodiment is not
limiting of the scope of the present disclosure. Thus, the
following aspects, features, embodiments and advantages are merely
illustrative and are not considered elements or limitations of the
appended claims except where explicitly recited in a claim(s).
Likewise, reference to "the invention" shall not be construed as a
generalization of any inventive subject matter disclosed herein and
shall not be considered to be an element or limitation of the
appended claims except where explicitly recited in a claim(s).
[0090] Aspects of the present disclosure may take the form of an
entirely hardware embodiment, an entirely software embodiment
(including firmware, resident software, micro-code, etc.) or an
embodiment combining software and hardware aspects that may all
generally be referred to herein as a "circuit," "module," or
"system."
[0091] The present disclosure may be a system, a method, and/or a
computer program product. The computer program product may include
a computer readable storage medium (or media) having computer
readable program instructions thereon for causing a processor to
carry out aspects of the present disclosure.
[0092] The computer readable storage medium can be a tangible
device that can retain and store instructions for use by an
instruction execution device. The computer readable storage medium
may be, for example, but is not limited to, an electronic storage
device, a magnetic storage device, an optical storage device, an
electromagnetic storage device, a semiconductor storage device, or
any suitable combination of the foregoing. A non-exhaustive list of
more specific examples of the computer readable storage medium
includes the following: a portable computer diskette, a hard disk,
a random access memory (RAM), a read-only memory (ROM), an erasable
programmable read-only memory (EPROM or Flash memory), a static
random access memory (SRAM), a portable compact disc read-only
memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a
floppy disk, a mechanically encoded device such as punch-cards or
raised structures in a groove having instructions recorded thereon,
and any suitable combination of the foregoing. A computer readable
storage medium, as used herein, is not to be construed as being
transitory signals per se, such as radio waves or other freely
propagating electromagnetic waves, electromagnetic waves
propagating through a waveguide or other transmission media (e.g.,
light pulses passing through a fiber-optic cable), or electrical
signals transmitted through a wire.
[0093] Computer readable program instructions described herein can
be downloaded to respective computing/processing devices from a
computer readable storage medium or to an external computer or
external storage device via a network, for example, the Internet, a
local area network, a wide area network and/or a wireless network.
The network may comprise copper transmission cables, optical
transmission fibers, wireless transmission, routers, firewalls,
switches, gateway computers and/or edge servers. A network adapter
card or network interface in each computing/processing device
receives computer readable program instructions from the network
and forwards the computer readable program instructions for storage
in a computer readable storage medium within the respective
computing/processing device.
[0094] Computer readable program instructions for carrying out
operations of the present disclosure may be assembler instructions,
instruction-set-architecture (ISA) instructions, machine
instructions, machine dependent instructions, microcode, firmware
instructions, state-setting data, or either source code or object
code written in any combination of one or more programming
languages, including an object oriented programming language such
as Smalltalk, C++ or the like, and conventional procedural
programming languages, such as the "C" programming language or
similar programming languages. The computer readable program
instructions may execute entirely on the user's computer, partly on
the user's computer, as a stand-alone software package, partly on
the user's computer and partly on a remote computer or entirely on
the remote computer or server. In the latter scenario, the remote
computer may be connected to the user's computer through any type
of network, including a local area network (LAN) or a wide area
network (WAN), or the connection may be made to an external
computer (for example, through the Internet using an Internet
Service Provider). In some embodiments, electronic circuitry
including, for example, programmable logic circuitry,
field-programmable gate arrays (FPGA), or programmable logic arrays
(PLA) may execute the computer readable program instructions by
utilizing state information of the computer readable program
instructions to personalize the electronic circuitry, in order to
perform aspects of the present disclosure.
[0095] Aspects of the present disclosure are described herein with
reference to flowchart illustrations and/or block diagrams of
methods, apparatus (systems), and computer program products
according to embodiments of the disclosure. It will be understood
that each block of the flowchart illustrations and/or block
diagrams, and combinations of blocks in the flowchart illustrations
and/or block diagrams, can be implemented by computer readable
program instructions.
[0096] These computer readable program instructions may be provided
to a processor of a general purpose computer, special purpose
computer, or other programmable data processing apparatus to
produce a machine, such that the instructions, which execute via
the processor of the computer or other programmable data processing
apparatus, create means for implementing the functions/acts
specified in the flowchart and/or block diagram block or blocks.
These computer readable program instructions may also be stored in
a computer readable storage medium that can direct a computer, a
programmable data processing apparatus, and/or other devices to
function in a particular manner, such that the computer readable
storage medium having instructions stored therein comprises an
article of manufacture including instructions which implement
aspects of the function/act specified in the flowchart and/or block
diagram block or blocks.
[0097] The computer readable program instructions may also be
loaded onto a computer, other programmable data processing
apparatus, or other device to cause a series of operational steps
to be performed on the computer, other programmable apparatus or
other device to produce a computer implemented process, such that
the instructions which execute on the computer, other programmable
apparatus, or other device implement the functions/acts specified
in the flowchart and/or block diagram block or blocks.
[0098] The flowchart and block diagrams in the Figures illustrate
the architecture, functionality, and operation of possible
implementations of systems, methods, and computer program products
according to various embodiments of the present disclosure. In this
regard, each block in the flowchart or block diagrams may represent
a module, segment, or portion of instructions, which comprises one
or more executable instructions for implementing the specified
logical function(s). In some alternative implementations, the
functions noted in the block may occur out of the order noted in
the figures. For example, two blocks shown in succession may, in
fact, be executed substantially concurrently, or the blocks may
sometimes be executed in the reverse order, depending upon the
functionality involved. It will also be noted that each block of
the block diagrams and/or flowchart illustration, and combinations
of blocks in the block diagrams and/or flowchart illustration, can
be implemented by special purpose hardware-based systems that
perform the specified functions or acts or carry out combinations
of special purpose hardware and computer instructions.
[0099] Embodiments of the disclosure may be provided to end users
through a cloud computing infrastructure. Cloud computing generally
refers to the provision of scalable computing resources as a
service over a network. More formally, cloud computing may be
defined as a computing capability that provides an abstraction
between the computing resource and its underlying technical
architecture (e.g., servers, storage, networks), enabling
convenient, on-demand network access to a shared pool of
configurable computing resources that can be rapidly provisioned
and released with minimal management effort or service provider
interaction. Thus, cloud computing allows a user to access virtual
computing resources (e.g., storage, data, applications, and even
complete virtualized computing systems) in "the cloud," without
regard for the underlying physical systems (or locations of those
systems) used to provide the computing resources.
[0100] Typically, cloud computing resources are provided to a user
on a pay-per-use basis, where users are charged only for the
computing resources actually used (e.g., an amount of storage space
consumed by a user or a number of virtualized systems instantiated
by the user). A user can access any of the resources that reside in
the cloud at any time, and from anywhere across the Internet. In
context of the present disclosure, a user may access applications
(e.g., a retail store app for a mobile computing device) or related
data (e.g., compiled shopping data) available in the cloud. Doing
so allows a user to access this information from any computing
system attached to a network connected to the cloud (e.g., the
Internet).
[0101] While the foregoing is directed to embodiments of the
present disclosure, other and further embodiments of the disclosure
may be devised without departing from the basic scope thereof, and
the scope thereof is determined by the claims that follow.
* * * * *