U.S. patent application number 14/590240 was filed with the patent office on 2016-04-21 for method, computer program product, and system for providing a sensor-based environment.
The applicant listed for this patent is TOSHIBA GLOBAL COMMERCE SOLUTIONS HOLDINGS CORPORATION. Invention is credited to Monsak Jason Chirakansakcharoen, Dean Frederick Herring, Ankit SINGH.
Application Number | 20160110791 14/590240 |
Document ID | / |
Family ID | 55748790 |
Filed Date | 2016-04-21 |
United States Patent
Application |
20160110791 |
Kind Code |
A1 |
Herring; Dean Frederick ; et
al. |
April 21, 2016 |
METHOD, COMPUTER PROGRAM PRODUCT, AND SYSTEM FOR PROVIDING A
SENSOR-BASED ENVIRONMENT
Abstract
Method, computer program product, and system to influence a
person within an environment having a plurality of items for
selection. The method includes capturing, using a first visual
sensor disposed within the environment, field of view information
for the person, performing analysis on the field of view
information using a computing device, and identifying, based on the
analysis, one or more first items of the plurality of items that
are included within the field of view of the person. The method
further includes selecting, based on the identified one or more
items, at least one second item of the plurality of items for
presentation to the person; and presenting information related to
the at least one second item to the person.
Inventors: |
Herring; Dean Frederick;
(Youngsville, NC) ; Chirakansakcharoen; Monsak Jason;
(Raleigh, NC) ; SINGH; Ankit; (Morrisville,
NC) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
TOSHIBA GLOBAL COMMERCE SOLUTIONS HOLDINGS CORPORATION |
Tokyo |
|
JP |
|
|
Family ID: |
55748790 |
Appl. No.: |
14/590240 |
Filed: |
January 6, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62064323 |
Oct 15, 2014 |
|
|
|
Current U.S.
Class: |
705/26.61 |
Current CPC
Class: |
G06Q 30/0609 20130101;
G06T 7/66 20170101; H04N 7/181 20130101; G06K 9/209 20130101; G06K
9/00221 20130101; G06Q 30/016 20130101; G01G 19/4144 20130101; G06K
9/00536 20130101; G06Q 20/209 20130101; G07G 1/0054 20130101; G06T
11/60 20130101; G06Q 10/087 20130101; G01B 11/02 20130101; G06K
2209/17 20130101; G06Q 30/0224 20130101; H04N 5/225 20130101; H04N
5/232 20130101; G06T 2207/30232 20130101; G06Q 20/204 20130101;
G06F 1/163 20130101; G06F 3/012 20130101; G06K 9/6201 20130101;
G06Q 20/322 20130101; G06Q 30/0623 20130101; G07G 1/0045 20130101;
G06K 9/6267 20130101; G06Q 20/18 20130101; G06Q 30/0269 20130101;
G06K 9/00362 20130101; G06Q 30/0235 20130101; G01B 11/00 20130101;
G01S 19/13 20130101; G06Q 30/0251 20130101; G07G 1/0072 20130101;
G01G 19/40 20130101; G06K 9/00288 20130101; G06Q 30/0629 20130101;
G06K 9/00577 20130101; G06K 9/78 20130101; G07G 1/0063 20130101;
G06K 9/46 20130101; G06Q 20/208 20130101; G06K 9/00624 20130101;
G06Q 30/0639 20130101; G06K 2009/00328 20130101; G06F 3/013
20130101; G06K 9/00255 20130101; G06Q 30/0633 20130101; G06Q
30/0635 20130101; G06Q 30/0631 20130101 |
International
Class: |
G06Q 30/06 20060101
G06Q030/06; G06F 3/01 20060101 G06F003/01; G06T 7/20 20060101
G06T007/20; G06K 9/00 20060101 G06K009/00; G06F 1/16 20060101
G06F001/16; G06T 7/00 20060101 G06T007/00 |
Claims
1. A computer-implemented method to influence a person within an
environment having a plurality of items for selection, the method
comprising: capturing, using a first visual sensor disposed within
the environment, field of view information for the person;
performing analysis on the field of view information using a
computing device; identifying, based on the analysis, one or more
first items of the plurality of items that are included within the
field of view of the person; selecting, based on the identified one
or more first items, at least one second item of the plurality of
items for presentation to the person; and presenting information
related to the at least one second item to the person.
2. The computer-implemented method of claim 1, wherein capturing
field of view information for the person includes using the first
visual sensor to determine at least one of an eye orientation, a
head orientation, and a body orientation of the person.
3. The computer-implemented method of claim 1, wherein performing
analysis on the field of view information includes comparing the
field of view information with location data for the plurality of
items of the environment.
4. The computer-implemented method of claim 1, wherein performing
analysis on the field of view information includes performing image
processing upon one or more images of portions of the environment
captured using at least a second visual sensor disposed within the
environment.
5. The computer-implemented method of claim 1, further comprising:
determining a focus of the person on at least one item selected
from the one or more first items included within the person's field
of view, wherein determining a focus is based on inclusion of the
at least one selected item within the person's field of view for at
least a predetermined first length of time.
6. The computer-implemented method of claim 5, further comprising:
calculating an interest score of the person for the at least one
selected item based on a length of time of inclusion within the
person's field of view.
7. The computer-implemented method of claim 6, wherein presenting
information related to the at least one second item includes
determining content of the information based on the interest
score.
8. The computer-implemented method of claim 1, wherein selecting
the at least one second item is based on data compiled from one or
more previous experiences of one or more persons.
9. The computer-implemented method of claim 1, wherein selecting
the at least one second item is based on personal preferences
included in a personal profile associated with the person.
10. The computer-implemented method of claim 1, wherein the at
least one second item is not included within the person's field of
view, and wherein presenting information related to the at least
one second item includes indicating a direction to a location of
the at least one second item within the environment.
11. A computer program product to influence a person within an
environment having a plurality of items for selection, the computer
program product comprising: a computer-readable storage medium
having computer-readable program code embodied therewith, the
computer-readable program code executable by one or more computer
processors to: capture, using a first visual sensor disposed within
the environment, field of view information for the person; perform
analysis on the field of view information using a computing device;
identify one or more first items of the plurality of items that are
included within the field of view of the person; select, based on
the identified one or more first items, at least one second item of
the plurality of items for presentation to the person; and present
information related to the at least one second item to the
person.
12. The computer program product of claim 11, wherein performing
analysis on the field of view information includes performing image
processing upon one or more images of portions of the environment
captured using at least a second visual sensor disposed within the
environment.
13. The computer program product of claim 11, wherein the
computer-readable program code is further executable to: determine
a focus of the person on at least one item selected from the one or
more first items included within the person's field of view,
wherein determining a focus is based on inclusion of the at least
one selected item within the person's field of view for at least a
predetermined first length of time.
14. The computer program product of claim 13, wherein the
computer-readable program code is further executable to: calculate
an interest score of the person for the at least one selected item
based on a length of time of inclusion within the person's field of
view.
15. The computer program product of claim 14, wherein presenting
information related to the at least one second item includes
determining content of the information based on the interest
score.
16. A system to influence a person within an environment having a
plurality of items for selection, the system comprising: one or
more computer processors; a first visual sensor disposed within the
environment and communicatively coupled with the one or more
computer processors; and a memory containing a program which, when
executed by the one or more computer processors, performs an
operation comprising: capturing, using a first visual sensor
disposed within the environment, field of view information for the
person; performing analysis on the field of view information using
a computing device; identifying, based on the analysis, one or more
first items of the plurality of items that are included within the
field of view of the person; selecting, based on the identified one
or more first items, at least one second item of the plurality of
items for presentation to the person; and presenting information
related to the at least one second item to the person.
17. The system of claim 16, wherein the first visual sensor is
included in a body-worn computing device of the person.
18. The system of claim 16, further comprising a plurality of
visual sensors distributed throughout the environment, and wherein
performing analysis on the field of view information includes
performing image processing upon one or more images of portions of
the environment captured using the plurality of visual sensors.
19. The system of claim 16, further comprising an output device
communicatively coupled with the one or more computer processors,
wherein the output device is configured to present the information
related to the at least one second item to the person using at
least one of audio and visual output.
20. The system of claim 19, wherein the output device is one of a
handheld computing device and a body-worn computing device of the
person.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims benefit of U.S. provisional patent
application Ser. No. 62/064,323, filed Oct. 15, 2014, entitled
"Integrated Shopping Environment," which is herein incorporated by
reference.
BACKGROUND
[0002] The present disclosure relates to a sensor-based
environment, and more specifically, to providing an adaptive
personal experience within such an environment using a determined
field of view of the person.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0003] FIG. 1 illustrates an integrated shopping environment,
according to one embodiment.
[0004] FIG. 2 illustrates a shopping environment system, according
to one embodiment.
[0005] FIG. 3 illustrates an integrated shopping environment,
according to one embodiment.
[0006] FIG. 4 illustrates a system of influencing shopping
experience based on a customer field of view, according to one
embodiment.
[0007] FIGS. 5A and 5B illustrate an example wearable computing
device for use in a shopping environment, according to one
embodiment.
[0008] FIGS. 6A-6C illustrate determining a customer field of view
and identifying items included within the field of view, according
to one embodiment.
[0009] FIG. 7 illustrates several example views of determining a
customer focus on an item within the customer's field of view,
according to one embodiment.
[0010] FIGS. 8A-8C illustrate several example presentations of
information related to an item, according to one embodiment.
[0011] FIG. 9A illustrates a method of influencing a shopping
experience of a customer within a shopping environment, according
to one embodiment.
[0012] FIG. 9B illustrates a method of determining content to
present to a customer based on a customer interest score, according
to one embodiment.
[0013] To facilitate understanding, identical reference numerals
have been used, where possible, to designate identical elements
that are common to the figures. It is contemplated that elements
disclosed in one embodiment may be beneficially utilized on other
embodiments without specific recitation. The illustrations referred
to here should not be understood as being drawn to scale unless
specifically noted. Also, the drawings are often simplified and
details or components omitted for clarity of presentation and
explanation. The drawings and discussion serve to explain
principles discussed below, where like designations denote like
elements.
DETAILED DESCRIPTION
[0014] Aspects of the current disclosure relate to an integrated
environment capable of providing a personalized, automated, and
adaptive experience for a person within the environment. A number
of different sensor devices may be employed within the environment,
and networked with various computing devices such as point-of-sale
(POS) terminals, digital signage, servers, and mobile or handheld
computing devices to provide a seamless integration of mobile
technologies and e-commerce into a traditional experience.
[0015] Using one or more visual sensors within the environment, a
retailer or other provider may determine a person's field of view,
and may compile personal behaviors and determine personal
preferences. This data may then be used to provide timely, tailored
recommendations in real-time to the person in order to more
effectively influence their experience. While generally discussed
within the context of a shopping environment, such as a retail
store or other commercial environment, it is contemplated that the
techniques disclosed herein may be applied to other environments
(some non-limiting examples include libraries, museums, classrooms,
hospitals, etc.) to provide an adaptive experience for persons
included therein.
[0016] FIG. 1 illustrates an integrated shopping environment,
according to one embodiment. As shown, environment 100 includes a
plurality of terminals 105, a plurality of servers 110.sub.1,
110.sub.2 coupled to a network 115, one or more sensors 120 of
different types, one or more user devices 140, and one or more
other devices 150. In some embodiments, the environment 100 may be
integrated into the layout of a retail store, market, or other
commercial environment that is known or hereinafter developed.
[0017] Terminals 105 generally include any structure that is
capable of receiving input from customers and/or producing output
to customers within the environment 100. The terminals 105 may
include computing systems, portions of computing systems, or
devices controllable by computing systems. In one example, a
terminal may include a computing device that is communicatively
coupled with a visual display and audio speaker(s), as well as
being communicatively coupled with one or more input devices. In
another example, a terminal may include a visual display and
associated driver hardware, but a computing device coupled to the
terminal and providing data for display is disposed separately from
the terminal. In some embodiments, terminals 105 may be implemented
as standalone devices, such as a kiosk disposed on the store floor
or monolithic device disposed on a shelf or platform. In some
embodiments, terminals 105 may be integrated partially or wholly
with other components of the environment 100, such as input or
output devices included with shelving or other structural
components in the environment (e.g., components used for product
display or storage). In some embodiments, terminals 105 may be
modular and may be easily attachable and detachable to elements of
the environment 100, such as the structural components.
[0018] Generally, terminals 105 may be distributed throughout the
environment 100 and may enhance various phases of the shopping
experience for customers. For example, terminals 105 may include
digital signage 108 disposed throughout the environment, such as
included in or near aisles, endcaps, displays, and/or shelving in
the environment. A customer may view and/or interact with the
digital signage 108 as he/she moves through the store environment.
The digital signage may be included in a static display or may be
movable, such as including digital signage within a customer's
shopping cart or basket. Terminals 105 may also include POS
terminals 106 that provide a checkout functionality, allowing the
customer to complete his/her shopping transaction (e.g., make
payment for selected items). In some embodiments, terminals 105 may
provide an integrated functionality. For example, the terminals may
function in one mode as digital signage, and when engaged by a
customer, the terminals function as a POS terminal.
[0019] Servers 110.sub.1, 110.sub.2 generally include processors,
memory, and communications capabilities, and may perform various
computing tasks to support the commercial operation of the
environment 100. Servers 110.sub.1, 110.sub.2 communicate using
various wired and/or wireless communications methods with terminals
105, sensors 120, and with other networked devices such as user
devices 140 and other devices 150. Servers 110.sub.1, 110.sub.2
generally execute computer program code in which input data is
received from networked devices, the input data is processed and/or
stored by the servers, and output data is provided to networked
devices for operation of the environment 100.
[0020] Sensors 120 may include various sensor devices, such as
video sensors 125, audio sensors 130, and other sensors 135. The
other sensors 135 generally include any sensors that are capable of
providing meaningful information about customer interactions with
the environment, e.g., location sensors, weight sensors, and so
forth. Sensors 120 may be deployed throughout the environment 100
in fixed and/or in movable locations. For example, sensors 120 may
be statically included in walls, floors, ceilings, displays, or
other devices, or may be included in shopping carts or baskets
capable of being transported around the environment. In one
embodiment, sensors 120 may include adjustable position sensor
devices, such as motorized cameras attached to a rail, wire, or
frame. In one embodiment, sensors 120 may be included on one or
more unmanned vehicles, such as unmanned ground vehicles (UGVs) or
unmanned aerial vehicles (UAVs or "drones"). Sensors 120 may also
include sensor devices that are included in user devices 140 or
other devices 150 (which in some cases may include body-worn or
carried devices). User devices 140 and other devices 150 may
include passive or actively-powered devices capable of
communicating with at least one of the networked devices of
environment 100. One example of a passive device (which may be worn
or carried) is a near-field communication (NFC) tag. Active devices
may include mobile computing devices, such as smartphones or
tablets, or wearable devices such as a Google Glass.TM. interactive
eyepiece (Glass is a trademark of Google Inc.). The user devices
140 generally denotes ownership or possession of the devices by
customers, while the other devices 150 denotes ownership or
possession by the retailer or other administrator of the
environment 100. In some cases, other devices 150 may be carried by
employees and used in the course of their employment. User devices
140 and other devices 150 may execute applications or other program
code that generally enables various features provided by the
servers and/or other networked computing devices.
[0021] FIG. 2 illustrates a shopping environment system, according
to one embodiment. Generally, the system 200 corresponds to the
environment 100 described above. System 200 includes one or more
processors 205, memory 210, and input/output 250, which are
interconnected using one or more connections 240. In one
embodiment, system 200 may be included in a singular computing
device, and the connection 240 may be a common bus. In other
embodiments, system 200 is distributed and includes a plurality of
discrete computing devices that are connected through wired or
wireless networking. Processors 205 may include any processing
element suitable for performing functions described herein, and may
include single or multiple core processors, as well as combinations
thereof. Processors 205 may be included in a single computing
device, or may represent an aggregation of processing elements
included across a number of networked devices such as user devices
140, POS terminals 105, etc.
[0022] Memory 210 may include a variety of computer-readable media
selected for their size, relative performance, or other
capabilities: volatile and/or non-volatile media, removable and/or
non-removable media, etc. Memory 210 may include cache, random
access memory (RAM), storage, etc. Storage included as part of
memory 210 may typically provide a non-volatile memory for the
networked computing devices (e.g., servers 110.sub.1, 110.sub.2),
and may include one or more different storage elements such as
Flash memory, a hard disk drive, a solid state drive, an optical
storage device, and/or a magnetic storage device. Memory 210 may be
included in a single computing device or may represent an
aggregation of memory included in networked devices. Memory 210 may
include a plurality of modules 211 for performing various functions
described herein. The modules 211 include program code that is
executable by one or more of the processors 205. As shown, modules
211 include user identification 212, item identification 214,
advertising 216, recommendations 218, virtual cart 220, assistance
222, security 224, power management 226, gaming 228, audit 230,
loyalty program 232, and inventory 234. The modules 211 may also
interact to perform certain functions. For example, a loyalty
program module 232 during operation may make calls to user
identification module 212, item identification module 214,
advertising module 216, and so forth. The person of ordinary skill
will recognize that the modules provided here are merely
non-exclusive examples; different functions and/or groupings of
functions may be included as desired to suitably operate the
shopping environment. Memory 210 may also include customer profiles
236 and customer images 238, which may be accessed and/or modified
by various of the modules 211. In one embodiment, the customer
profiles 236 and customer images 238 may be stored on the servers
110.sub.1, 110.sub.2 or on a separate database.
[0023] Input/output (I/O) 250 may include a number of different
devices interfacing with various computing devices and with the
shopping environment. I/O 250 includes sensors 120, which have been
described above. I/O 250 may further include input devices 252 and
output devices 254 that are included to enhance the shopping
experience for customers. In one embodiment, terminals 105, user
devices 140, and other devices 150 may include visual displays
and/or audio speakers (examples of the output devices 254), and
various input devices 252 (such as cameras, keyboards or keypads,
touchscreens, buttons, inertial sensors, etc.). I/O 250 may further
include wired or wireless connections to an external network 256
using I/O adapter circuitry. Network 256 may include one or more
networks of various types, including a local area or local access
network (LAN), a general wide area network (WAN), and/or a public
network (e.g., the Internet). In one embodiment, various networked
computing devices of the system 200 are interconnected using a LAN,
and one or more computing devices (e.g., servers 110.sub.1,
110.sub.2, user devices 140) include connections to the
Internet.
[0024] FIG. 3 illustrates an integrated shopping environment,
according to one embodiment. The environment 300 includes a
plurality of sensor modules 302 disposed in the ceiling 301 of the
store. The sensor modules 302 may each include one or more types of
sensors, such as video sensors (e.g., cameras), audio sensors
(e.g., microphones), and so forth. Sensor modules 302 may also
include actuating devices for providing a desired sensor
orientation. Sensor modules or individual sensors may generally be
disposed at any suitable location within the environment 300. Some
non-limiting examples of alternative locations include below,
within, or above the floor 330, within other structural components
of the environment 300 such as a shelving unit 303 or walls, and so
forth. In some embodiments, sensors may be disposed on, within, or
near product display areas such as shelving unit 303. The sensors
may also be oriented toward an expected location of a customer
interaction with items, to provide better data about a customer's
interaction, such as determining a customer's field of view.
[0025] Environment 300 also includes a number of kiosks (or
terminals) 305. Generally, kiosks 305 may be configured for
performing customer checkout and/or other shopping functions. Each
kiosk 305 may each include computing devices or portions of
computing systems, and may include various I/O devices, such as
visual displays, audio speakers, cameras, microphones, etc. for
interacting with the customer. In some embodiments, a customer 340
may have a mobile computing device, such as smartphone 345, that
communicatively couples with the kiosk 305 for completing the
checkout transaction. In one embodiment, the mobile computing
device may execute a store application that is connected to the
networked computing systems (e.g., through servers 110.sub.1,
110.sub.2), or may be directly connected to kiosk 305 through
wireless networks within the environment (e.g., over Wi-Fi or
Bluetooth). In one embodiment, the mobile computing device may
couple to the kiosk 305 when brought within range, e.g., using
Bluetooth or NFC.
[0026] Environment 300 also includes one or more shelving units 303
having shelves 310 that support various store items 315. Though not
shown, multiple shelving units 303 may be disposed in a particular
arrangement in the environment 300, and the space between adjacent
shelving units may form aisles through which customers may travel.
In some embodiments, the shelving unit 303 may include visual
sensors or other sensor devices or I/O devices. The sensors or
devices may couple to a customer's smartphone 345 and/or other
networked computing devices (including servers) within the
environment 300. For example, the front portions 320 of shelves 310
may include video sensors oriented outward from the shelving unit
303 to capture customer interactions with items 315 on the shelving
unit 305, and the data from the video sensors may be provided to
back-end servers for storage and/or analysis. In some embodiments,
portions of the shelving unit 303 (such as the front portions 320
of shelves 310) may include indicator lights or other visual
display devices or audio output devices that are used to
communicate with a customer.
[0027] FIG. 4 illustrates a system of influencing shopping
experience based on a customer field of view, according to one
embodiment. System 400 may be used in coordination with the various
shopping environments described herein. Generally, system 400 may
share at least portions of several components with the shopping
environment system 200, such as processors 205, memory 210, and I/O
250. System 400 may also utilize one or more of the modules 211 to
provide various aspects of the system's functionality, such as item
identification module 214, advertising module 216, recommendation
module 218, and so on.
[0028] I/O 250 includes output devices 254 and sensors 120. Output
devices 254 include one or more devices for presenting information
to customers and include audio output devices 440 and visual output
devices 445. The audio output devices 440 may include conventional
audio speakers having any suitable form factor (e.g., in a stereo,
headphones, etc.), as well as devices using alternative methods of
producing sound to a customer, such as bone conduction transducers
in a worn device. Visual output devices 445 may include visual
displays and various visual indicators such as light emitting
diodes (LEDs). Other output devices 450 may provide information to
customers through tactile feedback (e.g., haptic devices) or other
sensory stimuli. Sensors 120 may include visual sensors 455 which
may be carried or worn sensors 460, and distributed sensors 465
that are disposed throughout the shopping environment. Other
sensors 470 may be included that are suitable for collecting
information about a customer and his/her interactions within the
shopping environment. Examples of other sensors 470 may include
infrared (IR) sensors, thermal sensors, weight sensors, capacitive
sensors, magnetic sensors, sonar sensors, radar sensors, lidar
sensors, and so forth.
[0029] The visual sensors 455 may be used to capture one or more
images 415 of the customer and/or the shopping environment, which
may include views from various perspectives (e.g., a customer-worn
visual sensor, static or movable visual sensors at various
locations in the environment). The images 415 may be stored in
memory 210, and may be individually or collectively processed to
determine information about customers in the environment and their
respective interactions with items in the environment.
[0030] Memory 210 includes one or more programs 435 that
collectively receive data about customers and the shopping
environment, process the received data, and transmit information to
customers in order to influence the customers' shopping
experiences. Programs 435 may include program code to determine a
customer's field of view at a given time, including which items are
included in the field of view. In one embodiment, the customer's
field of view may be determined directly. For example, a body-worn
device may include a visual sensor (i.e., a worn visual sensor 460)
that, when the device is worn, gives the visual sensor an
orientation that is similar to the orientation of customer's head
or eyes (e.g., a forward-looking camera). Images captured from the
worn visual sensor may generally reflect the customer's field of
view.
[0031] In some embodiments, the customer's field of view may be
estimated (or determined indirectly) using other sensor
measurements. In one embodiment, the customer's field of view may
be estimated by determining the orientation of one or both of the
customer's eyes. Eye orientation may be determined using worn
visual sensors 460 (e.g., an inward-facing camera on a head-worn
device) and/or distributed visual sensors 465 (e.g., capturing
images of the customer's face and image processing to determine an
eye orientation). In other embodiments, the customer's field of
view may be estimated by determining the position and/or
orientation of the customer's head and/or body using various visual
sensor measurements. The customer's field of view may be
represented in any suitable data format, such as an image or as
coordinate data (e.g., Cartesian, polar, spherical
coordinates).
[0032] While it is possible that a single visual sensor 455 may be
used to determine a customer's field of view, several embodiments
employ a combination of a plurality of visual sensors 455 to
determine a field of view. These embodiments may be preferred as
providing additional data to support a more accurate estimate of
the field of view. Additionally, the plurality of visual sensors
used to determine a field of view may include visual sensors
selected from different categories (worn sensors 460, distributed
sensors 465) to provide additional robustness to the collected
data.
[0033] Programs 435 may also include program code to identify one
or more items included within the customer's field of view. The
identification process may be performed directly or estimated. One
example of direct identification is performing image processing on
images collected from a worn, forward-looking camera to visually
identify one or more items. Estimating items within a customer's
field of view may require combining sensor data with known
information about the shopping environment, such as the items in
the environment (item data 420) and their relative arrangement or
layout within the environment (location data 425).
[0034] Programs 435 may also include program code to present
information to customers based on the identified one or more items.
In some embodiments, the information may be used to influence the
customer's shopping experience. The information presented to the
customer may include information about the identified items (e.g.,
nutritional data, pricing, a customer's purchase history of the
items, etc.), information encouraging the purchase of identified
items (e.g., bringing particular items to the customer's attention,
touting the items' features, offering discounts or other promotions
on the items, etc.), and information encouraging the purchase of
alternatives to the identified items (e.g., highlighting
differences of the items, offering discounts, etc.).
[0035] To present relevant and persuasive information in real-time
to a particular customer, programs 435 may access and analyze
various additional data in memory 210 that is related to the
customer and perhaps other customers of the shopping environment.
In one embodiment, programs 435 may analyze shopping data 430
collected from previous shopping experiences for the particular
customer and/or for other customers or groups of customers. For
example, shopping data 430 may include customer views and the items
included therein, progressions of customer views (showing a
customer's interest over time), selection or purchase history for
items, and so forth. While shopping data 430 may be compiled and
used to generate information to present to customers and influence
their shopping experiences in real-time, the shopping data 430 may
also be used by the retailer or administrator of the shopping
environment to modify the layout of the environment. The shopping
data 430 may help the retailer identify trends in customer
shopping, and to optimize placement of items within the environment
to improve customer sales.
[0036] System 400 may also present information to customers based
on their personal preferences 405. The preferences 405 may
generally be stored in a corresponding customer profile 236, and
may reflect preferences that are explicitly specified by a
customer, or may be determined based on the customer's historical
shopping behavior (e.g., included in shopping data 430). For
example, a customer may have an allergy to a particular ingredient,
and the customer may enter this allergy information in preferences
405, e.g., using a mobile phone app for the retailer. Accordingly,
the system 400 when determining which information to present to the
customer may present information that highlights items within the
customer's field of view that include the ingredient, and may
further suggest alternative items that do not include the
ingredient.
[0037] A customer's shopping history may also be used to determine
customer preferences 405. For example, the customer's determined
fields of view and purchase history from shopping data 430 may be
processed to deduce which items, combinations of items, and/or
aspects of items are preferred by the customer. For example,
preferred aspects might include preferred brands, costs,
quantities, sizes, ingredients, nutritional properties (e.g.,
calorie content, fat, sugar, vitamins, minerals, etc.), and so
forth. For example, the customer may specify a preference for
low-fat foods, and the system may determine recommended items based
on the items included in the customer's field of view and the
customer's preferences. This may include suggesting a particular
item within the field of view for purchase (or alternatively, an
item located outside the field of view) and/or advising the
customer about the item's properties vis-a-vis the customer's
preferences (e.g., reporting fat content).
[0038] Of course, a customer's preferences may be included as a
logical combination of a plurality of these aspects (e.g., a
customer prefers Brand X items to Brand Y, so long as the cost of
the Brand X item is no more than 150% of the Brand Y item). In some
embodiments, other customers' shopping data may also be used to
deduce a particular customer's preferences. Of course, the
preferences may be dynamically updated to identify whether deduced
preferences are accurate or not. The dynamic updating may be caused
by the customer's explicit indication and/or by the customer's
shopping patterns following the deduced preference. For example,
the system 400 may deduce that a customer has a categorical
preference for Brand X items over similar Brand Y items. However,
the customer's historical shopping data indicated that the customer
looked at a Brand X item (e.g., field of view data) before
selecting and purchasing a similar Brand Y item (e.g., field of
view data and/or purchase history data). The system in response may
adapt or may entirely remove the deduced (and determined
inaccurate) preference.
[0039] In some embodiments, system 400 may present information to
customers that is also based on other programs 410. Examples of
programs 410 may include fitness, nutrition, or health goals, money
management goals, etc. In some embodiments, the programs 410
themselves may be integrated into a store application and
accessible by the customer's mobile computing device or wearable
device. In other embodiments, the store application may interface
with applications from other providers to determine the customer's
goals and present appropriate information during the shopping
experience. For example, the system 400 could include a
nutrition-oriented program, and may make suggestions for more
nutritious items to a customer who is looking at junk food items
(e.g., candy).
[0040] FIGS. 5A and 5B illustrate an example wearable computing
device for use in a shopping environment, according to one
embodiment. Portions of wearable computing device 500 may be
head-worn or worn on other portions of the body. The wearable
computing device 500 includes a housing 505 that includes several
structural components. The band 505 may be used as a structural
frame, supporting other portions while itself being supported by a
customer's head when worn. Other structural components may include
nose pieces 520, ear piece 515, and enclosure 512. Enclosure 512
may include a computing device 525, which includes video I/O
components 530 and audio I/O components 540. The enclosure formed
by the earpiece 515 may also house components, such as a battery
545 for powering the computing device 525. Although not shown,
computing device 525 also includes wireless networking components
for communicating with other computing devices. The video I/O
components 530 may include a forward-looking camera 513 that
provides an estimate of the wearer's field of view based on their
head orientation, and a transparent prism 514 that is used to
project light onto the wearer's retina, displaying information to
the wearer over the wearer's natural field of view. Other video I/O
components may include an inward-looking camera that is configured
to capture the eye orientation of the wearer, a conventional
display device (e.g., a LCD), and so forth. Audio I/O components
540 may include one or more microphones and/or audio output
devices, such as speakers or bone-conducting transducers.
[0041] FIG. 5B illustrates the wear of wearable computing device
500. As shown, scene 550 includes the wearer's natural view 560. A
portion of the area of the natural view 560 may be used for
displaying an overlay 565 (e.g., using the prism 514) that provides
additional information to the wearer. The display of the overlay
565 may be adequately transparent to permit the wearer to continue
to observe their natural view through the overlay area. As shown,
the overlay 565 includes a map view of the wearer's current
location (e.g., the wearer is at W 34.sup.th street). In some
embodiments, information may be selected and visually presented in
text and/or graphics to a customer wearing such a device to
influence his/her shopping experience.
[0042] FIGS. 6A-6C illustrate determining a customer field of view
and identifying items included within the field of view, according
to one embodiment. In scene 600, a shelving unit 603 is depicted
having a plurality of shelves 610 that each support and display a
number of items 612 that are available for selection and purchase
by a customer.
[0043] Within the scene 600 are defined a customer's field of view
615 and an area 605 outside the customer field of view. In one
embodiment, the customer's field of view 615 may be represented by
an image captured from a forward-looking camera. While shown as
generally rectangular, the customer field of view 615 may have any
suitable alternative shape and size. For example, the customer's
actual vision may encompass a significantly larger area, but
determining the field of view for purposes of the shopping
environment may include applying a threshold or weighting scheme
that emphasizes areas that are closer to the center of a customer's
vision. Of course, data provided by various visual sensors and/or
other sensors may be used to make these determinations.
[0044] The field of view 615 may include a plurality of fully
included items 620, as well as a plurality of partially included
items 625. When determining which items to identify as "included"
in the field of view, certain embodiments may categorically include
or exclude the partially included items 625. An alternative
embodiment may rely on image processing to determine whether a
partially included item 625 should be identified as included. For
example, if the processing cannot recognize the particular item
with a certain degree of confidence, the item may be excluded. In
another alternative embodiment, partially included items 625 may be
included, and the amount of item inclusion (e.g., the percentage of
surface area of the item included) may be used to determine a
customer focus or calculate a customer interest score, which are
discussed further below.
[0045] Items that are fully or partially included in the customer
field of view 615 may be recognized by performing image processing
techniques on images captured by various visual sensors. For
example, images that include the items may be compared against
stock item images stored in a database or server. To aid image
processing, items may also include markers or distinctive symbols,
some of which may include item identification data such as barcodes
or quick response (QR) codes. Of course, other processing
techniques may be employed to recognize a particular item, such as
textual recognition, determining the item's similarity to adjacent
items, and so forth.
[0046] Scene 650 of FIG. 6B depicts a customer 660 in a shopping
environment. The customer 660 is standing in an aisle 655 adjacent
to a shelving unit 603, which has a plurality of shelves 610.
Visual sensors may capture one or more images of scene 650 from
various spatial perspectives, and the images may be used to
determine the customer's field of view. Specifically, various
aspects of the scene that are captured in the images may be used to
estimate the customer's field of view.
[0047] In one embodiment, the relative position and/or orientation
of portions of the customer's body may be determined. In one
embodiment, the position and orientation of the customer's eyes 680
may be determined. For example, eye position within the environment
may be determined in Cartesian coordinates (i.e., determining x, y,
and z-direction values) and eye orientation may be represented by
an angle a defined relative to a reference direction or plane (such
as horizontal or an x-y plane corresponding to a particular value
of z). In other embodiments, other portions of the customer's body
may (also) be used to determine the field of view, such as the
position and orientation of the customer's head 665, or of one or
both shoulders 670. In other embodiments, the customer's
interaction with the shelving unit 603 by extending her arm 675 may
also be captured in one or more images, and the direction of the
extended arm may be used to determine her field of view.
[0048] Of course, embodiments may use combinations of various
aspects of the scene to determine the customer's field of view. In
some embodiments, the combinations may be weighted; for example,
data showing a customer 660 reaching out her arm 675 towards a
specific item may be weighted more heavily to determine her field
of view than the orientation of her shoulders. In some embodiments,
the weights may be dynamically updated based on the customer's
shopping behaviors following an estimate of the customer's field of
view. For example, if a customer reached for (or selected) an item
that was not included in the determined field of view, the system
may adjust the relative weighting in order to accurately capture
the customer's field of view. This adjustment may include
determining correlation values between particular captured aspects
of the scene to the selected item; for example, the customer's head
may be partly turned towards the selected item, but their eye
orientation may generally be more closely tied to the selected
item. In some embodiments, the correlation values may be more
useful where one or more aspects of the scene cannot be determined
(e.g., the system may be unable to determine eye orientation for a
customer wearing sunglasses, non-optimal visual sensor positioning,
etc.).
[0049] Scene 685 of FIG. 6C illustrates an overhead view of several
customers 660 in a shopping environment. In one embodiment, the
view of scene 685 may be represented by an image captured from a
ceiling mounted camera, or from a drone.
[0050] Certain additional aspects depicted in scene 685 and
captured in images may be used to estimate a customer's field of
view. In one example, the orientation of customer 660A may be
estimated using the relative position of his/her shoulders 670. As
shown, a line connecting the two shoulders may be compared to a
reference direction or plane (e.g., parallel to the length of
shelving unit 603A) and represented by an angle .beta.. In another
example, the orientation of customer 660B may be estimated using
the orientation of his/her head 665, comparing a direction of the
customer's head to a reference direction or plane, which may be
represented by an angle .gamma.. Images may also capture a customer
660C interacting with the shelving unit 603B, and the position
and/or orientation of the customer's arm 675 may be used to
determine the customer's field of view.
[0051] FIG. 7 illustrates several example views of determining a
customer focus on an item within the customer's field of view,
according to one embodiment. While the computing systems may
determine information to present to a customer based on all of the
items identified within a determined field of view, in some
embodiments it may be advantageous to make a further identification
of one or more items that the customer is specifically focused on.
Presenting information to the customer that is based on the
customer-focused items may generally provide a more relevant and
more persuasive information to influence the customer's shopping
experience.
[0052] Items 715A-D are included on a shelf 710 of a shelving unit
703. The determined field of view 705 of a customer may include
different groups of items at different times, and the progression
of the customer's field of view over time may help determine which
item(s) within the field of view are specifically being focused on
by the customer. Generally, the customer's focus on a particular
item may indicate that the item is merely attracting the customer's
attention, or that the customer is deciding whether or not to
purchase the item. Either way, understanding the object of a
customer's focus may help retailers or suppliers to improve
packaging and placement of items or to influence the customer's
shopping experience in real-time.
[0053] View 1 illustrates a field of view 705 that includes items
715A, 715B, and a portion of 715C. In one embodiment, items 715A
and 715B may be included in a customer focus determination due to
the items' full inclusion within the field of view 705. Conversely,
item 715C may be excluded from a customer focus for being only
partially included in the field of view 705. In an alternative
embodiment, all three items may be included by virtue of being at
least partially included in the field of view 705. In some
embodiments, a customer focus on particular items may be a
time-based determination. For example, if the customer's field of
view 705 remained relatively steady during a preset amount of time
(e.g., 5 or 10 seconds), such that both items 715A, 715B remained
within the field of view during this time, the computing system may
determine that the customer is focused on items 715A-C. In some
embodiments, the particular item(s) must continuously remain in the
field of view 705 during the preset amount of time (e.g., remain
across several samples of the field of view during this time).
[0054] View 2 illustrates a field of view 705 that includes item
715B, and portions of items 715A and 715C. View 2 could represent
the same field of view as View 1 at a later time. In some
embodiments, the customer's focus may be determined to include item
715B but not 715A. For example, the system could determine that
715B is the lone customer-focused item based on its relatively
central position within the field of view 705, or perhaps based on
the changes to the field of view from View 1 (i.e., a shift away
from previously fully included item 715A). In one embodiment, item
715B must also remain within the field of view 705 for the
predetermined amount of time to be considered as a customer-focused
item. Of course, other methods are possible to identify one or more
relatively prominent items within a field of view to determine a
customer's focus, such as determining of the percentage of the item
that is included within the field of view, determining the
percentage of the field of view occupied by the item, and so forth.
In alternative embodiments, the customer-focused items may still
include item 715A and/or item 715C.
[0055] View 3 illustrates a field of view 705 that includes item
715B, and portions of items 715A and 715C. View 3 differs from View
2 in that the field of view 705 is "closer" to the shelving unit
703 and items 715 in View 3 than in View 2. View 3 could represent
the same field of view as View 2 at a later time, e.g., as the
customer moves towards the shelving unit 703. Here, the system
could determine that 715B is a customer-focused item based on its
relatively central position within the field of view 705 and/or
based on the changes to the field of view from View 2 (i.e., item
715B occupies an increased percentage of the field of view
705).
[0056] View 4 also illustrates a field of view 705 that includes
item 715B, and portions of items 715A and 715C. View 4 could
represent the same field of view as View 3 at a later time. In View
4, the customer has selected item 715B and is holding the item in
his/her hand 725. Here, the system could determine that 715B is a
customer-focused item based on its relatively central position
within the field of view 705, based on the changes to the field of
view from View 3 (e.g., item 715B occupies an increased percentage
of the field of view 705), and/or based on the presence of a
portion 730 of the customer's hand 725 included in the field of
view 705.
[0057] In some embodiments, a customer interest score may be
determined for items included within the customer's field of view.
The customer interest score is generally time-based, and score may
be used to timely interject desired information in order to
influence a customer's shopping decisions (e.g., whether to buy a
particular item) in real-time. In one embodiment, a customer
interest score is calculated and updated concurrent with
determining a customer focus, and the customer interest score for
an item exceeding a threshold value may be used to determine that
the customer has focused on the item. The customer interest score
may also be adjusted based on various other aspects shown in the
Views 1-4, such as position within the field of view (e.g., a
central position may mean more interest), percentage occupied
within the field of view (e.g., a larger percentage may mean more
interest), customer selection and/or manipulation of the item, and
so forth. Of course, the calculation of a customer's interest score
may reflect their particular shopping behaviors (e.g., a customer
is determined to be 95% likely to pick up and manipulate a
customer-focused item within their field of view, so the mere fact
of picking up an item might not indicate an increased level of
customer interest in the item).
[0058] After determining that a customer has focused on a
particular item, the customer interest score may generally decrease
over time as it becomes more likely that the customer will not
purchase the particular item. To maintain or improve customer
interest, and to thereby increase the likelihood that the customer
will purchase the item, the system may provide information about
the item at desired times (corresponding to customer interest
scores). This may include providing information about the items or
alternatives, and/or offering promotional pricing on the item (or
an alternative item). In one embodiment, to provide more effective
promotions (e.g., more customer-tailored and more timely), and to
prevent customer abuse of a time-based promotional pricing scheme,
the customer shopping history may include data regarding the number
of promotions already offered to the customer, the length of time
(or customer interest score) before offering the promotion, the
customer utilization rate of the previous promotions, etc.
[0059] FIGS. 8A-8C illustrate several example presentations of
information related to an item, according to one embodiment. As
discussed above, the presentation of information may be based upon
some or all of the items that are identified as being within the
customer's field of view. For ease of description, FIG. 8A
illustrates a single item 805 within the customer's field of view
705; this could also represent an example in which one item is
identified from several items as being a customer focused item.
However, similar techniques may be applied to a field of view or a
customer focus that includes more than one item. In some
embodiments, the computing system may present information to the
customer about a particular item, based on the item 805. The
presented information may relate to the same item 805 and/or to one
or more different items. The different items may include other
items located within the customer's field of view 705, and may
include items (such as item 810) that are not included within the
field of view 705. In some embodiments, the information may be
determined and timely presented so as to influence the customer to
purchase the particular item(s).
[0060] FIG. 8B illustrates several example presentations of
information to a customer. The presentations may be delivered in
real-time to the customer using any suitable manner, such as a
visual output (e.g., graphical and/or textual), audio-based output,
and/or other sensory output (e.g., haptic). While the presentations
815, 820, 825 are shown as textual presentations for simplicity, it
is possible for the presentations to be spoken or presented in any
suitable alternative manner. In one example, a presentation may be
displayed by a wearable computing device of the customer, such as a
visual overlay on a display of the wearable computing device as
discussed above with respect to FIGS. 5A and 5B. In another
example, a presentation may be transmitted as a message to the
customer's mobile computing device, causing text or graphics to be
displayed on the mobile computing device and causing the device to
vibrate.
[0061] Presentation 815 indicates a customer preference for a
second item 810 to the currently viewed or focused item 805.
Presentation 815 generally suggests that the customer may also
prefer the second item 810. The customer preference may be
determined based on compiled previous shopping data for other
customers. The shopping data may include separate statistics for
the two items, for example, based on historical purchase data for
each item. The shopping data may also include compiled information
that directly interrelates the two items. For example, in previous
shopping experiences, other customers also focused on item 805
(e.g., based on determined fields of view) but ultimately declined
to purchase item 805 in favor of item 810.
[0062] Presentation 820 indicates that a promotion is available for
the second item 810. Presentation 820 may reflect a sales promotion
or prices already in effect (e.g., a reporting function), or the
promotion may be dynamically generated to encourage the customer to
purchase item 810. For example, manufacturers or stores may wish to
promote item 810 over item 805, and the promotion may be presented
to the customer in real-time. As discussed above, in some
embodiments, the promotion may be determined, as well as the timing
of its presentation to the customer, based on a determined level of
customer interest in an item.
[0063] Presentation 825 indicates that item 805 would have a
negative impact on the customer's goals. The presentation 825 may
be based on customer profile data such as customer preferences
and/or programs. Examples of programs may include fitness,
nutrition, or health goals, money management goals, etc. Although
not shown here, one embodiment may present the information to the
customer upon which the negative impact determination was made. If
the system determines that item 805 is incompatible with specified
goals or preferences, or that better alternative items exist
consistent with the customer's goals/preferences, the system may
recommend one or more alternative items for presentation to the
customer.
[0064] FIG. 8C illustrates several example presentations of
information to a customer. Like the presentations discussed above,
the presentations may be delivered in real-time to the customer
using any suitable manner, such as a visual output (e.g., graphical
and/or textual), audio-based output, and/or other sensory output
(e.g., haptic). The presentations may be output using a customer's
worn or carried computing device, or using output hardware (e.g.,
displays, speakers, etc.) that is disposed within the shopping
environment. For example, LEDs or other visual display elements may
be included in the shelving units, such as display elements 835a-b
and 836c-d.
[0065] As shown, a single item 805 is included within a customer's
field of view 705, and item 810 is located outside the field of
view. Of course, similar techniques may be applied to a field of
view or a customer focus that includes more than one item. In cases
where the item 805 (already included in the field of view 705) is
recommended for presentation to the customer, the system may
display various indicators that highlight the item 805 to the user.
For example, if the field of view 705 represents the customer view
through a head-worn computing device such as wearable computing
device 500, the system may display colors, shapes, etc. that
visually enhance the item 805, or may reduce the visibility of
other items in the field of view (e.g., gray out). The system may
also display location information of the item 805, for example, by
illuminating one or more of the display elements 835a-b nearest to
the item 805.
[0066] In cases where an item is recommended for presentation that
is not disposed within the field of view 705 (e.g., item 810), the
system may output directional information from the current field of
view 705 to the item. For example, the system may display an arrow
830 overlay indicating the relative direction from the field of
view 705 or item 805 to the target item 810. The system may also
provide directional information using one or more of the display
elements 835, 836. For example, the display elements 835, 836 may
be illuminated in sequence from left to right to guide the
customers' eye toward item 810. Of course, the various types of
information discussed here (e.g., highlighting, location, and
directional) may additionally or alternately be provided by
non-visual means to the customer, such as audio or haptic
outputs.
[0067] FIG. 9A illustrates a method of influencing a shopping
experience of a customer within a shopping environment, according
to one embodiment. Method 900 may generally be used in conjunction
with the various environments and systems discussed above. Method
900 begins at block 905, where a field of view is determined for
the customer. Determining a field of view may include a "direct"
determination, such as capturing an image from a body-worn,
forward-looking camera (i.e., the image strongly corresponds to the
customer's eye gaze). Alternatively (or in addition to the direct
determination), the field of view may be determined "indirectly" by
capturing images of the shopping environment and performing image
processing to determine various aspects. The aspects may include
the determined position and orientation of one or more of the
customer's body, head, shoulders, eyes, arms, hands, etc. The
images may be captured by visual sensors that are worn or disposed
throughout the shopping environment.
[0068] At block 915, one or more first items that are included
within the customer's field of view are identified. In one
embodiment, identifying items may include performing image
processing on a captured image that reflects the customer's field
of view (e.g., from a forward-looking camera). In another
embodiment, identifying items may include referencing coordinate
data determined for the customer's field of view against a known
store layout that includes item location information. In such an
embodiment, image processing may not be necessary to identify the
one or more first items.
[0069] At block 925, at least one second item is selected for
presentation to the customer. The selection is based on the
identified one or more first items. In one embodiment, at least one
of the second item(s) may be included in the identified first
items. In other embodiments, each of the second item(s) may be
different than the identified first items. The selection may be
based on decisional logic for recommending a particular item for
sale, based on historical shopping data for the customer (and/or
other customers) and/or customer preferences or programs.
[0070] At block 935, information related to the at least one second
item is presented to the customer. Presenting information may be
done in any suitable manner, such as displaying textual or
graphical information in a visual overlay, displaying directional
information to the customer, or outputting sound or other feedback.
The information may be related to the properties of the second item
(e.g., price, nutrition information, etc.) or to historical
shopping data, promotions, or other customer-specific preferences
or programs. The timing of the presentation may be determined based
on customer interest scores or other timing considerations that are
collectively intended to influence a customer's shopping
purchases.
[0071] FIG. 9B illustrates a method of determining content to
present to a customer based on a customer interest score, according
to one embodiment. Method 940 may generally be used in conjunction
with the various shopping environments and systems discussed above.
In one embodiment, method 940 is performed as part of performing
method 900. Method 940 begins at block 945, where a customer focus
is determined on at least one item selected from one or more items
that are included within the customer's field of view. The customer
focus may be time-based, such as requiring the item to remain
within the field of view for a minimum amount of time.
[0072] At block 955, a customer interest score is determined for
the at least one customer-focused item. The customer interest score
may also be time-based, and may be influenced by various
characteristics of the interaction of the customer with the item,
which may be observed through the determined customer field of
view. In one embodiment, the customer interest score may be
determined concurrently with determining the customer focus on an
item. In one embodiment, the customer-focused item is determined
based on the customer interest score for that item reaching a
predetermined threshold value. In another embodiment, the
customer-focused item is determined by the item's score that
exceeds scores for other items within the field of view.
[0073] At block 965, the content of the information to present to
the customer is determined based on the determined customer
interest score. Determining the content may include determining
whether to present information for the customer-focused item or for
an alternate item, whether or not to present a promotion for the
item or the alternate item, determining an amount of the promotion,
and so forth.
[0074] The descriptions of the various embodiments of the present
disclosure have been presented for purposes of illustration, but
are not intended to be exhaustive or limited to the embodiments
disclosed. Many modifications and variations will be apparent to
those of ordinary skill in the art without departing from the scope
and spirit of the described embodiments. The terminology used
herein was chosen to best explain the principles of the
embodiments, the practical application or technical improvement
over technologies found in the marketplace, or to enable others of
ordinary skill in the art to understand the embodiments disclosed
herein.
[0075] In the preceding, reference is made to embodiments presented
in this disclosure. However, the scope of the present disclosure is
not limited to specific described embodiments. Instead, any
combination of the following features and elements, whether related
to different embodiments or not, is contemplated to implement and
practice contemplated embodiments. Furthermore, although
embodiments disclosed herein may achieve advantages over other
possible solutions or over the prior art, whether or not a
particular advantage is achieved by a given embodiment is not
limiting of the scope of the present disclosure. Thus, the
following aspects, features, embodiments and advantages are merely
illustrative and are not considered elements or limitations of the
appended claims except where explicitly recited in a claim(s).
Likewise, reference to "the invention" shall not be construed as a
generalization of any inventive subject matter disclosed herein and
shall not be considered to be an element or limitation of the
appended claims except where explicitly recited in a claim(s).
[0076] Aspects of the present disclosure may take the form of an
entirely hardware embodiment, an entirely software embodiment
(including firmware, resident software, micro-code, etc.) or an
embodiment combining software and hardware aspects that may all
generally be referred to herein as a "circuit," "module," or
"system."
[0077] The present disclosure may be a system, a method, and/or a
computer program product. The computer program product may include
a computer readable storage medium (or media) having computer
readable program instructions thereon for causing a processor to
carry out aspects of the present disclosure.
[0078] The computer readable storage medium can be a tangible
device that can retain and store instructions for use by an
instruction execution device. The computer readable storage medium
may be, for example, but is not limited to, an electronic storage
device, a magnetic storage device, an optical storage device, an
electromagnetic storage device, a semiconductor storage device, or
any suitable combination of the foregoing. A non-exhaustive list of
more specific examples of the computer readable storage medium
includes the following: a portable computer diskette, a hard disk,
a random access memory (RAM), a read-only memory (ROM), an erasable
programmable read-only memory (EPROM or Flash memory), a static
random access memory (SRAM), a portable compact disc read-only
memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a
floppy disk, a mechanically encoded device such as punch-cards or
raised structures in a groove having instructions recorded thereon,
and any suitable combination of the foregoing. A computer readable
storage medium, as used herein, is not to be construed as being
transitory signals per se, such as radio waves or other freely
propagating electromagnetic waves, electromagnetic waves
propagating through a waveguide or other transmission media (e.g.,
light pulses passing through a fiber-optic cable), or electrical
signals transmitted through a wire.
[0079] Computer readable program instructions described herein can
be downloaded to respective computing/processing devices from a
computer readable storage medium or to an external computer or
external storage device via a network, for example, the Internet, a
local area network, a wide area network and/or a wireless network.
The network may comprise copper transmission cables, optical
transmission fibers, wireless transmission, routers, firewalls,
switches, gateway computers and/or edge servers. A network adapter
card or network interface in each computing/processing device
receives computer readable program instructions from the network
and forwards the computer readable program instructions for storage
in a computer readable storage medium within the respective
computing/processing device.
[0080] Computer readable program instructions for carrying out
operations of the present disclosure may be assembler instructions,
instruction-set-architecture (ISA) instructions, machine
instructions, machine dependent instructions, microcode, firmware
instructions, state-setting data, or either source code or object
code written in any combination of one or more programming
languages, including an object oriented programming language such
as Smalltalk, C++ or the like, and conventional procedural
programming languages, such as the "C" programming language or
similar programming languages. The computer readable program
instructions may execute entirely on the user's computer, partly on
the user's computer, as a stand-alone software package, partly on
the user's computer and partly on a remote computer or entirely on
the remote computer or server. In the latter scenario, the remote
computer may be connected to the user's computer through any type
of network, including a local area network (LAN) or a wide area
network (WAN), or the connection may be made to an external
computer (for example, through the Internet using an Internet
Service Provider). In some embodiments, electronic circuitry
including, for example, programmable logic circuitry,
field-programmable gate arrays (FPGA), or programmable logic arrays
(PLA) may execute the computer readable program instructions by
utilizing state information of the computer readable program
instructions to personalize the electronic circuitry, in order to
perform aspects of the present disclosure.
[0081] Aspects of the present disclosure are described herein with
reference to flowchart illustrations and/or block diagrams of
methods, apparatus (systems), and computer program products
according to embodiments of the disclosure. It will be understood
that each block of the flowchart illustrations and/or block
diagrams, and combinations of blocks in the flowchart illustrations
and/or block diagrams, can be implemented by computer readable
program instructions.
[0082] These computer readable program instructions may be provided
to a processor of a general purpose computer, special purpose
computer, or other programmable data processing apparatus to
produce a machine, such that the instructions, which execute via
the processor of the computer or other programmable data processing
apparatus, create means for implementing the functions/acts
specified in the flowchart and/or block diagram block or blocks.
These computer readable program instructions may also be stored in
a computer readable storage medium that can direct a computer, a
programmable data processing apparatus, and/or other devices to
function in a particular manner, such that the computer readable
storage medium having instructions stored therein comprises an
article of manufacture including instructions which implement
aspects of the function/act specified in the flowchart and/or block
diagram block or blocks.
[0083] The computer readable program instructions may also be
loaded onto a computer, other programmable data processing
apparatus, or other device to cause a series of operational steps
to be performed on the computer, other programmable apparatus or
other device to produce a computer implemented process, such that
the instructions which execute on the computer, other programmable
apparatus, or other device implement the functions/acts specified
in the flowchart and/or block diagram block or blocks.
[0084] The flowchart and block diagrams in the Figures illustrate
the architecture, functionality, and operation of possible
implementations of systems, methods, and computer program products
according to various embodiments of the present disclosure. In this
regard, each block in the flowchart or block diagrams may represent
a module, segment, or portion of instructions, which comprises one
or more executable instructions for implementing the specified
logical function(s). In some alternative implementations, the
functions noted in the block may occur out of the order noted in
the figures. For example, two blocks shown in succession may, in
fact, be executed substantially concurrently, or the blocks may
sometimes be executed in the reverse order, depending upon the
functionality involved. It will also be noted that each block of
the block diagrams and/or flowchart illustration, and combinations
of blocks in the block diagrams and/or flowchart illustration, can
be implemented by special purpose hardware-based systems that
perform the specified functions or acts or carry out combinations
of special purpose hardware and computer instructions.
[0085] Embodiments of the disclosure may be provided to end users
through a cloud computing infrastructure. Cloud computing generally
refers to the provision of scalable computing resources as a
service over a network. More formally, cloud computing may be
defined as a computing capability that provides an abstraction
between the computing resource and its underlying technical
architecture (e.g., servers, storage, networks), enabling
convenient, on-demand network access to a shared pool of
configurable computing resources that can be rapidly provisioned
and released with minimal management effort or service provider
interaction. Thus, cloud computing allows a user to access virtual
computing resources (e.g., storage, data, applications, and even
complete virtualized computing systems) in "the cloud," without
regard for the underlying physical systems (or locations of those
systems) used to provide the computing resources.
[0086] Typically, cloud computing resources are provided to a user
on a pay-per-use basis, where users are charged only for the
computing resources actually used (e.g., an amount of storage space
consumed by a user or a number of virtualized systems instantiated
by the user). A user can access any of the resources that reside in
the cloud at any time, and from anywhere across the Internet. In
context of the present disclosure, a user may access applications
(e.g., a retail store app for a mobile computing device) or related
data (e.g., compiled shopping data) available in the cloud. Doing
so allows a user to access this information from any computing
system attached to a network connected to the cloud (e.g., the
Internet).
[0087] While the foregoing is directed to embodiments of the
present disclosure, other and further embodiments of the disclosure
may be devised without departing from the basic scope thereof, and
the scope thereof is determined by the claims that follow.
* * * * *