U.S. patent number 8,108,255 [Application Number 11/863,094] was granted by the patent office on 2012-01-31 for methods and systems for obtaining reviews for items lacking reviews.
This patent grant is currently assigned to Amazon Technologies, Inc.. Invention is credited to Scott Allen Mongrain, Terrance R. Nightingale, James G. Robinson.
United States Patent |
8,108,255 |
Robinson , et al. |
January 31, 2012 |
**Please see images for:
( Certificate of Correction ) ** |
Methods and systems for obtaining reviews for items lacking
reviews
Abstract
A system includes an interactive item catalog and a data
repository that stores purchase histories for users, first reviews
for items submitted by users, and first review helpfulness scores.
A search engine system identifies catalog items that do not have
reviews from users. The system includes a user interface that
enables the user to select an item identified by the search engine
system as not having user reviews. The systems includes review
entry user interface via which the user can enter a first review
for an item selected by the user. A game system is configured to
calculate for a first review game first review scores and first
review rankings for users based at least in part on data stored in
the data repository and the first review scores. The system
includes an interface configured to provide information to users
related to their first review scores and rankings.
Inventors: |
Robinson; James G. (Olympia,
WA), Nightingale; Terrance R. (Marysville, WA), Mongrain;
Scott Allen (Seattle, WA) |
Assignee: |
Amazon Technologies, Inc.
(Reno, NV)
|
Family
ID: |
45508224 |
Appl.
No.: |
11/863,094 |
Filed: |
September 27, 2007 |
Current U.S.
Class: |
705/14.44;
705/347; 705/26.7 |
Current CPC
Class: |
G06Q
30/0631 (20130101); G06Q 30/0282 (20130101); G06Q
30/06 (20130101); G06Q 30/02 (20130101); G06Q
30/0245 (20130101) |
Current International
Class: |
G06Q
30/00 (20060101) |
Field of
Search: |
;705/14.72,26.7,347 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
0751471 |
|
Feb 1997 |
|
EP |
|
409265478 |
|
Oct 1997 |
|
JP |
|
2001029098 |
|
Apr 2001 |
|
KR |
|
WO 99/23591 |
|
May 1999 |
|
WO |
|
WO 00/62223 |
|
Oct 2000 |
|
WO |
|
WO 01/15002 |
|
Mar 2001 |
|
WO |
|
Other References
B Tedeschi, "Fulfilling Orders Is a Challenge for Many Internet
Companies" The New York Times on the Web, dated Sep. 27, 1999 (5
pages). cited by other .
Waddington, Alan: "Mail Order Inventory Control": 1983. cited by
other .
Colacecchi, Mary Beth, "Conquering last-minute mania; inventory,
service and staff planning are key; mail order business": Mar.
1994. cited by other .
U.S. Appl. No. 11/864,521, filed Sep. 28, 2007. cited by other
.
Office Action dated Sep. 21, 2010 from U.S. Appl. No. 11/864,521.
cited by other .
Final Office Action dated Feb. 1, 2011 from U.S. Appl. No.
11/864,521. cited by other .
U.S. Appl. No. 11/864,521, filed Sep. 28, 2007, Robinson et al.
cited by other .
"eBay Launches the Most Comprehensive Trust and Safety Upgrades to
the World's Largest Person-to-Person Trading Site." PR Newswire,
Jan. 15, 1999. cited by other .
"Epinions.com Announces the Launch of its Free Online Shopping
Guide Powered by Consumer Opinions." PR Newswire, p. 2829, Sep. 8,
1999. cited by other .
"Home, Netscape, Yahoo! Veterans Announce Epinions.com." PR
Newswire, p. 8921, Jul. 12, 1999. cited by other .
"MyGiftList.com Announces launch of Online Gift Registry" Dec.
1999, Business Wire. cited by other .
A. Barrett, "What's Your Epinion?", PC World, Sep. 13, 1999 (2
pages printed from PCWORLD.COM website). cited by other .
Abstract: "Just Add People--Collaborative filtering brings human
input to information retrieval in the enterprise". cited by other
.
Amazon: Paquet, Cheri; Amazon.com Buys Junglee and PlanetAll: PC
World Online, Aug. 5, 1998, Proquest #58947830. cited by other
.
Amazon: Weil, Nancy; "PlanetAll Keeps Online Masses Organized," PC
World Online, Aug. 14, 1998, Proquest #33761930. cited by other
.
Barrett, Alexandra. `What`s Your Epinion? Network World, page NA,
Sep. 13, 1999. cited by other .
C. Kirby, "Everyone's A Critic--Web sites Hope online reviews of
products lead to online buying," San Francisco Chronicle, Jan. 22,
2000 (3 pages printed from sfgate.com website). cited by other
.
D. Willmott, "Deja View," PC Magazine Online. dated May 26, 1999 (2
pages). cited by other .
D. Willmott, "epinions.com," PC Magazine, dated Sep. 10, 1999 (2
pages printed from zdnet.com website). cited by other .
eBay's Feedback Forum web site, archived on Oct. 12, 1999 by
web.archive.org [URL:
http://web.archive.org/web/19991012065814/pages.ebay.com/...].
cited by other .
Eplnlons.com. www.epinions.com, Dec. 4, 1999 [retrieved May 17,
2004], pp. 1021, retrieved from: Google.com and archlve.org. cited
by other .
Lenatti, Chuck. "Auction Mania." Upside, vol. 11, No. 7, pp. 84-92,
Jul. 1999. cited by other .
M. Roscheisen, C. Mogensen and T. Winograd, "Beyond browsing:
shared comments, SOAPs, trails, and on.line communities," Computer
Networks and ISDN Systems, Proceedings of the Third International
World-Wide Web Conference, pp. 739-749, dated Apr. 1995. cited by
other .
PlanetAll: Internet Archive Wayback Machine, www.archive.org;
www.planetall.com; Nov. 14, 1997, 19 pgs. cited by other .
PlanetAll; Cardwell, Annette; Cyberscene PlanetAll Keeps You in
Touch, Boston Herald, Dec. 6, 1996, Proquest #17350285. cited by
other .
Rafter, "10 Companies to Watch: Epinions.Com," The Industry
Standard Magazine, dated Jan. 17, 2000 (3 pages printed from
TheStandard.com website). cited by other .
Tedeschi, Bob. "Consumer Products are Being Reviewed on More Web
Sites, Some Featuring Comments From Anyone With an Opinion." (Full
Text Version) New York Times, Late Edition--Final ED, col. 1, p.
16, Oct. 25, 1999. cited by other .
UU: Carrol, Matt: "Can't Keep Track of Your Friends? A Cambridge
Firm Will," Boston Globe, Jan. 4, 1998, Proquest #25464125. cited
by other .
W. Andrews, "Building Communities As Way of Building Sales," Web
Week, vol. 2, Issue 20, Dec. 16, 1996 (2 pages). cited by other
.
Watt, Peggy; Internet Article; Varied Collaborative Approaches
Entice Intranet Users: Network World, Apr. 21, 1997 ProQuest
#11525439. cited by other .
Wohl, Amy. "User Review--Your Opinions Are Highly Valued on the
Web." VarBusiness, No. 1528, p. 69, Nov. 8, 1999. cited by other
.
"A few well placed Questions", Inc., Sep. 15, 1998 [retrieved on
Jun. 26, 2002]. vol. 20, Issue 13, 2 pages, Received from: Proquest
Direct. cited by other .
"BizRate Names `The Buyers` Best' Online Merchants in Customer
Satisfaction for June", PR Newswire, Jul. 2, 1998 [retrieved Feb.
20, 2003], pp. 1-2, retrieved from: Proquest Direct. cited by other
.
"Instant Survey Gets Info from Customers, Fast", Small Business
Information, Dec. 21, 1998 [retrieved on Jun. 26, 2002], 2 pages,
retrieved from: http://sbinformation.about.com/library/weekly.
cited by other .
"L.L. Bean, Inc: Corporate Strategy", Harvard Business School, May
18, 1988, pp. 7, 8, and 17. cited by other .
"NetReflector's www.InstantSurvey.com Goes Live Today", Business
Wire, Mar. 23, 1998 [retrieved on Jun. 26, 2002], 3 pages,
retrieved from: Proquest Direct. cited by other .
Dragan, Richard, et al., "Advice from the Web", PC Magazine, Sep.
9, 1997, [retrieved Jun. 9, 2005], vol. 16, No. 15, pp. 1-9,
retrieved from: Dialog, file 275. cited by other .
Hiam, Alexander, "Marketing for Dummies", IDG Books Worldwide,
1997, pp. 98-108. cited by other .
Hof, Robert D., "Technology, Technology, Technology", Business
Week, Dec. 13, 1998 [retrieved on Jun. 25, 2002], 2 pages,
retrieved from: Proquest Direct. cited by other .
Labounty, Char, "Perception Predominates", Service News, Nov. 1998
[retrieved on Jun. 26, 2002], vol. 18, Issue 12, 2 pages, retrieved
from : Proquest Direct. cited by other .
Leong, Kathy Chin, "Customer Service Gets the Royal Treatment",
Internetweek, Sep. 14, 1998 [retrieved Jun. 26, 2002], pp. 1-4,
retrieved from: Proquest Direct. cited by other .
Magazzine, Tom, "Re-evaluating business processes", May/Jun. 1998
[retrieved Jul. 11, 2002], vol. 12, Issue 3, 7 pages, retrieved
from: Proquest Direct. cited by other .
Net Perceptions, Inc., "Net Perceptions: The GroupLens Solution",
www.netperceptions.com, Jan. 18, 1997 [retrieved Feb. 11, 2005],
pp. 1-31, retrieved from: Google.com and archive.org. cited by
other .
Planned Maintenance Program, "What is PMP?", www.followup.com, Apr.
15, 1997 [retrieved Jun. 26, 2002], pp. 1-4, retrieved from:
Google.com and archive.org. cited by other .
Sheff, David, "Mr. Satisfaction", Los Angeles Times, Oct. 13, 1996
[retrieved Oct. 28, 2005], pp. 1-5, retrieved from: Proquest
Direct. cited by other.
|
Primary Examiner: Weiss; John G.
Assistant Examiner: Chornesky; Adam
Attorney, Agent or Firm: Knobbe, Martens, Olson & Bear
LLP
Claims
What is claimed is:
1. A computing system, comprising: an interactive catalog of items;
a data repository that stores: purchase histories for users, first
reviews for items submitted by users and an identification as to
which user submitted a given first review, and indicators related
to how helpful others have found the first reviews for items; a
search engine system configured to identify catalog items that do
not have reviews from users; a recommendation system configured to
recommend candidate items for a user to provide first reviews for,
wherein the recommendation system uses data stored in the data
repository to determine user item affinity and the identification
of catalog items that do not have reviews to generate
recommendations for first review candidate items; a first review
candidate selection interface that provides functionality for the
user to select an item from the recommended candidate items; a
review entry user interface via which the user can enter a first
review for an item selected by the user; a game system configured
to calculate for a review game first review scores and first review
rankings based at least in part on data stored in the data
repository; and an interface configured to provide information to
users related to their first review scores and rankings.
2. The computing system as defined in claim 1, wherein the search
engine system is configured to provide functionality for users to
conduct searches to locate items in the interactive catalog that do
not have user reviews.
3. The computing system as defined in claim 1, wherein the game
system is configured to calculate a first review score for a first
user based at least in part on a quantity of first reviews provided
by the first user and on indications as to how many other users
found the user's first reviews helpful.
4. The computing system as defined in claim 1, wherein the game
system is configured to calculate a first review score for a first
user based at least in part on a quantity of first reviews provided
by the first user and on indications as to how many other users
found the user's first reviews not helpful.
5. The computing system of claim 1, wherein the recommendation
system is configured to recommend candidate first review items to
the user based at least in part on item-to-item similarity mappings
with respect to items the user has provided reviews for and/or has
purchased.
6. The computing system of claim 1, wherein the recommendation
system is configured to offer enhanced first review points for
first reviews submitted for categories of items identified as
having more than a specified percentage and/or amount of items
without user reviews.
7. The computing system of claim 1, wherein the computing system
includes a user interface via which the user can specify that the
system is to email message and/or transmit a message to the user's
mobile phone when one or more user specified events related to the
review game occur.
8. A computer-implemented method of providing a review game,
comprising: recommending, by a computer system including hardware,
to a first user first review candidates for items in an electronic
catalog that do not have user reviews; calculating, by the computer
system, a first review score for the first user based at least in
part on the quantity of reviews submitted by the first user in a
first time period and on helpfulness indications provided by other
users with respect to first user's first reviews within the time
period; calculating, by the computer system, a first review ranking
for the first user relative to other users playing the review game
based least in part of the user's first review score and the first
review scores of other users; and assigning a badge and/or title to
the first user based at least in party on the first user's first
review score.
9. The method of claim 8, further comprising: identifying at least
a first category of items having a first characteristic; and
offering additional first review points to users that submit first
reviews for items within the first category as compared to review
points offered for a category of items that do not have the first
characteristic.
10. The method of claim 8, wherein the first characteristic relates
to having a higher percentage of items that do not have user
reviews than other categories.
11. The method of claim 8, further comprising storing in a computer
readable medium: purchase histories for users, first reviews for
items submitted by users and an identification as to which user
submitted a given first review, and indicators related to how
helpful others have found the first reviews for items.
12. The method of claim 8, wherein the first review candidates are
selected based at least in part on item affinity determinations for
the first user.
13. The method of claim 8, further comprising providing a user
interface via which the first user can search for items in a
selected category in the electronic catalogue that do not have user
reviews.
14. The method of claim 8, further comprising providing a user
interface displaying information related to the first user's
performance in the review game, including the first user's ranking
relative to other users.
15. The method of claim 8, wherein the first review candidates
include at least one item that has a review from a distributor
and/or manufacturer.
16. The method of claim 8, further comprising providing review game
performance trend information for the first user to the first
user.
17. A computing system, comprising: an electronic catalog of items;
a data repository that stores: purchase histories for users, first
reviews for items submitted by users and an identification as to
which user submitted a given first review, and scores related to
how helpful others have found the first reviews for items; a first
review candidate selection interface that provides functionality
for a user to select an item not having any user reviews; a review
entry user interface via which the user can enter a first review
for an item selected by the user; a game system configured to
calculate for a review game first review scores and first review
rankings for users based at least in part on data stored in the
data repository and the first review scores; and an interface
configured to provide information to users related to their first
review scores and rankings.
18. The computing system as defined in claim 17, further comprising
a recommendation system configured to recommend candidate items for
the user to provide first reviews for.
19. The computing system as defined in claim 17, wherein the
computing system is configured to offer enhanced first review
points for first reviews submitted for categories of items
identified having a first characteristic.
20. The computing system as defined in claim 17, further comprising
a search engine system configured to identify catalog items that do
not have reviews from users.
21. The computing system as defined in claim 17, wherein the game
system is configured to calculate a first review score for a first
user based at least in part on a quantity of first reviews provided
by the first user and on indications as to how many other users
found the user's first reviews helpful and how many other users
found the user's first reviews unhelpful.
22. The computing system as defined in claim 17, wherein the game
system is configured to prevent a score associated with a given
first review from exceeding a predefined cap.
23. The computing system as defined in claim 17, wherein the
computing system includes a user interface that displays to a first
review ranking, badge and/or title, and score for at least one
user.
24. The computing system of claim 17, wherein the computing system
includes a user interface that provides information regarding how
to improve at least one first review at least partly in response to
a current score associated with the at least one review reaching a
first level.
25. The computing system of claim 17, wherein the computing system
includes a user interface via which the user can specify that the
system is to automatically transmit a message to the user when one
or more user specified events related to the review game occur.
Description
BACKGROUND
Description of the Related Art
Certain conventional online systems accept, process and post
reviews for items. Such reviews aid other users in evaluating such
items and in deciding whether to acquire such items. One drawback
of conventional online systems is that typically many items or
categories of items may not receive any reviews, depriving others
of such helpful information and making acquisition of such items a
riskier and more uncertain task.
BRIEF DESCRIPTION OF THE DRAWINGS
Example embodiments will now be described with reference to the
drawings summarized below. These drawings and the associated
description are provided to illustrate particular example
embodiments and implementations, and not to limit the scope of the
invention.
FIG. 1 illustrates a web site system that implements services
described herein, and illustrates the flow of information between
components of the system.
FIGS. 2-9 illustrate example user interfaces.
FIG. 10 illustrates an example process performed by the system
illustrated in FIG. 1.
FIG. 11 illustrates another example process performed by the system
illustrated in FIG. 1.
FIG. 12 (including FIGS. 12A1-6) illustrates another example user
interface.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
Various features and methods will now be described in the context
of a system and a set of services used to identify database records
having a paucity of certain types of data. By way of illustration,
the database records may be associated with products or services,
and the deficiency in data may be product or service reviews.
Certain example embodiments described herein search for and
identify items in an interactive electronic catalogue that do not
have an associated item review or are missing other desirable
information. Such items are identified to certain users and such
users are prompted or otherwise encouraged to submit the missing
information. For example, users may participate in an information
submission game, where users may be assigned points for submitting
certain types of information for an item. Additional points are
optionally assigned for a given submission based on the number of
votes from other users indicating that the helpfulness and/or
accuracy of the submitted information. Points are optionally
subtracted based on the number of votes from other users indicating
that the submission is not helpful and/or inaccurate. Optionally, a
cap is provided on the number of points a given information
submission can be awarded. Optionally, a floor is provided with
respect how low a point score an information submission may
receive.
Certain example embodiments track information submission scores for
a plurality of users and using such information, assign rankings
and/or titles (such as badges). Certain example embodiments provide
user interfaces that display information on a first review online
game, including player rankings, titles/badges, top player(s) in a
given period of time, players having submitted the most or more
helpful information.
By way of example, the information submission may be a review in
the form of a textual review, a point review (which may be in the
form of a number of stars), a grade review, an icon review (e.g.,
happy or sad face), an audio review (e.g., recorded live or
submitted as an audio file, such as an MP3, ACC, OGG, WAV, AC3,
WMA, or other file format), a video review (e.g., recorded live, or
submitted as an MPEG, Flash, WMV, AVI, or other file format, and
optionally including an audio component), and/or other form of
review. The items/products may, for example, include physical
products that are shipped to users, digital products that are
downloaded or transmitted to users, and/or services provided to
users. As will be recognized, the disclosed methods are also
applicable to other types of items, such as authors, musical
artists, users, businesses, brands, web sites, web pages, user
communities, chat rooms, blogs, etc.
Throughout the description, the term "product" will be used to
refer generally to both an actual product (e.g., an Acme Widget)
and to its record or description in a computer data repository. A
more specific meaning may be implied by context. The more general
term "item" will be used in the same manner.
While the following description may refer to incentivizing or
motivating users to submit first product reviews, the processes and
systems discussed herein can be used to incentivize or motivate
users to perform other actions, such as submitting reviews
generally, submitting tags, providing votes as to the quality or
helpfulness/unhelpfulness of reviews or other submissions,
submitting pictures of products, submitting product catalogs,
submitting product manuals, submitting guides, submitting articles,
participating in an online discussion, updating product
information, recommending accessories (e.g., products that can be
used with a given item, such as ink for a printer, batteries for an
electronic toy, memory cards for cameras, software for computers,
service contracts for electronic products, phone service for a
cellular phone, etc.), providing recommendations as to similar
alternate products, submitting useful software programs or widgets,
adding information or corrections to a wiki/collaborative website,
participating in a forum, etc.
To facilitate an understanding of the specific embodiments
described below, an overview will initially be provided of an
example web site system. Throughout the description, reference will
be made to various implementation-specific details of this system.
These details are provided in order to fully illustrate specific
embodiments, and not to limit the scope of the invention.
In this particular embodiment, the web site system ("web site")
comprises functionality for users to search, browse, and make
purchases from an interactive electronic catalog of products, such
as book titles, music titles, video/DVD titles, and electronics
products over the Internet, or other network, and to provide
reviews for items in the catalog. The web site may also provide
functionality for renting products (such as software, DVDs or
downloaded movies or music), for listing products for sale in an
online marketplace, and/or for performing other types of
product-related actions. The various products are optionally
arranged within a hierarchical browse structure or "browse tree" in
which a node represents a product category or subcategory.
Detailed information about a product can be obtained by accessing
that product's detail page. As used herein, a "detail page" is a
page that predominantly contains information about a particular
product or other item. In a particular example embodiment, a
product detail page may comprise a description, picture, and price
of the product, customer reviews of the product (if any have been
submitted) and a listing of helpful/unhelpful review votes, lists
of related products, and information about the product's
availability from one or more sellers. When an audio or video
review is posted on a detail page, optionally playback controls
(e.g., play, pause, rewind, fast forward, increase volume, decrease
volume, etc.) are provided as well. When a video review is posted
on an item page, optionally a still (e.g., the first frame) from
the video review is statically displayed prior to the user
activating a playback control. Optionally, a textual title
submitted by a user in association with a video or audio review may
be displayed as well. The product detail pages may also provide
functionality for users to perform such actions as purchasing the
product, creating a listing for selling the product, submitting a
product review, voting on a product review, and/or adding the
product to a personal shopping cart, wish list, or rental
queue.
The product detail pages, and/or other pages of the web site,
optionally also include elements for users to assign tags (e.g., a
keyword or category label) to particular products. These tags may
be used for various purposes, including obtaining product
recommendations that are specific to user-definable product
categories, interests, and contexts. In one embodiment, users can
also obtain personalized recommendations without tagging any
products; these recommendations may be based on the target user's
purchase history, rental history, item viewing history, first
reviews for products submitted by the user, other product reviews
submitted by the user, and/or product ratings.
The recommendations provided by the web site are generated by a
recommendation service or "engine" that, may for example, accept an
input list of items (e.g., product IDs), and returns an output list
of items that are deemed to be collectively similar or related to
the items on the input list. The recommendation service may also
optionally take into consideration information about the known or
predicted affinity the user has for particular items on the input
list. For example, if a user provided a review on an item
(optionally even if there is no record on the system of a purchase
of the item by the user), other similar products or products
purchased by other users that purchased the reviewed product may be
recommended to the user. Products that the user provided first
reviews on may be weighted more heavily in determining an affinity
and providing recommendations then products where the user provided
a review but did not provide the first review.
Further, the content of the review provided by the user (e.g., the
point rating or a text rating parsed by the system) may be used in
determining what products to recommend to the user. For example, if
the user rated one item on the input list highly, and gave an
average rating or no rating to another item, the highly rated item
may be given more weight. Examples of methods and data structures
that may be used to implement the recommendations service are
described in U.S. Pat. No. 6,912,505, the disclosure of which is
hereby incorporated by reference.
It has been observed that users are more likely to purchase
products that have associated reviews available. User written
reviews often give other users a more credible evaluation of a
product than the description provided by the product manufacturer
or distributor. Thus, a significant increase in sales may occur
upon the presence of even a single positive review. Further, the
existence of at least one review may encourage others to submit
reviews. Thus, in order to encourage users to submit reviews for
products or other items where no reviews have yet been submitted
and accepted, certain example embodiments identify which products
have not yet been reviewed and/or have less than a certain number
of reviews. The system may then encourage or incentivize users to
submit reviews for such products. This is enables the system to
better ensure that review coverage is provided for a larger number
and distribution of products and product types.
By way of example, a game or contest may be provided wherein the
submission of initial reviews (also referred to as first reviews)
for a product or other item may result in the user being rewarded
or otherwise recognized, thereby providing an incentive to
participate and play. Optionally, a user is entered into the game
automatically upon submission of a review by the user. Optionally,
a user can opt in or out of the game via an account maintenance Web
page or otherwise. If the user opts out of the game, the system
does not display the user's first review game ranking or badges,
and optionally, the user's first reviews are not considered in
assigning rankings to other users play the game.
In an example review game, a user may be given a certain number of
points for each review the user submits that is the first review
for the corresponding product. The user may be enabled to compete
against himself/herself, wherein if the user achieves certain point
levels the user is assigned a certain designation (e.g., a
title/badge) that indicates such achievement. For example, if the
user reaches a first number of points (e.g., 40 points), the user
is assigned a first title (e.g., bronze medal winner). If the user
reaches a second number of points (e.g., 100 points), the user is
assigned a second title (e.g., silver medal winner). If the user
reaches a third number of points (e.g., 500 points), the user is
assigned a third title (e.g., gold medal winner). If the user
reaches a fourth number of points (e.g., 2000 points), the user is
assigned a fourth title (e.g., platinum medal winner), and so on.
Optionally, the user's points may be reset to zero or other number
periodically (e.g., every three months), and the user would need to
earn new points in order to achieve the title or badge. Other
titles or badges may be assigned to a user based on the number of
first reviews submitted by the user or based on other first review
parameters.
In addition or instead, a user can compete with other users
participating in the first review game. For example, a user can
compete with other users based on the relative number of points,
titles, badges, and/or other forms of recognition (e.g., having a
review highlighted on a helpful review Web page or designated as a
most helpful first review) they acquire. A list of first review
submitters can be presented in order of their point rankings and/or
titles. The rankings and/or titles can be displayed in association
with a ranked list entry. For example, a listing entry can include
some or all of the following: current ranking; previous ranking
(e.g., during a specified period, such as the previous week, month,
and/or year); a graphic indicating a trend or change in ranking
(e.g., indicating whether the ranking has improved from a previous
period, gotten worse, or is unchanged); a title/badge (e.g., best
first reviewer, top 10 first reviewer, top first reviewer within a
specified period of time (e.g., today, this week, this month, last
90 days, this year, etc.)); a user name (which may be the user's
actual name, or a pseudo name/nickname); the total current points
for the user's first reviews (which may be restricted to points
received within a specified period of time, such as the previous 90
days or 1 month); the number of first reviews for products
submitted by the user; a helpfulness or usefulness score (e.g., a
percentage of the total helpful/not-helpful votes for the user's
first product reviews that indicate the reviews are helpful, and/or
a tally of the number of helpful votes submitted by others for the
user's first reviews within a specified time period, optionally
with points detracted for unhelpful votes); number of first reviews
submitted by the user.
Optionally, some or all of the foregoing information is updated in
substantially real time. Optionally, the list can be sorted based
on one or more of the entry information types (e.g., based on
ranking, badges, helpfulness score, number of reviews, etc.).
The rankings, titles, badges, trend indicators, and/or other
information will often motivate users to improve their ranking,
title, badge, and/or ranking performance trend. Information is
optionally provided to users indicating how they can improve their
ranking, title, badge, and/or ranking performance trend.
Optionally, the ranking and title assignment process and criteria
are explicitly disclosed to users so as to provide transparency
with respect to scoring, ranking, and the assignment of titles.
A user can be provided with notifications (e.g., via email, Short
Message Service (SMS), instant messaging, a Web page, or otherwise)
when a significant event has occurred with respect to their
ranking, title, badge, or trend or that of other users. Optionally,
the user can specify via an account setup Web page the type of
notifications that are to be provided and how the notifications are
to be provided. For example, the notifications can be triggered by
one or more of the following events: change in the user's ranking;
change in the user's title; change in another specified user's
ranking; change in another specified user's title; surpassing
another specified user's ranking; being surpassed by another
specified user's ranking; reaching a specified number of points;
dropping below a specified number of points; change in the number
and/or percentage of helpful votes for the user's first reviews; a
user review score falls to a certain level (e.g., 0 points); a
determination that an item purchased by the user does not have any
reviews on the item web page; availability of bonus points for
submitting a first product review for a specified product or class
of products (e.g., underwear).
Review requests can also be transmitted to users using techniques
and processes described in U.S. Pat. No. 6,963,848, the content of
which is incorporated herein by reference.
Points can be assigned to a user based solely on the number of
first product reviews submitted by the user (optionally within a
specified period of time, such as the previous 3 months, or within
other specified period, such as the beginning of the second quarter
to the previous day), or points can be based on additional or
different considerations. For example, a user can be assigned a
first number of points (e.g., 40 points) for submitting the first
review for a product ("first product review"), an additional number
of points (or fraction of a point) for each helpful vote received
from other users regarding the helpfulness/usefulness of the
review, and a negative number of points (or fraction of a point)
for each not helpful or not useful vote received. The votes may be
provided via an item Web page that displays the user's review and
voting buttons (e.g., helpful/unhelpful, good/bad, etc.).
By way of further illustration, a "helpful" vote may have a value
of "1," and a "not helpful" vote may have a value of "-1." For
example, a reviewer with 100 "helpful" votes and 20 "not helpful"
votes may have a tally or score of 80. In another embodiment, the
tally may be based on only "helpful" or on only "not helpful"
votes, or equivalents thereof. The "helpful" and "not helpful"
votes may optionally be assigned different values. For example, a
"helpful" vote may be assigned a value of "2" and a "not helpful"
vote may be assigned a value of "-0.5," so that "helpful" votes are
weighted more heavily than "not helpful" votes.
Optionally, the tally of votes/ratings given to anonymous reviews
may be calculated differently from those given to non-anonymous
reviews. In an example embodiment, a rating of a reviewer's
anonymous review is given less weight then that of a rating of a
non-anonymous review by the reviewer. For example, a "helpful" vote
for a non-anonymous review by a given reviewer may be assigned a
value of "2," while a "helpful" vote for an anonymous review by the
given reviewer may be assigned a value of "1." Generally, this
allows non-anonymous reviews and their authors to more easily
attain higher scores or ratings than anonymous reviews, thereby
encouraging reviewers to provide non-anonymous reviews, which
generally appear more credible to readers.
Similarly, the tally of votes/ratings given to reviews by users
that have been verified to have purchased the item being reviewed
(e.g., by accessing a purchase history database record associated
with the user or via a receipt provided by the user) may be
calculated differently (e.g., have different weightings) from those
given to users whose purchase or use of the item being reviewed has
not been verified. Further example systems and methods for
calculating a rating for a review that may be utilized are
described in U.S. patent application Ser. No. 09/715,929, the
disclosure of which is hereby incorporated by reference.
Optionally, if a user's review receives a certain number of
negative reviews or the user's review score falls to a certain
level (e.g., 0 points) as a result of negative reviews, the user is
so notified (e.g., via a Web page, email, SMS/mobile phone message,
or other communication). The notification may include a link to the
user's review via which the user may edit the review to improve its
quality. Optionally, guidelines may be provided on how the review
can be improved (e.g., provide additional substantive details, use
shorter sentences, compare to other similar products, do not be
insulting, etc.). The guidelines may be generic, or may be selected
based on an automated or manual analysis of the review text.
Optionally, bonus points (e.g., extra points, such as a specific
number of points, or a point multiplier, such a doubling the points
for a review submission and/or for each helpful votes) can be added
or included in the user's scores for first product reviews for
specified products and/or product classes. Such bonus points can be
used to further encourage users to provide reviews for certain
products or product classes that do not have any reviews or that
are generally under-reviewed.
For example, if few reviews are submitted for undershirts, users
can be notified (e.g., via email, SMS or other mobile phone
communication, Web page, or otherwise) that bonus points will be
awarded to a user that submits a first product review for an
undershirt product generally or for a specific undershot product
(e.g., having a specific SKU). The communication can optionally
include a link to a corresponding product page, wherein a control
is provided via which the user can submit a review (e.g., a first
product review).
In an example embodiment, bonus points are automatically or
manually awarded for product reviews submitted that meet the some
are all of the following criteria:
the product has been offered via the system for at least a first
specified period of time (e.g., 15 days, 30 days, 90 days, or other
specified period) and no review has been posted;
the product is in a product category wherein less than a specified
amount and/or specified percentage of products have received a
first review (e.g., less the 70%, 50%, 25%, or other
percentage);
the product detail page has been viewed a specified number of times
(e.g., 1 time, 5 times, or other specified number of times), and no
review has yet been posted;
the product is being offered by a relatively new seller that became
a seller on the web site hosted by the system within a specified
period of time (e.g., the last 30 days, 60 days, or other specified
number of times);
the product price;
the profit that would be realized from a sale of the product.
Optionally, a minimum and/or maximum point restriction may be set
and stored in memory for first product reviews generally and/or for
specific first product reviews. Thus, for example, a maximum of 100
points may be set and/or a minimum of 0 points may be set. The
foregoing point limitations ensure that a single first product
review (e.g., on a book that is very likely to be highly popular)
with a large number of helpful votes does not unduly weight the
user's ranking. Similarly, if a user submits a negative review for
a controversial book, it is likely that the user may receive many
biased unhelpful votes. Providing a minimum point score ensures
that a single unpopular review will not unduly lower the user's
ranking.
Optionally, however, if a user violates certain review rules (e.g.,
by submitting large number of duplicate supposed reviews that are
not substantive, such as "good product"; by submitting reviews
within a very short period of time (e.g., 30 seconds) after an item
is posted, indicating that the user is trolling for first review
opportunities; by submitting offensive reviews, etc.) the review
may be removed and the user may be given the standard minimum per
review that is in violation of the review rules, less than the
standard minimum per review (e.g., -20 points), or the user may be
prohibited from submitting additional reviews generally or first
reviews specifically. Optionally, a review may be removed,
associated points cancelled, and/or negative points awarded if the
community votes or requests that review be removed.
Further, if it is determined that a user is participating in a
voting circle (e.g., where a group of people automatically submit
helpful votes for each other's review rather than on the basis of
the quality of the reviews) the user can be similarly penalized as
discussed above and/or the user's helpful votes are removed and not
counted towards the corresponding review score. Similarly, if a
user (e.g., a parent) always submits helpful votes for another user
(e.g., a child), optionally the user's helpful votes are removed
and not counted towards the corresponding review score. Other types
of biased voting (e.g., voting by a reviewer enemy or friend) or
non-credible voting (e.g., voting on review after review in very
quick succession, wherein it is unlikely that the vote has had time
to actually read the reviews) can be detecting and discounted as
desired. Thus, different types of biased votes can be processed
differently and the penalties for biased votes are optionally
increased as the votes appear to be more fraudulent.
Optionally, a search service is provided via which a user can
search for products that have not yet been reviewed. Optionally,
the search may be limited by the user to specific product areas
(e.g., books, text books, music, movies, magazines, newspapers,
kitchen, home, outdoors, clothing, shows, jewelry, watches,
accessories, toys, electronics, computers, office products,
software, cameras, audio, video, cell phone, cell phone service,
musical instruments, video games, food, pet supplies, beauty,
healthcare, furniture, home improvement, exercise, automotive,
industrial, scientific, lawn & garden equipment, tools,
hardware, etc.) and/or to products that the system has a record of
the user purchasing or licensing.
FIGS. 2-9 illustrate example user interfaces. As will be
recognized, the particular features of the illustrated user
interfaces are merely illustrative, and many other possible
embodiments may be used as well. For example, elements from
different user interfaces can be combined into the same user
interface, elements from a given user interface can be divided up
among multiple user interfaces, and users interfaces can include
additional or fewer elements than those illustrated. Further, the
language provided in the user interfaces is merely illustrative and
optional, and other language and terminology may be used. The user
interfaces are optionally provided as Web pages.
FIG. 2 (including FIGS. 2A1-A2) illustrates a user interface, which
can act as a first review game landing page, providing information
regarding users that have provided first product reviews. Links or
menus are provided via which the user can access other user
interfaces described herein.
Referring to FIG. 2, interface 202 provides a listing of monthly
top first reviewers for a number of months (three in the
illustrated example). One or more controls (e.g., a links) 204 are
provided, which when activated, cause the user interface to display
a listing of additional or all top first reviewers for a given
month (or other specified time period).
A helpful first reviews interface 206 displays one or more first
product reviews that have received the highest or among the highest
helpful scores (e.g., wherein a helpful score equals or corresponds
to the number of helpful votes from others minus the number of
unhelpful votes). The first product reviews are displayed in
association with a product title, a star review (where the greater
the number of stars, the more favorable the user is rating the
product), a review title, a review date, the name (actual or pseudo
name) of the first reviewer, the number of points earned by the
review, the number of helpful votes, the number of unhelpful votes,
the difference between the number of helpful and unhelpful votes,
and the number of first product review points assigned to the
reviewer. The textual review is also provided. Optionally, a
photograph or avatar/icon representing the reviewer and/or the
product is presented. A control 208 is provided, which when
activated, causes the user interface to display a listing of
additional or all top first reviews.
A badge interface 210 is provided which lists the available badges
(e.g., gold, silver, bronze, top first reviewer (lifetime)), and
the number of users that have earned the badges.
Interface 212 lists, via a leaderboard, the users that have the
most first review points earned is a specified period of time
(e.g., within the last 30, 60, or 90 days). In this example, the
top 10 users are listed, although fewer, additional, or all users
that have provided first product reviews within the specified time
period (or other time period) can be listed. In this example, a
given entry includes the user's rank, a symbolic indication (e.g.,
an upwards or downwards pointing arrow or triangle) indicating
whether the user's rank has improved or gotten worse as compared to
a previous ranking, a photograph or icon corresponding to the user,
the user name (real or pseudo name), the current number of first
review points, the number of first reviews submitted within the
specified period of time, and the percentage of helpful votes
relative to the sum of (helpful--not helpful votes) received for
the user's first product reviews within the specified period of
time. A control is provided that when activated causes an expanded
version of the leaderboard to be presented, as illustrated in FIG.
3 (including FIGS. 3A1-A2).
Interface 214 provides information regarding how to play the game,
how points and badges are awarded, how certain titles are awarded,
and a notification regarding the possibility of rule changes.
A forum interface 216 is provided via which a user can access, read
and submit posts in forums related to the game/contest. For
example, users can discuss matters that they are happy or unhappy
with, provide information, ask advice, give advice, discuss
accomplishments (negative or positive), or chat. The forum may
include sub-forums dedicated to certain topics.
Interface 218 displays the user's name, current number of points
received within a specified period of time (e.g., the current
month), the user's current ranking, and an explanation regarding
how many points are needed to receive certain corresponding badges,
and regarding the badge currently awarded to the user (e.g., the
gold badge in this example as indicated by the visual emphasis).
Thus, the user can visit this Web page to check on their progress
in terms of points, ranking and/or title. A control is provided
which the user can activate to log-in to another account.
Interface 220 displays information regarding potential first review
candidates and the number of points that would be earned by
submitting a first review, wherein the candidates are products that
the system has determined from the user account database that the
user has purchased. The candidates may be presented via a
scrollable list (including a product name and/or photograph), and
when the user sees a product that the user is interested in review,
the user can activate a "write a first review" link. A user
interface (not shown), including a text entry field is displayed
via which the user can enter a textual review, and a star selection
user interface is optionally provided via which the user can assign
an appropriate number of stars corresponding with the item
quality.
Several other different interfaces are provided via which a user
can identify and select a product to provide a first review for.
The user interfaces may be used to select a specific product,
browse a product category, or search for a product that has not yet
been reviewed. Interface 222 displays several first review
candidates of particular products or categories of products via
which the user can browse and select a product to provide a first
review for. The categories can be selected to be or to include
those categories in which the user has made a product purchase.
Interface 224 includes search fields via which the user can search
for products that are in need of a first review. A drop down menu
may be used to select one or more product categories within which
the search is to be conducted. A search text field is provided via
which the user can enter one or more search terms, optionally using
Boolean equations. The user can activate a search control or
provide an enter command and, in response, the system will conduct
a search in accordance with the user's selection and search terms
and return corresponding search results for display to the
user.
Interface 226 displays a scrollable list of first review candidates
for which bonus points may be earned. The candidates optionally may
be limited to or include products purchased by the user via the
system. Optionally, products purchased as gift by the user for
another (e.g., as indicated by a corresponding database record that
indicates that the user has indicated (explicitly and/or by
instructing that the product be gift wrapped) are excluded from the
first review candidates provided via interface 226.
FIG. 3 illustrates an expanded view of the leaderboard 212
illustrated in FIG. 2, including an extended list 302 of first
reviewers, including the following information for a given
customer: rank, the rank in a previous period, name, the number of
first reviews submitted in a period of time (e.g., the last 90
days), the number of review points (e.g., the points received for
submitting reviews), the number of helpful votes, the number of
unhelpful votes, the helpfulness percentage (e.g., 100 (helpful
votes/(helpful+unhelpful votes)), and the number of current points
(e.g., the score based the user's review points, the number of
helpful votes, and the number of unhelpful votes). An extended
display 304 of monthly top reviewers is also provided in this
example. A selection menu 306 is provided via which the user can
specify the time period from which to display monthly top reviewers
(e.g., the last week, the last month, the last 6 months, the last
12 months, of all time, etc.). The types of information provided
via the user interface illustrated in FIG. 3 are substantially the
same as that provided in the leaderboard 212 and the monthly top
reviewers interface 202 illustrated in FIG. 2, although in other
embodiments, less, more, or additional information may be
provided.
FIG. 4 (including FIGS. 4A1-A3) illustrates an expanded view of the
most helpful first reviews interface 206 illustrated in FIG. 2,
including an extended list of helpful first reviews. An extended
display of monthly top reviewers is also provided. The types of
information provided in this view are substantially the same as
that provided in the leaderboard 212 and monthly top reviewers
interface 202 illustrated in FIG. 2, although in other embodiments,
less, more, or additional information may be provide.
FIG. 5 (including FIGS. 5A1-A3) provides an expanded view of the
user interface 228 illustrated in FIG. 2. The illustrated user
interface includes interfaces 518, 520, 522, 524, 526 corresponding
to interfaces 218, 220, 222, 224, 226 illustrated in FIG. 2. In
addition, user interface 502 provides a listing of first reviews
submitted by the user. A listing entry in this example includes a
product name and/or photograph, the number of points initially
assigned for submitting the first review (e.g., 10 points, or if a
bonus has been awarded, 20 points), the number of helpful votes
received for the first review, the number of unhelpful points
received for the first review, and the total score for the first
review (e.g., Total points=first review points+number of helpful
votes-number of unhelpful votes). The totals for each review
(optionally including the displayed and undisplayed totals for a
given time period or without a time limitation) are summed and
displayed. Optionally, the first review points list 502 can be
expanded to display the review points entries for all the user's
reviews (optionally limited to a time period, such as the previous
3 months or the time period that the user's current ranking is
based upon).
A "leaderboard position" user interface 504 is displayed which
shows the user's ranking and the users whose rankings are within a
specified range above and below the user (e.g., based on users that
earned review points in a specified period of time such as the last
90 days). In this example, the user interface 504 shows the two
users (having rankings of 21 and 22) ranked immediately above the
user accessing the interface (having a rank of 23) and the two
users ranked immediately below the user (having rankings of 24 and
25). A review user entry in this example includes the user name,
user ranking, a photographic or icon representing the user, the
number of recent points (e.g., earned in the last 90 days or other
time period), and the number of lifetime reviews. A control is
provided that when activated causes an expanded list of active
first reviewers to be displayed.
A rank history user interface 506 includes a graphical display
(e.g., an area, bar, and/or line graph) of the user's rank over a
period of time (e.g., the time period on which the current rank is
based, over the last 90 days, over the lifetime of the user's
rankings, etc.), and provides a numerical rank for a certain number
of recent months and the points earned during those months. A
control is provided that when activated causes an expanded list of
the rank history to be displayed (e.g., showing each month over the
lifetime of the user's rank history).
A recent progress user interface 508 provides information on the
user's recent progress relative to others and in absolute terms.
For example, the recent progress user interface 508 may report some
or all of the following (or other types of information): a change
in the user's ranking; when the user's ranking has changed by a
certain amount (e.g., an increase or decrease of at least 5
rankings); when the user's ranking has reached a specified ranking
level; when the user has reached or surpassed a certain number of
points; when the user has dropped below a specified number of
points; when the user has dropped a certain number of points; when
the user has submitted certain number(s) of first reviews; a change
in the user's title/badge; when the user has been surpassed in
ranking by another specified user or users; when the user has
received a certain number and/or percentage of helpful votes for
the user's first reviews; when a review score for a user review
falls to a certain level (e.g., 0 points); a determination that an
item purchased by the user does not have any reviews on the item
web page; that a product purchased by the user is eligible for
first review bonus points.
Referring to interface 508, in this example the user is notified of
certain other users the user passed in ranking and the associated
time of the passing in ranking. The interface 508 also identifies a
first review submitted by the user that is being provided with
special treatment. In this example, the user is notified that the
identified user's review is being highlighted to other users (e.g.,
via the most helpful review user interface 206, referred to as a
"Spotlight First Review")). In addition, the user is notified that
the user was awarded a certain level of badge (e.g., a silver
badge), and in congratulated on having submitted a notable number
of first reviews (e.g., 100 reviews).
FIG. 6 (including FIGS. 6A1-A2) illustrates an expanded view 602 of
interface 506, including an expanded graphical representation of
the of user's rank over a period of time and including numerical
representations of the user's performance on a monthly (or other
time period) basis. The numerical information can include the
user's rank during a specified time period (e.g., the rank achieved
at the end of the time period), the number of first reviews
submitted in the period of time (e.g., that month), the number of
review points, the number of helpful votes, the number of unhelpful
votes, the helpfulness percentage. FIG. 6 further includes
interfaces 604, 606, 608, and 610 similar to interfaces 220, 222,
224 and 226 illustrated in FIG. 2.
FIG. 7 (including FIGS. 7A1-A2) illustrates an expanded view 702 of
interface 502, and further includes interfaces 704, 706, 708, and
710 similar to interfaces 220, 222, 224 and 226 illustrated in FIG.
2.
FIG. 8 (including FIGS. 8A1-A3) illustrates an expanded view 802 of
first review candidates interface 220 (listing a plurality of first
review candidates with associated points for submitting a first
review), as well as interfaces 804, 806 similar to interfaces 222
and 226. FIG. 9 (including FIGS. 9A1-A3) illustrates an expanded
view 902 of the first reviews discussions/forums interface 228,
wherein an entry listing includes a subject line, a number of
posts, and when the last post was posted. Interfaces 904, 906
corresponding to interfaces 202, 206 are also provided.
FIG. 1 illustrates an example web site system that implements
services described herein, and illustrates the flow of information
between components of the system. As illustrated in FIG. 1, web
servers 90 retrieve catalog content for particular products from a
catalog service 94, which includes or accesses a data store, such
as repository 96, of item content. The item content may, for
example, include photos, reviews, price and availability data, and
other types of descriptive information about particular products
that are available to purchase, rent, download, review, post for
sale, etc. via the web site.
The web servers 90 further communicate with a first review game
system 110. The first review game system 110 further communicates
with the item content database 96 and a user data database 108. The
first review game system 110 provides services related to the first
review game described above. For example, the first review game
system 110 accesses the item content database 96 to identify items
and categories of items lacking in reviews (e.g., those categories
that have more than an average number of items without first
reviews, or that have above a certain percentage of items (e.g.,
more the 50%, 60%, 70%, 80% or 90%), including items that do not
have first reviews. In addition, the first review game system 110
determines when bonus points are to be offered for a first
review.
The first review game system 110 accesses the user database 108 to
identify users playing the first review game, determine which
products they have purchased and to recommend first review
candidates using such information. Optionally, the search engine
101 discussed below can be used to determine which products users
have purchased and recommend first review candidates using such
information and/or other information available to the search engine
service 101 to determine potential affinities.
The first review game system 110 further accesses the number of
first points a user has overall, on a monthly (and/or other time
period) basis, and how many points the user has received for a
given first review. Based on the user points and a specified
ranking period (e.g., the previous 90 days), the service 110
determines rankings, assigns badges, identifies most helpful
reviews, and provides other related services described herein.
The web servers 90 also optionally communicate with a tagging
service 98 that maintains a database 100 of user-specific tag data.
The tag data stored for each user may, for example, include a set
of tag-item ID pairs, optionally together with various other types
of data such as permission data and a creation timestamp. The
tagging service 98 may receive both read requests from the web
servers (e.g., when a user requests a page that displays personal
tag data), and update requests (e.g., when a user tags an item). In
an example embodiment, each tag is stored in association with the
corresponding user, meaning that if two or more users create
identical tags, these tags are treated as separate and distinct
from each other.
The web servers 90 also communicate with the search engine 101 that
allows users to search for information stored in the item content,
such as products that do not have any associated user/customer
reviews, and tag data repositories 96 and 100. As mentioned above,
the search engine 101 may include functionality (including a
keyword index) for users to search for items using their own
respective tag associations, and/or the tag associations of others.
A user interface for the search engine 101 is described above with
reference to FIG. 2.
The web servers 90 also access a recommendations service 102 which
generates item recommendations as described above. To request
recommendations for a particular user, the web servers 90 send a
list of one or more items (e.g., product IDs) to the
recommendations service 102 (optionally together with associated
item ratings), which responds with a list (e.g., of recommended
items and/or first review candidates).
In the illustrated embodiment, the recommendations service 102
accesses a Similar Items Table 104 which contains item-to-item
mappings used to generate the recommendations. The item-to-item
mappings represented in this table 104 may reflect purchase-based
similarities (e.g., items A and B are similar because a relatively
large portion of the users who purchased item A also bought item
B), item viewing based similarities (e.g., items A and B are
related because a significant portion of those who viewed item A
also viewed item B), tag-based similarities (e.g., items A and B
are similar because a relatively large portion of the users who
tagged item A assigned the same tag to item B), content-based
similarities, review-based similarities, and/or similarities based
on other criteria.
As similarly discussed above, the recommendations service 102 can
utilize reviews submitted by a user in order to generate product
recommendations. For example, if a user provided a review on an
item, other similar products or products purchased or viewed by
other users that purchased or viewed the reviewed product may be
recommended to the user. Products that the user provided first
reviews on may optionally be weighted more heavily in determining
an affinity and providing recommendations then products where the
user provided a review but did not provide the first review.
Further, the content of the review provided by the user (e.g., the
point rating and/or a text rating parsed by the system) may be used
in determining what products to recommend.
Optionally, if the user has in the past indicated a willingness to
provide first reviews (e.g., as determined by examining the number
of first reviews submitted by the user within a certain time period
and/or over the lifetime of the user's account), the service may
increase the prominence of un-reviewed products that the service
determines the user may otherwise have an affinity for in the
recommendations provided to the user. The prominence can be
provided by listing un-reviewed products first, by using a color
emphasis/highlighting, by providing a text or symbol indicating
that the product have not yet been reviewed, or otherwise.
The Similar Items Table 104 is generated or updated offline by the
Item-Association Mining Application 106, which may use the same or
similar data analysis methods to those described in U.S. Pat. No.
6,912,505, referenced above. In one embodiment, a similar items
list generated by the Item-Association Mining Application 106 and
stored in the Similar Items Table 104 consists of the N (e.g., 20)
items which, based on correlations between items, are deemed to be
the most closely related to the item for which associations are
sought. An item in the Similar Items Table 104 is stored together
with a commonality index ("CI") value which indicates the
relatedness of that item to the item for which associations are
sought. Other recommendation engines or services capable of
generating a set of recommended items given an input set of items
may be used as well.
The web servers 90 also access one or more additional repositories
of user data, logically represented in FIG. 1 by the repository
108. Because a group of individuals can share an account, a given
"user" may include multiple individuals (e.g., two family members
that share a computer). As illustrated by FIG. 1, the data stored
for each user may include one or more of the following types of
information (among other things) that can be used to generate
recommendations: (a) the user's purchase history, including dates
of purchase, (b) a history of items recently viewed by the user,
(c) the user's item review/ratings profile, if any, (d) the user's
first reviews, first review scores, helpful/not helpful review
votes, ranking and badges/titles, and (e) the current contents of
the user's personal shopping cart. Various other types of user
information, such as wish list/registry contents, email addresses,
shipping addresses, and clickstream histories, may additionally be
stored.
The various components of the web site system 86 may run, for
example, on one or more servers (not shown). In one embodiment, the
Similar Items Table 104 is be stored as a B-tree data structure to
permit efficient look-up, and may be replicated across multiple
machines to accommodate heavy loads. As will be appreciated, the
systems and processes may also be implemented using a
recommendations algorithm that does not make use of a similar items
table or other item-to-item similarity mappings.
Referring to FIG. 10, an example first review incentive process
that can be executed via the system illustrated in FIG. 1 will now
be described. Other embodiments can perform the process with fewer
or additional states, or the states may be performed in a different
order.
The process may be invoked in real-time in response to an online
action of the user or may be continuously running. For example, the
user may specifically access a first review landing page, such as
that illustrated in FIG. 2, which causes the process to update the
information to be displayed on the landing page. In addition or
instead, the process may be invoked in response to an online action
of the user that does not directly relate to playing the first
review game. For example, the process may be invoked in response to
a user having purchased an item that does not have a review and
inferring or estimating that the user has had time to use the
item.
Any of a variety of other methods can be used to initiate the first
review incentive process and to display or otherwise convey the
information discussed above with respect to the example user
interfaces. For example, the information can automatically be
generated periodically and sent to the user by e-mail, in which
case the e-mail listing may contain hyperlinks to the user
interfaces or to products needing reviews. Further, the information
can be generated in advance of any request or action by the user,
and cached until requested.
At state 1002, a user action is detected initiating the process.
The user's account is accessed (e.g., from user data database 108),
and the user's current first review points, current ranking,
current title/badge, and the number of the user's helpful/not
helpful first review votes are read. At state 1003, the current
scores and rankings of other users are accessed (e.g., from user
data database 108). At state 1004, the current ranking of the user
is calculated based on the user's first review score (e.g., which
is calculated based on the number of first reviews within a
specified time period, the number of helpful/unhelpful first review
votes, the number of bonus points, and/or other criteria) and that
of other users.
At state 1006, the system determines whether the user's ranking,
and/or title/badge entitlement has changed and whether any new or
recent notable first review-related events have occurred with
respect to the user.
At state 1008, the item content repository 96 is accessed and
products without reviews are identified. At state 1010, items that
have previously been purchased by the user that do not have reviews
and that an inference or indication that the user has used the item
are identified. At state 1012, the process updates and displays
first review game-related information to the user using the web
servers 90. For example, the information can include some or all of
the information discussed above with respect to the various user
interfaces (e.g., the user's current ranking, the user's ranking
trend, the user's current badge assignment, the user's position
relative to user's having somewhat higher and lower rankings, the
current most or more helpful reviews, first review candidates,
etc.).
Optionally, the user can request additional information (e.g., by
activating an appropriate control or menu selection), access an
item page in order to provide a first item review, or take other
action.
As discussed above, in addition to first reviews, the embodiments
and electronic interactive games/contests described herein may be
utilized with respect to encouraging other types of user behavior,
such as encouraging the submission of reviews generally (and not
just first reviews), the submission of tags, the submission of
votes as to the quality or helpfulness/unhelpfulness of reviews,
the submission of pictures of products, the submission of product
catalogs, product manuals, the submission of recommendations as to
similar alternate products, the submission of lists of related
items the user finds interesting or recommends (e.g., a list of
children books, a list of novels, a list of toys, a list of cooking
equipment, etc.), the submission of updates to product information,
the submission of recommendations of accessories, the submission of
useful software programs or widgets, the submission of information
or corrections to a wiki/collaborative website, participating in a
forum, etc. Users can be rewarded for behaving in the desired
manner with points, bonus points, rankings, titles, and/or badges
as similarly discussed above.
In certain instances, users may be incentivized to provide
information where a complete lack of such information has been
identified (e.g., via a search engine or otherwise) with respect to
the relevant data record/data store (e.g., in the item content data
repository 96, the tag data repository 100, the user data
repository 108, or other data repository). In certain instances,
the system may incentivize users to provide information where a
complete lack of such information has been identified with respect
to the relevant data record/data store and/or where there is such
information in the data store, but the amount, quantity, or quality
of such information is insufficient or less than a desirable amount
(e.g., as measure relative to a corresponding threshold setting
stored in system memory, wherein, for example, the threshold can be
a single submission of a given type information, three submissions,
five submissions, or other specified number of submissions) or
quality (e.g., as determined by the numbers and/or rations of
helpful/unhelpful votes, accurate/inaccurate votes, etc.).
For example, with respect to a product manual, in certain
situations a single submission of a manual for a given item may be
sufficient. Thus, in an example embodiment, points are awarded for
the first person to submit the manual and not for subsequent
submissions. Optionally, bonus points can be awarded for providing
product manuals for certain types of products (e.g., as similarly
discussed above with respect to first reviews). Optionally, users
may be able to vote on the legibility or illegibility of a
submitted manual, and such votes can be used to modify the score.
For example, a user can be assigned a first number of points (e.g.,
40 points) for submitting the first manual for a product, an
additional number of points (or fraction of a point) for each
"legible and complete" vote received from other users regarding the
legibility and completeness of the manual, and a negative number of
points (or fraction of a point) for each "illegible or incomplete"
vote received. The votes may be provided via an item Web page that
displays the manual and voting buttons (e.g., legible, illegible,
complete, incomplete, etc.). Rankings, titles, badges, and/or
information can be provided as similarly described above with
respect to first reviews. Optionally, a search engine may be used
to locate items without product manuals. Optionally, a
recommendation engine is provided that identifies a user's
potential affinity for an item lacking a product manual and
suggests that the user provide a product manual for the identified
item. Optionally, the system identifies users that have purchased
an item lacking a product manual and asks some or all of those
users (via email, an SMS/MMS message, a Web page, or otherwise) to
provide a product catalog (e.g., via an upload of a digital version
of the manual, as an email attachment, via fax, mail, or
otherwise).
By way of further example, with respect to an item tag, in certain
situations a certain minimum number of tags may desirable. Thus, in
an example embodiment, points are awarded for the first person to
submit a tag for an item and for subsequent tag submissions for the
item. Optionally, a user may be awarded a first number of points
for the being the first user to submit a tag for an item, and
subsequent users submitting tags for the item are awarded a second
number of points (e.g., equal to the first number of points, or
less than the first number of points). Optionally, bonus points can
be awarded for providing tags for certain types of products (e.g.,
as similarly discussed above with respect to first reviews), and
not for others. Optionally, users may be able to vote on the
helpfulness or accuracy of a submitted tag, and such votes can be
used to modify the score for a tag submission. For example, a user
can be assigned a first number of points (e.g., 40 points) for
submitting an item tag, an additional number of points (or fraction
of a point) for each helpful or accurate vote received from other
users regarding the helpfulness or accuracy of the tag, and a
negative number of points (or fraction of a point) for each
unhelpful or inaccurate vote received. The votes may be provided
via an item Web page that displays the tags and voting buttons
(e.g., helpful, unhelpful, accurate, inaccurate, etc.). Rankings,
titles, badges, and/or information can be provided as similarly
described above with respect to first reviews. Optionally, a search
engine may be used to locate items that have less then a specified
threshold of tags (e.g., wherein, for example, the threshold can be
a single tag, two tags, five tags, or other specified number of
tags). Optionally, a recommendation engine is provided that
identifies a user's potential affinity for an item lacking
sufficient tags and suggests that the user provide a tag for the
identified item. Optionally, the system identifies users that have
purchased an item lacking sufficient tags and asks some or all of
those users (via email, an SMS/MMS message, a Web page, or
otherwise) to provide a tag.
By way of still further example, with respect to recommendations as
to similar alternate products, in certain situations a certain
minimum number of such recommendations may desirable. Thus, in an
example embodiment, points are awarded for the first person to
submit a recommendation as to similar alternate products for an
item, and for subsequent alternate product recommendations for the
item. Optionally, a user may be awarded a first number of points
for the being the first user to submit an alternate product
recommendation for an item, and subsequent users submitting
alternate product recommendations for the item are awarded a second
number of points (e.g., equal to the first number of points, or
less than the first number of points). Optionally, bonus points can
be awarded for providing alternate product recommendations for
certain types of products (e.g., as similarly discussed above with
respect to first reviews). Optionally, users may be able to vote on
the helpfulness of a submitted alternate product recommendation,
and such votes can be used to modify the score for a alternate
product recommendation submission. For example, a user can be
assigned a first number of points (e.g., 40 points) for submitting
an alternate product recommendation, an additional number of points
(or fraction of a point) for each helpful vote received from other
users regarding the helpfulness of the alternate product
recommendation, and a negative number of points (or fraction of a
point) for each unhelpful vote received. The votes may be provided
via an item Web page that displays the alternate product
recommendation and voting buttons (e.g., helpful, unhelpful, etc.).
Rankings, titles, badges, and/or information can be provided as
similarly described above with respect to first reviews.
Optionally, a search engine may be used to locate items that have
less then a specified threshold of alternate item recommendations.
Optionally, a recommendation engine is provided that identifies a
user's potential affinity for an item lacking sufficient alternate
product recommendations and suggests that the user provide an
alternate product recommendation for the identified item.
Optionally, the system identifies users that have purchased an item
lacking alternate product recommendations and asks some or all of
those users (via email, an SMS/MMS message, a Web page, or
otherwise) to provide an alternate product recommendation.
By way of yet further example, with respect to lists of related
products, in certain situations a certain minimum number of lists
for a given product area may desirable. Thus, in an example
embodiment, points are awarded each list containing at least a
specified number (e.g., 1, 2, 3, or other specified number) of
items (e.g., related items). Optionally, bonus points can be
awarded for providing product lists for certain types of products
(e.g., as similarly discussed above with respect to first reviews).
Optionally, users may be able to vote on the helpfulness/usefulness
of a submitted list, and such votes can be used to modify the score
for a alternate product recommendation submission. For example, a
user can be assigned a first number of points for submitting a
related product lists, an additional number of points (or fraction
of a point) for each helpful vote received from other users
regarding the helpfulness of the alternate product recommendation,
and a negative number of points (or fraction of a point) for each
unhelpful vote received. The votes may be provided via an item Web
page that displays the alternate product recommendation and voting
buttons (e.g., helpful, unhelpful, etc.). Optionally, the number of
points awarded for a list submission can related to the number of
items in the list (e.g., 10 points for each additional item after a
minimum initial number of items). Rankings, titles, badges, and/or
information can be provided as similarly described above with
respect to first reviews. Optionally, a search engine may be used
to locate product areas that have less then a specified threshold
of item lists and/or having less than a certain percentage of items
on a list. Optionally, a recommendation engine is provided that
identifies a user's potential affinity for a product area lacking
sufficient item lists and suggests that the user provide an item
list for the identified product area. Optionally, the system
identifies users that have purchased an item from a product area
lacking sufficient items lists (e.g., relative to a specified
threshold) and asks some or all of those users (via email, an
SMS/MMS message, a Web page, or otherwise) to provide item
lists.
The item lists, alternate product recommendations, tags, reviews,
catalogs, pictures, manuals, recommend accessories, guides,
articles, other feedback, and/or other submissions are optionally
displayed on, accessed from (e.g., via a hyperlink), or submitted
via one or more corresponding relevant item detail pages. For
example, the user interface illustrated in FIG. 12 (including FIGS.
12A1-6) provides links or fields for submitting pictures 1202,
updates for product information 204, feedback on images 206, tag
submissions 208, user reviews/ratings 1210, 1212, discussion
content (e.g., questions, opinions, information, etc.) for an
online discussion forum, submission of item-related or other
articles 1216, submission of item lists 1218, guides 1220 (for
using an item, selecting an item, providing recommendations on a
grouping of items, proving information on a grouping of related
items, performing a process, etc.), or providing other feedback
1222.
Referring to FIG. 11, an example process for incentivizing users to
submit information or to take a desired action via an interactive
game that can be executed via the system illustrated in FIG. 1 will
now be described. Other embodiments can perform the process with
fewer or additional states, or the states may be performed in a
different order.
As similarly discussed above with respect to FIG. 10, the process
may be invoked in real-time in response to an online action of the
user or may be continuously running. For example, the user may
specifically access one or more game user interfaces. In addition
or instead, the process may be invoked in response to an online
action of the user that does not directly relate to accessing a
game user interface. For example, the process may be invoked in
response to a user having purchased an item whose catalog listing
is missing or has an insufficient amount of certain types of
information (e.g., a manual, sufficient numbers of tags,
photographs, alternate item suggestions, helpful/unhelpful review
votes, etc.).
Any of a variety of other methods can be used to initiate the
illustrated process and to display or otherwise convey the
information discussed above with respect to the example user
interfaces. For example, the information can automatically be
generated periodically and sent to the user by e-mail, in which
case the e-mail listing may contain hyperlinks to the user
interfaces or to products needing the identified information.
Further, the information can be generated in advance of any request
or action by the user, and cached until requested.
Referring again to FIG. 11, at state 1102, a user action is
detected by the system, initiating the process. The user's account
is accessed (e.g., from user data database 108), and the user's
current game points, current ranking, current title/badge, and
other information related to the user's score (e.g., the number of
the user's helpful/not helpful votes for reviews, tags,
photographs, accuracy votes, legibility votes, etc.; the number of
legible/illegible votes for manuals, etc.) are read. At state 1103,
the current scores and rankings of other users are accessed (e.g.,
from user data database 108). At state 1104, the current ranking of
the user is calculated based on the user's current game score
(e.g., which is calculated based on the number of the user's
submissions within a specified time period, the number of votes
from other users related to the submission, the number of bonus
points, and/or other criteria) and that of other users.
At state 1106, the system determines whether the user's ranking,
and/or title/badge entitlement has changed and whether any new or
recent notable game-related events have occurred with respect to
the user.
At state 1108, the item content repository 96 is accessed and
products lacking the desired information are identified. At state
1110, items that have previously been purchased by the user lacking
the desired information are identified. At state 1112, the process
updates and displays game-related information to the user using the
web servers 90. For example, the information can include some or
all of the information discussed above with respect to the various
user interfaces (e.g., the user's current ranking, the user's
ranking trend, the user's current badge assignment, the user's
position relative to user's having somewhat higher and lower
rankings, candidates that need information submissions, etc.). The
system optionally searches for and identifies candidates that need
certain types of submissions (e.g., Reviews, tags, identification
of similar or alternate products, item lists, pictures, product
catalogs, product manuals, guides, articles, online discussion
participation, updated product information, and/or accessory
recommendations, etc.).
Thus, as described herein, certain example embodiments search for
and identify items in an interactive electronic catalogue that do
not have an associated item review or other types of desired
information. Such items are identified selectively to users and
such users are prompted or otherwise encouraged to submit a first
review or other desired information. Certain example embodiments
utilize a game format, where players are assigned points for
submitting first reviews or other desired information and
optionally further assignment points if other users find the first
review or other desired information helpful, accurate, and/or
legible.
Although certain example embodiments are described herein, other
embodiments that are apparent to those of ordinary skill in the
art, including embodiments which do not provide all of the benefits
and features set forth herein, may also be utilized.
* * * * *
References