U.S. patent application number 13/534579 was filed with the patent office on 2013-10-10 for annotations based on hierarchical categories and groups.
This patent application is currently assigned to MICROSOFT CORPORATION. The applicant listed for this patent is Syed Muhammad Mujahid Hasan, Jiayuan Huang, Sadi Khan, Yi Mao, Qiang Wu. Invention is credited to Syed Muhammad Mujahid Hasan, Jiayuan Huang, Sadi Khan, Yi Mao, Qiang Wu.
Application Number | 20130268513 13/534579 |
Document ID | / |
Family ID | 48916757 |
Filed Date | 2013-10-10 |
United States Patent
Application |
20130268513 |
Kind Code |
A1 |
Wu; Qiang ; et al. |
October 10, 2013 |
ANNOTATIONS BASED ON HIERARCHICAL CATEGORIES AND GROUPS
Abstract
Systems and methods for recommending entities to a user are
presented. In at least one embodiment, a user is identified as
belonging to one or more groups of users, or to a hierarchy of
groups. A category of entities, preferred by at least of the
hierarchy of identified groups, is identified. The category of
entities preferred by the at least one identified group corresponds
to the category of a user-preferred entity preferred by the user.
An entity from the category of entities is selected; the selected
entity is not the user-preferred entity. The selected entity is
provided to the user as a recommended entity to the user.
Inventors: |
Wu; Qiang; (Sammamish,
WA) ; Hasan; Syed Muhammad Mujahid; (Bellevue,
WA) ; Huang; Jiayuan; (Sammamish, WA) ; Khan;
Sadi; (Redmond, WA) ; Mao; Yi; (Redmond,
WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Wu; Qiang
Hasan; Syed Muhammad Mujahid
Huang; Jiayuan
Khan; Sadi
Mao; Yi |
Sammamish
Bellevue
Sammamish
Redmond
Redmond |
WA
WA
WA
WA
WA |
US
US
US
US
US |
|
|
Assignee: |
MICROSOFT CORPORATION
Redmond
WA
|
Family ID: |
48916757 |
Appl. No.: |
13/534579 |
Filed: |
June 27, 2012 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61621566 |
Apr 8, 2012 |
|
|
|
Current U.S.
Class: |
707/722 ;
707/E17.108 |
Current CPC
Class: |
G06F 16/9535 20190101;
G06F 16/00 20190101; G06F 7/00 20130101; G06Q 30/02 20130101 |
Class at
Publication: |
707/722 ;
707/E17.108 |
International
Class: |
G06F 17/30 20060101
G06F017/30 |
Claims
1. A computer-implemented method for providing a recommended entity
to a user, the method comprising: identifying a group to which the
user belongs; identifying a category of entities preferred by the
identified group, wherein the category of entities preferred by the
identified group corresponds to the category of a user-preferred
entity preferred by the user; selecting an entity of the category
of entities having the greatest affinity value between the group
and the selected entity, wherein the selected entity is not the
user-preferred entity; and providing the selected entity as a
recommended entity to the user and associating an annotation with
the selected entity, the annotation including an annotation
relationship identifying that the identified group has an affinity
to the selected entity.
2. The method of claim 1 further comprising: obtaining a plurality
of search results responsive to a search query received from the
user; and generating a search results page, the search results page
including a subset of the obtained plurality of search results and
further including the recommended entity and the associated
annotation; wherein providing the selected entity as a recommended
entity to the user comprises providing the generated search results
page to the user.
3. The method of claim 2, wherein at least one search result of the
subset of search results included in the generated search results
page corresponds to the selected entity, and wherein the annotation
associated with then entity is placed proximately to the at least
one search result in the generated search results page.
4. The method of claim 1, wherein the annotation relationship
describes a basis for which the recommended entity is relevant to
the identified group.
5. The method of claim 1, wherein the annotation relationship
describes a positive affinity between the identified group and the
recommended entity.
6. The method of claim 1, wherein the annotation relationship
describes a negative affinity between the identified group and the
recommended entity.
7. The method of claim 1, wherein the affinity values between the
group and the entities of the category of entities are determined
according to one or more probability density functions.
8. The method of claim 1, wherein the search query is directed to
items of the category of entities.
9. A computer-readable medium bearing computer-executable
instructions which, when executed on a computing device having at
least a processor and a memory, carry out a method for providing
recommendations of entities to a user, the method comprising:
obtaining a plurality of search results responsive to receiving a
search query from a user; determining a group from a hierarchy of
groups to which the user belongs; identifying a plurality of
entities of a category that are preferred by the group; for each of
the plurality of entities, determining an affinity value between
the group and the entity; selecting the entity having the greatest
affinity value to the group; generating a search results page
responsive to the search query, the search results page including:
a subset of the obtained search results; the selected entity
presented as a recommended entity to the user; and an annotation,
the annotation comprising an annotation relationship identifying
that the identified group has an affinity to the selected entity;
and providing the generated search results page to the user
responsive to the search query.
10. The computer-readable medium of claim 9, wherein at least one
search result of the subset of search results included in the
generated search results page corresponds to the selected entity,
and wherein the annotation associated with then entity is placed
proximately to the at least one search result.
11. The computer-readable medium of claim 9, wherein the annotation
relationship describes a basis for which the recommended entity is
relevant to the identified group.
12. The computer-readable medium of claim 9, wherein the annotation
relationship describes a positive affinity between the identified
group and the recommended entity.
13. The computer-readable medium of claim 9, wherein the annotation
relationship describes a negative affinity between the identified
group and the recommended entity.
14. The computer-readable medium of claim 9, wherein the affinity
values between the group and the entities of the category of
entities are determined according to one or more probability
density functions.
15. The computer-readable medium of claim 9, wherein the search
query is directed to items of the category of entities.
16. A system for recommending an entity to a user, wherein the
recommendation system is implemented on a computer system
comprising at least a processor executing instructions stored in a
memory, and further comprising: an annotation store, the annotation
store storing relationship information between a plurality of
groups of users and a plurality of entities; and a recommendation
component that, responsive to an entity recommendation for a user:
identifies a group of users to which the user belongs; identifies a
category of entities preferred by the identified group, wherein the
category of entities preferred by the identified group corresponds
to the category of a user-preferred entity preferred by the user;
and selects an entity of the category of entities having the
greatest affinity value between the group and the selected entity,
wherein the selected entity is not the user-preferred entity;
wherein the system provides the selected entity as a recommended
entity to the user.
17. The system of claim 16 further comprising an annotation
component that associates an annotation with the selected entity,
and wherein the system provides the selected entity as a
recommended entity to the user with the associated annotation.
18. The system of claim 17 further comprising a search results
retrieval component that obtains a plurality of search results
responsive to the system receiving a search query from the user;
and a search results page generator that generates a search results
page responsive to the system receiving the search query from the
user, the search results page including a subset of the obtained
plurality of search results and further including the recommended
entity and the associated annotation.
19. The system of claim 18, wherein at least one search result of
the subset of search results included in the generated search
results page corresponds to the selected entity, and wherein the
associated annotation is placed proximately to the at least one
search result in the generated search results page.
20. The system of claim 19, wherein the affinity values between the
group and the entities of the category of entities are determined
according to one or more probability density functions.
Description
BACKGROUND
[0001] More and more, people are interacting with and through
online services, including but not limited to social networking
sites, search engines, online shopping sites, libraries,
entertainment/gaming sites, music and video streaming sites, and
the like. All of these online services work at a basic level of
functionality with each new (or unidentified) user, yet nearly all
of these online services work "better" when a user is identified
and has provided information about himself/herself to the service.
With specific information about the user, these online services are
able to "personalize" their services--i.e., provide services
specifically tailored and targeted to the user. However, when an
online service makes a personalized recommendation to a user, the
quality of the personalized recommendation has a direct correlation
to user engagement and user satisfaction with that
recommendation.
SUMMARY
[0002] The following presents a simplified summary in order to
provide a basic understanding of various embodiments described
herein. This summary is not an extensive overview, and it is not
intended to identify key and/or critical elements or to delineate
the scope thereof. The sole purpose of this summary is to present
some concepts in a simplified form as a prelude to the more
detailed description that follows.
[0003] According to aspects of the disclosed subject matter, a
method for recommending entities to a user is presented. In at
least one embodiment, a user is identified as belonging to one or
more groups of users, or to a hierarchy of groups. A category of
entities, preferred by at least of the hierarchy of identified
groups, is identified. The category of entities preferred by the at
least one identified group corresponds to the category of a
user-preferred entity preferred by the user. An entity from the
category of entities is selected; the selected entity is not the
user-preferred entity. The selected entity is provided to the user
as a recommended entity to the user.
[0004] According to additional aspects of the disclosed subject
matter, a method embodied on a computer-readable medium bearing
computer-executable instructions is presented. The method is
configured to provide recommendations of entities to a user. In
response to receiving a search query from a user (or other
search-triggering event), a plurality of search results is
obtained. A group from a hierarchy of groups, to which the user
belongs, is identified and a corresponding category of entities
that is preferred by the group is identified. An entity is selected
of the category of entities. A search results page is generated in
response to the search query, the search results page including a
subset of the obtained search results, the selected entity and an
annotation associated with the selected entity, the annotation
identifying that the identified group has an affinity to the
selected entity. The search results page is then provided to the
user in response to receiving the search query.
BRIEF DESCRIPTION OF THE DRAWINGS
[0005] The foregoing aspects and many of the attendant advantages
of the disclosed subject matter will become more readily
appreciated as they are better understood by reference to the
following description when taken in conjunction with the following
drawings, wherein:
[0006] FIG. 1 is a diagram illustrating an exemplary networked
environment suitable for implementing aspects of the disclosed
subject matter;
[0007] FIG. 2 is a pictorial diagram of an exemplary browser view
showing annotated recommended entities in accordance with aspects
of the disclosed subject matter;
[0008] FIG. 3 is a flow diagram illustrating an exemplary routine
suitable for annotating recommended entities on a search results
page;
[0009] FIG. 4 is a flow diagram illustrating an exemplary
sub-routine for determining annotations between a set of entities
and a user;
[0010] FIG. 5 is a block diagram visually illustrating the
annotation of an entity;
[0011] FIGS. 6A and 6B show pictorial diagrams for illustrating
annotating entities with regard to an image;
[0012] FIG. 7 is a flow diagram illustrating an exemplary routine
suitable for annotating entities within an image;
[0013] FIG. 8 is a block diagram illustrating exemplary components
of a search engine configured to respond to search queries with a
search results page including annotated entities;
[0014] FIG. 9 is a block diagram illustrating exemplary components
of an annotation system suitable for annotating a plurality of
entities according to aspects of the disclosed subject matter;
[0015] FIG. 10 is a pictorial diagram of illustrative user signals
used in generating user information for one of many users in the
annotation store from which annotations are derived;
[0016] FIG. 11 is a pictorial diagram illustrating an exemplary
user interface enabling a user to exercise control of the signals
used in generating that portion of the annotation store pertaining
to the user;
[0017] FIG. 12 is a pictorial diagram for illustrating the
promotion of a user to a group, and of an entity to a category in
determining annotations for the user; and
[0018] FIG. 13 is a flow diagram illustrating an exemplary routine
for determining a recommendation of an entity of a category to a
specific user.
DETAILED DESCRIPTION
[0019] For purposed of clarity, the use of the term "exemplary" in
this document should be interpreted as serving as an illustration
or example of something, and it should not be interpreted as an
ideal and/or leading illustration of that thing.
[0020] As used in this document, the term "entity" refers to a
concept, a person, or a thing. An entity is a "something" which can
be annotated. For example, a user will submit a search query
including one or more query terms, and these query terms relate to
one or more entities--i.e., the intent of the search query. For
example, a search query "Paris, France" relates to a single entity,
the capital city in France. Search queries may specify multiple
entities. For example, the search query "Paris France Eiffel Tower"
may be reduced to two entities: (1) the capital of France and (2)
the "Eiffel Tower." A "recommended entity" refers to an entity that
has been recommended (typically through personalization) to the
user. In the context of a search engine, a recommended entity may
include, but is not limited to, a search result (that references
suggested content), a suggested search query, a product, an
advertisement, and the like. A recommended entity may also comprise
a group (or set) of entities and/or a category or subcategory of a
product (e.g., "shirts" or "yellow" shirts). For example, a video
streaming service may recommend a collection of videos within a
genre to the user, the collection being a single recommended
entity.
[0021] The term, "annotation," as used throughout this document,
refers to a set of relationships between an entity and a user,
i.e., the rationale or basis as to how and/or why an entity relates
or is relevant to the user. An annotation is comprised of one or
more annotation relationships, each relationship describing a
single basis for which the user and entity are related. While
annotation relationships typically describe a positive affinity
between the user and the entity, an annotation relationship may
describe a negative affinity between the user and the entity.
"Annotating an entity" identifying and associating an annotation
with an entity. To visually indicate that an entity has been
annotated, an indicator (typically a user-actionable indicator,
such as an icon or a hyperlink) is placed in proximity to the
entity through which the user can view/access the annotation for
that entity. As an alternative to user-actionable indicators, the
entire textual annotation may be placed next to the annotated
entity.
[0022] According to aspects of the disclosed subject matter, an
annotation system is present that is configured to annotate one or
more entities with regard to a particular user. The annotation
system provides an annotation service in which the annotation
service receives a set of one or more entities along with the
identity of a user and provides annotations from each of the one or
more entities.
[0023] Advantageously, the annotation system identifies or
determines the annotation for an entity independent of their
selection or recommendation by another service. In this sense,
then, the annotation system is a pluggable system, capable of
working with any number of services. This is, in part, accomplished
by the fact that the annotation system maintains its own annotation
store and annotation analysis engine. With its annotation store and
analysis engine, the annotation service issues an annotation
independent of the basis by which a cooperating system identifies
or recommends the set of entities. For example, a video streaming
service may identify a set of videos that it (the video streaming
service) wishes to recommend to the user. In annotating the set of
videos (either as a group of entities or individually) the
annotation system relies upon the information in the annotation
store and analysis engine to identify and/or determine the
corresponding annotations.
[0024] The annotation store includes information (attributes,
categories, preferences, relationships, metadata, etc.) about
entities, users, and relationships between the two. In conjunction
with the information in the annotation store, the annotation
service identifies and/or determines a set of annotation
relationships between a given entity and user. According to one
embodiment annotation relationships between an entity and a user
are determined according to probability density functions that
predict the likelihood of relevance between the user and the
entity.
[0025] Clearly, one of the advantages of annotating entities
independent of the service that identifies them for annotation is
that the cooperative service does not need to gather, ingest, and
maintain the robust information that the independent annotation
system keeps and uses in annotating entities. In the example above
of the video streaming service, the video streaming service may not
have access to the identified user's browsing history, the user's
purchase history of videos, the user's social network, or any other
number of interesting details regarding the user. However,
information gathered from these and other sources may be the best
rationale of one or more annotation relationships between the
entity and the user. Thus, the video streaming service can focus
its efforts on providing video streaming services.
[0026] While the annotation system may be implemented as a
cooperative, stand-alone system, in accordance with aspects of the
disclosed subject matter the annotation system may be incorporated
within another service. For example, a search engine may be
configured to comprise an annotation system such as will be
discussed in regard to FIG. 8. However, even when incorporated into
another system, the annotation system determines the annotations
for a set of entities independent of the process by which the
entities were identified or recommended.
[0027] Much of the following discussion is made in regard to
responding to a search query with from a computer user. While this
is one embodiment in which aspects of the disclosed subject matter
may operate, and it should be appreciated that the disclosed
subject matter is not so limited. Indeed, there are various
conditions that may trigger or initiate a search by a search engine
or service. User-initiated search queries are search events.
Proximity-based apps, such as an app on the user's mobile device
for finding restaurants in the device's immediate vicinity, will
trigger a search event that obtains search results for the
corresponding computer/device user. Recognition services may also
cause a search event. For example, a recognition app running on a
user's mobile device may initiate a search event to provide
information regarding a location or person as the user takes a
picture with the mobile device. Accordingly, while much of the
discussion that follows is made in regard to responding to a search
query from a computer user, it is just one example of a
search-triggering event ("search event") and should not be viewed
as limiting upon the disclosed subject matter.
[0028] Turning now to FIG. 1, this figure shows is a diagram
illustrating an exemplary networked environment 100 suitable for
implementing aspects of the disclosed subject matter. The
illustrative environment 100 includes one or more user computers,
such as user computers 102-106, connected to a network 108, such as
the Internet, a wide area network or WAN, and the like. Also
connected to the network 108 is a search engine 110. Those skilled
in the art will appreciate that a search engine 110 corresponds to
an online service hosted on one or more computers on, or computing
systems distributed throughout, the network 108. The search engine
110 receives and responds to search queries submitted over the
network 108 from various users, such as the users connected to user
computers 102-106. In response to receiving a search query, the
search engine 110 obtains search results information related and/or
relevant to the received search query (as defined by the terms of
search query.) The search results information includes search
results, i.e., references (typically in the form of hyperlinks) to
relevant/related content available from various target sites (such
as target sites 112-114) on the network 108. The search results
information may also include other information such as related
and/or recommended alternative search queries, data and facts
regarding the subject matter of the search query, products and/or
services related/relevant to the search query, advertisements, and
the like. The search engine 110 generates one or more search
results pages responsive to the search query based on the search
results information. According to various embodiments of the
disclosed subject matter, the search engine 110 includes annotated
entities with the generated search results pages.
[0029] Also shown in the exemplary networked environment 100 is an
annotation system 116 for annotating entities, including
personalized entities from a search engine 110. While this
annotation system 116 is shown as being a separate service/entity
in the networked environment 100, it should be appreciated that
this is illustrative only and should not be construed as limiting
upon the disclosed subject matter. The process of the annotation
system in annotating an entity is described in greater detail
below.
[0030] As those skilled in the art will appreciate, target sites,
such as target sites 112-114, host content that is available and/or
accessible to users (via user computers) over the network 108. The
search engine 110 will be aware of at least some of the content
hosted on the many target sites located throughout the network 108,
and will store information regarding the hosted content of the
target sites in a content index (620 of FIG. 7). The search engine
110 draws from the content index when obtaining search results
information in response to receiving a search query. As shown in
FIG. 1, the target sites include, by way of illustration, a news
organization 112, and a shopping site 114. Of course, those skilled
in the art will appreciate that any number and type of target sites
may be connected to the network 108. Moreover, as is known in the
art, some search engines are aware of millions of target sites and
the content that is hosted by those target sites.
[0031] Suitable user computers for operating within the
illustrative environment 100 include any number of computing
devices that can communicate with the search engine 110 or target
sites 112-114 over the network 108. In regard to the search engine
110, communication between the user computers 102-106 and the
search engine 110 include both submitting search queries and
receiving a response in the form of one or more search results
pages from the search engine 110. User computers 102-106 may
communicate with the network 108 via wired or wireless
communication connections. These user computers 102-106 may
comprise, but are not limited to: laptop computers such as user
computer 102; desktop computers such as user computer 104; mobile
phone devices such as user computer 106; tablet computers (not
shown); on-board computing systems such as those found in vehicles
(not shown); mini- and/or main-frame computers (not shown); and the
like.
[0032] Turning now to FIG. 2, this figure is a pictorial diagram of
an exemplary browser view 200 showing annotated recommended
entities within the browser view in accordance with aspects of the
disclosed subject matter. In this example, a user has submitted the
search query, "Owen Roe Sharecroppers 2008, and browser view 200
illustrates a portion of a generated search results page that
includes search results information, including recommended
entities. More particularly, the recommended entities include
recommended queries 202-204, a recommended search result 206, and a
recommended product group 208. As can be seen, each of these
recommended entities is annotated with an indicator through which
the user can view the rationale for which each entity may have been
recommended to the user. For example, a user-actionable icon 210 is
placed next to entity group 208 indicating that the entity has been
annotated. Annotation view 212 illustrates exemplary annotation
relationships between the user who submitted the query and the
recommended entity (i.e., the group of wines.) In regard to the
annotation information, according to aspects of the disclosed
subject matter, for each annotation there may be one or more
annotation relationships identified between the user and the
entity.
[0033] To better understand the process by which entities within a
search results page are annotated, reference is now made to FIG. 3.
FIG. 3 is a flow diagram illustrating an exemplary routine 300
suitable for annotating recommended entities on a search results
page. Beginning at block 302, the search engine 110 receives a
search query from a user. At block 304, the search engine 110
obtains search results information responsive to the search query.
The search results information includes search results relevant to
the search query, i.e., references (typically hyperlinks) to
content stored throughout the network 108. Further, however, the
search results information will typically include, without
limitation, data related to the search query, images, videos,
alternative related search queries, search histories, recommended
search queries (such as recommended search queries 202-204),
related products, and the like.
[0034] At block 306, an annotation system associated with the
search engine 110 (or incorporated as a part of the search engine)
obtains a set of recommended entities from the search results
information that was obtained in response to the search query from
the user. Once a set of recommended entities is identified, at
block 308 those recommended entities are annotated, i.e.,
annotation information for each recommended entity is obtained.
Obtaining annotation information for the recommended entities is
described in regard to FIG. 4.
[0035] Turning, then, to FIG. 4, this figure is a flow diagram
illustrating an exemplary sub-routine 400 for determining
annotations between a set of entities and a user. Beginning at
block 402, a looping construct is begun to iterate through the set
of entities that are to be annotated, such that the steps of blocks
404-408 are repeated for each entity. At block 404, the annotation
system determines a set of annotation relationships according to
information in a relationship store between the user and the
current entity being processed. These relationships are often of a
positive nature but may also be a negative relationship (i.e.,
"product A is recommended because you don't like product B" or
"your friend Joe did not like Restaurant X so we do not recommend
it to you"). According to various aspects of the disclosed subject
matter, the annotation system may determine/identify any number of
relationships between the user and the entity, even more than can
be reasonably displayed in any one annotation. In limiting the
number of annotation relationships that can be used an effort is
made to select those relationships that are the strongest,
presumptively the most meaningful to the user. Thus, at block 406
an affinity value is assigned to each of the annotation
relationships identified between the user and the entity. At block
408, the top n annotation relationships, as determined according to
the affinity value between user and the entity, are selected. In
this, n is a configurable number as determined according to
implementation details, and may be a static or dynamic value.
[0036] At block 410, if there are any remaining entities in the set
of entities to be annotated the subroutine 400 selects the next
entity and returns to block 402 to process that entity.
Alternatively, if all of the entities have been annotated, the
subroutine 400 proceeds to block 412 where the annotations
corresponding to the set of entities is returned.
[0037] Returning again to FIG. 3, once the recommended entities are
annotated (i.e., are associated with an annotation describing one
or more relationships between the user and the entity), the routine
300 proceeds to block 310. At block 310, the search engine 110
generates a search results page. According to aspects of the
disclosed subject matter, as part of generating the search results
page, the annotated entities are so indicated by an annotation
indicator. In general, the annotation indicator is a
user-actionable indication that is configured to display the
annotation to the user upon its activation. User-actionable icon
210 (FIG. 2) is a non-limiting example of an annotation indicator.
After generating the search results page, at block 312 the
generated page is returned to the user (in response to the search
query) and the routine 300 terminates.
[0038] Regarding the routines of FIGS. 3 and 4, (as well as other
routines described below) it should be appreciated that while they
are expressed with discrete steps, these steps should be viewed as
being logical in nature and may or may not correspond to any
actual, discrete steps. Nor should the order that these steps are
presented be construed as the only order in which the various steps
may be carried out. Those skilled in the art will appreciate that
logical steps may be combined together or be comprised of multiple
steps. Further, while novel aspects of the disclosed subject matter
are expressed in routines or methods, this functionality may also
be embodied in computer-readable media. As those skilled in the art
will appreciate, computer-readable media can host
computer-executable instructions for later retrieval and execution.
When executed on a computing device, the computer-executable
instructions carry out various steps or methods. Examples of
computer-readable media include, but are not limited to: optical
storage media such as digital video discs (DVDs) and compact discs
(CDs); magnetic storage media including hard disk drives, floppy
disks, magnetic tape, and the like; transitory and non-transitory
memory such as random access memory (RAM), read-only memory (ROM),
memory cards, thumb drives, and the like; cloud storage (i.e., an
online storage service); and the like. For purposes of this
document, however, computer-readable media expressly excludes
carrier waves and propagated signals.
[0039] In regard to the process by which the annotation system
identifies annotations for entities, FIG. 5 is a block diagram
visually illustrating the annotation of an entity. As shown in FIG.
5, the top portion of the diagram shows an annotation system 500
that accepts a user identifier 502 as one input to the annotation
system and an entity identifier 504 ("Entity.sub.A ID") as a second
input. It should be appreciated that while a single entity (via its
identifier) is input into the annotation system 500, this is
illustrative only. As discussed above, in an alternative embodiment
a group of entities may be submitted to the annotation system 500
for annotation of the group.
[0040] Also shown is an annotation store 506 from which the
annotation system 500 obtains information regarding the
relationships between the entity (as represented by entity
identifier 504) and the user (as represented by the user identifier
502). The annotation system 500 obtains the relationship
information by way of an analysis engine 514, which analyzes the
information from the annotation store (as well as other sources of
information) and determines/identifies the various annotation
relationships between the user and an entity. The output of the
annotation service 500 is the entity annotation 512.
[0041] With reference to the lower portion of the diagram, the
annotation service obtains a first set 508 of annotation reasons
that describe one or more bases for a relationship between the
entity and the user--as described in block 404 of FIG. 4. This
initial set of annotation relationships is then scored, as shown in
the second set 510 of annotation relationships. Generally speaking,
these scores represent the strength of affinity between the entity
and the user that the annotation relationship represents. In this
example, those scores that are the greatest represent the most
affinity, but this is illustrative and should not be viewed as
limiting on the disclosed subject matter. The third set 512 of
annotation relationships represents a selected subset of the
annotation relationships of the earlier sets, with these restricted
to the n highest scoring relationships, where n=4 (n being a
predetermined threshold value). Of course this is illustrative but
shows that not all possible annotation relationships need to be
included in an annotation.
[0042] Regarding the selection of the best (or highest scoring)
annotation relationships, while this illustrative diagram shows
that the annotation system 500 is responsible for selecting a
subset of the best relationships, in an alternative embodiment the
annotation system returns all of the identified relationships,
along with the affinity scores, such that the requesting service
can make the selection itself.
[0043] As suggested above, a search engine 110 may be configured
with an annotation system (or annotation component) in annotating
recommended entities from among search results information.
However, the annotation system is not constrained to operate solely
as a component of the search engine and, in many cases, operates as
an independent service with regard to other online services.
Indeed, according to aspects of the disclosed subject matter the
annotation system may be implemented as a "pluggable" system that
can work (as an independent system) with any number of other
systems or services. Examples of this include, but are limited to:
associating the annotation system with a video streaming service in
which the annotation system annotates video content that the video
streaming search recommends to a user; an on-line book store in
annotating recommended titles; a social network site in annotating
friend and group recommendations; an app or music marketplace;
image annotation as described in conjunction with FIGS. 6A-6B and
7; and the like. Annotations may also be made as changes in the
current environment occur. For example, the annotation system may
annotate offers from local merchants that are periodically sent to
subscribers.
[0044] In regard to FIGS. 6A and 6B, these two figures show
pictorial diagrams for illustrating annotating entities with regard
to an image 600. In this example, we can assume that image 600
represents an image that is taken on a user's mobile phone device
(that frequently includes a camera for taking images.) A service on
the user's mobile phone device (through which the image is taken)
provides so-called augmented reality services. This service
determines the location and image view just that the subject matter
of the image 600 can be recognized. The various entities within the
recognized subject matter are then identified and a portion of the
entities are recommended to the user. These recommended entities
are then submitted to the annotation system (operating as a
pluggable service to the augmented reality service, to annotate the
recommended entities. An exemplary routine for annotating an image
is discussed in regard to routine 700 of FIG. 7. As shown in FIG.
6B, the recommended entities annotated, the augmented reality
service overlays the image 600' with user actionable icons 602-608.
Assuming that a user activated icon 602, an annotation window 610
is presented showing one or more annotation relationships between
the user and the entity identified in the image 600.
[0045] Turning to FIG. 7, this figure is a flow diagram
illustrating an exemplary routine 700 suitable for annotating
entities within an image, such as the image 600 of FIGS. 6A and 6B.
Beginning at block 702, the pluggable annotation system (i.e., an
annotation system that operates autonomously or semi-autonomously
from other systems) obtains a set of entities within the image that
are to be annotated. At block 704, the sub-routine 400 (FIG. 4) is
called to annotate the various entities. Thereafter, at block 706,
annotation controls (i.e., user-actionable indicators) are overlaid
in the image such that the user can interact and discover one or
more rationale as to why the corresponding entity is recommended to
the user.
[0046] FIG. 8 is a block diagram illustrating exemplary components
of a search engine 110 configured with an annotation system (as
suggested earlier) to respond to search queries with a search
results page that includes annotated entities. The search engine
110 includes a processor 802 and a memory 804. As those skilled in
the art will appreciate, the processor 802 executes instructions
retrieved from memory 804 in carrying out various aspects of the
search service, including annotating recommended entities within
the search results information.
[0047] The search engine 110 also includes a network communications
component 806 through which the search engine sends and receives
communications over the network 108. For example, it is through the
network communication component 806 that the search engine 110
receives search queries from user computers, such as user computers
102-106, and returns results responsive to the search queries. The
search engine 110 further includes a search results retrieval
component 808 and a search results page generation component 810.
Regarding the search results retrieval component 808, this logical
component is responsible for retrieving or obtaining search results
information relevant to a user's search query from the content
index 814. Once the set of search results information responsive to
a search query have been retrieved, an entity recommendation
component 812 identifies various entities as recommended entities
for the user. These recommendations, as well as other
personalization information, are typically based on information in
a user profile store 816.
[0048] It should be appreciated, of course, that many of these
components should be viewed as logical components for carrying out
various functions of a suitably configured search engine 110. These
logical components may or may not correspond directly to actual
components. Moreover, in an actual embodiment, these components may
be combined together or broke up across multiple actual
components.
[0049] Also included as part of the search engine 110 is the
annotation system. More particularly, this search engine 110 is
configured with an annotation system that includes an annotation
component 818 that accepts one or more recommended entities and
provides an annotation for that entity (as previously described.)
Also included as part of the annotation system of the search engine
110 is an annotation store 506 from which the annotation component
818 obtains/identifies the relationships between an entity and the
user. In at least one embodiment, these entities are identified
through an entity identification and extraction component 820. This
entity identification and extraction component identifies a given
set of entities with text, such as a user query. Of course, while
shown as part of the annotation system portion of the search engine
110, in one embodiment the entity identification and extraction
component may be an external component to the search engine.
[0050] While the annotation system of FIG. 8 is shown as being a
part of the search engine 110, as already mentioned, in an
alternative embodiment the annotation system can be implemented as
an autonomous or semi-autonomous system. FIG. 9 is a block diagram
illustrating exemplary components of an annotation system 900
suitable for annotating a plurality of entities according to
aspects of the disclosed subject matter. This exemplary annotation
system includes a processor 902 and a memory 904 implementing
similar functionality as described above in regard to FIG. 8. The
annotation system 900 further includes a network communication
component 906 through which the annotation system communicates with
other systems in carrying out its annotation function. Also
included is an annotation component 818 that accepts one or more
entities and provides an annotation for each entity (as previously
described in regard to FIG. 8). Still further included as part of
the annotation system 900 is an annotation store 506 from which the
analysis engine 514 obtains/identifies the relationships between an
entity and the user. The entity identification and extraction
component, if not included as part of the annotation component 818,
identifies a given set of entities from natural language text.
[0051] Regarding the various components identified in FIGS. 8 and
9, while certain components are identified as parts of the various
computing systems, it should be appreciated that these components
should be viewed as logical components for carrying out various
functions of suitable configured search engine 110. These logical
components may or may not correspond directly to actual components.
Moreover, in an actual embodiment, these components may be combined
together or broke up across multiple actual components.
[0052] As mentioned above in regard to FIG. 5, the annotation
system 500 identifies and/or determines the annotation
relationships for a given user/entity pair according to information
in the annotation store 506. The information in the annotation
store includes attributes, aspects, values, and metadata for
entities, and similarly includes information regarding users.
Numerous user signals--information/data sources relating to a
user--are gathered, analyzed and mined for specific pieces of
information, attributes, preferences, and the like of a user, and
this is done for many users.
[0053] Regarding the gathering of information of the users, FIG. 10
is a pictorial diagram of illustrative user signals 1000 used in
generating user information for one of many users in the annotation
store from which annotations are derived. For each user represented
in the annotation store, a set of one or more user signals 1000 is
obtained, analyzed and mined to generate the data for the user in
the annotation store 506. While user signals 1000 illustrate
various illustrative user signals that could be obtained for a
given user, they are illustrative only and should not be construed
as a limiting or mandatory set of user signals. User profile stores
of other services, such as the user profile store 816 of the search
engine 800 could be used as a user signal for many users. Moreover,
each set of user signals obtained for each user need not be the
same as those obtained for another user. For example, while
User.sub.A and User.sub.B may belong to a social media site such
that social signals 1002 are obtained for each, User.sub.A may have
a mobile phone device that enables the user to download apps, thus
app signals 1004 (including any apps that have been downloaded by
the user, the location of use of the app, whether the app was
purchased or free, and the like) may be gathered for User.sub.A,
whereas User.sub.B might not download apps to a mobile device but
is an online "gamer" such that games signals 1006 are gathered for
User.sub.B. Other user signals may include by way of illustration,
but not exclusive or limiting, explicit user preferences (that may
originate from a user profile store 816 or other source), the
geolocation of the user, music preferences (including which songs
have been purchased, ratings associated with songs, and the like),
and search history information including previously submitted
search queries, locations associated and/or specified in search
queries, search results that were visited, and the like.
[0054] As mentioned, the user signals 1000 are gathered and fed as
input into an analysis and mining process 1008. In this process,
the signals are converted into specific values, attributes,
categories, and the like and are stored in the annotation store 506
and associated with the specific user. Of course, gathering and
analyzing various user signals is typically an ongoing, likely
periodic, process for all of the users represented in the
annotation store 506 as new information may be obtained and the
annotation store 506 is updated.
[0055] With the ability to gather so many user signals 1000 for any
one user, it is advantageous to enable each user to identify
whether any signals should be analyzed (i.e., opt in or out) and,
if so, which signals can be used. To this end, FIG. 11 is a
pictorial diagram illustrating an exemplary user interface 1100
enabling a user to exercise control of the signals used in
generating that portion of the annotation store pertaining to the
user. The top portion of the user interface 1100 enables the user
to opt in or out of annotation system gathering and utilizing the
various user signals 1000 in creating user information that is used
by the annotation system 500. Assuming that a user is "opted in",
the lower portion of the exemplary user interface 1100 includes
various tabs 1102-1114 each corresponding to a specific user
signal. As shown in FIG. 1100, the specific user signal shown (via
tab 1106) is in regard to the user's music preferences. Moreover,
the user may be given an opportunity to opt in or opt out of the
annotation system using that specific user signal (as indicated by
option 1120, and specific categories within that signal (as
indicated by radio buttons 1122-1126). Opting out of any one signal
will have the result that the annotation system will not identify
annotation relationships between the user and an entity based on
that particular signal.
[0056] Information provided by an annotation system can be used to
provide recommendations to a user. User/entity annotations are
scored and the best scoring annotations could be used as the basis
for making recommendations. Indeed, there is a strong correlation
between the quality of a personalized recommendation and the
quality of user engagement with the recommendation, i.e., as the
quality of a personalized recommendation increases, so too does the
quality of the user interaction with that recommendation. As online
services continue to monetize user engagement with items, providing
quality, personalized recommendations is important.
[0057] One way in which an online service can boost the quality
and, therefore, the user engagement with a personalized
recommendation is to take advantage of the cognitive behavior of a
typical user's need to belong to a group. In particular, users
often identify themselves as belonging to networks or groups.
Online social media services provide numerous avenues in which a
user can associate with a group or network. Moreover, while users
will often associate with networks or groups, online social media
services (among other online services) may be able to automatically
associate a user with one or more groups and/or networks. Often,
but not exclusively, this automatic association is based on user
online behaviors, preferences, and activities. For example, a
Stanford alumnus may identify himself (or be automatically
identified) as belonging to that group of people. Moreover, it can
often be determined that members of a certain group will prefer
certain items, or attend certain functions, and the like. For
example, assume it could be determined that Stanford alumni attend
a local sports bar whenever Stanford's football team is playing a
game. Thus, for User.sub.A who associates himself with the Stanford
alumni group, a reasonable recommendation would be the local sports
bar during a Stanford football game with an annotation clearly
denoting why the sports bar is recommended (i.e., because Stanford
alumni apparently prefer to watch the game on the big-screen TV
there.)
[0058] Similar concepts (making recommendations based on group
associations) can be applied to entities. Indeed, it is sometimes
more interesting to note that user's aren't as interested in a
particular entity as they are in the entity category. For example,
a user may watch a specific "James Bond" movie, such as the movie
"Quantum Solace," not because of any particular aspect of the
specific movie, but because it is a "James Bond" movie.
[0059] By combining the notion of associating a user with one or
more groups of users, and categorizing entities into categories,
more annotation relationships can be identified and quality
recommendations between a user and one or more entities can be
made. More particularly, recommendations can be made to a user of
items with which the user has no experience, and these
recommendations can have a high likelihood of user engagement.
Turning, then, to FIG. 12, this is a pictorial diagram 1200 for
illustrating the promotion of a user to a group, and of an entity
to a category in determining annotations and/or recommendations for
the user. To begin with, we assume that user 1202 is known to
prefer a particular wine brand, entity 1204. Thus, when annotating
the entity 1204, at least one annotation relationship could be
something such as, "You often prefer Owen Roe Sharecroppers 2008
vintage."
[0060] However, in expanding the recommendations that may be
presented to the user 1202, the process (as will be described below
in regard to FIG. 13) determines that the user is a member of group
1206. Since user 1202 likes the wine represented by entity 1204, at
least a portion of the group 1206 likes entity 1204. A sampling of
the group members can be made to determine a probabilistic
statement that members of group 1206 have an affinity to entity
1204. Correspondingly, it may have already been determined (or
could similarly be determined) that group 1206 has an affinity to
wines represented by the category 1208 and that entity 1204 is a
member of category 1208. Moreover, another wine represented by
entity 1212 is also a member of the category 1208. Hence, a
probability (an affinity value) that a member of group 1206 will
also enjoy entity 1212 can be determined/assumed. In other words,
since user 1202 is a member of 1206, there is a probability that
user 1202 will also have an affinity to entity 1212. In terms of an
annotation the relationship if the user 1202 is recommended the
entity 1212, the relationship may be as follows: "Your friends of
the local wine taster group really like wines like this from Yakima
Valley." The format of this relationship is: "members of group G,
of which you are a member, prefer entities of category C, to which
entity E belongs." As can be seen, as users associate themselves
with groups, such as user 1210 associating with group 1206,
annotation relationships can be formed between the user and
entities preferred by the group where no direct relationship
between the user and the entities previously existed. Moreover,
recommendations can then be made based on these newly established
annotation relationships.
[0061] Those skilled in the art will appreciate that a user (and/or
an entity) will often belong to more than one group (or be
automatically associated with multiple groups) that may, in the
aggregate, either strengthen or weaken the probability that the
user has an affinity to an entity, such as entity 1204. By sampling
members of multiple groups to which a user may belong, a stronger
probabilistic statement that a user will have an affinity to an
entity may be made.
[0062] A hierarchy can be established among groups and categories.
For example, while not show, group 1206 may belong to a larger
group (i.e., members of the group of Stanford Law School alumni are
also members of the group of Stanford alumni). The same holds true
for categories of entities (i.e., Canon EOS t2i is a member of
digital SLR cameras, which is of the category of digital cameras.)
Annotation relationships and corresponding recommendations can be
identified by traversing up the hierarchy of groups and
entities.
[0063] FIG. 13 is a flow diagram illustrating an exemplary routine
for determining a recommendation of an entity of a category to a
specific user. Beginning with block 1302, a selected group of all
of the groups to which a user belongs are identified. This
selection/identification of a group may include identification from
social groups, super-groups, as well as a group of a hierarchy of
groups to which the user belongs, either directly or as a group
belonging to another group. Moreover, inclusion within a group may
be an explicit inclusion by the user (or another party) or by
implicit/implied inclusion of a group.
[0064] At block 1304, a category (of potentially many categories)
that is preferred by the selected group is identified. As above,
this preferences or affinity between the group and the identified
category may be based on explicit preference or according to an
implicit/implied affinity or preference. At block 1306, affinity
values between the entities of the identified category and the
selected group are determined. In at least one embodiment, the
affinity values are determined according to probability that the
user will prefer the corresponding entity using one or more
probability density functions. At block 1308, the entity having the
greatest affinity value is selected for recommendation to the user.
At block 1310, the selected pair is returned. As this is a
recommendation process, in at least one embodiment the selected
entity is not already a user-preferred entity, such as in the
example discussed above in regard entity 1212 (FIG. 12) that was
not related to user 1202.
[0065] As mentioned in block 1310, the recommended entity is
accompanied by an annotation that identifies the basis on which the
selected entity is recommended to the user. By way of example, for
the returned entity the annotation relationship underlying the
annotation may be "Members of Group G, of which you are a member,
tend to prefer Entity E." Thereafter, the routine 1300
terminates.
[0066] While routine 1300 is presented as an entity recommendation
routine, it should be appreciated that this routine could be
included in any number settings, including within a process of
responding to a search query. Moreover, in at least one embodiment,
the category is selected according to the subject matter of the
search query. By way of example and not limitation, when search
results obtained in response to a search query do not correspond to
an entity with which the user already has a relationship, the
search engine can provide a recommendation (and annotation) to the
user of an entity that corresponds to an obtained search result.
For example (and with reference to FIG. 12), should user 1202
submit a search query for wines that would not include a search
result directed to entity/wine 1204, yet would include search
results for wines from the category 1208 of wines, then the search
engine could recommend entity/wine 1212 that is a member of
category 1208.
[0067] Throughout the previous discussion, reference has been made
to a discover system for identifying relationships between a user
and an entity based on hierarchies of groups and classes of
entities in the context of a user submitting a search query to a
search engine. However, as suggested in regard to FIG. 9, the
disclosed subject matter should not be viewed as limited to the
search query/search engine context. It is anticipated that any
number of environments may be enhanced according to aspects of the
disclosed subject matter. By way of example, but not limitation: a
streaming video service may classify a user into one or more groups
(or allow a user to self-classify) and a hierarchy of groups may be
used to recommend one or more videos to the user according to
affinities between the groups and genres (classifications) of
videos; a college guidance system may obtain information about a
student (both explicit and implicit information) and provide highly
relevant guidance information to the student regarding colleges
(based on classification hierarchies of the various educational
institutions); and the like.
[0068] While various novel aspects of the disclosed subject matter
have been described, it should be appreciated that these aspects
are exemplary and should not be construed as limiting. Variations
and alterations to the various aspects may be made without
departing from the scope of the disclosed subject matter.
* * * * *