U.S. patent application number 15/048125 was filed with the patent office on 2016-09-22 for domain expertise determination.
The applicant listed for this patent is Microsoft Technology Licensing, LLC. Invention is credited to Susan T. Dumais, Jaime Teevan, Ryen W. White.
Application Number | 20160275082 15/048125 |
Document ID | / |
Family ID | 42076618 |
Filed Date | 2016-09-22 |
United States Patent
Application |
20160275082 |
Kind Code |
A1 |
White; Ryen W. ; et
al. |
September 22, 2016 |
DOMAIN EXPERTISE DETERMINATION
Abstract
A user's domain expertise may be estimated through several
factors by monitoring different aspects of a user's interaction
with a web browser. Based on the user's domain expertise, search
results may be presented to the user that is commensurate with
their expertise, resulting in a more efficient and productive on
line session. A user's expertise in a knowledge domain may be
determined from the user's behavior, including analyzing search
queries, monitoring how the user navigates between and through
websites, and analyzing the specific cites visited. As a user
interacts with a browser, the user's estimated domain expertise may
be updated and used to provide appropriate and useful search
results. In many embodiments, a user may have different expertise
levels for different technical domains.
Inventors: |
White; Ryen W.; (Kirkland,
WA) ; Dumais; Susan T.; (Kirkland, WA) ;
Teevan; Jaime; (Bellevue, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Microsoft Technology Licensing, LLC |
Redmond |
WA |
US |
|
|
Family ID: |
42076618 |
Appl. No.: |
15/048125 |
Filed: |
February 19, 2016 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14550096 |
Nov 21, 2014 |
9268864 |
|
|
15048125 |
|
|
|
|
13669455 |
Nov 6, 2012 |
8930357 |
|
|
14550096 |
|
|
|
|
13350928 |
Jan 16, 2012 |
8402024 |
|
|
13669455 |
|
|
|
|
12246046 |
Oct 6, 2008 |
8122021 |
|
|
13350928 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 16/24578 20190101;
G06F 16/9535 20190101; G06F 16/24575 20190101; G06F 16/248
20190101 |
International
Class: |
G06F 17/30 20060101
G06F017/30 |
Claims
1. A system, the system comprising: a processor; system memory; an
expert rating system, using the processor, configured to: identify
a search query submitted by a user during a browser session, the
search query generating a search results page, the search results
page including links to sites in a technical domain; and during the
browser session, monitor the user's interaction with one or more
pages of the linked sites, including identifying one or more
technical words in said one or more pages; and determine a
technical domain expertise of the user within the technical domain
by comparing the one or more technical words to a mapping of
technical words to expertise for the technical domain; and store
the technical domain expertise of the user.
Description
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of and priority to U.S.
patent application Ser. No. 14/550,096, entitled "Domain Expertise
Determination", filed 21 Nov. 2014 by Ryen W. White, et. al., the
entire contents of which are expressly incorporated by reference.
That application claims the benefit of and priority to U.S. patent
application Ser. No. 13/669,455, now U.S. Pat. No. 8,930,357,
entitled "Domain Expertise Determination", filed 6 Nov. 2012 by
Ryen W. White, et. al., the entire contents of which are expressly
incorporated by reference. That application claims the benefit of
and priority to U.S. patent application Ser. No. 13/350,928, now
U.S. Pat. No. 8,402,024, entitled "Domain Expertise Determination",
filed 16 Jan. 2012 Ryen W. White, et. al., the entire contents of
which are expressly incorporated by reference. That application
claims the benefit of and priority to U.S. patent application Ser.
No. 12/246,046, now U.S. Pat. No. 8,122,021, entitled "Domain
Expertise Determination", filed 6 Oct. 2008 by Ryen W. White, et.
al., the entire contents of which are expressly incorporated by
reference.
BACKGROUND
[0002] Users of the world wide web use many different techniques
and mechanisms to find information they desire. Users may perform
queries in general purpose or specialized search engines. Users may
browse to various sites with which they may be familiar.
[0003] Users often use the world wide web to find information
relating to their expertise. For example, a physician may research
a disease or condition for a patient that they may treat. Because
of the physician's expertise, the physician may wish to see a
different level of technical information than the patient or other
layperson would want to see concerning the same disease. The
patient may want to see broad overview of the disease and some
general explanation of different treatments for the disease, while
the physician may wish to understand the details of the potential
causes of the condition and tradeoffs between each course of
treatment so that an appropriate treatment may be selected.
SUMMARY
[0004] A user's domain expertise may be estimated through several
factors by monitoring different aspects of a user's interaction
with a web browser or other application. Based on the user's domain
expertise, search results may be presented to the user that is
commensurate with their expertise, resulting in a more efficient
and productive on line session. A user's expertise in a knowledge
domain may be determined from the user's behavior, including
analyzing search queries, monitoring how the user navigates between
and through websites, and analyzing the specific sites visited. As
a user interacts with a browser, the user's estimated domain
expertise may be updated and used to provide appropriate and useful
search results. In many embodiments, a user may have different
expertise levels for different technical domains.
[0005] This Summary is provided to introduce a selection of
concepts in a simplified form that are further described below in
the Detailed Description. This Summary is not intended to identify
key features or essential features of the claimed subject matter,
nor is it intended to be used to limit the scope of the claimed
subject matter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0006] In the drawings,
[0007] FIG. 1 is a diagram illustration of an embodiment showing a
system with an expertise rating system.
[0008] FIG. 2 is a process diagram illustration of an embodiment
showing a process for searching using an expertise level.
[0009] FIG. 3 is a process diagram illustration of an embodiment
showing a process for evaluating a user expertise level based on a
user's browsing behavior.
DETAILED DESCRIPTION
[0010] A user's domain or technical expertise may be determined
through a user's behavior with a web browser. Once the level of
domain expertise is determined, search results may be tailored to
the user.
[0011] Users with technical expertise in a particular field tend to
use the World Wide Web in a different manner than novices who do
not possess the same technical expertise. In many cases, the same
person may use the World Wide Web differently when operating within
their area of technical expertise than when the same person is
searching for something with which they are not familiar.
[0012] As a general rule, people with technical expertise in a
specific field tend to perform search queries in different manners
than novices, visit technically oriented websites, and perform
searches with a greater number of branches. An expert's queries
tend to use more technical language and tend to be longer and more
directed. A technical expert may visit sites that contain technical
and detailed information, and technical experts tend to have longer
sessions, visit more sites, and branch their searches many more
times than novices.
[0013] By monitoring a user's behavior with a web browser, an
assessment of a user's technical domain expertise may be
determined. In many cases, a user's expertise may be determined and
refined over many browsing sessions. As measurement of the user's
expertise is refined, the user expertise factor may be used by a
search engine to enhance the search results for the user. In cases
where the user is an expert, the search results may increase the
rank of sites with technical details, and increase the rank of
sites that are visited by other experts. In cases where the user is
not an expert, the search results may increase the rank of summary
or overview sites that have little technical jargon and detail.
[0014] A system for determining technical expertise may be
implemented in several manners. In one embodiment, a system may
operate as part of a search engine and may track a person's web
browsing from the interaction with the search engine. In another
embodiment, a monitoring system may operate at least in part on a
local device or a remote server to process a user's search queries
as well as other web browsing.
[0015] In embodiments where a monitoring system may have access to
Personally Identifiable Information (PII), a permission system or
agreement may be in place to permit or deny access based on the
permission granted by the user. In some embodiments, a monitoring
system may use data that has been scrubbed of any PII and such data
may be anonymous and untraceable to the user.
[0016] Throughout this specification, the term `domain` and
`technical domain` is used to refer to a technical area of
competence, as opposed to a `web domain` which refers to a second
or third level domain address for a website. A technical domain of
expertise refers to a technical area or subject matter. A technical
domain may be broad, such as the technical expertise associated
with mathematical or computer science, or the technical domain may
be narrow, such as the technical expertise associated with virtual
server architecture within the broad technical domain of computer
science.
[0017] A technical domain can be any classification of human
knowledge and are not limited to science or technology based
fields. Examples of a technical domain may include hobbies such as
knitting, biking, or raising tropical fish. Other examples may
include sports topics, such as professional football, athletic
activities at a specific college, or coaching children's soccer.
Still more examples may include a general topic of interest such as
landscape photography, the writings of Plato, or any other
classification of human knowledge.
[0018] Throughout this specification and claims, reference is made
to a "web browser". The term "web browser" may include any device,
application, or system that may gather and display information from
disparate sources across a network, and where a user may search and
browse such information. One example of such a system may be a
browsing application that may display information from servers
using Hyper Text Transport Protocol (HTTP). Some servers may
contain databases of links to other servers, and may respond to
search queries by presenting a list of such links that are related
to a search query.
[0019] Throughout this specification, like reference numbers
signify the same elements throughout the description of the
figures.
[0020] When elements are referred to as being "connected" or
"coupled," the elements can be directly connected or coupled
together or one or more intervening elements may also be present.
In contrast, when elements are referred to as being "directly
connected" or "directly coupled," there are no intervening elements
present.
[0021] The subject matter may be embodied as devices, systems,
methods, and/or computer program products. Accordingly, some or all
of the subject matter may be embodied in hardware and/or in
software (including firmware, resident software, micro-code, state
machines, gate arrays, etc.) Furthermore, the subject matter may
take the form of a computer program product on a computer-usable or
computer-readable storage medium having computer-usable or
computer-readable program code embodied in the medium for use by or
in connection with an instruction execution system. In the context
of this document, a computer-usable or computer-readable medium may
be any medium that can contain, store, communicate, propagate, or
transport the program for use by or in connection with the
instruction execution system, apparatus, or device.
[0022] The computer-usable or computer-readable medium may be, for
example but not limited to, an electronic, magnetic, optical,
electromagnetic, infrared, or semiconductor system, apparatus,
device, or propagation medium. By way of example, and not
limitation, computer readable media may comprise computer storage
media and communication media.
[0023] Computer storage media includes volatile and nonvolatile,
removable and non-removable media implemented in any method or
technology for storage of information such as computer readable
instructions, data structures, program modules or other data.
Computer storage media includes, but is not limited to, RAM, ROM,
EEPROM, flash memory or other memory technology, CD-ROM, digital
versatile disks (DVD) or other optical storage, magnetic cassettes,
magnetic tape, magnetic disk storage or other magnetic storage
devices, or any other medium which can be used to store the desired
information and which can accessed by an instruction execution
system. Note that the computer-usable or computer-readable medium
could be paper or another suitable medium upon which the program is
printed, as the program can be electronically captured, via, for
instance, optical scanning of the paper or other medium, then
compiled, interpreted, of otherwise processed in a suitable manner,
if necessary, and then stored in a computer memory.
[0024] Communication media typically embodies computer readable
instructions, data structures, program modules or other data in a
modulated data signal such as a carrier wave or other transport
mechanism and includes any information delivery media. The term
"modulated data signal" means a signal that has one or more of its
characteristics set or changed in such a manner as to encode
information in the signal. By way of example, and not limitation,
communication media includes wired media such as a wired network or
direct-wired connection, and wireless media such as acoustic, RF,
infrared and other wireless media. Combinations of the any of the
above should also be included within the scope of computer readable
media.
[0025] When the subject matter is embodied in the general context
of computer-executable instructions, the embodiment may comprise
program modules, executed by one or more systems, computers, or
other devices. Generally, program modules include routines,
programs, objects, components, data structures, etc. that perform
particular tasks or implement particular abstract data types.
Typically, the functionality of the program modules may be combined
or distributed as desired in various embodiments.
[0026] FIG. 1 is a diagram of an embodiment 100 showing a system
for monitoring a web browser and determining a user expertise
level. Embodiment 100 is a simplified example of the functional
components that may make up a system that can determine a user
expertise level and use the expertise level to return useful
results from a search engine.
[0027] The diagram of FIG. 1 illustrates functional components of a
system. In some cases, the component may be a hardware component, a
software component, or a combination of hardware and software. Some
of the components may be application-level software, while other
components may be operating system level components. In some cases,
the connection of one component to another may be a close
connection where two or more components are operating on a single
hardware platform. In other cases, the connections may be made over
network connections spanning long distances. Each embodiment may
use different hardware, software, and interconnection architectures
to achieve the functions described.
[0028] Embodiment 100 is an example of the functional components
that may determine and use a user's expertise within a subject
matter domain. Based on the user's expertise, a search engine may
return results that are tailored or modified to better match the
user's expertise.
[0029] Embodiment 100 may determine a user's expertise level using
many different classification schemes and mechanisms. In some
embodiments, a user may be classified as either an "expert" or
"non-expert". In other embodiments, a user's expertise may be
represented on a continuum, such as a numerical value.
[0030] The information about a user's expertise level may be used
in several different ways in the search experience. For example,
the expertise level may be used to retrieve different results for
experts and non-experts. In another example, the results may be
displayed in different ways based on expertise level. In yet
another example, query or site suggestions may be made based on
expertise level. In still another example, strategic guidance may
be provided for non-experts that may not be provided for
experts.
[0031] A web browser 102 may be connected to a monitoring system
104 that may process or monitor communications between the web
browser 102 and the internet 106. The monitor system 104 may
interact with an expertise rating system 108 that may be used to
determine a user expertise level 110. The monitor system 104 may
actively or passively receive messages from the web browser 102.
When actively receiving messages, messages may be routed through
the monitor system 104 for processing. For example, one embodiment
may have messages sent from the web browser 102 that are addressed
to the monitor system 104. The monitor system 104 may process the
messages, receive responses from various web servers, such as the
search engines 112, and transmit the responses back to the web
browser 102. Such an embodiment may include a monitoring system
that is an add-on or other component of a web browser 102 operating
on a local device, or as a server, proxy, or other component that
operates on a remote server. In some active embodiments, the
monitor system 104 may be incorporated into a search engine, for
example.
[0032] In a passive operation, the monitor system 104 may examine
communications between the web browser 102 and servers located on
the internet 106, but without having the messages addressed to and
processed by the monitoring system 104. For example, such an
operation may include monitoring log files of a communication
session or collecting the activities of the web browser.
[0033] The monitor system 104 may collect information or route
communications that may be used by an expertise rating system 108.
The expertise rating system 108 may determine a user expertise
level 110 through several different mechanisms.
[0034] The user expertise level 110 may be an estimated or
calculated measurement of a user's expertise or knowledge in a
specific technical domain. In many cases, each user may have
different levels of expertise in different technical domains. For
example, an accomplished astrophysicist may have a very high level
of expertise in astrophysics, but little expertise in cooking.
Similarly, an executive chef who is a culinary school graduate may
have a high level of expertise in cooking, but only a passing
interest in astronomy.
[0035] When people with different levels of expertise perform a
search or browse the Internet 106, their expectations and abilities
to comprehend the subject matter can be vastly different. Using the
previous example, when the astrophysicist searches for "baking" and
"cinnamon", the astrophysicist may be searching for a recipe for
cinnamon buns. When the executive chef searches for the same
general terms, the chef may be searching for articles that discuss
the identification and selection of specific varieties of cinnamon
or how the spice interacts with other ingredients during baking.
Similarly, the chef may search for "Mars" to understand the basic
properties of the fourth planet, while the astrophysicist may want
to find scientific papers discussing Mars.
[0036] In each case, the level of detail that could be comprehended
and understood by the recipient is different and is based on their
domain expertise. Thus, the search results that are relevant to an
expert in a subject may be irrelevant to a novice, and vice versa.
By determining an estimated expertise level for a user, more
relevant search results may be presented, which may enhance the
user's search experience.
[0037] As a user gathers more information and learns more about a
subject, the user may desire to see increasingly detailed and
increasingly expert-level material on the subject. In many cases,
the process of searching the Internet may raise a user's expertise
level over time.
[0038] When search results are presented to a novice, some
embodiments may have a section of `expert` level results that may
also be presented. The expert level results may be present to
enable a user to explore some of the more detailed information on a
topic and thus increase their knowledge even further than a typical
novice level search result. In the example above, the search
results for the astrophysicist's search for "baking" and "cinnamon"
may include a section of expert results that may be more
appropriate to the culinary school graduate, in addition to recipes
for cinnamon buns. Some embodiments may present the expert level
material under a separate heading or somehow visually demarked.
[0039] The expertise rating system 108 may determine or estimate a
user expertise level 110 through several different mechanisms. In
some embodiments, the expertise rating system 108 may store and
periodically update a user expertise level 110 for various
technology domains. Each time the expertise rating system 108 may
analyze a user's action, the user expertise level 110 may be
updated and stored. In embodiments where the user expertise level
is stored, the user expertise level may be recalled at the
beginning of a new session.
[0040] A user's expertise level 110 may be stored in a manner that
associates a particular user with the user expertise level. For
example, a cookie or other storage mechanism may be stored on a
device on which the web browser 102 operates. The cookie may
contain the user expertise level for specific technology domains
and may be queried and updated by the expertise rating system
108.
[0041] In some embodiments, the user expertise level 110 may be
stored on a remote server. In one such embodiment, a search engine
may store and update a user expertise level associated with a
specific search session, but such an expertise level may or may not
be associated with a specific user or device on which the web
browser 102 operates. When the expertise level is associated with
the specific user or device, the user may log onto an account with
a search engine and may set various permission settings 116 that
may acknowledge that the search engine may store parameters that
may include the user's expertise level.
[0042] One mechanism for accessing the user's expertise level is to
analyze a query to a search engine 112. The query may be examined
against a lexicon 114 of technical terms to determine a level of
expertise based on the query alone. The lexicon 114 may contain a
database of technical terms that may be associated with different
technical domains. In many embodiments, the analysis with the
lexicon 114 may determine both a technical domain as well as a
level of expertise within that domain. In some embodiments, a
sequence of queries may be analyzed over time or collectively to
determine a user's expertise.
[0043] Queries from experts tend to contain more technical words
than novices, and experts tend to use some technical words that
would not be used by novices. Since each technical domain may have
its own jargon and terminology, those who use the terminology in a
search are more likely to be expert in the field.
[0044] In a simple embodiment, a lexicon 114 may include terms
that, if used, indicate that the query is from an expert. For
example, a query for a medical condition that includes a complex
medical term "amyotrophic lateral sclerosis" may indicate that the
query is from an expert in the medical field, while a similar query
for "Lou Gehrig's Disease" would be assumed to be from a novice.
Both queries are for the same medical condition, but different
results may be presented, with the former containing mainly
technical information that a physician or other expert may
comprehend while the latter may have more information suited for a
layperson.
[0045] In a more complex embodiment, a lexicon 114 may include
expert ratings for different words or phrases. Some words or
phrases may be more indicative of an expert in the field than
others. The analysis with the lexicon 114 may include combining the
various words or phrases to give a score or numerical value for
expertise based on the query. For example, a query may contain
multiple terms, some of which have a higher expert rating than
others. A query against the lexicon 114 may return expert ratings
for each technical term, and a formula or algorithm may be used to
calculate an expertise level based on an average, maximum, or other
computation of the expert ratings.
[0046] Some embodiments may estimate a user expertise level using a
numerical score. Other embodiments may use a qualitative mechanism,
such as defining a user as `expert` or `novice`. Whether a user
expertise level is defined using a numerical score or
qualitatively, such a definition may be used in a ranking algorithm
to rank or group search results that would be applicable to a
particular user.
[0047] In many embodiments, a monitor system 104 and expertise
rating system 108 may be incorporated into a search engine 112. In
such an embodiment, the functions of analyzing a query and
determining a user expertise level 110 based on the query may be
performed separately for each search session. For example, a user
may establish an HTTP session with a search engine, transmit a
query, the query may be analyzed for a user expertise level, and
the search engine may return results based at least in part on the
calculated user expertise level. In such an embodiment, the user
expertise level 110 may be calculated new for each search
session.
[0048] In some such embodiments, a user may receive a cookie
associated with the web browser 102 that includes the user
expertise level 110. In other embodiments, the user expertise level
110 may be stored by the search engine and recalled when the user
logs onto an account with the search engine or otherwise is
identified to the search engine. In cases where user information
could be stored with personally identifiable information, the
permission settings 116 may allow or deny such practice based on a
user's consent.
[0049] In some embodiments, the monitor system 104 may monitor
other portions of a browsing session, such as monitoring
branchiness of a search. Experts in a domain tend to have searches
that have many branches, where the expert may follow a link, then
retrace their steps and follow a different link. In general, an
expert in a domain often attempts to compare or confirm one source
against another when doing a search within their field of
specialty, and an expert will visit more different web sites, as
measured by a second level domain or third level domain web
address.
[0050] A logging and analysis system 115 may be capable of
monitoring and measuring the branchiness of a search. In an
embodiment where the logging and analysis system 115 is a component
of a search engine, the logging and analysis system 115 may monitor
a user's interaction with a search results page to log when a user
selects one search result then returns to the search page and
selects another search result.
[0051] In another embodiment, the monitor system 104 may be capable
of measuring branchiness from any web browsing session, including
browsing through websites that are not related to a search engine.
In such an embodiment, the monitor system 104 may collect website
visitation information that may be in a log or history file and
determine a branchiness evaluation based on the actual history of
one or more browsing sessions.
[0052] The logging and analysis system 115 may determine the
branchiness for a single session or for two or more sessions that
relate to a specific topic. In many cases, a session may be defined
as a session that operates until the web browser is closed or until
the web browser is idle for 30 minutes. Such a definition of a
session is arbitrary and in some cases, a user may let a session
lapse for more than 30 minutes or close a web browser for a variety
of reasons.
[0053] In many cases, a web browser 102 may have tabs or windows in
which different web pages may be displayed. In some cases, a user
may have two or more different web browsers operating on a single
device, and may have multiple instances of each web browser
displayed. In such a case, the logging and analysis system 115 may
aggregate the web interactions from different instances of web
browsers, from different tabs or windows within each web browser,
and from web browsers manufactured by different manufacturers. In
such a case, the aggregated actions of a single user across
multiple browsers, multiple instances of each browser, and multiple
windows or tabs within each instance may be consolidated and
analyzed to determine branchiness.
[0054] Technical domain experts and novices tend to visit different
websites. In many cases, specific pages or second level domains may
indicate that a user is a technical expert. For example, registered
users of for-fee websites or websites that have a registration
process can be a very strong indicator of a user's expertise in a
field. In one example, a user who accesses Private PAIR through the
uspto.gov website may be considered an expert in patent law. In the
example, registered patent attorneys, agents, and their associates
are permitted to access Private PAIR after verifying their identity
and receiving credentials. Similarly, a user who visits a site
directed toward laypersons may be considered a novice.
[0055] In many cases, users may start a browsing session with a
website as opposed to a search engine. Such a website may be a good
indicator for the user's technical expertise. For example, a user
who starts a session with a technically oriented website may be
considered to be an expert, as opposed to a novice who may start a
session with a website oriented toward laypersons.
[0056] Experts may also select technically-oriented websites over
layperson-oriented websites during a search or throughout a
browsing session. As a user becomes expert in a topic, the user may
become familiar with the content provided on certain websites and
may wish to revisit the sites. Thus, some embodiments may increase
a user's expertise level when the user visits such sites
frequently.
[0057] In some embodiments, an expert site database 118 may contain
a list of websites that are frequently visited by experts and
novices. A website may be queried against the expert site database
118 to determine a technical domain and an expert level for the
website.
[0058] The expert site database 118 may be populated in several
different ways. In one method, the websites visited by other
experts in a technical domain may be cataloged and ranked based on
the visits by the other experts. In such an embodiment, a website
may have an expert level repeatedly updated as experts or novices
access the website.
[0059] In another method, a website that is not in the database may
be assigned an expert level based on an analysis of keywords from
the website. The website may be scanned to identify one or more
keywords and the keywords may be analyzed using the lexicon 212 to
determine an expert level for the website.
[0060] In still another method, a website that is not in the
database may be assigned an expert level based on an analysis of
link patterns. For example, a website's expert classification may
be inferred based on the classification of websites that link to
the website or classification of websites to which the website
links. In yet another method, a website's classification may be
determined by analyzing the expertise level of users who visit the
website.
[0061] The expert level of a website may be static or dynamic. In a
static ranking, a single determination of a website's expert level
may be performed by keyword analysis, through session log analysis,
or through a combination of analyses. The single determination may
be stored in an expert site database 118 and may not be further
analyzed. Some embodiments may perform a second analysis when the
website has changed or been updated.
[0062] In a dynamic ranking, an initial determination of a website
expert level may be made through session log analysis or keyword
analysis and stored in the expert site database 118. The expert
level may be updated over time as users subsequently visit the
website. The expert level may be changed based on users that were
previously identified as experts or novices.
[0063] The expert site database 118 may contain references to
specific pages within a website for which individual expert levels
are determined, or may contain a summary expert level that may
apply to groups of pages, third level domains within a website, or
an entire second level domain for a website.
[0064] The architecture of embodiment 100 may be structured
differently in various embodiments.
[0065] In many embodiments, the functions of a monitor system 104
and expertise rating system 108 may be part of a general purpose or
other search engine. When a user sends a query to the search
engine, an expertise rating system 108 may evaluate the query to
determine an expert level based on the query, and may return search
results that are ranked differently, grouped differently, or may
present different results altogether based on the user's expertise
level. Such an embodiment may also monitor the sites selected by
the user and monitor the branchiness of a search based on the
user's selection from a search results page.
[0066] In many such embodiments, a user's expertise level may be
stored and recalled by a search engine. In some cases, a search
engine may store and update a user's expertise level just for a
session, or may identify the user in some manner and may recall the
user's expertise level and update the same with each session. One
mechanism for storing a user's expertise level is to store such
information in a cookie on the device operating the web browser
102.
[0067] When a search engine or other remote server stores a user's
expertise level, the stored data may be anonymous and may be
tracked by some identifier with which no personally identifiable
information may be associated. In some embodiments, a user may log
onto a search engine or have an account in which the user may give
express permission to store data about the user, including the
user's expertise level.
[0068] In some embodiments, some or all of the functions of the
monitor system 104 may be operable as part of the web browser 102
or as a local process operating on the same device as the web
browser 102. In such an embodiment, a more detailed and complete
log of a browser session may be analyzed. For example, such a
monitor system 104 may be able to detect when a user starts a
browser session at a technical website and browses from one website
to another website without using a search engine. Such an
embodiment may be able to assess and update a user's expertise
level based on the websites visited and the branchiness of their
browsing.
[0069] Some embodiments may implement the monitoring system 104 as
a proxy or other site through which all web browsing and searching
messages may pass. Such a system may be within a local area
network, such as an employer's network. Another version may be a
proxy service that is offered by a third party which may or may not
provide search services. Such a version may store user information
anonymously or may have a permission setting 116 that permits such
information to be gathered with or without personally identifiable
information.
[0070] FIG. 2 is a process illustration of an embodiment 200
showing a process or method for searching using a user's expertise
level. Embodiment 200 is a simplified example of some of the steps
that may occur when processing a search query to determine a user's
expertise level, then using the expertise level with a search
engine to tailor search results for the user.
[0071] Other embodiments may use different sequencing, additional
or fewer steps, and different nomenclature or terminology to
accomplish similar functions. In some embodiments, various
operations or sets of operations may be performed in parallel with
other operations, either in a synchronous or asynchronous manner.
The steps selected here were chosen to illustrate some principles
of operations in a simplified form.
[0072] Embodiment 200 is an example of some of the operations that
may be performed to analyze a search query for a user's expertise
level. The expertise level may be used by a search engine in
various manners to tailor search results for a user's expertise
level.
[0073] A user 202 may interact with a web browser 204. The web
browser 204 may be any form of application through which a user may
select a web page to view, and through which the page may be
displayed. Many web pages have hyperlinks or other navigational
tools that may allow the user to move from one page to another
within a website or to move to other websites to view pages. Many
web browsers may have a list of bookmarked web pages that a user
may access directly, and/or may have text boxes in which a user may
type a website address to access a website.
[0074] In many embodiments, the web browser 204 may be used to
access web content from another device. Most web browser
embodiments are used for browsing content accessible over the
Internet. Other web browser embodiments may be used to access
content available on a local device or local area network.
[0075] A user 202 may generate a query 206 that may be transmitted
from the web browser 204 to an expertise rating system 208. The
expertise rating system 208 may be a function operable on a search
engine website, for example. Other embodiments may use an expertise
rating system 208 that operates within a local area network or may
operate in part on the local device on which the web browser 204
may operate.
[0076] The expertise rating system 208 may identify one or more
technical words 210 from the query 206, and may send the technical
words 210 to a lexicon 212. A query to the lexicon 212 may return
an expertise level and domain 214.
[0077] The lexicon 212 may contain a database for which various
technical words may have an expertise level defined. Some such
databases may define an expertise level to individual words or may
include multiple words as phrases. Some embodiments may have an
expertise level based on a numerical scale. A lexicon 212 may
include an algorithm for calculating an expertise level based on
the number and expertise level of the technical words in the query
206. In general, a higher expertise level may be given for those
queries with the largest number of technical words, and for those
queries having technical words with high expertise levels.
[0078] In an example above, a search query for "amyotrophic lateral
sclerosis" may indicate that a user has more of a technical
expertise in medicine than a search query for "Lou Gehrig's
Disease", even though both queries refer to the same topic. A
lexicon 212 may identify the technical terms "amyotrophic",
"lateral", and "sclerosis" as technical terms in the medical field,
and may rate each word as highly expert. The combination of three
highly technical terms may return a result that the query is from
an expert. In some embodiments, the lexicon 212 may analyze the
phrase "amyotrophic lateral sclerosis" as a single phrase and may
return a similar analysis.
[0079] When the lexicon 212 analyzes the phrase "Lou Gehrig's
Disease", each word "Lou", "Gehrig's", and "Disease" may not be a
highly technical term, but the reference to "Disease" may indicate
that the query relates to the medical field. The term "Lou Gehrig"
may be found to relate to the technical field of baseball, and thus
an analysis may return a low expertise level relating to the
technical field of medicine and a medium expertise level relating
to the technical field of baseball, for example.
[0080] In many embodiments, a lexicon 212 may contain technical
words relating to many different technical domains. Some
embodiments may include broad technical domains, such as computer
science or medicine, while others may include detailed
sub-categories of technical domains, such as neurosurgery within
medicine or compiler design within computer science.
[0081] Many embodiments may define technical domains for any type
of knowledge, pursuit, hobby, avocation, or interest imaginable.
Examples of technical domains may include tropical fish, dairy
cattle operations, knitting, amateur radio, college football, state
politics, or any other imaginable interest.
[0082] After determining an expertise level, the expertise rating
system 208 may transfer the query and expertise level 216 to a
search engine 218. The search engine 218 may use the expertise
level 216 in several different manners to return sorted search
results 220 that are tailored to the user expertise level.
[0083] In one embodiment, a search engine 218 may perform a generic
search using the query as defined by the user 202. After the
generic search is performed, the results may be sorted or organized
based on the user expertise level. Many search engines perform a
query and rank the results of the query based on several factors.
In such an embodiment, the user expertise level may be used to rank
technical results higher than non-technical results for an expert
in a field, but may rank less technical results higher than
technical results when the user is a novice.
[0084] In another embodiment, the search engine 218 may use the
expertise level to adjust a query to increase or decrease the
technical sophistication of results. Such an adjustment may be made
prior to submitting the query to a search engine database. In some
embodiments, the expertise rating system 208 may perform such an
adjustment or modification to a query prior to sending the query to
the search engine. An example of such an adjustment may be to
include an expertise rating as a variable within a search
query.
[0085] An example of adjusting the search query may be to add or
remove technical terms to a search query so that the search query
returns results that are commensurate with the user's expertise
level. For example, when a user who has a layperson's expertise
level in medicine searches for "amyotrophic lateral sclerosis", the
search query may be modified to add "Lou Gehrig's Disease". In some
cases, the search query may replace "amyotrophic lateral sclerosis"
with "Lou Gehrig's Disease".
[0086] An expertise rating may be a parameter to a search engine
that may be used to fetch different results based on expertise. In
some cases, an expertise rating may be used to categorize, rank, or
group results differently. A search engine may present results
using different visual formats or different visual organization
based on expertise level.
[0087] After performing a search, the search engine 218 may return
a group of search results 220. The expertise rating system 208 may,
in some embodiments, organize, group, or further sort the search
results using the expertise level to send a set of sorted search
results 222 to the web browser 204 for display.
[0088] In many embodiments, a user's expertise level may be saved
224 and stored as an expertise level 226. On subsequent searches,
the expertise level 226 may be recalled 228 and used as a basis for
further expertise level determination.
[0089] In many embodiments, a previously stored user expertise
level 226 may be a starting point for determining a user's
expertise level by an expertise rating system. For example, a user
may have previously performed a search using a query with a very
high expertise level. A subsequent query within the same technology
domain may have a different calculated expertise level, but the
user's new expertise level may be calculated based on the old
expertise level and the newly analyzed query. Various formulas,
heuristics, or algorithms may be used to determine a new expertise
level. One such mechanism may be to apply an exponentially weighted
moving average time series technique or some other mechanism to
calculate a new expertise level.
[0090] FIG. 3 is a process illustration of an embodiment 300
showing a process for determining a user's expertise level from
browsing behaviors. Embodiment 300 is a simplified example of some
of the steps that may occur when monitoring a user's browsing
behavior and determining an expertise level based on branchiness,
the sites visited during one or more browsing sessions, and/or
other similar features of the users' querying and browsing behavior
during one or more search sessions.
[0091] Other embodiments may use different sequencing, additional
or fewer steps, and different nomenclature or terminology to
accomplish similar functions. In some embodiments, various
operations or set of operations may be performed in parallel with
other operations, either in a synchronous or asynchronous manner.
The steps selected here were chosen to illustrate some principles
of operations in a simplified form.
[0092] Embodiment 300 is an example of some of the functional
elements and steps performed by those functional elements to
determine a user's expertise level in a particular technical domain
from a user's browsing behavior. Embodiment 300 is similar to
embodiment 200, in that an expertise rating system 208 may perform
similar analyses. In some embodiments, one or more of the functions
of embodiment 200 and embodiment 300 may be combined.
[0093] Embodiment 300 shows a user 202 that may interact with a web
browser 204. The user 202 may be able to select a website and send
a site selection 302 to an expertise rating system 208. The browser
204 may allow a user 202 to select a website through several
different mechanisms. Many browsers may have a text box in which a
Uniform Resource Locator (URL) or Internet Protocol (IP) address
may be entered to access a specific website or web page. Many
browsers may also have a bookmark system whereby an address to a
website or web page may be stored and recalled.
[0094] In many cases, a site may be selected from another site or
web page. Many web pages have hyperlinks or other mechanisms by
which another web page may be selected. In one instance, such a web
page may be a search results page that may display many links to
different websites as the result of a query. In some cases, the
query may be the same query discussed in Embodiment 200.
[0095] The expertise rating system 208 may receive a site selection
302 from a web browser 204 and determine a user's expertise level
226 from at least one of several different factors.
[0096] The expertise rating system 208 may determine a user's
expertise level 226 by determining an expertise level for the
requested website. In general, an expert in a technology domain may
request and visit those websites that are more complex and are
visited by other experts. Thus, by analyzing the requested site, a
user expertise in a domain may be inferred.
[0097] A varying degree of expertise may be inferred by how a user
selects a website. When a user selects a website from a list of
search results, a small correlation may be inferred about the
user's expertise, as opposed to a website that is selected from a
bookmark, from which a higher degree of correlation of expertise
may be inferred. The presumption being that a website selected from
a bookmark is one that is previously visited and from which the
user found something of value. By analyzing websites favored by a
user, an estimation of the user's expertise can be drawn.
[0098] One mechanism for determining an expertise level for a
website may be to scan one or more web pages within a website for
keywords. The page keywords 304 may be analyzed by the lexicon 212
to return an expertise level and knowledge domain 214. The lexicon
212 may be the same lexicon 212 used and discussed in Embodiment
200. In many embodiments, the expertise rating system 208 may send
a site request 306 to the website on the Internet 308 and receive
the site page 310. Prior to or in parallel with sending the site
page 312 to the web browser 204 for display, the expertise rating
system 208 may determine an expert level for the user in the
technical domain.
[0099] Another mechanism for determining an expertise level for a
website may be to query an expert site database 326. The expert
site database 326 may receive a site 324 and return an expertise
level and domain 328 based on the second level domain, third level
domain, or individual pages within a website.
[0100] In many embodiments, an expertise rating system 208 may send
the site 314 to a session log 316. A branchiness evaluator 320 may
analyze the contents of the session log 316 to determine an
expertise level 322. In many cases, the greater the branchiness of
a user's search, the higher level of user expertise in a technical
domain.
[0101] The session log 316 may be implemented differently in
various embodiments. In many cases, a web browser 204 may keep a
session log. In some cases, a proxy or firewall may keep a session
log that may be analyzed by a branchiness evaluator. In some cases,
an expertise rating system 208 may keep a separate session log.
[0102] The branchiness evaluator 320 may determine the relative
branchiness of a user's browsing and searching patterns. In many
cases, experts in a technology field may perform searches with many
branches and may evaluate different sources of information on a
single topic. Novices, on the other hand, tend to find a single
source and stop their search. The branchiness evaluator 320 may
analyze some or all of a session log 316 to determine how many
branches were performed by a user, and thus give an expertise level
322.
[0103] The expertise rating system 208 may determine an expertise
level based on the session length from the session log 316. In many
cases, experts in a technical domain have longer and more extensive
browsing sessions than do novices. In some embodiments, the
branchiness evaluator 320 may assess the session length to
determine an expertise level.
[0104] Similar to Embodiment 200, a user's expertise level 226 may
be saved 224 and recalled 228 with each analysis and update of the
user's expertise level.
[0105] In many embodiments, the expertise rating system 208 may use
many different factors to estimate a user's expertise level for a
particular technology domain. Some embodiments may use one or more
of the various mechanisms discussed above in different combinations
to estimate a user's expertise level. The mechanisms include
analysis of search queries, analysis of visited sites, branchiness
of a browsing session, and length of browsing sessions.
[0106] The foregoing description of the subject matter has been
presented for purposes of illustration and description. It is not
intended to be exhaustive or to limit the subject matter to the
precise form disclosed, and other modifications and variations may
be possible in light of the above teachings. The embodiment was
chosen and described in order to best explain the principles of the
invention and its practical application to thereby enable others
skilled in the art to best utilize the invention in various
embodiments and various modifications as are suited to the
particular use contemplated. It is intended that the appended
claims be construed to include other alternative embodiments except
insofar as limited by the prior art.
* * * * *