U.S. patent application number 17/650996 was filed with the patent office on 2022-06-02 for determining real-time impact of digital content through digital surveys.
The applicant listed for this patent is Qualtrics, LLC. Invention is credited to Evan Child, Nicholas Perona, Bryce Winkelman.
Application Number | 20220172234 17/650996 |
Document ID | / |
Family ID | |
Filed Date | 2022-06-02 |
United States Patent
Application |
20220172234 |
Kind Code |
A1 |
Perona; Nicholas ; et
al. |
June 2, 2022 |
DETERMINING REAL-TIME IMPACT OF DIGITAL CONTENT THROUGH DIGITAL
SURVEYS
Abstract
The present disclosure is directed toward systems, methods, and
computer-readable media for determining an impact of digital
content by administering a digital survey. In particular, the
systems described herein generate a survey administration packet
for distribution together with digital content. The survey
administration packet causes a respondent client device to perform
various functions, including detecting a trigger event and
replacing a presentation of digital content with a presentation of
a digital survey question. The systems and methods also generate
survey results indicating an impact of the digital content based on
received digital survey responses.
Inventors: |
Perona; Nicholas; (Orem,
UT) ; Child; Evan; (Provo, UT) ; Winkelman;
Bryce; (Lehi, UT) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Qualtrics, LLC |
Provo |
UT |
US |
|
|
Appl. No.: |
17/650996 |
Filed: |
February 14, 2022 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15859269 |
Dec 29, 2017 |
11250451 |
|
|
17650996 |
|
|
|
|
International
Class: |
G06Q 30/02 20060101
G06Q030/02 |
Claims
1. A computer-implemented method comprising: associating a survey
administration packet with digital content for distribution to a
respondent client device, the survey administration packet
comprising at least one digital survey question from a digital
survey, wherein distribution of the survey administration packet
and the associated digital content to the respondent client device
causes the respondent client device to provide a presentation of
the digital content; receiving, from the respondent client device,
an indication of a user interaction with the presentation of the
digital content; and automatically populating an answer to the at
least one digital survey question from the digital survey based on
the indication of the user interaction with the digital content and
without receiving user interaction in relation to the at least one
digital survey question.
2. The computer-implemented method of claim 1, further comprising:
determining a level of interest in the digital content based on the
indication of the user interaction with the presentation of the
digital content; and wherein populating the answer to the at least
one digital survey question comprises generating a survey response
that indicates the level of interest in the digital content.
3. The computer-implemented method of claim 1, further comprising:
determining that the indication of the user interaction with the
presentation of the digital content indicates a lack of user
interaction with the digital content; and wherein populating the
answer to the at least one digital survey question comprises
generating a survey response to the at least one digital survey
question that indicates a low level of interest in the digital
content based on the lack of user interaction with the digital
content.
4. The computer-implemented method of claim 1, further comprising:
determining a time attribute corresponding with the indication of
the user interaction with the presentation of the digital content;
determining a level of interest in the digital content based on the
time attribute; and wherein populating the answer to the at least
one digital survey question comprises generating a survey response
that indicates the level of interest in the digital content
determined based on the time attribute.
5. The computer-implemented method of claim 1, further comprising:
determining an interaction type corresponding with the indication
of the user interaction with the presentation of the digital
content; and wherein populating the answer to the at least one
digital survey question comprises generating a survey response
based on the interaction type.
6. The computer-implemented method of claim 5, wherein the
interaction type is a hover interaction.
7. The computer-implemented method of claim 1, wherein populating
the answer to the at least one digital survey question from the
digital survey based on the indication of the user interaction with
the digital content and without receiving user interaction in
relation to the at least one digital survey question comprises
populating answers to two or more digital survey questions from the
digital survey.
8. A non-transitory computer readable medium comprising
instructions that, when executed by at least one processor, cause a
computer device to: associate a survey administration packet with
digital content for distribution to a respondent client device, the
survey administration packet comprising at least one digital survey
question from a digital survey, wherein distribution of the survey
administration packet and the associated digital content to the
respondent client device causes the respondent client device to
provide a presentation of the digital content; receive, from the
respondent client device, an indication of a user interaction with
the presentation of the digital content; and automatically populate
an answer to the at least one digital survey question from the
digital survey based on the indication of the user interaction with
the digital content and without receiving user interaction in
relation to the at least one digital survey question.
9. The non-transitory computer readable medium of claim 8, further
comprising instructions that, when executed by the at least one
processor, cause the computer device to: determine a level of
interest in the digital content based on the indication of the user
interaction with the presentation of the digital content; and
wherein populating the answer to the at least one digital survey
question comprises generating a survey response that indicates the
level of interest in the digital content.
10. The non-transitory computer readable medium of claim 8, further
comprising instructions that, when executed by the at least one
processor, cause the computer device to: determine that the
indication of the user interaction with the presentation of the
digital content indicates a lack of user interaction with the
digital content; and wherein populating the answer to the at least
one digital survey question comprises generating a survey response
to the at least one digital survey question that indicates a low
level of interest in the digital content based on the lack of user
interaction with the digital content.
11. The non-transitory computer readable medium of claim 8, further
comprising instructions that, when executed by the at least one
processor, cause the computer device to: determine a time attribute
corresponding with the indication of the user interaction with the
presentation of the digital content; determine a level of interest
in the digital content based on the time attribute; and wherein
populating the answer to the at least one digital survey question
comprises generating a survey response that indicates the level of
interest in the digital content determined based on the time
attribute.
12. The non-transitory computer readable medium of claim 8, further
comprising instructions that, when executed by the at least one
processor, cause the computer device to: determine an interaction
type corresponding with the indication of the user interaction with
the presentation of the digital content; and wherein populating the
answer to the at least one digital survey question comprises
generating a survey response based on the interaction type.
13. The non-transitory computer readable medium of claim 8, wherein
populating the answer to the at least one digital survey question
from the digital survey based on the indication of the user
interaction with the digital content and without receiving user
interaction in relation to the at least one digital survey question
comprises populating answers to two or more digital survey
questions from the digital survey.
14. A system comprising: at least one processor; and a
non-transitory computer readable medium comprising instructions
that, when executed by the at least one processor, cause the system
to: associate a survey administration packet with digital content
for distribution to a respondent client device, the survey
administration packet comprising at least one digital survey
question from a digital survey, wherein distribution of the survey
administration packet and the associated digital content to the
respondent client device causes the respondent client device to
provide a presentation of the digital content; receive, from the
respondent client device, an indication of a user interaction with
the presentation of the digital content; and automatically populate
an answer to the at least one digital survey question from the
digital survey based on the indication of the user interaction with
the digital content and without receiving user interaction in
relation to the at least one digital survey question.
15. The system of claim 14, further comprising instructions that,
when executed by the at least one processor, cause the system to:
determine a level of interest in the digital content based on the
indication of the user interaction with the presentation of the
digital content; and wherein populating the answer to the at least
one digital survey question comprises generating a survey response
that indicates the level of interest in the digital content.
16. The system of claim 14, further comprising instruction that,
when executed by the at least one processor, cause the system to:
determine a time from beginning the presentation of the digital
content to receiving the indication of the user interaction with
the digital content; and automatically populate, without receiving
user interaction in relation to the at least one digital survey
question, the answer to the at least one digital survey question
based on the time from beginning the presentation of the digital
content to receiving the indication of the user interaction.
17. The system of claim 14, wherein the at least one digital survey
question comprises two or more digital survey questions that are
both populated based on the indication of the user interaction with
the digital content.
18. The system of claim 14, further comprising instructions that,
when executed by the at least one processor, cause the system to:
determine an interaction type corresponding with the indication of
the user interaction with the presentation of the digital content;
and wherein populating the answer to the at least one digital
survey question comprises generating a survey response based on the
interaction type.
19. The system of claim 18, wherein the interaction type is a hover
interaction.
20. The system of claim 14, further comprising instructions that,
when executed by the at least one processor, cause the system to:
determine that the indication of the user interaction with the
presentation of the digital content indicates a lack of user
interaction with the digital content; and wherein populating the
answer to the at least one digital survey question comprises
generating a survey response to the at least one digital survey
question that indicates a low level of interest in the digital
content based on the lack of user interaction with the digital
content.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] The present application is a continuation of U.S.
application Ser. No. 15/859,269, filed on Dec. 29, 2017. The
aforementioned application is hereby incorporated by reference in
its entirety.
BACKGROUND
[0002] Advancements in computing devices and networking technology
have led to a variety of innovations in providing digital content
across computer networks. For example, online digital content
systems are now able to serve digital content to users spanning the
globe almost instantaneously. Indeed, whether in entertainment,
employment, or advertising, modern online digital content systems
are able to provide instantaneous digital content to thousands of
users via various types of client devices.
[0003] Despite these advances however, conventional digital content
distribution systems continue to suffer from a number of
disadvantages. For instance, while conventional digital content
distribution systems can execute digital content campaigns and
provide digital content to selected users (e.g., a target
audience), these systems are often rigid and inflexible. To
illustrate, some conventional digital content distribution systems
seek to ascertain the effectiveness of particular digital content
(e.g., advertisements) that result in conversions (e.g., purchases)
based on a post hoc analysis of purchase statistics. By relying on
an analysis of events that occur or are only detectable after a
various amount of time after a user is exposed to digital content,
conventional digital content distribution systems are imprecise.
That is to say, conventional systems gather a diluted sense of the
effect that the digital content has on a user or audience because
other intervening events that are extraneous to the system may also
play a role in influencing a user to purchase a product or
service.
[0004] In addition, because of the delay in determining
conversions, conventional systems are often unable to provide
meaningful impression data to a content administrator within an
effective timeframe. For example, conventional systems are often
delayed by waiting on back-end data (e.g., purchase information)
that matches with a presentation of digital content. This delay
causes a content advertiser to potentially continue with an
ineffective presentation of digital content due to the inefficient
system and the delayed data. Thus, the inefficient impression data
retrieval from conventional systems results in an inefficient of
use of bandwidth and content resources based on potentially
providing large amounts of ineffective content to a large number of
users.
[0005] Such rigidity and imprecision further results in systems
that waste computing resources. As mentioned above, some
conventional digital content distribution systems are either too
slow to conclude, or altogether fail to conclude, that digital
content being provided is ineffective to a user or group of users.
Thus, the systems store content that is not effective, expend
processor resources packaging and sending digital content that is
not effective, and utilize bandwidth resources by transmitting
digital content that is not effective. In sum, by engaging less
receptive users for significant periods of time, conventional
digital content distribution systems expend an inefficient amount
of resources generating and providing digital content.
[0006] Thus, there are several disadvantages with regard to
conventional digital content distribution systems.
SUMMARY
[0007] One or more embodiments described herein provide benefits
and solve one or more of the foregoing or other problems in the art
with systems, methods, and non-transitory computer-readable media
that determine an impact of digital content by generating a survey
administration packet that causes a client device to replace a
presentation of the digital content with a digital survey. For
instance, In some embodiments, the systems distribute digital
content and a generated survey administration packet to a
respondent client device. Based on instructions associated with the
survey administration packet, a respondent client device presents a
digital survey question after a presentation of digital content
within the same content presentation platform (e.g., a website).
The disclosed systems determine an impact of the digital content to
the respondent (and overall on an audience composed of multiple
respondents) based on received responses to the digital survey
question.
[0008] The systems, methods, and computer-readable media accurately
and flexibly determine a real-time or near real-time impact of
digital content based on providing digital survey questions
together with digital content. For instance, based on detecting a
trigger event associated with a presentation of digital content,
and, in response to the trigger event, replace the presentation of
the digital content to with a presentation of or more digital
survey questions as part of a digital survey. Thus, the systems
allow for the collection of survey responses corresponding to
digital content to generate digital survey results that reflect an
impact in real-time or near real-time of the digital content in
relation to a respondent or a group of respondents.
[0009] Additional features and advantages of the present
application will be set forth in the description which follows, and
in part will be obvious from the description, or can be learned by
the practice of such example embodiments.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] This disclosure will describe one or more embodiments of the
invention with additional specificity and detail by referencing the
accompanying figures. The following paragraphs briefly describe
those figures, in which:
[0011] FIG. 1 illustrates a schematic diagram of an example
environment of a digital content survey system in accordance with
one or more embodiments;
[0012] FIGS. 2A-2B illustrate a sequence diagram for determining an
impact of digital content via a digital survey in accordance with
one or more embodiments;
[0013] FIG. 3 illustrates an example graphical user interface
provided to a publisher client device in accordance with one or
more embodiments;
[0014] FIG. 4 illustrates an example sequence for distributing
digital content and a digital survey to an exposure group and a
control group in accordance with one or more embodiments;
[0015] FIGS. 5A-5B illustrate example process flows using a survey
administration packet in accordance with one or more
embodiments;
[0016] FIGS. 6A-6C illustrate example respondent client devices
presenting example graphical user interfaces in accordance with one
or more embodiments;
[0017] FIG. 7 illustrates a schematic diagram of a digital content
survey system in accordance with one or more embodiments;
[0018] FIG. 8 illustrates a flowchart of a series of acts in a
method of determining an impact of digital content in accordance
with one or more embodiments;
[0019] FIG. 9 illustrates a block diagram of an exemplary computing
device in accordance with one or more embodiments; and
[0020] FIG. 10 illustrates an example digital content survey system
in accordance with one or more embodiments.
DETAILED DESCRIPTION
[0021] One or more embodiments described herein provide benefits
and solve one or more of the foregoing or other problems in the art
with a digital content survey system that determines an impact of
digital content by generating a survey administration packet that
replaces a presentation of digital content with a presentation of a
digital survey on a respondent client device. For example, a
respondent may access a webpage using a respondent client device.
The digital content survey system (or other content distribution
provider) can include a presentation of digital content within the
webpage. In addition, and in conjunction with the digital content,
the digital content survey system can provide instructions that
cause the digital content to be replaced by a digital survey
question within the webpage. Thus, the digital content survey
system provides a survey question related to a specific exposure
instance of digital content to respondent on the respondent's
client device.
[0022] For example, the digital content survey system can
distribute, by way of a digital content distribution service,
digital content and a generated survey administration packet to a
respondent client device. Based on instructions associated with the
survey administration packet, the respondent client device first
presents digital content (e.g., within a location of webpage), then
second presents a digital survey question corresponding to the
digital content (e.g., within the same location of the webpage).
Accordingly, the digital content survey system can determine an
impact of the digital content on the respondent (and on an overall
audience composed of multiple respondents) based on a received
response to the digital survey question corresponding to the
digital content. In this way, the digital content survey system can
provide real-time or near real-time responses to flexibly,
efficiently, and quickly adapt digital content based on the
responses to digital survey questions connected directly to a
presentation of digital content.
[0023] As described in further detail below, the digital content
survey system can generate a survey administration packet for a
digital survey that includes at least one digital survey question.
The survey administration packet can include instructions for
administering a digital survey question in conjunction with digital
content. In addition, a digital content publisher (or simply
"publisher") can provide digital content (e.g., video, images,
advertisement digital media) along with digital content
distribution instructions. In turn, the digital content survey
system associates the digital content with the survey
administration packet.
[0024] The digital content survey system can provide the digital
content together with the survey administration packet to a digital
content distribution service for distribution to respondent client
devices. Upon a respondent client device receiving the digital
content and the associated survey administration packet, the survey
administration packet causes the respondent client device to
present the digital content and, after detecting a trigger event
defined by the survey administration packet, replace the digital
content with a digital survey question. Based on a respondent
providing a response to the digital survey question via the
respondent client device, the digital content survey system
receives a response to the digital survey question. The digital
content survey system analyzes the response (and additional
responses from several respondents) to determine an impact of the
digital content for the respondent as well across an audience of
respondents.
[0025] As mentioned above, the digital content survey system
generates a survey administration packet that corresponds to a
digital survey associated with digital content. To generate the
survey administration packet, the digital content survey system
provides a graphical user interface ("GUI") to a publisher whereby
the publisher can select or otherwise input preferences,
instructions, survey questions, and other information pertaining to
a digital survey corresponding to a presentation of digital content
on a respondent client device. For example, the digital content
survey system provides a GUI including selectable options to
associate a digital survey with a digital content item.
[0026] The digital content survey system receives, by way of the
above-mentioned GUI and from the publisher, instructions and
information pertaining to digital content and a digital survey.
Based on the received instructions and information, the survey
administration packet can include a survey identifier, a question
identifier for each digital survey question, and a link or other
indicator that, in response to a detection of a trigger event
(e.g., an expiration of a period of time), retrieves, requests or
otherwise accesses a digital survey question from the digital
content survey system. In these embodiments, upon receiving the
request from the respondent client device, the digital content
survey system can provide the corresponding digital survey question
maintained within a digital survey database.
[0027] While in some embodiments the survey administration packet
includes a link or other indicator that retrieves or requests a
digital survey question from the digital content survey system, the
survey administration packet itself can include one or more digital
survey questions (e.g., question text and answer choices).
Accordingly, in these embodiments, the survey administration packet
includes instructions to reconfigure or replace a presentation of
digital content and present a digital survey question, stored
within the survey administration packet itself, by way of the
respondent client device. In other words, the survey administration
packet includes instructions and information to present a digital
survey question without accessing a digital survey database to
request a digital survey question.
[0028] In addition to referencing or include a digital survey
question, the survey administration packet can further define a
trigger event. For example, upon detection of a trigger event,
instructions within the survey administration packet can cause a
respondent client device to replace the presentation of the digital
content with a presentation of a digital survey question. For
instance, the survey administration packet can define a trigger
event such as, for example, an expiration of a threshold period of
time since the digital content was delivered to the respondent
client device, or alternatively or in addition, a user interaction
with the digital content.
[0029] In addition to generating the survey administration packet,
the digital content survey system further associates the survey
administration packet with digital content. For instance, the
digital content survey system associates the survey administration
packet with a particular item of digital content, such as a
particular video, image, text, or other media content. The digital
content survey system associates the survey administration packet
with the digital content based on instructions or preferences the
publisher provides (e.g., by way of the GUI mentioned above). In
particular, the digital content survey system receives an item of
digital content that the publisher indicates is to accompany the
digital survey having digital survey questions related to the
digital content.
[0030] After the survey administration packet is associated with
digital content, the digital content survey system provides the
survey administration packet and the digital content to a digital
content distribution service for distribution to respondent client
devices. In some embodiments, the digital content survey system
generates an exposure data set and a control data set. For example,
the survey exposure data set includes the survey administration
packet and the digital content for distribution to the respondent
client devices. The survey control data set, on the other hand, can
include the survey administration packet, while excluding the
digital content.
[0031] Based on the exposure data sent and the control data set,
the digital content survey system causes both the exposure group
and the control group to receive a digital survey question, while
only the exposure group receives a particular item of digital
content. Thus, based on responses received from respondent client
devices associated with the exposure data set and respondent client
devices associated with the control data set, the digital content
survey system can perform real-time or near real-time analysis on
an impact of the particular item of digital content. In particular,
the digital content survey system analyzes the received responses
to generate survey results. Based on the generated results, the
digital content survey system determines an impact (e.g., an
effectiveness or other value) that the digital content has had on
an audience (e.g., an audience composed of the exposure group or
the control group). Additional detail regarding determining the
impact will be provided below with reference to the figures.
[0032] The digital content survey system described herein provides
several advantages over conventional digital content distribution
systems. For example, the digital content survey system is more
flexible than conventional digital content distribution systems. To
illustrate, the digital content survey system can flexibly adapt to
provide digital content that is more relevant to a given
respondent. Indeed, the digital content survey system is capable of
tailoring digital content to fit individual respondents based on
responses to digital survey questions that are provided directly
after exposing a respondent to digital content.
[0033] As another advantage, the digital content survey system can
determine a real-time or near real-time impact of digital content.
Whereas conventional digital content distribution systems may
require days or weeks to estimate the impact of digital content,
and are typically only capable of doing so for a target audience in
general, the digital content survey system described herein
determines an impact of digital content almost immediately, and
does so on an individual basis. Thus, the digital content survey
system provides faster actionable results than conventional digital
content distribution systems.
[0034] Because the digital content survey system can determine
real-time impact of digital content, the digital content survey
system is also more effective than conventional digital content
distribution systems. For instance, because conventional digital
content distribution systems require longer periods of time to
gather feedback and other data to determine the effect that a
particular digital content item has on a target audience, these
conventional systems suffer from inaccuracies due to forgetfulness
of respondents, hindsight and/or recency bias, and fading
impressions or emotional responses to digital content. By contrast,
the digital content survey system collects responses during or
immediately after a respondent is exposed to digital content.
[0035] Furthermore, the digital content survey system requires
fewer communication resources than conventional digital content
distribution systems. Particularly, where conventional systems send
follow-up messages to solicit feedback from respondents on a post
hoc basis, the digital content survey system reduces the number of
communications by generating the survey administration packet for
distribution together with the digital content, where the survey
administration packet includes links to digital survey questions or
the digital survey questions themselves, as mentioned above and as
described in further detail below.
[0036] As used herein, a "digital survey" (or sometimes simply
"survey") refers to a set of digital survey questions intended for
distribution over a network (e.g., network 102) by way of
electronic devices (e.g., smart home devices, computers,
smartphones, tablets), and further intended to collect responses to
the digital survey questions for generating survey results from the
collected responses. A digital survey can include one or more
digital survey questions and corresponding answer choices that
accompany the given question.
[0037] As used herein, a "digital survey question" (or simply
"survey question") refers to a prompt included in a digital survey
that invokes a response from a respondent, or that requests
information from a respondent. Example types of questions include,
but are not limited to, multiple-choice, slider, open-ended,
ranking, scoring, summation, demographic, dichotomous,
differential, cumulative, dropdown, matrix, net promoter score
(NPS), singe textbox, heat map, and any other type of prompt that
can invoke a response from a respondent. In one or more
embodiments, when one or more answer choices are available for a
digital survey question, a digital survey question may include a
question portion as well as an available answer choice portion that
corresponds to the survey question. For example, when describing a
multiple-choice survey question, the term survey question may
include both the question itself as well as the multiple-choice
answers associated with the multiple-choice question.
[0038] As used herein, a "survey response" (or simply "response")
refers to any type of electronic data representing a response
provided to a digital survey question. Depending on the question
type, the response may include, but is not limited to, a selection,
a text input, an indication of an answer, an actual answer, and/or
an attachment. For example, a response to a multiple-choice
question may include a selection of one of the available answer
choices associated with the multiple-choice question. As another
example, a response may include a numerical value, letter, or
symbol that that corresponds to an available answer choice. In some
cases, a response may include a numerical value that is the actual
answer to a corresponding survey question. In addition, a survey
response may include other information associated thereto such as,
for example, geographical information pertaining to the location of
the respondent, a device ID associated with the device by which the
respondent responds to the digital survey question, or other
metadata associated with respondent/user input.
[0039] In addition, the term "respondent" refers to an individual,
group of individuals, company, organization, or other entity that
responds to one or more digital survey questions as part of a
digital survey. Such responses include, but are not limited to a
selection of an option from within a set of options (e.g., a set of
answer choices) associated with a multiple-choice question, an
open-ended text input response, a selection of a rating on a slider
bar, etc.
[0040] As used herein, a "survey result" or the term "survey
results" (or simply "results") refers to a representation of
information acquired from a survey response. A survey result can
include a simplified representation of a sentiment, opinion,
impact, effectiveness, or other information expressed by a
respondent in a survey response. In particular, survey results can
refer to a report of customer feedback ratings or a summary of poll
results based on extracting or otherwise synthesizing quantitative
and/or qualitative information from a received survey response.
[0041] The term "digital content," as used herein generally refers
to any data or information (i.e., content) that a computing device
can present to a user. Examples of digital content can include
text-based content (e.g., text blocks, titles, tables, etc.) and
image-based content (e.g., charts, graphs, graphics, pictures,
digital images, maps, gifs, movies, etc.) Additional examples of
digital content include digital media, audio/music files, location
information, notifications, advertisements, and/or search results.
Digital content can be included within a group of digital content
items. A group of digital content items does not necessarily
include the same or similar digital content items. For example, a
group of content items can include a plurality of several different
types of digital content items.
[0042] More detail regarding the digital content survey system will
now be provided with reference to the figures. In particular, FIG.
1 illustrates a schematic diagram of an example environment 100 for
implementing a digital content survey system 106 in accordance with
one or more embodiments. An overview of the digital content survey
system 106 is described in relation to FIG. 1. Thereafter, a more
detailed description of the components and processes of the digital
content survey system 106 is provided in relation to subsequent
figures.
[0043] As illustrated by FIG. 1, the chat response environment 100
includes (e.g., may be implemented by) server(s) 104, digital
survey database 108, publisher client device 110 associated with
digital content publisher 114 (or simply "publisher 114"),
respondent client devices 116a-116n (referred to herein
collectively as "respondent client devices 116") associated with
respondents 120a-120n (referred to herein collectively as
"respondents 120"), digital content distribution service 122, and a
network 102. Each of the components of the environment 100 can
communicate with others of the components via the network 102, and
network 102 may be any suitable network over which computing
devices can communicate. Example networks are discussed in more
detail below with reference to FIG. 10.
[0044] As illustrated in FIG. 1, the server(s) 104 include (e.g.,
host) the digital content survey system 106. The digital content
survey system 106 may be implemented by and/or installed on the
server(s) 104 as hardware, software, or both. The digital content
survey system 106 may be implemented by (e.g., located entirely or
in part) on one or more other components of the environment 100.
For example, the digital content survey system 106 may be
implemented by the publisher client device 110, the server(s) 104,
the respondent client devices 116, and/or the network 102.
[0045] Additionally, as shown in FIG. 1, the server(s) 104 may
include, manage, and/or maintain a digital survey database 108.
Indeed, the digital survey database 108 may be located on the
server(s) 104 or may be external or from, but accessible to, the
server(s) 104 (e.g., within the cloud on remote servers). The
digital survey database 108 stores information for digital surveys
including digital survey characteristics, digital survey
identifiers, digital survey question identifiers, digital survey
questions, trigger event information, and digital survey
responses.
[0046] As used herein, a "publisher client device" refers to a
computing device operated by a digital content publisher (e.g., an
advertiser, marketer, author, streaming provider, studio,
distributor or other publisher of digital content). For example, a
publisher client device can include one or more servers, computers,
or mobile devices (e.g., smartphones) that communicate with the
digital content survey system 106 and/or the respondent client
devices 116 via network 102. A publisher client device can generate
or maintain digital content (e.g., advertisements) and provide the
generated digital content to the digital content survey system 106,
whereupon the digital content survey system 106 can provide the
digital content to the digital content distribution service 122 for
distribution to the respondent client devices 116. The digital
content survey system 106 can provide the publisher client device
110 access to the digital content survey system to allow a
publisher 114 to create, setup or otherwise input preferences
associated with a digital survey to be associated with a digital
content campaign.
[0047] As used herein, the term "digital content campaign" refers
to a series of actions, rules, and/or processes for disseminating
digital content. For example, a digital content campaign includes
one or more digital content items and one or more campaign
parameters for disseminating the digital content. To illustrate, a
digital content campaign includes digital content together with
campaign parameters for bidding on impression opportunities,
sending content items to client devices, or targeting particular
client devices and/or users.
[0048] As illustrated in FIG. 1, the digital content survey system
106 may communicate with the publisher client device 110 to provide
a GUI whereby the publisher 114 can create a digital survey by way
of survey creation application 112. A "survey creation application"
refers to software and/or hardware that presents a GUI by way of a
publisher client device 110 and that enables the publisher 114 to
input settings, preferences, digital content, and other
information, and to provide the information to the digital content
survey system 106 via network 102. A survey creation application
may refer to a webpage, a mobile application, a software program,
executable hardware instructions, or a combination thereof.
[0049] In addition to the survey creation application 111, the
publisher client device 110 can include digital content 111. As
described above, the publisher client device 110 can create and
publish digital content 111 for a digital content campaign.
Furthermore, the publisher client device 110 can present the GUI
received from the digital content survey system 106 to the
publisher 114. Accordingly, the publisher can enter or input
preferences, instructions, digital survey questions, and other
information pertaining to a digital survey to accompany the digital
content 111 by way of the survey creation application 112.
[0050] Returning to FIG. 1, the digital content survey system 106
can communicate with the publisher client device 110 to receive the
digital content 111 and the information, preferences, etc.,
pertaining to the digital survey. The digital content survey system
106 can generate a survey administration packet based on the
received information, preferences, etc. As used herein, the term
"survey administration packet" refers to a packet or package of
data that is grouped together, linked, or otherwise associated, and
that includes information for a digital survey and/or a digital
survey question. In some example embodiments, a survey
administration packet can include a survey identifier, a question
identifier, and/or trigger event information.
[0051] In addition, a survey administration packet can include an
indicator or link that instructs a computing device to request,
retrieve, or access a digital survey that corresponds to the survey
identifier and/or a digital survey question that corresponds to the
question identifier. Alternatively, or additionally, the survey
administration packet can include digital survey content. For
instance, the survey administration packet can include digital
survey question text and answer choices for each of a number
digital survey questions. Additional detail regarding the survey
administration packet is provided below with reference to FIGS. 5A
and 5B.
[0052] As further illustrated in FIG. 1, the digital content survey
system 106 can communicate with the digital content distribution
service 122. For instance, the digital content survey system 106
can provide, to the digital content distribution service 122, the
digital content 111 and the survey administration packet. As used
herein, the "digital content distribution service" refers to a
service that includes one or more servers that distribute digital
content (e.g., digital content 111) via a network (e.g., network
102). For example, a digital content distribution service can
include a media provider such as an internet service provider
and/or can include an advertising server, or other service that is
capable of distributing digital content (e.g., digital content 111)
to computer devices. In some embodiments, the digital content
distribution service 122 may be associated with (e.g., located on
or maintained by) the publisher client device 110.
[0053] Indeed, the digital content survey system 106 and/or the
digital content distribution service 122 can communicate with the
respondent client devices 116 to distribute the digital content 111
together with the survey administration packet. To illustrate, the
digital content distribution service 122 can provide a video to
respondent client device 116a via network 102, whereby respondent
client device 116a can present the video to the respondent
120a.
[0054] As used herein, the term "respondent client device" refers
to any of a variety of computing devices such as mobile devices,
desktop computers, laptop computer, tablets, or wearable devices. A
respondent client device is capable of communicating across network
102 with the other components of the environment 100 to receive and
transmit data including digital content 111 and the survey
administration packet, and is further capable of presenting, via a
display, the digital content 111 and a digital survey question.
Each of the respondent client devices 116 can include the same type
of functionality consistent with the disclosure herein, but need
not be the same type of device. Additional information regarding
computing devices (e.g., servers and client devices) is provided
below with respect to FIG. 9.
[0055] As illustrated in FIG. 1, the respondent client devices 116
are capable of receiving user input from respondents 120 in the
form of user interaction with digital content 111 or input of
responses to digital survey questions. For instance, the respondent
client devices 116 each include survey response application 118.
The survey response application 118 is capable of presenting
digital content 111 as well as a digital survey question and answer
choices. The survey response application 118 further provides the
respondent client devices 116 the ability to communicate with the
digital content survey system 106 and/or the digital content
distribution service 122 to receive the digital content 111 and to
provide responses to digital survey questions.
[0056] As used herein, the term "survey response application"
refers to hardware and/or software that enables a respondent client
device (e.g., respondent client device 116a) to receive digital
content, present digital content, receive a digital survey, present
digital survey questions associated with the digital survey,
receive user input for answers to digital survey questions, and
provide responses to digital survey questions corresponding to the
answers received via user input. A survey response application may
refer to a webpage, a mobile application, a software program,
executable hardware instructions, or a combination thereof.
[0057] Based on the received responses, the digital content survey
system 106 can generate survey results, as mentioned above. In
particular, the digital content survey system 106 can analyze the
received responses to determine an impact of the digital content
111 with relation to each of the respondents 120 individually and
as a group. The digital content survey system 106 can further
communicate with the publisher client device 110 to provide the
survey results. Additional detail regarding generating the survey
results, including a discussion of determining the impact of the
digital content 111, is provided below with reference to the
subsequent figures.
[0058] With reference to FIGS. 2-8, one or more components of the
digital content survey system 106 can manage, along and/or in
combination with other components of the environment 100, a digital
survey corresponding to digital content. For example, FIGS. 2A-2B
illustrate an example sequence 200 that includes a number of acts
202-226. One or more of the components described in FIG. 1 perform
the acts described in relation to FIGS. 2A-2B. Although FIGS. 2A-2B
depict particular components performing particular acts 202-226, in
some embodiments alternative or additional components can perform
the acts 202-226 described in relation to FIGS. 2A-2B.
Additionally, although FIGS. 2A-2B illustrate a particular number
of acts 202-226 and a particular sequence in which the acts 202-226
are performed, in some embodiments the sequence 200 can include
more or fewer acts, and the acts can be perform in an alternative
order or arrangement.
[0059] As illustrated in FIG. 2A, the sequence 200 includes acts
performed by the publisher client device 110, the digital content
survey system 106 (e.g., as implemented by the server(s) 104), the
digital content distribution service 122, and a respondent client
device 116a. Although the environment 100 includes multiple
respondent client devices 116, it will be beneficial to describe
the functioning of the digital content survey system 106 and other
actors/components with reference to a single respondent client
device 116a--and in some cases a different or additional respondent
client device 116b.
[0060] The digital content survey system 106 performs act 202 to
provide a graphical user interface ("GUI") for creating a digital
survey to the publisher client device 110. The digital content
survey system 106 provides a GUI that includes options for setting
a trigger event, setting a control group size, an exposure group
size, indicating whether the digital survey is linked or embedded,
etc. Additional detail regarding the GUI is provided below with
reference to FIG. 3.
[0061] The digital content survey system 106 provides the GUI to
the publisher client device 110, whereupon the publisher client
device 110 provides digital content 111 and digital survey
instructions to the digital content survey system 106, as indicated
by act 204 of sequence 200. To illustrate, the publisher 114 inputs
digital survey instructions in the GUI by way of the survey
creation application 112. The digital survey instructions include
the above-mentioned settings for a trigger event, a control group
size, an exposure group size, indicating whether the digital survey
is linked or embedded, etc., as described further in FIG. 3. To
provide the digital content 111, the publisher client device 110
uploads, transfers, or otherwise transmits the digital content 111
together with the digital survey instructions to the digital
content survey system 106 (e.g., via network 102).
[0062] The digital content survey system 106 receives the digital
content 111 and the digital survey instructions, whereupon the
digital content survey system 106 generates a survey administration
packet, as illustrated by act 206. To generate the survey
administration packet, the digital content survey system 106
analyzes the received digital survey instructions and performs the
necessary functions to create a data package that includes computer
instructions and information capable of causing a computer device
to perform various functions. For instance, the survey
administration packet can include the settings for the trigger
event, a control group size, an exposure group size, a survey
identifier associated with the digital survey, and/or one or more
question identifiers associated with each digital survey question
that are part of the digital survey.
[0063] In addition, the digital content survey system 106 can
generate a linked survey administration packet that includes a link
(e.g., a pointer or other indicator) to each digital question that
is part of the digital survey as stored on the digital survey
database 108. The digital content survey system 106 can also
generate an embedded survey administration packet that includes
digital survey content (e.g., the digital survey questions
themselves, including question text and answer choices). Indeed,
the digital content survey system 106 may generate a survey
administration packet to be either linked, embedded, or a
combination of linked and embedded. For instance, the survey
administration packet links to certain digital survey questions and
directly includes digital survey questions.
[0064] Based on the above description of the survey administration
packet, "linked survey," as used herein, refers to digital survey
wherein the corresponding survey administration packet is of the
variety that includes the link or pointer to the digital survey
database 108 (e.g., does not include digital survey question text
or answer choices therein). This type of survey administration
packet may be referred to as a "linked survey administration
packet." In addition, an "embedded survey" refers to a digital
survey wherein the corresponding survey administration packet is of
the variety that includes the digital survey (e.g., including
digital question text and answer choices) therein. This type of
survey administration packet may be referred to as an "embedded
survey administration packet." A combination survey that includes
both linked survey information and embedded survey information may
be referred to as a "hybrid survey administration packet."
[0065] As illustrated in FIG. 2A, the digital content survey system
106 further performs an act 208 to associate the survey
administration packet with the digital content 111. In particular,
the digital content survey system 106 generates an exposure data
set that includes the digital content 111 and the survey
administration packet. As used herein, the term "exposure data set"
refers to a package of computer data or information composed of two
or more distinct portions that are grouped together for
simultaneous distribution to computer devices. An exposure data set
can include digital content (e.g., digital content 111) and the
survey administration packet. Indeed, the term "exposure" refers to
the act of exposing a respondent to the digital content 111.
[0066] Additionally, the digital content survey system 106
generates a control data set that includes only the survey
administration packet and does not include the digital content 111.
As used herein, the term "control data set" refers to a package of
computer data or information composed of a single portion that
includes digital survey information, but does not include digital
content. The control data set is purposefully lacking digital
content because the control data set is for distribution to a
control group.
[0067] As mentioned, the digital content survey system 106
generates both a control data set and an exposure data set. Indeed,
the digital content survey system 106 generates the control data
set for distribution to a number of the respondent client devices
116 that corresponds with a control group size indicated by the
publisher 114 via the GUI mentioned above. Generally, the control
group is a number of respondent client devices 116 fewer than the
total number of respondent client devices 116. The digital content
survey system 106 generates the control data set for distribution
to respondent client devices 116 to cause the respondent client
devices 116 to present one or more digital survey questions without
presenting the digital content 111.
[0068] The digital content survey system 106 generates the exposure
data set for distribution to an exposure group. The exposure group
is a number of respondent client devices 116 fewer than the total
number of respondent client devices 116 and different from the
respondent client devices 116 associated with the control group.
Indeed, the digital content survey system 106 generates the
exposure data set for distribution to respondent client devices 116
to cause the respondent client devices 116 to present the digital
content 111 and, after detecting a trigger event, to present a
digital survey question.
[0069] As used herein, the term "trigger event" refers to an
occurrence, action, or instance of a particular event that triggers
or causes a computer device to perform specific functions. For
example, a trigger event can include the expiration of a threshold
period of time, where, upon detecting the expiration of the
threshold period of time, a respondent client device (e.g.,
respondent client device 116a) replaces a presentation of digital
content (e.g., digital content 111) with a presentation of a
digital survey question. As another example, a trigger event can
include detecting a user input (e.g., a user interaction with the
digital content 111), whereupon the respondent client device 116a
replaces the presentation of the digital content 111 with a
presentation of a digital survey question. Alternatively, upon
detecting a user interaction with the digital content 111,
respondent client device 116a (as instructed by the survey
administration packet) can automatically (e.g., without user
interaction) populate, generate, and/or provide survey responses
for digital survey questions to the digital content survey system
106.
[0070] For example, in some embodiments, the digital content survey
system 106 generates a control data set that includes a linked
survey administration packet (referred to herein as a "linked
control data set") and also generates an exposure data set that
includes a linked survey administration packet (referred to herein
as a "linked exposure data set"). In other embodiments, the digital
content survey system 106 generates a control data set that
includes a linked survey administration packet and generates an
exposure data set that includes an embedded survey administration
packet (referred to herein as an "embedded exposure data set"). In
still other embodiments, the digital content survey system 106
generates a control data set that includes an embedded survey
administration packet (referred to herein as an "embedded control
data set") and an exposure data set that includes a linked survey
administration packet. In further embodiments, the digital content
survey system 106 generates a control data set that includes an
embedded survey administration packet and an exposure data set that
also includes an embedded survey administration packet.
[0071] In still further embodiments, the digital content survey
system 106 generates a control data set of one type (e.g., a linked
control data set or an embedded control data set) for distribution
to respondent client device 116a and generates a control data set
of a different type for distribution to a different respondent
client device 116b. Likewise, the digital content survey system 106
can generate different types of exposure data sets for distribution
to different respondent client devices 116.
[0072] Returning to FIG. 2A, as shown by the sequence 200, the
digital content survey system 106 further performs act 210 to
provide the digital content 111 and the survey administration
packet to a digital content distribution service 122. As mentioned
above, the digital content survey system 106 provides the digital
content 111 and the survey administration packet together in the
form of an exposure data set. Though not explicitly shown in FIG.
2A, in some embodiments, the digital content survey system 106
further provides a control data set to the digital content
distribution service 122, which includes the survey administration
packet but does not include the digital content 111.
[0073] As also illustrated in FIG. 2A, the digital content
distribution service 122 distributes the digital content 111 and
the survey administration packet to the respondent client device
116a, as shown in act 212. Indeed, the digital content distribution
service 122 distributes the exposure data set to the respondent
client device 116a. Accordingly, the respondent client device 116a
belongs to the exposure group described above. Thus, the survey
administration packet causes the respondent client device 116a to,
by way of the survey response application 118, perform specific
acts or functions described hereafter.
[0074] Though not illustrated in FIG. 2A, the digital content
distribution service 122 further distributes the control data set
to a different respondent client device (e.g., respondent client
device 116b). As mentioned above, the digital content survey system
106 designates a control group apart from the exposure group that
consists of a subset of the respondent client devices 116 that do
not receive the digital content 111 but receive only the survey
administration packet.
[0075] As mentioned, the survey administration packet causes the
respondent client device 116a to perform specific acts 214-218.
Continuing the sequence 200 to FIG. 2B, the respondent client
device 116a performs the act 214 to present the digital content
111. For example, the respondent client device 116a presents a
video or image by way of the survey response application 118. For
example, the respondent client device presents digital content
(e.g., an advertisement) within a webpage. In the case of a
webpage, the exposure data set can be included within the webpage
code that, when rendered by the survey response application (e.g.,
a web browser), causes the webpage to present the digital content
within a portion of the webpage.
[0076] The respondent client device 116a further performs act 216
to detect a trigger event. As mentioned above, the survey
administration packet includes the trigger event information (e.g.,
the information that defines the trigger event). Accordingly, the
survey administration packet causes the respondent client device
116a to detect a trigger event such as, for example, the expiration
of a threshold period of time since the respondent client device
116a began presenting the digital content 111. Other trigger events
include, but are not necessarily limited to, detecting user
interaction (e.g., input) from the respondent 120a with the digital
content 111 or the expiration of a threshold period of time since
the digital content distribution service 122 first distributed the
digital content 111 and/or the survey administration packet.
[0077] As further illustrated by FIG. 2B, the respondent client
device 116a replaces the presentation of the digital content 111
with a presentation of a digital survey question, as depicted by
act 218. In particular, the survey administration packet includes
instructions to cause the respondent client device 116a to, in
response to detecting the trigger event (act 216), replace the
digital content with a digital survey question. For example,
referencing the above case of presenting digital content within a
portion of a webpage, upon detecting the trigger event,
instructions within the survey administration packet cause the
respondent client device to replace the digital content 111 with a
digital survey question within the portion of the webpage within
which previously included the digital content. In other
embodiments, the digital survey question does not replace the
digital content, but rather modifies the digital content to allow
the survey question to be included within the portion of the
webpage (e.g., reduces the size of the digital content). In other
embodiments, the digital survey question is provided within another
portion of the webpage, thus, allowing the presentation of the
digital content to remain within the webpage, if desired.
[0078] In embodiments where the respondent client device 116a
receives a linked survey administration packet, the respondent
client device 116a requests access to a digital survey question
maintained by the digital content survey system 106 on the digital
survey database 108. On the other hand, in embodiments where the
respondent client device 116a receives an embedded survey
administration packet, the respondent client device 116a replaces
the digital content 111 with a local digital survey question
included with (e.g., as part of) the survey administration packet,
again per the instructions of the survey administration packet.
[0079] Continuing with the sequence 200 of FIG. 2B, the respondent
client device 116a further performs the act 220 to provide a
response to the digital content survey system 106. In particular,
the respondent client device 116a receives user input from the
respondent 120a to select or input answer choices for one or more
digital survey questions. In response to receiving the answer
choices, the respondent client device 116a generates a survey
response including the answer choices and provides (e.g.,
transmits) the survey response to the digital content survey system
106.
[0080] In some embodiments, the respondent client device 116a
provides (as instructed by the survey administration packet) a
survey response for each individual digital survey question at a
time, providing a response each time the respondent 120a provides
an answer for a given digital survey question. In other
embodiments, the respondent client device 116a provides a survey
response as a bundle at the end of the digital survey (e.g., as
indicated by the last digital survey question of the digital
survey) that includes survey responses for each digital survey
question.
[0081] In some embodiments, such as (but not limited to) where the
survey administration packet defines the trigger event as detecting
user interaction with the digital content 111, the respondent
client device 116a automatically generates survey responses. To
illustrate, the survey administration packet may include, or link
to, a digital survey question that asks, for example, "Was this
digital content interesting to you?" Based on detecting a user
interaction (e.g., a click, tap, touch, press, or other user input)
with the digital content, the respondent client device 116a can
automatically generate a survey response to the digital survey
question indicating that the respondent 120a found the digital
content 111 interesting. In other words, without ever explicitly
providing a digital survey question for presentation to a
respondent, the survey administration packet can include
instructions to automatically provide a survey response to one or
more digital survey questions based on a user interaction or lack
of user interaction with the digital content.
[0082] Similarly, based on the information included in the survey
administration packet, the respondent client device 116a can
automatically generate survey responses that vary along a spectrum
of possible answer choices. For example, a survey administration
packet may include, or link to, a digital survey question that
includes multiple answer choices in a sliding scale type digital
survey question (e.g., a scale from 1 to 10) that asks, "How likely
are you to purchase the depicted product in the next month, with 1
being least likely and 10 being most likely?" The survey
administration packet may cause the respondent client device 116a
to time how long it takes for the respondent 120a to interact with
the digital content 111.
[0083] To illustrate, the respondent client device 116a may
determine an amount of time that elapses between when the
respondent client device 116a first begins presenting the digital
content 111 and when the respondent 120a interacts with the digital
content 111. Based on how long it takes for the respondent 120a to
interact with the digital content 111, the survey administration
packet may cause the respondent client device 116a to automatically
generate a survey response corresponding to a particular number on
the scale from 1 to 10 that illustrates the desire or necessity of
the respondent 120a in purchasing the product depicted in the
digital content 111. Indeed, a very short time (i.e., a fast user
interaction) may correspond with a higher number (e.g., 8, 9, or
10), while a longer time may correspond with a lower number. In
another embodiment, the survey administration packet may cause the
client device to detect that a user hovers over (e.g., with a mouse
pointer) the digital content, but does not select it. Based on the
hover interaction, the client device can automatically send one or
more responses as described above.
[0084] Furthermore, the survey administration packet may cause the
respondent client device 116a to swap digital content. To
illustrate, the survey administration packet may include more than
one digital content item and may cause the respondent client device
116a to, upon detecting a negative sentiment (e.g., a selection of
an answer choice that reflects negative sentiment), replace the
digital content 111 with different digital content. In some
embodiments, the survey administration packet includes only digital
content 111 and does not include any additional digital content. In
these or other embodiments, the survey administration packet causes
the respondent client device 116a to, upon detecting a negative
sentiment, replace the digital content 111 by requesting additional
or alternative digital content from the digital content survey
system 106, the digital content distribution service 122, and/or
the publisher client device 110.
[0085] Although not illustrated in FIG. 2B, the digital content
distribution service 122 distributes the control data set to a
control group that includes those respondent client devices 116
that are not part of the exposure group, as mentioned above. In the
case of one these control group respondent client devices 116, such
as respondent client device 116b for example, the survey
administration packet causes the respondent client device 116b to
perform a different series of acts than those depicted in relation
to the respondent client device 116a of the exposure group.
[0086] To illustrate, the survey administration packet causes the
respondent client device 116b to present a digital survey question
without first presenting any digital content (e.g., digital content
111). Thus, the respondent client device 116b need not detect a
trigger event to cause the respondent client device 116b to replace
digital content with a digital survey question. Instead, the
respondent client device 116b presents the digital survey question
in lieu of the digital content 111, in the place where the
respondent client device 116b would otherwise have presented the
digital content 111. Additional detail regarding the differences in
acts performed by respondent client devices 116a (exposure group)
and 116b (control group) is provided below in relation to FIG.
4.
[0087] In any case, the respondent client device 116a (or 116b)
provides a survey response to the digital content survey system
106, as illustrated by act 220. Accordingly, the digital content
survey system 106 performs act 222 of sequence 200 to receive the
survey response. Upon receiving the survey response to each digital
survey question, either all at once or one at a time (as described
above), the digital content survey system 106 performs act 224 of
sequence 200 to determine an impact of the digital content 111.
[0088] As used herein, the term "impact" refers to an indication of
an effectiveness of, or attitude or sentiment that a respondent
(e.g., respondent 120a) exhibits toward, digital content (e.g.,
digital content 111). An impact may refer to an overall impact or
effectiveness as an average or collective determination over an
entire audience (e.g., respondents 120 corresponding to respondent
client devices 116), or else an impact may refer to an impact that
the digital content 111 has with regard to a single respondent
(e.g., 116a or 116b) on an individual level.
[0089] As mentioned, the digital content survey system 106
determines an impact of the digital content 111. Based on the
determined impact, the digital content survey system 106 further
performs act 226 to provide results to the publisher client device
110. Indeed, the digital content survey system 106 generates
results that indicate the impact of the digital content 111, based
on, for example, analyzing the responses received from the
respondent client devices 116. For instance, the digital content
survey system 106 may analyze the survey responses using text
analysis techniques, machine learning models (e.g., deep learning,
neural networks, etc.), statistical models, and other techniques as
mentioned above to determine the impact and then provide an
indication of the impact to the publisher client device 110 via
survey results.
[0090] As described briefly above, and as FIG. 3 illustrates, the
digital content survey system 106 generates and provides a GUI to
the publisher client device 110, whereby the publisher 114 enters
digital survey preferences, uploads digital content (e.g., digital
content 111), sets control and exposure group preferences, and/or
otherwise organizes a digital content campaign. FIG. 3 illustrates
publisher client device 110 presenting the GUI 300 that includes
trigger event settings 302, a survey mode option 304, control group
settings 306, digital content portion 308, and digital content
option 310. Additional detail regarding the elements of the GUI 300
are provided hereafter. Furthermore, although FIG. 3 illustrates a
particular layout and arrangement of the GUI 300, the GUI 300 is
merely an example, and other layouts and arrangements are possible
and may include more or fewer elements than those illustrated in
FIG. 3.
[0091] The GUI 300 of FIG. 3 includes trigger event settings 302.
As illustrated in FIG. 3, the trigger event settings 302 enable a
publisher 114 to select which type of trigger event to use. For
instance, the publisher 114 can select either a time-based trigger
event (as described above) or a user interaction-based trigger
event (also described above), or both. In addition to providing
options to selection which type of trigger event to use, the GUI
300 may also include options for the publisher 114 to enter an
amount (e.g., duration) of time to define a time-based trigger
event. Additionally, the GUI 300 may include options for the
publisher to select a type of user interaction to use as the user
interaction-based trigger event. For example, the type of user
interaction can include a selection (e.g., click, touch, tap, etc.)
of the digital content 111, a muting of any audio included in the
digital content 111, a selection to turn on any audio included in
the digital content 111, or a selection of a terminate/close option
associated with the digital content.
[0092] As illustrated in FIG. 3, the GUI 300 further includes a
survey mode option 304. The survey mode option 304 may include user
selectable options for the publisher 114 to select which type of
digital survey to use. For example, the survey mode option 304 can
include an option to select either an embedded survey or a linked
survey, as described above. Indeed, in response to receiving an
indication from the publisher client device 110 that the publisher
114 selects the embedded survey option, the digital content survey
system 106 generates an embedded survey administration packet. Upon
receiving an indication that the publisher 114 selects the option
to use a linked survey, on the other hand, the digital content
survey system 106 generates a linked survey administration packet,
as described in further detail above.
[0093] As further illustrated in FIG. 3, the GUI 300 includes
control group settings 306. In particular, the control group
settings 306 include an option for the publisher to select to use a
control group or to select to not use a control group. Based on
detecting that the publisher 114 selects the option to use a
control group, the GUI 300 is configured to present options for the
publisher 114 to enter a control group size and an exposure group
size. The control group size and the exposure group size may refer
to a number of respondents 120 that will receive the control data
set and the exposure data set, respectively.
[0094] FIG. 3 further illustrates that the GUI 300 includes a
digital content portion 308. In particular, the digital content
portion 308 represents the digital content 111 that the publisher
chooses to include with the digital survey as part of the digital
content campaign. The digital content portion 308 may include one
or more digital content items. Indeed, the publisher 114 can choose
to add additional or alternative digital content by selecting the
digital content option 310.
[0095] As mentioned, the GUI 300 of FIG. 3 further includes a
digital content option 310. Upon detecting a selection of the
digital content option 310, the GUI 300 is configured to enable the
publisher to upload or otherwise add additional or alternative
digital content to be associated with the digital survey. In cases
where the publisher 114 chooses to include more than one digital
content item as part of the digital content campaign, the digital
content survey system 106 may perform the acts described above in
relation to FIGS. 2A and 2B with respect to each digital content
item. For example, the digital content survey system 106 may
determine an impact of each individual digital content item.
Alternatively, the digital content survey system 106 may determine
a collective impact of the digital content campaign as a whole,
across all included digital content items. Accordingly, by
providing the GUI 300 to the publisher client device 110, the
digital content survey system 106 enables the publisher 114 to
organize a digital content campaign. Based on the input of the
publisher 114, the digital content survey system 106 generates a
digital survey and carries out a digital content campaign.
[0096] FIG. 4 illustrates a sequence 400 of acts 402-426 performed
by the digital content survey system 106 and/or other components of
the environment 100 described above. As illustrated in FIG. 4, the
digital content survey system 104 receives the digital content
(e.g., digital content 111) and the survey instructions (e.g., the
input received from the publisher client device 110 by way of the
GUI 300), as shown in act 402. For example, the digital content
survey system 106 receives indications of trigger event type,
survey mode (e.g., embedded or linked), control group size,
exposure group size, etc. The digital content survey system 106
further receives the digital content 111 that is to accompany the
digital survey.
[0097] Based on the received information, the digital content
survey system 106 generates a digital survey for the digital
content 111. Indeed, as illustrated in FIG. 4, the digital content
survey system 106 performs act 404 to generate control data set and
an exposure data set. As described above, the control data set
includes a survey administration packet but does not include the
digital content 111. On the other hand, the exposure data set
includes both the survey administration packet and the digital
content 111. As also described above, the survey administration
packet included within the control data set and/or the exposure
data set may be either linked or embedded.
[0098] FIG. 4 further illustrates that the digital content survey
system 106 performs act 406 to provide the control data set and the
exposure data set to the digital content distribution service 122.
Indeed, based on the information received from the publisher client
device 110 (e.g., via the GUI 300), the digital content survey
system 106 provides a control data set for distribution to a
particular number of respondent client devices 116 that are
associated with the control group. Likewise, the digital content
survey system 106 provides an exposure data set for distribution to
a particular number of respondent client devices that are
associated with the exposure group.
[0099] To determine which respondents 120 belong to the control
group and which respondents 120 belong to the exposure group, the
digital content survey system 106 may analyze a potential audience.
For example, the digital content survey system 106 may determine a
number of respondent client devices 116 to include within each of
the exposure group and the control group based on the information
received from the publisher client device 110 via GUI 300. In
addition, the digital content survey system 106 generates a control
group for the purpose of determining the impact of the digital
content 111--e.g., to show the effect that the digital content 111
has on respondents 120 who are exposed to the digital content 111
versus those respondents 120 who are not.
[0100] In some embodiments, the digital content survey system 106
generates a random control group, while in other embodiments the
digital content survey system 106 generates a control group
consisting of respondents 120 who represent various demographics.
For instance, the digital content survey system 106 generates a
control group that includes respondents 120 of each sex, of varying
ages, location, income, ethnicity, race, or other demographic.
[0101] In any case, the digital content survey system 106 provides
the control data set and the exposure data set to the digital
content distribution service 122. As illustrated in FIG. 4, the
digital content distribution service 122 distributes the exposure
data set by performing act 408 and distributes the control data set
by performing act 410. Indeed, as illustrated in FIG. 4, the box
428 represents the exposure group and the box 430 represents the
control group. Additionally, the acts depicted within the boxes 428
and 430 represent acts that the survey administration packet causes
a given respondent client device (e.g., 116a for the exposure group
or 116b for the control group) to perform.
[0102] As illustrated in FIG. 4, the sequence 400 further includes
act 410 to show that the survey administration packet causes the
respondent client device 116a to present the digital content 111.
The respondent client device 116a presents the digital content 111
to the respondent 120a and further performs act 412 to detect a
trigger event. As described above, the respondent client device
116a detects a time-based trigger event and/or a user
interaction-based trigger event. Based on detecting the trigger
event, the respondent client device 116a then replaces the
presentation of the digital content 111 with a presentation of a
digital survey question (e.g., the first digital survey question of
the digital survey).
[0103] To replace the digital content 111 with the digital survey
question, the respondent client device 116a may access or otherwise
load the digital survey question and present the digital survey
question in the same location on a screen of the respondent client
device 116a--i.e., the location where the respondent client device
116a presented the digital content 111. In other embodiments, the
respondent client device 116a presents the digital survey question
together with the digital content 111. For instance, the respondent
client device 116a presents the digital survey question within a
sub-window or nested portion of the presentation of the digital
content 111.
[0104] As further illustrated in FIG. 4, the respondent client
device 116a receives user input for a response, as shown by act
416. In particular, the respondent client device 116a receives user
input in the form of a selection or other input of an answer to the
digital survey question. Based on the received answer, the
respondent client device 116a provides a survey response that
includes each answer to each respective digital survey question
(e.g., either in turn or altogether, as described above) to the
digital content survey system 106, as shown by act 418 of sequence
400.
[0105] As mentioned above, the digital content distribution service
122 performs act 420 to distribute the control data set to the
control group (e.g., including respondent client device 116b). Upon
receiving the control data set, the respondent client device 116b
presents the digital survey question, as illustrated by act 422. In
particular, the respondent client device 116b refrains from
presenting the digital content 111 or else simply does not present
the digital content 111 because the respondent client device 116b
does not receive the digital content 111. Indeed, as described
above, the control data set does not include the digital content
111.
[0106] Continuing the discussion of FIG. 4, sequence 400
illustrates that the respondent client device 116b performs act 424
to receive user input for a response. Similar to the discussion
above in relation to the respondent client device 116a and the
exposure data set, the respondent client device 116b also receives
user input from a respondent 120b. In particular, the respondent
client device 116b receives user input in the form of answers to
digital survey questions. Based on those received answers, the
respondent client device 116b performs act 426 to provide a
response to the digital content survey system 106 that indicates
the answers input by the respondent 120b.
[0107] As described above, the digital content survey system 106
generates a survey administration packet. The digital content
survey system 106 can generate a linked survey administration
packet or an embedded survey administration packet. FIGS. 5A and 5B
illustrate the differences between the two and the different
functions that each survey administration packet causes a computing
device to perform.
[0108] FIG. 5A illustrates a flow 500a in relation to a linked
survey administration packet. As illustrated in FIG. 5A, the flow
500a includes an exposure data set 502 that includes the digital
content 111 and the linked survey administration packet 503. The
flow 500a further includes the digital content distribution service
122, the digital content survey system 106 including the digital
survey database 108, and acts 504-510 performed by one or more of
the digital content distribution service 122, the digital content
survey system 106, and a respondent client device (e.g., respondent
client device 116a).
[0109] As illustrated in FIG. 5A, the linked survey administration
packet 503 includes a survey identifier, a question identifier, an
indicator or link to a digital survey (including digital survey
questions) located on a digital survey database 108, a control
group size, and an exposure group size. Indeed, the survey
administration packet 503 includes the information necessary to
cause the digital content distribution service 122 to distribute
the exposure data set 502 to exposure group and the control group
(although FIG. 5A depicts acts only for the exposure group). In
addition, the survey administration packet 503 includes the
information and instructions necessary to cause the respondent
client device 116a to perform the acts described above.
[0110] As mentioned, and as illustrated in FIG. 5A, the digital
content distribution service 122 performs act 504 to distribute the
exposure data set 502 to a respondent client device (e.g.,
respondent client device 116a). Upon receiving the exposure data
set 502, the respondent client device 116a performs act 506 to
detect a trigger event, as described in further detail above. For
instance, the respondent client device 116a detects the expiration
of a threshold period of time since the respondent client device
116a began presenting the digital content 111. The trigger event is
defined by the trigger event information included in the survey
administration packet 503.
[0111] Upon detecting the trigger event (act 506), the respondent
client device 116a requests access or otherwise retrieves a digital
survey question maintained by the digital content survey system 106
on a digital survey database 108, as shown by act 508 of FIG. 5A.
Indeed, because the survey administration packet 503 is a linked
survey administration packet, the survey administration packet 503
causes the respondent client device 116a to perform act 508 to
request a digital survey question that is not included directly
within the survey administration packet 503. To request the digital
survey question, the respondent client device 116a provides the
survey identifier and the question identifier included in the
survey administration packet 503 to access the corresponding
digital survey question within the digital survey database 108.
[0112] In response to receiving the request to access the digital
survey question from the digital survey database 108, the digital
content survey system 106 performs act 510 to provide the digital
survey question to the respondent client device 116a. In
particular, the digital content survey system 106 provides the text
of the digital survey question from a digital survey that matches
the survey identifier included in the survey administration packet
503. Likewise, the digital content survey system 106 provides the
digital survey question that matches the question identifier
included as part of the survey administration packet 503, as shown
in FIG. 5A. Additional acts that the digital content survey system
106 and respondent client device 116a perform based on the digital
content survey system 106 providing the digital survey question to
the respondent client device 116a are discussed above with
reference to the previous figures.
[0113] FIG. 5B illustrates a flow 500b in relation to an embedded
survey administration packet. As illustrated in FIG. 5B, the flow
500b includes an exposure data set 512 that includes the digital
content 111 and an embedded survey administration packet 513. The
flow 500b further includes the digital content distribution service
122 and acts 514-518.
[0114] As shown in FIG. 5B, the embedded survey administration
packet 513 includes a survey identifier, a question identifier,
digital survey text, trigger event information, control group size,
and exposure group size. Unlike the linked survey administration
packet 503 of FIG. 5A, the embedded survey administration packet
513 includes digital survey text. In particular, the digital survey
text includes digital survey questions and answer choices for one
or more digital survey questions that are part of the digital
survey.
[0115] FIG. 5B illustrates that the digital content distribution
service 122 distributes the exposure data set to a respondent
client device (e.g., respondent client device 116a). As mentioned
above, the survey administration packet 513 includes instructions
that cause the respondent client device 116a to present the digital
content 111 and to perform act 516 to detect a trigger event.
However, unlike the discussion in relation to FIG. 5A, the survey
administration packet 513 does not cause the respondent client
device to request access to a digital survey question from the
digital content survey system 106 and/or digital survey database
108. Instead, the survey administration packet 513 causes the
respondent client device 116a to present the digital survey
question that is stored within the survey administration packet 513
itself. Accordingly, the respondent client device 116a need not
request access via network 102, but may instead only require local
processes to access and present--i.e., replace the digital content
111 with--the digital survey question, as shown by act 518 of FIG.
5B.
[0116] While FIGS. 5A and 5B illustrate exposure data sets 502 and
512, respectively, it will be understood from the disclosure herein
that the digital content distribution service 122 also distributes
a control data set in one or more of the embodiments described in
relation to FIGS. 5A and 5B. As described above, the control data
set includes a survey administration packet (e.g., the linked
survey administration packet 503 or the embedded survey
administration packet 513), but does not include the digital
content 111. Therefore, the respondent client device that receives
the control data set does not present any digital content, but
instead presents a digital survey question. Additional detail
regarding the control data set is provided above.
[0117] Additional acts that the digital content survey system 106
and respondent client device 116a perform based on the digital
content survey system 106 providing the digital survey question to
the respondent client device 116a are discussed above with
reference to the previous figures.
[0118] FIGS. 6A-6C illustrate the respondent client device 116a and
the GUIs that the respondent client device 116a presents based on
the acts described above. For example, FIG. 6A illustrates the
respondent client device 116a presenting digital content 604 via a
display 602. As shown in FIG. 6A, digital content 604 is a
graphical (e.g., still image) advertisement for a pet store. The
respondent client device 116a presents the digital content 604 in
response to receiving the exposure data set described above.
[0119] FIG. 6B illustrates the same respondent client device 116a
presenting a digital survey question 606 by way of the display 602.
The respondent client device 116a may present the digital survey
question 606 as a result of detecting a trigger event. For example,
the respondent client device 116a may detect an expiration of a
threshold period of time whereupon the respondent client device
116a replaces the presentation of the digital content 604 with the
presentation of the digital client device 606. Alternatively, the
respondent client device 116a may detect a user interaction such as
a tap on the display 602 by the respondent 120a to trigger the
respondent client device 116a to replace the digital content 604
with the digital survey question 606.
[0120] As illustrated in FIG. 6B, the digital survey question 606
also includes answer choices 608. Indeed, the answer choices 608
include options for the respondent 120a to answer the digital
survey question. In response to receiving a selection of one of the
answer choices 608, the respondent client device 116a may provide a
survey response indicating the answer choice to the digital content
survey system 106, as described above. Additionally, the respondent
client device 116a may further advance the digital survey to
present a subsequent digital survey question, as illustrated in
FIG. 6C.
[0121] As illustrated in FIG. 6B, the digital survey question 606
asks "What company did the advertisement depict?" Based on the
response that the digital content survey system 106 receives (e.g.,
whether the response is correct or incorrect), the digital content
survey system 106 can analyze the response together with other
responses from other respondent client devices 116 to determine an
impact of the digital content 604. For example, the digital content
survey system 106 can determine whether the digital content 604 is
effective enough to cause respondents 120 to answer the digital
survey question 606 correctly. Thus, the digital content survey
system 106 can determine how effective (e.g., based on a scale or
score of effectiveness) the digital content 604 is at portraying a
brand.
[0122] While FIG. 6B represents the respondent client device 116a
as part of the exposure group, it will be understood from the
disclosure herein that respondent client device 116b may present
the same digital survey question 606 and the same answer choices
608 without first presenting the digital content 604 of FIG. 6A.
Likewise, the respondent client device 116b can present the digital
survey question 610 and the answer choices 612 of FIG. 6C,
described in more detail hereafter.
[0123] As mentioned, in response to receiving user input to select
an answer choice for the digital survey question 606, the
respondent client device 116a replaces the presentation of the
digital survey question 606 with a presentation of digital survey
question 610. Indeed, the survey administration packet may cause
the respondent client device 116a to iteratively present one
digital survey question after another until the digital survey is
complete--i.e., until every digital survey question is presented
and/or answered.
[0124] As illustrated in FIG. 6C, the digital survey question 610
asks "Did the advertisement increase the likelihood you will
purchase from the depicted company?" Based on responses that the
digital content survey system 106 receives to this digital survey
question 610, the digital content survey system 106 can determine
an impact of the digital content 604 that, in addition to
indicating an effectiveness in portraying a brand, also indicates
an effectiveness in influencing conversions (e.g., purchases).
Accordingly, based on responses to the digital survey questions 606
and 610, the digital content survey system 106 can generate survey
results that indicate the impact of the digital content 604, and
can further provide the survey results to the publisher client
device 110. Thus, the publisher 114 may act accordingly to adjust a
digital content campaign as necessary to increase
effectiveness.
[0125] Turning now to FIG. 7, additional detail will be provided
regarding components and capabilities of the digital content survey
system 106. Specifically, FIG. 7 illustrates an example schematic
diagram of the digital content survey system 106 on an exemplary
computing device 700 (e.g., the server(s) 104). As shown in FIG. 7,
the digital content survey system 106 may include a graphical user
interface manager 702, a digital survey manager 704, a survey
response manager 706, a survey results manager 708, and a storage
manager 710 that can include a digital survey database 712. While
FIG. 7 depicts a particular number of components, in some
embodiments, the digital content survey system 106 may include more
or fewer components. In addition, the components may perform
additional or alternative tasks than those described hereafter.
[0126] As mentioned, the digital content survey system 106 includes
a graphical user interface manager 702. In particular, the
graphical user interface manager 702 generates and provides a GUI
such as GUI 300. For instance, the graphical user interface manager
702 generates a GUI with a particular layout of GUI elements and
selectable options, whereby the publisher 114 can input
preferences, settings, and other information for a digital content
campaign that includes a digital survey together with digital
content. Furthermore, the graphical user interface manager 702 can
generate those portions of the survey administration packet that
cause a respondent client device (e.g., respondent client device
116a or 116b) to present the various elements described above.
[0127] The digital content survey system 106 also includes a
digital survey manager 704. In particular, the digital survey
manager creates or generates the survey administration packet for
distribution to respondent client devices 116. For example, the
digital survey manager 704 generates a linked survey administration
packet and an embedded survey administration packet for
distribution to respondent client devices 116 based on the
disclosure herein. Furthermore, the digital survey manager 704
generates the exposure data set and the control data set to include
the digital content 111 and the survey administration packet for
distribution to the exposure group and the control group, as
described above. In addition, the digital survey manager 704
associates the survey administration packet with the digital
content for distribution to the respondent client devices 116. To
illustrate, the digital survey manager 704 assigns a survey
identifier to a particular digital survey, and assigns question
identifiers to each digital survey question included as part of the
digital survey.
[0128] As illustrated in FIG. 7, the digital content survey system
106 further includes a survey response manager 706. In particular,
the survey response manager 706 receives survey responses from
respondent client devices 116. The survey response manager 706
collects or gathers responses, and associates each response with
its respective digital survey question and digital survey by
identifying the survey identifier and question identifier
associated with each received survey response.
[0129] As further illustrated in FIG. 7, the digital content survey
system 106 includes a survey results manager 708. The survey
results manager 708 generates survey results based on the received
survey responses. For instance, the survey results manager 708
determines an impact of digital content (e.g., digital content 111)
based on the responses received in relation to the digital content
111. Additionally, the survey results manager 708 provides survey
results indicating the impact of the digital content 111 to the
publisher computing device 110, thereby providing actionable
information to a publisher 114 to improve the effectiveness of a
digital content campaign.
[0130] The digital content survey system 106 further includes a
storage manager 710. In particular, the storage manager 710
includes a digital survey database 712. To illustrate, the digital
survey database 712 may represent digital survey database 108
described above. The storage manager 710 may store, maintain, and
manage digital survey information pertaining to digital surveys
within the digital survey database 712. For example, the storage
manager 710 may organize each digital survey according to a survey
identifier associated with the digital survey, and may further
organize each digital survey question within a given digital survey
according to a question identifier associated with the digital
survey question. Indeed, the survey identifier and the question
identifier may be metadata attached to, included within, or
otherwise associated with a digital survey or digital survey
question.
[0131] Furthermore, the storage manager 710 may store and organize
survey results within the digital survey database 712 as well. In
addition, the survey results may also include a survey identifier
whereby the storage manager 710 associates the survey results to a
particular digital survey. The storage manager 710 may maintain
survey results to provide the survey results to digital content
publishers (e.g., publisher 114) on more than one occasion (e.g.,
upon request).
[0132] FIGS. 1-7, the corresponding text, and the examples provide
a number of different systems and methods for determining an impact
of digital content by administering digital surveys. In addition to
the foregoing, embodiments can also be described in terms of
flowcharts comprising acts steps in a method for accomplishing a
particular result. For example, turning now to FIG. 8, this figure
illustrates a flowchart of a series of acts 800 of determining an
impact of digital content based on a survey administration packet.
While FIG. 8 illustrates acts according to one embodiment,
alternative embodiments may omit, add to, reorder, and/or modify
any of the acts shown in FIG. 8. The acts of FIG. 8 can be
performed as part of a method. Alternatively, a non-transitory
computer readable medium can comprise instructions, that when
executed by one or more processors, cause a computing device to
perform the acts of FIG. 8. In still further embodiments, a system
can perform the acts of FIG. 8.
[0133] As illustrated in FIG. 8, the series of acts 800 may include
an act 802 of generating a survey administration packet. In
particular, the act 802 may involve generating a survey
administration packet for a digital survey including at least one
digital survey question. The act 802 can include receiving, from a
publisher client device, instructions for administering the digital
survey. The survey administration packet can include the at least
one digital survey question and instructions to present the at
least one digital survey question in response to the respondent
client device detecting the trigger event.
[0134] The series of acts 800 may also include an act 804 of
associating digital content with the survey administration packet.
In particular, the act 804 can involve associating, by at least one
processor, digital content to be provided to a respondent client
device with the survey administration packet. For example, the act
804 can include generating an exposure data set comprising the
digital content and the survey administration packet.
[0135] As further illustrated in FIG. 8, the series of acts 800 can
include an act 806 of providing the digital content and the survey
administration packet. In particular, the act 806 can involve
providing the digital content and the survey administration packet
to a digital content distribution service for distribution to the
respondent client device, wherein receiving the digital content and
the survey administration packet causes the respondent client
device to present the digital content and, in response to detecting
a trigger event defined by the survey administration packet,
replace the presentation of the digital content with a presentation
of the at least one digital survey question of the digital survey.
The act 806 can include providing the exposure data set to the
digital content distribution service for distribution to the
respondent client device. Detecting the trigger event can include
detecting that a threshold period of time has expired since the
respondent client device began presenting the digital content.
[0136] The series of acts 800 can further include an act 808 of
receiving a response. In particular, the act 808 can involve
receiving, from the respondent client device, a response to the at
least one digital survey question. As illustrated in FIG. 8, the
series of acts 800 can still further include an act 810 of
determining an impact. In particular, the act 810 can involve
determining an impact of the digital content based at least in part
on the response.
[0137] Though not illustrated in FIG. 8, the series of acts 800 can
further include an act of generating a control data set comprising
the survey administration packet, wherein the survey control set
does not include the digital content. The series of acts 800 can
also include an act of providing the survey control set to the
digital content distribution service for distribution to a
different respondent client device, wherein, upon the different
respondent client device receiving the survey control set, the
survey administration packet causes the different respondent client
device to present the at least one digital survey question without
the digital content. In addition, the series of acts 800 can
include an act of receiving, from the different respondent client
device, a response to the at least one digital survey question.
Indeed, determining the impact of the digital content can be
further based on the response received from the different
respondent client device.
[0138] The series of acts 800 can still further include acts of
receiving an indication of a user interaction with the digital
content by way of the respondent client device and automatically
populating an answer to the at least one digital survey question
based on the indication of the user interaction with the digital
content. The series of acts 800 can also include an act of
providing, to a publisher client device, a graphical user interface
comprising a plurality of selectable options, wherein the plurality
of selectable options comprises options to define the trigger event
and the survey administration packet, as well as an act of
providing, based on the received response, alternative digital
content to the digital content distribution service for
distribution to replace the digital content distributed to the
respondent client device.
[0139] In addition, the series of acts 800 can include an act of
receiving an indication of a user interaction with the digital
content by way of the respondent client device, and can include an
act of automatically populating an answer to the at least one
digital survey question based on the indication of the user
interaction with the digital content, wherein the at least one
digital survey question comprises two or more digital survey
questions that are both populated based on the received indication
of the user interaction with the digital content.
[0140] Embodiments of the present disclosure may comprise or
utilize a special purpose or general-purpose computer including
computer hardware, such as, for example, one or more processors and
system memory, as discussed in greater detail below. Embodiments
within the scope of the present disclosure also include physical
and other computer-readable media for carrying or storing
computer-executable instructions and/or data structures. In
particular, one or more of the processes described herein may be
implemented at least in part as instructions embodied in a
non-transitory computer-readable medium and executable by one or
more computing devices (e.g., any of the media content access
devices described herein). In general, a processor (e.g., a
microprocessor) receives instructions, from a non-transitory
computer-readable medium, (e.g., a memory, etc.), and executes
those instructions, thereby performing one or more processes,
including one or more of the processes described herein.
[0141] Computer-readable media can be any available media that can
be accessed by a general purpose or special purpose computer
system. Computer-readable media that store computer-executable
instructions are non-transitory computer-readable storage media
(devices). Computer-readable media that carry computer-executable
instructions are transmission media. Thus, by way of example, and
not limitation, embodiments of the disclosure can comprise at least
two distinctly different kinds of computer-readable media:
non-transitory computer-readable storage media (devices) and
transmission media.
[0142] Non-transitory computer-readable storage media (devices)
includes RAM, ROM, EEPROM, CD-ROM, solid state drives ("SSDs")
(e.g., based on RAM), Flash memory, phase-change memory ("PCM"),
other types of memory, other optical disk storage, magnetic disk
storage or other magnetic storage devices, or any other medium
which can be used to store desired program code means in the form
of computer-executable instructions or data structures and which
can be accessed by a general purpose or special purpose
computer.
[0143] A "network" is defined as one or more data links that enable
the transport of electronic data between computer systems and/or
modules and/or other electronic devices. When information is
transferred or provided over a network or another communications
connection (either hardwired, wireless, or a combination of
hardwired or wireless) to a computer, the computer properly views
the connection as a transmission medium. Transmissions media can
include a network and/or data links which can be used to carry
desired program code means in the form of computer-executable
instructions or data structures and which can be accessed by a
general purpose or special purpose computer. Combinations of the
above should also be included within the scope of computer-readable
media.
[0144] Further, upon reaching various computer system components,
program code means in the form of computer-executable instructions
or data structures can be transferred automatically from
transmission media to non-transitory computer-readable storage
media (devices) (or vice versa). For example, computer-executable
instructions or data structures received over a network or data
link can be buffered in RAM within a network interface module
(e.g., a "NIC"), and then eventually transferred to computer system
RAM and/or to less volatile computer storage media (devices) at a
computer system. Thus, it should be understood that non-transitory
computer-readable storage media (devices) can be included in
computer system components that also (or even primarily) utilize
transmission media.
[0145] Computer-executable instructions comprise, for example,
instructions and data which, when executed by a processor, cause a
general-purpose computer, special purpose computer, or special
purpose processing device to perform a certain function or group of
functions. In some embodiments, computer-executable instructions
are executed on a general-purpose computer to turn the
general-purpose computer into a special purpose computer
implementing elements of the disclosure. The computer executable
instructions may be, for example, binaries, intermediate format
instructions such as assembly language, or even source code.
Although the subject matter has been described in language specific
to structural features and/or methodological acts, it is to be
understood that the subject matter defined in the appended claims
is not necessarily limited to the described features or acts
described above. Rather, the described features and acts are
disclosed as example forms of implementing the claims.
[0146] Those skilled in the art will appreciate that the disclosure
may be practiced in network computing environments with many types
of computer system configurations, including, personal computers,
desktop computers, laptop computers, message processors, hand-held
devices, multiprocessor systems, microprocessor-based or
programmable consumer electronics, network PCs, minicomputers,
mainframe computers, mobile telephones, PDAs, tablets, pagers,
routers, switches, and the like. The disclosure may also be
practiced in distributed system environments where local and remote
computer systems, which are linked (either by hardwired data links,
wireless data links, or by a combination of hardwired and wireless
data links) through a network, both perform tasks. In a distributed
system environment, program modules may be located in both local
and remote memory storage devices.
[0147] Embodiments of the present disclosure can also be
implemented in cloud computing environments. In this description,
"cloud computing" is defined as a model for enabling on-demand
network access to a shared pool of configurable computing
resources. For example, cloud computing can be employed in the
marketplace to offer ubiquitous and convenient on-demand access to
the shared pool of configurable computing resources. The shared
pool of configurable computing resources can be rapidly provisioned
via virtualization and released with low management effort or
service provider interaction, and then scaled accordingly.
[0148] A cloud-computing model can be composed of various
characteristics such as, for example, on-demand self-service, broad
network access, resource pooling, rapid elasticity, measured
service, and so forth. A cloud-computing model can also expose
various service models, such as, for example, Software as a Service
("SaaS"), Platform as a Service ("PaaS"), and Infrastructure as a
Service ("IaaS"). A cloud-computing model can also be deployed
using different deployment models such as private cloud, community
cloud, public cloud, hybrid cloud, and so forth. In this
description and in the claims, a "cloud-computing environment" is
an environment in which cloud computing is employed.
[0149] FIG. 9 illustrates, in block diagram form, an exemplary
computing device 900 that can be configured to perform one or more
of the processes described above. One will appreciate that the
digital content survey system 112 can comprise implementations of
the computing device 900. As shown by FIG. 9, the computing device
can comprise a processor 902, memory 904, a storage device 906, an
I/O interface 908, and a communication interface 910. In certain
embodiments, the computing device 900 can include fewer or more
components than those shown in FIG. 9. Components of computing
device 900 shown in FIG. 9 will now be described in additional
detail.
[0150] In particular embodiments, processor(s) 902 includes
hardware for executing instructions, such as those making up a
computer program. As an example, and not by way of limitation, to
execute instructions, processor(s) 902 can retrieve (or fetch) the
instructions from an internal register, an internal cache, memory
904, or a storage device 906 and decode and execute them.
[0151] The computing device 900 includes memory 904, which is
coupled to the processor(s) 902. The memory 904 can be used for
storing data, metadata, and programs for execution by the
processor(s). The memory 904 can include one or more of volatile
and non-volatile memories, such as Random Access Memory ("RAM"),
Read Only Memory ("ROM"), a solid-state disk ("SSD"), Flash, Phase
Change Memory ("PCM"), or other types of data storage. The memory
1204 can be internal or distributed memory.
[0152] The computing device 900 includes a storage device 906
includes storage for storing data or instructions. As an example,
and not by way of limitation, storage device 906 can comprise a
non-transitory storage medium described above. The storage device
906 can include a hard disk drive (HDD), flash memory, a Universal
Serial Bus (USB) drive or a combination of these or other storage
devices.
[0153] The computing device 900 also includes one or more input or
output ("I/O") devices/interfaces 908, which are provided to allow
a user to provide input to (such as user strokes), receive output
from, and otherwise transfer data to and from the computing device
900. These I/O devices/interfaces 908 can include a mouse, keypad
or a keyboard, a touch screen, camera, optical scanner, network
interface, modem, other known I/O devices or a combination of such
I/O devices/interfaces 908. The touch screen can be activated with
a stylus or a finger.
[0154] The I/O devices/interfaces 908 can include one or more
devices for presenting output to a user, including, but not limited
to, a graphics engine, a display (e.g., a display screen), one or
more output drivers (e.g., display drivers), one or more audio
speakers, and one or more audio drivers. In certain embodiments,
devices/interfaces 908 is configured to provide graphical data to a
display for presentation to a user. The graphical data can be
representative of one or more graphical user interfaces and/or any
other graphical content as can serve a particular
implementation.
[0155] The computing device 900 can further include a communication
interface 910. The communication interface 910 can include
hardware, software, or both. The communication interface 910 can
provide one or more interfaces for communication (such as, for
example, packet-based communication) between the computing device
and one or more other computing devices 900 or one or more
networks. As an example, and not by way of limitation,
communication interface 910 can include a network interface
controller (NIC) or network adapter for communicating with an
Ethernet or other wire-based network or a wireless NIC (WNIC) or
wireless adapter for communicating with a wireless network, such as
a WI-FI. The computing device 900 can further include a bus 912.
The bus 912 can comprise hardware, software, or both that couples
components of computing device 900 to each other.
[0156] FIG. 10 illustrates an example network environment 1000 of a
digital content survey system. Network environment 1000 includes a
client system 1008 and a digital content survey system 1004
connected to each other by a network 1006. Although FIG. 10
illustrates a particular arrangement of client system 1008, digital
content survey system 1004, and network 1006, this disclosure
contemplates any suitable arrangement of client system 1008,
digital content survey system 1004, and network 1006. As an
example, and not by way of limitation, two or more of client system
1008, digital content survey system 1004, bypassing network 1006.
As another example, two or more of client system 1008, and digital
content survey system 1004 can be physically or logically
co-located with each other in whole or in part. Moreover, although
FIG. 10 illustrates a particular number of client systems 1008,
digital content survey system 1004, and networks 1006, this
disclosure contemplates any suitable number of client systems 1008,
digital content survey system 1004, and networks 1006. As an
example, and not by way of limitation, network environment 1000 can
include multiple client system 1008, digital content survey system
1004, and networks 1006.
[0157] This disclosure contemplates any suitable network 1006. As
an example, and not by way of limitation, one or more portions of
network 1006 can include an ad hoc network, an intranet, an
extranet, a virtual private network (VPN), a local area network
(LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless
WAN (WWAN), a metropolitan area network (MAN), a portion of the
Internet, a portion of the Public Switched Telephone Network
(PSTN), a cellular telephone network, or a combination of two or
more of these. Network 1006 can include one or more networks
1006.
[0158] Links can connect client system 1008, and digital content
survey system 1004 to communication network 1006 or to each other.
This disclosure contemplates any suitable links. In particular
embodiments, one or more links include one or more wireline (such
as for example Digital Subscriber Line (DSL) or Data Over Cable
Service Interface Specification (DOCSIS)), wireless (such as for
example Wi-Fi or Worldwide Interoperability for Microwave Access
(WiMAX)), or optical (such as for example Synchronous Optical
Network (SONET) or Synchronous Digital Hierarchy (SDH)) links. In
particular embodiments, one or more links each include an ad hoc
network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a
WWAN, a MAN, a portion of the Internet, a portion of the PSTN, a
cellular technology-based network, a satellite communications
technology-based network, another link, or a combination of two or
more such links. Links need not necessarily be the same throughout
network environment 1000. One or more first links can differ in one
or more respects from one or more second links.
[0159] In particular embodiments, client system 1008 can be an
electronic device including hardware, software, or embedded logic
components or a combination of two or more such components and
capable of carrying out the appropriate functionalities implemented
or supported by client system 1008. As an example, and not by way
of limitation, a client system 1008 can include any of the
computing devices discussed above in relation to FIG. 10. A client
system 1008 can enable a network user at client system 1008 to
access network 1006. A client system 1008 can enable its user to
communicate with other users at other client systems 1008.
[0160] In particular embodiments, client system 1008 can include a
web browser, such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME or
MOZILLA FIREFOX, and can have one or more add-ons, plug-ins, or
other extensions, such as TOOLBAR or YAHOO TOOLBAR. A user at
client system 1008 can enter a Uniform Resource Locator (URL) or
other address directing the web browser to a particular server
(such as server), and the web browser can generate a Hyper Text
Transfer Protocol (HTTP) request and communicate the HTTP request
to server. The server can accept the HTTP request and communicate
to client system 1008 one or more Hyper Text Markup Language (HTML)
files responsive to the HTTP request. Client system 1008 can render
a webpage based on the HTML files from the server for presentation
to the user. This disclosure contemplates any suitable webpage
files. As an example, and not by way of limitation, webpages can
render from HTML files, Extensible Hyper Text Markup Language
(XHTML) files, or Extensible Markup Language (XML) files, according
to particular needs. Such pages can also execute scripts such as,
for example and without limitation, those written in JAVASCRIPT,
JAVA, MICROSOFT SILVERLIGHT, combinations of markup language and
scripts such as AJAX (Asynchronous JAVASCRIPT and XML), and the
like. Herein, reference to a webpage encompasses one or more
corresponding webpage files (which a browser can use to render the
webpage) and vice versa, where appropriate.
[0161] In particular embodiments, digital content survey system
1004 can be a network-addressable computing system that can host an
online chat network. Digital content survey system 1004 can
generate, store, receive, and send chat question, chat response
data, such as, for example, user-profile data, concept-profile
data, text data, or other suitable data related to the online chat
network. Digital content survey system 1004 can be accessed by the
other components of network environment 1000 either directly or via
network 1006. In particular embodiments, digital content survey
system 1004 can include one or more servers. Each server can be a
unitary server or a distributed server spanning multiple computers
or multiple datacenters. Servers can be of various types, such as,
for example and without limitation, web server, news server, mail
server, message server, advertising server, file server,
application server, exchange server, database server, proxy server,
another server suitable for performing functions or processes
described herein, or any combination thereof. In particular
embodiments, each server can include hardware, software, or
embedded logic components or a combination of two or more such
components for carrying out the appropriate functionalities
implemented or supported by server. In particular embodiments,
digital content survey system 1004 can include one or more data
stores. Data stores can be used to store various types of
information. In particular embodiments, the information stored in
data stores can be organized according to specific data structures.
In particular embodiments, each data store can be a relational,
columnar, correlation, or other suitable database. Although this
disclosure describes or illustrates particular types of databases,
this disclosure contemplates any suitable types of databases.
Particular embodiments can provide interfaces that enable a client
system 1008, or a digital content survey system 1004 to manage,
retrieve, modify, add, or delete, the information stored in data
store.
[0162] In particular embodiments, digital content survey system
1004 can provide users with the ability to take actions on various
types of items or objects, supported by digital content survey
system 1004. As an example, and not by way of limitation, the items
and objects can include groups or chat networks to which users of
digital content survey system 1004 can belong, events or calendar
entries in which a user might be interested, computer-based
applications that a user can use, transactions that allow users to
buy or sell items via the service, interactions with advertisements
that a user can perform, or other suitable items or objects. A user
can interact with anything that is capable of being represented in
digital content survey system 1004 or by an external system of a
third-party system, which is separate from digital content survey
system 1004 and coupled to digital content survey system 1004 via a
network 1006.
[0163] In particular embodiments, digital content survey system
1004 can be capable of linking a variety of entities. As an
example, and not by way of limitation, digital content survey
system 1004 can enable users to interact with each other or other
entities, or to allow users to interact with these entities through
an application programming interfaces (API) or other communication
channels.
[0164] In particular embodiments, digital content survey system
1004 also includes user-generated content objects, which can
enhance a user's interactions with digital content survey system
1004. User-generated content can include anything a user can add,
upload, send, or "post" to digital content survey system 1004. As
an example, and not by way of limitation, a user communicates chats
to digital content survey system 1004 from a client system 1008.
Chats can include data such as chat questions or other textual
data, location information, photos, videos, links, music or other
similar data or media. Content can also be added to digital content
survey system 1004 by a third-party through a "communication
channel," such as a newsfeed or stream.
[0165] In particular embodiments, digital content survey system
1004 can include a variety of servers, sub-systems, programs,
modules, logs, and data stores. In particular embodiments, digital
content survey system 1004 can include one or more of the
following: a web server, action logger, API-request server,
relevance-and-ranking engine, content-object classifier,
notification controller, action log,
third-party-content-object-exposure log, inference module,
authorization/privacy server, search module,
advertisement-targeting module, user-interface module, user-profile
store, connection store, third-party content store, or location
store. Digital content survey system 1004 can also include suitable
components such as network interfaces, security mechanisms, load
balancers, failover servers, management-and-network-operations
consoles, other suitable components, or any suitable combination
thereof. In particular embodiments, digital content survey system
1004 can include one or more user-profile stores for storing user
profiles. A user profile can include, for example, biographic
information, demographic information, behavioral information,
social information, or other types of descriptive information, such
as work experience, educational history, hobbies or preferences,
interests, affinities, or location.
[0166] The web server can include a mail server or other messaging
functionality for receiving and routing messages between digital
content survey system 1004 and one or more client systems 1008. An
action logger can be used to receive communications from a web
server about a user's actions on or off digital content survey
system 1004. In conjunction with the action log, a
third-party-content-object log can be maintained of user exposures
to third-party-content objects. A notification controller can
provide information regarding content objects to a client system
1008. Information can be pushed to a client system 1008 as
notifications, or information can be pulled from client system 1008
responsive to a request received from client system 1008.
Authorization servers can be used to enforce one or more privacy
settings of the users of digital content survey system 1004. A
privacy setting of a user determines how particular information
associated with a user can be shared. The authorization server can
allow users to opt in to or opt out of having their actions logged
by digital content survey system 1004 or shared with other systems,
such as, for example, by setting appropriate privacy settings.
Third-party-content-object stores can be used to store content
objects received from third parties. Location stores can be used
for storing location information received from client systems 1008
associated with users.
[0167] In the foregoing specification, the invention has been
described with reference to specific exemplary embodiments thereof.
Various embodiments and aspects of the invention(s) are described
with reference to details discussed herein, and the accompanying
drawings illustrate the various embodiments. The description above
and drawings are illustrative of the invention and are not to be
construed as limiting the invention. Numerous specific details are
described to provide a thorough understanding of various
embodiments of the present invention.
[0168] The present invention can be embodied in other specific
forms without departing from its spirit or essential
characteristics. The described embodiments are to be considered in
all respects only as illustrative and not restrictive. For example,
the methods described herein can be performed with less or more
steps/acts or the steps/acts can be performed in differing orders.
Additionally, the steps/acts described herein can be repeated or
performed in parallel with one another or in parallel with
different instances of the same or similar steps/acts. The scope of
the invention is, therefore, indicated by the appended claims
rather than by the foregoing description. All changes that come
within the meaning and range of equivalency of the claims are to be
embraced within their scope.
* * * * *