U.S. patent application number 14/682648 was filed with the patent office on 2015-07-30 for perpetual memoire.
This patent application is currently assigned to Forget You Not, LLC. The applicant listed for this patent is Forget You Not, LLC. Invention is credited to Richard J.W. Mansfield, Daniel Shani, Nissim Shani.
Application Number | 20150215242 14/682648 |
Document ID | / |
Family ID | 51533504 |
Filed Date | 2015-07-30 |
United States Patent
Application |
20150215242 |
Kind Code |
A1 |
Mansfield; Richard J.W. ; et
al. |
July 30, 2015 |
Perpetual Memoire
Abstract
Among other things, an intelligent automated proxy for an party
who wants to communicate a multimedia message about a life event an
integrated, conversational manner using natural language dialog to
a second party and provides information from which the life event,
the communication, the manner of delivery, the future time, and the
recipient can be determined.
Inventors: |
Mansfield; Richard J.W.;
(Cambridge, MA) ; Shani; Nissim; (Waban, MA)
; Shani; Daniel; (Waban, MA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Forget You Not, LLC |
Waban |
MA |
US |
|
|
Assignee: |
Forget You Not, LLC
Waban
MA
|
Family ID: |
51533504 |
Appl. No.: |
14/682648 |
Filed: |
April 9, 2015 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
13866748 |
Apr 19, 2013 |
9043423 |
|
|
14682648 |
|
|
|
|
61790789 |
Mar 15, 2013 |
|
|
|
Current U.S.
Class: |
709/206 |
Current CPC
Class: |
G06Q 10/02 20130101;
G06Q 10/10 20130101; H04L 51/02 20130101; H04L 51/10 20130101; H04L
67/325 20130101; H04L 51/32 20130101; H04L 67/18 20130101 |
International
Class: |
H04L 12/58 20060101
H04L012/58; H04L 29/08 20060101 H04L029/08 |
Claims
1. A method comprising: maintaining first information
characterizing a person; receiving second information indicative of
contents of a message to be delivered to a recipient on behalf of
the person, an identity of the recipient, and a delivery time for
delivery of the message; and delivering the message to the
recipient at the delivery time, including conducting a natural
language interaction with the recipient on behalf of the person
based on the first information characterizing the person.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of and claims priority to
U.S. application Ser. No. 13/866,748, filed Apr. 19, 2013, which
claims priority to U.S. Provisional Application Serial No.
61/790,789, filed Mar. 15, 2013, the contents of both of which are
incorporated here by reference.
[0002] This application is related to U.S. patent application Ser.
No. 13/489,210, filed Jun. 5, 2012; and to U.S. patent application
Ser. No. 13/650,941, filed Oct. 12, 2012; and to patent application
Ser. No. 13/715,517, filed Dec. 14, 2012, the entire contents of
all three of which are incorporated here by reference.
BACKGROUND
[0003] This description relates generally to social network
environments, more particularly, to systems and methods for
efficient and effective search for and communication of life
history information to present and future generations.
SUMMARY
[0004] In general, in an aspect, a method includes maintaining
first information characterizing a person; receiving second
information indicative of contents of a message to be delivered to
a recipient on behalf of the person, an identity of the recipient,
and a delivery time for delivery of the message; and delivering the
message to the recipient at the delivery time, including conducting
a natural language interaction with the recipient on behalf of the
person based on the first information characterizing the
person.
[0005] Implementations may include one or a combination of any two
or more of the following features.
[0006] The message includes a multimedia message.
[0007] Delivering the message includes delivering the message using
a digital representation of the person. The method includes
generating the digital representation of the person based on the
first information characterizing the person. The digital
representation of the person includes a representation of one or
more of the following: a voice of the person, a behavior of the
person, and an appearance of the person.
[0008] Conducting a natural language interaction includes
responding to a question or statement from the recipient.
Responding to the question or statement from the recipient includes
responding to the question or statement with a natural language
response. The method includes forming a response to the question or
statement based on the first information characterizing the
person.
[0009] The method includes generating the message based on the
first information, the second information, or both.
[0010] The second information includes a specification of the
identity of the recipient.
[0011] The second information includes a specification of a
relationship between the person and the recipient.
[0012] The method includes identifying the recipient based on the
second information indicative of the identity of the recipient.
[0013] The second information includes a specification of the
delivery time.
[0014] The second information indicative of the delivery time
includes an identification of an event. The second information
indicative of the delivery time includes a definition of a
condition for occurrence of the event.
[0015] The method includes identifying the delivery time based on
the second information indicative of the delivery time.
[0016] The second information includes a specification of the
contents of the message.
[0017] The method includes determining the contents of the message
based on the second information indicative of the contents of the
message.
[0018] The contents of the message include autobiographical
information about the person.
[0019] The second information is indicative of a manner of delivery
of the message.
[0020] The person is not alive or not competent at the time when
the message is delivered.
[0021] The recipient is not alive or not competent at the time when
the second information is received.
[0022] The method includes receiving at least some of the first
information characterizing the person from the person. Receiving
the first information comprises: presenting a question to the
person and receiving an answer to the question from the person.
Receiving the first information includes conducting a natural
language interaction with the person.
[0023] The method includes obtaining at least some of the first
information characterizing the person from an automated analysis of
publicly available online information.
[0024] The first information characterizing the person includes
information about one or more of the following: a voice of the
person, a behavior of the person, and an appearance of the
person.
[0025] The first information characterizing the person includes
information about a time period during which the person lived.
[0026] The first information characterizing the person includes
autobiographical information about the person.
[0027] In general, in an aspect, a method includes receiving first
information characterizing a person and second information
indicative of an identity of a recipient; and conducting a natural
language interaction with the recipient using a digital
representation of the person that is generated based on the first
information, in which the person is not alive or not competent when
the natural language interaction is conducted.
[0028] Implementations may include one or a combination of any two
or more of the following features.
[0029] Conducting the natural language interaction includes
conducting the natural language interaction based on the first
information.
[0030] The method includes determining a context of the natural
language interaction based on the first information. The context of
the natural language interaction includes a time at which the
natural language interaction is to be conducted, a style of the
natural language interaction, or both.
[0031] The method includes determining a content of the natural
language interaction based on the first information. The content of
the natural language interaction includes autobiographical
information about the person.
[0032] The digital representation of the person includes a
representation of one or more of the following: a voice of the
person, a behavior of the person, and an appearance of the
person.
[0033] Conducting the natural language interaction includes
responding to the recipient.
[0034] The method includes determining a response to the recipient
based on the first information.
[0035] The method includes identifying the recipient based on the
second information.
[0036] The recipient is not alive or not competent when the second
information is received.
[0037] In general, in an aspect, a system includes a storage device
for maintaining first information characterizing a person; and an
analytics engine for receiving second information indicative of
contents of a message to be delivered to a recipient on behalf of
the person, an identity of the recipient, and a delivery time for
delivery of the message, and delivering the message to the
recipient at the delivery time, the analytics engine configured to
conduct a natural language interaction with the recipient on behalf
of the person based on the first information characterizing the
person.
[0038] Implementations may include one or a combination of any two
or more of the following features.
[0039] The analytics engine is configured to generate a digital
representation of the person for display on a user interface.
[0040] The analytics engine is configured to respond to a question
or statement from the recipient with a natural language
response.
[0041] The system includes a communication engine configured to
identify the recipient based on the second information.
[0042] In general, in an aspect, a system includes a communication
engine for receiving first information characterizing a person and
second information indicative of an identity of a recipient; and an
analytics engine for conducting a natural language interaction with
the recipient using a digital representation of the person that is
generated based on the first information. The person is not alive
or not competent when the natural language interaction is
conducted.
[0043] Implementations may include one or a combination of any two
or more of the following features.
[0044] The analytics engine is configured to generate the digital
representation of the person for display on a user interface.
[0045] In general, in an aspect, information is received
information from which at least one of the following can be
derived: content of a message to be delivered from a first party to
a second party about a life event, a manner of delivery of the
message, a future time when the message is to be delivered, and the
recipient of the message. The, on behalf of the first party, an
integrated, conversational multimedia message is automatically
formed that is to be part of a natural language dialog with the
second party.
[0046] Implementations may include one or a combination of any two
or more of the following features.
[0047] The integrated, conversational multimedia message is
automatically formed by an intelligent proxy on behalf of the first
party.
[0048] The integrated, conversational multimedia message is formed
to be responsive to natural language communication of the second
party. The intelligent proxy enables a user to interact with online
services to obtain information related to the message.
[0049] The automatic forming of the message includes imbuing a
digital surrogate with qualities associated with the first party,
the manner of delivery of the message, the time of delivery of the
message, or the recipient of the message.
[0050] The information is received interactively. The information
is received interactively by an interrogation avatar.
[0051] The information is received in the form of images, video,
voice, or behavioral gestures of the party from whom the messages
to be delivered, or combinations of any two or more of them.
[0052] The message is formed to provide an answer to a
conventionally asked question.
[0053] The natural language dialog includes an interactive obituary
or a digitized life history.
[0054] In general, in an aspect, digital information is maintained
that enables the formation of an interactive digital surrogate of
an originating person based on digital audio or video information
derived from the originating person. At a time when the originating
person is not alive or otherwise not available, the digital
surrogate is caused to engage in a natural language dialog with a
receiving person who is determined on the basis of information that
had been provided by the originating person.
[0055] Implementations may include one or a combination of any two
or more of the following features.
[0056] The context of the natural language dialog is determined
based on information that had been provided by the originating
person. The context includes the time of the natural language
dialog or the style of the natural language dialog.
[0057] In general in an aspect an intelligent automated proxy for
an individual is implemented on an electronic device, to facilitate
user interaction with the individual's life story in an integrated,
conversational manner using natural language dialog. In an aspect
the intelligent automated proxy is programmed to enable the
individual to more effectively engage with local and remote
services to obtain information and perform various actions at
future times.
[0058] These and other aspects, features, implementations, and
advantages, and combinations of them, can be expressed as methods,
apparatus, systems, components, program products, business methods,
and means or steps for performing functions, or combinations of
them.
[0059] Other features, aspects, implementations, and advantages
will become apparent from the description, the drawings, and the
claims.
DESCRIPTION
[0060] FIG. 1 is a flowchart for developing an intelligent avatar
for life history recording, transformation, and publishing in
digital interactive format.
[0061] FIG. 2 is a flowchart for imparting advance directives to a
intelligent surrogate for execution at a future time.
[0062] FIG. 3 is a block diagram of a social network or other
system.
[0063] FIG. 4 is a view of an editing interface for avatar
construction and surrogate development.
[0064] FIG. 5 is a view of an editing interface for entering
surrogate directives.
[0065] FIG. 6 is a block diagram of an Analytics Engine.
[0066] FIG. 7 is a block diagram of a Communication Engine.
[0067] FIG. 8 is a block diagram of a Notification Engine.
[0068] FIG. 9 is a view of the recipient interface.
[0069] FIG. 10 is a flowchart for delivering presentations.
[0070] FIG. 11 is flowchart for delivering gifts.
[0071] FIG. 12 is a flowchart for managing an event directive.
[0072] Devices that are in communication with each other need not
be in continuous communication with each other, unless expressly
specified otherwise. In addition, devices that are in communication
with each other may communicate directly or indirectly through one
or more intermediaries.
[0073] A description of an embodiment with several components in
communication with each other does not imply that all such
components are required. To the contrary, a variety of optional
components are described to illustrate the wide variety of possible
embodiments of the invention.
[0074] Further, although process steps, method steps, algorithms or
the like may be described in a sequential order, such processes,
methods and algorithms may be configured to work in alternate
orders. In other words, any sequence or order of steps that may be
described in this patent application does not in and of itself,
indicate a requirement that the steps be performed in that order.
The steps of described processes may be performed in any order
practical. Further, some steps may be performed simultaneously
despite being described or implied as occurring non-simultaneously
(e.g., because one step is described after the other step).
Moreover, the illustration of a process by its depiction in a
drawing does not imply that the illustrated process is exclusive of
other variations and modifications thereto, does not imply that the
illustrated process or any of its steps are necessary to the
invention, and does not imply that the illustrated process is
preferred.
[0075] When a single device or article is described, more than one
device/article (whether or not they cooperate) may be used in place
of a single device/article. Similarly, where more than one device
or article is described (whether or not they cooperate), a single
device/article may be used in place of the more than one device or
article.
[0076] Techniques and mechanisms described or reference herein will
sometimes be described in singular form for clarity. However,
particular embodiments include multiple iterations of a technique
or multiple instantiations of a mechanism unless noted
otherwise.
[0077] The system described here enables members of a community of
users on a network to imbue a surrogate, which is matched in
digital appearance, voice, and manner, with their personal
knowledge, expertise and discernment in order to carry out the
user's directives and intent at some predetermined future time. For
example, a senior user with access to a social media network to
embed his life story in digital format together with a personalized
digital avatar to present that life story or portions thereof in
response to natural language questions and using an intelligent
surrogate to act on the senior's behalf to carry out the senior's
directives in the future after the senior dies or becomes
incompetent.
[0078] FIG. 1 shows the conceptual framework 100 of the interaction
of the system with the Principal Person 102 for extracting and
transforming a life history. The Principal Person 102 logs onto the
system and answers some standard identification questions, 104, so
the system aided by the Analytic Engine 106 can search for relevant
internee accessible information on the individual and events that
occurred during his lifetime found in external data sources 106A or
currently in internal data sources 106B. The system then progresses
to collecting answers, 108 to autobiographical questions, which
cover aspects usually published in a biographical note or obituary
article. To assist the Principal Person 102 in answering the
questions and to be consistent with the historical record, the
system aided by the Analytic Engine 106 assembles material from
internal data sources 1068 or interact accessible data sources 106A
about the Principal Person 102 or the events and presents that
informational material along with the questions. The image of the
Principal Person 102 along with his verbal answers are collected
digitally by the system 108. The system then progresses to more
personal, individual questions presented by an Interrogation Avatar
during a session of Elaboration 112 with whom the Principal Person
102 is relaxed and comfortable. The Interrogation Avatar, which is
generated by the Analytics Engine 106 using techniques similar to
those for LivingActor.TM. and can be selected by gender, age,
ethnicity, voice type and other variables, is programmed with
natural language understanding similar to that of Apple's SIRI or
Nuance Communications Systems' Nuance Interactive Natural Assistant
(Nina.TM.), and exhibit a number of behavioral expressions as do
those avatars of LivingActor.TM. in response to the answers of the
Principal Person 102. This Elaboration 112 of autobiographical
information then uses a third set of open-ended questions presented
by the Interrogation Avatar to elicit life stories, such as those
recorded by Story Corps, covers more personal topics, such as "What
is you earliest memory?", "Who was your favorite teacher?" "What is
your favorite flavor of ice cream?", "What was it like growing up
in [neighborhood, city, region, wartime, depression]? The
Elaboration 112 using the Analytic Module 106 can interactively
expand upon the questions along the lines of Weizenbaum's ELIZA
program but in a more sophisticated manner guiding the Principal
Person 102 then a series of life experiences. The responses of the
Principal Person 102 are recorded digitally both in terms of voice,
and three-dimensional appearance with behavioral gestures.
[0079] The system aided by the Analytic Engine 106 then creates a
surrogate of the Principal Person 102 consisting of an avatar of
the Principal Person 102 linked with the personal knowledge of the
Principal Person 102 and the ontology of his world at that point in
time. The avatar of the Principal Person is generated using the
digitized three-dimensional video image of the Principal Person 102
parsed into behavioral expressions employed to morph the avatar's
image, for example, facial expressions of the Principal Person 102
using software such as that of FaceShift.com and employing the
Principal Person's voice to generate the avatar's voice in pitch
and other sound attributes using articulatory synthesis and a
database of speech units, which may amount to a billion or more for
responding to arbitrary questions but some life stories or jokes
may be recorded and used in their entirety.
[0080] The accuracy of the Surrogate Transformation is assessed 114
by the Principal Person 102 or a Second Person. The Principal
Person 102 or a Second Person then interrogates the system's
Surrogate of the Principal Person using a script of frequently
asked conventional questions. If there are gaps or errors the
process iterates until the performance of the Surrogate of the
Principal Person reaches a predetermined level of correctness in
terms of this limited Turing Test.
[0081] As a result of the training and testing, the system enables
the Surrogate of the Principal Person 102 to answer general and
specific questions by means of an Avatar matching in appearance,
voice, manner and gesture the Principal Person 102 and by means of
the Analytic Module 106 equipped with a digital representation of
the transformed personal knowledge in the form of an ontology
covering the lifetime of the Principal Person 102 using the
techniques described by Gruber et al. (U.S. patent application Ser.
No. 12/987,982, filed Jan. 10, 2011, the contents of which are
incorporated herein by reference) to include an automated assistant
receiving user input; an active ontology with representations of
concepts and relations among concepts drawn from various databases
of historical data and a language interpreter parsing the
questioner's input to derive a representation of questioner's
intent in terms of the active ontology.
[0082] Using the Communications Module 110 and the Analytic Module
106 the transformed digital representation of the Principal Person
102 is edited 116 for particular formats and audiences then
communicated 118 when appropriate to that audience 120. So the
Surrogate of the Principal Person can be used to generate an
interactive obituary, a digitized life history in the Story Corps
sense or as a published autobiography using the book assembly
techniques of Parker (Philip M. Parker, U.S. Pat. No. 7,266,767 B2,
Sep. 4, 2007 Method and Apparatus for Automated Authoring and
Marketing, which is incorporated by reference) or serve in place of
the Principal Person 102 at a memorial service or at subsequent
anniversaries or on special occasions.
[0083] It will be apparent, however, to one skilled in the art,
that the framework described may be practiced using only voice
communication or text communication and or more broadly for
expertise in various domains. For example, a business may wish to
deploy the expertise of one of its employees to several remote
location simultaneously interactively or in the form of a manual or
multimedia-based expert system.
[0084] FIG. 2 shows the conceptual framework 200 for the entering
and execution of advance directives for future actions. The
Principal Person 102 logs in to his account for Identification 204,
to access identification information and all the information known
or inferred by the Analytic Module that has been collected,
organized and transformed into an intelligent personal assistant
that can act as a Surrogate when the Principal Person 102 becomes
incompetent. The advance directives enable the system acting as the
Surrogate of the Principal Person to compose and deliver messages,
to purchase gifts and arrange for their delivery to living persons
specifically identifiable at the time the directives were entered
or to persons unborn or not yet part of the social network; and to
convene and organize social gatherings to similarly chosen guests.
The Principal Person 102 initiates directives with a process of
Selection 206 of the occasion or event chosen from a set provided
by the Analytic Module 106, then a Designation 210 of the person or
persons to be contacted and using the Notification Module specifies
a future date of Notification 214 or a means to determine a future
date to be notified. When the system detects the said date has
occurred the Communication Module 110 aided by Analytic Module 106
initiated a Notification 216 of the person or person previously
selected and manage carrying out the directives 218. For example
the system with the aid of the Analytic Module 106 which can access
an ontology of future time, can determine the birthday of selected
individuals, analyze with the aid of the Communication Module 110
their preferences from their profiles then contact a vendor for a
suitable gift and arrange for its delivery. In some embodiments,
the Analytics Engine acts as a surrogate of a Principal Person 102
to generate responses (e.g., messages, and/or delivery
instructions) based on historical data specific to the Recipient
220. For instance, the Analytics Engine may be configured as
described by Gruber et al. (U.S. patent application Ser. No.
12/987,982, filed Jan. 10, 2011, the contents of which are
incorporated herein by reference) and by Basea-Forte et al (U.S.
patent application Ser. No. 13/492,809 to include an automated
assistant receiving user input; an active ontology with
representations of concepts and relations among concepts drawn from
various databases of historical; and a language interpreter parsing
the publisher user input to derive a representation of Principal
Person's intent in terms of the active ontology together with a
services orchestration to output responses and instructions to
implement the Principal Person's intent.
[0085] It will be apparent, however, to one skilled in the art,
that the framework described may be practiced for more than
communicating multimedia messages at future times, delivering gifts
to relatives or friends on future occasions, or organizing future
meetings and presentations. For example, a business in the event of
an emergency may wish to deploy the expertise of one of its
employees to several remote locations simultaneously interactively
or in the form of a manual or multimedia-based expert system.
[0086] FIG. 3 is an exemplary configuration of a system 300 for
embodiment of the present invention. The Principal Person 102 by
means of his computing device 306, which is connected to the
Internet 302, is enabled to generate a representational Surrogate
imbued with his personal knowledge and capable of discerning his
intent for executing directives for future actions. The Internet
302 further connects to the Analytic Engine 106, the Communication
Engine 110, the Notification Engine 208 and the services of a
Vendor 310. These engines and services draw up internet-accessible
databases on the network 304, which include the data and
relationship supporting the lifetime ontology of the Principal
Person 102 as well as the present and future ontologies in which
the directives of the Principal Person 102 will be carried out. The
communications, interactive presentations and directives of the
Principal Person 102 are targeted at internet-enabled Recipients
312.
[0087] FIG. 4 shows an exemplary embodiment of the Editing
Interface 400 presented to the Principal Person 102. The
internet-accessible computing device 402 is a means of presenting
an Avatar 410, which can be the interrogating avatar for
interactively collecting life history information from the
Principal Person 102 or the avatar of the Principal Person to
testing determine if the system surrogate has met a predetermined
criterion of correctness. The Window Title 404 indicates the stage
of surrogate development. The website is noted by 406. The video
start button 408 can be clicked or commanded verbally to start the
session. When the system surrogate has met the predetermined
criterion of correctness the Principal Person 102 can proceed to
set up directives future actions.
[0088] FIG. 5 is a view of an editing interface 500 for entering
surrogate directives. The internet-accessible computing device 502
is a means of presenting an Avatar of 510 to receive instructions
by natural language dialog for executing future actions. The
Windows title 506 indicates the type of directive being edited. The
website URL is indicated by 504. A progress bar 508 indicates the
steps completed and still to be completed
[0089] FIG. 6 is a block diagram of the Analytics Engine 106 of the
system 300. The Analytics Engine 106 facilitates identifying
notification trigger events for significant event persons, curating
an appropriate message upon notification, then publishing or
delivery of communications in response to a significant event on
behalf of Principal Person 102 by inferring information about the
person described in the communication item from one or more data
sources or determining or inferring milestone events for the
recipient user 314 or for the person described in the communication
item, among other things. The person described in the communication
item may be the intended reader of the communication item, the
publisher of the communication item, both the reader and the
publisher, or neither the reader nor the publisher.
[0090] A predictive analytics module 602 in Analytics Engine 106
applies search algorithms and forecasting models stored in a search
and forecasting models library 606 to analyze present and/or
historical subject user data in view of data stored in a database
608 containing data related to identify milestone events and to
suggest appropriate media messages or items for a reader user. The
predictive analytics module 602 implements a variety of forecasting
techniques beyond simple algorithms, such as future date
calculation, including statistical techniques such as machine
learning (e.g., as applied by IBM's Watson computer), game theory,
and data mining, to analyze current and historical data to
determine significant event triggers for notification, to make
predictions about future readers, to identify communication media
messages, and to identify appropriate delivery times and methods,
among a wide variety of other analyses and algorithms. For example
in the case of an autobiographical communication item of a
grandfather (the publisher user), the reader user is the projected
offspring of a current member in the system but the publisher user
may incapacitated at the time of publication and delivery or both.
By a computer-implemented method, data relevant to the publisher
user may be extracted from his profile in the system 300 and/or
generated implicitly based, at least in part, on the publisher
user's stored profile together with historical data by a predictive
analytics algorithm based on historical data. Based on the
extracted or generated data, a significant event is identified,
content for a communication is selected, appropriate reader users
are identified, and/or other analysis is performed, in some cases
without input from the publisher user.
[0091] In some examples, Analytics Engine 106 may use data specific
to a user to identify potential milestone events (e.g., to identify
a bar mitzvah date for a child based on the child's birth date or
to identify appropriate religious holidays based on the user's
specified religious affiliation or a prevailing religion in the
region where the user lives). In some cases, the predictive
analytics module 602 may also incorporate present or historical
data or both to identify relevant information for the communication
subject or reader user or both that are in keeping with preferences
of the publisher user. In other examples, the Analytics Engine 106
analyzes publically available sources of information and/or private
sources of information to which it has access, including, e.g.,
social networking facilities, online or mobile periodicals such as
newspapers or magazines, and websites, to identify the occurrence
of significant events associated with the reader user or with the
person whom the communication is about. For instance, the Analytics
Engine 106 may analyze trade journal websites to set up criteria so
the Notification Engine 208 can identify when a target person
receives an award or a promotion.
[0092] A display module 504 of the Analytics Engine 106 may display
the significant events of an individual of interest (e.g., a reader
user or another individual specified by the publisher user) on a
webpage belonging to or accessible by the publisher user. These
events with associated actions can be selected for notifications
stored in the Notification Engine 208. In some embodiments, upon
notification by the Notification Engine, the Analytics Engine 106
may automatically communicate messages of a significant event
relevant to a reader user, such as a birthday, to the Communication
Engine 110, which sends the notices to relatives of the reader
user, who can act as publisher users. The Analytics Engine 106 may
use information about a reader user found in various databases to
determine when to send out the notices.
[0093] In some embodiments, the predictive analytics implemented by
the Analytics Engine 106 incorporate the robust, optimizing
forecasting techniques of Pinto et al. (U.S. Pat. No. 7,499,897,
issued on Mar. 3, 2009: U.S. Pat. No. 7,562,058, issued on Jul. 14,
2009; U.S. Pat. No. 7,725,300, issued on May 25, 2010; U.S. Pat.
No. 7,730,003, issued on Jun. 1, 2010; U.S. Pat. No. 7,933,762,
issued on Apr. 26, 2011; U.S. Pat. No. 8,165,853. issued Apr. 24,
2012; U.S. Pat. No. 8,170,841, issued May 1, 2012; and U.S. patent
application Ser. No. 10/826,949, filed Apr. 16, 2004, the contents
of all of which are incorporated herein by reference), that manage
historical data using missing values, which must be inferred.
[0094] The Analytics Engine 106 can act as a surrogate for the
Principal Person 102 by specifying communications, goods, and
services or any other communication items for placement into
storage. Provided the potential reader user 314 is registered in
the system 300, the Communication Engine 110 will determine the
location of the reader user, e.g., by GPS tracking or similar
means. The reader user's interests and status may be available in
the reader user's profile, which, taken together with historical
data, enable the Analytics Engine 106 utilizing inferences from
missing data, if necessary, to determine an appropriate gift to
accompany the message and an optimal time for delivery. The
delivery time is selected by a predictive analytics algorithm,
based on historical data specific to the reader user.
[0095] In some embodiments, the Analytics Engine 106 acts as a
surrogate of a Principal Person 102 to generate communication items
(e.g., messages, sympathy tribute selections, and/or delivery
instructions) based on data, e.g., historical data, specific to the
reader user. For instance, the Analytics Engine may be configured
as described by Gruber et al. (U.S. patent application Ser. No.
12/987,982, filed Jan. 10, 2011, the contents of which are
incorporated herein by reference) to include an automated assistant
receiving user input; an active ontology with representations of
concepts and relations among concepts drawn from various databases
of historical data (for instance, for the case in which the
publisher user is the agent of a corporation, the corporate
personnel database may be referenced); and a language interpreter
parsing the publisher user input to derive a representation of
publisher user intent in terms of the active ontology together with
a services orchestration to output responses and instructions to
implement the publisher user's intent. By a computer-implemented
method, data relevant to the reader user may be extracted from his
profile, generated implicitly based, at least in part, on the
reader user's stored profile with the system together with
historical data by a predictive analytics algorithm based on
historical data, and/or obtained from publically available or
privately accessible information sources. Based on the extracted or
generated data a communication is selected, generated, and
delivered, in some cases without input from the publisher user.
[0096] FIG. 7 is a block diagram 700 of a Communication Engine 110
of the system 300. The Communication Engine 110 facilitates
identification of a significant event person along with criteria
used by the Notification Engine 208 to trigger notification of the
curator for composing a message, publishing or delivery of
communication items) for a Principal Person 102 by providing
information about a reader user's physical location, profile
interest information, and/or affinity information. In some
examples, the location, interest, and affinity and other current
and historical information may be displayed by a display module 704
on a webpage belonging to or accessible by the publisher user. The
communication engine 110 includes a affinity database 702 for
storing significant event person and reader user affinity
information and a reader location database 710 for storing the
physical location of a reader user. The reader location database
710 may receive data relevant to the location of a reader user from
a GPS translation module 706 or may obtain the reader user's
location from the Analysis Engine's analysis of websites or other
sources of information. For example, the publisher user may have
composed a sympathy note regarding an individual killed in an
accident; the Communication engine determines that the next of kin
is traveling in another country and thus likely to be reachable by
email but not by phone.
[0097] FIG. 8 is a block diagram 800 of a Notification Engine 208
of the system 300. The Notification Engine 208 facilitates
publishing or delivery of communications for a Principal Person 102
by detecting significant events of interest to the Principal Person
102 as listed in a data file 802. For example, a publisher user may
be notified about the death of friend by means of a search module
804 supported by a parsing and interpretation library 808. The
publisher user may then compose and publisher or deliver a sympathy
message to the friend's family. In some examples, information the
individual whose significant event has been detected (referred to
here as the tracked individual) may be displayed by a display
module 806 on a webpage belonging to the publisher user. The.
Notification Engine 108 may also display the tracked individual's
profile within the system 300 for confirmation of the individual's
significant event.
[0098] FIG. 9 is a view 900 of a recipient interface. The
internet-connected computing device 902 enables a multimedia
display that shows an Avatar of the Principal Person 910 programmed
to deliver answers to natural language inquiries using the detailed
knowledge of the Principal Person 102 previously collected, curated
and transformed.
[0099] FIG. 10 is a flowchart of an example process 1000 for
communicating a multimedia message in the system 300. In the
process 1000, a Principal Person 102 has directed his surrogate
with an occasion and an specific audience (Step 1002). The
Surrogate using the Communication Engine (110) requests the
profiles of the audience members (Step 1004). The Analytic Engine
then provides event constraints for the communication (Step 1006).
Using the Notification Engine 208 there is a pause for the first
question of request (Step 1008). When the question or request is
received the intent is determined by the Analytic Engine (Step
1012) and the response is prepared (Step 1014). The Communication
Engine using the Surrogate Avatar delivers the communication (Step
1016). Then there is a pause for follow-up (Step 1018) and the
cycle repeats until the Notification Engine 208 determines the
presentation is complete. Then there is a closure of the
presentation (Step 1020). The audience may be given the opportunity
to acknowledge receipt and confirm delivery and even reply to the
Principal Person 102 or to his proxy.
[0100] FIG. 11 is a flowchart of an example process 1100 for
distributing a gift or tribute in the system 300. In the process
1100, a Principal Person 102 has selected or set up a directive to
select a recipient person (Step 1102) and a particular life cycle
event (Step 1104) and using the communication engine for profile of
the person and analytic engine, for predictive forecasting a gift
or tribute is chosen (Step 1106). Then there is a pause (Step 1108)
but at the appropriate date as determined by the notification
engine (208) (Step 1110) the recipient is located by means of the
communication engine (110) (Step 1112) using the current recipient
profile, for location (Step 1114). Under the directives the system
makes arrangement with a vendor (Step 1116) using the analytics
engine (106) for vendor selection (Step 1118) and delivery time
selection (1120). The gift vendor or another vendor then delivers
the gill to the recipient (Step 1122). The recipient 3141 may be
given the opportunity to acknowledge receipt and confirm delivery
and to reply to the Principal Person 102 or the publisher user's
proxy.
[0101] FIG. 12 shows a flowchart 1200 of the management of an
advance event directive planned by the Principal Person 102 The
Principal Person selects the type of event to host (Step 1202) and
selects the attendees either directly (Step 1204) or by description
using profiles if available (Step 1206). The Principal Person 102
then sets the date of the event or some triggering criterion, e.g.,
birthday party or graduation party (Step 1208) and the venue
directly or by description, e.g., function room at Waldorf Astoria
in New York (Step 1212). Based on the guest's profiles, the Proxy
Surrogate selects the menu and caterer which may be the regular
venue caterer (Step 1214). In advance of the appropriate date or
immediately after notice of the triggering event, the Proxy
Surrogate of the Principal Person sends out the invitations (Step
1216) using a service such as EventBrite.TM. to collect RSVPs (Step
1220) then confirms the appropriate venue and catering arrangements
(Step 1222). Then the Proxy Surrogate arranges for an
internet-accessible telepresence display (Step 1224). On the date
of the event, the Proxy Surrogate uses the telepresence display to
attend the event and greet, the attendees and during the event the
Proxy Surrogate interacts with attendees. At the end of the end the
Proxy Surrogate thanks the attendees, coordinates with the caterer
and venue provider and shuts off the telepresence display.
[0102] As desired, the life cycle event social network may include
more or fewer than the components illustrated.
[0103] The life cycle event social network is described above with
reference to block and flow diagrams of systems, methods,
apparatuses, and/or computer program products according to
examples. In some instances, the publisher and reader users may
access the social network by desktop or laptop computers. In some
embodiments, the publisher and reader users may access the social
network by mobile devices such as smart phones. In some
embodiments, the publisher and reader users may access the social
network by tablet computers or any commercial computing device
connected to the internet, in some cases, the social network may be
constructed to operate on the internet independent of existing
social networks. The life cycle event social network may operate
using existing social networks, e.g., Facebook, Google+, or Yammer
as platforms using existing application interfaces open to website
developers.
[0104] Generally, the intelligent automated assistant techniques
disclosed herein may be implemented on hardware or a combination of
software and hardware. For example, they may be implemented in an
operating system kernel, in a separate user process, in a library
package bound into network applications, on a specially constructed
machine, or on a network interface card. In a specific embodiment,
the techniques disclosed herein may be implemented in software such
as an operating system or in an application running on an operating
system.
[0105] Software/hardware hybrid implementation(s) of at least some
of the intelligent automated assistant embodiment(s) disclosed
herein may be implemented on a programmable machine selectively
activated or reconfigured by a computer program stored in memory,
Such network devices may have multiple network interfaces which may
be configured or designed to utilize different types of network
communication protocols. A general architecture for some of these
machines may appear from the descriptions disclosed herein.
According to specific embodiments, at least some of the features
and/or functionalities of the various intelligent automated
assistant embodiments disclosed herein may be implemented on one or
more general-purpose network host machines such as an end-user
computer system, computer, network server or server system, mobile
computing device (e.g., personal digital assistant, mobile phone,
smartphone, laptop, tablet computer, or the like), consumer
electronic device, music player, or any other suitable electronic
device, router, switch, or the like, or any combination thereof in
at least some embodiments, at least some of the features and/or
functionalities of the various intelligent automated assistant
embodiments disclosed herein may be implemented in one or more
virtualized computing environments (e.g., network computing clouds,
or the like).
[0106] One or more blocks of the block diagrams and flow diagrams,
and combinations of blocks in the block diagrams and flow diagrams,
respectively, can be implemented by computer-executable program
instructions. Some blocks of the block diagrams and flow diagrams
may not necessarily need to be performed in the order presented, or
may not necessarily need to be performed at all, in some cases.
[0107] These computer-executable program instructions may be loaded
onto a general-purpose computer, a special-purpose computer, a
processor, or other programmable data processing apparatus to
produce a particular machine, such that the instructions that
execute on the computer, processor, or other programmable data
processing apparatus create means for implementing one or more
functions specified in the flow diagram block or blocks. These
computer program instructions may also be stored in a
computer-readable memory that can direct a computer or other
programmable data processing apparatus to function in a particular
manner, such that the instructions stored in the computer-readable
memory produce an article of manufacture including instruction
means that implement one or more functions specified in the flow
diagram block or blocks. As an example, embodiments may provide for
a computer program product, comprising a computer-usable medium
having a computer-readable program code or program instructions
embodied therein, said computer-readable program code adapted to be
executed to implement one or more functions specified in the flow
diagram block or blocks. The computer program instructions may also
be loaded onto a computer or other programmable data processing
apparatus to cause a series of operational elements or steps to be
performed on the computer or other programmable apparatus to
produce a computer-implemented process such that the instructions
that execute on the computer or other programmable apparatus
provide elements or steps for implementing the functions specified
in the flow diagram block or blocks.
[0108] Accordingly, blocks of the block diagrams and flow diagrams
support combinations of means for performing the specified
functions, combinations of elements or steps for performing the
specified functions and program instruction means for performing
the specified functions, it will also be understood that each block
of the block diagrams and flow diagrams, and combinations of blocks
in the block diagrams and flow diagrams, can be implemented by
special-purpose, hardware-based computer systems that perform the
specified functions, elements or steps, or combinations of special
purpose hardware and computer instructions.
[0109] While the life cycle event social network has been described
in connection with certain examples, is the system is not limited
to the disclosed embodiments, but on the contrary, includes various
modifications and equivalent arrangements. Embodiments of the
subject matter and the functional operations described in this
specification can be implemented in digital electronic circuitry,
in tangibly-embodied computer software or firmware, in computer
hardware, including the structures disclosed in this specification
and their structural equivalents, or in combinations of one or more
of them. Embodiments of the subject matter described in this
specification can be implemented as one or more computer programs,
i.e., one or more modules of computer program instructions encoded
on a tangible non-transitory program carrier for execution by, or
to control the operation of, data processing apparatus.
Alternatively or in addition, the program instructions can be
encoded on an artificially generated propagated signal, e.g., a
machine-generated electrical, optical, or electromagnetic signal
that is generated to encode information for transmission to
suitable receiver apparatus for execution by a data processing
apparatus. The computer storage medium can be a machine-readable
storage device, a machine-readable storage substrate, a random or
serial access memory device, or a combination of one or more of
them.
[0110] The term "data processing apparatus" encompasses all kinds
of apparatus, devices, and machines for processing data, including
by way of example a programmable processor, a computer, or multiple
processors or computers. The apparatus can include special purpose
logic circuitry, e.g., an FPGA (field programmable gate array) or
an ASIC (application specific integrated circuit). The apparatus
can also include, in addition to hardware, code that creates an
execution environment for the computer program in question, e.g.,
code that constitutes processor firmware, a protocol stack, a
database management system, an operating system, or a combination
of one or more of them.
[0111] A computer program (which may also be referred to or
described as a program, software, a software application, a module,
a software module, a script, or code) can be written in any form of
programming language, including compiled or interpreted languages,
or declarative or procedural languages, and it can be deployed in
any form, including as a standalone program or as a module,
component, subroutine, or other unit suitable for use in a
computing environment. A computer program may, but need not,
correspond to a file in a file system. A program can be stored in a
portion of a file that holds other programs or data, e.g., one or
more scripts stored in a markup language document, in a single file
dedicated to the program in question, or in multiple coordinated
files, e.g., files that store one or more modules, sub programs, or
portions of code. A computer program can be deployed to be executed
on one computer or on multiple computers that are located at one
site or distributed across multiple sites and interconnected by a
communication network.
[0112] The processes and logic flows described in this
specification can be performed by one or more programmable
computers executing one or more computer programs to perform
functions by operating on input data and generating output. The
processes and logic flows can also be performed by, and apparatus
can also be implemented as, special purpose logic circuitry, e.g.,
an FPGA (field programmable gate array) or an ASIC (application
specific integrated circuit).
[0113] Computers suitable for the execution of a computer program
include, by way of example, can be based on general or special
purpose microprocessors or both, or any other kind of central
processing unit. Generally, a central processing unit will receive
instructions and data from a read only memory or a random access
memory or both. The essential elements of a computer are a central
processing unit for performing or executing instructions and one or
more memory devices for storing instructions and data. Generally, a
computer will also include, or be operatively coupled to receive
data from or communication data to, or both, one or more mass
storage devices for storing data, e.g., magnetic, magneto optical
disks, or optical disks. However, a computer need not have such
devices. Moreover, a computer can be embedded in another device,
e.g., a mobile telephone, a personal digital assistant (PDA), a
mobile audio or video player, a game console, a Global Positioning
System (GPS) receiver, or a portable storage device, e.g., a
universal serial bus (USB) flash drive, to name just a few.
[0114] Computer readable media suitable for storing computer
program instructions and data include all forms of non volatile
memory, media and memory devices, including by way of example
semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory
devices; magnetic disks, e.g., internal hard disks or removable
disks; magneto optical disks; and CD ROM and DVD-ROM disks. The
processor and the memory can be supplemented by, or incorporated
in, special purpose logic circuitry.
[0115] To provide for interaction with a user, embodiments of the
subject matter described in this specification can be implemented
on a computer having a display device, e.g., a CRT (cathode ray
tube) or LCD (liquid crystal display) monitor, for displaying
information to the user and a keyboard and a pointing device, e.g.,
a mouse or a trackball, by which the user can provide input to the
computer. Other kinds of devices can be used to provide for
interaction with a user as well; for example, feedback provided to
the user can be any form of sensory feedback, e.g., visual feed
hack, auditory feedback, or tactile feedback; and input from the
user can be received in any form, including acoustic, speech, or
tactile input. In addition, a computer can interact with a user by
sending documents to and receiving documents from a device that is
used by the user; for example, by sending web pages to a web
browser on a user's client device in response to requests received
from the web browser.
[0116] Embodiments of the subject matter described in this
specification can be implemented in a computing system that
includes a back end component, e.g., as a data server, or that
includes a middleware component, e.g., an application server, or
that includes a front end component, e.g., a client computer having
a graphical user interface or a Web browser through which a user
can interact with an implementation of the subject matter described
in this specification, or any combination of one or more such back
end, middleware, or front end components. The components of the
system can be interconnected by any form or medium of digital data
communication, e.g., a communication network. Examples of
communication networks include a local area network ("LAN") and a
wide area network ("WAN"), e.g., the Internet.
[0117] The computing system can include clients and servers. A
client and server are generally remote from each other and
typically interact through a communication network. The
relationship of client and server arises by virtue of computer
programs running on the respective computers and having a
client-server relationship to each other.
[0118] While this specification contains many specific
implementation details, these should not be construed as
limitations on the scope of any invention or of what may be
claimed, but rather as descriptions of features that may be
specific to particular embodiments of particular inventions.
Certain features that are described in this specification in the
context of separate embodiments can also be implemented in
combination in a single embodiment. Conversely, various features
that are described in the context of a single embodiment can also
be implemented in multiple embodiments separately or in any
suitable subcombination. Moreover, although features may be
described above as acting in certain combinations and even
initially claimed as such, one or more features from a claimed
combination can in some cases be excised from the combination, and
the claimed combination may be directed to a subcombination or
variation of a subcombination.
[0119] Similarly, while operations are depicted in the drawings in
a particular order, this should not be understood as requiring that
such operations be performed in the particular order shown or in
sequential order, or that all illustrated operations be performed,
to achieve desirable results. In certain circumstances,
multitasking and parallel processing may be advantageous. Moreover,
the separation of various system modules and components in the
embodiments described above should not be understood as requiring
such separation in all embodiments, and it should be understood
that the described program components and systems can generally be
integrated together in a single software product or packaged into
multiple software products.
[0120] Particular embodiments of the subject matter have been
described. Other embodiments are within the scope of the following
claims. For example, the actions recited in the claims can be
performed in a different order and still achieve desirable results.
As one example, the processes depicted in the accompanying figures
do not necessarily require the particular order shown, or
sequential order, to achieve desirable results, in certain
implementations, multitasking and parallel processing may be
advantageous.
[0121] Other implementations are also within the scope of the
following claims.
* * * * *