U.S. patent application number 13/334065 was filed with the patent office on 2013-04-11 for mobile device as a local server.
This patent application is currently assigned to Microsoft Corporation. The applicant listed for this patent is Avi Bar-Zeev, Brian C. Beckman, Meir Ben-Itay, Scott Isaacs, Gur Kimchi, Blaise Aguera y Arcas, Eran Yariv. Invention is credited to Avi Bar-Zeev, Brian C. Beckman, Meir Ben-Itay, Scott Isaacs, Gur Kimchi, Blaise Aguera y Arcas, Eran Yariv.
Application Number | 20130091197 13/334065 |
Document ID | / |
Family ID | 48042811 |
Filed Date | 2013-04-11 |
United States Patent
Application |
20130091197 |
Kind Code |
A1 |
Bar-Zeev; Avi ; et
al. |
April 11, 2013 |
MOBILE DEVICE AS A LOCAL SERVER
Abstract
Architecture that embeds a server (a local server) inside a
mobile device operating system (OS) close to the data (but under
the OS services) such that the server has access to native
capabilities, and offers an Internet-like frontend with which a
browser or application can communicate. The local server appears as
a web server, and small programs can be pushed into the local
server from the browser or a remote server such that the local
server can be made to perform work more effectively. Local and
remote events can be triggered such as launching a browser (or
other application(s)), initiating remote server calls, triggering
battery save mode, locking the phone, etc. The local server can run
a script execution environment such as node.js, an event driven I/O
model where callbacks are invoked to handle emergent conditions
(e.g., explicit requests, state changes, etc.).
Inventors: |
Bar-Zeev; Avi; (Redmond,
WA) ; Kimchi; Gur; (Bellevue, WA) ; Beckman;
Brian C.; (Newcastle, WA) ; Isaacs; Scott;
(Bellevue, WA) ; Ben-Itay; Meir; (Sammamish,
WA) ; Yariv; Eran; (Zichron Yaakov, IL) ; y
Arcas; Blaise Aguera; (Seattle, WA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Bar-Zeev; Avi
Kimchi; Gur
Beckman; Brian C.
Isaacs; Scott
Ben-Itay; Meir
Yariv; Eran
y Arcas; Blaise Aguera |
Redmond
Bellevue
Newcastle
Bellevue
Sammamish
Zichron Yaakov
Seattle |
WA
WA
WA
WA
WA
WA |
US
US
US
US
US
IL
US |
|
|
Assignee: |
Microsoft Corporation
Redmond
WA
|
Family ID: |
48042811 |
Appl. No.: |
13/334065 |
Filed: |
December 22, 2011 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
61546049 |
Oct 11, 2011 |
|
|
|
Current U.S.
Class: |
709/203 |
Current CPC
Class: |
H04L 67/02 20130101;
H04L 67/42 20130101; H04L 67/32 20130101; G06F 1/3203 20130101;
H04L 67/34 20130101; G06F 1/3215 20130101 |
Class at
Publication: |
709/203 |
International
Class: |
G06F 15/16 20060101
G06F015/16 |
Claims
1. A system, comprising: a mobile operating system of a mobile
device having a server embedded therein, the server a computational
unit that accesses native capabilities of the mobile operating
system and emulates a web server for client application
interaction, the server operates in combination with a reactive
framework of events and callbacks; and a processor that executes
computer-executable instructions associated with the server.
2. The system of claim 1, wherein the embedded server receives
instructions from a local application, the instructions received
via a secure protocol.
3. The system of claim 1, wherein the embedded server receives
instructions from a remote web service authorized to provide the
instructions.
4. The system of claim 1, wherein the events include launch of an
application or exercise of an application function.
5. The system of claim 1, wherein the events include low-level
hardware events that originate in the operating system.
6. The system of claim 1, wherein the events and callbacks are
communicated to and from the embedded server using a scripting
language.
7. The system of claim 1, wherein the mobile device is a cellular
telephone and the server invokes phone functionality in response to
a local or remote event.
8. The system of claim 1, wherein the events are associated with
device sensors and environmental conditions that are bound to the
callbacks.
9. The system of claim 1, wherein the reactive framework employs
data storage that unifies code and data as a graph.
10. The system of claim 9, wherein the data includes personal data
that is accessible only to an authorized user.
11. A method, comprising acts of: operating a server in a mobile
device that accesses native capabilities of a mobile operating
system of the mobile device; executing an event-driven environment
on the server to process conditions and callbacks at least one of
locally or remotely; and utilizing a processor that executes
instructions stored in memory to perform the act of operating or
executing.
12. The method of claim 11, further comprising embedding the server
in the mobile operating system and emulating a web server for
client application interaction.
13. The method of claim 11, further comprising receiving
instructions into the server from a local application and via a
secure protocol.
14. The method of claim 11, further comprising receiving
instructions into the server from an authorized remote web service
via a secure protocol.
15. The method of claim 11, further comprising launching an
application in response to processing of a condition.
16. The method of claim 11, further comprising executing an
application function in response to processing of a condition.
17. The method of claim 11, further comprising storing data and
code as a graph, the graph having personal data accessible only by
an authorized user.
18. A method, comprising acts of: operating a server in a mobile
device that accesses native capabilities of a mobile operating
system of the mobile device; executing an event-driven environment
on the server to process events and callbacks at least one of
locally or remotely; communicating instructions to and from the
server via a secure protocol; invoking local functionality of at
least one of an application or a function in response to an event
using a scripting language; and utilizing a processor that executes
instructions stored in memory to perform the act of operating,
executing, communicating, or invoking
19. The method of claim 18, further comprising communicating the
instructions from a local application or a remote web service based
on hardware events, sensory events, applications, and application
functions.
20. The method of claim 18, further comprising storing code and
data in association with nodes of a storage graph.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional
Patent Application Ser. No. 61/546,049 entitled "MOBILE DEVICE AS A
LOCAL SERVER" and filed Oct. 11, 2011.
BACKGROUND
[0002] Mobile devices (e.g., mobile phones) typically operate in a
client/server relationship--the device (including the device's web
browser) is a client, and a server sends content, determines needs
of the device, and receives state changes, as needed. This misses a
principle of distributed systems in that the computation and data
reside closest to the source.
[0003] If a mobile phone, for example, is resolving geo-fences, the
data the phone needs is generally on the phone. Privacy concerns
may further indicate that this data remain on the phone. Similarly,
a phone user may need computation performed which uses cloud
information, including private information the user may not have
access to in specific detail, but a generalized result may be fine.
There again, the computation is moved to a server.
SUMMARY
[0004] The following presents a simplified summary in order to
provide a basic understanding of some novel embodiments described
herein. This summary is not an extensive overview, and it is not
intended to identify key/critical elements or to delineate the
scope thereof. Its sole purpose is to present some concepts in a
simplified form as a prelude to the more detailed description that
is presented later.
[0005] The disclosed architecture moves computation to the location
where it can be most efficient by embedding (e.g., daemon, service,
built-in) a server (a local server) inside a mobile device
operating system (OS) close to the data (but under the OS services)
such that this server has access to native capabilities, and offers
a host-like frontend with which a browser or application can
communicate. The embedded server offers web server capabilities,
but carries greater efficiencies in that small programs can be
pushed into the local server from the browser or a remote server
such that the local server can be made to perform work more
effectively. Accordingly, the server operates in combination with a
reactive framework where events trigger other actions.
[0006] The architecture enables the capability to launch "web
toast" (automatically launching a browser page of information)
where the local server can invoke mobile device (e.g., phone)
functionality such as a browser when computation determines that
this is what the user likely wants. More generally, the capability
is provided to initiate local and remote events such as launching a
browser (or other application(s)) with deep link, remote server
calls, trigger battery save mode, lock the phone, etc.
[0007] The local server can be implemented to host a script
execution environment such as node.js, an event driven I/O model
where callbacks are invoked to handle emergent conditions (e.g.,
explicit requests, state changes, etc.). In one implementation,
javascript is loaded into the node.js server for evaluation to
execute a set of callbacks when given conditions are met. For
example, a condition (e.g., described as a full javascript
evaluation function) can determine that the user is approaching a
geo-fence. Web toast, or other notification means, can then be
invoked to notify the user or launch an application, update an
"above the lock" screen, or add data to a running application
(e.g., a map) configured for this purpose, to name just a few. Web
pages, local applications, and remote servers can communicate with
the local server in a protected way (e.g., submit computation in
the form of javascript, setup new conditions that invoke callbacks
when evaluated to be true, etc.).
[0008] To the accomplishment of the foregoing and related ends,
certain illustrative aspects are described herein in connection
with the following description and the annexed drawings. These
aspects are indicative of the various ways in which the principles
disclosed herein can be practiced and all aspects and equivalents
thereof are intended to be within the scope of the claimed subject
matter. Other advantages and novel features will become apparent
from the following detailed description when considered in
conjunction with the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0009] FIG. 1 illustrates a system in accordance with the disclosed
architecture.
[0010] FIG. 2 illustrates a method in accordance with the disclosed
architecture.
[0011] FIG. 3 illustrates further aspects of the method of FIG.
2.
[0012] FIG. 4 illustrates an alternative method in accordance with
the disclosed architecture.
[0013] FIG. 5 illustrates further aspects of the method of FIG.
4.
[0014] FIG. 6 illustrates a schematic block diagram of an exemplary
multimode handset that can employ an embedded server and reactive
framework in accordance with the disclosed architecture.
[0015] FIG. 7 illustrates a block diagram of a computing system
that can execute an embedded server and reactive framework in
accordance with the disclosed architecture.
DETAILED DESCRIPTION
[0016] The disclosed architecture embeds a server (local server) on
a mobile device such as a mobile phone. Employing the local server
on a mobile device offers additional possibilities not seen
conventionally. For example, the local server on the phone can
connect to facilities that normally are not exposed to webpages on
the phone, including hardware sensor subsystems such as the
accelerometer, the compass, the speakers, the camera, etc. This
does not require modification of the web browser itself. Rather,
existing protocols can be used. In one example, rather than
requesting an image from a remote website, the image can be
requested from the phone camera. Moreover, the camera expresses a
URL (uniform resource locator) such that rich data (camera) can be
communicated back and forth between the local server (local
resources using local hosts) and remote servers.
[0017] In addition to opening access to these kinds of local
facilities synchronously or in realtime, the architecture can
facilitate an AJAX (asynchronous JavaScript.TM. and XML (extensible
markup language)) style, which includes being reactive. (JavaScript
is a scripting language that normally runs only on the client in a
web browser, and hereinafter is denoted as javascript). AJAX is a
client-side development tool for creating asynchronous web
applications that perform background asynchronous data transmit and
retrieval with a server. AJAX-type applications utilize coding
constructions called callbacks. Accordingly, rather than
implementing a program that executes synchronously as a webpage is
opened, only bindings of a number of different functions to events
that happen on that webpage execute. For example, if the mouse
hovers over a button, then a javascript function can be made to
execute to open a menu, and so on. This style of execution never
blocks and precludes code running immediately when the webpage is
opened.
[0018] This is a reactive approach to the local server 104 where
rather than querying the local server 104 for state (e.g., the
current accelerometer value or current the battery level) a
javascript function is only getting called when there is a material
change or triggering event.
[0019] The capability is enabled to push a javascript call list or
more complex functions based on complex events (e.g., given an
event when the battery is 10% and the phone is determined to be
associated with walking or running, or the phone arrives a certain
point in time, etc.). In other words, code pieces can be
dynamically downloaded for the local server 104 that run
autonomously based on complex events.
[0020] The architecture is easily and immediately updatable. The
javascript can be streamed to change requests for new notifications
and new types of notifications at any time or location. The
architecture provides the capability to benefit from the dynamic
scripting capabilities in any language between the local server 104
and remote servers (e.g., remote server 110).
[0021] Reference is now made to the drawings, wherein like
reference numerals are used to refer to like elements throughout.
In the following description, for purposes of explanation, numerous
specific details are set forth in order to provide a thorough
understanding thereof. It may be evident, however, that the novel
embodiments can be practiced without these specific details. In
other instances, well known structures and devices are shown in
block diagram form in order to facilitate a description thereof.
The intention is to cover all modifications, equivalents, and
alternatives falling within the spirit and scope of the claimed
subject matter.
[0022] FIG. 1 illustrates a system 100 in accordance with the
disclosed architecture. The system 100 includes a mobile device 102
having an embedded server (local server 104) in association with a
mobile operating system (OS) 106. The local server 104 communicates
with a mobile browser 108, as one example of an application
interaction with the local server 104. As shown, the browser 108
can communicate directly with a remote server 110. Alternatively,
the local server 104 can communicate directly with the remote
server 110. Still further, the browser 108 and the local server 104
can communicate with the remote server 110.
[0023] In other words, a local server 104 is embedded in
association with a mobile operating system of the mobile device
102. The local server 104 is a computational unit that accesses
native capabilities of the mobile OS 106 and emulates a web server
for client application interaction. The local server 104 operates
in combination with a reactive framework of events and callbacks.
In one implementation, this is facilitated via a scripting
language.
[0024] The embedded server 104 receives instructions from a local
application. The instructions can be received via a secure protocol
(e.g., encryption). The embedded server receives instructions from
a remote web service (the remote server 110) authorized (e.g., by
authentication, user login) to provide the instructions. The events
include launch of an application or exercise of an application
function. The events include low-level hardware events that
originate in the operating system 106. The events and callbacks are
communicated to and from the embedded server using a scripting
language (e.g., javascript).
[0025] The cross-loading of instructions to the local server from
the local browser or the remote web service can be accomplished
using REST/LINQ. LINQ (language integrated query) is a component of
the .NET framework by Microsoft Corporation that adds native data
querying capabilities to .NET languages. REST (representational
state transfer) is an architecture for distributed media networks.
A RESTful web service is a simple web service implemented using
HTTP and the principles of REST. REST Services focus on resources,
each resource represented by a unique URI (uniform resource
identifier), and users interact with the resources via the resource
URI using the HTTP uniform interface. LINQ standard query
operators, implemented in JavaScript over node.js, web sockets, and
the reactive framework, as a mechanism for bi-directional query
injection over REST in web applications and mobile applications.
REST/LINQ can be the "algebra" for specifying complex event
triggers beyond what is hardwired into the daemon. (A daemon is a
program that runs continuously and exists for the purpose of
handling periodic service requests that a computer system expects
to receive. The daemon program forwards the requests to other
programs (or processes) as appropriate. Some of these processes may
exist outside of the computer system and require the daemon to
communicate with an outside service. The daemon can also receive
remote instructions and process them locally on the device, with
our without the user's interaction.)
[0026] The mobile device 102 can be a cellular telephone and the
server 110 invokes phone functionality of the device 102 in
response to a local or remote event. The events can be associated
with device sensors and environmental conditions that are bound to
the callbacks. Additionally, the reactive framework employs data
storage that unifies code and data as a graph. The data can include
personal data that is accessible only to an authorized user.
[0027] In a specific exemplary implementation, the architecture
employs a graph-structured database, and a reactive, functional
programming model. The functional programming model can be
implemented using an event driven input/output (I/O) server-side
script (e.g., node.js) can be employed. Similarly, the database can
be a graph data storage (e.g., GraphDB), which is built on a
document-oriented database (e.g., MongoDB).
[0028] Node.js is an environment designed to implement services
using javascript. With node.js, the server-side script is written
in javascript as well, and similar to AJAX, a client-side
development tool for creating interactive web applications on the
client, node.js is event-driven.
[0029] In an AJAX web application, a "callback" (e.g., a javascript
function) can be bound (bind) to an event such as a mouse click on
an HTML (hypertext markup language) button. (The words "register"
and "subscribe" are also often used interchangeably for "bind".
Registration of a callback is an instance of a publish-subscribe
pattern.)
[0030] Following is code that illustrates a callback and
binding.
TABLE-US-00001 var http = require(`http`); http.createServer(
function (req,res) { res.writeHead(200, {`Content-Type`:
`text/plain`}); res.end(`Hello World\n`); } ).listen(1337,
"127.0.0.1"); console.log(`Server running at
http://127.0.0.1:1337/`);
The "function" instruction is the callback, and the "createServer"
instruction binds the callback to an HTTP (hypertext transfer
protocol) GET request on localhost (the virtual loopback network
interface of a computer system) port 1337.
[0031] The callback executes quickly; otherwise, the web browser
may "hang" or become unresponsive (following the "button click" or
selection). If clicking on the button requires making web service
calls to a server, the callback should not "block", meaning,
directly invoke a synchronous function such as a HTTP command (HTTP
GET) that will not return until the services request is fulfilled
(which could be many seconds). Rather, when a long or blocking
operation is needed, the callback invokes the function
asynchronously with a call that returns immediately, the call
having an argument that is another callback to be triggered when
the request is fulfilled.
[0032] By writing an entire AJAX experience in terms of
non-blocking callbacks that can, in turn, bind further callbacks, a
web application is created that is both responsive and efficient.
Polling operations, locks, and multiple javascript threads are
avoided. Threads and locks are used at a lower and
better-encapsulated level within the libraries that perform the
slow operations.
[0033] With respect to javascript and the event model server-side,
node.js generalizes this same pattern to the server side. Instead
of UI (user interface) events, bindable events now include server
requests; the outer part of a node.js server script includes the
binding of the HTTP request to a callback (similar to the server's
"main( ) function). All I/O, database reads or writes, etc., can be
treated as "slow", and are invoked asynchronously. The server-side
and client-side scripts now both consist entirely of callbacks, and
are both written in javascript.
[0034] It is conventionally understood that the phone and personal
computer can have javascript running in a web browser, while
servers are normally considered as computers (e.g., rack-mounted)
running in a data center and serving content to the user's mobile
browser and computer-based web browsers.
[0035] A user's device can function as a server or connect to local
server. It is also possible for a device client (e.g., a web
browser) to connect to the local device as a server (e.g., referred
to as "localhost", or IP address 127.0.0.1). When a mobile device
is considered to be a server, and is also running an event-driven
software component (e.g., node.js), a variety of device sensors
and/or environmental conditions now become available as events that
can be bound to callbacks (e.g., similar to AJAX events and HTTP
connection requests).
[0036] Examples of events a phone server can expose include, but
are not limited to, a hardware button press, the phone is unlocked,
the launching of an application, the appearance of a new WiFi
access point, a change in the radio signal strength, a change in
the battery level, a feature or gesture (e.g., a figure-8 hand
wave) recognizable with an onboard motion sensor (e.g.,
accelerometer), a complex pattern recognizable with signal
processing applied to the accelerometer and other sensors (e.g.,
the user changes states between walking, running, sitting or
driving), the phone is removed from the pocket (e.g., recognizable
with the infrared sensor), a loud noise, features in the microphone
input that suggest there is music playing in the environment, an
incoming call, a Bluetooth device is paired, an email is sent,
etc.
[0037] Broadly speaking, these events can fall into several
categories. One category is low-level hardware events (e.g., button
presses). These can originate in the OS. Another category is
high-level sensory events (e.g., music detection, gait detection,
gesture recognition, etc.). These events can also originate in the
OS, but unlike low-level hardware events, these events utilize
nontrivial processing. This is the case especially if the
processing is restricted to occurring in the background or without
waking-up the phone's processor. However, low-power background
signal processing hardware can assist to wake up the CPU (central
processing unit) to generate an event only when the event actually
takes place, and only if one or more callbacks have subscribed to
this event.
[0038] Another category can be defined as phonetop events (e.g.,
launching an application). These events are generated from the
phone OS shell. Yet another category can be described as
application events (e.g., sending an email). These events form an
open-ended set. Since access to the full network stack is provided
at the application level, arbitrarily named and parameterized
events can be named with an HTTP call to localhost (a standard
hostname for the local computer). New applications can define their
own events, using a parameterized namespace. It is desired to
"standardize" as much of the event namespace as possible, so that
patterns written in terms of these events can run on many devices
and operating systems. Library names can also follow simple
conventions (e.g., with standard battery events accessible via
node.js).
[0039] In terms of data, a graph-structured database (e.g.,
GraphDB), the benefits of a non-SQL-type graph-structured database
are as follows. Arbitrary objects can be stored in the graph
database (represented using javascript). In particular, these
objects can be pure data, in which case the objects appear as JSON
(javascript object notation, a simple javascript-compatible way of
representing values, key-value pairs, and arrays). The objects can
also include code (e.g., functions and/or methods). There is no
pre-determined schema; thus, fields, properties, and functions can
be added as needed. Another benefit is the relationships between
objects (the graph edges) are similarly flexible, and can be
extended to encompass new types as needed. Yet another benefit is
that arbitrary fields can be indexed when and as needed to allow
fast search and retrieval.
[0040] In one exemplary implementation, GraphDB (an open source
graph database and universal access layer) and Read/Write World.TM.
(RWW) can be employed as the data management systems; however, it
is to be understood that these are only examples, and that other
suitable systems can be employed for the desired results. (RWW is a
project by Microsoft Corporation to index, unify, and connect
geo-linked media, which comprises cloud-based geo-indexing,
matching, and processing services.) GraphDB implements this kind of
realtime read/write storage and access, optimized as an in-memory
server-side cache, and in general, backed by slower, persistent
stores. MapReduce.TM. (a software framework for distributed
computing on large datasets) type operations are supported as
well.
[0041] By extension, all callbacks can be stored in the graph as
function in nodes. Javascript code stored this way in the database
is by definition the deployed version, since callback lookups can
be queries (e.g., "Which callbacks do I run when this RSS feed is
updated?", "Which callbacks do I delete if this restaurant is
removed from the graph?", "Which callback do I invoke to render a
panorama?", "Which callbacks are set to trigger on user X's
phone?", and "Which callbacks refer in any way to user Y?").
[0042] Alternatively, it is also possible to develop and deploy
callbacks in the live environment. Web applications written as AJAX
enable live changes and operation in realtime.
[0043] With respect to user data, privacy, and inversion of
control, private data of users can be stored. When logged in, a
user view of the graph can include nodes and edges containing
personal data that are otherwise invisible. This interface does not
imply that the physical storage of this sensitive data be
co-located or even implemented using the same technology, but is
only an API (application programming interface) requirement.
[0044] The user data on servers can be managed securely. For
example, the following can be provided:
[0045] Signal dashboard. The user can be in control of signals that
are observable. For example, the user experience can be
sufficiently granular to enable tracks of music played to be
observed while keeping the location secured, or allow only location
information by neighborhood to be observed, but not actual GPS
(global positioning system) traces.
[0046] Notification dashboard. The user can understand and manage
all notifications currently set for the user (e.g., using metaphor
like icons or badges, and perhaps with hierarchy).
[0047] No black boxes. When making an inference about user behavior
or interests, the user can inquire, and receive a clear
explanation. Pattern-based event triggering renders the triggering
conditions for any notification into a user understandable
format.
[0048] Opt-in sharing. If user signals or data are to be shared
with another user or a third party, the user can explicitly
opt-in.
[0049] Weak pointers to user data. A user can view any and all
recipients or observers of the user's data, and revoke access for
any or all of them.
[0050] Compute at the edge. It is desirable to push algorithms and
event logic to the mobile device (e.g., phone), rather than pulling
phone signals to the cloud for evaluation on the server. If both
node.js and GraphDB run in the phone as well as in the cloud, then
user-inference-making algorithms (e.g., event triggering conditions
and callbacks) can be run on the phone. For example, knowing only
the neighborhood a user lives in, all callbacks associated with
entities in that neighborhood can be queried and streamed, with
associated triggering conditions, to the mobile device phone.
[0051] In operation, this can look like a subscription to the
neighborhood, such that all additions or removals of events and
callbacks applying to that neighborhood are streamed to the phone,
perhaps only when connected by WiFi or at intervals of no more than
1.times./day. The subscription model implies that this is itself a
callback, and can be implemented entirely within the system. This
enables allows many of the events/callback to trigger without any
communication back to the server, which not only protects user
privacy, but also saves bandwidth, and enables many events to
trigger without an Internet connection (e.g., geo-fencing can work
abroad without an international data plan). Both code and content
can be cached locally.
[0052] With respect to no client-side changes, without a
specialized client of any kind, a backend system can still use SMS
(short message service) to send notifications to the phone. SMS
messages can also contain hyperlinks to render brief text messages
with one (or even multiple) links to webpages with richer expanded
notifications or interactions. (Note that many of the benefits of
deep integration into the client can also be achieved with
integration into the phone operator's servers, especially when the
signals are rich.)
[0053] With respect to a client-side application, a
user-installable application can be provided that is capable of
generating rich notifications and, to one degree or another,
operating in a background thread even when not visible. The
background application can track location.
[0054] With respect to using a daemon, modified builds of existing
devices can be implemented that enable scenarios without requiring
the full nodejs/GraphDB stack on the client.
[0055] With respect to a local server, enabling a full
nodejs/GraphDB/localhost stack on the client can provide the
following: a web-oriented protocol for registering new kinds of
events using a generic event and client-side javascript in the web
toast, with localhost support, an "algebra" for specifying complex
event triggers beyond what is hardwired into the daemon, and the
ability to issue server requests and perform additional processing
in response to events in javascript without pulling up a web
toast.
[0056] Pages in the web browser can create and subscribe to local
phone events, thereby enabling a web phone development platform
that greatly expands the domain of what can be implemented as a
cross-platform web application. Moreover, a low-level application
such as the telephone application itself (used for dialing and
receiving calls), can be written as a web page. Furthermore, such
applications can be combined with each other and with native
applications.
[0057] As an example of the disclosed capabilities, consider the
following instant answer when typing a publicly listed phone number
into a search engine. Suppose a callback is registered to the
"incoming phone call" event, which first checks whether the number
matches information in the address book; and if not, queries the
search engine using the number. If the number is matched in the
address book, slides in a toast showing a condensed (and
expandable) rendition of the instant answer.
[0058] In this way, an apparent alert registration is now viewed as
an installed plug-in, which enhances the behavior of an existing
application. This general pattern, in which online services augment
existing functionality with a slide-out panel or other
notification, can be referred to as a "sidecar".
[0059] The sidecar can be used not only as an enhancement, but also
to redirect traffic from existing websites or applications, to
instrument them, or even to supplant altogether (e.g., by offering
an alternate dialer altogether on launch of the phone application).
On mobile devices such as the cell phone, there are many more
signals than on a typical computer, because the device is more
personal and the web (Internet) and applications can be
instrumented and intercepted. Moreover, the uniform security and
permissioning architecture built into the system eliminates the
lack of transparency typical with computer toolbars, for
example.
[0060] Many of the in-search widgets can be reframed in this
context. Using javascript as the universal language and securing
the system also provides the opportunity for third-party
development of many simple but useful utilities.
[0061] Computers and other devices with larger screens than cell
phones encourage immersive exploration of remote environments in a
way that smaller mobile devices (phones) do not.
[0062] If considering the problem of intent, this can also be
considered in terms of notifications, using an event subscription
model. Moving through space is in effect subscribing to a moving
window in space; anything triggered purely by "geo-fencing" shows
up as movement progresses. If interested in a transit "overlay",
simply type or say "transit", and an "alert" can be offered that in
effect subscribes to experiences relevant to transit. These can
include a "where would you like to go?" input box that travels with
the user as the user moves through space, as well as interactable
surfaces near all bus stops and subway stations. These
"intent-lens-like" subscriptions can be rendered iconically in the
first-person view, just as more latent, standing alert
subscriptions show up in a user profile.
[0063] Still further, devices can be created that combine the
personal augmentation of mobile phones with the large, immersive
displays of computers or home theaters, allowing real life and
virtual reality (or markup) to be combined. (These experiences are
referred to as "immersed", as opposed to "immersive".)
[0064] Included herein is a set of flow charts representative of
exemplary methodologies for performing novel aspects of the
disclosed architecture. While, for purposes of simplicity of
explanation, the one or more methodologies shown herein, for
example, in the form of a flow chart or flow diagram, are shown and
described as a series of acts, it is to be understood and
appreciated that the methodologies are not limited by the order of
acts, as some acts may, in accordance therewith, occur in a
different order and/or concurrently with other acts from that shown
and described herein. For example, those skilled in the art will
understand and appreciate that a methodology could alternatively be
represented as a series of interrelated states or events, such as
in a state diagram. Moreover, not all acts illustrated in a
methodology may be required for a novel implementation.
[0065] FIG. 2 illustrates a method in accordance with the disclosed
architecture. At 200, a server is operated in a mobile device that
accesses native capabilities of a mobile operating system of the
mobile device. At 202, an event-driven environment is executed on
the server to process conditions and callbacks at least one of
locally or remotely.
[0066] FIG. 3 illustrates further aspects of the method of FIG. 2.
Note that the flow indicates that each block can represent a step
that can be included, separately or in combination with other
blocks, as additional aspects of the method represented by the flow
chart of FIG. 2. At 300, the server can be embedded in the mobile
operating system and emulating a web server for client application
interaction. At 302, instructions are received into the server from
a local application and via a secure protocol. At 304, instructions
are received into the server from an authorized remote web service
via a secure protocol. At 306, an application is launched in
response to processing of a condition. At 308, an application
function is executed in response to processing of a condition. At
310, data and code are stored as a graph. The graph has personal
data accessible only by an authorized user.
[0067] FIG. 4 illustrates an alternative method in accordance with
the disclosed architecture. At 400, a server is operated in a
mobile device that accesses native capabilities of a mobile
operating system of the mobile device. At 402, an event-driven
environment is executed on the server to process events and
callbacks at least one of locally or remotely. At 404, instructions
are communicated to and from the server via a secure protocol. At
406, local functionality of at least one of an application or a
function is invoked in response to an event using a scripting
language.
[0068] FIG. 5 illustrates further aspects of the method of FIG. 4.
Note that the flow indicates that each block can represent a step
that can be included, separately or in combination with other
blocks, as additional aspects of the method represented by the flow
chart of FIG. 4. At 500, the instructions are communicated from a
local application or a remote web service based on hardware events,
sensory events, applications, and application functions. At 502,
code and data are stored in association with nodes of a storage
graph.
[0069] As used in this application, the terms "component" and
"system" are intended to refer to a computer-related entity, either
hardware, a combination of software and tangible hardware,
software, or software in execution. For example, a component can
be, but is not limited to, tangible components such as a processor,
chip memory, mass storage devices (e.g., optical drives, solid
state drives, and/or magnetic storage media drives), and computers,
and software components such as a process running on a processor,
an object, an executable, a data structure (stored in volatile or
non-volatile storage media), a module, a thread of execution,
and/or a program. By way of illustration, both an application
running on a server and the server can be a component. One or more
components can reside within a process and/or thread of execution,
and a component can be localized on one computer and/or distributed
between two or more computers. The word "exemplary" may be used
herein to mean serving as an example, instance, or illustration.
Any aspect or design described herein as "exemplary" is not
necessarily to be construed as preferred or advantageous over other
aspects or designs.
[0070] FIG. 6 illustrates a schematic block diagram of an exemplary
multimode handset 600 that can employ an embedded server and
reactive framework in accordance with the disclosed architecture.
In order to provide additional context for various aspects thereof,
FIG. 6 and the following discussion are intended to provide a
brief, general description of a suitable environment in which the
various aspects of the innovation can be implemented. While the
description includes a general context of computer-executable
instructions, those skilled in the art will recognize that the
innovation also can be implemented in combination with other
program modules and/or as a combination of hardware and
software.
[0071] Generally, applications (e.g., program modules) can include
routines, programs, components, data structures, etc., that perform
particular tasks or implement particular abstract data types.
Moreover, those skilled in the art will appreciate that the
inventive methods can be practiced with other system
configurations, including single-processor or multiprocessor
systems, minicomputers, mainframe computers, as well as personal
computers, hand-held computing devices, microprocessor-based or
programmable consumer electronics, and the like, each of which can
be operatively coupled to one or more associated devices.
[0072] The handset 600 (e.g., a cell phone) can typically include a
variety of computer-readable media. Computer-readable media can be
any available media accessed by the handset systems and includes
volatile and non-volatile media, removable and non-removable media.
By way of example, and not limitation, computer-readable media can
comprise device storage media and communication media. Storage
media includes volatile and/or non-volatile, removable and/or
non-removable media implemented in any method or technology for the
storage of information such as computer-readable instructions, data
structures, program modules or other data. Storage media can
include, but is not limited to, RAM, ROM, EEPROM, flash memory or
other memory technology, CD-ROM, digital video disc (DVD) or other
optical disk storage, magnetic tape, magnetic disk storage or other
magnetic storage devices, or any other medium which can be used to
store the desired information and which can be accessed by the
handset systems.
[0073] The handset 600 includes a processor 602 for controlling and
processing onboard operations and functions. A memory 604
interfaces to the processor 602 for the storage of data and one or
more applications 606 (e.g., a video player software, user feedback
component software, etc.) and operating system. The one or more
application programs 606, and/or program data can include entities
and components of the system 100 of FIG. 1, and the methods
represented by the flowcharts of FIGS. 2-5, for example.
[0074] The applications 606 can also include a user interface (UI)
application 608 that operates with a client 610 (e.g., operating
system)to facilitate user interaction with handset functionality
and data, for example, answering/initiating calls,
entering/deleting data, configuring settings, address book
manipulation, multimode interaction, etc. The applications 606 can
include other applications 612 that came installed with the handset
606 and/or can be installed as add-ons or plug-ins to the client
610 and/or UI 608, for example, or for other purposes (e.g.,
processor, firmware, etc.).
[0075] The other applications 612 can include voice recognition of
predetermined voice commands that facilitate user control, call
voice processing, voice recording, messaging, e-mail processing,
video processing, image processing, music play, as well as
subsystems or components described infra. Some of the applications
606 can be stored in the memory 604 and/or in a firmware 614, and
executed by the processor 602 from either or both the memory 604
or/and the firmware 614. The firmware 614 can also store code for
execution in power-up initialization and control during normal
operation of the handset 600.
[0076] A communications component 616 can interface to the
processor 602 to facilitate wired/wireless communications with
external systems, for example, cellular networks, VoIP
(voice-over-IP) networks, local wireless networks or personal
wireless networks such as Wi-Fi, Wi-Max, and so on. Here, the
communications component 616 can also include a multimode
communications subsystem for providing cellular communications via
different cellular technologies. For example, a first cellular
transceiver 618 (e.g., GSM) can be one mode and an Nth transceiver
620 can provide cellular communications via an Nth cellular network
(e.g., UMTS), where N is a positive integer. The communications
component 616 can also include a transceiver 622 for unlicensed
communications (e.g., Wi-Fi, Wi-Max, Bluetooth, etc.) for
corresponding communications. The communications component 616 can
also facilitate communications reception from terrestrial radio
networks (e.g., broadcast), digital satellite radio networks, and
Internet-based radio services networks.
[0077] The handset 600 can process IP data traffic via the
communications component 616 to accommodate IP traffic from an IP
network such as, for example, the Internet, a corporate intranet, a
home broadband network, a personal area network, etc., via an ISP
or broadband cable provider. Thus, VoIP traffic can be utilized by
the handset 600 and IP-based multimedia content can be received in
an encoded and/or decoded format.
[0078] The handset 600 includes a display 624 for displaying
multimedia that include text, images, video, telephony functions
(e.g., a Caller ID function), setup functions, menus, etc. The
display 624 can also accommodate the presentation of multimedia
content (e.g., music metadata, messages, wallpaper, graphics,
etc.).
[0079] An input/output (I/O) interface 626 can be provided for
serial/parallel I/O of data and/or signals (e.g., USB, and/or IEEE
1394) via a hardwire connection, and other I/O devices (e.g., a
keyboard, keypad, mouse, interface tether, stylus pen, touch
screen, etc.). The I/O interface 626 can be utilized for updating
and/or troubleshooting the handset 600, for example.
[0080] Audio capabilities can be provided via an audio I/O
component 628, which can include a speaker for the output of audio
signals related to, for example, indication that the user pressed
the proper key or key combination to initiate the user feedback
signal, call signals, music, etc. The audio I/O component 628 also
facilitates the input of audio signals via a microphone to record
data and/or telephony voice data, and for inputting voice signals
for telephone conversations.
[0081] The handset 600 can include a slot interface 630 for
accommodating a subscriber identity system 632 that can accommodate
a SIM or universal SIM (USIM), and interfacing the subscriber
identity system 632 with the processor 602. However, it is to be
appreciated that the subscriber identity system 632 can be
manufactured into the handset 600, and updated by downloading data
and software thereinto.
[0082] An image capture and processing system 634 (e.g., a camera)
can be provided for decoding encoded image content. Additionally,
as indicated, photos can be obtained via an associated image
capture subsystem of the image system 634. The handset 600 can also
include a video component 636 for processing video content received
and, for recording and transmitting video content.
[0083] Optionally, a geolocation component 638 (e.g., GPS-global
positioning system) facilitates receiving geolocation signals
(e.g., from satellites via the communications component 616) that
define the location of the handset 600. Alternatively, or in
combination therewith, the geolocation component 638 can facilitate
triangulation processing for locating the handset 600.
[0084] The handset 600 also includes a power source 640 in the form
of batteries and/or an AC power subsystem, which power source 640
can interface to an external power system or charging equipment
(not shown) via a power I/O component 642.
[0085] Referring now to FIG. 7, there is illustrated a block
diagram of a computing system 700 that can execute an embedded
server and reactive framework in accordance with the disclosed
architecture. However, it is appreciated that the some or all
aspects of the disclosed methods and/or systems can be implemented
as a system-on-a-chip, where analog, digital, mixed signals, and
other functions are fabricated on a single chip substrate. In order
to provide additional context for various aspects thereof, FIG. 7
and the following description are intended to provide a brief,
general description of the suitable computing system 700 in which
the various aspects can be implemented. While the description above
is in the general context of computer-executable instructions that
can run on one or more computers, those skilled in the art will
recognize that a novel embodiment also can be implemented in
combination with other program modules and/or as a combination of
hardware and software.
[0086] The computing system 700 for implementing various aspects
includes the computer 702 having processing unit(s) 704, a
computer-readable storage such as a system memory 706, and a system
bus 708. The processing unit(s) 704 can be any of various
commercially available processors such as single-processor,
multi-processor, single-core units and multi-core units. Moreover,
those skilled in the art will appreciate that the novel methods can
be practiced with other computer system configurations, including
minicomputers, mainframe computers, as well as personal computers
(e.g., desktop, laptop, etc.), hand-held computing devices,
microprocessor-based or programmable consumer electronics, and the
like, each of which can be operatively coupled to one or more
associated devices.
[0087] The system memory 706 can include computer-readable storage
(physical storage media) such as a volatile (VOL) memory 710 (e.g.,
random access memory (RAM)) and non-volatile memory (NON-VOL) 712
(e.g., ROM, EPROM, EEPROM, etc.). A basic input/output system
(BIOS) can be stored in the non-volatile memory 712, and includes
the basic routines that facilitate the communication of data and
signals between components within the computer 702, such as during
startup. The volatile memory 710 can also include a high-speed RAM
such as static RAM for caching data.
[0088] The system bus 708 provides an interface for system
components including, but not limited to, the system memory 706 to
the processing unit(s) 704. The system bus 708 can be any of
several types of bus structure that can further interconnect to a
memory bus (with or without a memory controller), and a peripheral
bus (e.g., PCI, PCIe, AGP, LPC, etc.), using any of a variety of
commercially available bus architectures.
[0089] The computer 702 further includes machine readable storage
subsystem(s) 714 and storage interface(s) 716 for interfacing the
storage subsystem(s) 714 to the system bus 708 and other desired
computer components. The storage subsystem(s) 714 (physical storage
media) can include one or more of a hard disk drive (HDD), a
magnetic floppy disk drive (FDD), and/or optical disk storage drive
(e.g., a CD-ROM drive DVD drive), for example. The storage
interface(s) 716 can include interface technologies such as EIDE,
ATA, SATA, and IEEE 1394, for example.
[0090] One or more programs and data can be stored in the memory
subsystem 706, a machine readable and removable memory subsystem
718 (e.g., flash drive form factor technology), and/or the storage
subsystem(s) 714 (e.g., optical, magnetic, solid state), including
an operating system 720, one or more application programs 722,
other program modules 724, and program data 726.
[0091] The operating system 720, one or more application programs
722, other program modules 724, and/or program data 726 can include
entities and components of the system 100 of FIG. 1, and the
methods represented by the flowcharts of FIGS. 2-5, for
example.
[0092] Generally, programs include routines, methods, data
structures, other software components, etc., that perform
particular tasks or implement particular abstract data types. All
or portions of the operating system 720, applications 722, modules
724, and/or data 726 can also be cached in memory such as the
volatile memory 710, for example. It is to be appreciated that the
disclosed architecture can be implemented with various commercially
available operating systems or combinations of operating systems
(e.g., as virtual machines).
[0093] The storage subsystem(s) 714 and memory subsystems (706 and
718) serve as computer readable media for volatile and non-volatile
storage of data, data structures, computer-executable instructions,
and so forth. Such instructions, when executed by a computer or
other machine, can cause the computer or other machine to perform
one or more acts of a method. The instructions to perform the acts
can be stored on one medium, or could be stored across multiple
media, so that the instructions appear collectively on the one or
more computer-readable storage media, regardless of whether all of
the instructions are on the same media.
[0094] Computer readable media can be any available media that can
be accessed by the computer 702 and includes volatile and
non-volatile internal and/or external media that is removable or
non-removable. For the computer 702, the media accommodate the
storage of data in any suitable digital format. It should be
appreciated by those skilled in the art that other types of
computer readable media can be employed such as zip drives,
magnetic tape, flash memory cards, flash drives, cartridges, and
the like, for storing computer executable instructions for
performing the novel methods of the disclosed architecture.
[0095] A user can interact with the computer 702, programs, and
data using external user input devices 728 such as a keyboard and a
mouse. Other external user input devices 728 can include a
microphone, an IR (infrared) remote control, a joystick, a game
pad, camera recognition systems, a stylus pen, touch screen,
gesture systems (e.g., eye movement, head movement, etc.), and/or
the like. The user can interact with the computer 702, programs,
and data using onboard user input devices 730 such a touchpad,
microphone, keyboard, etc., where the computer 702 is a portable
computer, for example. These and other input devices are connected
to the processing unit(s) 704 through input/output (I/O) device
interface(s) 732 via the system bus 708, but can be connected by
other interfaces such as a parallel port, IEEE 1394 serial port, a
game port, a USB port, an IR interface, short-range wireless (e.g.,
Bluetooth) and other personal area network (PAN) technologies, etc.
The I/O device interface(s) 732 also facilitate the use of output
peripherals 734 such as printers, audio devices, camera devices,
and so on, such as a sound card and/or onboard audio processing
capability.
[0096] One or more graphics interface(s) 736 (also commonly
referred to as a graphics processing unit (GPU)) provide graphics
and video signals between the computer 702 and external display(s)
738 (e.g., LCD, plasma) and/or onboard displays 740 (e.g., for
portable computer). The graphics interface(s) 736 can also be
manufactured as part of the computer system board.
[0097] The computer 702 can operate in a networked environment
(e.g., IP-based) using logical connections via a wired/wireless
communications subsystem 742 to one or more networks and/or other
computers. The other computers can include workstations, servers,
routers, personal computers, microprocessor-based entertainment
appliances, peer devices or other common network nodes, and
typically include many or all of the elements described relative to
the computer 702. The logical connections can include
wired/wireless connectivity to a local area network (LAN), a wide
area network (WAN), hotspot, and so on. LAN and WAN networking
environments are commonplace in offices and companies and
facilitate enterprise-wide computer networks, such as intranets,
all of which may connect to a global communications network such as
the Internet.
[0098] When used in a networking environment the computer 702
connects to the network via a wired/wireless communication
subsystem 742 (e.g., a network interface adapter, onboard
transceiver subsystem, etc.) to communicate with wired/wireless
networks, wired/wireless printers, wired/wireless input devices
744, and so on. The computer 702 can include a modem or other means
for establishing communications over the network. In a networked
environment, programs and data relative to the computer 702 can be
stored in the remote memory/storage device, as is associated with a
distributed system. It will be appreciated that the network
connections shown are exemplary and other means of establishing a
communications link between the computers can be used.
[0099] The computer 702 is operable to communicate with
wired/wireless devices or entities using the radio technologies
such as the IEEE 802.xx family of standards, such as wireless
devices operatively disposed in wireless communication (e.g., IEEE
802.11 over-the-air modulation techniques) with, for example, a
printer, scanner, desktop and/or portable computer, personal
digital assistant (PDA), communications satellite, any piece of
equipment or location associated with a wirelessly detectable tag
(e.g., a kiosk, news stand, restroom), and telephone. This includes
at least Wi-Fi.TM. (used to certify the interoperability of
wireless computer networking devices) for hotspots, WiMax, and
Bluetooth.TM. wireless technologies. Thus, the communications can
be a predefined structure as with a conventional network or simply
an ad hoc communication between at least two devices. Wi-Fi
networks use radio technologies called IEEE 802.11x (a, b, g, etc.)
to provide secure, reliable, fast wireless connectivity. A Wi-Fi
network can be used to connect computers to each other, to the
Internet, and to wire networks (which use IEEE 802.3-related media
and functions).
[0100] What has been described above includes examples of the
disclosed architecture. It is, of course, not possible to describe
every conceivable combination of components and/or methodologies,
but one of ordinary skill in the art may recognize that many
further combinations and permutations are possible. Accordingly,
the novel architecture is intended to embrace all such alterations,
modifications and variations that fall within the spirit and scope
of the appended claims. Furthermore, to the extent that the term
"includes" is used in either the detailed description or the
claims, such term is intended to be inclusive in a manner similar
to the term "comprising" as "comprising" is interpreted when
employed as a transitional word in a claim.
* * * * *
References