U.S. patent application number 15/870563 was filed with the patent office on 2018-07-19 for method and apparatus for bidirectional control connecting hardware device action with url-based web navigation.
The applicant listed for this patent is Roger Wagner. Invention is credited to Roger Wagner.
Application Number | 20180205569 15/870563 |
Document ID | / |
Family ID | 62841468 |
Filed Date | 2018-07-19 |
United States Patent
Application |
20180205569 |
Kind Code |
A1 |
Wagner; Roger |
July 19, 2018 |
METHOD AND APPARATUS FOR BIDIRECTIONAL CONTROL CONNECTING HARDWARE
DEVICE ACTION WITH URL-BASED WEB NAVIGATION
Abstract
A method and apparatus, using a responsive electronic hardware
device and Uniform Resource Locators (URLs) with an Internet
browser display simultaneously, both to control web navigation
through hardware device action and to control hardware device
action through web navigation. The method exploits the basic
functionality of a hardware micro controller and the universal and
bidirectional effectiveness of URLs through a data table connecting
URLs with actions and actions with URLs and including timing
conditions. Embodiments employ the invention's understandability
and ease of construction in educational, diagnostic, and
entertainment applications.
Inventors: |
Wagner; Roger; (El Cajon,
CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Wagner; Roger |
El Cajon |
CA |
US |
|
|
Family ID: |
62841468 |
Appl. No.: |
15/870563 |
Filed: |
January 12, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
62445410 |
Jan 12, 2017 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 16/954 20190101;
G06F 16/9566 20190101; G06F 9/4411 20130101; H04L 67/125 20130101;
H04L 12/2816 20130101; H04L 67/025 20130101; G06F 9/44505 20130101;
G05B 15/02 20130101 |
International
Class: |
H04L 12/28 20060101
H04L012/28; H04L 29/08 20060101 H04L029/08; G05B 15/02 20060101
G05B015/02 |
Claims
1. A method for controlling a hardware device based on Uniform
Resource Locator (URL)-based web navigation, the method comprising:
identifying a URL associated with a web browser; determining, using
an electronic lookup table, an action from a set of actions for a
hardware device to perform based at least in part on the identified
URL; and causing the hardware device to perform the determined
action.
2. The method of claim 1, the determined action comprises turning
the hardware device on or off.
3. The method of claim 1, wherein the hardware device is a light
source, and wherein the determined action comprises turning the
light source on or off.
4. The method of claim 1, wherein the hardware device is a camera,
and wherein the determined action comprises changing one or more
settings of the camera.
5. The method of claim 4, wherein one or more settings of the
camera comprises a combination of one or more of a direction or a
zoom of the camera.
6. The method of claim 1, wherein the hardware device comprises one
or more of an LED, a motor, or a servomechanism.
7. The method of claim 1, wherein the electronic lookup table
correlates the URL and the determined action.
8. A method for navigating to a Uniform Resource Locator (URL)
based on data corresponding to a hardware device, the method
comprising: receiving data indicative of a hardware state of a
hardware device; identifying, using an electronic lookup table, a
URL location from a set of URLs based at least in part on the
received data; and causing a web browser to navigate to the
identified URL.
9. The method of claim 8, wherein the hardware device comprises a
sensor, and wherein the data corresponds to data sensed by the
sensor.
10. The method of claim 9, wherein the sensor is a photocell.
11. The method of claim 8, wherein the hardware device comprises
one or more of an LED, a motor, a servomechanism, or a buzzer.
12. The method of claim 8, wherein the hardware device is a camera,
and wherein data of the hardware device corresponds to one or more
settings of the camera.
13. The method of claim 12, the data of the hardware device
corresponds to a change in direction or zoom of the camera.
14. The method of claim 8, wherein the electronic lookup table
stored in a computer-readable medium.
15. The method of claim 8, wherein the electronic lookup table
correlates the hardware state of the hardware device and the URL
location.
16. A system for bidirectional control between hardware device
action and URL-based web navigation, the system comprising: a
microcontroller in communication with at least one hardware device;
and one or more processors in communication with the
microcontroller, and a non-transitory computer-readable storage
medium storing computer-executable instructions that when executed
by the one or more processors cause the one or more processors to:
receive data from the hardware device, wherein the data corresponds
to the one or more actions, monitor a web browser; responsive to
receiving first data, cause the web browser to navigate to a first
URL, and responsive to determining that the web page navigated to a
second URL, causing the hardware device to perform a first
action.
17. The system of claim 16, the first data corresponds to hardware
state data of the hardware device.
18. The system of claim 16, wherein the hardware device is a light
source, and wherein the first action turning the light source
OFF.
19. The system of claim 16, wherein the hardware device is a light
source, and wherein the first action turning the light source ON.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. Provisional
Application No. 62/445,410, entitled "METHOD AND APPARATUS FOR
BIDIRECTIONAL CONTROL CONNECTING HARDWARE DEVICE ACTION WITH
URL-BASED WEB NAVIGATION," filed on Jan. 12, 2017, the disclosure
of which is incorporated herein in its entirety for all
purposes.
BACKGROUND OF THE INVENTION
Background Art
[0002] Uniform Resource Locators (URLs), which are here taken as
synonymous to Uniform Resource Identifiers (URIs), have resulted in
everyday browser-related experiences, of which a few examples are
web search, shopping, and filling out of forms. Their applicability
to special-purpose electronic hardware (hereafter called "hardware"
or "device") has been more restricted, but is nevertheless common.
Among obvious cases are webcams, security notification, and
environmental monitoring which reports sensor data via the
Internet.
[0003] Examples may be Internet-of-Things (IoT) systems, such as:
[0004] (a) device to Internet to browser display, or an http-based
web app, telling you that motion has been detected in your home,
and, [0005] (b) browser display, or an http-based web app to
Internet to device to turn on the lights in the home.
[0006] In (a), one type of prior art may have dedicated software
that responds to the sensor condition with a new dedicated
(non-browser) screen or display. In another type of prior art, the
software displays data or messages within a screen on a webpage on
the mobile device, such as weather station data being graphically
displayed on a website.
[0007] In prior art of type (b) a person may use an app or an
http-based web app to touch a screen display button to turn on
lights, or it might even be automatically done by specific
software, but there are not any existing methods where simply
navigating to a new screen with a different URL on the device would
turn on a light in the user's home.
[0008] In the device-to-webpage direction, the prior art only
displays data on the webpage, it does not change the webpage based
on the device-gathered data. In the webpage-to-device direction,
prior art may be able to turn on something on the device through
controls on the webpage, but the action does not occur as a direct
result of simply being on the webpage.
[0009] It can be seen that the prior art discloses the display of
data from a remote sensor on a given web-page, or the control of a
remote sensor or actuator from the webpage, but that the URL of the
webpage itself is not controlled by the sensors data, nor are
actuators controlled by the act of changing the current URL of the
display device. This deficiency in the prior art is corrected by
the present invention.
[0010] A further example of the prior art involves a scenario of
testing the security lights around a home. In all embodiments to
date, the user clicks on different buttons presented on a webpage
or in an app, and the lights come on, through methods very
well-known to those skilled in the art. All these methods, however,
involve the complication of programming the buttons and the
response to the buttons.
[0011] In addition to the unidirectional URL-level control,
bidirectional control is found in other prior art, e.g., use a
webpage user interface to change the settings (direction, zoom,
etc.) on a webcam, and with that in a fixed state, the webcam data
flow is passed back as a display to the webpage. However, absent
the method taught in this invention, the webpage does not change in
response to the camera, and the camera does not change in response
to changing the webpage being viewed.
[0012] The prior art approaches require programming of at least
four pieces: [0013] the mobile or web app, [0014] the
special-purpose communication on the app side that allows
communication to the hardware, [0015] the hardware programming, and
[0016] the special-purpose communication module that makes it
possible for the hardware to communicate to the app. All of this
must be done with dedication to a particular hardware and software
combination, and it does not provide a general multi-application
system that can be easily repurposed for different hardware sensor
inputs and webpage displays.
[0017] The special-purpose communication can be designed to provide
great speed if necessary, but it has the disadvantage of adding
great complexity to the synergy of the hardware/web interface
combination. That disadvantage is such as to restrict the
development of a device with such a combination to very determined
programmers. Thus the programming of a whole, responsive device
involving both embedded hardware and an easily usable web interface
becomes a specialty task.
SUMMARY
[0018] Accordingly, it is an object of the present invention to
provide method and apparatus for bidirectional control connecting
hardware device action with URL-based web navigation.
[0019] These and other objects and advantages of the present
invention will become clear to those skilled in the art in view of
the description of the best presently known mode of carrying out
the invention and the industrial applicability of the preferred
embodiment as described herein and as illustrated in the figures of
the drawings.
BRIEF DESCRIPTION OF THE DRAWING(S)
[0020] The purposes and advantages of the present invention will be
apparent from the following detailed description in conjunction
with the appended tables and figures of drawings in which:
[0021] TBL. 1 is a description of an exemplary playlist;
[0022] FIG. 1 shows some timelines of a typical operation of a
typical embodiment of the invention called the "HyperDuino";
and
[0023] FIG. 2 is a picture of the communicating pieces of one
embodiment of the invention, HyperDuino For Chrome running an
embedded device with a serial connection and connected via the
Internet to servers from which URLs can draw videos or stills.
[0024] In the various figures of the drawings, like references are
used to denote like or similar elements or steps.
DETAILED DESCRIPTION
[0025] A preferred embodiment of the present invention is a method
and apparatus for bidirectional control connecting hardware device
action with URL-based web navigation. As illustrated in the various
drawings herein, preferred embodiments of the invention are
depicted.
[0026] The invention is a system including a responsive electronic
hardware device (the "hardware"), a browser with a user interface
(the "browser"), a server or servers with content displayable by
the browser (the "server"), and a software program (the "handler")
capable of: [0027] (a) communicating in either direction with the
hardware, [0028] (b) of emitting URLs to the browser and server,
[0029] (c) of receiving information from the browser about the
current state of its web interface, including URLs displayed,
anchors, selections, and optionally, [0030] (d) timing of changing
media content such as videos. The browser and server are standard,
and the hardware is specialized only in the sense that it is
capable of monitoring sensors and inputs, and activating outputs,
usually, but not limited to, actuators such as LEDs, motors,
servos, buzzers, etc. and that the hardware communicates with the
handler. The inventive content is focused on the combination of the
browser, server, hardware and handler, as described below.
[0031] The handler is event-coded to comply with a table, herein
called the "playlist," which is a data table correlating hardware
state input and browser URL location transmitted to the handler to
actions performed by the handler. See TBL. 1. In what follows, the
term "actuator" means the handler drives the hardware, either
causing internal hardware state change or further effects within or
beyond the hardware, or both, while the term "sensor" means the
hardware that drives the handler, either as a result of the
internal hardware state or conditions detected within or beyond the
hardware, or both. The word "microcontroller" includes any
electronics within the hardware.
[0032] Turning now to TBL. 1, this is a description of an exemplary
playlist. The Fields 1a-zzz hold extra parameters, such as start
and end times for a video which may have not been implicitly
written in the URL, but which are still detectable by the Software
while that URL is actively displayed in the browser. (these latter
parameters may have been used in the prior art, but not within the
context of the URL-associated I/O disclosed in this invention).
[0033] The Field 2: ResponseFlag (indicator) has three possible
states. The first state is RespondNewURL. This initiates
microcontroller output (actuator(s)) based on Fields 2a-zzz in
response to the URL appearing in the browser display during the
cycled review of the list, when the cycle before, that URL was not
the active URL. That is, URL=(URL in list) AND (CurrentURL<
>Past URL) AND (ActionTaken=False).
[0034] The second state is RespondLeaveURL. This initiates
microcontroller output (actuator) based on Fields 2a-zzz in
response to the URL NO LONGER appearing in the browser display
during the cycled review of the list, when the cycle before, that
URL WAS an active URL=found-in-list. That is, (CurrentURL<
>Past URL) AND ActionTaken=False.
[0035] And the third state is RespondSense. This passes the URL to
the browser in response to microcontroller output (sensor) meeting
conditions.
[0036] The Fields Fields 2a-zzz are I/O channels, as appropriate to
the microcontroller being used by the Software, and conditions to
be tested for in case RespondSense or output to be actuated in
cases RespondNewURL or RespondLeaveURL.
[0037] Causality goes from hardware state to URL in the Field 2
RespondSense case, and from web interface state to hardware state
in the Field 2 RespondNewURL or RespondLeaveURL state.
[0038] The handler is simultaneously quickly responsive to web
interface state, including URLs, anchors, selections, and timing
such as time elapsed during a video, and to microcontroller
(sensor) output to the handler. This can be accomplished by any
sufficient programming construct, such as select( ) frequent enough
polling, or event coding implied by the structure of a higher-level
programming language like Javascript.
[0039] Since URLs can have multiple queries attached, including
variable=value pairs, this kind of single-URL event coding can
imply a rich branching structure and thus result in as arbitrarily
complex event responses as are normally associated with deeply
nested event-based coding with callbacks.
[0040] With local caching of data, internet latency, and even total
disconnects, operation of the invention can still continue to
operate satisfactorily by just passing the URL to the browser that
itself uses cached data, and this is an example of how the
URL-centric system provides an advantage that would be much more
complex with custom-made computer code and applications.
[0041] Thus, the invention permits an arbitrarily complex event
response to be coded as a flat table of URL responses. This table,
which in one embodiment is called a' "playlist," responds simply to
each specified URL, using its queries to guide the activation of
actuators (defined earlier in the patent text), and also permits
the display of any URL in the playlist in response to the output of
sensors.
Example Embodiments
[0042] The first embodiment is the HyperDuino, an educational
device. It builds on the simplicity inherent in the invention by
introducing a synergistic simplicity to hardware construction,
through the attachment of a shield called the "HyperDuino shield"
to any Arduino or Arduino-equivalent device, such as the Funduino.
The educational value of this is well described on the inventors
web page [Wagner, Roger: "Hardware & Instructional
Design--HyperDuino"; www.rogerwagner.com], for example: [0043]
Physical classroom projects can now be made into interactive maker
projects, but creating a two-way link between digital student-made
content and physical student-created models. [0044] The HyperDuino
solves key obstacles for those just entering the maker movement, by
eliminating the need for breadboards and resistors, and combining
with the HyperDuino for Chrome app to eliminate the need for
scripted (written) coding, while still teaching the logic of
programming.
[0045] The HyperDuino handler is instantiated as a Chrome OS app,
written in Javascript. The creation of Chrome apps in Javascript is
well known to web programmers skilled in the art. The HyperDuino
handler is responsive to Web interface state, and it is also
written so as to be responsive to asynchronous input from serial or
other channels that are driven by the hardware.
[0046] After standard setup is complete and all communication
channels both to the hardware and to the browser are functioning,
the response of the HyperDuino handler is completely determined by
its playlist, as described in TBL. 1. The HyperDuino handler
continues operating until a HaltEvent is received, which can come
either from internal programming, such as the operating session is
done, or from an external signal, such as a user stop signal or
arrival at an URL in the playlist, or the detection of a hardware
state.
[0047] The transmission of URLs caused by the handler in the Field
2 RespondSense case can also include any web interface programming
of which the browser is capable, and can fetch content from any
accessible Internet server reached by the URL. The actuator
response in the Field 2 RespondNewURL or RespondLeaveURL case can
be driven by any knowledge of Web interface state that the browser
is capable of sharing with the handler. Indirect immediate or
delayed feedback responses, both as a result of hardware behavior
and as a result of Web interface behavior, are possible and
expected implications of the playlist.
[0048] The second embodiment presented here is an instructional and
diagnostic tool for actual breadboard circuits, where analog and
digital readings from the digital and analog inputs would directly
lead to the display of informational and diagnostic webpages purely
by virtual of how the existing playlist responded to the
values.
[0049] For example, if the Arduino and a breadboard are being used
to construct a circuit demonstrating a photocell, the HyperDuino
software playlist will see "0" for a short circuit (for example
wires crossed leading to the photocell), "1023" for an open circuit
(for example, the photocell has not yet been inserted), and then an
expected range when the photocell is inserted.
[0050] A lesson on a system like Google Slides may on slide N say
"insert a photocell into this part of the circuit". Until the
photocell is inserted, the slide will not advance. When it is
inserted, the slide will congratulate the user. Should a different
item be inserted that is very different in expected resistance from
a photocell, a different slide will be presented asking the
operator to confirm and re-do the insertion of a photocell.
[0051] A third embodiment would be the application of the invention
in program code on the server of a webpage itself, and not
requiring a local application to be running. In that case, the
webpage could display other webpages within a sub-window of the
main webpage, as is commonly done with embedded videos and other
html content. Code running on the server would then use the method
taught in this invention to correlate the URL displayed in the
sub-window with hardware inputs and outputs connected to the local
computer by serial or other type of communication.
FIGURES AND EXPLANATION
[0052] FIG. 1 shows some timelines of a typical operation of a
typical embodiment of the invention, hereafter called the
"HyperDuino." [0053] 101 is a timeline from left to right. 102 is
the time spent in the HyperDuino Build stage, building the
playlist. 103 is an idle time of indefinite length, during which
the playlist is stored. 104 is the time spent in the HyperDuino Run
stage, reading the playlist and acting upon it. 105 is a drawing
representing the HyperDuino Builder, with its UI windows controlled
by the builder, active during 102. 106 is a drawing representing
the HyperDuino Handler, the standard browser and its screen and
connection to the Internet, and its connected embedded device, all
active during 104. [0054] 107 is a timeline within 104 during which
a video running on the browser is synchronized by the HyperDuino
Handler with action on the embedded device. 108 is the start of the
video, 109 is an appropriate first time period, 110 is a command
causing action such as lights or movement on the embedded device,
111 is an appropriate second time period, 112 is another command to
the embedded device, 113 is an appropriate third time period, 114
is a third command to the embedded device. [0055] 115 is a timeline
within 104 during which a video awaits selection by an embedded
action. 116 is the start of the video, 117 is an appropriate first
time period, 118 is a pause of the video, 119 is an indefinite time
awaiting input from the embedded device (which could be a running
state or a user button push, for instance), 120 is the awaited
input, and 121 is the video resuming or another display taking its
place if so programmed to be dependent on the input. [0056] 122 is
a timeline within 104 during which the HyperDuino responds to a
test result. 123 is a video, a still URL, or other display
appropriate to the test result being unknown, 124 is a test result
or state of the embedded device that may arrive unexpectedly and
cause a branching action by the handler (time for branching not
shown because it is very quick), and 125 is the chosen video or
still URL appropriate to the test result detected.
[0057] FIG. 2 is a picture of the communicating pieces of one
embodiment of the invention, HyperDuino For Chrome running an
embedded device with a serial connection and connected via the
Internet to two Servers from which URLs can draw videos or stills.
This is during phase 104 of the timeline of FIG. 1. [0058] 201 is
the embedded device. 202 is the HyperDuino Handler, and 206 is the
standard Chrome browser, both running on the same system. 203, 204,
and 205 are parts of the Handler: 203 is the playlist, which at
this stage is read-only; 204 is HyperDuino For Chrome; and 205 is
the HyperDuino Web Driver. 206 is the standard Chrome browser, 207
its screen. 208 is the Internet, and 209a and 209b are servers
accessed by the browser through the Internet.
[0059] Bidirectional serial communication, in which either side can
take the initiative, is 210 (embedded device to HyperDuino Web
Driver) and 211 (HyperDuino Web Driver to embedded device). 212
represents HyperDuino For Chrome reading the playlist at will. 213
is data from HyperDuino For Chrome to HyperDuino Web Driver, and
214 is data from HyperDuino Web Driver to HyperDuino For Chrome, to
effect the division of labor required by the OS and/or browser. 215
is an URL and/or timing sent from HyperDuino Web Driver to be acted
upon by the browser, and 216 is the browser transmitting
information about an URL to HyperDuino Web Driver. 217, 219a, and
219b represent the URL request going from browser through the
Internet to a server, and 220a, 220b, and 218 represent the return
of data by a server to the browser.
[0060] The embedded device and serial connection of 201, 210, and
211 are completely standard, as are the browser 206, its display
207, and the connection 217 and 218 and the Internet and servers
beyond it. The breakdown of the HyperDuino Handler into HyperDuino
For Chrome and HyperDuino Web Driver is due to requirements of the
Chrome OS/browser. Other embodiments will use a different
structure, but perform the same essential function.
[0061] While various embodiments have been described above, it
should be understood that they have been presented by way of
example only, and that the breadth and scope of the invention
should not be limited by any of the above described exemplary
embodiments, but should instead be defined only in accordance with
the following claims and their equivalents.
TABLE-US-00001 TBE. 1. DESCRIPTION OF PLAYLIST Field 1: URL (text)
Fields 1a-zzz: Sub-field(s) to Field 1 Field 2: ResponseFlag
(indicator) as to whether this entry is, Field 2 = RespondNewURL
or, Field 2 = RespondLeaveURL or, Field 2 = RespondSense Fields
2a-zzz: Sub-fields to Field 2
* * * * *
References