U.S. patent application number 17/150538 was filed with the patent office on 2021-05-13 for scalable smart environment with gateway thermostat.
The applicant listed for this patent is Computime Ltd.. Invention is credited to Chi Lung Chan, Leung Yin Chan, Hung Bun Choi, Wai-Leung Ha, Tsz Kin Lee, Luke Li, Chi Chung Liu, Yau Wai Ng, Hamza Yilmaz.
Application Number | 20210144025 17/150538 |
Document ID | / |
Family ID | 1000005348555 |
Filed Date | 2021-05-13 |
![](/patent/app/20210144025/US20210144025A1-20210513\US20210144025A1-2021051)
United States Patent
Application |
20210144025 |
Kind Code |
A1 |
Choi; Hung Bun ; et
al. |
May 13, 2021 |
Scalable Smart Environment with Gateway Thermostat
Abstract
Building blocks for a smart device such as a thermostat include
a user interface (UI) unit and a terminal (TML) unit. A UI unit may
support one or more input data from a user and/or sensors and/or
one or more control terminals. The UI unit may process each input
datum or a combination of the input data, generate a control signal
to one or more control terminals based on the processing, and send
the control signal to one or more control terminals over a
communication channel. A terminal unit, which may consist of one or
more control terminals, transforms the received control signal into
one or more controls to one or more associated environmental
generators. One or more UI units may control one or more controlled
apparatuses in conjunction with a mobile app to allow a unified
user experience.
Inventors: |
Choi; Hung Bun; (Hong Kong,
CN) ; Ha; Wai-Leung; (Hong Kong, CN) ; Chan;
Leung Yin; (Hong Kong, CN) ; Ng; Yau Wai;
(Hong Kong, CN) ; Liu; Chi Chung; (Hong Kong,
CN) ; Li; Luke; (Shenzhen, CN) ; Lee; Tsz
Kin; (Hong Kong, CN) ; Chan; Chi Lung; (Hong
Kong, CN) ; Yilmaz; Hamza; (Gilroy, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Computime Ltd. |
New Territories |
|
HK |
|
|
Family ID: |
1000005348555 |
Appl. No.: |
17/150538 |
Filed: |
January 15, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16037140 |
Jul 17, 2018 |
10897374 |
|
|
17150538 |
|
|
|
|
62581910 |
Nov 6, 2017 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04L 12/28 20130101;
H04L 12/2814 20130101; G05B 19/042 20130101; G05B 15/02 20130101;
H04L 12/2832 20130101; H04L 12/282 20130101; G05B 2219/2642
20130101; H04L 12/281 20130101; H04L 12/2834 20130101; H04L 12/2809
20130101 |
International
Class: |
H04L 12/28 20060101
H04L012/28; G05B 19/042 20060101 G05B019/042; G05B 15/02 20060101
G05B015/02 |
Claims
1. One or more non-transitory computer-readable media storing
instructions that, when executed by a computing device comprising
at least one processor, memory, and a communication interface,
cause the computing device to: generate a first control flow and a
first user graphics of a first generated user interface identical
to that in a first physical user interface; generate a second
control flow and a second user graphics of a second generated user
interface identical to that in a second physical user interface;
display a dash board with a first status of the first generated
user interface and a second status of the second generated user
interface; receive an selection input to select one of the
generated user interfaces of the dash board, wherein the selected
generated user interface corresponds to a corresponding physical
user interface; and modify the selected generated user interface
based on inputted manipulations.
2. The one or more non-transitory computer-readable media of claim
1 storing instructions that, when executed by the computing device,
further cause the computing device to: control at least one smart
device associated with the corresponding physical user
interface.
3. The one or more non-transitory computer-readable media of claim
2 storing instructions that, when executed by the computing device,
further cause the computing device to: select one or more of the at
least one smart device associated with the corresponding physical
user interface.
4. The one or more non-transitory computer-readable media of claim
1 storing instructions that, when executed by the computing device,
further cause the computing device to: synchronize a current status
of the first physical user interface with the first status of the
first generated user interface.
5. The one or more non-transitory computer-readable media of claim
1 storing instructions that, when executed by the computing device,
further cause the computing device to: control a controlled device
in accordance with the inputted manipulations through the selected
generated user interface.
6. The one or more non-transitory computer-readable media of claim
5 storing instructions that, when executed by the computing device,
further cause the computing device to: update a third status of the
corresponding physical user interface from a fourth status of the
selected generated user interface.
7. The one or more non-transitory computer-readable media of claim
5 storing instructions that, when executed by the computing device,
further cause the computing device to: subsequently control the
controlled device in accordance with a user input through the
corresponding physical user interface.
8. The one or more non-transitory computer-readable media of claim
7 storing instructions that, when executed by the computing device,
further cause the computing device to: update the fourth status of
selected generated user interface from the third status of the
corresponding physical user interface.
9. A method for controlling a controlled apparatus in conjunction
with a plurality of user interface (UI) units and at least one
terminal unit, wherein the plurality of UI units includes first and
second UI units, wherein the at least one terminal unit includes a
first terminal unit, and wherein the controlled apparatus is
controlled through the terminal unit, the method comprising:
distributing execution of a machine learning program module over
the first and second UI units; receiving, by the first UI unit,
operational information from the second UI unit; determining, by
the first UI unit, a first rule about operation of the controlled
apparatus; determining, by the second UI unit, a second rule about
the operation of the controlled apparatus; downloading the first
and second rules into the terminal unit; and instructing, by the
terminal unit, the controlled apparatus based on the first and
second rules.
10. The method of claim 9, further comprising: executing the
machine learning program module, the first rule, and the second
rule when the plurality of UI units and the at least one terminal
unit are disconnected from an internet.
11. The method of claim 9, further comprising: locally applying
machine learning in one of the plurality of UI unit of a local
loop, wherein said one UI unit is paired with an individual
terminal unit.
12. A method for controlling a controlled apparatus in conjunction
with a plurality of user interface (UI) units, wherein the
plurality of UI units includes first and second UI units, the
method comprising: obtaining first and second performance metrics
from the first and second UI units, respectively; selecting one of
the first and second UI units based on the first and second
performance metrics; executing a machine learning programing module
on the selected UI unit; obtaining, by the selected UI unit,
operational information from at least one of the first and second
UI units; determining, by the selected UI unit, a rule about
operation of the controlled apparatus from the operational
information; and instructing the controlled apparatus based on the
rule.
13. The method of claim 12, further comprising: downloading the
rule into a terminal unit; and instructing, by the terminal unit,
the controlled apparatus based on the rule.
14. The method of claim 12, further comprising: executing the
machine learning program module and the rule at the selected UI
unit when the selected UI is disconnected from an internet.
15. A mobile computing device for controlling a plurality of smart
devices, the mobile computing device comprising: at least one
processor; a wireless communication interface; and memory storing
computer-readable instructions that, when executed by the at least
one processor, cause the mobile computing device to: generate a
first control flow and a first user graphics of a first generated
user interface identical to that in a first physical user
interface; generate a second control flow and a second user
graphics of a second generated user interface identical to that in
a second physical user interface; display a dash board with a first
status of the first generated user interface and a second status of
the second generated user interface; receive an selection input to
select one of the generated user interfaces of the dash board,
wherein the selected generated user interface corresponds to a
corresponding physical user interface; modify the selected
generated user interface based on inputted manipulations; and
control, through the wireless communication interface, at least one
smart device associated with the corresponding physical user
interface.
16. The mobile computing device of claim 15, wherein the memory
storing computer-readable instructions that, when executed by the
at least one processor, further cause the mobile computing device
to: select one of the at least one smart device associated with the
corresponding physical user interface.
17. The mobile computing device of claim 15, wherein the memory
storing computer-readable instructions that, when executed by the
at least one processor, further cause the mobile computing device
to: control a first smart device in accordance with the inputted
manipulations via the selected generated user interface.
18. The mobile computing device of claim 17, wherein the memory
storing computer-readable instructions that, when executed by the
at least one processor, further cause the mobile computing device
to: update a third status of the corresponding physical user
interface from a fourth status of the selected generated user
interface.
19. The mobile computing device of claim 17, wherein the memory
storing computer-readable instructions that, when executed by the
at least one processor, further cause the mobile computing device
to: subsequently control the first smart device in accordance with
a user input through the corresponding physical user interface.
20. The mobile computing device of claim 19, wherein the memory
storing computer-readable instructions that, when executed by the
at least one processor, further cause the mobile computing device
to: update the fourth status of selected generated user interface
from the third status of the corresponding physical user
interface.
21. A control system for controlling a controlled device, the
system comprising: a first user-interface (UI) unit; a second UI
unit; a terminal unit; a terminal unit is configured to control the
controlled device; the first UI unit and the second UI unit
configured to execute a distributed machine learning program
module; the first UI unit configured to receive operational
information from the second UI unit and determine a first rule
about operation of the controlled device; the second UI unit
configured to determine a second rule about the operation of the
controlled device; the first and second UI units configured to
download the first and second rules, respectively, into the
terminal unit; and the terminal unit configured to instruct the
controlled device based on the first and second rules.
Description
[0001] This patent application is a divisional application of and
claims priority to U.S. patent application Ser. No. 16/037,140,
entitled "A Scalable Smart Environment Using A Gateway Thermostat"
filed on Jul. 17, 2018 which claims priority to provisional patent
application Ser. No. 62/581,910 entitled "A Scalable Smart
Environment Using A Gateway Thermostat" filed on Nov. 6, 2017,
which is hereby incorporated by reference in its entirety.
TECHNICAL FIELD
[0002] Aspects of the disclosure relate to a smart control system.
Some embodiments may include a user interface and a terminal unit
to control one or more controlled apparatuses.
BACKGROUND OF THE INVENTION
[0003] There is frequently a need to support a scalable smart
environment that may span environmental controllers, smart devices,
and/or other appliances.
SUMMARY OF THE INVENTION
[0004] An aspect supports building blocks for a smart device, such
as a thermostat. Building blocks include one user interface (UI)
unit and one terminal (TML) unit. A UI unit may support one or more
input data from a user and/or sensors and/or one or more control
terminals. The UI unit may process each input datum or a
combination of the input data, generate a control signal to one or
more control terminals based on the processing, and send the
control signal to one or more control terminals over a
communication channel. A terminal unit, which may consist of one or
more control terminals, transforms the received control signal into
one or more controls to one or more associated environmental
generators. Environmental generators (controlled apparatuses) may
include furnaces, boilers, air conditioners, heat pumps, fans,
de-humidifiers, thermostatic radiator valves (TRVs), and so
forth.
[0005] With another aspect, a user interface unit communicates to a
terminal unit via a communication channel. The communication
channel may utilize a wired or wireless transmission medium.
[0006] With another aspect, the number of UI and terminal units is
flexible. The wired/wireless connections/communications between UI
and terminal units are also flexible, to form a scalable smart
control system.
[0007] With another aspect, a smart control system comprises two
individual smart devices. Examples include a thermostat (having one
UI unit and one terminal unit) and a home console (having one UI
unit and multiple terminal units including those for lightings,
switches, TV's, air conditioners, and so forth).
[0008] With another aspect, one or more UI units may communicate
with one or more terminal units via a connection hub. Control
commands and status reports may be collected in the connection hub
and may be routed to target devices within the web. The connection
hub may be implemented in a public cloud provided by a service
provider or within one of the many UI units within the smart
control system.
[0009] With another aspect, a UI unit may be a gateway which
connects all other smart devices to a public cloud.
[0010] With another aspect, a UI unit may support a private cloud,
where controls and status reports are manipulated within the user's
home network before going to a public domain. The UI unit further
supports a rule engine for smart devices within a home environment
and only user-defined control and information are sent to public
cloud for further manipulation. The UI unit may be equipped with
the capabilities of a connection hub, a gateway and private
cloud.
[0011] With another aspect, different UI units may have different
user interface capabilities such as an LCD with touchscreen,
7-segment LEDs with mechanical buttons, and so forth.
[0012] With another aspect, there is no UI unit at all and the
control of terminal units is done via mobile devices, such as a
smart phone or a tablet.
[0013] With another aspect, a terminal unit may support one or more
control terminals for heating units, cooling units, heat pumps,
fans, de-humidification, and the like.
[0014] With another aspect, the terminal units may be extended to
include those for controlling different electrical appliances.
Flexibility may be supported for interactions between different
electrical appliances and different UI units. The electrical
appliances can be, but not limit to, smart home devices (such as
thermostats, light bulbs, smart sensors, smart plugs, and so
forth), home appliances (such as air conditioners, washing
machines, electric grill, oven, and so forth), a game console,
media devices, healthcare devices, and the like.
[0015] With another aspect, a mobile application is scalable such
that it can select all the smart devices which connect to a UI unit
(which has the capabilities of user interface, gateway, connection
hub and private cloud). Conversely, it may de-select a smart device
from the list.
[0016] With another aspect, the user experience design (UX) and UI
graphic of each UI unit in the mobile app is identical to the
actual UX and UI graphic of the pair to improve user experience. No
extra learning for using the mobile app is thus needed.
[0017] With another aspect, a rule engine may execute on a terminal
unit, and/or a UI unit, and/or public cloud.
[0018] With another aspect, a UI unit may comprise a mobile app
that controls all the associated terminal units via either WiFi or
BLUETOOTH.RTM. Low Energy (BLE). In this case, a rule engine may be
downloaded to individual terminal units to reduce the latency
caused by network performance or when the network connection is
unavailable or when the mobile phone is absent. More complex rule
engines and learning algorithms may be implemented in the mobile
phone or public cloud.
[0019] With another aspect, a number of UI and terminal units may
also be connected and one of the UI units which may have enough
computational and memory power to implement the functions of a
connection hub, a gateway and a private cloud which has a rule
engine with a learning algorithm to execute.
[0020] With another aspect, an individual terminal unit may have
its own UI unit. A rule engine and learning algorithm may be
implemented separately into each UI/terminal pair and the
associated input/output devices to form a local loop. Status
reports for each UI/terminal pair may be sent to a public cloud via
one UI/terminal pair. Rule engine and learning algorithms for the
interactions between of all the UI/terminal pairs may be
implemented in the public cloud.
[0021] With another aspect, a number of local loops can be grouped
together to form a network, such as a home network, a building
network, a community network, a regional network, etc.
[0022] With another aspect, rule engines and learning algorithms
may be implemented with four different levels. First, a simple rule
engine may be implemented in the terminal unit based on its
individual internal inputs (for example, sensor inputs) and
associated external inputs (for example, external sensor inputs).
The rule engine may be downloaded to the terminal unit using WiFi
or BLE in the mobile phone. Second, a rule engine may be
implemented within individual UI unit which controls one or more
terminal units. Third, a rule engine between different UI/terminal
pairs may be implemented in the private cloud (which may be one of
the UI/terminal pairs). Learning algorithms may be implemented in
the private cloud to reduce the latency introduce by the internet.
Fourth, a rule engine and learning algorithm may involve external
Things of Interest (for example, weather information, location
data, sales information, and the like) that may be implemented in a
public cloud.
[0023] With another aspect, rules may be automatically generated
via machine learning algorithm. Machine learning may be applied
locally in the UI unit of a local loop. Machine learning can also
applied to the information received from multiple UI units of a
larger network. Machine learning may be conducted within a single
UI unit which has better system hardware. Learning may also be
distributed to different UI units to reduce the computation loading
for individual UI unit and summarized within the primarily UI
units.
[0024] With another aspect, the configuration of each UI/terminal
pair may be set in the mobile app and committed to an individual
UI/terminal pair one after another by using near-field
communications (NFC). Alternatively, a user can commit to the
private cloud (which may be one of the UI/terminal pairs) via NFC
and the private cloud can then route the configuration to an
individual UI/terminal pair.
[0025] With another aspect, system security may be realized via
audio signature when audio service is available in the private
cloud or individual UI/terminal pair. A user may use voice commands
to unlock the system. Alternatively, the NFC of the user's mobile
device may also be used, assuming that only the user can unlock
his/her mobile device. Voice signature or NFC access may also be
used for system personalization to configure the system based on
the individual user's historical data to meet his/her preference
system behavior(s).
[0026] With another aspect, a scalable smart control system
controls at least one controlled apparatus, where the control flow
and the user interface graphics of a UI unit in a mobile app are
identical to that in a physical UI unit. The status of each UI unit
in the dash board of the mobile app is identical to the current
status as shown in the physical UI unit. A user may choose a UI
unit from the dash board of the mobile app for further
manipulation, as it is done on the spot. This eliminates the
learning needs in using the mobile app and allows a unified user
experience in using the UI unit as well as the mobile app.
BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The foregoing summary of the invention, as well as the
following detailed description of exemplary embodiments of the
invention, is better understood when read in conjunction with the
accompanying drawings, which are included by way of example, and
not by way of limitation with regard to the claimed invention.
[0028] FIG. 1 shows system building blocks of a gateway thermostat
in accordance with an embodiment.
[0029] FIG. 2 shows multiple user interfaces (UI) units controlling
a single terminal unit in accordance with an embodiment.
[0030] FIG. 3 shows a single UI unit controlling multiple terminal
units in accordance with an embodiment.
[0031] FIG. 4 shows a connection hub between multiple UI units and
terminal units in accordance with an embodiment.
[0032] FIG. 5 shows a user interface unit in accordance with an
embodiment.
[0033] FIG. 6 shows a terminal unit in accordance with an
embodiment.
[0034] FIG. 7 shows a system with multiple UI units, multiple
terminal units, a mobile app, and a web browser app in accordance
with an embodiment.
[0035] FIGS. 8-12 show exemplary embodiments with one or more UI
units and one or more terminal units.
[0036] It will be apparent to one skilled in the art after review
of the entirety disclosed that the steps illustrated in the figures
listed above may be performed in other than the recited order, and
that one or more steps illustrated in these figures may be
optional.
DETAILED DESCRIPTION
[0037] In the following description of various illustrative
embodiments, reference is made to the accompanying drawings, which
form a part hereof, and in which is shown, by way of illustration,
various embodiments in which aspects of the disclosure may be
practiced. It is to be understood that other embodiments may be
utilized, and structural and functional modifications may be made,
without departing from the scope of the present disclosure.
[0038] It is noted that various connections between elements are
discussed in the following description. It is noted that these
connections are general and, unless specified otherwise, may be
direct or indirect, wired or wireless, and that the specification
is not intended to be limiting in this respect.
[0039] Illustrative embodiments of the present invention will now
be described more fully hereinafter with reference to the
accompanying drawings, in which some, but not all, embodiments of
the invention are shown. Indeed, the invention may be embodied in
many different forms and should not be construed as limited to the
embodiments set forth herein; rather, these embodiments are
provided so that this disclosure will satisfy applicable legal
requirements. Like numbers refer to like elements throughout.
[0040] As will be appreciated by one of skill in the art in view of
this disclosure, the present invention may be embodied as an
apparatus (for example, a system, computer executable product,
and/or other device), a method, or a combination of the foregoing.
Accordingly, embodiments of the present invention may take the form
of an entirely hardware embodiment, an entirely software embodiment
(including firmware, resident software, micro-code), or an
embodiment combining software and hardware aspects that may
generally be referred to herein as a "system." In addition, various
signals representing data or events as described herein may be
transferred between a source and a destination in the form of light
or electromagnetic waves traveling through signal-conducting media
such as metal wires, optical fibers, or wireless transmission media
(e.g., air or space). In general, the one or more computer-readable
media may comprise one or more non-transitory computer-readable
media. Embodiments of the present invention are described below
with reference to flowchart illustrations and/or block diagrams of
processes or apparatuses (the term "apparatus" including systems
and computer executable products). It will be understood that each
block of the flowchart illustrations and/or block diagrams, and
combinations of blocks in the flowchart illustrations and/or block
diagrams, can be implemented by computer executable instructions.
These computer executable instructions may be provided to a
processor of a special purpose computer or other executable data
processing apparatus to produce a particular machine, such that the
instructions, which execute by the processor of the computer or
other executable data processing apparatus, create mechanisms for
implementing the functions/acts specified in the flowchart and/or
block diagram block or blocks.
[0041] These computer executable instructions and algorithms
described herein may also be stored in a computer-readable memory
that can direct a computer or other executable data processing
apparatus to function in a particular manner, such that the
instructions stored in the computer readable memory produce an
article of manufacture including instructions, which implement the
function/act specified in the flowchart and/or block diagram block
or blocks.
[0042] FIG. 1 shows system building blocks 101 and 102 of gateway
thermostat 100 in accordance with an embodiment.
[0043] System 100 includes terminal unit 102 and user interface
(UI) unit 101. Each unit may be housed separately and connected via
either wireless means or wired means 151.
[0044] A wireless connection may support Sub-1G, ZWave, WiSun,
Zigbee, Thread, BLE, WiFi, NFC, 2G/3G/4G modem or IrDa or any
combination of these wireless technologies.
[0045] A wired connection may support a direct connection of the
system control signals. It may also support any wired interface,
such as SPI, I2C, UART, USB, CAN, Ethernet, Opentherm, RS232/RS485
running FSK, eBus, EMS, MP Bus, Modbus, BACnet, and the like.
[0046] A proprietary protocol may be defined for exchanging system
information as well as terminal controls.
[0047] System flexibility for multiple connection scenarios may be
enabled by using the design topology shown in FIG. 1.
[0048] FIG. 2 shows multiple user interfaces (UI's) 201-204
controlling single terminal unit 205 in accordance with an
embodiment.
[0049] Multiple UI units 201-204 may be able to control single
terminal unit 205. The feature requirements for different UIs may
be different. With one embodiment, UI unit 201 (for example,
located in the sitting room) may be a comprehensive UI unit that is
configured to control other appliances, such as a TV (not
explicitly shown), game console 206, and/or multimedia console 207.
UI unit 202 may be the control panel of a home appliance 208, such
as an oven.
[0050] FIG. 3 shows single UI 301 controlling multiple terminal
units 302-305 in accordance with an embodiment.
[0051] Single UI unit 301 is able to control a range of terminal
units 302-305. The feature requirements for different terminal
units 302-305 may be different. For example, UI unit 301 may
control a first terminal unit at home (which may have three-stages
of heat and cool, a heat pump, a fan and a de-humidifier) and a
second terminal unit in an individual store room away from home
(which may only have one-stage heat and cool, a heat pump, and a
fan).
[0052] The following is an example illustrating the operation
between UI unit 301 and a terminal unit 302. A user sets a target
temperature to UI unit 301. UI unit 301 then sends the target
temperature to terminal unit 302. Terminal unit 302 may act as a
thermostat or may control an external thermostat (not explicitly
shown). Using its rule engine to estimate how long it will take to
reach the target temperature, terminal unit 302 decides when to use
different stages of a heating and cooling system (not explicitly
shown). If a first stage of heating/cooling cannot reach the
desired temperature in a reasonable amount of time, terminal unit
302 may activate additional stages of heating or cooling to ensure
achieving the target temperature.
[0053] The following is another example illustrating the operation
between UI unit 301 and a terminal unit 302. A user is on the way
back home and performs the following actions via a smart home app
executing on his/her mobile device. [0054] Sets the target
temperatures of his/her sitting room and executes the setting.
[0055] Sets the target time and temperatures of his/her bed room as
well as his/her children's rooms. [0056] Programs the washing
machine [0057] Sets the time and temperature of the oven [0058]
When the user arrives home, the user puts all the dirty clothes
into the washing machine and a turkey into the oven. Referring to
FIG. 3, the user then puts the near field communications (NFC)
device of his/her mobile device near central UI unit 301 in the
sitting room (which also functions as the connection hub of his
home). Central UI unit 301 then routes the settings to the
corresponding terminal units 302-305 within the local loop,
including the HVAC terminal units for the bed rooms, the terminal
unit of the washing machine, and the terminal unit of the oven.
[0059] FIG. 4 shows a connection hub between multiple UI units
401-404 and terminal units 405-408 in accordance with an
embodiment.
[0060] With some embodiments, multiple UI units 401-404 and
terminal units 405-408 are connected together via connection hub
412. Control commands and status reports are collected in
connection hub 412 and may be routed to target devices within the
web. Connection hub 412 may be implemented in a public/private
cloud 413 provided by a service provider or within one of the UI
units 401-404.
[0061] In the embodiments shown in FIGS. 2-4, the UI unit can be a
gateway which connects all other devices to a public cloud.
Alternatively, the UI unit may be a connection hub which routes the
commands and status reports to target devices within the web.
Alternatively, the UI unit may be a private cloud, in which all the
controls and status report may be manipulated within the user's
home network before going to a public domain. Embodiments may
include a rule engine (for example mapping events to actions) for
all the smart devices within a home network and only the
user-defined control and information are sent to public cloud for
further manipulation. The UI unit may also be equipped with
capabilities of a gateway, a connection hub and private cloud.
[0062] FIG. 5 shows a user interface (UI) unit 500 in accordance
with an embodiment. Components of UI unit 500 comprise a basic
subsystem, a sensor subsystem, a UI subsystem, a connectivity
subsystem, and a security subsystem.
[0063] The basic subsystem includes main processor 501, memory
device 502, power management unit 503, and other system components
as will be discussed.
[0064] Main processor 501 may be a MCU, a SoC of a single processor
core, or a multi-core SoC with numerous co-processor or
accelerators. Multiple processors may also be used depending on the
system architecture design.
[0065] Different combination of memory device(s) 502 may be used
(such as SSD, eMMC, NAND, NOR, DDR3, SDRAM, EEPROM) depending on
the feature requirements. If a private cloud is implemented into a
UI unit, a large size eMMC or an SSD may be used for data
warehousing and data mining as well as machine learning.
[0066] The core building block of the power management unit (PMU)
503 may be discrete components or a complex power management IC
(PMIC) which may include multiple DC-DC converts, LDOs as well as
battery charging circuitry 521 and/or external battery 522. Some
embodiments may require separate real-time clock (RTC) 523 with
backup power source (a re-charge coil cell battery or a super-cap
524), in order to provide an accurate clock source when system 500
in deep sleep mode.
[0067] Other basic system components may include debug port(s) 504
for system debugging, keys 525 for resetting system 500 and
configuring different system boot modes, and information apparatus
526 that is indicative of the hardware version and the like.
[0068] A sensor subsystem may include multiple sensors to enable a
"smart environment". The sensors include, but not limit to the
following:
[0069] Temperature and relative humidity sensor (T/RH) 505: A combo
temperature (T) and relative humidity (RH) sensor may be used to
monitor the ambient temperature and relative humidity.
[0070] Temperature sensor(s): Multiple temperature sensors (Temp)
506 may be installed for measuring the ambient temperature as well
as the internal temperature of the UI unit for temperature
compensation.
[0071] Accelerometer: Accelerometer 507 may be installed to detect
the orientation of the UI unit and to use the corresponding UI/UX.
Accelerometer 507 may also be used as user inputs, through
different tapping patterns and directions.
[0072] PIR sensor: Passive infrared (PIR) sensor 508 may be
installed to detect the presence of human object(s) within the area
of interests. One of the applications is directed to energy saving,
in which some appliances may only be operated when human object(s)
are detected. For example the backlight of a UI unit may be turned
on when a human object is detected.
[0073] Proximity sensor: Proximity sensor 509 may be equipped to
detect the proximity of a human object. In such case, the UI unit
may prepare all the essence information for device operation, for
example prompting up the setting page for temperature and operation
mode when an object is detected within a pre-defined distance.
[0074] Ambient light sensor: An ambient light sensor (not
explicitly shown) may be equipped for backlight adjustment
according to ambient light intensity for user experience
enhancement.
[0075] The UI unit can also receive inputs from external sensor
through wired or wireless means.
[0076] Inputs from combination of sensors can be used together to
provide means to enhance user experience in the smart
environment.
[0077] As will be discussed, the user interface subsystem may
support inputs from touch panel 511, microphone array 512, and
feature keys 513 as well as support outputs to display 514, speaker
array 515, LED(s) 516, and vibrator 517. Other user inputs and
outputs means may also be added to enhance the user experience of
the smart environment.
[0078] Display and capacitive touch panel 511 may include
traditional UI means.
[0079] Depending on the application requirement, a display may be
segment LEDs, or a segment TN LCD, or a PMOLED LCD, or a dot-matrix
TN LCD, or a color TFT LCD, or an IPS TFT LCD, or an AMOLED
LCD.
[0080] For touch panels, resistive touch or capacitive touch may be
supported. Gesture inputs as well as multiple touch inputs may also
be supported.
[0081] Regarding speaker array 515 and microphone array 512, audio
UI means may also be used to provide a more natural user
experience. Microphone array 512 together with audio processing
algorithms (supported by device 527) may be used to provide a clear
voice command understand noisy environment. The speech recognition
algorithm may be deployed to understand the commands. Speaker array
515 may be used to provide a voice feedback.
[0082] Microphone array 512 may be used to detect the position of a
user and hence the voice feedback can be focused towards the user's
position when a more complex speaker array is used.
[0083] Other system inputs include the count and direction of taps
from accelerometers 507, feature keys 513, and the like.
[0084] Other system outputs include system LEDs 516 and vibrator
517. System outputs may incorporate a combination of different LED
colors, mark space ratios of the LED's, vibration patterns, and so
forth.
[0085] The connectivity subsystem includes wireless connectivity
and wired connectivity.
[0086] Wireless connectivity subsystem, 518 depending on the
application requirements, may be a combination of Sub-1G, Wisun,
Zwave, Zigbee, Thread, BLE, WiFi, NFC, 2G/3G/4G modem or IrDa, and
so forth.
[0087] Wired connectivity subsystem 519 may be a combination of
I2C, SPI, UART, RS232, RS485, USB, CAN, Ethernet, Opentherm, and so
forth.
[0088] Security subsystem 520 may be included to avoid
un-authorized system access and to protect user identity and data
from the smart devices. This may be important for IoT application,
where data of smart devices are uploaded to the public cloud for
data mining and machine learning of user behavior.
[0089] General system access can be done through password
input.
[0090] Voice signature may also be used to prevent un-authorized
system access, verification of user identity and hence to enable
personalized system profile.
[0091] Encryption may be applied when data is exchanged between a
UI unit and public cloud.
[0092] Data encryption may be done by SW implementation.
Alternatively, an external data encryption IC (not explicitly
shown) can be used to further strengthen data security.
[0093] FIG. 6 shows terminal unit 600 in accordance with an
embodiment.
[0094] The building blocks of terminal unit 600 include a basic
subsystem, a connectivity subsystem, and a terminal control
subsystem.
[0095] Similar to the UI unit, the basic subsystem of terminal unit
600 may include main processor 601, its corresponding memory
devices (not explicitly shown), power management unit 602, and
other system components.
[0096] Main processor 601 may be a MCU with limited memory that is
powerful enough to detect and control its associated terminals as
well as communicating to its associated devices, including smart
sensor(s) and/or UI unit(s).
[0097] Power management unit (PMU) 602 may be implemented with
discrete components, including AC/DC converter, DC/DC converters
and LDOs.
[0098] RTC 604 may be used to provide the clock when terminal unit
600 is in deep sleep mode and no RTC is available from the UI
unit(s).
[0099] Other basic system components include debug port(s) 605 for
system debugging, keys 606 for resetting system 600 and configuring
different system boot modes, information apparatus 607 for storing
information indicative of the hardware version and so forth.
[0100] The terminal control subsystem includes terminal detection
and control.
[0101] Detection circuit 608 is implemented to detect if an
appliance is connected to a terminal and/or if the appliance is
disconnected (malfunction) during operation.
[0102] Terminal control (ON/OFF) 609 is enabled when an appliance
is detected.
[0103] The connectivity subsystem includes wireless connectivity
and/or wired connectivity, depending on the application
requirements.
[0104] Wireless connectivity subsystem 610 may support a unique
protocol which may be sub-1G, Wisun, Zwave, Zigbee, Thread, BLE,
WiFi, NFC, 2G/3G/4G modem or IrDa, and the like.
[0105] Wired connectivity subsystem 611 may include an I2C, SPI,
UART, RS232, RS485, USB, CAN, Ethernet, Opentherm interfaces. Wired
protocol(s) may be implemented using the connection, such as
Modbus, BACnet, EBus, EMS Bus, MP Bus, FSK or a proprietary
protocol. Direct connection of the control signal may be allowed if
UI unit 500 and terminal unit 600 are at the same location.
[0106] With reference to FIGS. 5 and 6, the computing system
environment may include a computing device wherein the processes
discussed herein may be implemented. The computing device may have
a processor for controlling overall operation of the computing
device and its associated components, including RAM, ROM,
communications module, and memory device. The computing device
typically includes a variety of computer readable media. Computer
readable media may be any available media that may be accessed by
computing device and include both volatile and nonvolatile media,
removable and non-removable media. By way of example, and not
limitation, computer readable media may comprise a combination of
computer storage media and communication media.
[0107] Computer storage media may include volatile and nonvolatile,
removable and non-removable media implemented in any method or
technology for storage of information such as computer readable
instructions, data structures, program modules or other data. A
program module may include computer-executable instructions that
are executed by one or more processors. Computer storage media
include, but is not limited to, random access memory (RAM), read
only memory (ROM), electronically erasable programmable read only
memory (EEPROM), flash memory or other memory technology, CD-ROM,
digital versatile disks (DVD) or other optical disk storage,
magnetic cassettes, magnetic tape, magnetic disk storage or other
magnetic storage devices, or any other medium that can be used to
store the desired information and that can be accessed by the
computing device.
[0108] Communication media typically embodies computer readable
instructions, data structures, program modules or other data in a
modulated data signal such as a carrier wave or other transport
mechanism and includes any information delivery media. Modulated
data signal is a signal that has one or more of its characteristics
set or changed in such a manner as to encode information in the
signal. By way of example, and not limitation, communication media
includes wired media such as a wired network or direct-wired
connection, and wireless media such as acoustic, RF, infrared and
other wireless media.
[0109] The following capabilities may be supported by the
embodiments. [0110] A thermostat system which consists of two
separate sub-blocks for system flexibility. [0111] The first
sub-block is for terminal control, the terminal unit. The number of
control terminals and their functions are configurable according to
the product requirements. [0112] The second sub-block is for user
interface (UI), the UI unit. The complexity of the UI unit may be
changed according to the product requirements, the complexity of
user experience and the graphics. Examples include portrait or
landscape detection, voice control and feedback, user
identification, human presence detection, user proximity detection,
ambient light tracking, fun lighting, haptic feedback, 2D/3D
animations, and so forth. [0113] The communication means between
the two sub-blocks may depend on the user cases, including: [0114]
a. Directly connecting the control signals through connectors when
the two sub-blocks are located at the same spot. [0115] b. Wireless
communication when the two units are separated apart from each
other. [0116] c. Wired communication in scenario where wireless
communication is not feasible. [0117] Embodiments may utilize the
thermostat design topology shown in Figure [0118] a. A UI unit is
able to control a single terminal unit or multiple terminal units.
[0119] b. A terminal unit may be controlled by a single Li unit or
multiple UI units. [0120] c. Multiple UI units and terminal units
may be connected together through a connection hub. [0121] A
thermostat system may be extended to include different smart
devices. Moreover, flexibility is allowed for different terminal
units to be associated with different UI units, for interactions.
The smart devices may be, but not limit to, smart home devices
(e.g. thermostats, light bulbs, smart sensors, smart plugs, etc.),
home appliances, a game console, media devices, healthcare devices,
and so forth. [0122] The terminal unit implements only one or
numerous control terminal. Examples include: [0123] a. A
comprehensive set of control terminals including heats, cools, heat
pumps, fans, fan speeds, de-humidifier, auxiliary, and so forth.
[0124] b. A single control terminal for boiler on/off. [0125] c. A
single control terminal to produce a pulse width modulation output,
such as those used for motor control or heating elements. [0126]
The UI unit is equipped with gateway capability to connect the
terminal unit and other smart devices to public cloud. [0127] The
UI unit is also equipped with private cloud capability, including
but not limited to: [0128] a. Data warehouse to record the history
of the terminal unit and other smart devices. [0129] b. rule engine
which implements all the rules for the interaction between the
terminal unit and other smart devices, [0130] As an example, a user
sets a target temperature to warm up a room. The rule engine may
set a first stage algorithm to control a terminal unit or more
terminal units to reach the target temperature in a reasonable
time. The first stage algorithm may be obtained from a user's input
or predefined heuristic algorithm or may be generated by artificial
intelligence (AI) of the rule engine based on collecting data from
the environment. The terminal unit(s) may feedback temperature(s)
to the rule engine. If the target temperature is not achieved in a
reasonable time, the rule engine may control terminal(s) with a
second stage algorithm or even a third stage algorithm in order to
achieve the target temperature and thus ensure comfort to the user.
[0131] As another example, a user always turns on the thermostats
of the bed rooms and turns off the thermostat of the sitting room,
the oven, the microwave and cooker in the kitchen. All of these
actions may be captured by the UI units in the corresponding smart
device and sent to the central UI unit. The central UI unit learns
the user profile. One day, the user goes to bed and forgets to turn
off the oven. The central UI unit automatically sends a turn off
command to the UI unit of the oven and then sends a warning to the
user. [0132] c. A learning model to learn and adapt to the user
behavior in using the terminal unit and other smart devices. [0133]
The wireless connectivity subsystem of the UI unit supports, but is
not limited to, WiFi, BLE, Zigbee, ZWave, and NFC. [0134] a. The
WiFi subsystem enables internet access to the UI unit and other
smart devices connected to it. The WiFi subsystem also allows
mobile app control via WiFi direct. [0135] b. The Zigbee subsystem,
ZWave subsystem and BLE subsystem connect to the smart devices
under the same protocols, for example, Zigbee thermostat, smart
plugs and light bulbs, ZWave thermostats and sensors, BLE heartbeat
monitor and weight scale, and so forth. [0136] c. The BLE subsystem
also enables directly control of smart devices through mobile app.
[0137] d. The NFC subsystem provides a direct and easy way for UI
units to pair with terminal units and/or transfer the profiles to
the UI units, thereafter routing to the target UI unis and/or
terminal units. [0138] The wired connectivity subsystem of the UI
unit consists of, but not limited to, an Ethernet and an RS485
connections. [0139] a. The Ethernet connects UI unit directly to
internet. [0140] b. The RS485 allows wired protocol to be
implemented for controlling the smart devices which only have wired
connectivity available, for example Modbus or BACnet [0141] The UI
unit equips with an accelerometer for portrait or landscapes UI
mode. Moreover, it allows user inputs by tapping the UI unit from
different directions with different count of tabs. [0142] The audio
subsystem of the UI unit consists of a mic array with two digital
microphones and a speaker array with two speakers. [0143] a. The
audio subsystem is used for user identification, speech
recognition, audio feedback as well as audio playback. [0144] b.
User identification is used for access control to the UI unit.
[0145] c. User identification is also used for retrieving personal
profile for automatic control of all the smart devices as well as
learning of user behavior. [0146] d. Speech recognition is used for
controlling the smart devices. During the speech recognition
process, the direction of the user is also estimated. [0147] e. The
audio feedback can be focused towards the direction of the user
estimated. [0148] The UI unit has a large memory to store the data
from a range of connected smart devices. A rule engine may be
implemented to enable the interaction between smart devices. The
interactive rules may be set by users through the UI unit or via a
mobile device. Alternatively, it may be learnt directly via a
learning algorithm implemented within the UI unit(s), private cloud
or public cloud. [0149] A learning model is implemented to enable
learning of user behavior. [0150] a. Supervised learning is started
by using the rule engine to understand the "Things of Interests"
(TOI). [0151] b. The model learns the user behavior through the
data from the smart devices and associated user inputs. [0152] c.
The model repeats the user behavior when a pre-defined confidence
level is reached. [0153] d. Reinforce learning is implemented when
user correction is made to the machine action. [0154] e. Machine
learning may be conducted in the public cloud server or private
cloud server (which reside within a UI unit) or distributed amongst
the number of UI units within a network. [0155] A simple rule
engine is also implemented in the terminal unit, when a UI unit or
network connection is not available, which take the inputs from
associated smart devices to execute the controls. An example would
be the terminal control would be enabled when user presence is
confirmed from an external PIR sensor. [0156] Rules related to the
terminal unit a downloaded to the terminal unit when it is
established and executed accordingly within the terminal unit.
[0157] A mobile application clones the user interface of the UI
unit. The usage model is such that a user is using the UI unit on
the spot. For example, when a user uses a mobile app to control a
smart device, the UI unit and the control flow of the mobile app
cooperate together. An advantage of this approach is a unique user
experience in using the smart device and its associated mobile app
so that a user does not have to learn two user interface schemes
for a single smart device (one in the UI unit and the one in the
mobile app). [0158] The mobile application is scalable such that it
can select all the smart devices which connected to the UI unit.
Conversely, it can de-select a smart device from the list. [0159]
Referring to the embodiment shown in FIG. 7, system 700 includes 2
UI units 701 and 702a, 4 TML units 702b, 703, 704, and 705, mobile
app 706, and PC web browser app 707. While not explicitly shown,
some of the elements may interact with each other via wired or
wireless communication channels. UI1 701 comprises a home console
that supports a home gateway, a rule engine, and a connection hub.
Device control from UI1 701 may be performed via a capacitve touch
panel and/or voice commands. UI2/TML2 pair 702a,702b supports a
thermostat in which UI unit 702a and terminal unit 702b are wiredly
connected via a proprietary control protocol. TML3 703 may be a
simple terminal unit which controls a relay of a heating element.
TML4 704 and TML5 705 may be terminal units controling the LED
lighting at different zones. There was no dedicated UI for TML3
703, TML4 704, and TML5 705; rather, control was performed via
mobile app 706, PC web browser app 707, and/or UI1 701. All the
devices may be added to mobile app 706, PC web browser app 707,
and/or UI1 701 for smart home control. As shown in FIG. 7, the
UI/UX for all UI devices 701 and 702a, mobile app 706, and PC web
browser app 707 are the same to enable a good user experience.
(Display components 701-755 correspond to UI1 701, UI2/TML2
702a,702b, TML3 703, TML4 704, and TML5 704, respectively.) The
status of all the devices are synchronized and displayed in mobile
app 706, and PC web browser app 707.
[0160] Exemplary embodiments include:
[0161] As shown in FIG. 8, a scalable smart control system 800
controls a plurality of controlled apparatuses, where the system
comprises:
first and second user-interface (UI) units 801 and 802; a terminal
unit 803 controlling a first controlled apparatus 804; [0162] when
the first UI unit 801 receives first information, the first UI unit
801 sends the first information to the terminal unit 803 and
controls the first controlled apparatus 804 via the terminal unit
803; [0163] when the second UI unit 802 receives second
information, the second UI unit 802 sends the second information to
the terminal unit 803 and controls the first controlled apparatus
804 via the terminal unit 803; and [0164] when the first UI unit
801 receives third information, the first UI unit 801 directly
controls a second controlled apparatus 805.
[0165] As shown in FIG. 9, a scalable smart control system 900
controls a plurality of controlled apparatuses, where the system
comprises:
a user-interface (UI) unit 901; a first terminal unit 902
controlling a first controlled apparatus 904; a second terminal
unit 903 controlling a second controlled apparatus 905; the UI unit
901 receiving first target data and second target data; the UI unit
sending the first target data to the first terminal unit 902 and
the second target data to the second terminal unit 903; the first
terminal unit 902 controlling the first controlled apparatus 904 in
accordance with the first target data; and the second terminal unit
903 controlling the second controlled apparatus 905 in accordance
with the second target data.
[0166] As shown in FIG. 10, a scalable smart control system 1000
controls a plurality of controlled apparatuses, where the system
comprises:
first and second user-interface (UI) units 1001 and 1002; first and
second terminal units 1003 and 1004 controlling first and second
controlled apparatuses 1005 and 1006, respectively; a connection
hub 1007; the first UI unit 1001 receiving first control data and
sending a subset of the first control data to a cloud computing
resource 1008; the first UI unit 1001 receiving, from the cloud
computing resource 1008, a processed subset of the first control
data; the first UI unit 1001 obtaining processed first control data
from the processed subset; the second UI 1002 unit receiving second
control data; the first UI unit 1001 sending the processed first
control data and the second UI unit 902 sending the second control
data to the connection hub 1007; the connection hub 1007 collecting
and routing the processed first control data and the second control
data to the first and second terminal units 1003 and 1004,
respectively; the first terminal unit 1003 controlling the first
controlled apparatus 1005 based on the processed first control
data; and the second terminal unit 1004 controlling the second
controlled apparatus 1006 based on the second control data.
[0167] As shown in FIG. 11, a scalable smart control system 1100
controls at least one controlled apparatus, where the control flow
and the user interface graphics of a UI unit in a wireless device
app 1107 (executed by a wireless device) is identical to that in
the physical UI unit 1101, 1102, and 1103. The status of each UI
unit in the dash board of the mobile app 1107 is identical to the
current status shown in the physical UI unit 1101, 1102, and 1103.
A user can choose a UI unit from the dash board of the wireless
device app 1107 for further manipulation, as it is done on the
spot. This eliminates the learning needs in using the wireless
device app 1107 and allows a unified user experience in using the
UI unit as well as the wireless device app 1107.
[0168] Physical UI unit 1101, 1102, 1103 controls smart devices
1104, 1105, and 1106, respectively, and may do so through terminal
units not explicitly shown in FIG. 11.
[0169] The wireless device may include a computing device for
executing wireless device app 1107. may include a computing device
wherein the processes discussed herein may be implemented. The
computing device may have a processor for controlling overall
operation of the computing device and its associated components,
including RAM, ROM, communications module, and memory device. The
computing device typically includes a variety of computer readable
media. Computer readable media may be any available media that may
be accessed by computing device and include both volatile and
nonvolatile media, removable and non-removable media. By way of
example, and not limitation, computer readable media may comprise a
combination of computer storage media and communication media.
[0170] As shown in FIG. 12, a scalable smart control system 1200
controls at least one controlled apparatus, where the system
comprises:
first and second user-interface (UI) units 1201 and 1202; a
terminal unit 1203 controlling a controlled apparatus through the
second UI unit 1201; the first UI unit 1201 comprising a near-field
communications (NFC) interface 1206; the first UI unit 1201
receiving configuration data about the controlled apparatus 1204
from a NFC device 1205 through the NFC interface 1206; the first UI
unit 1201 routing the configuration data to the second UI unit
1202; the second UI unit 1202 sending the configuration data to the
terminal unit 1203; and the terminal unit 1203 controlling the
controlled apparatus 1204 in accordance with the configuration
data.
[0171] With another aspect of the embodiments, rules may be
automatically generated via machine learning program module.
Machine learning may be applied locally in the UI unit of a local
loop. Machine learning may also be applied to information received
from multiple UI units of a larger network. With some embodiments,
the machine learning program module may be executed within a single
UI unit that has system hardware with better performance. A machine
learning program module may also be distributed over different UI
units to reduce the computation loading for individual UI units.
The resulting rules determined by the distributed machine learning
module may be loaded to the corresponding terminal units to reduce
response latency. With some embodiments, the machine learning
program module and resulting rules may be executed when the
internet is disconnected.
[0172] One or more aspects of the disclosure may be embodied in
computer-usable data or computer-executable instructions, such as
in one or more program modules, executed by one or more computers
or other devices to perform the operations described herein.
Generally, program modules include routines, programs, objects,
components, data structures, and the like that perform particular
tasks or implement particular abstract data types when executed by
one or more processors in a computer or other data processing
device. The computer-executable instructions may be stored on a
computer-readable medium such as a hard disk, optical disk,
removable storage media, solid-state memory, RAM, and the like. The
functionality of the program modules may be combined or distributed
as desired in various embodiments. In addition, the functionality
may be embodied in whole or in part in firmware or hardware
equivalents, such as integrated circuits, application-specific
integrated circuits (ASICs), field programmable gate arrays (FPGA),
and the like. Particular data structures may be used to more
effectively implement one or more aspects of the disclosure, and
such data structures are contemplated to be within the scope of
computer executable instructions and computer-usable data described
herein.
[0173] As can be appreciated by one skilled in the art, a computer
system with an associated computer-readable medium containing
computer-executable instructions for controlling the computer
system can be utilized to implement the exemplary embodiments that
are disclosed herein. The computer system may include at least one
computer such as a microprocessor, digital signal processor, and
associated peripheral electronic circuitry.
[0174] Aspects of the invention have been described in terms of
illustrative embodiments thereof. Numerous other embodiments,
modifications and variations within the scope and spirit of the
disclosed invention will occur to persons of ordinary skill in the
art from a review of this entire disclosure. For example, one of
ordinary skill in the art will appreciate that the steps
illustrated in the illustrative figures may be performed in other
than the recited order, and that one or more steps illustrated may
be optional in accordance with aspects of the disclosure.
* * * * *