U.S. patent application number 14/555170 was filed with the patent office on 2015-07-02 for display apparatus, terminal apparatus, and controlling methods thereof.
This patent application is currently assigned to SAMSUNG ELECTRONICS CO., LTD.. The applicant listed for this patent is SAMSUNG ELECTRONICS CO., LTD.. Invention is credited to Ga-hyun JOO, Chang-won KIM, Jung-hyun SHIM.
Application Number | 20150188724 14/555170 |
Document ID | / |
Family ID | 52102383 |
Filed Date | 2015-07-02 |
United States Patent
Application |
20150188724 |
Kind Code |
A1 |
KIM; Chang-won ; et
al. |
July 2, 2015 |
DISPLAY APPARATUS, TERMINAL APPARATUS, AND CONTROLLING METHODS
THEREOF
Abstract
A display apparatus, a terminal apparatus, and controlling
methods are provided. The display apparatus includes: a
communicator to communicate with a terminal apparatus or a relay
server in order to receive collected status information about at
least one peripheral device from the terminal apparatus, a display
unit to display a user interface (UI) screen; and a controller to
generate a control UI for controlling the at least one peripheral
device or a security UI for monitoring a peripheral space through
the at least one peripheral device based on the status information
about the at least one peripheral device and based on whether a
communication with the terminal apparatus is possible, thereby
controlling a home device t in real time according to a
communication status with the terminal apparatus.
Inventors: |
KIM; Chang-won; (Asan-si,
KR) ; JOO; Ga-hyun; (Suwon-si, KR) ; SHIM;
Jung-hyun; (Yongin-si, KR) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
SAMSUNG ELECTRONICS CO., LTD. |
Suwon-si |
|
KR |
|
|
Assignee: |
SAMSUNG ELECTRONICS CO.,
LTD.
Suwon-si
KR
|
Family ID: |
52102383 |
Appl. No.: |
14/555170 |
Filed: |
November 26, 2014 |
Current U.S.
Class: |
340/3.71 ;
340/12.53 |
Current CPC
Class: |
G06F 21/305 20130101;
H04L 12/4625 20130101; H04N 21/43615 20130101; H04N 21/41407
20130101; G06F 2221/2101 20130101; G05B 2219/2642 20130101; G06F
2221/2111 20130101; G08B 3/10 20130101; G06F 3/048 20130101; G08C
2201/30 20130101; H04N 21/4227 20130101; H04N 7/181 20130101; G08C
2201/93 20130101; H04L 12/282 20130101; H04N 21/4725 20130101; H04N
21/4753 20130101; G06F 2221/2141 20130101; H04N 21/4131 20130101;
H04M 1/72533 20130101; H04L 12/2818 20130101; H04W 4/021 20130101;
H04N 21/4223 20130101; G08C 2201/42 20130101; H04L 12/2834
20130101; G05B 15/02 20130101; H04N 21/2187 20130101 |
International
Class: |
H04L 12/28 20060101
H04L012/28; G05B 15/02 20060101 G05B015/02; G06F 3/048 20060101
G06F003/048 |
Foreign Application Data
Date |
Code |
Application Number |
Dec 31, 2013 |
KR |
10-2013-0169113 |
Claims
1. A display apparatus comprising: a communicator configured to
communicate with at least one of a terminal apparatus and a relay
server to receive from the terminal apparatus collected status
information corresponding to at least one peripheral device; a
display unit configured to display a user interface (UI) screen;
and a controller configured to generate at least one of a control
UI for controlling the at least one peripheral device and a
security UI for monitoring a peripheral space using the at least
one peripheral device based on the status information corresponding
to the at least one peripheral device and based on whether
communication with the terminal apparatus is possible, and display
the at least one of the control UI and the security UI on the
display unit.
2. The display apparatus of claim 1, wherein, if communication with
the terminal apparatus is possible, the controller generates a
control list information about at least one controllable peripheral
device, from among the at least one peripheral device, based on the
received status information, and generates the control UI
comprising a control menu corresponding to one of the at least one
controllable peripheral device selected based on the control list
information.
3. The display apparatus of claim 1, wherein, if communication with
the terminal apparatus is not possible, the controller generates a
security list information comprising identification information
indicating the at least one peripheral device to be monitored and
an area information depicting a monitoring area based on the
received status information and setting information corresponding
to each of the at least one peripheral device.
4. The display apparatus of claim 3, wherein the controller is
further configured to : generate a user certification UI for
certifying a user in response to one of the at least one peripheral
device being selected based on the security list information,
request the terminal apparatus, using the relay server, to monitor
the selected one of the at least one peripheral device in response
to an encryption key input on the user certification UI
corresponding to preset encryption information, and generate a
security UI for displaying image information that is captured by
the selected peripheral device and received from the relay server,
according to the monitoring request.
5. The display apparatus of claim 4, wherein the security UI
comprises a monitoring control menu for controlling a monitoring
operation of the selected peripheral device.
6. The display apparatus of claim 5, wherein the security UI
further comprises an editing menu for editing an image captured by
the selected peripheral device.
7. The display apparatus of claim 2, wherein the controller
generates the control list information based on an execution mode
of the display apparatus, wherein the execution mode is at least
one of a first execution mode activated on a quick menu, a second
execution mode activated on a lock screen, and a third execution
mode in which a home network service-related application is
activated.
8. The display apparatus of claim 7, wherein the controller
generates a first control list information comprising the at least
one controllable peripheral devices in the first execution mode and
generates a second control list information comprising one
activated controllable peripheral device from among the at least
one controllable peripheral device in the second execution
mode.
9. The display apparatus of claim 7, wherein, if in the third
execution mode, the controller generates a control UI comprising
status-related detailed information and a control menu of the at
least one controllable peripheral device based on the received
status information regardless of whether communication with the
terminal apparatus is possible.
10. A terminal apparatus comprising: a communicator configured to
periodically communicate with at least one peripheral device; a
storage unit configured to store status information about the at
least one peripheral device; and a controller configured to, in
response to a status information request message requesting status
information of the at least one peripheral device being received
from a display apparatus, control the communicator to transmit, to
the display apparatus, at least one of status information about at
least one controllable peripheral device, from among the at least
one peripheral device, and the status information about the at
least one peripheral device to be monitored based on the status
information stored in the storage unit and based on whether a
communication with the display apparatus is possible.
11. The terminal apparatus of claim 10, wherein, if communication
with the display apparatus is possible, the controller transmits
the status information about the at least one controllable
peripheral device based on the status information stored in the
storage unit and, in response to a control command being received
from the display apparatus, transmits the control command to one of
the at least one controllable peripheral device corresponding to
the received control command.
12. The terminal apparatus of claim 10, wherein, if communication
with the display apparatus is not possible, the controller
transmits the status information about the at least one peripheral
device to be monitored to the display apparatus through a relay
server based on the status information stored in the storage unit
and, in response to a monitoring performance command being received
from the display apparatus through the relay server, transmits the
monitoring performance command to one of the at least one
peripheral device corresponding to the monitoring performance
command, and transmits image information captured by the one of the
at least one peripheral device to the display apparatus through the
relay server according to the monitoring performance command.
13. A method of controlling a display apparatus, the method
comprising: periodically communicating with a terminal apparatus or
a relay server to receive collected status information about at
least one peripheral device from the terminal apparatus; generating
at least one of a control UI for controlling the at least one
peripheral device and a security UI for monitoring a peripheral
space through the at least one peripheral device based on the
status information about the at least one peripheral device and
based on whether a communication with the terminal apparatus is
possible; and displaying the at least one of the control UI and the
security UI.
14. The method of claim 13, wherein the generating at least one of
the control UI and the security UI comprises: if communication with
the terminal apparatus is possible, generating control list
information about at least one controllable peripheral device, from
among the at least one peripheral device, based on the status
information; and in response to one of the at least one
controllable peripheral device being selected based on the control
list information, generating the control UI comprising a control
menu corresponding to the selected controllable peripheral
device.
15. The method of claim 13, wherein the generating at least one of
the control UI and the security UI comprises: if communication with
the terminal apparatus is not possible, generating security list
information comprising identification information indicating the at
least one peripheral device to be monitored and an area information
depicting a monitoring area based on the status information and
setting information corresponding to each of the at least one
peripheral device.
16. The method of claim 15, wherein the generating at least one of
the control UI and the security UI further comprises: generating a
user certification UI for certifying a user in response to one of
the at least one peripheral device being selected based on the
security list information; requesting, in response to an encryption
key input on the user certification UI corresponding to preset
encryption information, the terminal apparatus, using the relay
server, to monitor the selected peripheral device; and generating,
in response to image information being captured by the selected
peripheral device and being received according to the monitoring
request, the security UI for displaying the received image
information.
17. The method of claim 16, wherein the security UI further
comprises a monitoring control menu for controlling a monitoring
operation of the selected peripheral device.
18. The method of claim 17, wherein the security UI further
comprises an editing menu for editing an image captured by the
selected peripheral device.
19. The method of claim 14, wherein the control list information is
generated based on an execution mode of the display apparatus,
wherein the execution mode is at least one of a first execution
mode activated on a quick menu, a second execution mode activated
on a lock screen, and a third execution mode in which a home
network service-related application is activated.
20. The method of claim 19, wherein the generating of the control
list information comprises: generating a first control list
information comprising the at least one controllable peripheral
device in the first execution mode; and generating a second control
list information comprising one activated controllable peripheral
device from among the at least one controllable peripheral device
in the second execution mode.
21. The method of claim 19, wherein the generating of the control
UI comprises: if in the third execution mode, generating the
control UI comprising status-related detailed information and a
control menu of the at least one controllable peripheral device
based on the received status information regardless of whether
communication with the terminal apparatus is possible.
22. A method of controlling a terminal apparatus, the method
comprising: periodically communicating with at least one peripheral
device to receive status information about the at least one
peripheral device; storing the received status information about
the at least one peripheral device; receiving a status information
request message requesting status information of the at least one
peripheral device from a display apparatus; and transmitting at
least one of status information about at least one controllable
peripheral device, from among the at least one peripheral device,
and the status information about the at least one peripheral device
to be monitored, to the display apparatus based on the stored
status information and based on whether the communication with the
display apparatus is possible.
23. The method of claim 22, wherein the transmitting of the status
information comprises: if communication with the display apparatus
is possible, transmitting the status information about the at least
one controllable peripheral device to the display apparatus based
on the stored status information; and in response to a control
command being received from the display apparatus, transmitting the
control command to one of the at least one controllable peripheral
device corresponding to the received control command.
24. The method of claim 22, wherein the transmitting of the status
information comprises: if communication with the display apparatus
is not possible, transmitting the status information about the at
least one peripheral device to be monitored, to the display
apparatus through a relay server based on the stored status
information; in response to a monitoring performance command being
received from the display apparatus through the relay server,
transmitting the monitoring performance command to one of the at
least one peripheral device corresponding to the monitoring
performance command; and transmitting image information captured by
the one of the at least one peripheral device to the display
apparatus through the relay server according to the monitoring
performance command.
25. A method of controlling at least one peripheral device, the
method comprising: transmitting, to the at least one peripheral
device, a request for status information of the at least one
peripheral device; receiving, from the at least one peripheral
device, the requested status information; generating at least one
of a control user interface (UI) for controlling the at least one
peripheral device and a security UI for monitoring a peripheral
space using the at least one peripheral device based on the
received status information and a communication status information;
and transmitting, to the at least one peripheral device, a user
command to control the at least one peripheral device, input by a
user in the at least one of the control UI and the security UI.
Description
CROSS-REFERENCE TO RELATED APPLICATION(S)
[0001] This application claims priority from Korean Patent
Application No. 10-2013-0169113, filed on Dec. 31, 2013, in the
Korean Intellectual Property Office, the disclosure of which is
incorporated herein by reference in its entirety.
BACKGROUND
[0002] 1. Field
[0003] Apparatus and methods consistent with exemplary embodiments
relate to providing a display apparatus, a terminal apparatus, and
controlling methods thereof, and more particularly, to providing a
display apparatus that provides a home network service, a terminal
apparatus, and controlling methods thereof.
[0004] 2. Description of the Related Art
[0005] The recent development of communication technologies, has
led to an emergence of various technologies directed towards a home
network system establishing, integrating and/or managing a network
between home devices such as a smart TV, a smart air conditioner,
etc. installed in a home. According to the various technologies, a
user may integrate and manage the home devices by using a portable
terminal apparatus such as a smartphone.
[0006] According to most technologies that have been commonly used,
researched and/or developed, an operation status of a home device
is monitored on a portable terminal apparatus through a data
communication performed using relay terminal apparatuses that
integrate and manage the portable terminal apparatus and the home
device. The home device is controlled according to a user command
input through the portable terminal apparatus.
[0007] Although home devices, such as a smart TV, a smart air
conditioner, a robot cleaner, or the like, have various functions,
these various functions of the home device have not been used via
the above described and currently prevalent home network
system.
[0008] Therefore, there is a need for methods of using various
functions of a home device through a home network system.
SUMMARY
[0009] Exemplary embodiments address at least the above problems
and/or disadvantages and other disadvantages not described above.
Also, the exemplary embodiments are not required to overcome the
disadvantages described above, and an exemplary embodiment may not
overcome any of the problems described above.
[0010] An aspect of the present disclosure provides using various
functions of a home device through a home network system.
[0011] Another aspect of the present disclosure provides a home
network service appropriate for a user according to a position of
the user.
[0012] According to an aspect of an exemplary embodiment, a display
apparatus includes a communicator configured to communicate with at
least one of a terminal apparatus and a relay server to receive
from the terminal apparatus collected status information
corresponding to at least one peripheral device, a display unit
configured to display a user interface (UI) screen, and a
controller configured to generate at least one of a control UI for
controlling the at least one peripheral device and a security UI
for monitoring a peripheral space using the at least one peripheral
device based on the status information corresponding to the at
least one peripheral device and based on whether communication with
the terminal apparatus is possible, and display the at least one of
the control UI and the security UI on the display unit.
[0013] According to another exemplary embodiment, if communication
with the terminal apparatus is possible, the controller generates a
control list information about at least one controllable peripheral
device, from among the at least one peripheral device, based on the
received status information, and generates the control UI
comprising a control menu corresponding to one of the at least one
controllable peripheral device selected based on the control list
information.
[0014] According to another exemplary embodiment, if communication
with the terminal apparatus is not possible, the controller
generates a security list information comprising identification
information indicating the at least one peripheral device to be
monitored and an area information depicting a monitoring area based
on the received status information and setting information
corresponding to each of the at least one peripheral device.
[0015] According to another exemplary embodiment, the controller
further includes generate a user certification UI for certifying a
user in response to one of the at least one peripheral device being
selected based on the security list information, request the
terminal apparatus, using the relay server, to monitor the selected
one of the at least one peripheral device in response to an
encryption key input on the user certification UI corresponding to
preset encryption information, and generate a security UI for
displaying image information that is captured by the selected
peripheral device and received from the relay server, according to
the monitoring request.
[0016] According to another exemplary embodiment, the security UI
comprises a monitoring control menu for controlling a monitoring
operation of the selected peripheral device.
[0017] According to another exemplary embodiment, the security UI
further comprises an editing menu for editing an image captured by
the selected peripheral device.
[0018] According to another exemplary embodiment, the controller
generates the control list information based on an execution mode
of the display apparatus, and the execution mode is at least one of
a first execution mode activated on a quick menu, a second
execution mode activated on a lock screen, and a third execution
mode in which a home network service-related application is
activated.
[0019] According to another exemplary embodiment, the controller
generates a first control list information comprising the at least
one controllable peripheral devices in the first execution mode and
generates a second control list information comprising one
activated controllable peripheral device from among the at least
one controllable peripheral device in the second execution
mode.
[0020] According to another exemplary embodiment, if in the third
execution mode, the controller generates a control UI comprising
status-related detailed information and a control menu of the at
least one controllable peripheral device based on the received
status information regardless of whether communication with the
terminal apparatus is possible.
[0021] According to another exemplary embodiment, According to
another aspect of an exemplary embodiment, a terminal apparatus
includes a communicator configured to periodically communicate with
at least one peripheral device, a storage unit configured to store
status information about the at least one peripheral device, and a
controller configured to, in response to a status information
request message requesting status information of the at least one
peripheral device being received from a display apparatus, control
the communicator to transmit, to the display apparatus, at least
one of status information about at least one controllable
peripheral device, from among the at least one peripheral device,
and the status information about the at least one peripheral device
to be monitored based on the status information stored in the
storage unit and based on whether a communication with the display
apparatus is possible.
[0022] According to another exemplary embodiment, if communication
with the display apparatus is possible, the controller transmits
the status information about the at least one controllable
peripheral device based on the status information stored in the
storage unit and, in response to a control command being received
from the display apparatus, transmits the control command to one of
the at least one controllable peripheral device corresponding to
the received control command.
[0023] According to another exemplary embodiment, if communication
with the display apparatus is not possible, the controller
transmits the status information about the at least one peripheral
device to be monitored to the display apparatus through a relay
server based on the status information stored in the storage unit
and, in response to a monitoring performance command being received
from the display apparatus through the relay server, transmits the
monitoring performance command to one of the at least one
peripheral device corresponding to the monitoring performance
command, and transmits image information captured by the one of the
at least one peripheral device to the display apparatus through the
relay server according to the monitoring performance command.
[0024] According to another exemplary embodiment, According to
another aspect of an exemplary embodiment, a method of controlling
a display apparatus includes periodically communicating with a
terminal apparatus or a relay server to receive collected status
information about at least one peripheral device from the terminal
apparatus, generating at least one of a control UI for controlling
the at least one peripheral device and a security UI for monitoring
a peripheral space through the at least one peripheral device based
on the status information about the at least one peripheral device
and based on whether a communication with the terminal apparatus is
possible, and displaying the at least one of the control UI and the
security UI.
[0025] According to another exemplary embodiment, the generating at
least one of the control UI and the security UI includes, if
communication with the terminal apparatus is possible, generating
control list information about at least one controllable peripheral
device, from among the at least one peripheral device, based on the
status information, and in response to one of the at least one
controllable peripheral device being selected based on the control
list information, generating the control UI comprising a control
menu corresponding to the selected controllable peripheral
device.
[0026] According to another exemplary embodiment, the generating at
least one of the control UI and the security UI includes, if
communication with the terminal apparatus is not possible,
generating security list information comprising identification
information indicating the at least one peripheral device to be
monitored and an area information depicting a monitoring area based
on the status information and setting information corresponding to
each of the at least one peripheral device.
[0027] According to another exemplary embodiment, the generating at
least one of the control UI and the security UI further includes
generating a user certification UI for certifying a user in
response to one of the at least one peripheral device being
selected based on the security list information, requesting, in
response to an encryption key input on the user certification UI
corresponding to preset encryption information, the terminal
apparatus, using the relay server, to monitor the selected
peripheral device, and generating, in response to image information
being captured by the selected peripheral device and being received
according to the monitoring request, the security UI for displaying
the received image information.
[0028] According to another exemplary embodiment, the security UI
further comprises a monitoring control menu for controlling a
monitoring operation of the selected peripheral device.
[0029] According to another exemplary embodiment, the security UI
further comprises an editing menu for editing an image captured by
the selected peripheral device.
[0030] According to another exemplary embodiment, the control list
information is generated based on an execution mode of the display
apparatus, and the execution mode is at least one of a first
execution mode activated on a quick menu, a second execution mode
activated on a lock screen, and a third execution mode in which a
home network service-related application is activated.
[0031] According to another exemplary embodiment, the generating of
the control list information includes generating a first control
list information comprising the at least one controllable
peripheral device in the first execution mode, and generating a
second control list information comprising one activated
controllable peripheral device from among the at least one
controllable peripheral device in the second execution mode.
[0032] According to another exemplary embodiment, the generating of
the control UI includes if in the third execution mode, generating
the control UI comprising status-related detailed information and a
control menu of the at least one controllable peripheral device
based on the received status information regardless of whether
communication with the terminal apparatus is possible.
[0033] According to another aspect of an exemplary embodiment, a
method of controlling a terminal apparatus includes periodically
communicating with at least one peripheral device to receive status
information about the at least one peripheral device, storing the
received status information about the at least one peripheral
device, receiving a status information request message requesting
status information of the at least one peripheral device from a
display apparatus, and transmitting at least one of status
information about at least one controllable peripheral device, from
among the at least one peripheral device, and the status
information about the at least one peripheral device to be
monitored, to the display apparatus based on the stored status
information and based on whether the communication with the display
apparatus is possible.
[0034] According to another exemplary embodiment, the transmitting
of the status information includes if communication with the
display apparatus is possible, transmitting the status information
about the at least one controllable peripheral device to the
display apparatus based on the stored status information, and in
response to a control command being received from the display
apparatus, transmitting the control command to one of the at least
one controllable peripheral device corresponding to the received
control command.
[0035] According to another exemplary embodiment, the transmitting
of the status information includes if communication with the
display apparatus is not possible, transmitting the status
information about the at least one peripheral device to be
monitored, to the display apparatus through a relay server based on
the stored status information, in response to a monitoring
performance command being received from the display apparatus
through the relay server, transmitting the monitoring performance
command to one of the at least one peripheral device corresponding
to the monitoring performance command, and transmitting image
information captured by the one of the at least one peripheral
device to the display apparatus through the relay server according
to the monitoring performance command.
[0036] According to another aspect of an exemplary embodiment, a
method of controlling at least one peripheral device includes
transmitting, to the at least one peripheral device, a request for
status information of the at least one peripheral device,
receiving, from the at least one peripheral device, the requested
status information, generating at least one of a control user
interface (UI) for controlling the at least one peripheral device
and a security UI for monitoring a peripheral space using the at
least one peripheral device based on the received status
information and a communication status information, and
transmitting, to the at least one peripheral device, a user command
to control the at least one peripheral device, input by a user in
the at least one of the control UI and the security UI.
BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and/or other aspects will be more apparent and
more readily appreciated by describing certain exemplary
embodiments with reference to the accompanying drawings, in
which:
[0038] FIG. 1 is a block diagram of a home network system according
to an exemplary embodiment;
[0039] FIG. 2 is a flowchart of a method for controlling a
peripheral device in a home network system, according to an
exemplary embodiment;
[0040] FIG. 3 is a flowchart of a method for providing a security
service using a home device in a home network system, according to
an exemplary embodiment;
[0041] FIG. 4 is a block diagram of a display apparatus according
to an exemplary embodiment;
[0042] FIG. 5 is views illustrating a method for displaying a
control user interface (UI), for controlling a peripheral device,
on a display apparatus, according to an exemplary embodiment;
[0043] FIG. 6 is views illustrating a method for displaying
monitoring list information, about a peripheral device that is
monitored, on a display apparatus, according to an exemplary
embodiment;
[0044] FIG. 7 is views illustrating a method for displaying a
security UI, for monitoring a peripheral space through a peripheral
device, on a display apparatus, according to an exemplary
embodiment;
[0045] FIG. 8 is views illustrating a method for displaying a
control UI, for controlling a peripheral device, on a display
apparatus, activated on a lock screen, according to an exemplary
embodiment;
[0046] FIG. 9 is views illustrating a method for displaying a
security UI, for monitoring a peripheral space through a peripheral
device, on a display apparatus, activated on a lock screen,
according to an exemplary embodiment;
[0047] FIG. 10 is views illustrating a method for displaying a
control UI on a display apparatus on which a home network
service-related application is activated, according to an exemplary
embodiment;
[0048] FIG. 11 is views illustrating a method for monitoring a
peripheral status through a peripheral device on a display
apparatus, activated on a lock screen, according to another
embodiment;
[0049] FIG. 12 is views illustrating a method for displaying a
control UI on a display apparatus on which a home network
service-related application is activated, according to an exemplary
embodiment;
[0050] FIG. 13 is a block diagram of a terminal apparatus according
to an exemplary embodiment;
[0051] FIG. 14 is a flowchart of a method for controlling a display
apparatus, according to an exemplary embodiment;
[0052] FIG. 15 is a flowchart of a method for generating a control
UI for controlling a peripheral device on a display apparatus,
according to an exemplary embodiment;
[0053] FIG. 16 is a flowchart of a method for generating a security
UI for monitoring a peripheral device on a display apparatus,
according to an exemplary embodiment; and
[0054] FIG. 17 is a flowchart of a method for controlling a
terminal apparatus, according to an exemplary embodiment.
DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS
[0055] Exemplary embodiments are described in greater detail with
reference to the accompanying drawings.
[0056] In the following description, the same drawing reference
numerals are used for the same elements even in different drawings.
The matters defined in the description, such as detailed
construction and elements, are provided to assist in a
comprehensive understanding of the exemplary embodiments. Thus, it
is apparent that the exemplary embodiments can be carried out
without those specifically defined matters. Also, well-known
functions or constructions are not described in detail since they
would obscure the exemplary embodiments with unnecessary
detail.
[0057] FIG. 1 is a block diagram of a home network system according
to an exemplary embodiment.
[0058] Referring to FIG. 1, the home network system includes four
peripheral devices 100-1 through 100-4, a terminal apparatus 200, a
display apparatus 300, and a relay server 400.
[0059] The four peripheral devices 100-1 through 100-4 may be home
devices installed in a home. According to an exemplary embodiment,
the first peripheral device 100-1 may be a smart electronic device
such as a robot cleaner, a washing machine, a smart TV, or a
refrigerator, the second peripheral device 100-2 may be a security
device such as an internet protocol (IP) camera, a door lock, a
closed-circuit television (CCTV) camera, a fire detector, or the
like, the third peripheral device 100-3 may be one of the various
types of lighting equipment, and the fourth peripheral device 100-4
may be an energy device such as a boiler, an air conditioner, or
the like. The four peripheral devices 100-1 through 100-4 may
perform wired or wireless communications through the terminal
apparatus 200 and may be controlled according to a control command
from the display apparatus 300.
[0060] The terminal apparatus 200 may be a home gateway and
communicate with the four peripheral devices 100-1 through 100-4
that are registered and the display apparatus 300 via wire or
wirelessly. In detail, the terminal apparatus 200 may periodically
perform data communication with the four peripheral devices 100-1
through 100-4 to store status information about the four peripheral
devices 100-1 through 100-4. The status information may include
activation information of the four peripheral devices 100-1 through
100-4, communication status information between the four peripheral
devices 100-1 through 100-4 and the terminal apparatus 200,
etc.
[0061] If a status information request message is received from the
display apparatus 300, the terminal apparatus 200 transmits status
information about a controllable peripheral device or status
information about a peripheral device that may be monitored, to the
display apparatus 300 based on the stored status information of the
four peripheral devices 100-1 through 100-4 based on a
communication status with the display apparatus 300. In detail, the
terminal apparatus 200 may set one of the registered four
peripheral devices 100-1 through 100-4 selected by a user as a
peripheral device that may be monitored and may store the
peripheral device. Thus, the terminal apparatus 200 may transmit
status information about a controllable peripheral device or status
information about a peripheral device that may be monitored, to the
display apparatus 300 based on stored status information based on a
communication status with the display apparatus 300. According to
an exemplary embodiment, if a communication with the display
apparatus 300 is possible, the terminal apparatus 200 may receive
the status information request message from the display apparatus
300. In this case, the terminal apparatus 200 transmits status
information about a controllable peripheral device from among the
four peripheral devices 100-1 through 100-4 to the display
apparatus 300 based on the stored status information. If the
communication with the display apparatus 300 is impossible, the
terminal apparatus 200 may receive the status information request
message from the display apparatus 300 through the relay sever 400.
In this case, the terminal apparatus 200 transmits status
information about one of the four peripheral devices 100-1 through
100-4 that may be monitored, to the display apparatus 300 based on
the stored status information and setting information.
[0062] If communication between the terminal apparatus 200 and the
display apparatus is possible, the terminal apparatus 200 and the
display apparatus 300 exist on the same network. Therefore, the
communication between the terminal apparatus 200 and the display
apparatus 300 may be directly performed through a short-range
wireless communication. However, if communication between the
terminal apparatus 200 and the display apparatus 300 is impossible,
the terminal apparatus 200 and the display apparatus 300 exist on
different networks. Therefore, the communication between the
terminal apparatus 200 and the display apparatus 300 is possible
through the relay server 400.
[0063] The display apparatus 300 may generate and display a control
user interface (UI) for controlling at least one of the four
peripheral devices 100-1 through 100-4 based on the status
information received from the terminal apparatus 200 or may
generate and display a security UI for monitoring at least one of
the four peripheral devices 100-1 through 100-4.
[0064] In detail, if status information about a controllable
peripheral device is received from the terminal apparatus 200, the
display apparatus 300 may generate a control UI for controlling at
least one of the four peripheral devices 100-1 through 100-4 based
on the received status information. If status information about a
peripheral device that may be monitored is received from the
terminal apparatus 200, the display apparatus 300 may generate a
security UI for monitoring a peripheral space through at least one
of the four peripheral devices 100-1 through 100-4 based on the
received status information.
[0065] However, if the status information request message is
received from the display apparatus 300 regardless of a
communication status with the display apparatus 300, the terminal
apparatus 200 may transmit the stored status information about the
four peripheral devices 100-1 through 100-4.
[0066] In this case, the display apparatus 300 may generate a
control UI or a security UI according to the communication status
with the terminal apparatus 200. In detail, if the display
apparatus 300 directly communicates with the terminal apparatus 200
by wireless, the display apparatus 300 may generate a control UI
for controlling at least one of the four peripheral devices 100-1
through 100-4 based on the status information received from the
terminal apparatus 200. If the display apparatus 300 indirectly
communicates with the terminal apparatus 200 through the relay
server 400, the display apparatus 300 may generate a security UI
for monitoring at least one of the four peripheral devices 100-1
through 100-4 based on the status information received from the
terminal apparatus 200.
[0067] As described above, the display apparatus 300 of the home
network system, according to the present exemplary embodiment, may
generate a control UI for controlling at least one peripheral
device installed in a home according to a communication status
between the terminal apparatus 200 and the display apparatus 300 or
may generate a security UI for monitoring a peripheral space
through at least one peripheral device. Therefore, if a user of the
display apparatus 300 stays in a home, the user may control an
operation of a peripheral device through the display apparatus 300.
If the user stays outside the home, the user may monitor a status
of the home/display device through the display apparatus 300 in
real time.
[0068] The above-described operation of the home network system
will now be described in more detail with reference to FIGS. 2 and
3.
[0069] FIG. 2 is a flowchart of a method for controlling a
peripheral device in a home network system, according to an
exemplary embodiment. FIG. 3 is a flowchart of a method for
providing a security service by using a home device in the home
network system, according to an exemplary embodiment.
[0070] Referring to FIG. 2, in operation S10, the terminal
apparatus 200 periodically requests status information of at least
one registered peripheral device 100. If the status information is
received from the at least one peripheral device 100 according to
the request, the terminal apparatus 200 parses and stores the
status information in operation S11. Here, the status information
may include at least one of activation information including at
least one of on/off information, setting information, sensing
information of the peripheral device 100 and communication status
information between the peripheral device 100 and the terminal
apparatus 200, according to an exemplary embodiment.
[0071] In operation S12, the display apparatus 300 generates a
status information request message for requesting the status
information about the at least one peripheral device 100 and
transmits the status information request message to the terminal
apparatus 200 according to a user command. If the status
information request message is received, the terminal apparatus 200
determines whether a direct communication is possible with the
display apparatus 200 that transmitted the status information
request message, in operation S13. In detail, the terminal
apparatus 200 may determine whether the display apparatus 300
exists on the same network using the received status information
request message. If it is determined that the display apparatus 300
exists on the same network, the terminal apparatus 200 transmits
status information about a controllable peripheral device, from
among the stored status information, about the at least one
peripheral device 100, to the display apparatus 300 in operation
S14. If it is determined that the display apparatus does not exist
on the same network, the terminal apparatus 200 performs a
monitoring-related operation in operation S15. In other words, if
it is determined that the display apparatus 300 does not exist on
the same network, the terminal apparatus 200 transmits status
information about a peripheral device that may be monitored, from
among the stored status information, about the at least one
peripheral device 100, to the display apparatus 300 through the
relay server 400. The monitoring-related operation will be
described later with reference to FIG. 3.
[0072] If the status information about the controllable peripheral
device 100 is received from the terminal apparatus 200, the display
apparatus 300 generates control list information and a control UI
based on the received status information in operation S16. Here,
the control list information may be a list information about the
controllable at least one peripheral device 100, and the control UI
may be a UI for controlling the peripheral device 100 selected from
a plurality of peripheral devices 100 by the user. If the
controllable peripheral devices 100 are the four peripheral devices
100-1 through 100-4 as in the above-described exemplary embodiment
with reference to FIG. 1, the display apparatus 300 may generate
and display control list information including device information
about the four peripheral devices 100-1 through 100-4. If device
information about the first peripheral device 100-1 is selected by
the user in response to the control list information being
displayed, the display apparatus 300 may generate and display a
control UI including a control menu for controlling the selected
first peripheral device 100-1, according to an exemplary
embodiment.
[0073] If a user command is input when the control UI is displayed,
the display apparatus 300 transmits a control command corresponding
to the input user command to the terminal apparatus 200, and the
terminal apparatus 200 transmits the control command to the
peripheral device 100 selected by the user in operation S17. In
operation S18, the peripheral device 100 that receives the control
command from the terminal apparatus 200 performs a control
operation based on the control command.
[0074] If it is determined in operation S13 that the display
apparatus 300 and the terminal apparatus 200 do not exist on the
same network, the terminal apparatus 200 transmits status
information about the peripheral device 100 that may be monitored,
from among stored status information, about the at least one
peripheral device 100, to the display apparatus 300 through the
relay server 400 in operation S20 as shown in FIG. 3. If the status
information is received, the display apparatus 300 generates and
displays security list information and a user certification UI
based on the received status information in operation S21. Here,
the security list information may be list information about the
peripheral device 100 that may monitor a peripheral space, from
among a plurality of peripheral devices 100, according to an
exemplary embodiment. For example, the first and second peripheral
devices 100-1 and 100-2 of the four peripheral devices 100-1
through 100-4 may be peripheral devices that may be monitored. In
this case, the display apparatus 300 generates and displays
security list information including device information about the
first and second peripheral devices 100-1 and 100-2 that may be
monitored. If the first peripheral device 100-1 is selected by the
user when the security list information is displayed, the display
apparatus 300 generates and displays the user certification UI to
determine whether the corresponding user is a registered user. If
an encryption key is input by the user, the display apparatus 300
determines whether the encryption key input from the user
corresponds to a preset encryption information to determine whether
the corresponding user is a registered user in operation S22.
[0075] If it is determined that the corresponding user is a
registered user through the user certification process, the display
apparatus 300 requests the terminal apparatus 200 through the relay
server 400 to monitor the peripheral device 100, previously
selected by the user, in operation S23. If a monitoring request
message is received, the terminal apparatus 200 transmits a
monitoring performance command to the peripheral device 100
requested by the user based on the received monitoring request
message in operation S24, and the peripheral device 100 that
receives the monitoring performance command captures an image of a
peripheral space based on the received monitoring performance
command in operation S25. In operation S26, the peripheral device
100 transmits captured image information to the terminal apparatus
200, and in operation S27 the terminal apparatus 200 transmits the
captured image information received from the peripheral device 100
to the display apparatus 300 through the relay server 400. If the
captured image information is received, the display apparatus 300
generates a security UI and displays the captured image information
through the generated security UI in operation S28. Here, the
security UI may include an image captured by the peripheral device
100 and a monitoring control menu for controlling a monitoring
operation of the corresponding peripheral device 100. The
monitoring control menu may further include a first control menu
for controlling an operation of the peripheral device 100 that
captures the image and a second menu for editing and storing the
image captured by the peripheral device 100. If a user command is
input with respect to the first menu for controlling the operation
of the peripheral device 100, the display apparatus 300 may
transmit a control command corresponding to the input user command
to the terminal apparatus 200 through the relay server 400, and the
terminal apparatus 200 may transmit the corresponding control
command to the peripheral device 100 that performs the
monitoring-related operation. Therefore, the peripheral device 100
stops an image capturing operation or performs an operation of
moving an image capturing direction or another operation based on
the control command received from the terminal apparatus 200 in
operation S29.
[0076] If the status information request message is received, the
terminal apparatus 200 may not perform operation S13 as described
above and may transmit stored status information about at least one
peripheral device 100 to the display apparatus 300 directly or
through the relay server 400. In this case, the display apparatus
300 may generate a control UI for controlling a controllable
peripheral device 100 or a security UI for controlling the
peripheral device 100 that may monitor a peripheral space, based on
the status information of the at least one peripheral device 100,
wherein the status information is transmitted from the terminal
apparatus 200. In detail, if a direct communication with the
terminal apparatus 200 is possible, the display apparatus 300 may
generate control list information about the controllable peripheral
device 100 based on the received status information. If the direct
communication with the terminal apparatus 200 is impossible, the
display apparatus 300 may generate security list information about
the peripheral device 100 that may monitor the peripheral space,
based on the received status information. The terminal apparatus
200 and the display apparatus 300 may generate a control UI for
controlling the peripheral device 100 or a security UI for
displaying image information captured by the peripheral device 100
according to the operations described with reference to FIGS. 2 and
3.
[0077] Structures of the terminal apparatus 200 and the display
apparatus 300 will now be described in detail.
[0078] FIG. 4 is a block diagram of the display apparatus 300,
according to an exemplary embodiment.
[0079] Referring to FIG. 4, the display apparatus 300 may be a
portable terminal apparatus that performs wireless communications
like a smartphone, a tablet PC, or the like and may include a
communicator 310, a controller 320, a display unit 330, a storage
unit 340, and an input unit 350.
[0080] The communicator 310 wirelessly communicates with the
terminal apparatus 200, or the relay server 400 which in-turn
communicates with the terminal apparatus 200, that communicates
with the at least one peripheral device 100 that is a home device
installed in a home, to receive the collected status information
about the at least one peripheral device 100 from the terminal
apparatus 200. Here, the status information may include at least
one of activation information including at least one of on/off
information, setting information, sensing information of the
peripheral device 100, and communication status information between
the peripheral device 100 and the terminal apparatus 200, according
to an exemplary embodiment.
[0081] The communicator 310 may be a wireless communication module
and include at least one of a short-range communication module (not
shown) and a wireless local area network (WLAN) module (not shown).
Here, the short-range communication module is a communication
module that performs a wireless short-range communication between
the display apparatus 300 and the terminal apparatus 200 according
to a control command of the controller 320. The short-range
communication module may include at least one of a Wi-Fi module, a
Bluetooth module, an infrared data association (IrDA) module, a
near field communication (NFC) module, and a Zigbee module,
according to an exemplary embodiment.
[0082] The WLAN module accesses the relay server 400 such as an
access point (AP) (not shown) existing within a preset range to be
connected to the Internet according to the control command of the
controller 320. The WLAN module supports WLAN standard IEEE802.11x
of Institute of Electrical and Electronic Engineers (IEEE).
[0083] The communicator 310 may further include at least one of a
cellular communication module (not shown), a global positioning
system (GPS) communication module (not shown), and a broadcast
communication module (not shown). These communication modules are
well-known technologies, and thus their detailed descriptions are
omitted herein.
[0084] The controller 320 controls overall operations of the
elements of the display apparatus 300. In particular, the
controller 320 may generate a control UI for controlling at the
least one peripheral device 100 based on status information of the
at least one peripheral device 100 or may generate a security UI
for monitoring a peripheral space through the at least one
peripheral device 100 according to whether a communication with the
terminal apparatus 200 is possible.
[0085] The display unit 330 displays the generated control UI or
security UI on a screen according to the control command of the
controller 320. The display unit 330 may be realized as a touch
screen that senses a touch, according to an exemplary
embodiment.
[0086] In detail, if the collected status information regarding the
at least one peripheral device 100 is received from the terminal
apparatus 200 through the communicator 310, the controller 320
determines a communication status with the terminal apparatus 200.
According to an exemplary embodiment, if a communication with the
terminal apparatus 200 is possible, the controller 320 generates
control list information about a controllable peripheral device
based on the received status information about the at least one
peripheral device 100. Here, if the communication with the terminal
apparatus 200 is possible, the display apparatus 300 may directly
communicate with the terminal apparatus 200 through a short-range
wireless communication. If a direct communication between the
display apparatus 300 and the terminal apparatus 200 is possible as
described above, the controller 320 generates the control list
information regarding the controllable peripheral device based on
the received status information about the at least one peripheral
device 100 and displays the control list information through the
display unit 330.
[0087] If a command to select the peripheral device 100 is input
through the input unit 350 when the control list information is
displayed on the screen through the display unit 330, the
controller 320 generates a control UI including a control menu for
controlling the peripheral device 100 corresponding to the input
command. Therefore, the display unit 330 may display the control UI
including the control menu corresponding to the peripheral device
100 selected by the user.
[0088] Here, the control list information may be list information
including device information about the controllable at least one
peripheral device 100. The input unit 350 receives a selection
command according to various types of user controls of the user and
transmits the selection command to the controller 320. The input
unit 350 may include a touch input unit, such as a touch screen or
a touch pad, and a key input unit, such as a key pad having various
functional keys, numerical keys, character keys, etc.
[0089] Therefore, if a command to select the controllable
peripheral device 100 is input through the input unit 350 when the
control list information including the device information about the
at least one peripheral device 100 is displayed, the controller 320
may generate a control UI including a control menu of the
peripheral device 100 corresponding to the input command, from
among control menus of peripheral devices 100 stored in the storage
unit 340, and display the control UI through the display unit
330.
[0090] The controller 320 that generates the control UI for
controlling the selected peripheral device 100 as described above
may differently generate the control list information according to
an execution mode of the display apparatus 300. Here, the execution
mode may include at least one of a first execution mode activated
on a quick menu, a second execution mode activated on a lock
screen, and a third execution mode in which a home network
service-related application is activated, according to an exemplary
embodiment.
[0091] In detail, in the first execution mode activated on the
quick menu, the controller 320 may generate and display first
control list information including all controllable peripheral
devices 100 based on the received status information. In the second
execution mode activated on the lock screen, the controller 320 may
generate and display second control list information including the
peripheral device 100 that is activated among the all controllable
peripheral devices 100. In the third execution mode in which the
home network service-related application is activated, the
controller 320 may generate and display a control UI including
status-related detailed information and a control menu of at least
one peripheral device 100 based on the received status information
regardless of whether a communication with the terminal apparatus
200 is possible.
[0092] As described above, the controller 320 may differently
generate control list information and a control UI according to an
execution mode of the display apparatus 300 and display the control
list information through the display unit 330. An operation of
differently generating and displaying control list information and
a control UI according to an execution mode will now be described
in detail.
[0093] If it is determined that the communication with the terminal
apparatus 200 is impossible, the controller 320 may generate
security list information about the peripheral device 100 that may
monitor a peripheral space, based on the received status
information about the at least one peripheral device 100 and
setting information of each peripheral device 100 registered in the
storage unit 340 and display the security list information through
the display unit 330. Here, if the communication with the terminal
apparatus 200 is impossible, the display apparatus 300 and the
terminal apparatus 200 may communicate with each other through the
relay server 400. The security list information may include
identification information indicating the peripheral device 100
that may monitor the peripheral space and area information showing
an area that may be monitored through the corresponding peripheral
device 100.
[0094] If the display apparatus 300 and the terminal apparatus 200
indirectly communicate with each other through the relay server 400
as described above, the controller 320 may generate security list
information including identification information and area
information of the peripheral device 100 that may be monitored,
based on received status information and registered setting
information of at least one peripheral device and display the
security list information through the display unit 330.
[0095] If a command to select the peripheral device 100, that may
monitor a peripheral space, is input through the input unit 350
when the security list information is displayed, the controller 320
may generate a user certification UI for certifying the user and
display the user certification UI through the display unit 330. If
an encryption key is input through the input unit 350 when the user
certification UI is displayed, the controller 320 determines
whether the input encryption key corresponds to a preset encryption
information. If it is determined that the encryption key
corresponds to the preset encryption information, the controller
320 requests the terminal apparatus 200, via the relay server 400,
to monitor the peripheral device 100 corresponding to the input
selection command. Therefore, the terminal apparatus 200 transmits
a monitoring performance command to the peripheral device 100 that
is requested to be monitored and, if image information of a
peripheral space captured by the corresponding peripheral device
100 is received from the peripheral device 100, transmits the
received image information to the display apparatus 300 through the
relay server 400.
[0096] If the image information is received, the controller 320
generates a security UI for displaying an image captured by the
peripheral device 100, that is requested to be monitored. The
controller 320 controls the display unit 330 to display the image
corresponding to the received image information through the
generated security UI. Therefore, the display unit 330 may display
the image captured by the peripheral device 100, that is requested
to be monitored, in an area of the security UI. Therefore, the user
may monitor a peripheral space, which is related to the peripheral
device 100 selected by the user, in real time through the image
displayed in the area of the security UI displayed on the screen of
the display apparatus 300.
[0097] According to an exemplary embodiment, the controller 320 may
generate a security UI including a monitoring control menu for
controlling a monitoring operation of the peripheral device 100
that is requested to be monitored. Also, the controller 320 may
generate a security UI further including an editing menu for
editing the image captured by the peripheral device 100, that is
requested to be monitored. Here, the monitoring control menu may
include at least one of a menu for switching on and/or off the
peripheral device 100 that is monitored, and a menu for controlling
a monitoring direction of the corresponding peripheral device 100,
according to an exemplary embodiment. The controlling menu is not
limited thereto and may contain several other menus. The editing
menu may include at least one of a menu for capturing or recording
the image captured by the peripheral device 100, that is requested
to be monitored, and a menu for ending a currently displayed
security UI, according to an exemplary embodiment.
[0098] Therefore, the user may control an operation of the
peripheral device 100 that is monitored or may edit the image
captured by the corresponding peripheral device 100 using the
monitoring control menu or the editing menu included in the
security UI displayed on the screen of the display apparatus
300.
[0099] An operation of controlling at least one peripheral device
100 or providing a security service through the at least one
peripheral device 100 in the display apparatus 300 will now be
described in more detail.
[0100] FIGS. 5A and 5B are views illustrating a method for
displaying a control UI for controlling a peripheral device in a
display apparatus, according to an exemplary embodiment.
[0101] The display apparatus 300 may directly perform a data
communication with the terminal apparatus 200 through a short-range
wireless communication to receive status information about at least
one peripheral device 100 from the terminal apparatus 200. If the
status information is received, the display apparatus 300 generates
and displays control list information about a controllable
peripheral device 100 according to an execution mode of the display
apparatus 300.
[0102] In detail, as shown in FIG. 5A, if the execution mode of the
display apparatus 300 is a first execution mode activated on a
quick menu, the display apparatus 300 may generate and display
first control list information 510 about all controllable
peripheral devices 100 based on status information received from
the terminal apparatus 200.
[0103] For example, the display apparatus 300 may receive status
information about a smart TV, a robot cleaner, an air conditioner,
and a smart washing machine belonging to the first peripheral
device 100-1 from the terminal apparatus 200. The display apparatus
300 may also store setting information for setting devices
belonging to the four peripheral devices 100-1 through 100-4. The
setting information may include control menus of the devices
belonging to the four peripheral devices 100-1 through 100-4.
[0104] In this case, the display apparatus 300 may generate and
display the first control list information 510 including
identification icons indicating the smart TV, the robot cleaner,
the air conditioner, and the smart washing machine belonging to the
first peripheral device 100-1 related to the received status
information, from among the devices belonging to the four
peripheral devices 100-1 through 100-4, based on the received
status information and stored setting information. If an
identification icon 511, indicating the smart TV according to an
exemplary embodiment, is selected according to a request of the
user when the first control list information 510 is displayed, the
display apparatus 300 may generate and display a control UI 520
including a control menu corresponding to the smart TV, based on a
control menu included in the stored setting information, as shown
in FIG. 5B.
[0105] As shown in FIG. 5B, the control UI 520 for controlling the
smart TV corresponding to the selected identification icon 511 may
include a control menu for controlling a channel and a volume of
the smart TV, a control menu for switching on and/or off the smart
TV, etc. If the control menu for switching on and/or off the smart
TV 521 is selected according to a user request when the control UI
520 including the control menu is displayed, the display apparatus
300 transmits a control command corresponding to the selected
control menu to the terminal apparatus 200. Therefore, the terminal
apparatus 200 may transmit the control command to the smart TV
based on the control command received from the display apparatus
300, and the smart TV may be switched on and/or off according to
the received control command.
[0106] FIGS. 6A and 6B are views illustrating a method for
displaying monitoring list information about a peripheral device
that may be monitored, in a display apparatus, according to an
exemplary embodiment.
[0107] The display apparatus 300 may indirectly perform a data
communication with the terminal apparatus 200 through the relay
server 400 to receive collected status information about at least
one peripheral device 100 from the terminal apparatus 200. If the
status information is received, the display apparatus 300 generates
and displays monitoring list information about the peripheral
device 100 that may be monitored, according to an execution mode of
the display apparatus 300.
[0108] In detail, as shown in FIG. 6A, if the execution mode of the
display apparatus 300 is a first execution mode activated on a
quick menu, the display apparatus 300 may generate and display
monitoring list information 610 about the peripheral device 100
that may be monitored, based on status information and registered
setting information received from the terminal apparatus 200.
[0109] For example, the display apparatus 300 may receive status
information about a smart TV, a robot cleaner, an air conditioner,
and a smart washing machine belonging to the first peripheral
device 100-1 through the relay server 400. The display apparatus
300 may store setting information for setting the smart TV and the
robot cleaner belonging to the first peripheral device 100-1 of the
four peripheral devices 100-1 through 100-4 and setting information
for a security device belonging to the second peripheral device
100-2 to devices that may be monitored. The setting information for
a security device may include identification information indicating
a device that may be monitored and area information showing an area
of each device that may be monitored.
[0110] In this case, the display apparatus 300 displays a
notification message 610 notifying the number of devices that may
be monitored among the smart TV, the robot cleaner, the air
conditioner, and the smart washing machine belonging to the first
peripheral device 100-1 related to the received status information,
based on the received status information and the stored setting
information. If the notification message 610 is selected according
to a user command, the display apparatus 300 generates and displays
monitoring list information 620 including the identification
information and the area information corresponding to the device
that may be monitored.
[0111] If devices that may be monitored are the smart TV and the
robot cleaner as in the above-described exemplary embodiment, the
display apparatus 300 may generate and display the monitoring list
information 620 including identification information (an
identification icon) and area information indicating the smart TV
and identification information (an identification icon) and area
information indicating the robot cleaner as shown in FIG. 6B.
[0112] If an identification icon 621 indicating the smart TV is
selected according to a request of the user when the monitoring
list information 620 is displayed, the display apparatus 300 may
perform a user certification and display an image captured by the
smart TV through a security UI as shown in FIGS. 7A and 7B.
[0113] FIGS. 7A and 7B are views illustrating a method for
displaying a security UI for monitoring a peripheral space through
a peripheral device in a display apparatus, according to an
exemplary embodiment.
[0114] If the identification icon 621 indicating the smart TV is
selected as described with reference to FIGS. 6A and 6B, the
display apparatus 300 generates and displays a user certification
UI 710 for certifying a user as shown in FIG. 7A. If an encryption
key is input by the user when the user certification UI 710 is
displayed, the display apparatus 300 determines whether the input
encryption key corresponds to a preset encryption information. If
it is determined that the input encryption key corresponds to the
preset encryption information, the display apparatus 300 transmits
a monitoring performance command for performing a
monitoring-related operation to monitor the smart TV to the
terminal apparatus 200 through the relay server 400. Therefore, the
terminal apparatus 200 transmits the monitoring performance command
to the security device to monitor the smart TV based on the
received monitoring performance command. Therefore, the security
device captures an image of a peripheral space according to the
monitoring performance command received from the terminal apparatus
200 and transmits captured image information to the terminal
apparatus 200. The terminal apparatus 200 transmits the image
information received from the security device to the display
apparatus 300 through the relay server 400. If the image
information captured by the security device is received from the
terminal apparatus 200 through the relay server 400, the display
apparatus 300 may generate a security UI and display the image
captured by the smart TV through the generated security UI as shown
in FIG. 7B.
[0115] Here, as shown in FIG. 7B, the display apparatus 300 may
display a monitoring control menu for controlling a monitoring
operation of monitoring the smart TV in an area 720 of a security
UI in which the image captured by the security device is displayed.
The display apparatus 300 may display a security UI including an
editing menu for editing the image captured by the smart TV in an
area 730.
[0116] In detail, the display apparatus 300 may display a
monitoring control menu in the area 720 of the security UI in which
the image captured by the security device is displayed. Here, the
monitoring control menu includes a direction menu 721 for changing
a capturing area captured by the security device and a control menu
723 for stopping the security UI. The display apparatus 300 may
include a power menu 731 for controlling an operation of switching
on and/or off the security device, a capture menu 733 for capturing
the image displayed in the area 720 of the security UI, and a
recording menu 735 for recording the corresponding image in the
area 720 of the security UI.
[0117] For example, if the user selects a direction menu for moving
a capturing direction into a left direction, the display apparatus
300 transmits a control command corresponding to the selected
direction menu to the terminal apparatus 200 through the relay
server 400, and the terminal apparatus 200 transmits the control
command to the security device. Therefore, the security device
moves the capturing direction into the left direction based on the
received control command to capture a peripheral space in the moved
direction and transmit captured image information to the terminal
apparatus 200. Thereafter, the terminal apparatus 200 may transmit
the image information captured by the security device to the
display apparatus 300 through the relay server 400, and the display
apparatus 300 may display the image that the security device moves
into the left direction to capture, in the area 720 of the security
UI.
[0118] If an identification icon of a robot cleaner, in which area
information "Free Zone" is set, is selected as shown in FIG. 6B,
the display apparatus 300 may display an image captured by the
robot cleaner through the above-described operation in the area 720
of the security UI. Here, if a direction menu displayed in the area
720 of the security UI is selected by the user, the display
apparatus 300 transmits a control command corresponding to the
selected direction menu to the terminal apparatus 200 through the
relay server 400, and the terminal apparatus 200 transmits the
control command to the robot cleaner. Therefore, the robot cleaner
may move into a direction corresponding to the selected direction
menu to capture an image, and the display apparatus 300 may display
the image that the robot cleaner moves into the direction
corresponding to the selected direction menu to capture, in the
area 720 of the security UI.
[0119] As described above, the display apparatus 300 may control a
movement of the peripheral device 100 that may be monitored, in a
camera capturing direction and remotely move the corresponding
peripheral device 100 to capture an image. Therefore, the user may
monitor all areas of a home through the display apparatus 300 in
real time to check whether risk elements are detected.
[0120] FIGS. 8A and 8B are views illustrating a method for
displaying a control UI for controlling a peripheral device in a
display apparatus activated on a lock screen, according to an
exemplary embodiment.
[0121] The display apparatus 300 may directly perform a data
communication with the terminal apparatus 200 through a short-range
wireless communication to receive status information about at least
one peripheral device 100 from the terminal apparatus 200. If the
status information is received, the display apparatus 300 generates
and displays control list information about a controllable
peripheral device 100 according to an execution mode of the display
apparatus 300.
[0122] In detail, if the execution mode of the display apparatus
300 is a second execution mode activated on a lock screen as shown
in FIG. 8A, the display apparatus 300 may generate and display
second control list information 810 about one of the activated
controllable peripheral devices 100 based on the status information
received from the terminal apparatus 200.
[0123] For example, the display apparatus 300 may receive status
information about a smart TV, a robot cleaner, an air conditioner,
and a smart washing machine belonging to the first peripheral
device 100-1 from the terminal apparatus 200. The display apparatus
300 may also store setting information for setting devices
belonging to the four peripheral devices 100-1 through 100-4. The
setting information may include a control menu of the devices
belonging to the four peripheral devices 100-1 through 100-4.
[0124] In this case, the display apparatus 300 classifies the smart
TV, the robot cleaner, the air conditioner, and the smart washing
machine belonging to the first peripheral device 100-1, from among
the devices belonging to the four peripheral devices 100-1 through
100-4, as controllable peripheral devices 100 based on the received
status information and the stored setting information. If the
controllable peripheral devices 100 are classified, the display
apparatus 300 may classify only devices, which operate in an
activation mode, from among the smart TV, the robot cleaner, the
air conditioner, and the smart washing machine belonging to the
first peripheral device 100-1, as the controllable peripheral
devices 100, based on activation information included in the
receives status information.
[0125] The devices that operate in the activation mode may be in a
mode in which the devices currently operate. For example, if the
smart TV and the air conditioner operate, from among the smart TV,
the robot cleaner, the air conditioner, and the smart washing
machine, the display apparatus 300 may generate and display second
control list information 810 including identification icons
indicating the smart TV and the air conditioner and activation
information included in status information of the smart TV and the
air conditioner.
[0126] Therefore, the second control list information 810 may
include activation information including the identification icon
and current channel information of the smart TV and activation
information including the identification icon of the air
conditioner and a current room temperature sensed by the air
conditioner, according to an exemplary embodiment.
[0127] If an identification icon 811 indicating the smart TV is
selected by the user when the second control list information 810
is displayed, the display apparatus 300 may generate and display a
control UI 820 including a control menu corresponding to the smart
TV, based on a control menu included in stored setting information,
as shown in FIG. 8B.
[0128] Therefore, the user may remotely control the smart TV
through the control menu included in the control UI 820.
[0129] FIGS. 9A through 9C are views illustrating a method for
displaying a security UI for monitoring a peripheral space through
a peripheral device in a display apparatus activated on a lock
screen, according to an exemplary embodiment.
[0130] As shown in FIG. 9A, the display apparatus 300 may
indirectly perform a data communication with the terminal apparatus
200, through the relay server 400, to receive collected status
information about at least one peripheral device 100 from the
terminal apparatus 200. If the status information is received, the
display apparatus 300 may generate and display monitoring list
information about the peripheral device 100 that may be monitored,
according to an execution mode of the display apparatus 300.
[0131] In detail, as shown in FIG. 9A, if the execution mode of the
display apparatus 300 is a second execution mode activated on a
lock screen, the display apparatus 300 may generate and display
monitoring list information 910 about the peripheral device 100
that may be monitored, based on the status information and the
stored setting information received from the terminal apparatus
200.
[0132] For example, the display apparatus 300 may receive status
information about a smart TV, a robot cleaner, an air conditioner,
and a smart washing machine belonging to the first peripheral
device 100-1 through the relay server 400. The display apparatus
300 may also store setting information for setting the smart TV and
the robot cleaner belonging to the first peripheral device 100-1
and store setting information for setting a security device
belonging to the second peripheral device 100-2 from among devices
belonging to the four peripheral devices 100-1 through 100-4. Also,
the setting information may include identification information
indicating a peripheral device that may be monitored and area
information showing an area that may be monitored by each
device.
[0133] In this case, the display apparatus 300 generates and
displays monitoring list information 910 about a device that may be
monitored, from among the smart TV, the robot cleaner, the air
conditioner, and the smart washing machine belonging to the first
peripheral device 100-1, based on the received status information
and the stored setting information. Here, the monitoring list
information 910 may include identification icons identifying the
smart TV and the robot cleaner and area information showing an area
that may be monitored through the selection of the smart TV and the
robot cleaner.
[0134] If an identification icon 911 indicating the smart TV is
selected by the user when the monitoring list information 910 is
displayed, the display apparatus 300 generates and displays a user
certification UI 920 for performing a user certification as shown
in FIG. 9B.
[0135] If an encryption key is input from the user when the user
certification UI 920 is displayed, the display apparatus 300
determines whether the input encryption key corresponds to a preset
encryption information. If the input encryption key corresponds to
the preset encryption information, the display apparatus 300
determines that the user certification is completed, and generates
and displays a security UI 930 for displaying an image captured by
the security device covering the living room/smart TV selected by
the user as shown in FIG. 9C.
[0136] In detail, if the user certification is completed, the
display apparatus 300 transmits a monitoring performance command to
the terminal apparatus 200 through the relay server 400. Here, the
monitoring performance command is to perform a monitoring-related
operation through the security device covering the living
room/smart TV. Therefore, the terminal apparatus 200 transmits the
monitoring performance command to the security device covering the
living room/smart TV based on the received monitoring performance
command. Therefore, the security device covering the living
room/smart TV captures an image of a peripheral space and transmits
the captured image information to the terminal apparatus 200
according to the monitoring performance command received from the
terminal apparatus 200. Thereafter, the terminal apparatus 200
transmits the image information transmitted from the security
device covering the living room/smart TV to the display apparatus
300 through the relay server 400. If the image information captured
by the security device covering the living room/smart TV is
received from the terminal apparatus 200 through the relay server
400, the display apparatus 300 may generate a security UI 930 and
display the image captured by the security device covering the
living room/smart TV through the generated security UI 930.
[0137] FIGS. 10A through 10C are views illustrating a method for
monitoring a peripheral status through a peripheral device in a
display apparatus activated on a lock screen, according to an
exemplary embodiment.
[0138] As shown in FIG. 10A, the display apparatus 300 may display
a virtual object on a screen in a second execution mode activated
on a lock screen. Here, the virtual object may include a home
object 1010 indicating a home of a user and a peripheral object
1040 indicating an environment status around the home of the user,
according to an exemplary embodiment.
[0139] Therefore, the display apparatus 300 may provide a status of
the home according to an occurrence of an event through the home
object 1010 displayed on the lock screen, based on status
information that is corresponding to at least one peripheral device
100 and received through the relay server 400. The display
apparatus 300 may also provide the environment status around the
home of the user through the peripheral object 1040 based on the
status information about the at least one peripheral device 100 or
environment information received through a data communication with
an external server (not shown).
[0140] As shown in FIG. 10B, the display apparatus 300 may parse an
occurrence of an event indicating whether the peripheral device 100
installed in the home is behaving abnormally and/or is in
operation, based on the received status information about the at
least one peripheral device 100. For example, if it is determined
that the peripheral device 100 of a smart illumination lamp
installed in a living room is currently switched on based on the
received status information about the at least one peripheral
device 100, the display apparatus 300 displays a device status icon
1020 in an area of the home object 1010.
[0141] Therefore, the user may recognize that at least one of the
peripheral devices 100 installed in the home is behaving abnormally
and/or is in operation, through the device status icon 1020
displayed on the home object 1010 displayed on the lock screen. The
user may select the device status icon 1020 displayed on the home
object 1010 to recognize a more detailed status. If a selection
command, as described above, is input, the display apparatus 300
may display a control UI including detailed information and a
control menu of the peripheral device 100 corresponding to the
occurrence of the event (the light bulb in the above exemplary
embodiment) on a screen displaying the lock screen (not shown).
[0142] In detail, if a command to select the device status icon
1020 is input as in the above-described exemplary embodiment, the
display apparatus 300 may display a control UI, including detailed
information "The illumination lamp in the living room is switched
on." and a control menu for controlling switching on and/or off the
illumination lamp, on the lock screen. Therefore, the user may
determine that the illumination lamp in the living room of the home
is switched on and may switch off the illumination lamp by using
the control menu included in the control UI.
[0143] As shown in FIG. 10C, the display apparatus 300 may parse
whether a security-related event occurs, based on the received
status information about the at least one peripheral device 100.
For example, if it is determined that an event occurrence signal of
the peripheral device 100 is generated in a security system
installed at a front door based on the received status information
about the at least one peripheral device 100, the display apparatus
300 displays a security warning icon 1030 in an area of the home
object 1010.
[0144] Therefore, the user may recognize that a security problem
such as an intrusion has occurred in the home, through the security
warning icon 1030 displayed in the home object 1010 displayed on
the lock screen, and select the security warning icon 1030 to
recognize a detailed status. If the security warning icon 1030 is
selected, the display apparatus 300 may generate a security UI as
shown in FIGS. 9A through 9C and display an image captured by the
at least one peripheral device 100 through the generated security
UI.
[0145] Therefore, the user may monitor an indoor security status in
real time using the image displayed through the security UI of the
display apparatus 300.
[0146] FIGS. 11A through 11C are views illustrating a method for
monitoring a peripheral status through a peripheral device in a
display apparatus activated on a lock screen, according to another
exemplary embodiment.
[0147] As shown in FIG. 11A, the display apparatus 300 may display
a home object 1110 indicating a home of a user on a screen in a
second execution mode activated on a lock screen.
[0148] When the home object 1110 is displayed on the lock screen,
as shown in FIG. 11B, the user may select an area 1020 of the home
object 1110 to check who stays in the home and perform a dragging
operation in a preset direction based on the selected area 1020. If
a touch command of the user is input, the display apparatus 300
determines that the touch command is a command to request an indoor
family member status, and generates and displays a family
monitoring UI 1030 for showing a family member status in the home
based on received status information about at least one peripheral
device 100, as shown in FIG. 11C.
[0149] In detail, the status information that is about the at least
one peripheral device 100 and received from the relay server 400
may include status information about home devices such as the four
peripheral devices 100-1 through 100-4 and status information about
a peripheral display apparatus (not shown) existing on the same
network as the terminal apparatus 200. The status information about
the peripheral display apparatus existing on the same network as
the terminal apparatus 200 may include at least one of user
information and device information of the peripheral display
apparatus.
[0150] Therefore, if a user command to request the family status is
input, the display apparatus 300 may acquire the user information
and the device information of the peripheral display apparatus
based on the status information about the peripheral display
apparatus, from among the received status information. If the user
information and the device information are acquired, the display
apparatus 300 extracts user face images corresponding to the
acquired user information and displays the extracted user face
images on family list information 1040 of the family monitoring UI
1030. Therefore, as shown in FIG. 11C, the display apparatus 300
may display the family monitoring UI 1030, including the family
list information 1040 providing user face images corresponding to
family members staying in the home, on the lock screen. Therefore,
the user may check who stays in the home, through the family list
information 1040 of the family monitoring UI 1030 displayed on the
lock screen of the display apparatus 300.
[0151] As shown in FIG. 11C, the user may select a first family
member 1050 of family members provided through the family list
information 1040. If such a selection command is input, the display
apparatus 300 initiates a call to a display apparatus (not shown)
of the first family member 1050 corresponding to the input
selection command based on stored contact information. Therefore,
the user may call family members through the family list
information 1040 without being additionally well-acquainted with or
inputting the contact information that belongs to the family
members.
[0152] FIGS. 12A and 12B are views illustrating a method for
displaying a control UI in a display apparatus in which a home
network service-related application is activated, according to an
exemplary embodiment.
[0153] As shown in FIG. 12A, the display apparatus 300 may execute
a home network-related application 1210 according to a request of
the user. If the home network-related application 1210 is executed,
the display apparatus 300 may generate and display a control UI
1230 including status-related detailed information and a control
menu of the corresponding peripheral device 100 based on status
information that is about at least one peripheral device 100 and
received through the terminal apparatus 200 or the relay server
400, regardless of whether a communication with the terminal
apparatus 200 is possible.
[0154] Here, the peripheral devices 100 may include home devices
such as the four peripheral devices 100-1 through 100-4 and a
peripheral display apparatus (not shown) existing on the same
network as the terminal apparatus 200. In this case, status
information about the peripheral display apparatus, existing on the
same network as the terminal apparatus 200, may include at least
one of user information and device information of the peripheral
display apparatus (not shown).
[0155] Therefore, if the home network service-related application
1210 is executed as shown in FIG. 10A, the display apparatus 300
may provide a briefing menu 1220, which is to show user information
of a user staying in the home and a current status of the
peripheral device 100 corresponding to dynamic elements, from among
the four peripheral devices 100-1 through 100-4, through the home
network service-related application 1010. Here, the dynamic
elements may include an illumination intensity, a temperature, and
TV editing information, and the peripheral devices 100 performing
operations of the dynamic elements may be a smart illumination
lamp, an air conditioner, and a smart TV, according to an exemplary
embodiment.
[0156] If the briefing menu 1220 is selected by the user when the
home network service-related application 1010 is executed, the
display apparatus 300 may display a control UI 1230 including
status-related detailed information about the peripheral device 100
corresponding to the dynamic elements and a control menu for
controlling the corresponding peripheral device 100, based on the
received status information. Here, the control menu displayed in
the control UI 1230 may be a menu for controlling a basic operation
of each peripheral device 100 (for example, an operation of
switching on and/or off each peripheral device 100, an operation of
recording a program, or the like). The status-related detailed
information may be detailed result information that is parsed based
on status information about the peripheral device 100 corresponding
to the dynamic elements.
[0157] In detail, if a command to select the briefing menu 1220 is
input, the display apparatus 300 may display the control UI 1230
including detailed result information, which is parsed based on
status information about the smart illumination lamp, the air
conditioner, and the smart TV corresponding to the dynamic
elements, and a control menu for controlling operations of the
smart illumination lamp, the air conditioner, and the smart TV,
according to an exemplary embodiment.
[0158] In other words, as shown in FIG. 12B, the display apparatus
300 may display the control UI 1230 including detailed result
information "The smart illumination lamp is switched off. Do you
want to switch on the smart illumination lamp?" and a control menu
for switching on and/or off the smart illumination lamp in relation
to the smart illumination lamp, and detailed result information
"The room temperature is higher than an appropriate temperature. Do
yon want to switch on the air conditioner?" and a control menu for
operating the air conditioner in relation to the air
conditioner.
[0159] Therefore, the user may control the corresponding peripheral
device 100 by using a control menu corresponding to the peripheral
device 100 that is to be controlled, using the detailed result
information about the peripheral device 100 corresponding to the
dynamic elements.
[0160] FIG. 13 is a block diagram of the terminal apparatus 200,
according to an exemplary embodiment.
[0161] Referring to FIG. 13, the terminal apparatus 200 includes a
communicator 210, a storage unit 220, and a controller 230.
[0162] The communicator 210 periodically communicates with
registered at least one peripheral device 100 via a wired
connection or wirelessly to receive status information from the at
least one peripheral device 100. The communicator 210 also
communicates with the display apparatus 300 or the relay server
400. Here, the status information may include at least one of
activation information including at least one of on/off
information, setting information, and sensing information of the
peripheral device 100 and communication status information between
the peripheral device 100 and the terminal apparatus 200, according
to an exemplary embodiment.
[0163] The storage unit 220 stores the status information of the at
least one peripheral device 100 received through the communicator
210. The storage unit 220 may further include identification
information of the display apparatus 300 existing on the same
network, identification information of the relay server, etc.
[0164] If a status information request message of the peripheral
device 100 is received from the display apparatus 300, the
controller 230 controls the communicator 210 to transmit status
information about a controllable peripheral device 100 or status
information about a peripheral device 100 that may be monitored, to
the display apparatus 300 based on the status information of the at
least one peripheral device 100 stored in the storage unit 220
according to whether a communication with the display apparatus 300
is possible.
[0165] In detail, if the communication with the display apparatus
300 is possible (i.e., the terminal apparatus 200 exists on the
same network as the display apparatus 300), the controller 230
transmits the status information about the controllable peripheral
device 100 to the display apparatus 300 based on the status
information of the at least one peripheral device 100 stored in the
storage unit 220. If a control command is received from the display
apparatus 300 that receives the status information about the
controllable peripheral device 100, the controller 230 transmits
the received control command to the corresponding peripheral device
100. Therefore, the corresponding peripheral device 100 may perform
an operation based on the received control command.
[0166] If the communication with the display apparatus 300 is
impossible (i.e., the terminal apparatus 200 exists on a different
network from the display apparatus 300 and thus receives the status
information request message from the display apparatus 300 through
the relay server 400), the controller 230 transmits the status
information about the peripheral device 100 that may be monitored,
to the display apparatus 300 through the relay server 400 based on
the status information of the at least one peripheral device 100
stored in the storage unit 220.
[0167] If a monitoring performance command is received from the
display apparatus 300 through the relay server 400, the controller
230 transmits the monitoring performance command to the
corresponding peripheral device 100. The peripheral device 100 that
receives the monitoring performance command captures an image of a
peripheral space according to the monitoring performance command
and transmits captured image information to the terminal apparatus
200. Thereafter, the controller 230 may transmit the image
information received from the peripheral device 100 to the display
apparatus 300 through the relay server 400.
[0168] If a direct data communication with the display apparatus
300 is possible as described above, the terminal apparatus 200
transmits the status information about the controllable peripheral
device 100 to the display apparatus 300 based on the stored status
information about the at least one peripheral device 100.
Therefore, the display apparatus 300 may generate and display a
control UI for controlling the peripheral device 100 based on the
received status information, and the user may control the
peripheral device 100 installed in the home through the control UI
displayed on the display apparatus 300.
[0169] If an indirect data communication with the display apparatus
300 is possible through the relay server 400, the terminal
apparatus 200 transmits the status information about the peripheral
device 100 that may be monitored, to the display apparatus 300
based on the stored status information about the at least one
peripheral device 100. Therefore, the display apparatus 300 may
generate a security UI for monitoring a peripheral space based on
the received status information and display an image, which is
captured by the peripheral device 100, as requested by the user,
through the generated security UI. As a result, the user may
monitor the image captured by the peripheral device 100 selected by
the user through the security UI displayed on a screen of the
display apparatus 300 to check in real time whether risk elements
are detected in the home.
[0170] A method of controlling operations of the display apparatus
300 and the terminal apparatus 200 will now be described in
detail.
[0171] FIG. 14 is a flowchart of a method for controlling the
display apparatus 300, according to an exemplary embodiment.
[0172] Referring to FIG. 14, in operation S1410, the display
apparatus 300 communicates with the terminal apparatus 200 or the
relay server 400 to receive collected status information about at
least one peripheral device 100 from the terminal apparatus 200.
Here, the status information may include at least one of activation
information including at least one of on/off information, setting
information, sensing information of the peripheral device 100 and
communication status information between the peripheral device 100
and the terminal apparatus 200, according to an exemplary
embodiment.
[0173] If the status information about the at least one peripheral
device 100 is received in operation S1410, the display apparatus
300 determines whether a communication with the terminal apparatus
200 is possible in operation S1420. In detail, if a direct data
communication with the terminal apparatus 200 is not possible
through a short-range wireless communication, the display apparatus
300 determines whether an indirect data communication with the
terminal apparatus 200 is possible through the relay server
400.
[0174] If it is determined in operation S1420 that the direct data
communication with the terminal apparatus 200 is possible through
the short-range wireless communication, the display apparatus 300
generates and displays a control UI for controlling the at least
one peripheral device 100 based on the received status information
about the at least one peripheral device 100 in operation
S1430.
[0175] If it is determined in operation S1420 that indirect data
communication with the terminal apparatus 200 is possible through
the relay server 400, the display apparatus 300 generates and
displays a security UI for monitoring a peripheral space through
the at least one peripheral device 100 based on the received status
information about the at least one peripheral device 100 in
operation S1440.
[0176] A method of generating a control UI for controlling at least
one peripheral device 100 or a security UI for monitoring a
peripheral space through the at least one peripheral device 100
according to a communication status between the display apparatus
300 and the terminal apparatus 200 will now be described in
detail.
[0177] FIG. 15 is a flowchart of a method for generating a control
UI for controlling a peripheral device in a display apparatus,
according to an exemplary embodiment.
[0178] Referring to FIG. 15, if it is determined that a direct data
communication with the terminal apparatus 200 is possible through a
short-range wireless communication, the display apparatus 300
generates and displays control list information about a
controllable peripheral device 100, based on received status
information about at least one peripheral device 100, in operation
S1510. If a command to select at least one peripheral device 100 is
input from a user when the control list information is displayed,
the display apparatus 300 generates and displays a control UI
including a control menu for controlling the peripheral device 100,
corresponding to the input command, in operation S1520. Here, the
control list information may be list information including device
information about at least one controllable peripheral device
100.
[0179] Therefore, if a command to select the controllable
peripheral device 100 is input when the control list information
including the device information about the at least one peripheral
device 100 is displayed, the display apparatus 300 may generate and
display the control UI including a control menu for controlling the
peripheral device 100 corresponding to the input command, from
among stored control commands of respective peripheral devices
100.
[0180] The display apparatus 300 that generates and displays the
control UI for controlling the selected peripheral device 100 as
described above may differently generate the control list
information according to an execution mode. Here, the execution
mode may include at least one of a first execution mode activated
on a quick menu, a second execution mode activated on a lock
screen, and a third execution mode in which a home network
service-related application is activated. However, the execution
modes are not limited thereto.
[0181] In the first execution mode activated on the quick menu, the
display apparatus 300 may generate and display first control list
information including all controllable peripheral devices 100 based
on received status information. In the second execution mode
activated on the lock screen, the display apparatus 300 may
generate and display second control list information including an
activated peripheral device 100, from among all controllable
peripheral devices 100. In the third execution mode in which the
home network service-related application is activated, the display
apparatus 300 may generate and display a control UI including
status-related detailed information and a control menu of at least
one peripheral device 100 based on received status information
regardless of whether a communication with the terminal apparatus
200 is possible.
[0182] As described above, the display apparatus 300 may
differently generate and display the control list information and
the control UI according to an execution mode that is currently
executed.
[0183] If a command to select at least one control menu is input
when a control UI including a plurality of control menus is
displayed, the display apparatus 300 transmits a control command
related to a control menu, corresponding to the input command, to
the terminal apparatus 200, in operation S1530. Thereafter, the
terminal apparatus 200 transmits the control command to the
peripheral device 100 selected by the user based on the control
command received from the display apparatus 300. As a result, the
peripheral device 100 that receives the control command from the
terminal apparatus 200 may perform a control operation based on the
received control command.
[0184] FIG. 16 is a flowchart of a method for generating a security
UI for monitoring a peripheral device in a display apparatus,
according to an exemplary embodiment.
[0185] Referring to FIG. 16, if it is determined that an indirect
communication with the terminal apparatus 200 is possible through
the relay server 400, the display apparatus 300 generates and
displays security list information about a peripheral device 100
that may be monitored, based on received status information about
the peripheral device 100 and setting information of each
registered peripheral device 100 in operation S1610. Here, the
security list information may include identification information
indicating the peripheral device 100 that may monitor a peripheral
space/may be monitored in a peripheral space by a security device
and area information showing an area that may be monitored through
the corresponding peripheral device 100.
[0186] If a command to select the peripheral device 100 that may
monitor the peripheral space is input when the control list
information is displayed, the display apparatus 300 generates and
displays a user certification UI for certifying a user in operation
S1620.
[0187] If an encryption key is input when the user certification UI
is displayed, the display apparatus 300 determines whether the
input encryption key corresponds to a preset encryption information
in operation S1630. If it is determined in operation S1630 that the
encryption key corresponds to the preset encryption information,
the display apparatus 300 requests the terminal apparatus 200,
through the relay server 400, to monitor the peripheral device 100
corresponding to the input selection command in operation S1640.
The terminal apparatus 200 transmits a monitoring performance
command to the peripheral device 100 that is requested to be
monitored, according to the monitoring request and, if captured
image information of the peripheral space is received from the
corresponding peripheral device 100, transmits the received image
information to the display apparatus 300 through the relay server
400.
[0188] If the image information is received, the display apparatus
300 generates a security UI for displaying an image captured by the
peripheral device 100 and displays an image corresponding to the
received image information in an area of the generated security UI
in operation S1650.
[0189] Therefore, the user may monitor, in real time, the
peripheral space related to the peripheral device 100 selected by
the user through the image displayed in the area of the security UI
displayed on a screen of the display apparatus 300.
[0190] The display apparatus 300 may generate and display a
security UI including a monitoring control menu for controlling a
monitoring operation of the peripheral device 100 that is requested
to be monitored. Also, the display apparatus 300 may generate and
display a security UI further including an editing menu for editing
the image captured by the peripheral device 100, that is requested
to be monitored.
[0191] Here, the monitoring control menu, for example, may include
at least one of a menu for switching on and/or off the peripheral
device 100 that performs monitoring and a menu for controlling a
monitoring direction of the corresponding peripheral device 100,
according to an exemplary embodiment. Also, the editing menu may
include at least one of a menu for capturing or recording the image
captured by the peripheral device 100, that is requested to be
monitored, and a menu for stopping a currently displayed security
UI. Therefore, if a command to select the monitoring control menu
is input using the security UI, the display apparatus 300 transmits
a control command corresponding to the input command to the
terminal apparatus 200, through the relay server 400, in operation
S1660. Thereafter, the terminal apparatus 300 may transmit the
corresponding control command to the peripheral device 100 that
transmits the captured image, and the peripheral device 100 that
receives the control command may perform a control operation based
on the received control command.
[0192] Therefore, the user may monitor, in real time, a status of
the home through the image displayed in the area of the security UI
displayed on the screen of the display apparatus 300, and control a
capturing operation of the peripheral device 100 or edit the image
captured by the corresponding peripheral device 100 through the
monitoring control menu or the editing menu incorporated in the
security UI.
[0193] FIG. 17 is a flowchart of a method for controlling the
terminal apparatus 200, according to an exemplary embodiment.
[0194] Referring to FIG. 17, the terminal apparatus 200
periodically communicates with the registered at least one
peripheral device 100 to receive and store status information about
the at least one peripheral device 100 in operation S1710. Here,
the status information may include at least one of activation
information including at least one of on/off information, setting
information, sensing information of the peripheral device 100 and
communication status information between the peripheral device 100
and the terminal apparatus 200, according to an exemplary
embodiment.
[0195] If a status information request message requesting the
status information of the peripheral device 100 is received from
the display apparatus 300, the terminal apparatus 200 determines
whether a communication with the display apparatus 300 is possible
in operations S1720 and S1730.
[0196] In detail, if the status information request message is
received from the display apparatus 300, the terminal apparatus 200
may determine that the terminal apparatus 200 exists on a network
on which the terminal apparatus 200 directly communicates with the
display apparatus 300 (i.e., the terminal apparatus 200 exists on
the same network as the display apparatus 200). If it is determined
that the status information request message is received from the
display apparatus 300 through the relay server 400, the terminal
apparatus 200 may determine that the terminal apparatus 200
indirectly communicates with the display apparatus 300 (i.e., the
terminal apparatus 200 exists on a different network from the
display apparatus 300).
[0197] If it is determined that the status information request
message is received through a direct communication with the display
apparatus 300 as described above, the terminal apparatus 200
transmits status information about a controllable peripheral device
100 to the display apparatus 300, based on stored status
information about the at least one peripheral device 100, in
operation S1740.
[0198] If a control command is received from the display apparatus
300, the terminal apparatus 200 transmits the received control
command to the peripheral device 100 corresponding to the received
control command. Therefore, the peripheral device 100 may perform
an operation based on the received control command.
[0199] If it is determined that the status information request
message is received from the display apparatus 300 through the
relay server 400, the terminal apparatus 200 transmits status
information about the peripheral device 100 that may be monitored,
to the display apparatus 300 through the relay server 400, based on
the stored status information about the at least one peripheral
device 100, in operation S1750.
[0200] If a monitoring performance command is received through the
relay server 400, the terminal apparatus 200 transmits the
monitoring performance command to the peripheral device 100
corresponding to the received monitoring performance command. The
peripheral device 100 that receives the monitoring performance
command captures an image of a peripheral space according to the
received monitoring performance command and transmits the captured
image information to the terminal apparatus 200. If the image
information is received, the terminal apparatus 200 may transmit
the image information regarding the image captured by the
peripheral device 100 to the display apparatus 300 through the
relay server 400.
[0201] Therefore, the user may monitor the image captured by the
peripheral device 100 selected by the user through the security UI,
displayed on the screen of the display apparatus 300, to check in
real time whether risk elements are detected in the home.
[0202] According to various exemplary embodiments described above,
a display apparatus may control a home device that is a peripheral
device according to a communication status with a terminal
apparatus or monitor an environment status of a home through an
image captured by the home device in real time.
[0203] The foregoing exemplary embodiments and advantages are
merely exemplary and are not to be construed as limiting. The
present teaching can be readily applied to other types of
apparatuses. Also, the description of the exemplary embodiments is
intended to be illustrative, and not to limit the scope of the
claims, and many alternatives, modifications, and variations will
be apparent to those skilled in the art.
* * * * *