U.S. patent application number 16/896743 was filed with the patent office on 2020-12-03 for using voice commands from a mobile device to remotely access and control a computer.
This patent application is currently assigned to Voice Tech Corporation. The applicant listed for this patent is Voice Tech Corporation. Invention is credited to Todd R. Smith.
Application Number | 20200382601 16/896743 |
Document ID | / |
Family ID | 1000005030637 |
Filed Date | 2020-12-03 |
![](/patent/app/20200382601/US20200382601A1-20201203-D00000.png)
![](/patent/app/20200382601/US20200382601A1-20201203-D00001.png)
![](/patent/app/20200382601/US20200382601A1-20201203-D00002.png)
United States Patent
Application |
20200382601 |
Kind Code |
A1 |
Smith; Todd R. |
December 3, 2020 |
Using Voice Commands From A Mobile Device To Remotely Access And
Control A Computer
Abstract
A method of using voice commands from a mobile device to
remotely access and control a computer. The method includes
receiving audio data from the mobile device at the computer. The
audio data is decoded into a command. A software program that the
command was provided for is determined. At least one process is
executed at the computer in response to the command. Output data is
generated at the computer in response to executing at least one
process at the computer. The output data is transmitted to the
mobile device.
Inventors: |
Smith; Todd R.; (Houston,
TX) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Voice Tech Corporation |
Tyler |
TX |
US |
|
|
Assignee: |
Voice Tech Corporation
Tyler
TX
|
Family ID: |
1000005030637 |
Appl. No.: |
16/896743 |
Filed: |
June 9, 2020 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16896693 |
Jun 9, 2020 |
|
|
|
16896743 |
|
|
|
|
16655047 |
Oct 16, 2019 |
|
|
|
16896693 |
|
|
|
|
15704871 |
Sep 14, 2017 |
10491679 |
|
|
16655047 |
|
|
|
|
11809998 |
Jun 4, 2007 |
9794348 |
|
|
15704871 |
|
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
H04L 67/125 20130101;
H04M 11/007 20130101; G10L 15/26 20130101; H04M 2203/1033 20130101;
G06F 9/4843 20130101; H04M 3/4938 20130101; G06F 9/451 20180201;
H04L 63/083 20130101 |
International
Class: |
H04L 29/08 20060101
H04L029/08; G06F 9/48 20060101 G06F009/48; G06F 9/451 20060101
G06F009/451; H04M 3/493 20060101 H04M003/493; H04M 11/00 20060101
H04M011/00; H04L 29/06 20060101 H04L029/06 |
Claims
1. (canceled)
2. The method of claim 17, wherein the communications medium is an
analog communications medium.
3. The method of claim 17, wherein the communications medium is a
digital communications medium.
4. The method of claim 17, wherein the coupling the mobile device
to the general purpose processing platform is performed through a
physical connection.
5. The method of claim 4, wherein the physical connection comprises
a wire.
6. The method of claim 4, wherein the physical connection comprises
an optical fiber.
7. The method of claim 4, wherein the physical connection comprises
a telecommunications medium.
8. The method of claim 17, wherein the coupling the mobile device
to the general purpose processing platform is performed through a
virtual connection.
9. The method of claim 8, wherein the virtual connection comprises
randomly assigned memory locations of a data memory device.
10. The method of claim 8, wherein the virtual connection comprises
a hypertext transfer protocol (HTTP) link.
11. The method of claim 17, wherein the coupling the mobile device
to the general purpose processing platform is performed through a
logical connection.
12. The method of claim 11, wherein the logical connection
comprises communication between one or more semiconductor devices
in an integrated circuit.
13. The method of claim 17, wherein the coupling the mobile device
to the general purpose processing platform is performed through a
network.
14. The method of claim 13, wherein the network is a wire-line
network.
15. The method of claim 13, wherein the network is a wireless
network.
16. The method of claim 13, wherein the network is a public
switched telephone network.
17. A method of accessing and controlling a computer from a mobile
device, comprising: coupling a mobile device to a computer with a
communications medium, wherein the mobile device is any hardware
device capable of mobility and the computer is a general purpose
processing platform comprised of one or more discrete components;
receiving voice or data information from the mobile device, at the
general purpose processing platform, at an audio command interface;
the audio command interface determines whether the voice or data
information corresponds to a command; the audio command interface
selects, from at least one operating system and at least one
application, one operating system or one application, wherein the
audio command interface decides which is the appropriate operating
system or application to execute at least one process in response
to the command; and executing with the selected operating system or
application the at least one process in response to the command.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is a continuation of U.S. application Ser.
No. 16/896,693 filed on Jun. 9, 2020, which is a continuation of
U.S. application Ser. No. 16/655,047 filed on Oct. 16, 2019, which
is a continuation of U.S. Pat. No. 10,491,676 issued on Nov. 26,
2019, which is a continuation of U.S. Pat. No. 9,794,348 issued on
Oct. 17, 2017, the entire disclosures of which are incorporated by
reference.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
[0002] Not applicable.
BACKGROUND OF THE INVENTION
Field of the Invention
[0003] The present invention pertains to the field of computer
operating systems, and more specifically to a system for using
voice commands from a mobile device to remotely access and control
a computer, and a method of use that allows a person to use voice
commands from a mobile device to remotely access and control a
computer.
Background of the Related Art
[0004] Systems for allowing a person to use voice commands from a
mobile device to remotely access and control a computer are known
in the art. However, such prior art systems are
application-specific, meaning they are configured to allow the
person to use voice commands from a mobile device to remotely
access and control a specific application at a computer. Therefore,
the prior art systems require the person to have multiple mobile
devices and/or systems to remotely access and control the different
applications at a computer. Additionally, the prior art systems
limit the audible and visible feedback the person can receive from
a computer while using voice commands from a mobile device to
remotely access and control the computer.
SUMMARY OF THE INVENTION
[0005] In accordance with the present invention, a system is
provided for using voice commands from a mobile device to remotely
access and control a computer, and a method of use is provided that
allows a person to use voice commands from a mobile device to
remotely access and control a computer.
[0006] In particular, a system and method of use are provided that
allow a person to use voice commands from a mobile device to
remotely interact with a computer.
[0007] In accordance with an exemplary embodiment of the present
invention, a method of using voice commands from a mobile device to
remotely access and control a computer is provided. The method
includes receiving audio data from the mobile device at the
computer. The audio data is decoded into a command. A software
program that the command was provided for is determined. At least
one process is executed at the computer in response to the command.
Output data is generated at the computer in response to executing
at least one process at the computer. The output data is
transmitted to the mobile device.
[0008] The present invention provides many important technical
advantages. One important technical advantage of the present
invention is a system that allows a person to use voice commands
from a mobile device to remotely access and control a computer and
see and hear the images and sounds generated at the computer, in
response to the voice commands, in a manner similar to what the
person would see and hear, if the person were directly and locally
accessing and controlling the computer using a voice command
system.
[0009] Those skilled in the art will further appreciate the
advantages and superior features of the invention together with
other important aspects thereof on reading the detailed description
that follows in conjunction with the drawings.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
[0010] For a detailed description of the preferred embodiments of
the invention, reference will now be made to the accompanying
drawings in which:
[0011] FIG. 1 is a diagram of a system for using voice commands
from a mobile device to remotely access and control a computer in
accordance with an exemplary embodiment of the present
invention;
[0012] FIG. 2 is a diagram of a system for providing a mobile
device interface in accordance with an exemplary embodiment of the
present invention;
[0013] FIG. 3 is a diagram of a system for providing an audio
command interface in accordance with an exemplary embodiment of the
present invention; and
[0014] FIG. 4 is a diagram of a method for using voice commands
from a mobile device to remotely access and control a general
purpose computer in accordance with an exemplary embodiment of the
present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0015] In the description that follows, like parts are marked
throughout the specification and drawings with the same reference
numerals, respectively. The drawing figures might not be to scale,
and certain components can be shown in generalized or schematic
form and identified by commercial designations in the interest of
clarity and conciseness.
[0016] FIG. 1 is a diagram of system 100 for using voice commands
from a mobile device to remotely access and control a computer in
accordance with an exemplary embodiment of the present invention.
System 100 allows a person to use voice commands from a mobile
device to remotely access and control a computer, whereby the
person can operate the operating system at the computer, operate
native applications at the computer, and see and hear the images
and sounds generated at the computer in response to the voice
commands.
[0017] System 100 includes mobile device 102, which can be a
suitable mobile device such as a cellular phone, smart phone,
touch-screen device, personal digital assistant, tablet device,
notebook device, laptop device, or other suitable mobile device
that allows communication with a computer via a wireless or
wire-line network or a suitable combination of a wireless and
wire-line network.
[0018] System 100 also includes general purpose computer 104, which
can be a general purpose processing platform or other suitable
processing platforms. General purpose computer 104 includes mobile
device interface 106, audio command interface 108, operating system
interface 110, and native applications 112, each of which can be
implemented in hardware, software, or a suitable combination of
hardware and software, which can be one or more software systems
operating on a general purpose processing platform. As used herein,
"hardware" can include a combination of discrete components, an
integrated circuit, an application-specific integrated circuit, a
field programmable gate array, or other suitable hardware. As used
herein, "software" can include one or more objects, agents,
threads, lines of code, subroutines, separate software
applications, two or more lines of code or other suitable software
structures operating in two or more software applications or on two
or more processors, or other suitable software structures. In one
exemplary embodiment, software can include one or more lines of
code or other suitable software structures operating in a general
purpose software application, such as an operating system; and one
or more lines of code or other suitable software structures
operating in a specific purpose software application.
[0019] Mobile device 102 is coupled to general purpose computer 104
via communications medium 114. As used herein, "communications
medium" can include a wire-line communications medium, a wireless
communications medium, an optical communications medium, an analog
communications medium, a digital communications medium, other
suitable communications media or a suitable combination of
communications media. As used herein, the term "coupled" and its
cognate terms such as "couples" or "couple," can include a physical
connection (such as a wire, optical fiber, or a telecommunications
medium), a virtual connection (such as through randomly assigned
memory locations of a data memory device or a hypertext transfer
protocol (HTTP) link), a logical connection (such as through one or
more semiconductor devices in an integrated circuit), or other
suitable connections. In one exemplary embodiment, a communications
medium can be a network or other suitable communications media.
[0020] Mobile device interface 106 receives voice or data
information from mobile device 102. In one exemplary embodiment,
mobile device interface 106 is configured to monitor communications
medium 114 interface devices, or other suitable mechanisms for
interfacing with communications medium 114. In this exemplary
embodiment, mobile device interface 106 can monitor a public
switched telephone network (PS TN) modem that responds to ring
signals when a call is being made from mobile device 102 to general
purpose computer 104, such as through a dedicated telephone number.
Mobile device interface 106 can answer or monitor the answering of
the PSTN modem and determine whether mobile device 102 is calling
or whether some other device, such as a fax machine or other
computer is calling. Likewise, mobile device interface 106 can
operate or monitor a network connection, such as over a local area
network, DSL modem, cable modem or other suitable internet or
network connections and can determine whether mobile device 102 has
transmitted data to general purpose computer 104. In one exemplary
embodiment, mobile device interface 106 can receive addressing
data, such as through the use of keypad entries or other data
control devices that can transmit network addressed data to general
purpose computer 104 at a predetermined network address. Mobile
device interface 106 can receive such network addressed data and
determine whether it has been transmitted by mobile device 102 or
other suitable devices. If it is determined by mobile device
interface 106 that mobile device 102 has transmitted data to
general purpose computer 104, mobile device interface 106
establishes a session with mobile device 102 to allow mobile device
102 to interact with general purpose computer 104.
[0021] Audio command interface 108 receives audio commands from
mobile device interface 106. In one exemplary embodiment, audio
command interface 108 can receive data from mobile device interface
106 and detect audio commands in the data. For example, mobile
device interface 106 may receive data from mobile device 102
without determining whether the data includes audio data, keypad
entry data, keyboard entry data, or other suitable data. Likewise,
mobile device interface 106 can perform voice recognition and other
suitable processing and can provide voice data to audio command
interface 108.
[0022] Audio command interface 108 determines whether voice data
corresponds to an audio command. In one exemplary embodiment, audio
command interface 108 can determine whether voice data corresponds
to one of two or more predetermined audio commands. Audio command
interface 108 can also execute such detected commands. Audio
command interface 108 can also provide a list of available commands
to the person using mobile device 102, such as by presenting
prompts to the person, by allowing the person to request a list of
available audio commands, or in other suitable manners. Likewise,
audio command interface 108 can include one or more states, such
that certain audio commands are available depending upon the state
of audio command interface 108. In this exemplary embodiment, when
audio command interface 108 is being used to access and control a
native application 112 or operating system interface 110, audio
command interface 108 may change states to provide different audio
commands to the person using mobile device 102.
[0023] Operating system interface 110 allows audio command
interface 108 to activate various operating system commands. In one
exemplary embodiment, audio command interface 108 can include a
file of available operating system commands that can be matched
with voice data, such as operating system commands that would
otherwise be available to a person directly and locally accessing
and controlling general purpose computer 104 using a voice command
system. For example, an operating system command may include a file
search command that can be activated through audio command
interface 108 and operating system interface 110. In this exemplary
embodiment, the person using mobile device 102, after establishing
a session with mobile device interface 106, could state the command
"search", and audio command interface 108 could cause the search
functionality of operating system interface 110 to be activated.
The person could then further identify information to be searched
for such as documents, pictures, videos, all files and folders, or
other suitable processes can be performed.
[0024] Native applications 112 can include one or more native
applications accessed and controlled at general purpose computer
104. In one exemplary embodiment, native applications 112 can be
loaded into or interface with audio command interface 108, such as
by installing an applications program interface (API) or other
suitable data into audio command interface 108 that identifies
native applications 112 and provides available commands for audio
command interface 108 to interface with native applications 112. In
this exemplary embodiment, certain predetermined commands for
native applications 112 can be provided, such as commands that one
person would be authorized to use, whereas commands another person
would not be authorized to use can be inhibited or blocked for that
person (such as a print command, a command allowing the person to
turn off general purpose computer 104, or other suitable
commands).
[0025] In operation, system 100 allows a person to use voice
commands from mobile device 102 to remotely access and control
general purpose computer 104 over communications medium 114. Unlike
prior art systems that require voice command interoperability be
provided for each separate native application, system 100 allows
mobile device 102 to remotely access and control both operating
system functions and native applications at general purpose
computer 104. In this manner, the person using mobile device 102
can see and hear the images and sounds generated at general purpose
computer 104, regardless of the native application or operating
system function that is used to generate such images and sounds. As
such, each native application 112 or operating system of general
purpose computer 104 is not required to be configured to remotely
interact with mobile device 102. Likewise, where native
applications at general purpose computer 104 are configured to
remotely interact with mobile device 102, system 100 allows mobile
device 102 to switch between native applications that are
configured and operating system functions and native applications
that are not configured for interaction with mobile device 102.
[0026] FIG. 2 is a diagram of system 200 for providing a mobile
device interface in accordance with an exemplary embodiment of the
present invention. System 200 includes network interface 202,
native application video output conversion 204, and native
application audio output conversion 206, each of which can be
implemented in hardware, software, or a suitable combination of
hardware and software, which can be one or more software systems
operating on a general purpose processing platform.
[0027] Network interface 202 provides an interface between general
purpose computer 104 and communications medium 114 or other
suitable networks. In one exemplary embodiment, network interface
202 can monitor a modem, such as a PSTN modem, cable modem, DSL
modem, or other suitable modems for incoming data traffic that
indicates that mobile device 102 or other suitable devices are
attempting to interface with general purpose computer 104.
Likewise, network interface 202 can monitor a network card, such as
a local area network connection, network interface card, or other
suitable devices. Network interface 202 receives the incoming
mobile device 102 data and performs voice or data recognition on
the data.
[0028] Native application video output conversion 204 receives
native application video output from general purpose computer 104,
such as video data that has been generated by a native application,
and converts the video data into a format for transmission to
mobile device 102 or other suitable mobile devices. In one
exemplary embodiment, native application video output conversion
204 can receive screen information from general purpose computer
104 operating under a native application or operating system
command and can convert the screen information into a format for
transmission to mobile device 102. Likewise, native application
video output conversion 204 can convert the native application
video output into a different format for viewing at mobile device
102, such as a format that excludes data that would not provide
additional functionality, but would otherwise require excessive
bandwidth requirements for transfer to mobile device 102.
[0029] Native application audio output conversion 206 receives
native application audio output from general purpose computer 104,
such as audio data that has been generated by a native application,
and converts the audio data into a format for transmission to
mobile device 102 or other suitable mobile devices. Native
application audio output conversion 206 is coordinated with native
application video output conversion 204, so as to provide audio
output that is correlated with video output, allowing video
conferencing, playback of audio-visual data, or other suitable
processes. Likewise, native application audio output conversion 206
can convert the native application audio output into a different
format for hearing at mobile device 102, such as a format that
excludes data that would not provide additional functionality, but
would otherwise require excessive bandwidth requirements for
transfer to mobile device 102.
[0030] In operation, system 200 allows a person using mobile device
102 to remotely provide commands to general purpose computer 104 to
cause native applications or operating system functions to change
state or otherwise perform suitable functions at general purpose
computer 104, while at the same time allowing the person using
mobile device 102 to see and hear the images and sounds generated
by the native applications or operating system functions at general
purpose computer 104.
[0031] FIG. 3 is a diagram of system 300 for providing an audio
command interface in accordance with an exemplary embodiment of the
present invention. System 300 allows voice commands to be provided
to the operating system or native applications operating at general
purpose computer 104.
[0032] System 300 includes voice to command conversion 302,
operating system command system 304, native application command
system 306 and functionality limitation system 308, each of which
can be implemented in hardware, software, or a suitable combination
of hardware and software, which can be one or more software systems
operating on a general purpose processing platform. Voice to
command conversion 302 receives voice data and determines whether
the voice data matches one or more predetermined commands. In one
exemplary embodiment, voice to command conversion 302 can determine
if the voice data is in a proper sequence to be received as a
command, has occurred at a point in time to be interpreted as a
command, and can perform confirmatory functions such as repeating
the command to the person or other suitable functions. In one
exemplary embodiment, voice to command conversion 302 can include
one or more states, such as states based on an operating system
function, a native application, or other function being accessed
and controlled at a present time. In this exemplary embodiment,
when the person using mobile device 102 is accessing and
controlling a certain operating system function or native
application at general purpose computer 104, voice to command
conversion 302 can limit the number of commands to a predetermined
set of commands relevant to the operating system function or native
application. Likewise, voice to command conversion 302 can allow
the person using mobile device 102 to obtain a list of available
commands, such as by stating "list", can allow the person to
request confirmation of commands, such as by requesting "confirm,"
or can perform other suitable functions.
[0033] Operating system command system 304 allows a person to use
voice commands from mobile device 102 to remotely access and
control the operating system at general purpose computer 104. In
one exemplary embodiment, operating system command system 304 can
allow the person to perform predetermined operating system commands
such as a search command, a run command, a program list command, or
other suitable commands. In this exemplary embodiment, the person
can speak a command that causes the operating system to generate a
display of a predetermined number of last programs that were run by
the operating system. Likewise, operating command system 304 can
interact with the person, such as allowing the person to obtain a
list of available operating system commands, allowing the person to
query the operating system to obtain a list of native applications
that are available for operation, or in other suitable manners.
[0034] Native application command system 306 allows a person to use
voice commands from mobile device 102 to remotely access and
control the native applications at general purpose computer 104. In
one exemplary embodiment, native application command system 306 can
include one or more application programming interfaces having a
predetermined set of commands that can be used to operate a native
application. In this exemplary embodiment, native application
command system 306 can allow the person to request a list of
available native application commands, can confirm whether a spoken
command received from the person was meant to be one of two or more
similar native application commands, or can perform other suitable
functions. For example, native application command system 306 can
request the person to repeat a command that was not understood, can
ask the person to choose between one of two similar commands, can
allow the person to navigate backwards a predetermined number of
command steps or reset, can provide other suitable functions.
[0035] Functionality limitation system 308 interacts with network
interface 202 to perform password or other authorization processes,
requiring the person using mobile device 102 to be authenticated
before being allowed access and control of general purpose computer
104 in whole or in part. In one exemplary embodiment, speech or
image data can be received and compared with stored speech or image
data. The speech or image data can also include a plurality of sets
of speech or image data. In this exemplary embodiment, the speech
data can be selected from a set that causes one of a plurality of
predetermined processes to be performed, such as a first phrase or
term that causes a first native application to be executed, a
second phrase or term that causes a second native application to be
executed, and so forth. Likewise, certain terms or phrases can be
used in public to provide limited functionality to native
applications and other terms or phrases can be used in private to
provide access and control of additional native applications or
functions of such native applications.
[0036] In another exemplary embodiment, files of image data showing
different authorized persons or authorized persons in different
states (such as with long hair, short hair, unkempt hair, with or
without facial hair or make-up, or in other states) can also or
alternatively be provided; and an image of the person using mobile
device 102 can be transmitted from mobile device 102 to general
purpose computer 104 to authorize access and control of general
purpose computer 104. The transmitted image data can be compared to
the stored image data and it can be determined whether the data
matches within a predetermined tolerance, such as by using
traditional image data comparison algorithms or processes that
identify a plurality of points or features in the facial images for
comparison. Combinations of audio and video data can also or
alternatively be used for password or other authorization processes
and other suitable functions.
[0037] In operation, system 300 provides an audio command interface
that allows a person to use voice commands from mobile device 102
to access and control native applications or operating system
functions at general purpose computer 104. Audio command interface
108 provides the person with the ability to access and control both
native applications that are configured for direct interaction with
a mobile device as well as native applications that are not
configured for direct interaction with a mobile device, allowing a
person to start applications, to switch between applications, to
shut down applications, or to perform other suitable functions.
System 300 can also limit the access and control the person using
mobile device 102 can have of general purpose computer 104.
[0038] FIG. 4 is a diagram of method 400 for using voice commands
from a mobile device to remotely access and control a general
purpose computer in accordance with an exemplary embodiment of the
present invention. Method 400 begins at 402 where a call is
received from a mobile device. In one exemplary embodiment, the
call can be received over a public switched telephone network, a
wireless network, the internet, or other suitable networks. In
another exemplary embodiment, the call can be encrypted or coded in
other suitable manners. The method then proceeds to 404.
[0039] At 404, a session is established with the mobile device. In
one exemplary embodiment, a session can be established utilizing
mobile device identification and confirmation, utilizing a process
such as a text password and person ID, a spoken or visual password
and person ID where the person using the mobile device transmits
audio or image data of themselves, which is confirmed using
suitable processes at the general purpose processor, or in other
suitable manners. In another exemplary embodiment, the image of the
person using the mobile device can be compared with a plurality of
their images using known image comparison processes to determine
whether a match exists within a predetermined tolerance, allowing
the person using the mobile device to be recognized in different
states, such as with long hair, short hair, unkempt hair, with or
without facial hair or make-up, or in other suitable states. The
method then proceeds to 406.
[0040] At 406, it is determined whether a voice or data command has
been received. If it is determined that a voice or data command has
not been received, after a predetermined time, the method then
proceeds to 408 where a prompt can be provided to the person using
the mobile device to enter a voice or data command, or other
suitable processes are performed. In one exemplary embodiment, the
prompt can confirm that a session has been established with the
mobile device and the general purpose processing platform or other
suitable computer, and can allow or provide the person with
available voice commands. Likewise, other suitable processes can
also or alternatively be performed, such as receipt of data
commands entered by a keypad entry or keyboard entry. If it is
determined at 406 that a voice or data command has been received,
the method then proceeds to 410.
[0041] At 410, the voice or data command is decoded. In one
exemplary embodiment, a voice command can be decoded by determining
the equivalent word for a spoken word, whereas a data command can
be decoded by determining whether predetermined control data
precedes the data that identifies it as a data command such as a
key entry or other suitable data. The method then proceeds to
412.
[0042] At 412, it is determined whether the command is an operating
system command. If it is determined that an operating system
command has been received, the method then proceeds to 414 where
the operating system command is executed. In this exemplary
embodiment, an operating system command can be used to generate
audio and video output data, such as a command to play a movie file
using a video player of the operating system, or other suitable
processes. In another exemplary embodiment, the operating system
command can include a request to list available operating system
commands, such that the operating system command executed at 414 is
selected from a list of available operating system commands. In
another exemplary embodiment, operating system commands such as
"find" or "run" can be utilized in conjunction with spelling
commands, such as where the person spells an alphanumeric file
identifier or program name. The spelled name can then be repeated
back to the person for confirmation, or other suitable processes
can be used. In another exemplary embodiment, a document,
spreadsheet, photographic image, audio recording, video recording,
web page or other suitable data viewer/player can be used to find
and view/play a document, spreadsheet, photographic image, audio
recording, video recording, web page or other suitable data; and
navigation commands such as "back" or "forward" can be used to
navigate through a set of files of documents, spreadsheets,
photographic images, audio recordings, video recordings, web pages,
search results of such data, or other suitable data. The method
then proceeds to 422.
[0043] At 422, the audio output data, video output data, or audio
and video output data, generated at the computer is converted to a
mobile device format. In one exemplary embodiment, the audio output
data and video output data can be converted from a format generated
at the general purpose processing platform to a format compatible
with the mobile device, such as one having a reduced amount of
data, a reduced number of pixels, reduced definition, reduced audio
content, or other suitable formats that are optimized for the
mobile device. In another exemplary embodiment, the audio output
data and video output data can be converted into a format that
excludes data that would not provide additional functionality, but
would otherwise require excessive bandwidth requirements for
transfer to mobile device 102. In another exemplary embodiment, the
audio output data and video output data can be encrypted or coded
in other suitable manners. The method then proceeds to 424.
[0044] At 424, the audio output data, video output data, or audio
and video output data, converted to a mobile device format is
transmitted to the mobile device. In one exemplary embodiment, the
audio output data and video output data are coordinated with each
other, so as to provide audio output that is correlated with video
output, allowing video conferencing, playback of audio-visual data,
or other suitable processes. In another exemplary embodiment, the
audio output data can be transmitted without the video output data
being transmitted. In another exemplary embodiment, the video
output data can be transmitted without the audio output data being
transmitted. In another exemplary embodiment, audio output data and
video output data can be converted into a format that allows data
transmission to the mobile device in accordance with the bandwidth
available at the time of the transmission. The method then proceeds
to 426.
[0045] At 426, it is determined whether a response has been
received from the mobile device. If no response has been received,
the method proceeds to 428 and terminates. Likewise, the person
using the mobile device can be prompted to confirm whether or not
they have received the converted audio output data, the converted
video output data, or whether they are still active. A "ping"
command or other suitable operations can also or alternatively be
performed to determine whether a connection still exists with the
mobile device. If a response is received at 426, the method then
returns to 406.
[0046] If it is determined at 412 that an operating system command
has not been received, the method then proceeds to 416 where it is
determined whether a native application command has been received.
In one exemplary embodiment, a native application command can
include a request to list available native applications, a command
to request a list of available native application commands (such as
from an API), or other suitable native application commands. If it
is determined at 416 that a native application command has been
received, the method then proceeds to 418 where the command is
executed. The method then proceeds to 422. Otherwise, if it is
determined at 416 that the native application command has not been
received, the method proceeds to 420 where an error message is
generated. In one exemplary embodiment, the error message can
include feedback to the person using the mobile device that the
voice or data command received was not recognized as an available
operating system or native application command. The method then
returns to 406.
[0047] In operation, method 400 allows a person using a mobile
device to remotely interact with a computer, whereby the person is
able to command the computer from the mobile device and see and
hear the images and sounds generated at the computer, in response
to the commands received from the mobile device, in a manner
similar to what the person would see and hear, if the person were
directly and locally interacting with the computer.
[0048] Although exemplary embodiments of the system and method of
the present invention have been described in detail herein, those
skilled in the art will also recognize that various substitutions
and modifications can be made to the systems and methods without
departing from the scope and spirit of the appended claims.
* * * * *