U.S. patent application number 11/622934 was filed with the patent office on 2007-10-04 for method and apparatus for delivering a virtual reality environment.
This patent application is currently assigned to THE MISSION CORPORATION. Invention is credited to Maria Gaos.
Application Number | 20070233839 11/622934 |
Document ID | / |
Family ID | 26929213 |
Filed Date | 2007-10-04 |
United States Patent
Application |
20070233839 |
Kind Code |
A1 |
Gaos; Maria |
October 4, 2007 |
METHOD AND APPARATUS FOR DELIVERING A VIRTUAL REALITY
ENVIRONMENT
Abstract
A system is described for providing assistance and guidance to
users of a virtual reality environment that includes at least one
input device for each of the users, at least one client device,
each coupled to a network an operable to respond to a plurality of
commands from each input device issued by the users. A plurality of
servers are provided which are also coupled to the network that are
operable to execute the virtual reality environment and which
include at least one user profile server for storing a user profile
for each of the users. The system also includes an artificial
intelligence engine and at least one output device which is coupled
to each of the plurality of client devices, the artificial
intelligence being operable to invoke and execute at least one
intelligent assistant in the virtual reality environment for each
user.
Inventors: |
Gaos; Maria; (Seattle,
WA) |
Correspondence
Address: |
AXIOS LAW GROUP. PLLC
1525 FOURTH AVE., SUITE 800
SEATTLE
WA
98101
US
|
Assignee: |
THE MISSION CORPORATION
Medina
WA
|
Family ID: |
26929213 |
Appl. No.: |
11/622934 |
Filed: |
January 12, 2007 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
09964224 |
Sep 25, 2001 |
|
|
|
11622934 |
Jan 12, 2007 |
|
|
|
60235777 |
Sep 25, 2000 |
|
|
|
Current U.S.
Class: |
709/223 ;
348/E7.05; 348/E7.07; 386/E5.002 |
Current CPC
Class: |
H04N 5/765 20130101;
H04N 7/106 20130101; H04N 21/4755 20130101; H04N 21/47805 20130101;
H04N 5/775 20130101; H04N 21/42203 20130101; H04N 5/7755 20130101;
H04N 21/4516 20130101; G06Q 30/02 20130101; H04N 21/2543 20130101;
H04N 21/4622 20130101; H04N 21/466 20130101; H04N 21/44222
20130101; H04N 21/4782 20130101; H04N 21/454 20130101; H04N
21/25891 20130101; H04N 7/17309 20130101 |
Class at
Publication: |
709/223 |
International
Class: |
G06F 15/173 20060101
G06F015/173 |
Claims
1-4. (canceled)
5. A system operable for providing assistance and guidance to users
of a virtual reality environment, the system comprising: at least
one input device for each of the users, each of the input devices
operable to generate a plurality of commands; at least one client
device coupled to a network and operable to respond to the
plurality of commands from each of the at least one input devices;
a plurality of servers coupled to the network and operable to
execute the virtual reality environment, the plurality of servers
including at least one user profile server operable to store a user
profile in a memory for each of the users; an artificial
intelligence engine operable to invoke and execute at least one
intelligent assistant in the virtual reality environment for each
of the users; and at least one output device coupled to each of the
plurality of client devices.
6. The system of claim 5 wherein the at least one input device
includes at least one of a television, telephone, personal
computer, voice remote controller, a pointing device, a keyboard, a
mouse and a microphone.
7. The system of claim 5 wherein the at least one client device is
a set-top box.
8. The system of claim 7 wherein the set-top box includes the
memory.
9. The system of claim 5 wherein the artificial intelligence engine
is comprised of at least one of a speech recognition module, a
natural language understanding module, a speech synthesis module, a
digital video and audio processing module, a data mining module, a
knowledgebase and an expert system.
10. The system of claim 5 wherein the at least one client device is
at least one of a television, a personal computer, a DVD and a VCR,
each including an electronic means for responding to the plurality
of commands from each of the at least one input devices.
11. The system of claim 10 wherein the at least one client device
includes the memory.
12. The system of claim 5 wherein the at least one intelligent
assistant is invoked and executed to monitor each user's activity,
to gather data for storage in the user profile, to analyze a
history of selections and behaviors for each user based the data
gathered for storage in the user profile and to recommend at least
one of information data, product data and service data for the user
based on the data stored in the user profile.
13. The system of claim 5 wherein the at least one output device is
a television.
14. The system of claim 5 wherein the at least one output device is
a telephone.
15. The system of claim 5 wherein the at least one output device
includes at least one of a computer monitor, a mobile device
monitor and a printer.
16. The system of claim 9 wherein the speech synthesis module is
operable to generate a synthesized speech content on the at least
one output device in response to the plurality of commands from the
at least one input device.
17. The system of claim 5 wherein the network is the Internet.
18. The system of claim 5 wherein the intelligent assistant is a
human-like avatar having an appearance as a virtual person in the
virtual reality environment.
19. The system of claim 5 wherein the intelligent assistant is an
animated character having a plurality of human-like features.
20. The system of claim 5 wherein the virtual reality environment
is comprised of a plurality of content areas organized by topical
category.
21. The system of claim 5 wherein the virtual reality environment
is comprised of a plurality of content areas organized by market
segment.
22. The system of claim 5 the plurality of servers includes a game
application server operable to store one or more games for users of
the virtual reality environment.
23. The system of claim 21 wherein the game application server is
operable to download and store the one or more games to the at
least one client device in response to the plurality of commands
from each of the at least one input devices.
24. The system of claim 5 wherein the virtual reality environment
is operable to provide a plurality of services to each of the users
on an "on-demand" basis.
25. The system of claim 5 wherein the virtual reality environment
is operable to provide a plurality of services to a group of the
users on an "on-demand" basis.
26. A method of assisting and guiding users in a virtual reality
environment, the method comprising: monitoring for requests from
each of the users of the virtual reality environment, the virtual
reality environment including an artificial intelligence engine;
processing the monitored requests using the artificial intelligence
engine, the processing based in part on data stored in a user
profile for each of the users; retrieving data in response to the
processed requests using the artificial intelligence engine;
updating the user profile and the virtual reality environment based
on the processed requests and the retrieved data; and delivering
the retrieved data to each of the users.
27. The method of claim 26 wherein the virtual reality environment
includes a plurality of multimedia authoring and editing
products.
28. The method of claim 26 wherein the artificial intelligence
engine retrieves data in response to the processed requests based
on at least one of a semantic profile for each user and a cognitive
profile for each user.
29. The method of claim 26 wherein the virtual reality environment
learns, evolves and adapts to each of the users in response to the
monitored requests.
30. The method of claim 26 wherein the virtual reality environment
includes an intelligent assistant, the intelligent assistant using
the artificial intelligence engine to: monitor the requests;
determine at least one preference for each of the users from an
evaluation of a history of past requests for each of the users; and
offer at least one recommendation for each of the users based on
the monitoring of the requests and the determining of at least one
preference from the evaluation of the history of past requests for
each of the users.
31. The method of claim 30 wherein the intelligent assistant
monitors the requests from each of the users to determine whether a
response based on the retrieved data is to be delivered to each
user individually as an individual response or to a group of users
as a group response.
32. The method of claim 26 wherein the artificial intelligence
engine is comprised of at least one of a speech recognition module,
a natural language understanding module, a speech synthesis module,
a digital video and audio processing module, a data mining module,
a knowledgebase and an expert system.
33. The method of claim 30 wherein the history of past requests is
stored in the user profile by the intelligent agent.
34. The method of claim 33 wherein the user profile is continually
updated by the intelligent assistant from storing the monitored
requests.
35. The method of claim 30 wherein the intelligent assistant is a
human-like avatar having an appearance as a virtual person in the
virtual reality environment.
36. The method of claim 30 wherein the intelligent assistant is an
animated character having a plurality of human-like features.
37. In a system for execution of a virtual reality environment
including a plurality of client devices, each client device having
a voice-controlled interface, a method for providing assistance and
guidance to users of the virtual reality environment comprising:
monitoring for requests from the users of the virtual reality
environment on the voice-controlled interface of each client
device, the virtual reality environment including an artificial
intelligence engine; processing the monitored requests using the
artificial intelligence engine, the processing based in part on
data stored in a user profile for each of the users; presenting on
the voice-controlled interface a voice-generated output in response
to the processed requests; and updating the user profile and the
virtual reality environment based on the processed requests and the
voice-generated output.
38. The method of claim 37 wherein each client device includes a
voice-activated input device for receiving the requests from the
users.
39. The method of claim 37 wherein the voice-generated output is a
voice-generated catalog of items available in the virtual reality
environment.
40. The method of claim 37 wherein the artificial intelligence
engine retrieves data in response to the processed requests based
on at least one of a semantic profile for each user and a cognitive
profile for each user.
41. The method of claim 37 wherein the virtual reality environment
learns, evolves and adapts to each of the users in response to the
monitored requests.
42. The method of claim 37 wherein the virtual reality environment
includes an intelligent assistant, the intelligent assistant using
the artificial intelligence engine to: monitor the requests;
determine at least one preference for each of the users from an
evaluation of a history of past requests for each of the users; and
offer at least one recommendation for each of the users based on
the monitoring of the requests and the determining of at least one
preference from the evaluation of the history of past requests for
each of the users.
43. The method of claim 42 wherein the intelligent assistant
monitors the requests from each of the users to determine whether
the voice-generated output is to be delivered to each user
individually as an individual response or to a group of users as a
group response.
44. The method of claim 37 wherein the artificial intelligence
engine is comprised of at least one of a speech recognition module,
a natural language understanding module, a speech synthesis module,
a digital video and audio processing module, a data mining module,
a knowledgebase and an expert system.
45. The method of claim 42 wherein the history of past requests is
stored in the user profile by the intelligent agent.
46. The method of claim 45 wherein the user profile is continually
updated by the intelligent assistant from storing the monitored
requests.
47. A computer-readable medium having computer-executable
instructions for performing a method comprising: monitoring for
requests from users of a virtual reality environment, the virtual
reality environment including an artificial intelligence engine;
processing the monitored requests using the artificial intelligence
engine, the processing based in part on data stored in a user
profile for each of the users; retrieving data in response to the
processed requests using the artificial intelligence engine;
updating the user profile and the virtual reality environment based
on the processed requests and the retrieved data; and delivering
the retrieved data to each of the users.
48. The computer-readable medium of claim 47 wherein the virtual
reality environment includes a plurality of multimedia authoring
and editing products.
49. The computer-readable medium of claim 47 wherein the artificial
intelligence engine retrieves data in response to the processed
requests based on at least one of a semantic profile for each user
and a cognitive profile for each user.
50. The computer-readable medium of claim 47 wherein the virtual
reality environment learns, evolves and adapts to each of the users
in response to the monitored requests.
51. The computer-readable medium of claim 47 wherein the virtual
reality environment includes an intelligent assistant, the
intelligent assistant using the artificial intelligence engine to:
monitor the requests; determine at least one preference for each of
the users from an evaluation of a history of past requests for each
of the users; and offer at least one recommendation for each of the
users based on the monitoring of the requests and the determining
of at least one preference from the evaluation of the history of
past requests for each of the users.
52. The computer-readable medium of claim 51 wherein the
intelligent assistant monitors the requests from each of the users
to determine whether a response based on the retrieved data is to
be delivered to each user individually as an individual response or
to a group of users as a group response.
53. The computer-readable medium of claim 47 wherein the artificial
intelligence engine is comprised of at least one of a speech
recognition module, a natural language understanding module, a
speech synthesis module, a digital video and audio processing
module, a data mining module, a knowledgebase and an expert
system.
54. The computer-readable medium of claim 51 wherein the history of
past requests is stored in the user profile by the intelligent
agent.
55. The computer-readable medium of claim 54 wherein the user
profile is continually updated by the intelligent assistant from
storing the monitored requests.
56. The computer-readable medium of claim 51 wherein the
intelligent assistant is a human-like avatar having an appearance
as a virtual person in the virtual reality environment.
57. The computer-readable medium of claim 51 wherein the
intelligent assistant is an animated character having a plurality
of human-like features.
Description
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is based upon and claims the benefit of
priority from U.S. patent application Ser. No. 09/964,224, filed
Sep. 25, 2001, and Provisional Application No. 60/235,777, filed
Sep. 25, 2000, the entire contents of which are incorporated herein
by reference.
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND
DEVELOPMENT
[0002] Not applicable.
BACKGROUND OF THE INVENTION
[0003] 1. Field of the Invention
[0004] This invention relates to global communications networks,
and more specifically to delivering an easy to use virtual reality
environment to take advantage of existing technologies and global
communications networks such as the Internet without requiring the
use of computers nor computer literacy. This invention includes an
artificial intelligence assistant which receives commands through a
speech recognition device, adapts to each individual user, and
provides customized guidance to the user. The intelligent assistant
projects human-like features and behaviors appropriate to the
preferences of its user and appears as a virtual person to the
user.
[0005] 2. Discussion of the Background
[0006] In 1996, industry analysts propounded that, with 40 million
people connected around the world, $1 billion in electronic sales,
and over 10 million web addresses, the Internet as a commercial
medium had come of age; 25 years after its birth under the auspices
of the United States Department of Defenses Advanced Research
Project Agency as a small network (ARPANET) intended to promote the
sharing of super-computers among researchers in the United
States.
[0007] Five years after the experts accepted the Internet as a
commercial medium, the Internet today is a chaotic, overpopulated
road, accessible only to the relatively few people who own and can
operate computers. The virtual environment object of this invention
provides a means to bypass the computer dependency for accessing
today's Internet.
[0008] Today's Internet is generally deficient in several ways.
[0009] First, it is very generalized in the information it
provides. The Web pages are not designed for a particular person,
but rather, in the best of cases, for a particular group. They do
not rely on the personalized characteristics, needs or criteria of
the specific user in order to narrow the choices of selection to be
offered to the user. This requires excess time on the part of the
user to browse through the site in order to make the proper
selection. [0010] With existing search engines and query tools,
users must interact with the Internet using a limited set of
preestablished interactions. In addition, all users must navigate
through the pre-set paths and iterations even when the bulk of the
information accessible via those paths would not be applicable to
that user's needs nor to the user's level of familiarity with the
available information. This shortcoming is particularly significant
when one considers provisions of product information over the
Internet. With worldwide access, customers from geographically
diverse locations will have different requirements on available
product sets when interacting with the same Web site. [0011]
Different users will also have different interaction parameters
with respect to the level of product detail that they require. For
instance, when shopping for a camera, a professional photographer
will approach the interaction with more detailed specification
constraints than the user who is shopping for a camera as a
gift.
[0012] As a result of the foregoing shortcomings, the Internet has
modest penetration levels for a mass medium. ACNielsen Netwatch's
October 1999 survey shows a penetration rate of 38% in Canada with
the United States sharing second place with Australia 15 and
Singapore at 25% of the total population.
[0013] Nonetheless, the Internet has the potential to become a
significant global mass medium for conducting business, collecting,
and exchanging information, communication, and entertainment.
International Data Corporation ("IDC") estimates that the number of
Internet users worldwide will grow from approximately 142.2 million
at the end of 1998 to approximately 502.4 million by the end of
2003, representing a compound annual growth rate of 29%.
[0014] As Internet usage continues to grow, advertisers and 25
electronic commerce marketers are increasingly using the Web to
locate customers, advertise and facilitate transactions. The
Internet allows advertisers to more precisely target desired
audiences while tracking impression levels, user demographics, and
advertisement effectiveness. In addition, the Internet allows
advertisers to interact more effectively with users and to more
easily obtain relevant data about buying patterns, preferences and
demands.
[0015] According to Forrester Research, the total worldwide dollar
value of Internet advertising will increase from $3.3 billion in
1999 to $24.1 billion by 2003. Moreover, according to the IDC, the
percentage of Internet users buying goods and services on the
Internet will increase from approximately 22% in 1998 to
approximately 36% in 2003, and the total value of goods and
services purchased directly on the Web will increase from
approximately $27.0 billion in 1998 to approximately $842.7 billion
by 2003.
[0016] One of the factors slowing the wider deployment of the
Internet is that it requires an expertise level which is still
quite high. Today's computers are difficult to use and require
significant levels of expertise, generally built over many years of
computer use. Also, non-computer users are quite often intimidated
by the technology of computers. Current Internet products require
that their users have a certain level of familiarity with a
computer, the medium, its language, and its 20 unspoken rules. This
trend is becoming more entrenched with each new advance in
technology, since technology is being used to substitute for human
interaction between the Web site and its users.
[0017] This combination of factors results in more than half of the
25 world population bewildered, uninterested in using and enjoying
one of the best technologies developed in the twentieth century.
The general population is being excluded from the Internet because
it requires them to understand computers and learn computer skills
and technological language. Moreover, to use the Internet, users
have to pay more for a personal computer, Internet connections and
related requirements, than for a television set. In addition, there
is a natural, human fear of failure when challenged to master new
skills. This group excluded from the Internet represents every
segment of the general population--men and women, young and old,
professionals and non-professionals.
[0018] The Internet is a large network made up of a number of
smaller networks. It is made up of more than 100,000 interconnected
networks in over 100 countries, comprised of commercial, academic
and government networks. It has become commercialized into a
worldwide information highway and data base, containing information
on every subject known to humankind.
[0019] Thus, the Internet and its World Wide Web ("WWW" or "Web")
system have become an important means of providing access for many
users to vast amounts of information stored on the servers which
make up the Internet. By way of background, the WWW is an Internet
facility that links documents locally and remotely. The Web
document is called a Web page, and links in the page let users jump
from one page to another page (hyperlink) whether the pages are
stored on the same server or on servers around the world. The pages
are accessed and read via a Web browser such as Netscape Navigator
or Microsoft Internet Explorer.
[0020] The WWW has become the center of Internet activity since,
among other reasons, Web pages, containing text, graphics, audio
and video are easily accessible via a standard web browser. The WWW
contains the largest collection of online information in the world,
and the amount of information is increasing. Current schemes for
accessing a Web document require typing in the URL (Uniform
Resource Locator) address of the home page in the Web browser. From
there, the user starts surfing through the Internet via hypertext
links to other documents that can be stored on the same server or
on a server anywhere in the world.
[0021] The fundamental Web format is a text document embedded with
"HTML" tags that provide the formatting of the page as well as the
hypertext links (URLs) to other pages. HTML codes are common
alphanumeric characters that can be typed with any text editor or
word processor. Numerous Web publishing programs provide a
graphical interface for Web page creation and automatically
generate the codes. Many word processors and publishing programs
also export their existing documents to the HTML format. Thus, Web
pages can be created by users without learning any voluminous
coding system. The ease of page creation has helped fuel the Web's
growth. Web pages are maintained at Web sites, which are computers
or nodes that support the Web's HTTP protocol. The HTTP (Hyper Text
Transport Protocol) is a communications protocol used to connect
clients (for example, personal computers) to Web sites.
[0022] To date, the majority of users are people with technical or
professional backgrounds who use expensive personal computers or
work stations to access the Internet and the WWW. In order to
increase usage and acceptance of the Internet and WWW by the
public, it is necessary to provide systems which are less expensive
and easier to use than personal computers or the existing Internet
access devices.
[0023] The proper and efficient use of the great amount of
information and services available on the Internet has the
potential of providing Internet users with an environment in which
to attend to all their needs and wants. In particular, those
non-technical users interested in certain segments of the
information and services available on the Internet could benefit
tremendously from having their specific information or services of
interest available to them without much of a searching effort.
Moreover, such users would benefit greatly from being constantly
and automatically updated on new information and services as they
become available.
[0024] The sheer size of the information available on the Internet
and the Web has made it a necessity for users to efficiently and
constantly sift through the available information in order to find
and organize the information that is of interest to them. Stated
differently, users realize that the availability of information
itself is not useful unless the information is reduced to a
manageable size and unless the information so reduced in size is of
interest and of value to the specific user. According to IDC, the
number of web pages will increase from approximately 925 million at
the end of 1998 to over 13 billion by the end of 2003, representing
a compound annual growth rate of "Jo.
[0025] To date, the following strategies have been developed to
assist Internet users:
[0026] Internet Directories. Internet directories generally list
web sites by specific topics of interest and contain links to these
sites. With the rapid growth of content available on the Internet,
these directories are becoming increasingly difficult to build and
update with a high level of quality. In addition, the creators of
these directories often have limited relevant knowledge about the
directories' particular topic areas.
[0027] Internet Search Engines. Internet search engines capture,
store and index web site information in order to retrieve web site
listings in response to a user query. These software programs have
a limited ability to determine the quality or relevance of the web
sites they retrieve. Further, as the nature of the available
content has become more difficult to classify, many companies
offering search engines are being forced to employ significant
editorial staffs to ensure that responses to queries are
satisfactory. Search engines based on natural language have the
added difficulty of accurately determining sentence syntax and
nuances.
[0028] Internet Portals. Many traditional Internet directories and
search engines have added interactive tools and features, such as
weather, news feeds, stock portfolios! personal home pages, and
non-proprietary content from other media. These portals provide
Internet users with basic information and links on a broad range of
topics. Because most portals have many similar tools and features
and provide limited original content, there is little
differentiation among them.
[0029] Topic-Specific Sites. More in-depth content is generally
found on sites focused on particular topics where individuals
knowledgeable about those topics create the content. Because these
sites tend to be widely dispersed, Internet users may need to visit
a number of other unaffiliated sites in order to satisfy their
information needs.
[0030] User-Generated Content Sites. Many web sites generate and
contribute relevant content. However, the quality of this content
varies significantly and the content is frequently un-managed and
poorly organized and presented. As a result, advertisers may be
reluctant to entrust their brands to sites that rely upon
user-generated content.
[0031] Various "search engines", such as Yahoo, Excite, Lycos, and
5 AltaVista, among others, are known that permit a user to search
for specific information. However, such search engines must be used
on a daily basis to ensure that the updated information is
accessed. Moreover, almost any search string results in a huge
number of documents and the user would not know which documents
have already been ruled out as being of no interest. Further, the
search engines generally do not focus on particular sites that are
of interest to the user; they search the entire Internet. The
Internet is huge and complex. This is especially daunting to
non-technical users who are unschooled in the vagaries of search
techniques. The virtual environment object of this invention
develops a better, more powerful and intuitive search engine based
on semantic and cognitive profiling to allow for improved
information retrieval on the Internet by nontechnical users. The
same Artificial Intelligence driven semantic engine that determines
the meaning of user requests can be turned around and used to
interpret the validity of query results that answer requests.
Additionally data structures that support and store user oriented
semantic relationships provide a readily accessible "web of
knowledge".
[0032] Since commerce is an inherent activity of human beings and
an obvious part of life as we know it, it behooves us to develop a
largely friction free mass commerce potential to get buyers and
sellers together in a productive and mutually fulfilling
experience. In order to do this, ways must be developed to reduce
or remove the real or perceived barriers to commerce on the buyer's
side such as high pricing, return hassles, credit card concerns,
privacy protection, navigational difficulties and delayed
gratification. On the sellers' side ways of dealing effectively
with payment matters, the cost of customer service, inventory
control costs, including the cost of goods sold and a full range of
other business to customer and business-to-business concerns need
to be adopted. In addition, strong models to help businesses and
consumers deal effectively with business management issues that add
to supplier costs must be developed.
[0033] None of the devices or methods available in the prior art,
however, provides a way of integrating all available technologies
together in order to provide an user friendly virtual environment
which provides the user with an easy to use technological solution
while at the same time taking advantage several available
technologies. Thus, there is a need for a virtual environment and
method of creating said environment which overcomes these and other
deficiencies in the prior art.
SUMMARY OF THE INVENTION
[0034] It is therefore an object of the present invention to
fulfill the real potential of the Internet as a mass medium in a
Post-PC Environment that takes control of its users' experience
before, during, and after their initial connection by meeting and
anticipating their individual information, services, entertainment,
and transactional needs in real life scenarios through technology
that recreates everyday situations.
[0035] This invention will dynamically create a virtual reality
environment based on an user's individualized personality,
interests or needs using an artificial intelligence engine's
experience with the individual user of the invention.
[0036] It is a further object of this invention to humanize
information technology by making the technology truly easy to use,
accessible, affordable, and relevant in people's everyday lives
while bypassing the requirements of computer literacy and
ownership, deliver entertainment, current, and relevant
information, and useful services and merchandise on demand in a
manner that accurately reflects each user's reality at many levels.
The virtual environment of the present invention, respects users
desire for privacy, as the user only interacts with a set-top box
and not with retailers or service providers. The environment
interacts with and responds to users both individually and as a
group, learns, evolves and changes quickly in response to user
demands and preferences.
[0037] It is another object of the present invention to provide
simplified non-technical, and genuinely user friendly multimedia
authoring and editing tools tailored to deliver content that
effectively utilizes the virtual environment's front end and
supports a large array of content input sources such as subject
matter experts, free lance writers, staff writer/reporters,
graphic, video and audio artists.
[0038] The virtual environment is to engage users from all walks of
life by using their chosen natural, non-technical language at
personal and individual level. To accomplish this the environment
will use advanced Artificial Intelligence driven semantic
processing, voice recognition, and digital and audio technology to
create a unique environment that provides verbal, natural language,
intuitive and friendly navigational aides and online/of f line
human quality support.
[0039] The virtual environment of this invention is intended to be
a global, interactive "mass medium." The environment provides
appropriate user services "on demand" not following that ubiquitous
tendency for technology to develop one size fits all
approaches.
[0040] Every component in the virtual environment of this invention
has the capability to dynamically learn, change, and respond
quickly according to individual needs and desires by building dense
user profiles. At the same time, user profiles should be absolutely
secure and guarded against inappropriate access or use in
conformance to user desires and the varying laws on privacy from
jurisdiction to jurisdiction. This is an all-important feature. Not
only is the security of profile information required by law in an
increasing number of jurisdictions, but the lack of trust by users
on how their profiles are handled is a primary reason vast numbers
of potential users shun online activities. The virtual environment
of this invention guarantees privacy protection.
[0041] A primary feature of the virtual environment of this
invention is that it should, to the extent possible, make the
technology disappear. Currently, users must directly deal with
three artifacts of technology-operating systems, user interfaces
and browsers-before they even get to deal with applications that
could possibly affect their lives. Today's operating system/user
interface/browser combinations are not truly intuitive nor user
friendly. This invention provides a new non-technical and natural
integrated user interface/browser based on a totally transparent
and stable real time operating system. This innovative "front end"
is housed in a set-top box, which provides interaction and access
using the user's television and telephone. In other embodiments of
this invention this front end may be modularly constructed to
reside in any other user devices that provide appropriate access to
the virtual environment.
[0042] The system of this invention quickly goes online when
commanded without additional user interaction. The user does not
need to be involved in any operating system derived technical setup
routines. In this invention there is no chance for the loss of user
information and the user is continually apprized of the progress in
real time of all submitted requests.
[0043] The environment of this invention provides intelligent,
efficient and non-patronizing means to deliver marketing,
promotion, and merchandising services as a natural and unobtrusive
part of the users online experience. There is a tendency for the
online world to become over infatuated with mercantile e-commerce
as the primary driving force in the online experience. In life as
we know it, shopping and entertainment are not the only thing that
we do. We need to work productively to be able to shop and be
entertained as well as fulfill our sense of accomplishment. We seek
communications and relationships with the community at large in
order to share a common experience and social bonding. People over
time will not become loyal, persistent users of a medium that just
tries to sell them something all the time. The environment of this
invention teases out of the user experience strong models of local
community development that can be used to help people actually lead
what they perceive as better and more fulfilling lives. Among those
models are ones to facilitate high quality, user oriented
healthcare, expand the quality and availability of education at all
levels, help local people make a living thereby expanding local
community options, support community activism and participation and
increasing the general level of understanding between individuals
and communities. By facilitating these types of activities, the
environment of this invention will support electronic commerce as a
necessary by product.
[0044] A further object of this invention is to attract and satisfy
the vast majority of the population that does not use the Internet.
Conventional Internet companies set-up shop and wait in the
sidelines hoping to capture a share of the "inevitable" market
explosion. This invention develops a mass medium by pulling the
Internet outside of computers and into television sets, and
inviting people to use it as a medium that leads them to a friendly
and human virtual environment that delivers all of the services
that the users desire and need.
[0045] This invention tackles all of the aforementioned issues by
providing a full service affordable package to enable those who
previously were daunted by the perceived high financial and
intellectual costs of joining the Internet Age. The technological
dominance of the Internet industry has led to a critical
miscalculation--the belief that the consumer must adapt to
technology This obvious mistake permeates the thinking of current
Internet companies, including the most successful. This invention
fills this void by serving the consumer by technology, not vice
versa, by using and adapting those technologies which provide a
satisfying and value-adding experience to its users who do not have
to adapt to the invention.
[0046] A further object of this invention is to offer and create
individual virtual environments for each user in order to meet
their specific needs and wants. A user's nationality, language
preference and gender are only a start. The virtual environment
will meet the needs of the user in ease of use, richness of content
offerings, attractiveness in e-commerce availability, and any other
unmet need the user might express.
[0047] The virtual environment of the present invention also
provides simple processes to conduct electronic commercial and
non-commercial transactions and continued customer support before,
during, and after each transaction.
[0048] In the Post-PC Era, while there is a relatively small
universe of households with computers and Internet access, 95% of
households around the world own television sets. Thus, most people
may be connected instantly to the virtual environment object of
this invention, where the mouse, the keyboard and the PC monitor
are replaced by their television and telephone sets.
[0049] Connecting to the virtual environment does not require a PC.
The user only has to install the set-top box and turn it "On" to
enter the virtual environment. The direct connection between the
user and the virtual environment is totally seamless as it bypasses
the confusion of learning "to surf." Users are directed to an "all
inclusive" virtual environment as their first point of contact.
Navigation and communication with the virtual environment is
through humanized mechanisms, such as speech recognition.
[0050] The virtual environment object of this invention will relate
to its users as individuals in real, human, everyday language,
delivered through two familiar household items--the television and
the telephone. This will revolutionize today's Internet and
television, bypassing computers, to deliver information,
entertainment and commerce to its users interactively and on a
one-to-one basis, realizing for the first time the all-inclusive
potential of the new medium.
[0051] Recent market research, such as the study conducted in 1999
by Roper Starch Worldwide, reveals that consumers are willing to
rely on the new medium to satisfy a growing range of their
communications, information, entertainment, and business needs, and
consider it an important part of their lives. According to the
study, conducted via telephone in July 1999, among a random sample
of approximately 1,000 home Internet subscribers over 18 years of
age, electronic commerce is the fastest growing online activity,
with a 10% jump over the previous year. The Roper Study also
illustrates the limited vision of today's Internet--one question
asked of the respondents was "If you are stranded on a desert
island, would you prefer to have a television and telephone or a PC
with an Internet connection?" The respondents, presented this false
choice, selected an interactive connection. However, the
all-inclusive environment of this invention obliterates this false
dilemma--through a set-top box and environment which provides its
users with the interactivity they crave through household devices
they are familiar with, bypassing the perceived PC gauntlet.
[0052] It is also an object of the present invention to offer
highly-competitive pricing for all merchandise sold within the
environment by obtaining volume discount from retailers,
wholesalers, and manufacturers. By creating an environment that
removes the mass market's perceived barriers to electronic
commerce, a high-volume consumer audience that will be attractive
to vendors will be created. Vendors will seek to sell their goods
and services in the virtual environment. Commercial affiliates and
service providers may take orders from users of the virtual
environment, which will be executed by the operator of the
environment. Through volume, said operator will be able to
negotiate favorable financial terms for its members.
[0053] The technological infrastructure will learn from customer
interaction and immediately alert a customer service department of
problem areas and provide performance reports of content of the
virtual environment. Twenty four hour online and off line customer
service and support may facilitate all aspects of the customer
experience, from Internet connection to electronic commerce.
[0054] Video, audio and Speech-Recognition components interact with
users of the environment in a human, personal and individual
manner.
[0055] An important object of this invention is to provide the user
with a virtual intelligent assistant which adapts to each user and
offers a high quality of individualized user interaction.
[0056] Because current Internet products require that their users
have a certain level of familiarity with the medium, its language
and its unspoken rules, they all cater to the same audience while
excluding new users. It is an object of the present invention to
fill this vacuum, to entice and maintain non Internet users with a
virtual environment that delivers content that satisfies the
following requirements: [0057] Presented in an intelligent,
imaginative, attractive manner using real human language. [0058]
Edited for accuracy, timeliness, and completeness. [0059] Filtered
and edited by an Artificial Intelligence technology according to
each customer's individual needs and preferences before it is
published in the virtual environment. [0060] Organized and
catalogued by an Artificial Intelligence technology in order to
present the least possible number of relevant choices to each user.
[0061] Where marketing and promotions are introduced in an
intelligent, non patronizing way as a natural part of each user's
experience, through recommendations, suggestions and other
means.
[0062] The virtual environment of this invention appeals to the
general population because it relies on a personal human and
non-technological relationship to satisfy individual needs and
preferences.
[0063] An embodiment of the present invention provides a PostPC era
revenue model that eschews the current Internet advertising sales
strategies by offering retailers and Internet service providers the
opportunity to sell their products to a mass audience in a virtual
shopping zone.
[0064] Another object of the present invention is to maximize
marketing revenue while providing users a clean, uncluttered
content.
[0065] Still another object of the present invention is to offer an
attractive approach which would increase the size of the audience
and the length of their visits. Thus, increasing the traffic in the
virtual shopping zone allowing the host to set higher prices for
the retail space of the Shopping Zone and realizing higher sales
fees.
[0066] Offer retailers, manufactures, and service providers an
opportunity to create their own merchandising and marketing
presence within a virtual shopping environment, charging such
companies a fee for the preparation of their marketing area, with
cost dependent upon the complexity of the area and the prominence
of its location within the environment.
[0067] Retail space may be leased at a monthly fee.
[0068] Long-term contracts may be designed to support broad
marketing objectives, including brand promotion, awareness, product
introductions, online research and the integration of marketing
messages with editorial content.
[0069] The virtual environment itself, both as to its construction
and its mode of operation, will be best understood, and additional
objects and advantages thereof will become apparent, by the
following detailed description of a preferred embodiment taken in
conjunction with the accompanying drawings.
[0070] When the word "invention" is used in this specification, the
word "invention" includes "inventions", that is, the plural of
"invention". By stating "invention", the Applicant does not in any
way admit that the present application does not include more than
one patentably and non-obviously distinct invention, and Applicant
maintains that the present application may include more than one
patentably and non-obviously distinct invention. The Applicant
hereby asserts, that the disclosure of the present application may
include more than one invention, and, in the event that there is
more than one invention, that these inventions may be patentable
and non-obvious one with respect to the other.
BRIEF DESCRIPTION OF THE DRAWINGS
[0071] FIG. 1, is a block diagram of the set-top box, remote
control, and decryption box interface.
[0072] FIG. 2, is a block diagram of the interface between the
set-top box, the remote control, and the input and output
devices.
[0073] FIG. 3, is a block diagram of the verbal communication
devices interface.
[0074] FIG. 4, is a block diagram of the interaction of all the
components of the virtual environment of this invention.
[0075] FIG. 1, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0076] FIG. 2, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0077] FIG. 3, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0078] FIG. 4, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0079] FIG. 5, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0080] FIG. 6, is a block diagram of an embodiment of the set-top
box, VCR, and cable decryption box interface.
[0081] FIG. 7, is a block diagram for an Artificial Intelligence
System.
[0082] FIG. 8, is a block diagram for the Intelligence Center of
the Invention.
[0083] FIG. 9, is a block diagram of all likely features of the
set-top box.
[0084] FIG. 10, is a block diagram of a suggested assignment of
functions to JDTMF keypad.
[0085] FIG. 11, is a block diagram of all likely features of the 30
set-top box.
[0086] FIG. 12, is a block diagram of all likely features of the
set-top box.
[0087] FIG. 13, is a block diagram of the preferred configuration
of the set-top box.
[0088] FIG. 14, is a block diagram of the voice remote control
operation.
[0089] FIG. 15, is a block diagram of the preferred embodiment of
the voice remote control operation.
[0090] FIG. 16, is a block diagram of the Verbal Communication
Interface Program.
[0091] FIG. 17, is a block diagram of the Remote Control and Set
top box interface.
[0092] FIG. 18, is a block diagram of the Set top Device.
DESCRIPTION OF THE PREFERRED EMBODIMENT
[0093] The following detailed description sets forth numerous
specific details to provide a thorough understanding of the
invention. However, those of ordinary skill in the art will
appreciate that the invention may be practiced without these
specific details. In other instances, well-known methods,
procedures, protocols, components, algorithms, and circuits have
not been described in detail so as not to obscure the
invention.
I. The Virtual Reality Environment
[0094] By incorporating new applications of Artificial
Intelligence, speech recognition, and digital video and audio
technologies, this invention creates a unique virtual environment
that provides verbal, natural language, intuitive, and friendly
navigational aides and online/of f line human support to deliver
the most relevant, useful and current information and entertainment
options, and the most efficient means to conduct the widest
possible range of commercial and non-commercial transactions--from
shopping to banking and filing official documents and every other
of the users lifestyle needs. This virtual environment may engage
consumers from all walks of life in real, human, non-technological
language at the most personal and individual levels. Every
component in the virtual environment learns, responds, and changes
quickly according to the needs of each individual user, interacts
with users through video and audio technologies in natural
language, and is designed for smooth and fast scalability.
[0095] The virtual environment of this invention encompasses three
separate elements: a set-top box FIG. 11 and a voice equipped
remote controller, FIG. 11, 1101, which provide a very friendly
user interface to the system, a TV and or PC, and a group of
computer servers as the back-end to support all services available
from the set-top box. These services may include, among others,
home banking, e-commerce, e-services, education, entertainment,
voice over IP, and video over IP. In addition, hardware and
software infrastructure is required for supporting the above
components, e.g. connectivity means (telephone lines, cables, DSL,
satellite, etc.) and Application Service Provider (ASP)
[0096] The virtual environment of this invention takes the Internet
out of the PC environment and into the users television sets,
through a small and easy-to-use set-top box. The set-top box uses
televisions and telephone lines to instantly connect users with the
environment. Installation and connection of these devices is a
quick and simple process.
[0097] In addition, in another embodiment of the invention, circuit
boards, which serve the function of the set-top boxes, may be
incorporated in television sets. The users of the present invention
may access the aforementioned services via a natural and friendly
form of human communication with the familiar medium of television,
and traditional remote controls. In this way, people without
computer knowledge will be able to "surf" the Internet and enjoy
the vast array of services offered by current advanced technologies
without encountering any technical barriers.
[0098] The set-top box provides the user with a satisfying and
simple to understand experience by transparently taking care of the
technical details of the communications interface with television
sets and telephone lines to instantly connect people to the virtual
environment and allowing familiar paradigms and natural-feeling
controls to replace the arcane, un-intuitive, and awkward ones used
by other Internet products. In another embodiment of the invention,
circuit boards are pre-installed on PC's. Still in other
embodiments of the invention, circuit boards are pre-installed in
DVDs and VCRs.
[0099] The remote controller extends the functionality of the
system by allowing the user familiar with TV remote-controls and
wireless telephones to engage in Internet activities in an
intuitive way, and without learning a daunting amount of new jargon
and methods.
[0100] Through the use of the present invention, the Internet in
the Post-PC Era will be a vehicle that transports people to
destinations that enhance their lives and entertain them in ways
that existing media cannot. The present invention enables people to
access the Internet without the need of computers and/or computer
knowledge. Thus, enabling them to use the Internet for what it
truly is--a medium that connects people with the virtual
environment object of this invention. This virtual environment
empowers the users with true choices and one-on-one interactivity,
beyond the limitations of time, space and programming inherent to
both traditional media and today's Internet.
[0101] This invention will anticipate and meet all of the user's
lifestyle needs in a friendly and efficient way within a virtual
environment that emulates the world as the user knows it. As
technology and the communications infrastructure have evolved, the
time has come for television, radio, print, and the Internet to
meet in a multidimensional and Post-PC virtual environment that
offers, through connectivity and interactivity, individualized
information, entertainment, and communication options.
II. Host and Personal Assistant
[0102] Making reference to FIG. 7, the present invention provides
an artificial intelligence engine 701 to guide and assist the user
of the virtual environment. In its preferred embodiment, the
invention's intelligent assistant is represented by an animated
character or avatar having human-like features and gestures. The
character can communicate back to the user through text, voice, and
its own animated gestures 702. The invention's intelligent
assistant automatically and constantly collects information of
interest to the user in order to create a user profile 703.
[0103] Users are introduced to the virtual environment by a human
host who will be a trusted, real-life person presented through
digital video and audio technologies that will emulate human
interaction powered by Artificial Intelligence and Speech
Recognition technologies 704.
[0104] The host's task is to put the new user at ease by creating
an atmosphere of trust and respect, determine the user's level of
Internet and computer expertise, and gather basic personal data
through a friendly and brief conversation before turning the user
over to a virtual character or avatar who will be the user's
personal, discrete and unobtrusive personal assistant in the
virtual environment.
[0105] This initial encounter provides the basic information
necessary to shape the personalized virtual environment that will
be delivered to each user. The interaction between the assistant
and the user is more intense at the beginning for new Internet
users in order to gather all of the necessary information to
configure the virtual environment, and less intense for
intermediate and experienced users who do not want or need
assistance in the personalization process. The complete process of
personalization should generally be completed within 180 days.
[0106] The assistant is an imaginary virtual character or avatar
whose duties are the following: [0107] Determine the look and feel
of the virtual environment for each individual user. [0108]
Determine the entertainment, informational, and transactional needs
for each individual user. [0109] Guide and assist users as they
learn to use and enjoy the virtual environment. [0110] Monitor the
user's activity in the environment to anticipate and respond to new
needs and changing preferences. [0111] Introduce new features and
services that fit the individual user's profile.
[0112] The Artificial Intelligence engine that powers the Host and
the Assistant, will continually monitor the user's activities and
gather the data that will guide the continued production and
delivery of the virtual environment, and maximize the revenue
potential through the environment's commerce, maintenance and
subscription services. Moreover, the Artificial Intelligence engine
enables the assistant or avatar to create and maintain a user
profile. The intelligent assistant or avatar alerts the user of
products or services which he or she may be interested in based on
the user's profile.
[0113] In the preferred embodiment, the intelligent assistant is
portrayed as an animated character with human-like features. Other
embodiments may use any animated characters through the use of any
graphic image. The intelligent assistant incorporates the core
technology of artificial intelligence. In the preferred embodiment,
the intelligent assistant exhibits human-like intelligence and
behavior including reasoning, voice recognition, ability to learn
or adapt through experience, pattern recognition, and natural
language understanding.
[0114] The intelligent assistant is able to carry out its actions
based on its prior experience with the user. Thus, the intelligent
assistant not only performs the above mentioned tasks but also
provides a human-like personality.
[0115] The user may communicate with the intelligent assistant via
any of several input sources, including a speech recognition device
(a microphone or wireless headset provides input to the speech
recognition device). The user can issue commands to or communicate
with the intelligent assistant by voice commands entered via the
microphone which provides input to the speech recognition device
FIG. 18, 1803. The speech recognition device is used to provide
input to the invention's intelligent assistant by converting spoken
words into text. Speech is converted into text as if the words were
typed on a keyboard. In the preferred embodiment, the speech
recognition device used to provide input to the intelligent
assistant can recognize all types of vocabularies such as numeric
digits, words, and phrases.
[0116] The intelligent assistant can respond to or communicate with
the user in three different ways. The intelligent assistant can
respond to the user through text. The intelligent assistant can
display text on the TV screen 1807.
[0117] In the preferred embodiment of this invention, the
intelligent assistant communicates through speech via a speech
synthesis device 1805. The speech synthesis device generates
machine voice by arranging phonemes into words. The speech
synthesis device is used to turn text into spoken words. To achieve
this task, the speech synthesis device performs real time
conversion without a pre-defined vocabulary.
[0118] Referring to FIG. 16, the speech synthesis device feeds from
an expert system, providing outputs to a speaker through the speech
synthesis device. The invention's intelligent assistant utilizes a
standard audio device connected to the system. As the third form of
communication, the intelligent assistant may communicate with the
user through animation or gesture.
[0119] The assistant provides information and help to the user. The
assistant behaviors are sensitive to the history of user
behavior.
[0120] In a preferred embodiment of this invention the assistant
integrates data from several sources. The assistant accepts direct
interactions from the user as well as using the history of previous
user behavior stored in a user profile 703. In a preferred
embodiment, these inputs allow individualization of assistant
interaction. Alternative embodiments are responsive to additional
data types and sources. In a preferred embodiment of this invention
the diverse assistant behaviors adapt to the user based on a
variety of information about the user. The assistant modifies its
behavior on the basis of a growing history of interactions with the
user over time, as this history of user behavior is stored in the
user profile database.
[0121] The adaptation of the assistant to the user emerges from the
assistant's updating of data in the user profile database. In the
preferred embodiment, the assistant's knowledge of the user is
represented by data in the user profile database, in particular in
the user cognitive model, which stores general materials and
independent data describing how the user behaves. As the user
interacts with the virtual environment, the assistant receives
information regarding the user's preferences. The assistant updates
the user profile database with data from these interactions. As
this data is updated, the assistant adapts to the user, and thereby
individualizes to the user. This adaptation is maintained across
sessions with this user.
[0122] In the preferred embodiment, the data updated by the 20
assistant gives more weight to recent than past behavior. In
addition, the assistant tries to anticipate future behavior.
III. Artificial Intelligence Engine
[0123] The virtual environment of this invention is based on
cutting-edge technologies particularly powered by Artificial
Intelligence and speech recognition. The Artificial Intelligence
engine is the centerpiece of the environment. The engine combines
the strength of natural-language parsing software, speech
recognition technology, digital video and audio software, data
mining processes, knowledge-based, and expert systems.
[0124] Combined with the capabilities of human editors and customer
service representatives, the engine may anticipate and fulfill all
of the users' on-line needs by pro-actively pushing choices based
on preferences not on past behavior. Through this engine, the
virtual environment is able to deliver the defined services,
information, and assistance its users want and need in a simple
non-technical manner. The environment appeals to the general
population because it relies on a personal human and
non-technological relationship to satisfy individual needs and
preferences.
IV. Life in the Virtual Environment Post-PC Era: Content
Solution/Services that could be Provided
[0125] Content can take many shapes--text, audio, video, graphics,
among others--and it can be delivered in any number of ways, but it
is the critical element to attracting a significant number of
people to an environment not just one time, or once in a while, but
every day and even many times a day. However, a disorganized
proliferation of content only adds to the frustration of new
Internet users as they struggle to master an unfriendly medium that
does not even speak their own language.
[0126] There are five principal content areas in the preferred
embodiment of the virtual environment object of this invention
"Entertainment," "News," "Education," "Personal Growth," and
"Business or Professional Growth", divided by market segment and
topical categories, as shown in the following table. Many more
content areas may be available in other embodiments of the
invention. For instance, the following table presents one
embodiment of possible contents areas of the virtual environment of
this invention. Many other combinations are possible and fall
within the scope of this invention. TABLE-US-00001 CONTENT AREAS
MEN WOMEN CHILDREN PROFESSIONAL FASHION/BEAUTY (3-12 YEARS OLD)
SPORTS COLLEGE MOTHERHOOD/CHILD HOME AND DEVELOPMENT SCHOOL
PROJECTS/ SPORTS/AUTO/ HOME/DECORATION IDEAS/GAMES/ BOAT NEW TOYS
ELECTRONICS ROMANCE KID PAGES LEISURE SPORTS HEALTH/FITNESS
CONSUMER (GOLF, TENNIS, PRODUCT HUNTING) REVIEWS CONSUMER SCHOOL
DIRECTORY PRODUCT REVIEWS SUMMER CAMP DAYCARE & AFTER SCHOOL
DIRECTORY LEISURE ACTIVITIES HOROSCOPE CONSUMER PRODUCT REVIEWS
TEENAGERS BUSINESS PARTY PLANNING FAN CLUBS TOP NEWS BABY MUSIC
INTERNET NEWS CHILDREN TV/MOVIES OPINION/ANALYSIS TEEN THEME
FASHION/BEAUTY THE MARKETS SPECIAL OCCASION CYBER GAMES MOVERS
& SHAKERS WEDDINGS ROMANCE SMALL BUSINESS CORPORATE TEENPAGES
BUSINESS AGENDA CONSUMER SWAP BOARDS PERSONAL FINANCE PRODUCT
REVIEWS HOROSCOPE CAREER CENTER CONSUMER PRODUCT REVIEWS
Travel/Tourism Arts & Entertainment Politics Directories
Schedules Local Events Reviews National Travel Planning Services
Ticketing Services Virtual Vacations Special Events Daily News
Church/Religious Pets/Mascots Local Local Care National National
Health International Schedules General Advice Sports Activities
Vet/Grooming Services Consumer Product Reviews Senior Citizens
Physically Challenged Health Resources Resources Health Personal
Growth Personal Growth
[0127] Content may include original articles, professional and
commercial directories, as well as material generated through any
of the following information sources: [0128] Existing Web Sites
[0129] News Wires and Services [0130] Subject Matter Experts [0131]
Writers/Reporters [0132] Community, Professional, Civic, Religious
Organizations and Associations [0133] Civic/Religious/Cultural
Organizations and Associations [0134] Professional and Amateur
Sports Organizations and Associations [0135] Local Clubs
[0136] In the preferred embodiment, national and local celebrities
may be recruited as spokespersons or as hosts/presenters of the
virtual environment. In addition, content may be continually
refreshed by adding information, product, content, and services in
response to consumer demand.
[0137] In the preferred embodiment, production and presentation for
each content area is performed by an editorial staff composed of:
[0138] Editors/Coordinators. Responsible for coordinating the
production of material from all information sources, as well as
editing and filtering it for users. [0139] Writers/Reporters.
(Includes video/audio/photo/graphic material). Responsible for
producing material according to editors/coordinators guidelines.
[0140] Graphic Artists. Responsible for the visual presentation of
content, according to editors/coordinators guidelines. [0141]
Programmers. Responsible for developing/maintaining interactivity
between the virtual environment and its users. Are also responsible
for developing and maintaining content production and delivery
tools.
[0142] Some of the services that may provided by the virtual
environment can be categorized as follows: [0143] Services provided
by Application servers. Services in this category are provided
through the Internet. The virtual environment of this invention can
organize the contents of the service and deliver them to the users.
[0144] Basic Information Services. The information is current, not
repeated and the content is customized according to a user's
preferences. [0145] Any general and some special services may be
customized according to customer needs. [0146] Home banking.
Through its set-top box, The environment of this invention provides
secure connections to commercial banks, enabling customers to
transfer money between bank accounts, determine payment schedules,
complete credit card and loan applications, purchase certificates
of deposit and other investment products, safely pay bills, review
statement activity, make loan repayments, buy insurance, access
up-to-date account information and so on. [0147] E-commerce. The
environment of this invention provides a convenient and competitive
way of shopping at home. A customer can make a purchase at a retail
space on the virtual environment. A secure transaction is
paramount. Competitive prices, buying suggestions, and consumer
product reports will attract users. The virtual environment can be
an excellent shopping assistant and companion. [0148] The
environment contains a customer management system which:i.s manages
users' profiles and preferences information enabling the delivery
of customized content based on the user's preferences, and supports
promotions and one-to-one marketing. The system provides the
service of direct e-mail marketing campaigns based on subscriber
profiles and preferences. [0149] The Artificial Intelligence engine
provides many functions such as Voice Recognition, personal
preference analysis and so on. The knowledge-based database
contains some domain knowledge and expertise supporting research,
inference and expert systems, such as any E-service. A database
contains the data of the virtual environment. [0150] Customer
monitoring and customer feedback means, enable the system to
monitor a user's actions to the system and analyze user's behavior
and provide personal preference information to the Customer
Management System. The tool can process user's feedback timely and
adjust the system services accordingly. [0151] Video over IP:
(Needs digital video camera) Provides video conference service.
[0152] Internet conference: offering one-to-one and one-to-many
Internet conversation, such as chat rooms. [0153] Web-casting:
announcement, and distribution of software updates. V. Examples of
Uses of the Invention
[0154] A. Home Banking. In one embodiment of the invention, a home
banking service may be provided through the environment of the
invention. The home bank of said embodiment may include the
following servers: [0155] A browser or a conversation user
interface (UI): The interface that customers launch the home
banking service from front-end client program. [0156] A Proxy
server: The proxy server with a security server 20 provides
security connection to void web attacks. [0157] A web server: The
web server interprets customer requests from a web browser. [0158]
Application server: The application server provides application
specific program to process customer requests. [0159] Bank server:
The bank server provides the banking services listed above.
[0160] A home banking transaction in the environment of the
invention typically has the following steps: [0161] (1) The
customer requests a bank service by using the browser or the
conversation. [0162] (2) The set-top box opens a secure session
with the proxy server and a security server by using SSL
cryptography standard. [0163] (3) The proxy server will open an SSL
session with the application server. [0164] (4) The customer makes
a request to run the banking application. [0165] (5) The request is
encrypted and forwarded over the digital network to the proxy and
security server. [0166] (6) The request is authenticated,
re-encrypted, and is forwarded to the application server. [0167]
(7) The application server requests a service from the bank server
and sends the encrypted information back to the proxy server and
security server. [0168] (8) The proxy and security servers decrypt
the application, signs using system's digital certificate, and then
send the encrypted information back to the set-top box. [0169] (9)
The set-top box decrypted the information and opens a direct SSL
connection to the bank server. [0170] (10) The bank server
retrieves the requested financial information from the local
database and responds by sending the private data back to the
set-top box. [0171] (11) The set-top box interprets the requested
data and talks to the user and/or displays it on the television set
or other appropriate output mode.
[0172] B. E-commerce (encrypted credit card payments). The credit
card details are encrypted when an user of the environment submits
the information into the set-top e-commerce application. The
details are then sent from the subscriber to the particular
merchant as an encrypted message.
[0173] E-commerce transactions may include the following servers:
[0174] A browser or conversation user interface: to launch the
seti::top's e-commerce application. [0175] E-commerce applications:
Provide a catalog containing text and multimedia information on
every merchant. An user navigates the catalog of items and decides
to order a particular item. [0176] E-commerce application for
conversation: To analyze the user's need and find a merchant either
through an outside Ecommerce server interactively or through the
virtual environment directly. [0177] A merchant server: Processes
purchase requests and returns a 20 summary of the order to the
set-top box. [0178] A transaction server: A transaction server at
the financial institution that processed the transaction request.
[0179] A database server: A database server provides database
access capability requested by the transaction server for credit
card accounts of the customer. [0180] Outside server: for credit
card information. [0181] Accounting and inventory system: records
purchase information and arranges the delivery of the merchant.
[0182] (12) Verbal E-commerce interface provides shopping
information through voice communication.
[0183] A typical E-commerce transaction in the environment of this
invention includes the following steps: [0184] (1) The customer
launches the set-top box' e-commerce application (connecting to a
merchant server); [0185] (2) The merchant server provides a catalog
containing text and multimedia information on all available
merchandise; [0186] (3) The customer navigates the catalog of items
and decides to order a particular item, or the verbal E-commerce
interfaces talks with the customer and analyzes the customer's need
and finds a merchant either through an outside E-commerce server
interactively or through the virtual environment directly; [0187]
(4) The set-top box establishes an SSL connection with the merchant
s server; [0188] (5) The customer initiates the purchase process by
providing payment and shipping information; [0189] (6) The merchant
server receives the purchase request and returns a summary of the
order to the set-top box; [0190] (7) Details of the order,
including price, credit card number, shipping address, and order
number are displayed on the television set or other output medium;
[0191] (8) The customer verifies the information and confirms the
order back to the merchant server; [0192] (9) The merchant server
repacks the related customer information without the order details
by encryption and sends it to a financial institution; [0193] (10)
The transaction server at the financial institution receives the
request and processes it; [0194] (11) A database server accessed by
the transaction server will access a database of the credit card
accounts of the customer; [0195] (12) If the financial institution
doesn't have direct access to a credit card database, it may send
the information on to another institution or banking network, such
as Visa; [0196] (13) The bank's database server then sends a
request to the credit card database to debit the customer's
account; [0197] (14) similarly, it sends a request to its own
customer account database to credit the merchant store's account
with the relevant amount; [0198] (15) Once the financial
transactions are completed, information of an approval or denial
returns to the 20 server; [0199] (16) Provided the user's details
are correct, the merchant server will activate the other business
processes such as inventory, accounting and fulfillment; [0200]
(17) The merchandise is delivered.
[0201] C. Education. All areas of education including primary,
secondary and third level education will be moving into a new
dimension in the future (Supported cable, terrestrial, satellite
and MMDS). The education needs of the environment of the invention
encompasses the following components: [0202] The Set-top box'
educational application (a browser or a resident program): creates
virtual classrooms, so students in different schools interact and
learn as they where in the same classroom. [0203] A database
server: stores the various educational programs. [0204] A broadcast
server: Manages and injects the requested data into the MPEG video
system.
[0205] D. Entertainment. For example, playing games. Using a
standard set-top box and a remote control, a user can play a given
game. There are three primary system components that may be used
for supporting set-top gaming. [0206] Broadcast server: This server
is responsible for injecting or inserting the gaming application
into the digital MPEG-2 transport system. [0207] Games application
server: This server is used to store the programming code of the
various games that are available to users of the environment.
[0208] Set-top box with resident games application: 1. The
application can be downloaded by the server into the set-top box or
resides in the box. 2. Display the list of games. 3. Select a game.
4. Authorization. 5. Set-top box transformed into a game
console.
[0209] E. E-services. E-services include different topics such as
e-medical, e-travel, e-house improvement, e-gardening, and so on.
An e-service provides two levels of knowledge. One level is the
primary knowledge of the topic that customers want to know or get
in that field. The next level provides expertise of the field that
provides suggestions or solutions to some problems customers may
have. E-services in the environment of the invention have a rich
user interface, including text, video, audio, pictures, tables, and
their combinations, so that customers do not feel bored by the
program.
[0210] As an example of the potential applications of the virtual
environment of the invention, E-medical services may be offered as
an e-service. For example, the environment may offer: [0211]
Graphical presentations of the human body and detailed parts, such
as skeleton and organs. [0212] Graphical presentation of human body
functions categorized by sub-systems, such as digestive system and
blood system; [0213] Hierarchical presentation and back-forward
link so that any 0.15 part can be easily identified and presented
verbally and graphically; [0214] Basic question and answer
capability; [0215] Testing the knowledge a customer has mastered in
the field; [0216] Output description to a printer.
[0217] The expert level provides the following functions: [0218]
Interactive capability for answering and explaining deeper
questions; [0219] An expert system for common health problem
diagnostics; [0220] Provides suggestions for possible medical
treatment and 25 provides nonprescription treatment if available;
[0221] Interface to any medical supply web site for needed
medications; [0222] Reporting personal health problem to Subscriber
Management System; [0223] Sends an e-mail to a doctor for
assistance; [0224] Places a phone call for emergency
situations.
[0225] F. E-Mail
[0226] G. User Assistance and Learning. The User Assistance and
Learning module has two major roles. The first one is training an
user to use the system and providing online help documents. The
system will introduce the environment to an user in a natural and
user friendly way so that no prerequisite knowledge is needed in
learning the system. The other one is to play the role of an
assistant when an user has any problem in using the system. The
assistance includes solving problems and providing suggestions.
This system needs to talk with the Subscriber Management System and
provide possible preference information.
[0227] The system may include the following functions: [0228]
Generates online help document for any service topics; [0229]
Creates a training environment for any service topics; [0230] Be
able to train a customer in a natural and friendly way; [0231]
Provide content sensitive assistance; [0232] Log training topics
for preference analysis. VI. User Participation
[0233] Communication, interaction and participation needs in the
virtual environment of this invention may be met through a broad
array of communications modes and software applications, including
voice over IP, discussion style bulletin boards, person and
organizational web sites, chat rooms, instant messaging, e-mail,
swap boards, low and high value auctions and virtual flee markets,
classified listings, personals, user generated product review
boards, user managed events calendars, classified listings, and
professional and commercial directories, among others. All of these
features may be supported with 24-hour, immediate response customer
service, via e-mail, chats, instant messaging, and telephone. The
virtual environment of this invention provides the interfaces that
allow the user to integrate and use these modes of communications
in ways that are appropriate and customizable to their needs and
that raise an awareness of the value of the virtual reality
environment.
VII. User Support
[0234] The key to user acquisition and retention on the Internet is
effective user support. Thus, the virtual environment of the
invention should be supported by a staff that reflects the
composition of the general population: 30% technical and 70%
nontechnical personnel. Trained customer experience providers must
be available to assist users and members in every area of
interaction with the virtual environment. They may help users and
members connect to the environment, find information and services,
purchase products and conduct all of the commercial and
non-commercial transactions available in the virtual environment of
this invention.
VIII. Hardware Specification for System
[0235] The back-end system provides the services that less
fortunate users spend so much time searching-out and setting-up
with such unreliable satisfaction; Artificial Intelligence and
extension of the natural interface model allow tasks like banking
and shopping to be performed with speed and ease that up until now
required a human helper. [0236] User Interface: microphone/cordless
phone, speaker, mouse, keyboard, TV/monitor [0237] Set-top box:
CPU, memory, storage, operating system, Internet connectivity,
device connectivity (Input/Output) [0238] Internet Service Servers:
Customer servers, application servers, backup servers, database
servers [0239] Call Center [0240] Satellite Connectivity: Receiver,
Decoder Standards
[0241] This section lists the industrial standards recommended for
the preferred embodiment of the invention.
[0242] Network Communication Protocol [0243] TCP/IP (Transmission
Control Protocol/Internet Protocol) for transferring information
across the Internet
[0244] Document Publishing and Viewing Protocol [0245] HTML
(Hypertext Markup Language> [0246] XML (Extensible Markup
Language) [0247] DHTML (Dynamic HTML)
[0248] E-Mail protocol [0249] SMTP (Simple Message Transfer
Protocol) for sending and receiving e-mails among servers [0250]
S/MIME (Secure Multipurpose Internet Mail Extensions) for 25
encoding/decoding binary files and email encryption [0251] POP3
(Post Office Protocol) for accessing e-mail from a mail server
[0252] IMAP4 (Internet Message Access Protocol) similar functions
to POP3 with more features
[0253] Web Navigation Protocol [0254] HTTP (Hypertext Transfer
Protocol) for retrieving web pages 5 from web servers
[0255] Data Compression Protocol [0256] MPEG-2 [0257] MPEG-4
[0258] Security Protocol [0259] SSL (Secure Sockets layer) for
security [0260] SET (Secure Electronic Transaction) for supporting
credit card payment
[0261] Smart Card Reader
[0262] ISO/IEC 7816-3 (International Standards Organization)
System Services and Supporting Tools
[0263] The present invention will provide to its users two groups
of services. One is based on the Internet and related technologies
and another on television broadcasting This section lists these
services and describes the functions needed to fulfill each
service. Besides the customer services, there are some system tools
or functions to support customer services. These tools and their
descriptions are also presented in this section.
[0264] Since some of the services are based on these tools, we will
25 describe first the system tools then the services.
[0265] System supporting tools
[0266] User Management System ("UMS")
[0267] The UMS provides functions required in the virtual
environment to manage user's data such as profiles, preferences and
favorites. It accesses the user's database, provides user
information to user authentication system, sends requests to the
user authorization system and accepts updates from UMS.
[0268] In the preferred embodiment of this invention the UMS
includes the following functions: [0269] Register, modify, and
delete users profiles [0270] Track and modify users' preferences
and favorites [0271] Provide users' preference and favorite report
[0272] Statistical analysis of users' records
[0273] User Billing and Account Management System ("UBAMS)
[0274] The UBANS provides the support required in the virtual
environment to accurately manage individual's financial account. 5
It communicates with the UMS and user authorization system, and
have interface with related outside businesses such as banks and
credit card companies
[0275] This system includes the following functions: [0276]
Register, modify, and cancel subscriber financial records [0277]
Inventory management of set-top boxes and smart cards [0278]
Transaction recording and history logging [0279] Fault management
[0280] Bill preparation and formatting [0281] Presentation of bills
in electronic formats [0282] Accounting and auditing facilities
[0283] Multilingual and multi-currency capability [0284] Interface
with banks and credit card companies
[0285] User Authentication System (UANS) [0286] Encrypts and
decrypts data send to set-top box and received from set-top boxes;
[0287] Identifies a smart card [0288] Identifies a subscriber's
login name and password [0289] Identifies a smart card ID [0290]
Manages a subscriber's related data
[0291] User Authorization System (SARS)
[0292] The main task of the UANS is to translate the requests
coming from the SMS into entitlement management messages (EMMs) and
entitlement control messages (ECMs). An ECM is used to carry
authorization details and are subscriber specific. ECMs, on the
other hand, carry program and service specific information,
including control words that are used by the smart card to decrypt
the relevant program. These authorization messages are then sent
via the digital multiplex to the smart card, which is located in
the set-top-box. They are sent to users on a regular interval (for
instance, every month) to renew subscription rights on the smart
cards. In the case of Pay Per View (PPV) applications, the SAS may
send a certain amount of electronic tokens to the smart card that
will allow users to purchase a variety of PPV events. The SAS
contains database(s) that are capable of storing the following
items of information: [0293] Pay TV product information [0294] Data
to support the electronic TV guide [0295] Identification numbers of
smart cards [0296] Customer profiles [0297] Scheduling data
[0298] The User Authorization System includes the following
functions: [0299] Generates EMMs and ECMs [0300] Creates and
modifies user's data of authorization [0301] Manages pay TV product
information and TV guide
[0302] Basic Information Service--system requirements:
[0303] The environment of the invention may provide an user with
information like news, weather, sports, stock quotes, and so on.
The service can be customized according to individual tastes. The
medium of the service can be an audio broadcast or be displayed on
TV. The information will be provided during the set-top box
initialization so that customers will not notice possible delay of
system initialization. Another feature of the environment of the
invention is to provide alert or notice to users, reminding them
that a new e-mail just arrived, or a television program will be
broadcasted in about 10 minutes, or a chatter asks for a
conversation and waiting for his or her reply, and so on.
[0304] An important characteristic of the environment of the
invention is that the service is user friendly and as natural as
possible. In addition, the content meets users' preferences.
IX. Servers
[0305] The environment in the preferred embodiment may include the
following servers: [0306] A user profile server: To accept a
customer connection request and authenticating the service. [0307]
A basic service server: The server composes the service information
for each registered customer according to his/her profile. When a
customer logs in to a set-top box, the customer preferred media,
determined by previous connection, presents current contents of the
service, i.e. the contents should be different from the previous
logins for the same user. The contents are arranged according to
their preference and no repeated information is presented each
time. The user can go back to review previous content of the day.
[0308] An application server: When users' requests cannot be
provided by the environment, the basic server invokes the
application server to provide the needed services. X. Smart
Card
[0309] In E-commerce, using smart card instead of using a credit
card to purchase an article will replace the credit transaction
processes by E-cash. When the merchant server receives a confirmed
purchase request, a smart card reader connected with a set-top box
will accomplish the money transaction and tell the merchant server
to finish the other purchase process.
[0310] A smart card is just a very simple computer. It has a CPU,
RN'I, ROM and EEPROM for an operating system and a serial I/o
interface. It has no interface for keyboards and monitors. The
software for a smart card should include the following tasks:
[0311] Data transmission over the bi-directional, serial terminal
interface. [0312] Loading, operating, and management of
applications. [0313] Execution control and instruction processing.
[0314] Protected access to data. [0315] Memory management. [0316]
File management. [0317] Management and execution of cryptographic
algorithms.
[0318] A smart card reader connected to the set-top box performs
the following tasks: [0319] Identify card acceptability. [0320]
Authenticating an individual subscriber. [0321] Authorizing the
services. [0322] Data transaction. [0323] No repudiation: a third
party is required to verify that the transaction happened. [0324]
Cryptography: encrypting data when transmitting. [0325] Issuing a
smart card or modifying data in a card. [0326] Account management
and money transaction when using the card
Scope/Overview
[0327] Certain aspects of the design of the preferred embodiment
have several viable alternatives. Some of the main alternatives as
are identified herein. Nonetheless, other alternatives are possible
in other embodiments of the invention and fall within the scope
therewith. Its advantages and the recommended configuration
including mid-level block diagrams are described in the following
sections.
[0328] 1. A Vision of the Set-Top-Box as an Appliance
[0329] The Set-Top-Box ("STB") is the user's tool to visit the 20
environment of the invention; a virtual 3D world with an interface
optimized for human interaction and preferences e.g. music, movies,
TV, banking, shopping, among others. All may be available at a word
request and of course are faster and cheaper than a drive to the
store.
[0330] The hardware of the STB differs from that of a regular
computer. It has no floppy drive, no noisy cooling fan, will
connect to a TV, at least with NTSC/PAL and HDTV, VGA and SVideo,
will remain options. A tuner, a remote control and a keyboard are
essential in the design of the preferred embodiment of the
environment of the invention. The software includes Speech
Processing,
[0331] Virtual Reality (VR) and Artificial Intelligence
technologies, voice mail and video mail are also part of the
preferred embodiment.
[0332] The design and operation of the STB are geared towards
technically inexperienced users, who don't and won't have a
"computer," and who don't have the patience or inclination to deal
with software upgrades, cryptic error messages, installation
configuration or complicated instruction manuals.
[0333] 2. Participant's Interface
[0334] There should be no confusion as to how to use the STB, no
ambiguity, and no apprehension or frustration generated. If the
user dreads to use the environment of the invention, the
environment is not fulfilling its purpose.
[0335] The interface should be friendly, non-threatening even if
that does not contribute to the efficiency of an experienced user.
The user will have preemptive control, e.g. if a site, a picture or
a program is loading too slowly, the user will be able to
"instantly" abort the process or to select another option.
[0336] Hardware Interface
[0337] The system is designed to be ready-to-go right
out-of-the-box with minimum plug-in peripherals like a mouse,
keyboard, power-supply and video cable. More emphasis is placed on
the wireless technology and on the integration of functions into
one physical case like the Universal Serial Bus (USB) with its
"plug it in anywhere, it will fit" auto configurability.
[0338] User will interface with system hardware through mostly
three components: [0339] (1) The STB itself will have no real user
interface except a power switch (and possibly a microphone and/or
speaker). Certain external connections are user interfaces insofar
as the User will have to plug-in the appropriate cables. The STB
might incorporate a microphone jack, but certainly will need audio
and video outputs, etc. [0340] (2) The Voice Remote Control 1101,
provides at least rudimentary cursor control and select functions,
as well as a microphone. In an embodiment of the invention
telephone technology may be implemented in the design, because
cordless phones already incorporate audio and at least rudimentary
digital (button) communications channels. A standard IR remote
control might be modified to incorporate audio and use RF instead
of, or in addition to, IR for no directionality, continuous
operation and non-line-of sigh operation. [0341] (3) The Passive
Pointing Device (PPD) 1102: is in the form of a hardware device
sitting on top of the TV that provides modulated invisible IR
illumination and captures and analyses images to produce
cursor-control commands. It is designed to respond to the position
of a retro reflective "marker" that would be attached to the VCR,
or perhaps provided as a separate device. The mapping of the user's
control motions to the plane of the TV screen is direct, unlike
other types of pointing devices (mouse, trackballs, joysticks,
etc.) The actual set-up of the necessary connections of the STB to
the Internet, TV, and to the VCR has many permutations. Thus, in
order to accommodate for various users' needs, the user may be
provided with a step-by-step user manual that contains various
configurations such as those presented in FIGS. 1-6 and in Table
1.
[0342] Software Interface:
[0343] The underlying philosophy behind this invention is the
implementation of voice technology, virtual reality, artificial
intelligence, and an Intelligence Center (FIG. 8). An Illustration
on how users interface with the system during setup and upgrades is
presented in the following paragraphs:
[0344] Artificial Intelligence (AI) for Set-Up: One of the main
thrusts behind the invention is the implementation of artificial
intelligence. As an illustration, let us consider the first phase
when the new user interacts with the system:
[0345] To initiate the service, the participant hooks up the STB to
the TV and to his service provider through his/her phone line or
broadband cable, etc., turn it on, and uses its remote control
and/or its wireless keyboard to interact with the environment of
the invention. After turning on his new STB, the TV will show a
logo, or other suitable output, to indicate that it sees the box.
If the box detects the RF signal, i.e., if the user correctly
attached the antenna or cable TV lead, etc. to the appropriate
jack, then the STB will display a message on the TV screen
indicating that the connection is working too. At this point in
time the server starts to automatically interrogate the STE without
any input from the participant, inquiring its type and
capabilities. (e.g.: "I am a Meteoroid International Pentium XIII
8000 MHz w/512 Mb memory, 100 GB HDD, 1OXDVD, xyz-Type Graphics
Processor, running Windows NT 2001 rev 115.99, over DSTJ
communications, etc.") In response, the server selects the
appropriate code package and the requested services for the STB. It
then automatically downloads/installs the 10 appropriate software
without involving the participant. The software then determines
that the system is in a "Set-Up State". Accordingly an Avatar
appears and announces, "Can you see me? Say `Yes` or `Si` if you
can, Say `No` or hit `Delete` if you can't." "Can you hear me? Say
`Yes` or `Si` if you can, Say `No` or hit `Delete` if you can't."
etc. Communication may take place in any language. These words are
simultaneously displayed on the screen where further interaction
elucidates and guides the rest of the installation. With some help
from the user, the program will detect if TV/Cable is connected
correctly, if the microphone is connected correctly, and whether
the user can see and hear it, speak to it, use the Keyboard and
Remote Control, etc.
[0346] Using The Mission's Artificial Intelligence System as well
as the Artificial Intelligence Center of this invention would also
allow to log user's concerns, failure events, as well as some
diagnostics.
[0347] These assessments may be used to improve user services where
the server has the capability to upgrade the code as improved
versions become available, or to replace corrupted code, in a
non-transparent fashion to the user.
[0348] Artificial Intelligence for Upgrade: Once the system is
initially set-up and running, the user can have a variety of
interactions with it. The basic scenario includes those methods
that are necessary and sufficient for making effective use of the
system, but enhancements, additional hardware or features are also
possible. The setup AI routine will get the user to the optimal
configuration, and will be available to help optimize it at any
time. For example, the simplest Video connection might be by
connecting the STB to the TV A/V jacks or even through an external
RF modulator (like FIG. 3), but neither method is optimized until
some other connections are made. Software upgrades will make it
possible to provide functions to the user that are not part of the
basic or initial functionality, including the detection of the
installation of any optional or enhanced peripherals. Hardware
upgrades will be easily accommodated by an automatic
invocation/installation of the required software. If, for instance,
a new IR Remote Control is used for the first time, or the Passive
Pointing Device is installed, the user will not have to attend to
any technical manual, search for a disk or download a driver;
whatever setup is needed it will be taken care of automatically,
with code being downloaded from the internet (or the distribution
disk) as necessary.
[0349] Artificial Intelligence and Artificial Intelligence Center:
an illustration on how to implement these concepts is presented in
the mid and top-level diagram shown in FIGS. 7A, 7B, and 8.
[0350] STB Hardware Functional Blocks
[0351] The features intended for inclusion in the STE include, but
are not limited to, those shown in FIGS. 9 and 11-13 where the
major functional blocks/features are discussed below.
[0352] Video Programming enters the STB from Broadcast, Cable,
Satellite or other sources through either the RF or Video Inputs
901. This is one area where different service configurations may
occur. For example, certain sources may use a baseband form, i.e.,
not change the modulation of the digital signals. Others may
modulate the signals onto broadcast frequency 3 or 4. In general,
the signal quality is best if it is baseband video, so that it does
not have to lose quality from having to go through modulation and
demodulation, but this of course is limited to few miles. In many
installations only RF (modulated) will be available, so, to have
"picture-in-picture" or text/image overlay, one will need a
tuner/demodulator. By passing the DEMODULATED video through the
box, a sort of "picture-in-picture" function can be invoked, and
the system can be ready for invocation even when the user is just
watching TV. This convenience is key to getting the user to spend
time with the environment and he or she must come to take this
access for granted
[0353] The IR Receiver Port 902 permits compatibility with custom
or standard third-party IR keyboards 903, and with standard IR
Remote Controls.
[0354] If no IR Remote control is provided, the ability to control
the Power and Volume (and other functions) of a TV requires an
external IR source ("IR Blaster") 904, probably a little emitter on
the end of a cord, and either an IR receiver (to acquire the
necessary codes ["Learning Remote" function]) or the purchase of a
universal remote control library such as those available, for
example, from Sejin or Universal Electronics. The mode of operation
would involve the reception of a command from the Voice Remote
Control (VRC) 905, which is translated into an IR command.
[0355] The IR Pointing Device Receiver 906 uses the modulated IR
signal reflected from a button, baton or thimble of IR reflective
material 907 to control an on-screen cursor. The function of the IR
Receiver Port 902 might be combined into this to allow simultaneous
communications and cursor control. Likewise, the modulated IR
signal source, the Pointing Device Stimulus Generator 904, can also
serve as an IR communications transmitter. In order to be able to
control a TV (e.g. POWER or VOLUME), it is necessary to either have
IR transmission capability on the Remote Control, or from the STB
(an "IR Blaster" function). The IR codes may be either acquired
from an existing Remote Control ("Learning") or from a library of
codes ("Universal").
[0356] The Sound Processor 908 provides analog stereo signals to a
TV or Stereo 909, and preferably the ability to accept external
input for sampling or mixing with the output (needed for use with
the "Video In" function)
[0357] The Graphics Processing Unit (GPU) 910 produces the output
to go to the TV, as well as, preferably, an auxiliary VGA or other
high-resolution output. The NTSC output consists of the
STB-generated video overlaying ("picture-in-picture") or mixed with
the video-programming source. The massive graphics-processing load
presented by this application is served by a high-end GPU;
currently Nvidia makes the device with best price/performance that
is adequate. The Accelerated Graphics Port (AGP) is the interface
used by this class of GPU.
[0358] Communications with the combination Microphone/Remote
Control (Voice Remote Control, VRC) 905 must be RF, for continuous
function, as well as for non-Line-Of-Sight operation. There are
several choices as to the details of how this can be
accomplished.
[0359] FIG. 9 shows a method relying on the RF technology developed
for Cordless Phones. By providing a very simple circuit on a
phone-line modem, the phone may be connected directly to the phone
line through a relay, so that it would function normally, or it may
be switched to a simple circuit that imitates the phone line, so
that the user is now "talking to" the STB. DTMF may be decoded in
order to use the buttons to perform functions, and only the
DTMF-generating buttons may be usable for control. FIG. 10 has a
suggested assignment of the twelve standard phone buttons.
[0360] FIG. 11 shows another method of using the technology
developed for cordless phones. Qutek has developed a telephone with
a 56k modem integrated into its base station, with a USB connection
to a computer. Depending on the details of the internal circuitry,
it may be possible, in another embodiment of the invention, to
slightly modify the mode of 20 operation of this device to have it
function as a link, connecting audio between the computer and the
handset. Because the proprietary link between the handset and base
station is also available to them for programming, it should be
possible for the modified phone to send the computer button-press
information directly, rather than having to rely on DTMF decoding,
and this would also allow the assignment of control functions to
the other buttons on the handset. Note that a DTMF decoding
capability may still be useful in certain cases.
[0361] FIG. 12 shows the technologically most elegant method, with
a custom VRC based on some technology like BlueTooth (BT) The
easiest way to get there would be to incorporate a standard BT
transceiver card into a modified IR Remote Control (from Universal
Electronics or Sejin, etc.) to incorporate RF technology in
parallel with IR. This would permit choosing a model with whatever
fancy features one desires, including an integrated Pointing
Device.
[0362] At least one general-purpose Communications Port 911 may be
provided for features like printer and computer connectivity. USB
may be used, as it is designed for expandability, and to be used
for both digitally coded audio and other digital communications.
10/100 card may be used to support the current interface of choice
for DSL and other high-speed modems. It may be necessary to provide
an external DSL modem 912 because of all the different flavors that
exist.
[0363] An Internal 56k Modem 913 may be included in the STB to
provide a rudimentary means by which the STD can communicate via
the internet if no faster connection is available.
[0364] The CPU 914 and Graphics Processing Unit (GPU) 910 systems
perform the processing. A Hard Disk Drive (HOD) may be included as
well as a fast processor and large memory, see table 2.
[0365] The STB has a DVD-ROM drive 915. The operating system and
application code may be stored on a high-density medium, a CD disk
which makes it possible to load new machines quickly, and to give
the user a way to reload the system to a usable state in case of
system crash. The BIOS will determine which disk is considered for
restoring the system. This concept may be complemented by an
Intelligence Center in order to minimize user inconvenience. This
Disk will be always in the drive so that, if the user removes it to
play another disk, or for any other reason, the application should
request it back at the first reasonable opportunity, so that it
doesn't become lost.
[0366] The code on the disk will not correspond to the latest
version of the software, but just to a workable one, with upgrades
being via the connection to the servers. As the distance between
the disk version and the latest version increases, there will come
a point where it may be better to issue an upgraded CD. The DVD
player allows the opportunity to provide users with more option for
entertainment.
[0367] In an embodiment of the invention an RF output may be
provided. However, this is a relatively expensive function to add,
and would probably be needed by a small fraction of users: most TVs
and most VCRs these days have NTSC (baseband) Video (and Audio)
Inputs, the use of which is both more convenient and better quality
than RF-modulating the video, then demodulating it in the TV (or
VCR) back to baseband. An external RF modulator may be supplied as
an option.
[0368] The preferred embodiment of the STB is presented in the
configuration of FIG. 13. The adaptation of telephone technology
provides an adequate Voice Remote Control function and allows VoIP
as well as normal telephony. Functions that are not included in
this configuration may be added as part of other embodiments, e.g.
the PPD and the IR RC). Detailed Specifications of the preferred
embodiment of the STB are presented in table 2.
[0369] Voice Remote Control (VRC)
[0370] The Remote Control (RC) to be used for controlling the STB
must also transmit reasonable-quality audio for Voice Recognition
(VR). It also must serve to control the user's TV, and serve as a
cursor-control device.
[0371] As discussed above, there are two plausible methods of
obtaining a device capable of meeting these requirements: using
Remote Control technology and adding wireless (RF) audio capability
by implementing the emerging Blue Tooth technology or using
telephone technology for its wireless RF audio capability, and
adding Remote Control to it. This last approach is adopted in the
preferred embodiment.
[0372] The sound quality for VR provided by a phone-based
technology is excellent, because the telephone design is optimized
for this, and people are used to the associated speaking model. The
earpiece encourages people to hold the microphone in the correct
location, improving sound quality and stabilizing volume, and the
side tone generated in their ear provides a feedback of their
speaking volume that further stabilizes it. The telephone
technology might be incorporated in one of two ways, by allowing
any phone to be used, or by using a special design that
incorporates the necessary circuits into a phone.
[0373] To make any phone serve as a VRC, it is be necessary to
provide for connecting the phone to a regular phone line, so that
it can function normally, as well as to a "Telco simulator"
circuit. A pair of resistors connected to +12V and -I2V should be
sufficient, as this gives properties similar to those a phone sees
when it is picked-up as an extension on an established
conversation. Audio from the STB is generated and coupled to the
phone. The audio from the phone is then sampled and sent to the
STB. It would be reasonable to combine this function with either a
modem (a "software modem") or a sound card, or both. Because one of
the important advantages of this scheme is allowing the user to
employ his or her own phone, the one he already has, the user is
provided with a method by which the phone can be made to function
normally (by connecting it through a switch [relay] to the phone
line when it is not being used as a Remote Control). This method
would only allow control functions to be assigned to the twelve
DTMF-generating buttons, a significant limitation.
[0374] The Blue Tooth (BT) is supposed to eventually cost no more
than $5 to incorporate into a device: Once the base station is in
place, all sorts of wireless interactions with all sorts of devices
(cell phones, PEAs, printers, computers, etc.) are possible further
embodiments of the invention. To control the user's TV or other
devices, one would have to arrange to get access to the library of
IR codes maintained by a "Universal Remote Control" manufacturer
(Sejin and Universal are two), or we one have to incorporate an IR
receiver and implement a "Learning" method. Three major advantages
of this approach are the abundance of buttons provided, the
availability of universal libraries of codes (and the presence of
an IR emitter) for controlling TVs, and the existence of models
with various sorts of pointing devices.
[0375] All likely features intended for inclusion in the Smart
Control are included, but not limited to, in those shown in FIG.
14. Major functional blocks/features are discussed below:
[0376] The communications port labeled To PDA" 1401 allows this
control's micro controller 1402 to communicate with such a device
that may be used to contain e-mail addresses, telephone numbers,
and function-expanding software, possibly including Voice
Recognition (VR) The PDA may also communicate (indirectly) with the
Set-Top Box (STB) 1403 and also through the Infra Red (IR) transmit
1404 and receive 1405 blocks, which makes possible a very powerful
Universal Remote Control function.
[0377] RF communications 1406 allow the device to control functions
of the STB, to serve as a communications channel between the PDA
and STB, to be controlled by the STB (since BlueTooth or a similar
bi-directiona]. communications protocol may be used), and to
function as a wireless handset for a landline POTS telephone,
Internet telephone, or for VR control. Enhanced functionality may
incorporate the full functions of a PCS or Cellular telephone into
these blocks.
[0378] The IR transmitter 1404 allows control of existing
Audio/Video equipment and may be used in conjunction with the STB
to provide an on-screen cursor-control function in an embodiment
lacking RF.
[0379] The IR receiver 1405 allows for programming the IR control
codes for equipment not supported by an internal library of
codes.
[0380] A display 1407 may be used for such functions as Caller ID,
Number Last Dialed, or system status. A more extensive display may
be incorporated into the PDA.
[0381] The speaker 1408 can be used both for allowing the STB's
program to interact audibly with the user without disturbing others
present, in a "Remote Locate" beeping function, and when the device
is used as a telephone. By implementing side tone and audio from
the speaker, the user is encouraged to hold the microphone 1409 in
the optimal position and keep a regulated speaking volume. This is
important for VR (Voice Recognition).
[0382] The microphone 1409 can be used both for allowing the user
to interact verbally with the STB (through VR), and when the device
is used as a telephone, and for recording voice messages to be sent
by e-mail.
[0383] The keypad 1410 allows for system control, including cursor
control, and telephone or VoIP (Voice over Internet
Protocol/Provider) (Internet Phone) dialing.
[0384] FIG. 15 shows conceptually the preferred initial embodiment
obtained by making use of existing wireless telephone technology by
one method discussed above under "Preferred Configuration of the
Set-Top-Box." Some details of this implementation are omitted from
FIG. 15 that pertain to the connection of the base station of the
cordless phone adaptation. See discussion of DTMF decoding and
suggested keypad assignments. Note that the necessary control of TV
functions (Power, Volume, etc.) must, in this embodiment, be via an
"IR Blaster" function incorporated into the STB. If a telephone
incorporating a USB or other bidirectional digital communications
channel to the STB is constructed, it is important that certain
control commands be available for maximum ease in programming and
flexibility of function. Further, it is probably best to make as
little of the actual control sequence as possible incorporated in
the phone device itself, and provide a rich set of low-level
commands, as it is easier to correct program errors and add new
features by changing the STB's program.
[0385] Below is a list of suggested hardware commands to be made
available. In some embodiments, they are not actually necessary,
for example "Generate DTMF Digit" as this could be done in the
computer and transferred as an audio stream, but the phone can do
this easier, and reduce the computational load. Likewise with the
button press information: The DTMF generated by the numeral buttons
can be recognized/decoded in the computer, but it is already
necessary to report the key presses of the non-numeral buttons, if
they are to be used for remote control functions. Either in
response to a specific status request, or in response to any
successful command, the phone/RC should respond with a message that
reports status, for example, hookswitch and button conditions,
whether status has changed since last status report, etc. Here is a
suggestion of how many bits to report (bits)
[0386] (1) Hookswitch current status
[0387] ( ) Hookswitch control local or computer (maybe more bits
for "mode")
[0388] (1) Speakerphone activated.
[0389] (3) Speakerphone volume
[0390] (3) Earpiece volume
[0391] (6.times.2) Pipe status for each possible pipe (Open/Closed,
error)
[0392] (.about.20) Status of each button.
[0393] (1) Whether any button has changed since last status.
[0394] (1) Loop current detected.
[0395] (1) Ringing detected since last status (SLS)
[0396] (8) New Caller ID available (how many are available in phone
memory).
[0397] (1) User change of speed dial memory occurred SLS.
[0398] (8) Telephone number dialed SLS (how many are available in
phone memory).
[0399] (8) New DTMF digit detected SLS (how many are available in
phone memory).
[0400] (1) Handset in cradle (charging).
[0401] (1) Handset needs charging.
[0402] (1) Handset active (transmitting)
[0403] (1) Handset functioning. (Detects whether handset is
in-range, charged, turned-on.)
[0404] (.about.8) Model/capabilities.
[0405] So about 81 bits would be reported. Other, special reports
would be in response to queries for the contents of Last Number
Dialed register(s), Speed Dial registers, and Caller ID.
[0406] As far as the possible audio channels, there are a variety
of functions that may be provided if the hardware is sufficiently
flexible. Assuming there are six audio channels, three "in" and
three "out", as in the table below, here are some functions, and
the pipes that would be needed:
[0407] (1) Monitor the line for busy signals and dial tone and
such, for placing automatic phone calls, modem smart retries, "camp
on" function. (3)
[0408] (2) User calls in to tell machine "Record `Survivor` for me
tonight." (Remote Control) (3,4)
[0409] (3) Answering Machine. (3,4)
[0410] (4) Normal use as Voice Remote Control. (1,2) possibly
(5,6), (1,6) (1,2,6) etc.
[0411] Get Billy's attention because you've left him an emergency
voice message ( . . . BILL . . . BILL . . . BILL . . . ) or there's
an urgent e-mail addressed to Sally ( . . . SALLY . . . SALLY . . .
) (6)
[0412] Passive Pointing Device (PPD)
[0413] The Pointing Device (PD) supplied with the system is to be
in two parts, a part resident on or in or adjacent to the Set-Top
Box (STB), and a passive part that will be in the form of a
reflective button, thimble or baton. The mode of operation would
not require this to be either a unique material or object, nor for
it to be of expensive construction. A bicycle reflector (provided
it reflects Infra-Red light [IR]) would be adequate,
[0414] The operation of the PPD may be described as follows: A
device equipped with an optical array sensor tracks the position of
the pointing device target by its reflectivity. It does this in a
way that avoids interference caused by extraneous light sources by
providing its own signature illumination, readily distinguishable
from other sources of illumination. In the preferred embodiment the
illumination should be IR and should be modulated in a way that can
be detected in the presence of other light sources. By taking
"pictures" of the room scene with the illumination alternately ON
and OFF, the difference between two consecutive values of a given
pixel will usually be proportional to the reflectivity, in the IR,
of the objects visible in the solid volume whose projection
corresponds to the particular pixel. (Exceptions are instances
where a light source is suddenly occluded or extinguished, or the
reverse.) Most objects are not particularly reflective in the IR,
but retro reflectors are extremely so, so they will normally
dominate this reflective image if present.
[0415] The reflective image can be processed in several ways to
detect motion: an algorithm may identify and track the most
reflective objects (represented by individual pixels or contiguous
groups that have higher reflectivity than the background) or an
auto correlation function can repeatedly try fits in each of the
cardinal directions, reporting motion when one of these produces a
better fit than the null correlation, or a change in the position
of the center-of-mass or centroid of the whole distribution could
be reported. The first method requires a much greater degree of
abstraction and computation, as abstracting identifiable "objects"
from the scene's background and maintaining an identification of
them as they morph and move is a nontrivial problem. The second
method also involves considerable computation, as the
autocorrelation function would require on the order of 5n (n is the
number of pixels) multiplications for each frame. The last method
requires only on the order of n multiplications. The three methods
would also appear to offer decreasing levels of precision, but at
least the last method would allow the contamination of the visible
scene with extraneous reflective objects to cause only gradual and
graceful degradation of sensitivity, as the amount of motion of the
PD target needed to move the centroid will increase as the
proportion of the reflectivity contributed by the target decreases.
Even this might be remedied by a long time-constant per-pixel or
"Fatigue" function. The properties of the second method remain to
be investigated, but it appears that, in the absence of a Fatigue
function, it would take very little background reflectivity to
overwhelm the contribution that would be made to the overall
autocorrelation by a moving target.
[0416] A high-powered LED is the obvious choice for the
illumination source, a commercial retro reflector for the target,
leaving the sensor as the least defined of the system components. A
low-resolution array as used in the HDNS-2000 optical mouse
pointing engine from Agilent (HP) would be sufficient if only
resolution sufficient to navigate web sites on the TV or to select
menu items or control buttons is the intended application. It is
doubtful this PD idea could ever be sufficiently accurate for
purposes like drafting or editing text. The IR source could also be
used as a Command and Control (C&C) output for home electronics
(like a stereo), or to transmit to a device like a PDA that is
equipped similarly (see the Set-Top Box Analysis, discussed
above).
[0417] Operating the system at a high image rate compared to the
120 (or 100) Hz flash rate of fluorescent and other discharge-based
lighting would allow this source of interference to be ignored, as
would synchronizing the image rate to the flash rate (or a sub
harmonic), or using a rate that matches closely enough that the
changing contribution of this illumination source frame-to-frame is
minor. Otherwise, some form of frame-wide intensity normalization
may be helpful.
[0418] The density of the pixels into which the scene is divided
largely determines the resolution of the PD, even though the
methods above, especially "object tracking" and "centroid
tracking," can obtain sub-pixel motion information. Even if an
image array with finer granularity than needed is used, summing the
outputs of blocks of pixels can reduce the computational load The
distribution of the projected pixels, and therefore of the PD
sensitivity in the field need not be uniform. If the peripheral
pixels were radially "stretched" by the optical system, or if the
effective pixels used by the system were composed of the unions of
non uniformly distributed physical sensor pixels, the sensitivity
at the center of the scene (presumably directly in front of the
display in use [probably a TV]) could be greater than towards the
periphery, allowing the pointing device to operate over a given
projected "area" (solid angle) with a given (peak) resolution with
reduced computational load.
[0419] Of the arrays listed, the TV-type camera ones are the most
prevalent, and are likely cheaper than the low-density ones due to
their volume manufacturing economics. The only low-density
array-based product that could be found that is in actual
production is the HP/Agilent HDNS2000 optical mouse sensor, which
would, with a change of the embedded code, serve the purposes of
this invention, except that this would probably change the price
considerably from what the mouse manufacturers are paying. On the
other hand, if it is possible to synchronize an illuminator with
the electronic shutter function of a normal digital camera, then,
provided the camera is sensitive to the IR illumination, one should
be able to extract cursor-control information from its output. The
camera would still be able to function as a "web cam," except that
there might be some objectionable artifacts introduced while the
tracking system of the PD is active.
[0420] Glossary
[0421] AGP (Accelerated Graphics Port) A high-speed graphics port
from Intel that provides a direct connection between the display
adapter and memory. AGP is faster than PCI, and only one AGP slot
is provided on AGP-equipped motherboards. The PCI slot that would
normally hold the display adapter can be used for another device.
The brown AGP slot is slightly shorter than the white PCI slot and
is located about an inch farther back.
[0422] AGP uses a 32-bit bus. The original AGP standard (AGP Ix)
provides a data transfer rate of 264 Mbytes/sec. AGP 2x is 528
Mbytes/sec. AGP 4x is 1 Gbytes/sec. AGP 8x is 2 Gbytes/sec.
[0423] BIOS: (Basic Input Output System) An essential set of
routines in a PC, which is stored on a chip and provides an
interface between the operating system and the hardware. The BIOS
supports all peripheral technologies and internal services such as
the real time clock (time and date). The BIOS tests the system and
prepares the computer for operation by querying its own small CMOS
memory bank for drive and other configuration settings. It searches
for other BIOS's on the plug-in boards and sets up pointers
(interrupt vectors) in memory to access those routines. It then
loads the operating system and passes control to it. The BIOS
accepts requests from the drivers as well as the application
programs. BIOS's must periodically be updated to keep pace with new
peripheral technologies. If the BIOS is stored on a ROM chip (ROM
BIOS), it must be replaced. Newer BIOSs are stored on a flash
memory chip that can be upgraded via software. See BIOS
upgrades.
[0424] On startup, the BIOS searches all peripheral controllers in
the system to obtain the current configuration, which it makes
available to the software.
[0425] Bluetooth: A wireless personal area network (PAN) technology
from the Bluetooth Special Interest Group, (www.bluetooth.com),
founded in 1998 by Ericsson, IBM, Intel, Nokia and Toshiba.
Bluetooth is an open standard for short-range transmission of
digital voice and data between mobile devices (laptops, PDAs,
phones) and desktop devices. It supports point-to-point and
multipoint applications
[0426] Bluetooth provides up to 720 Kbps data transfer within a
range of 10 meters and up to 100 meters with a power boost. Unlike
IrDA, which requires that devices be aimed at each other (line of
sight), Bluetooth uses omni directional radio waves that can
transmit through walls and other non-metal barriers. Bluetooth
transmits in the unlicensed 2.4 GHz band and uses a frequency
hopping spread spectrum technique that changes its signal 1600
times per second. If there is interference from other devices, the
transmission does not stop, but its speed is downgraded. The name
Bluetooth comes from King Harald Blatan (Bluetooth) of Denmark. In
the 10th century, he began to Christianize the country. Ericsson
(Scandinavian company) was the first to develop this
specification
[0427] CPU: (Central Processing Unit) The computing part of the
computer. Also called the "processor," it is made up of the control
unit and ALU. Today, the CPUs of almost all computers are contained
on a single chip.
[0428] DTMF: (Dual-Tone Multi Frequency) The type of audio signals
that are generated when you press the buttons on a touch-tone
telephone.
[0429] LED: (Light Emitting Diode) A display technology that uses a
semiconductor diode that emits light when charged. It usually gives
off a red glow, although other colors can be generated. It is used
in readouts and on/off lights in a myriad of electronic appliances.
It was the first digital watch display, but was superseded by LCD,
which uses less power.
[0430] LEDs are also used as a light source for fiber-optic
transmission. They are typically used with lower-bandwidth
multimode fibers.
[0431] NTSC: (National TV Standards Committee) A color TV standard
that was developed in the U.S. Administered by the FCC, NTSC
broadcasts 30 interlaced frames per second (60 half frames per 20
second, or 60 "fields" per second in TV jargon) at 525 lines of
resolution. The signal is a composite of red, green and blue and
includes an audio FM frequency and an MTS signal for stereo. NTSC
is used throughout the world including the U.S., Canada, Japan,
South Korea, and several Central and South American countries.
[0432] PAL: (Phase Alternating Line) A color TV standard that was
developed in Germany. It broadcasts 25 interlaced frames per second
(50 half frames per second) at 625 lines of resolution. Brazil uses
PAL M, which broadcasts 30 fps. PAL is used throughout Europe and
China as well as in various African, South American and Middle
Eastern countries. PAL's color signals are maintained
automatically, and the TV set does not have a user-adjustable hue
control.
[0433] RF: (Radio Frequency) The range of electromagnetic
frequencies above the audio range and below visible light. All
broadcast transmissions, from AM radio to satellites, fall into
this range, which is between 30 kHz and 300 GHz.
[0434] S-video: (Super-video) Recording and transmitting video by
keeping luminance (Y) and color information (C) on separate
channels. S-video uses a special 5-pin connector rather than the
common RCA phono plug. It is widely used on camcorders, VCRs and
A/V receivers and amplifiers. If S-video connectors are available
between the two devices you want to hook up, using an S-video cable
will improve transmission quality and the image at the receiving
end.
[0435] USB: (Universal Serial Bus) USE connects more than computers
and peripherals. It has the power to connect you with a whole new
world of PC experiences USB is your instant connection to the fun
of digital photography or the limitless creative possibilities of
digital imaging. A person can use USB to connect with other people
through the power of PC-telephony and video conferencing.
[0436] VGA: (Video Graphics Array) The minimum standard for PC
video display, which originated with IBM's PS/2 models in 1987. It
supports earlier CGA and EGA modes and requires an analog monitor.
VGA was initially 640.times.480 pixels with 16 colors, but non-IBM
vendors quickly boosted resolution and colors to so-called "Super
VGA" which was later standardized by VESA. All VGA display adapters
today start at 256 colors
[0437] IP telephony: The two-way transmission of audio over an IP
network. When used in a private intranet or WAN, it is generally
known as "voice over IP," or "VoIP." When the public Internet is
the transport vehicle, it is referred to as "Internet telephony,"
however; all three terms are used synonymously.
[0438] Private networks can provide from good to excellent quality,
matching that of the PSTN. Over the Internet, voice quality varies
considerably; however, protocols that support quality of service
(QoS) are expected to improve this condition. Nevertheless,
Internet telephony means free voice calls as long as sending and
receiving users have identical software that uses proprietary
techniques or compatible software that uses the H.323 standard.
They must also be willing to talk from their PCs and to prearrange
the times to talk. When users go online and launch their IP
telephony client software, the session and current IP address is
registered on a directory server on the Internet so that others may
contact them. Using Internet telephony service providers (ITSPs),
users can make PC-to-phone or phone-to-phone calls, using the
Internet as the network backbone. ITSPs have been initially popular
for international calls. Another IP telephony application that is
expected to take off is with call centers that can answer a
question directly from a Web site. This voice integration is
desired by users and vendors alike and is expected to flourish in
the 2002-2003 time frame. The IP protocol is either being used or
being considered for future network backbones by all the major
telecom carriers
[0439] 10/100 card: An Ethernet network adapter (NIC) that supports
both 10 BaseT (10 Mbps) and 100 BaseT (100 Mbps) access methods.
Most cards auto negotiate at startup, enabling them to run at the
higher speed if supported by the device they are connected to (hub
or switch).
[0440] PSTN--Plain Standard Telephone Network
[0441] ISDN--Integrated Services Digital Network
[0442] XDSL--Digital Subscriber Line
[0443] W-LAN--Wireless Local Area Network (WEE 802.11)
[0444] LAN--Local Area Network
[0445] WAN--Wide Area Network
[0446] STB--Set-Top Box
[0447] Set-Top Box Software
[0448] The set-top box software assists in providing the following
functions, some of which are described hereinafter: speech
recognition, speech synthesis, Internet surfing by using a browser,
E-mail, Voice over IP, Video over IP, support printer, subscriber
authentication, subscriber profile management, data package and
un-package, data encryption and decryption, data transmission and
communication, user interface, task dispatch and control, system
diagnostics and recovery, system initialization, load memory
residence programs, set-top box parameters, verifying system
hardware functions, and launch system start program.
[0449] Interactive user interface. The user interface includes two
types of media. One is voice input/output. The other is electronic
signal. It includes input from a keyboard, a mouse and a
controller; and output to a television, a monitor or any equipment
connected to output ports, such as a printer.
[0450] Authentication. Authentication may be carried out on both
the server side and on the user side. User side authentication will
initially come from the server and may be updated periodically. The
server side authentication may be carried out when a request is
initialed and every transaction may be verified. Simplified user
profile can be stored in the user side so that authentication can
be done in the user side to accelerate connectivity with
processing. Verification of a valid user input, like password,
should be done in the user side unless assistance is required.
[0451] Data encryption and decryption. Some data may be encrypted
before sending to and be decrypted after received from the
servers.
[0452] Data packaging and un-packaging. Data may be transmitted and
received in a standard package size and format.
[0453] User side communication mechanism to connect to a server and
send customer requests.
[0454] A web browser with special functions to display and retrieve
data from Internet.
[0455] A speech recognition program to accept some voice inputs.
Complicated recognition tasks may be implemented at server
side.
[0456] A dispatch program to control the behavior of all user
interfaces, including submitting and canceling customer
requests.
[0457] Output ports. Transmit data to connected equipment, such as
a printer if a printer is connected to the set-top box. System
diagnostics and recovery functions.
[0458] E-mail user interface for reviewing, composing, deleting
email, managing the address book, organizing e-mail in a virtual
mail box, printing an e-mail, allowing mail attachments, and so
on.
[0459] Create and modify customer favorites. Some utilities for
storing, accessing and changing downloaded and/or client side data;
teletex for customer alert or notice.
[0460] TV services. The client program will accept the traditional
TV services, support enhanced (interactive) TV programs, and have
the control ability over any other TV programs according to
customer requests. The traditional TV program or any audio/video
signals inputs to the set-top box can be sent to its output
directly.
[0461] Verbal Communication User Interface Components
[0462] The speech recognition system must include at least the
following components: a speech recognizer 1601 to convert a voice
input 1602 into text according to given vocabulary and grammar
models 1603 of the domain knowledge of a language, and a language
analyzer 1604 to extract meaning from text by syntactic parsing and
semantic interpretation of the text according to semantic rules
1605.
[0463] An expert system 1606 with a domain knowledge-base supports
the selection of desired actions from the meaning and issues
commands to the system to invoke proper actions 1607, such as
accessing data from a database system, communicating with other
systems, and reporting status of actions. The results of actions
are reported in voice communication by a voice synthesizer
1608.
[0464] The voice synthesizer must include the following components:
a text generator to construct a reply in text form according to the
status and results of executing commands and the text from; and a
text-to-speech synthesizer to convert the text reply into machine
generated speech according to pronunciation rules of the
language.
[0465] In order to improve the verbal communication user interface,
`virtual reality` technology is used.
[0466] C. Remote Controller
[0467] There are three different embodiments of remotes in the
system of the present invention. All are wireless and use IR or RF
signals or hybrid IR/RF frequency for communicating (transmitting
and receiving) with the Set-Top-Box, PC, TV, VCR, Stereo and
Camcorders.
[0468] 1. Universal Remote. This remote encompasses the following
components: wireless microphone and a speaker, wireless pointer
device, e.g. wireless mouse, wireless Controller for the TV, VCR,
stereo systems and camcorders.
[0469] 2. Global Remote. This remote encompasses the following
components: wireless microphone and speaker (for cordless phone
function), wireless pointer device, e.g. wireless mouse, wireless
controller for the TV, VCR, stereo systems and camcorders.
[0470] 3. Upgrade Remote. "Smart Remote". The Upgrade Remote
embodiment is in the form of a `Personal Digital Assistant` (PDA)
with a central process unit (CPU) and memory.
[0471] The remote includes two physical parts, a base and an
attachment. The two parts can be connected in following ways 1701:
IR, RF, Plug in or slide in.
[0472] The base includes the following functions: wireless
connection to the set-top box 1702 through IR and RF: [0473]
Functioning as a regular cordless telephone [0474] Wireless pointer
device, e.g. wireless mouse [0475] Wireless controller for the TV,
VCR, stereo system and camcorders [0476] Controls for surfing
Internet: Back, Forward, Search, Stop, Home, Refresh, Book Marks,
Address Book, Reload, and Arrow Keys [0477] Voice communication
capability with the set-top box.
[0478] The base contains the following physical buttons: adaptor
for the attachment to connect, connector to a set-top box 1703 for
battery recharge, buttons arranged for all functions
[0479] The attachment contains the CPU and the memory with battery
as power supply. Detached from the base, the attachment can work
independently with the functions: [0480] Scientific calculator
[0481] Address book and phone book with editing, storing data
functions [0482] Dictionary with searching and pattern matching
functions [0483] Scheduler with modifying, alerting functions
[0484] Calendar, for any years in the past and future, displayed by
day, month, or year.
[0485] The attachment contains the following physical parts and
buttons: connector to connect the base, LCD display, and an
alphanumeric input pad.
[0486] When the base and the attachment are connected by one of the
above mentioned connection mechanisms, additional functions are
available besides the functions provided in each separate part:
[0487] Switch controls for different functions (more details:
between TV mode and Internet mode) [0488] Number pad for telephone
functions [0489] Reduce sized alphanumeric pad for text editor
[0490] Data transmission from the attachment to the set-top box,
which is connected to the virtual environment.
[0491] Functional description of the Smart Control
[0492] The features in the Smart Control include, but are not
limited to, those shown in FIG. 4. Major functional blocks/features
are discussed below
[0493] The communications port labeled "To PDA" 1701 allows this
control's microcontroller 1704 to communicate with such a device
that may be used to contain e-mail addressed, telephone numbers,
and function-expanding software, possibly including Voice
Recognition ("VR"). The connection to the Personal Digital
Assistant ("PDA") from the microphone's A/D Converter 1705 (ADC)
allows this. This connection may be conceptual, mediated by the
microcontroller 1704, rather than actual and direct. The PDA may
also communicate (indirectly) with the Set-Top Box (STB) 1706 and
through the InfraRed (IR) transmit 1708 and receive 1709
blocks.
[0494] Digital 1710 and Analog 1711 RF communications allow the
device to control functions of the STB 1706, to serve as a
communications channel between the PDA and STB 1706, to be
controlled by the STB 1706, and to function as a wireless handset
for a land-line POTS telephone, internet telephone, or for VR
control. Enhanced functionality may incorporate the full functions
of a PCS or Cellular telephone into these blocks.
[0495] The IR transmitter 1708 allows control of existing
audio/video equipment 1712, 1713 and may be used in conjunction
with the STB 1706 to provide an on-screen cursor-control
function.
[0496] The IR receiver 1709 allows for programming the IR control
codes for equipment not supported by an internal library of
codes.
[0497] A display may be used for such functions as Caller ID, Last
Number Dialed, and system status. A more extensive display may be
incorporated into the PDA.
[0498] The speaker 1714 can be used both for allowing the STB's
1706 program to interact audibly with the user without disturbing
others present, in a "Remote Locate" beeping function, and when the
device is used as a telephone.
[0499] The microphone or headset 1715 can be used both for allowing
the user to interact verbally with the STB 1706 (through VR), and
when the device is used as a telephone, and for recording voice
messages to be sent by e-mail
[0500] The keyboard 1716 allows the composition of e-mail messages
and for system control.
Network Architecture
[0501] Set-Top Box Connectivity Considerations
[0502] Set-Top Box (STB) connectivity to the virtual environment's
POP (point of presence) could be achieved via the following
methods: [0503] (1) Classified by communication link: PSTN, analog
modem, ISDN, xDSL, cable, satellite, W-LAN (Wireless LAN), WAN,
LAN, WAP (Wireless Access Protocol) [0504] (2) Classified by
Applications service provider ("ASP") [0505] a. Connection directly
to ASP. Connection to local POP: Analog, ISDN, xDSL, cable, W-LAN,
WAN, LANAP [0506] b. Dial-up Networking to centralized ASP through
call forwarding: Analog, ISDN, xDSL, Cable, ISDN (toll-free),
Satellite, Cable, WAN, WAP [0507] c. Connection to centralized ASP:
Analog (toll-free), ISDN (toll-free), Satellite, Cable WAN, WAP
[0508] d. Connection to customer's own ASP and through Internet to
virtual environment; Analog, ISDN, XDSL, Cable, Satellite, W-LAN,
WAN, LAN, WAP [0509] e. Connection to ISP and through network to
virtual environment: Analog, ISDN, XDSL, Cable, Satellite, W-LAN,
WAN, LAN, WAP
[0510] There are several embodiments of server location
considerations for the virtual environment object of this
invention: [0511] Centralized model (all servers situated at the
same location) [0512] Distributed model (multiple servers serving
specific region or audience with different degrees of redundancy)
[0513] Distributed location with no backup at centralized location
[0514] Distributed location with some backup at centralized
location [0515] Distributed location with full backup at
centralized location (centralized mirroring) [0516] Distributed
location with backup at other regional locations (distributed
mirroring)
[0517] In addition, there are several Data Center considerations
for the different embodiments of the invention: [0518] Power
requirements [0519] Multiple sources of AC power (from different
power companies) UPS [0520] Generation Facilities [0521] Network
connectivity requirements [0522] Multiple fiber connections to
different Internet Access Providers [0523] High Availability
requirements [0524] Network connectivity [0525] Network equipment
[0526] Server farm [0527] Database backup and redundancy
requirements [0528] HVAC and Air Flow/Air Exchange requirements
[0529] Plant and facilities requirements
[0530] In addition to all of the foregoing, there are several
considerations that must be addressed in the different embodiments
of the invention such as location, staffing, and hours of
operation
[0531] Finally, in the different embodiments, there are several
server arrangement that may be considered such as: Development
Servers, Database Servers, QA Servers, Back office servers, and
Intranet/Internet Web servers.
[0532] Sales fees are earned by executing purchases by subscribers
from merchant partners and retailers, securing volume discounts
that will be sold in the environment with quality of service and
transactional support provided by a customer service staff.
Eventually, the virtual shopping zone will become an online
department store, offering any products desired by its members or
subscribers. The virtual shopping zone is not a mere conduit
between its members and vendors, earning a small transaction
fee.
[0533] There will be recurring special services and subscription
fees generated from customers who subscribe to a number of highly
individualized premium services and information and entertainment
options delivered by the environment.
[0534] Earn fees through the financing of purchases of all type of
consumer goods and financial products by members through the
environment (financial wallet and credit card fees. TABLE-US-00002
TABLE 1 Operation of the STB, VCR and TV is various different
hook-ups: Available Modes of Operation: * indicateIndicates mode
with STB in line, underline indicates TV
modewithpremium(derypted)channels. Cubic Box (RF IN The last entry
in a row is the tuner to use once the other settings in the Case:
Output: VCR setting assumed) row are established. 1 All Channels RF
Ch 3 (4) Baseband TVtuner (asAs Out + Baseband A/V In *
TVtuner="A/VIn" STBtuner though not A/V * TVtuner="A/VIn"
STBtunner="A/VIn" VCR tuner there) Out (standard) * TVtuner="A/VIn"
STB tuner = "3 (4)" VCR tuner 2 3 (4), RF Ch 3 (4) Baseband TV
tuner including Out + Baseband A/V In * TV tuner = "A/V In" STB
tuner Decrypted A/V * TV tuner = "A/V In" STB tuner = "A/V In" VCR
tuner = 3 (4) Cable Box tuner Channels Out (standard) * TV tuner =
"A/V In" STB tuner = "A/V In" VCR tuner = "A/V In" Cable Box tuner
but can be bypassed. 3 3 (4), RF Ch 3 (4) No * TV tuner = 3 (4) STB
tuner including Out + Baseband Baseband * TVtuner=3(4)
STBtuner="A/VIn" VCRtuner=3(4) CableBoxtuner Decrypted A/V A/V In *
TVtuner=3(4) STBtuner"A/V" VCRtuner="A/VIn" CableBoxtuner Channels
Out (standard) but can be bypassed. 4a 3 (4), RF Ch 3 (4) Baseband
TV tuner = 3 (4) VCR tuner = 3 (4) Cable Box tuner including Out +
Baseband A/V In * TVtuner=3(4) VCRtuner="A/VIn" STBtuner=3(4)
CableBoxtuner Decrypted A/V * TV tuner = 3 (4) VCR tuner = "A/V In"
STB tuner = "A/V In" Cable Box tuner Channels. Out (standard) TV
tuner = "A/V In" VCR tuner = 3 (4) Cable Box tuner *
TVtuner="A/VIn" VCRtuner="A/VIn" STBtuner=3(4) CableBoxtuner *
TVtuner="A/VIn" VCRtuner="A/VIn" STBtuner="A/VIn" CableBoxtuner 4b
3 (4), Rf Ch 3 (4) Baseband TV tuner = 3 (4) VCR tuner including
Out + Baseband A/V In * TV tuner = 3 (4) VCR tuner = "A/V In" STB
tuner Decrypted A/V * TVtuner=3(4) VCRtuner="A/VIn"
STBtuner="A/VIn" CableBoxtuner Channels Out (standard) TV tuner =
"A/V In" VCR tuner but can be * TV tuner = "A/V In" VCR tuner =
"A/V In" STB tuner bypassed. * TVtuner="A/VIn" VCRtuner="A/VIn"
STBtuner="A/VIn" CableBoxtuner 4c 3 (4), RF Ch 3 (4) Baseband TV
tuner = 3 (4) VCR tuner including Out + Baseband A/V In *
TVtuner=3(4) VCRtuner="A/VIn" STBtuner=3(4) CableBoxtuner Decrypted
A/V * TVtuner=3(4) VCRtuner="A/VIn" STBtuner="A/VIn" CableBoxtuner
Channels Out (standard) TV tuner = "A/V In" VCR tuner but can be *
TVtuner="A/VIn" VCRtuner="A/VIn" STBtuner="3(4) CableBoxtuner
bypassed. * TVtuner="A/VIn" VCRtuner="A/VIn" STBtuner="A/VIn"
CableBoxtuner
[0535] TABLE-US-00003 TABLE 2 STB Specifications* Component Rating
or Type (minimal) Comment Processor X88 (Pentium Celeron or Duron)
- class, minimum 800 MHz Main Memory 128 MB Hard Disk 20 GB
Probably as smaller available. Removable DVD CD would be Disk Drive
adequate, but a DVD has marketing advantages, A Floppy drive is out
of the question; the User would think of the machine as a scary
computer. Removable DVD-ROM w/ OS, Disk operating code. GPU NVIDIA
GeForce MX GPU Memory 32 MB Uncommitted 10/100 Base-T, I/O two USB
IR IR receiver for 40 kHz Communications kHz consumer band, IR
blaster port for transmitting. Peripherals Modified Cordless
Passive Pointing Device Telephone for Voice (PPD) for later, IR
Remote Remote Control (and Control, or IR Remote 56 k modem), IR
Control w/RF, Voice added keyboard (40 kHz, to be optional. not
IRDA). *Note that these are off-the-shelf components
[0536] TABLE-US-00004 TABLE 3 Phone VRC Commands Command Name
Accomplishes HookSw(R) R = 0: Just report bookswitch status. 1 R =
1: Open the bookswitch. R = 2: Close the bookswitch. R = 3: Restore
local bookswitch control. AudioPipeOpen (P) Create and Open an
isochronous pipe for 1 audio. Not all of these may be possible, not
all may be possible simultaneously, but the first two are
necessary, P = 1: From HS microphone to computer. P = 2: From
computer to HS speaker. P = 3: From phone line to computer (remote
control). P = 4: From computer to phone (remote control). P = 5:
From speakerphone microphone to computer. P = 6: From computer to
speakerphone speaker. AudioPipeClose (P) Similar to above. 1
DataPipeOpen ( ) For (56 k) modem use. 2 DataPipeClose ( ) For (56
k) modem use. 2 Various Modem For (56 k) modem use. 2 Commands
SpeedDialReport (N) Returns the contents of a Speed-dial entry. 4
SpeedDialSet (N, [n]) Sets the contents of a Speed-dial entry. 4
(Maintain the list on the computer, but it works when the computer
is off Magint) GetCallIDs ( ) Returns contents of all, if any new
Caller 4 ID entries. GetLastNumbers ( ) Returns contents of all, if
any new dialed 4 number entries. DialDigit (n, t) Generate a DTMF
digit (including pause) for 2 () ms (dial). (It may also be
possible to use this to invoice other "phone tones" like ringback,
busy, etc. if n > 16.) DialNumber ([a]) Generate a sequence of
DMTF digits 3 (including pause) (dial). MakeRingTone (t) Activate
the electronic ringer, either default 3 cadence (t = 0) or for a
multiple of 500 ms (t = integer). InvoiceButton (N) IIC, for volume
control and stuff. 3 SetSpkrVol (n) Speakerphone volume 4 SetEPVol
(n) Earpiece volume 4 Reset ( ) Always handy.
[0537] Thus, there has been shown and described a method and
apparatus for delivering a virtual reality environment which
fulfills all the objects and advantages sought therefor. The
invention is not limited to the precise configuration described 5
above. While the invention has been described as having a preferred
design, it is understood that many changes, modifications,
variations and other uses and applications of the subject invention
will, however 1 become apparent to those skilled in the art without
materially departing from the novel 10 teachings and advantages of
this invention after considering this specification together with
the accompanying drawings. Accordingly, all such changes,
modifications, variations and other uses and applications which do
not depart from the spirit and scope of the invention are deemed to
be covered by this as defined in the following claims.
* * * * *