U.S. patent number 7,191,211 [Application Number 10/238,643] was granted by the patent office on 2007-03-13 for portable high speed internet access device priority protocol.
This patent grant is currently assigned to Raja Tuli. Invention is credited to Raja Tuli.
United States Patent |
7,191,211 |
Tuli |
March 13, 2007 |
Portable high speed internet access device priority protocol
Abstract
The present invention aims to provide a system that allows
multiple users operating a PDA (Personal Digital Assistant) to
access the Internet or the World Wide Web (WWW), and to be able to
view and interact with these images remotely on a display screen.
The invention provides a client and server system to facilitate
multiple PDA users simultaneously on a single server, which has
multiple virtual machines contained within, with each virtual
machine containing a web browser. The invention relates to the
priority in which packets or blocks of information are transferred
to and displayed on the screen of the PDA device. The invention
also relates to the interlacing of blocks and the construction of
such blocks displayed on the PDA screen. The invention further
relates to the sending, receiving, and acknowledgement of these
blocks between the PDA and server.
Inventors: |
Tuli; Raja (Montreal,
CA) |
Assignee: |
Tuli; Raja (CA)
|
Family
ID: |
31991006 |
Appl.
No.: |
10/238,643 |
Filed: |
September 11, 2002 |
Prior Publication Data
|
|
|
|
Document
Identifier |
Publication Date |
|
US 20030041106 A1 |
Feb 27, 2003 |
|
Related U.S. Patent Documents
|
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
Issue Date |
|
|
09677857 |
Oct 3, 2000 |
6842777 |
|
|
|
Current U.S.
Class: |
709/203;
707/E17.121; 709/247; 715/744; 709/217 |
Current CPC
Class: |
G06F
16/9577 (20190101); H04L 1/1867 (20130101); H04L
29/08846 (20130101); H04L 67/2823 (20130101); H04L
67/2828 (20130101); H04L 67/2842 (20130101); H04L
67/327 (20130101); H04L 69/329 (20130101); G06F
3/14 (20130101); G09G 2310/0224 (20130101); G09G
2310/04 (20130101); G09G 2340/02 (20130101); G09G
2340/0407 (20130101); G09G 2340/0428 (20130101); G09G
2340/045 (20130101); G09G 2350/00 (20130101); H04L
67/289 (20130101); G09G 5/346 (20130101) |
Current International
Class: |
G06F
15/16 (20060101) |
Field of
Search: |
;709/200-203,208,217-219,247 ;715/744,748-749 ;370/466-467 |
References Cited
[Referenced By]
U.S. Patent Documents
Foreign Patent Documents
|
|
|
|
|
|
|
0 833 260 |
|
Jan 1998 |
|
EP |
|
0 889 402 |
|
Jan 1999 |
|
EP |
|
0 889 636 |
|
Jan 1999 |
|
EP |
|
890922 |
|
Jan 1999 |
|
EP |
|
1 001 613 |
|
May 2000 |
|
EP |
|
1026578 |
|
Aug 2000 |
|
EP |
|
1043876 |
|
Oct 2000 |
|
EP |
|
1109113 |
|
Jun 2001 |
|
EP |
|
2347766 |
|
Sep 2000 |
|
GB |
|
WO 97 30556 |
|
Aug 1997 |
|
WO |
|
WO 97 38389 |
|
Oct 1997 |
|
WO |
|
WO 98 40842 |
|
Sep 1998 |
|
WO |
|
WO 98 43177 |
|
Oct 1998 |
|
WO |
|
WO 99 09658 |
|
Feb 1999 |
|
WO |
|
WO 99/48007 |
|
Sep 1999 |
|
WO |
|
WO 0033232 |
|
Jun 2000 |
|
WO |
|
WO 01 09836 |
|
Feb 2001 |
|
WO |
|
WO 02/08878 |
|
Mar 2002 |
|
WO |
|
WO 02/057951 |
|
Jul 2002 |
|
WO |
|
WO 00/18054 |
|
Mar 2003 |
|
WO |
|
Other References
International Search Report, PCT/US 01/03382, 1 pages, 2001. cited
by other .
International Search Report, PCT/CA 01/00114, 3 pages, 2001. cited
by other .
International Search Report, PCT/CA 01/00126, 3 pages, 2001. cited
by other .
International Search Report, PCT/CA 01/00170, 4 pages, 2002. cited
by other .
International Search Report, PCT/CA 01/00169, 3 pages, 2002. cited
by other .
International Search Report, PCT/CA 02/00048, 5 pages, 2003. cited
by other .
Tagawa Norio, "Data Relay System and Data Repeater Applied to the
System", Patent Abstracts of Japan, vol. 1997, No. 10, Jun. 24,
1997. cited by other .
Armando Fox et al, "Experience with Top Gun Wingman: a proxy-based
graphical web browser fro the 3Com PalmPilot", Middleware, IFIP
International Conference on Distributed Systems, Platforms and Open
Distributed Processing, Sep. 1998, pp. 407-424. cited by other
.
Richard Han, et al., "Dynamic Adaptation in an Image Transcoding
Proxy for Mobile Web Browsing", IEEE Personal Communication, Dec.
1998, pp. 8-17. cited by other .
Timothy Bickmore, et al., "Web Page Filtering and Re-Authoring for
Mobile Users", The Computer Journal, vol. 42, No. 6, 1999. cited by
other .
Tristan Richardson, Quentin Stafford-Fraser, Kenneth R. Wood &
Andy Hopper, "Virtual Network Computing", IEEE Internet Computing,
vol. 2 No. 1, Jan./Feb. 1998 pp. 33-38. cited by other .
Citrix Systems, Inc.: "Citrix Announces New Product and Product
Enhancements that Speed Web Application",
citrix.com/press/news/releases/20000905.sub.--product.sub.--enhance.asp,
Sep. 5, 2000. cited by other .
Boutell, T. et al., "PNG (Portable Network Graphics) Specification
Version 1.0", RFC 2083, pp. 1-102, Mar. 1997. cited by other .
Halfhill, T.R., "Good-Bye GUI . . . Hello, NUI," BYTE Magazine,
byte.com, vol. 22, No. 7, pp. 60-64, 66, 68, 70, and 72, Jul. 1997.
cited by other .
Masinter, L., "Returning Values from Forms: multipart/form-data,"
RFC 2388, pp. Aug. 1-9, 1998. cited by other .
Jao, C.S. et al., "The display of photographic-quality images on
the Web: a comparison of two technologies," IEEE Transactions on
Information Technology In Biomediciene, vol. 3, iss. 1, pp. 70-73,
Mar. 1999. cited by other .
Kaljuvee, O. et al., "Efficient Web form entry on PDAs" ACM
Proceedings of the tenth International Conference on WWW, ACM
Press, ISBN: 1-58113-348-0, pp. 663-672, May 2001. cited by other
.
Selected pages from AT&T Wireless, Bell Atlantic Mobile, and
Nextel, as delivered through the Wayback Machine, web.archieve.org.
cited by other .
Lewis, Ted, "Information Appliances: Gadget Netopia" Computer, vol.
31, issue 1, Jan. 1998, pp. 59-68. cited by other .
Cimini, Leonard J., Jr., et al. "Advanced Cellular Internet Serivce
(ACIS)", IEEE communication Magazine, vol. 36, Issue 10, Oct. 1998,
pp. 150-159. cited by other .
Oliphant, Malcolm W. "The Mobile Phone Meets the Internet", IEEE
Spectrum, vol. 36, Issue 8, Aug. 1999, pp. 20-28. cited by other
.
Lind, R., et al. "The Network Vehicle--A Glimpse into the Future of
Mobile Multi-Media", IEEE ASE Systems Magazine, Sep. 1999, pp.
27-32. cited by other .
Gbaguidi, Constant, et al. "A programmable Architecture for the
Provision of Hybrid Services", IEEE communications Magazine, vol.
27, Issue 7, Jul. 1999, pp. 110-116. cited by other .
Narayanaswamy, Shankar, et al. "User Interface for a PCS Smart
Phone", IEEE Int.. Conference on Multimedia Computing and Systems,
vol. 1 Jun. 1999, pp. 777-781. cited by other .
Rumba Technical Bulletin, "Configuring RUMBA LPR & LPD
(16-bit)," 1996, from the web:
156.27.8.200/ts.sub.--rumba/solution/pdf/TCPIP/4002.pdf, pp. 1-11.
cited by other .
International Search Report, PCT/CA 02/00133, 3 pages, 2003. cited
by other .
International Search Report, PCT/CA 02/00048, 3 pages, 2003. cited
by other .
International Search Report, PCT/CA 01/01057, 2 pages, 2003. cited
by other .
Joel F. Bartlett, "Experience with a Wireless World Wide Web
Client", Mar. 1995. cited by other .
International Search Report, PCT/CA 03/01840, 3 pages, 2004. cited
by other .
International Search Report, PCT/CA 03/01326, 3 pages, 2004. cited
by other.
|
Primary Examiner: Barot; Bharat
Attorney, Agent or Firm: Blakely, Sokoloff, Taylor &
Zafman LLP
Parent Case Text
This application is a continuation in part of application Ser. No.
09/677,857 filed on Oct. 3, 2000, now U.S. Pat. No. 6,842,777.
Claims
What is claimed:
1. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the virtual display and
sends it to a remote device capable of receiving, decompressing,
storing into a remote device memory and displaying a portion of the
virtual display to a user such that a first priority of sending
information is dependent upon the portion of the virtual display
being displayed by the remote device, and such that a second
priority of sending information is dependent on a non-displayed
portion of the virtual display surrounding the displayed portion of
the virtual display up to a fixed distance.
2. The host computer as claimed in claim 1 such that the priority
changes if a relative position of the two displays changes.
3. The host computer as claimed in claim 1 such that a third
priority is to download the rest of the rendered virtual
display.
4. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the compresses the virtual
display and sends it to a remote device capable of receiving,
decompressing, storing into a remote device memory and displaying
the virtual display to a user such that a priority of sending
information depends on an intent of the user to move a remote
device display in a particular direction with respect to the
virtual display.
5. A host computer which receives information from an outside
source, contains a browser with a window which renders this the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the virtual display and
sends it to a remote device capable of receiving, decompressing,
storing into a remote device memory and displaying the virtual
display to a user such that a first priority is to download
information of primarily text portions of a lower bit depth and a
picture portion at a higher bit depth.
6. The host computer as claimed in claim 5 such that the picture
portion is downloaded first at a lower resolution.
7. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the virtual display and
sends it to a remote device capable of receiving, decompressing,
storing into a remote device memory and displaying the virtual
display to a user such that only a limited amount of information is
downloaded to the remote device until the remote device sends the
host computer an acknowledgement of receiving the limited amount of
information.
8. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the virtual display and
sends it to a remote device capable of receiving, decompressing,
storing into a remote device memory and displaying the virtual
display to a user such that no information is sent to the remote
device until the browser has fully rendered the information in the
host computer memory.
9. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the compresses the virtual
display and sends it to a remote device capable of receiving,
decompressing, storing into a remote device memory and displaying
the virtual display to a user such that a priority of download
depends on a past history of sending information.
10. A host computer which receives information from an outside
source, contains a browser with a window which renders the
information onto a virtual display in a host computer memory, the
host computer having a software program which reduces the color
depth of the virtual display, compresses the virtual display and
sends it to a remote device capable of receiving, decompressing,
storing into a remote device memory and displaying the virtual
display to a user such that some information comprising the remote
device display area is first sent to the remote device for display,
and more information is sent at a later time to display all
information within the remote device display area.
Description
BACKGROUND OF THE INVENTION
The invention relates to a host computer system or server, which
has a web browser running on it, and the display of the web browser
is transmitted via modem and received by a cellular phone connected
to a PDA device which displays the image. In particular, the
invention consists of multiple virtual machines which are contained
in the server, and each contains a browser which has applications
running in them. Multiple clients are represented by software,
which sends the display of each virtual machine to the remote PDA
device to be displayed, and also relays information back to the
virtual machine from the PDA device. The invention relates to the
priority in which packets or blocks of information are displayed on
the screen of the PDA device. The invention also relates to the
interlacing of blocks and the construction of such blocks displayed
on the PDA screen. The invention further relates to the sending,
receiving, and acknowledgement of these blocks between the PDA and
server.
SUMMARY OF THE INVENTION
The present invention relates to multiple portable high speed
Internet access PDA (Personal Digital Assistant) devices that can
access the Internet and World Wide Web as wireless devices,-using a
client and server system to facilitate multiple PDA users
simultaneously on a single server.
A Web server connected to the Internet contains a virtual browser
that takes the image displayed in the browser, and converts this
image into a bit map which is compressed, and communicates via
telephone lines to a cellular telephone. The cellular telephone is
connected to a high speed Internet access device commonly referred
to as a PDA (Personal Digital Assistant) which is comprised of a
display screen, battery and related micro-electronics. This enables
the PDA to receive, decompress and view the bit-map image sent from
the virtual browser, and more importantly, through cellular phone
connectivity to be able to input data or commands from the PDA
directly onto the server. The host computer or server receives
vector information or compressed data in the form of HTML, JPEG,
etc., which is displayed on a web page. The virtual browser
virtually displays a virtual image on the server. That image, in
whole or in parts, is recompressed and sent to the PDA. In
particular, the host computer contains an RDP (Remote Data
Protocol) server which has multiple virtual machines contained
within, with each virtual machine containing a web browser.
Multiple RDP clients interact with the virtual machines with a
dedicated virtual machine for each client on the server. Each
client is represented by software, which sends the display of the
virtual machine to a single remote PDA device via a dedicated modem
port. This modem port allows two way communication between a single
PDA and a dedicated virtual machine on the server, via a single RDP
client. By implementing multiple virtual machines and multiple RDP
clients with multiple modem ports, it is now possible to
communicate with multiple users of PDA devices on the RDP server to
facilitate Internet browsing, electronic message communication,
etc. The RDP client relays information received via a modem port
from the PDA, such as mouse clicks or keyboard commands, to the
application program in the virtual machine, which is then processed
and a refreshed display sent back to the PDA via the same dedicated
RDP client and modem port. The browser on a virtual machine relays
display information to the dedicated RDP client such as bitmap
files, vector files, commands, buffer information, etc. The RDP
client then rasterizes some of the information by drawing it into
memory and then proceeds to break up this file into smaller blocks
of information. These blocks are compressed and sent to the PDA
device through a dedicated modem port connected to each RDP client.
The PDA would then receive, decompress and assemble the blocks of
information in the original order as first received by the RDP
client before the RDP client breaks up into smaller blocks. The
browser running in each virtual machine rasterizes most of the
information which is sent to the RDP client such as text, etc., but
the RDP client may have to rasterize other information such as
blocks, etc.
The PDA sends specific data to the virtual machine informing of the
current location of the displayable area of the PDA screen with
respect to the larger image sent from the browser. This is
necessary, as the area displayed by the PDA is smaller than the
displayable image on the browser window. The RDP client would send
compressed blocks of data representing the image to be displayed in
order of priority, such that the first blocks sent to and
decompressed by the PDA are in the displayable area of the PDA,
which is the current area where the user is viewing. The PDA would
then decompress blocks surrounding the displayed area in a
particular sequence, such as left to right across rows, and store
the image in internal memory. The PDA assembles blocks of the image
in a virtual page, which comprises the entire image, thus enabling
the user to access any part of the present image without
communicating constantly with the RDP client. Hence, as the user
scrolls across the image in any direction, the blocks of data
comprising the image would already be decompressed and assembled
priority-wise for instant viewing. Blocks of the image in closer
proximity to the displayed area of the PDA screen get decompressed
and stored into internal memory on higher priority than blocks
further away. The PDA continuously sends its current location to
the virtual machine on the server to keep getting refreshed data
instantaneously as the location changes by the user scrolling or
sending commands. This enables a rapid refresh rate of the
displayed image especially when scrolling, as areas surrounding the
displayed image would be decompressed and already stored in
memory.
A mouse click or any keyboard command is given priority in
communication between the PDA and the RDP server. Such actions from
the PDA are sent instantaneously, interrupting the current
activities and this action directed to the web browser on the
dedicated virtual machine, which sends a refreshed image back to
the PDA device. This feature allows the user to have rapid response
to commands. If the user initiates a mouse click or keyboard
command on the PDA that does not change the displayed image on the
RDP server, then the original activities are continued almost
instantly with minimum interruption. However, if the image is
changed and refreshed on the RDP server, any old image being sent
is stopped and the new image is sent immediately to the PDA if it
is a full screen image occupying all or part of the PDA display
screen. If the refreshed image sent occupies a part of the PDA
display screen and also areas outside the display screen, then the
portion of the refreshed image that is displayed on the PDA display
screen is sent first to the PDA, and blocks of the image are sent
to the PDA which are decompressed and stored in internal memory, in
order of priority closest to the displayed image, as previously
described. Hence the PDA would assemble blocks of the image in the
virtual page, which comprises the entire image, starting first with
the current location of the PDA display screen then areas around
it.
A beacon is sent form the PDA device to the RDP client and
vice-versa many times per minute to confirm that a connection is
established and maintained, for the duration of use. In the
instance a user is disconnected, either the PDA or the RDP client
will not receive a beacon and a time-out will be initiated, whereby
both the PDA and RDP client disconnect and then reconnect. The user
would still be able to view the present image and scroll around it
as this image would already be decompressed in order of priority
and stored in internal memory. The PDA would indicate in a message
area that a reconnect sequence was initiated, and the status of
this connection to the same RDP client as before, which would
refresh the PDA with the image if it has changed once reconnection
is established. Similarly, the PDA would continue downloading
blocks of information after being reconnected if a disconnection
interrupted this operation. A beacon is not necessarily sent from
the RDP client at times when it is sending information to the PDA.
As long as the PDA is receiving information such as a refreshed
image, or a beacon, it knows a connection is established.
Consequently, when there is no new activity in the virtual machine,
the RDP client must send a beacon to the PDA to confirm the
connection is established. The PDA device must always send beacons
to the RDP client, as commands are not sent frequently from the PDA
and only for a short duration when sent.
An error protocol is implemented to verify that all information
blocks are received and can be decompressed successfully. As
previously described, the image displayed in the virtual browser in
the virtual machine on the RDP server is broken down into smaller
blocks of information and compressed and transmitted to the PDA
device. The blocks are then assembled in correct sequence,
decompressed and stored in internal memory on the PDA. Each block
of information received is acknowledged by the PDA, which sends a
signal confirming that each block is successfully received. Hence,
the RDP client can monitor the successful decompression of all
blocks of information sent to the PDA, and would know when a block
is not acknowledged. The RDP client would also inform the PDA of
the number of transmitted blocks of information, with each block
identified numerically. The PDA also initiates another error
protocol when a block of information received can not be
successfully decompressed and stored in memory. In this case, the
PDA would send an error message to the RDP client informing which
block of information needs to be sent again, and the RDP client
would send this block after it has completed sending the current
block of information. The RDP client would monitor the
acknowledgement of all blocks of information successfully
decompressed, including blocks sent again after receiving error
messages from the PDA device.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is described in more detail below with respect to an
illustrative embodiment shown in the accompanying drawings in
which:
FIG. 1 illustrates Prior Art whereby a PDA device is connected to a
cellular phone, which communicates wirelessly to a Host
Computer.
FIG. 2 illustrates Prior Art of a Remote Data Protocol system.
FIG. 3 illustrates the displayable area of the PDA device with
respect to portions of the image, which are sequentially
decompressed prior to viewing in accordance with Prior Art.
FIG. 4 illustrates the Remote Data Protocol in accordance with
Prior Art.
FIG. 5 illustrates a block diagram of the virtual page with respect
to the viewing area of the PDA in accordance with Prior Art.
FIG. 6 illustrates the error protocol as blocks of information are
sent from the RDP client to the PDA device in accordance with Prior
Art.
FIG. 7 illustrates communication methods between remote devices and
host computers in accordance with Prior Art.
FIG. 8 illustrates a wireless communication method between the PDA
and the Host Computer in accordance with Prior Art.
FIG. 9 illustrates the transfer of packets of data from a web page
on the server to the PDA, in accordance with the present
invention.
FIG. 10 illustrates blocks of data received at the PDA in an
instance where the user scrolls to a new location on the present
web page, demonstrating the priority of packet downloading, in
accordance with the present invention.
FIG. 11 illustrates the line composition of each block of data, in
accordance with the present invention.
FIG. 12 illustrates the subdivision of each block of data into "a"
blocks and "b" blocks which when interlaced produce the entire
block of FIG. 11, in accordance with the present invention.
FIG. 13 illustrates a first drawing sequence of each block as the
PDA receives the "a" blocks of data from the PDA, in accordance
with the present invention.
FIG. 14 illustrates a second drawing sequence as the "b" blocks are
received at the PDA and inserted in their correct location, in
accordance with the present invention.
FIG. 15 illustrates activities conducted by the PDA during defined
periods in accordance with priorities established, in accordance
with the present invention.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
To facilitate description, any numeral identifying an element in
one figure will represent the same element in any other figure.
The principal embodiment of the invention aims to provide a system
that allows multiple users operating PDA (Personal Digital
Assistant) devices similar to a palm top computer to access the
Internet or the World Wide Web (WWW), as demonstrated in FIG. 4, as
previously disclosed. Prior Art relied upon provides a RDP (Remote
Data Protocol) client and server system to facilitate multiple PDA
users simultaneously on a single server.
Prior Art is disclosed in FIG. 1 where Microsoft Remote Data
Protocol is demonstrated, using a main RDP server 1 in which
virtual machines 2 exist capable of running multiple application
programs 3. Each virtual machine 2 is connected to a dedicated
terminal 5 or client on a network system 4, which displays the
image of the virtual machine. The terminals 5 on the network allow
input of keyboard 6 and mouse 7 commands to the RDP server 1, with
all data processing done on the server and displayed on the
terminals. In this method of Prior Art, the RDP clients are the
terminals on the network. In the present invention however, the RDP
clients are represented by software which interact between the
virtual machines on the RDP server and the PDA devices which
display the image of the RDP clients, in a completely different
fashion.
A general description of the Prior Art is disclosed in FIG. 2 with
further reference to Prior patent application Ser. Nos. 09/496,172,
now U.S. Pat. No. 7,068,381, 09/501,585, now abandoned, 09/504,809,
now U.S. Pat. 6,874,009, 09/504,808, now U.S. Pat. No. 6,690,403,
09/504,807, now U.S. Pat. 6,633,314, 09/677,857, now U.S. Pat. No.
6,842,777, and 09/650,412, still pending. A host computer 8 is
depicted which is connected to the Internet, and that host may also
be a Web server. Running in the host computer, is a Web server
program 9. When a remote user 10 requests to view a Web page (or
electronic message etc.) the Web server software receives HTML,
JAVA, or other types of information and transmits this information
to another software, the Browser Translator 11. This software
translates the information, (i.e. the entire image comprising
graphics and text) received in the form of HTML, Java, etc. (as
information may be gathered from different sources) and translates
it to a black and white bit map or raster image. In another
embodiment, the software translates the information into a raster
or color image. The image contains the information that would
normally be displayed on a single Web page. The translation program
therefore, also acts as a virtual browser. The cellular telephone
12 of FIG. 2 is connected to the high speed internet access device
13 of the invention commonly referred to as a PDA (Personal Digital
Assistant) which is comprised of a display screen 14, battery and
related micro-electronics. This enables the PDA to receive,
decompress and view the bit map image sent from the virtual
browser, and more importantly, through cellular phone connectivity
to be able to input data from the PDA directly onto the server 8.
In particular, the host computer or server of FIG. 2 and FIG. 4
receives vector information or compressed data in the form of HTML,
JPEG, etc., which is displayed on a web page. The virtual browser
virtually displays a virtual image on the server by rasterizing the
image, or decompressing parts of the image and putting it into
memory. That image, in whole or parts, is recompressed and sent to
the PDA. The recompressed data format sent to the PDA, is not
necessarily in the same format as the compressed data format first
received by the server. For example, the incoming data from a Web
page may be in the form of JPEG which is decompressed and displayed
on the virtual browser. This data is recompressed and sent to the
PDA but can be in the form of TIFF G4 or other formats, and not
necessarily JPEG as initially received.
Another embodiment of the invention involves the server receiving
vector information such as HTML or text and then rasterizing it to
bit map format. It can then be shown in memory through the virtual
browser and is recompressed through a "loss less" method and sent
to the PDA.
The information is received by the device 13 in FIG. 2, which has
the ability to display a monochrome or color image 15, in its
display window 14. The information is decompressed and displayed in
the order of priority such that part of the image 17 of FIG. 3,
which substantially or completely covers the displayable area 14 of
the device, is decompressed and displayed first and then
sequentially the portions 18, 19 and 20 of the image are
decompressed, and stored in an internal memory of the device to be
displayed later when the user scrolls up, down, or sideways to
these parts of the image.
Prior application Ser. No. 09/677,857, now U.S. Pat. No. 6,842,777
demonstrates FIG. 4 to contain the host computer 21 which contains
an RDP server 22 which is a software unit having multiple virtual
machines 23 contained within, with each virtual machine containing
a web browser 24. Multiple RDP clients 25 interact with the virtual
machines with a dedicated virtual machine for each RDP client. Each
RDP client 25 is represented by software, which sends the display
of each virtual machine 23 to a single remote PDA device 26 via a
dedicated modem port 27. Each modem port 27 allows a two-way
communication between a single PDA 26, connected to a cellular
phone 64, and a dedicated virtual machine 23 on the RDP server, via
a single dedicated RDP client 25. By implementing multiple virtual
machines and multiple RDP clients with multiple modem ports, it is
now possible to communicate with multiple users of PDA devices on
the RDP server to facilitate Internet browsing, electronic message
communication, etc. The RDP client 25 relays information received
from the PDA 26, such as mouse clicks or keyboard commands, via a
modem port 27 to the Browser 24 in the virtual machine 23, which is
then processed and a refreshed display sent back to the PDA via the
same dedicated RDP client and modem port. The browser on each
virtual machine relays display information to the dedicated RDP
client such as bitmap files, vector files, commands, buffer
information, etc. The RDP client then rasterizes some of the
information by drawing it into memory and then proceeds to break up
this file into smaller blocks of information. These blocks are
compressed and sent to the PDA device through a dedicated modem
port connected to each RDP client, as further illustrated in FIG.
5. The PDA would then receive, decompress and assemble the blocks
of information in the original order as first received by the RDP
client before the RDP client breaks up into smaller blocks. The
browser running in each virtual machine rasterizes most of the
information which is sent to the RDP client such as text, etc., but
the RDP client may have to rasterize other information such as
blocks, etc.
The PDA 26 of FIG. 4, in accordance with Prior Applications
mentioned, sends specific data to the virtual machine 23 informing
of the current location of the displayable area 14 of the PDA
screen with respect to the larger image or virtual page 28 sent
from the browser to the RDP client then to the PDA, as illustrated
in FIG. 5. This is necessary, as the area displayed 14 by the PDA
is smaller than the displayable image on the RDP client, referred
to as the virtual page 28. The dedicated RDP client would send
compressed blocks of data representing the image to be displayed in
order of priority, such that the first blocks sent to and
decompressed by the PDA are in the displayable area 14 of the PDA,
which is the current area where the user is viewing. The PDA would
then decompress blocks surrounding the displayed area 14 in a
particular sequence, such as left to right across rows, and store
the image in internal memory. The PDA assembles blocks of the image
in a virtual page 28 stored in the PDA's internal memory, which
comprises the entire image, thus enabling the user to access any
part of the present image without communicating constantly with the
RDP client. Hence, as the user scrolls across the image in any
direction, the blocks of data comprising the image would already be
decompressed and assembled priority-wise for instant viewing.
Blocks of the image in closer proximity to the displayed area of
the PDA screen get decompressed and stored into internal memory on
higher priority than blocks further away. To explain in detail by
referring further to FIG. 5, the image displayed on the PDA screen
14 can be comprised of blocks 48, 49, 50, 53, 54 & 55, which
are sent from the RDP client first to the PDA, decompressed and
then stored into memory first on the PDA. Then blocks 42, 43, 44,
45, 47, 52, 57, 58, 59 & 60 would be sent immediately after in
that order from the RDP client to the PDA, to be decompressed and
stored in the PDA's internal memory. This enables a rapid refresh
rate of the displayed image especially when scrolling, as areas
surrounding the displayed image would be decompressed and already
stored in memory. The PDA continuously sends its current location
to the RDP client to keep getting refreshed data instantaneously,
should the location change by the user scrolling around or outside
the virtual page 28, to enable the RDP client to always have the
viewing area and surrounding blocks sent to the PDA.
A mouse click or any keyboard command on the PDA is given priority
in communication between the PDA and the RDP server. Such actions
from the PDA are sent instantaneously, interrupting the current
activities and this action directed to the web browser on the
dedicated virtual machine, which sends a refreshed image back to
the PDA device through the RDP client. This feature allows the user
to have rapid response to commands. If the user initiates a mouse
click or keyboard command on the PDA that does not change the
displayed image on the RDP server, then the original activities are
continued almost instantly with minimum interruption. However, if
the image is changed and refreshed on the RDP server, any old image
being sent is stopped and the new image is sent immediately to the
PDA if it is a full screen image occupying all or part of the PDA
display screen. If the refreshed image sent occupies a part of the
PDA display screen and also areas outside the display screen, then
the portion of the refreshed image that is displayed on the PDA
display screen is sent first to the PDA, and blocks of the image
surrounding the PDA displayed image are sent next to the PDA which
are decompressed and stored in internal memory, as previously
described. Hence the PDA would assemble blocks of the image in the
virtual page, which comprises the entire image, starting first with
the current location of the PDA display screen then areas around
it.
A beacon is sent from the PDA device to the RDP client and
vice-versa many times per minute to confirm that a connection is
established and maintained, for the duration of use as disclosed in
Prior Applications. In the instance a user is disconnected, either
the PDA or the RDP client will not receive a beacon and a time-out
will be initiated, whereby both the PDA and RDP client disconnect
and then reconnect. The user would still be able to view the
present image and scroll around it as this image would already be
decompressed in order of priority and stored in internal memory.
The PDA would indicate in a message area that a reconnect sequence
was initiated, and the status of this connection to the same RDP
client as before, which would refresh the PDA with the image if it
has changed once reconnection is established. Similarly, the PDA
would continue downloading blocks of information after being
reconnected if a disconnection interrupted this operation. A beacon
is not necessarily sent from the RDP client at times when it is
sending information to the PDA. As long as the PDA is receiving
information such as a refreshed image, or a beacon, it knows a
connection is established. Consequently, when there is no new
activity in the virtual machine, the RDP client must send a beacon
to the PDA to confirm the connection is established. The PDA device
must always send beacons to the RDP client, as commands are not
sent frequently from the PDA and only for a short duration when
sent.
An error protocol is implemented in accordance with Prior
Applications, to verify that all information blocks are received
and can be decompressed successfully. As previously described, the
image displayed in the virtual browser in the virtual machine on
the RDP server is broken down into smaller blocks of information
and compressed and transmitted to the PDA device. The blocks are
then assembled in correct sequence, decompressed and stored in
internal memory on the PDA. Each block of information received is
acknowledged by the PDA, which sends a signal confirming that each
block is successfully received. Hence, the RDP client can monitor
the successful decompression of all blocks of information sent to
the PDA, and would know when a block is not acknowledged. The RDP
client would also inform the PDA of the number of transmitted
blocks of information, with each block identified numerically. The
PDA also initiates another error protocol when a block of
information received can not be successfully decompressed and
stored in memory. In this case, the PDA would send an error message
to the RDP client informing which block of information needs to be
sent again, and the RDP client would send this block after it has
completed sending the current block of information. The RDP client
would monitor the acknowledgement of all blocks of information
successfully decompressed, including blocks sent again after
receiving error messages from the PDA device. To illustrate this
further, reference is made to FIG. 6 which shows a string of data
blocks 61 to be sent from the RDP client to the PDA device. As the
PDA receives the data blocks acknowledgements or error messages 62
are sent back to the RDP client. Since the RDP client responds to
all data from the PDA, the string of data blocks actually sent from
the RDP client to the PDA is represented by 63. To further explain
the events of the RDP client 63, blocks 31, 32, 33 & 34 are
sent in this order from the RDP client to the PDA, with successful
acknowledgements ack31 & ack32 sent from the PDA to the RDP
client, but whilst block 34 is being sent, an error message in
block 33 is relayed from the PDA, resulting in the RDP client
sending this block 33 again as soon as block 34 is sent. The RDP
client resumes sending blocks 35, 36 and 37, but notices no
acknowledgement for block 35 after receiving successful
acknowledgements ack34 & ack36 sent from the PDA, which results
in the RDP client sending block 35 after block 37 is sent. The RDP
client may also respond faster or slower to acknowledgements or
error messages as described above.
The illustration of FIG. 7 teaches of a standard serial connection
66 between the PDA device 13 and a cellular phone 12, with an AT
command set for communicating between modems. This allows the
cellular phone 12 to act as a modem in communicating with another
modem 65 attached to the host computer 8. Web pages 67 received
from the Internet are converted to G4 files 68 then sent via modem
to the PDA device.
For a different type of cellular phone that does not allow AT
command set communication but provides a TCPIP Internet connection,
another embodiment is disclosed, as illustrated in FIG. 8. The PDA
device would contain a browser to be able to view images sent by
the host computer, which may be connected to the Internet. In this
case, the PDA device 13 would contain a mini-browser, which
understands and is capable of translating compressed G4 images. The
PDA is connected to a cellular phone 12, which is connected to an
Internet Service Provider (providing standard Internet services)
instead of a modem at the Host Computer. The Host Computer is
connected to the Internet and translates all web pages 67 to G4
compressed files 69. These G4 compressed files 69 are sent to the
PDA in Internet protocol via the Internet Service Provider, and the
mini-browser in the PDA is capable of translating the received
images, and displaying on the PDA screen. Hence, the Host Computer
has Internet images coming in and also Internet images being sent
out. The Internet Service Provider (ISP) that the PDA connects to
would always log on to one web page, and when the user at the PDA
wants to go to a link or to a different web page, the click or the
information is sent through the ISP to the host computer which will
load the new page. This allows multiple users to dial up the
Internet Service Provider to be able to view web pages from the
Internet as desired, in this manner.
Prior application Ser. No. 09/650,412, still pending discloses the
image displayed in the virtual browser is compressed at the server
in various amounts, by different methods before sending to the PDA
device. A web page is usually comprised of text portions and
picture portions. Different parts of the image are compressed in
different ways. Parts of the image that are text images are
compressed by G4 compression techniques. Other parts of the image
containing pictures are compressed by JPEG compression techniques.
The entire image is converted to a raster image but different parts
are converted at different depths of color, and by different
compression techniques. Text portions can be compressed by "loss
less" techniques, which will result in complete image reproduction
with no errors in text upon decompression, whereas pictures can be
compressed by "lossy" techniques which upon decompression give a
slightly degraded image quality for each compression. An image with
black and white text and color picture portions would be converted
to a raster image, but only the text part of it reduced to black
and white 1 bit and other picture parts to 24 bit or other color
bit depths. Hence different parts of the image can be converted or
reduced to different depths of color to resemble an actual web page
with picture in color and text in black and white. The depths of
color can be set by the user or preset at the server depending on
the quality of image required, which affects the speed of
refreshing the screen which also depends on the amount of pictures
displayed and the amount of bandwidth used. Text and pictures from
a web page would get compressed separately at the server and then
sent to the PDA. In the PDA, text or black and white portions of
the image get decompressed first and displayed on the screen. Color
portions get decompressed and are overlain in the image on the
screen shortly after, enabling the user to view black and white
portions of the image in advance. At the browser, the priority of
decompression is determined by the depth of color, with the minimum
depth being decompressed first.
A further embodiment would take the entire image to be viewed
including all text and pictures and convert it to 1 bit raster.
This file is compressed by G4 or other loss less compatible methods
and sent to the PDA. When received by the PDA, this file or packets
are decompressed for the user to view the image almost
instantaneously. One of the ways this is accomplished is by taking
a web page image at the server, comprising text and picture
portions, and compress the text and picture portions at different
depths of color and by different compression techniques, so that a
black & white image is first displayed on the PDA screen after
decompressing, and then the picture portions of the image are
overlain with color. In particular, the text and picture portions
are initially displayed at the PDA in 1-bit color, and the picture
portions only are overlain on top of the initial 1-bit with 24-bit
color. Thus, the text will be displayed in 1-bit color and pictures
would be displayed in 24-bit color. This enables an image to be
viewed quickly without the full color details, which follow moments
after, by reducing the bandwidth of the information sent to the PDA
to be first displayed.
In another embodiment, the text and picture portions are initially
displayed at the PDA in 3-bit color and the picture portions are
overlain on top of the initial 3-bit with 8-bit color. Thus, the
text will be displayed in 3-bit color and pictures would be
displayed in 8-bit color. This method provides a means to view the
image rapidly while also conserving on bandwidth. The picture
portions may also be overlain with 24-bit color keeping the text at
3-bit color, in a further embodiment.
In a further embodiment, which discloses another method of reducing
the bandwidth consumed, consequently providing a fast display from
the time an image is sent from the server to the PDA, the text and
pictures may be sent in a low bit color and the picture portions
overlain with higher bit depths of color as previously described,
but the picture resolution is reduced before sending from the
server to the PDA. Therefore, by example a picture, which is 100
pixels.times.100 pixels, is reduced in resolution by four times and
then sent from the server to the PDA. This would reduce the amount
of data transferred from the server to the PDA, conserving on
bandwidth consumed providing an image, which is "blocky" at first
being of a low resolution, but more pixels are sent moments
afterwards thereby improving the picture resolution.
In accordance with the present invention, as illustrated in FIG. 9,
the server 71 contains many browsers 74, 75, and 76, in which each
browser is running its own web page, and each browser is meant for
a different PDA. A particular web page 70 that is displayed first
on one browser 74 at the server 71, is transmitted to the PDA
device 72 which is connected to a cellular telephone 73. The
displayable area or screen 78 of the PDA is shown to be much
smaller than the entire web page 70, thus it is important to
establish a priority in sending data of the web page to the PDA, as
the user will want to first view the location that the PDA screen
occupies on the web page. Data of each web page is broken down into
many packets or blocks, whereby these packets or blocks are
transmitted from the server to the PDA and assembled into memory in
the PDA, and then displayed on the screen. In an alternate
embodiment, the blocks transmitted from the server to the PDA may
be assembled first on the screen and then stored into memory. In a
further embodiment, the blocks transmitted from the server to the
PDA may be simultaneously assembled on the screen and stored into
memory. A packet of data may contain one or more blocks or a packet
may only consist of part of a block. For demonstration purposes, a
packet is defined as an entire block, which is transmitted from the
server to the PDA. When a new web page is displayed on the PDA
screen, the user 79 will want to first see the segment of that web
page occupied by the screen area 78. Thus, all blocks of data that
fill up the screen area of the current web page on display must be
transmitted to the PDA as a first priority.
To explain further in detail, reference is made to FIG. 10, which
illustrates a web page 70 enlarged. The web page is shown to
consist of blocks of data 80 that are numbered consecutively for
demonstration purposes. Blocks 80 exist across and down an entire
web page in a matrix array, and span beyond the displayable area,
or screen 78 as shown. When a web page is loaded at the server,
blocks 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12 & 13 are sent first
from the server to the PDA to be displayed, as only these blocks
comprise the displayable area or screen 78. Thus, the first
priority is sending data from the server to the PDA whereby the
displayable area on the PDA is filled up with blocks of data.
The second priority is sending data from the server to the PDA
whereby a small area around the displayable area on the PDA is
filled up with blocks of data, and this is only done after the
first priority is completed. To explain further in detail,
reference is made again to FIG. 10, which illustrates blocks 7, 14,
15, 16, 17, 18, 19, 20 & 21 sent from the server to the PDA, to
be stored in internal memory on the PDA, on the web page image
displayed on the PDA, with these blocks inserted in their correct
locations as they appear on the web page 70 at the server. This
will facilitate scrolling of small amounts immediately at the PDA
device to view the displayed image on the PDA screen so that it
gives better speed to the user, and can only be done after blocks
1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12 & 13 are sent first from the
server to the PDA (first priority).
The third priority is sending data from the server to the PDA
whereby animation or segments of the current PDA display area that
continuously change, keep getting refreshed continuously for a set
period of time. The user may have the option to change this period
according to the user's preference, or this may be preset at the
server. With further reference to FIG. 10, blocks 3, 4 & 5, or
portions thereof, may be an advertisement in which the GIF is
continuously changing. Thus, as an example, blocks 3, 4 & 5
would get continuously refreshed for the first 10 seconds (if the
user selects this period or it is preset at the server) as a third
priority, after the first and second priorities have been
completed, as defined previously. One of the reasons for
continuously refreshing animation or segments of the display that
continuously change for a certain period of time, is because the
web page itself may not be fully rendered in the browser at the
server. In this case, portions of the image on the web page for
example Yahoo logos, advertisement banners or other portions of the
web page, may not appear immediately as the web page is loaded at
the server and is being rendered for a certain period of time, but
more parts keep getting rendered during the first 10 seconds at the
server (if the user selects this period or it is preset at the
server) until the full page is finally rendered. This allows the
portions of the web page that appear in the displayable area of the
PDA at the server, to be continuously sent and refreshed at the PDA
during this period, as the web page is being rendered at the
browser which will take a few seconds. Therefore, as a new web page
appears on the browser at the server and parts are added to it for
this period until fully rendered, the image displayed on the PDA
screen will also be generated in a similar fashion with parts added
to it for this period. Thus, it may not be advertisements that are
appearing and changing during the first 10 seconds at the server
(if the user selects this period or it is preset at the server),
but it may be the web page that needs to be fully rendered during
this period. This period for example 10 seconds, may not be varied
by the user or preset at the server, but may be determined by the
browser itself. The browser is aware of how long it takes to
download the entire web page and it may use from the beginning to
the end of when it receives the entire web page, as the amount of
time it is going to continue refreshing. Therefore the browser will
refresh parts of the image on the main screen up to the end of this
time as a second priority, and after this time it will either not
send refreshes or send them based on a third priority protocol.
With further reference to FIG. 15 and the first priority, there
exists a period "A" (which is the period previously explained which
can be determined either by the user or preset at the server or
determined by the browser, when the web page is fully received)
whereby the browser is rendering the web page during this period,
and all blocks that change on the web page within the displayable
screen area will be sent from the server to the PDA to be
displayed. The browser does not distinguish between animation
portions such as advertisements or other parts of the web page that
do not change, but sends blocks of information that comprise the
display area 78 in accordance with the first and second priorities.
At the end of period "A" the browser tells the server software that
the web page is fully rendered. During period "A" the server checks
each block in the displayable area or screen 78 at set intervals.
If the last change was made to block 3 in a particular frame, the
server checks each block in the display area 78 for new blocks that
have changed, and if changes are made to blocks 3 and 4, the server
sends block 4 first then block 3 as the priority is shifted to
sending newly changed blocks that have not been changed since the
previous frame. Thus, blocks are not sent twice consecutively
unless there are no other changes to other blocks in consecutive
frames. The set intervals at which times the server checks the
display area 78 for changes to be sent to the PDA, may be done
every 2 to 3 seconds or as often as the user desires or is preset
at the server.
Again with reference to FIG. 15 and the third priority, after the
duration "A", the animation may be disabled by the user or it may
be preset at the server, as the user may not wish to see animation
changes such as advertisements. Thus, the user at the PDA or the
server may disable animation changes to the display area 78 to
conserve on bandwidth, as animation changes consume large amounts
of bandwidth. Therefore as a third priority the server will send
the blocks comprising the rest of the web page to the PDA.
As a further embodiment, if animation is enabled at the browser and
it is determined that animation is not to be displayed at the PDA
(this may be the case when animation is disabled at the browser,
nothing at all appears on the screen at the browser for that
particular location, but the user would like to see at least the
first still image) the first, second and third priorities are all
conducted as before but with animation enabled at the browser,
except that the third priority which fills the rest of the page
will only send each block once and not send changes.
As a further embodiment, if the user or server has predetermined
that animation is required, then the first and second priorities
are conducted as before, but the third priority is changed. With
animation enabled, and after period "A" has elapsed or after a few
seconds more than "A" represented by "X" seconds the server would
time slice blocks being sent to the PDA to fill up the larger area
87 with blocks of animation that keep changing continuously. A
reduction in the rate of updating animation would conserve
bandwidth and will allow other parts of the web page 70 to be
transmitted simultaneously to the PDA. Thus, after "A" seconds or
after "A"+"X" seconds, the server would check for changes to
animation and send those blocks that have changed, then send a few
blocks of the larger area 87, and alternate between these two
activities in this time slice mode, until area 87 is completely
received at the PDA.
When the user is viewing area 81 of a web page 70 of FIG. 10 and
this area is filled up with blocks of data at the PDA (or even
before), and the user scrolls to a new area 82, blocks of data fill
up the display area 78 in accordance with the first three
priorities established. This means that the server is told about
the new location and priorities 1, 2, and 3 start to get conducted
with this new location as the new parameter.
Another parameter that will restart the priority protocols 1, 2,
and 3 is if the user clicks anywhere on the display screen. Thus,
the emphasis will be on any changes to the main display when this
activity occurs.
Should the user scroll to a new location on the web page image
displayed on the PDA screen 78, then the PDA would immediately send
information on the new location scrolled to, and the server would
respond by sending those blocks that occupy the display screen of
the new location to the PDA, which would be added to the web page
image stored in memory on the PDA. This process would be handled as
a first priority also, since the criteria for first priority is to
fill up the display area on the PDA display screen. Thus, by
example with reference to FIG. 10, in a first location 81 blocks of
data 1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12 & 13 which only occupy
the displayable area 78, are sent from the server 71 to the PDA 72
to be displayed first. Should the user 79 scroll to the new
location 82 before the PDA is finished downloading and displaying
blocks from the first location 81, the PDA then sends the
coordinates of this new location to the server, and immediately
blocks of data 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42 & 43
which only occupy the displayable area 78 of the new location 82
are sent as a first priority in response from the server to the
PDA, to be decompressed, stored into memory (on the same web page
image at the PDA), and displayed on the screen 78 to the user. In
an alternate embodiment, the blocks transmitted from the server to
the PDA may be assembled first on the screen and then stored into
memory. In a further embodiment, the blocks transmitted from the
server to the PDA may be simultaneously assembled on the screen and
stored into memory. As defined previously, the second priority
would be for the server 71 to send blocks 37, 44, 45, 46, 47, 48,
49, 50 & 51 to the PDA, which surround the new location 82 at a
fixed distance surrounding the display area 78. The number of
blocks sent from the server to the PDA surrounding the display area
78 may be set at the server location. Thus, at any instant when
scrolling is performed, all activities are interrupted and the
first priority goes to sending blocks of data that only occupy the
displayable area 78, in the portion of the web page on the server
where the PDA has scrolled to, in accordance with the present
invention.
In an effort to conserve on bandwidth, the PDA may be set by the
user to receive only a few blocks of data around the screen
location 81, or alternately the server may be set to only send a
few blocks of data around the screen location 81, after which no
other blocks are sent to the PDA unless the user scrolls or clicks
elsewhere.
As mentioned previously, in the original application Ser. No.
09/677,857, now U.S. Pat. No. 6,842,777 pictures contained in the
image displayed in the virtual browser are compressed at the
server, in different methods than the text portions and the color
depth reduced by different methods than text portions before
sending to the PDA device. When received by the PDA, this image is
decompressed for the user to view text portions first, whereby the
picture portions are displayed at a lower depth of color next,
which can be done quickly, and these portions are overlaid
progressively with color at higher bit depths. This enables an
image to be viewed quickly without the fine details of graphics,
which follow moments afterwards. Thus, picture portions of a web
page are reduced in color depth at the server 71 and sent to the
PDA 72, which receives and displays these graphics at a lower color
depth, for example 3 bit. The fourth priority is for the color
depth of the pictures displayed on the screen 78 to be increased to
resemble the web page image presently rendered on the server 71.
Then by example, the 3-bit color depth graphics or picture portions
of the image are overlaid with 8-bit color. In particular, a web
page image 70 of FIG. 10 is usually comprised of text portions and
picture portions, and only the picture portions are overlaid with
8-bit color and the text portions are left in 3-bit color depth.
Therefore, after the third priority of sending data from the server
to the PDA whereby animation or segments of the display that
continuously change, keep getting refreshed continuously for a set
period of time, the color depth of the picture portions of the
image displayed on the PDA screen is increased to match the web
page image by being sent by the server as a fourth priority, in
accordance with the present invention.
Alternatively, if bandwidth reduction is required and the third
priority is eliminated, then this fourth priority is conducted
after the second priority. In general, a priority protocol is
referred to as being a priority. This fourth priority applies to
images only on the current display or at a fixed distance around
the display.
For a color PDA device, the fifth priority after the picture
portions are overlaid with 8-bit color (or other), is to have
blocks of data sent from the server to the PDA filling up all other
areas on the web page image at the PDA, which are decompressed and
stored into memory in their correct locations as they appear on the
rendered image at the server, in accordance with the present
invention. This activity, however, is not an exclusive activity and
may be done in conjunction with refreshing animation or segments of
the display that continuously change. Thus, this fifth priority of
filling the missing blocks around the display area 78 may be
time-sliced with refreshing animation or segments of the display
that continuously changes. Therefore, everything that changes on
the main screen becomes slower due to time-slicing. In the method
of time-slicing, there would be a few blocks of data sent from the
server to the PDA to fill up around the display area, then a few
blocks of data which refresh areas of animation within the display
area 78 are sent immediately after, in an alternating fashion until
the entire web page image is received at the PDA. In another
embodiment of this invention, this fifth priority of filling the
missing blocks around the display area 78 may be done
simultaneously with refreshing animation or segments of the display
that continuously change. A possible means for accomplishing this
is to use half the bandwidth to transmit each task from the server
to the PDA. In a further embodiment of the present invention, one
byte of data is dedicated to transmitting blocks of information to
fill up around the display area, and the next byte of data
dedicated to refreshing animation or segments of the display that
continuously change, with this procedure occurring in an
alternating fashion until the entire web page image is received at
the PDA. This is the same as third priority from before.
If instead of a circuit switched connection the network being used
is a packet-based network where conservation of bandwidth is
required, then priority three or priority five may not be sent at
all, i.e. no blocks are sent to fill up the rest of the web page
and the only way the user will see other parts of the web page is
by scrolling out of the current display area and waiting for new
blocks to be downloaded. Alternatively, the server may start to
send additional blocks (which are not in the current display area),
in a location determined by the direction the user starts to scroll
or shows intent of scrolling towards by other means. The server may
also save user preferences of certain web pages where a particular
user normally likes to scroll and downloads those portions after
the first and second priorities are done.
In another embodiment, the server may send no image during period
"A" or "A+x" and only starts the first priority after this time has
elapsed. Therefore, the current screen (or current screen plus
fixed distance around current screen) would start to get downloaded
after the period finishes, and all blocks would only download
once.
This way the web page would be fully rendered at the browser before
anything is sent to the PDA device. So, first the current screen
would be sent (after this period is finished) and then a fixed
distance around the current screen would be sent secondly, and in
the bandwidth conserving method nothing else would be sent (no
changes to the current screen or filling up the rest of the page)
unless the user shows intent of scrolling out of this current
screen or clicks on the current screen. If the user clicks on the
current screen then the first priority shifts to sending changes on
the current screen.
In accordance with the present invention, the PDA screen 78 is
comprised of a matrix array of liquid crystal dots, each dot of
equal uniform size and gap between dots. The image occupying the
entire PDA screen is sub-divided into blocks of data as previously
described, and for demonstration purposes each block of data
comprising a fixed amount of dots along its width and a fixed
amount of dots along its height. The actual blocks may be variable
in size and are determined by the server dynamically. Thus, for
demonstration purposes and with reference to FIG. 11, block I (of
FIG. 10) will be defined as a matrix array 10 dots wide by 10 dots
high. Therefore, line 1 83 will be comprised of a linear array of
10 dots only, with 10 such lines located one below the other
allocated to block 1. Each block of data may be further sub-divided
into smaller blocks as depicted in FIG. 12, whereby block 1 is
sub-divided into blocks 1a and 1b. Similarly, all other blocks (2,
3, 4, 5, etc.) are split into "a" and "b" sub-divisions with the
"a" sub-division comprising the odd numbered lines 1, 3, 5, 7 &
9, and the "b" sub-division comprising the even numbered lines 2,
4, 6, 8 & 10, for the corresponding block. Therefore, when a
web page is loaded at the server and blocks of data are transmitted
to the PDA, it is the "a" sub-divisions that are sent first for all
blocks of data transmitted from the server to the PDA. Thus, with
further reference to FIG. 10, for the screen 78 at location 81,
blocks 1a, 2a, 3a, 4a, 5a, 6a, 8a, 9a, 10a, 11a, 12a & 13a are
sent as a first priority from the server to the PDA. Since only
half the data for each block of information is sent to the PDA for
display by this method, the drawing sequence or display sequence is
such that each line of data is drawn twice as data blocks in the
"a" sub-division only contain 5 of the 10 lines of information in
each block, on the display screen 78. Otherwise, each displayed
block will have alternating lines missing or blank, making the
displayed image on the PDA screen difficult to recognize.
Therefore, with further reference to FIG. 13, the first drawing
sequence or display sequence for block 1 will be to display lines
1, 1, 3, 3, 5, 5, 7, 7, 9, & 9 in this order from top to
bottom, in which the "a" lines are temporarily repeated. By this
method, only half the data for each block is sent from the server
to the PDA to have a very fast display time on the PDA, though the
image will appear "blocky" as alternate lines of information
comprising the display for each block will be missing and replaced.
This condition is only temporary, as at a later time, for the
screen 78 at location 81, blocks 1b, 2b, 3b, 4b, 5b, 6b, 8b, 9b,
10b, 11b, 12b & 13b are sent from the server to the PDA, and
the "a" lines which are temporarily repeated in each block are then
replaced with these new "b" lines for each block. Blocks 1a and 1b
are combined to produce the final display of block 1, as each
contains only half the display lines, and this process is repeated
for each block on the PDA display. These fully reconstructed blocks
are then stored in memory on the PDA device. Thus, block 1 of the
display screen will now be comprised of lines 1, 2, 3, 4, 5, 6, 7,
8, 9 & 10 as illustrated in to FIG. 14, which is the second
drawing sequence or display sequence for block 1. This method of
interlacing blocks of data enables the user to view a web page with
very little delay from the server, though at first it may not be
very clear as alternate lines of information are replaced, enabling
the user to have a preview of the web page at a display speed much
faster than if all lines of each block were transmitted at once
from the server. This is one method of sending interlaced images,
and there may be other methods of sending a lower information
content block and add on the rest of the information at the next
step.
In accordance with priorities established in the present invention,
and with further reference made to FIG. 10 for the screen 78 in
location 81, when the image to be displayed on the PDA screen is
sent from the server, blocks 1a, 2a, 3a, 4a, 5a, 6a, 8a, 9a, 10a,
11a, 12a & 13a are sent as a part of the first priority. Next,
blocks 1b, 2b, 3b, 4b, 5b, 6b, 8b, 9b, 10b, 11b, 12b & 13b are
sent from the server to the PDA to complete the first priority, and
these "b" blocks are interlaced with corresponding blocks in the
"a" sub-division previously sent. Thus, it is important to have the
image of what the user views as close to perfect before any other
areas are sent, in accordance with the first priority of the
present invention. Afterwards, and in accordance with priorities
previously established, blocks 7a, 14a, 15a, 16a, 17a, 18a, 19a,
20a & 21a which surround the display area 78, are sent from the
server as part of the second priority. This will allow the user the
opportunity to scroll a little outside the display area almost
immediately, without having to wait the time it takes for the
entire blocks surrounding the display to be sent to the PDA.
Subsequently, blocks 7b, 14b, 15b, 16b, 17b, 18b, 19b, 20b &
21b are sent to the PDA to complete the second priority, and these
"b" blocks are interlaced with corresponding blocks in the "a"
sub-division previously sent. In further detail as an example, with
the screen 78 in location 81, as the first string of blocks 1a, 2a,
3a, 4a, 5a, 6a, 8a, 9a, 10a, 11a, 12a & 13a are sent from the
server to the PDA, and the user scrolls to a new location 82 before
blocks 1b, 2b, 3b, 4b, 5b, 6b, 8b, 9b, 10b, 11b, 12b & 13b are
sent, then immediately blocks 31a, 32a, 33a, 34a, 35a, 36a, 38a,
39a, 40a, 41a, 42a & 43a which only occupy the new displayable
area are sent from the server to the PDA. Sending blocks 31a, 32a,
33a, 34a, 35a, 36a, 38a, 39a, 40a, 41a, 42a & 43a will now be
treated as a first priority (as this is the area the user wishes to
view), and these are followed by blocks 31b, 32b, 33b, 34b, 35b,
36b, 38b, 39b, 40b, 41b, 42b & 43b sent from the server to the
PDA to complete the first priority, and these blocks are interlaced
with corresponding blocks in the "a" sub-division previously sent.
Next, blocks 37a, 44a, 45a, 46a, 47a, 48a, 49a, 50a & 51a which
surround the new location 82 of the display area are sent from the
server to the PDA, in accordance with a part of the second
priority. To complete the second priority, corresponding blocks
37b, 44b, 45b, 46b, 47b, 48b, 49b, 50b & 51b are sent from the
server to the PDA, and these blocks are interlaced with
corresponding blocks in the "a" sub-division previously sent. As a
third priority animation or segments of the display that
continuously change get refreshed in a similar manner, with the "a"
sub-divisions sent first from the server and the "b" subdivisions
sent immediately after, to be interlaced in a similar fashion. The
fourth priority of increasing the color depth follows immediately
after. The fifth priority however, would be to next send the "b"
subdivisions to the first area 81 which was first presented to the
user before the scrolling action. Thus, blocks 1b, 2b, 3b, 4b, 5b,
6b, 8b, 9b, 10b, 11b, 12b & 13b are sent from the server to the
PDA, and this is followed by all surrounding areas, with the "a"
sub-divisions sent first from the server and the "b" subdivisions
sent immediately after, to be interlaced in a similar fashion
thereby completing the web page image stored into memory on the
PDA.
In accordance with another embodiment of the present invention, and
with reference to FIG. 9, blocks of data 84 are transmitted from
the server 71 to the PDA 72, through the cellular phone
infrastructure. Inside the cellular phone infrastructure there is a
large pipe, and the blocks that are sent from the server to the PDA
are inside this pipeline. The pipeline 85 for transmitting blocks
of data is quite large, and as a result there is a significant lag
or delay in time between sending a particular block at the server
and receiving it at the PDA. Because of this there are many blocks
in the pipeline at the time a particular block is sent from the
server, and the blocks in the pipeline must be removed or received
before that particular block is received at the PDA. In particular,
a situation may occur whereby blocks 1, 2, 3, 4, 5, 6, 7 & 8
may be sent form the server and only block 1 is received at the PDA
after block 6 is sent from the server, due to the significant lag
or delay in the pipeline. The PDA must then transmit an
acknowledgement for each block received, otherwise the server has
no information on any blocks that may not be successfully received,
decompressed and displayed on the PDA screen. Should there be any
missing blocks or blocks that can't be displayed successfully, then
the server would reschedule such blocks to be sent again next into
the pipeline, as no acknowledgement for such blocks would be
received at the server. One problem with this arrangement is the
lengthy delay in any rescheduled block being received at the PDA,
as many other blocks in the pipeline must be received and
acknowledged by the PDA before the rescheduled block is received,
acknowledged and put in its correct location. This problem of a
lengthy delay also exists when the user clicks to a new web page,
as many blocks from the old web page already in the pipeline need
to be removed from the pipeline or received by the PDA before any
new blocks representing the new web page are received at the PDA.
To significantly reduce such a lengthy delay, the number of blocks
transmitted in the pipeline is limited to a smaller quantity, say 3
blocks instead of 6 blocks for demonstration purposes only. Thus
blocks 1, 2 & 3 are sent from the server to the PDA, and the
server waits for acknowledgement of block 1 from the PDA first,
before sending block 4. If no acknowledgement is sent for a
particular block within a designated time period, then the server
assumes this block is not successfully received, decompressed and
displayed at the PDA, and automatically reschedules this block for
sending next in the pipeline. By example, ideally blocks 1, 2 &
3 are sent first from the server to the PDA, then an
acknowledgement for block 1 must be received at the server first
before block 4 is sent to the PDA. After the acknowledgement for
block 2 is received at the server, then block 5 is sent, etc., so
the acknowledgements are not behind by more than 3 blocks from what
was sent. This system quickens the response time for displaying a
new image significantly, without having to wait for many packets or
blocks to be cleared from the pipeline first.
Also, in accordance with the present invention, the priority of
downloading data to the PDA changes if the user clicks anywhere on
the display screen 78. For example, the main screen display area is
already downloaded and displayed on the PDA screen, and the
remaining blocks surrounding the display area are being downloaded,
and the priority has already changed from refreshing animation to
putting blocks around the display area time-sliced with animation
changes. Thus, everything that changes on the main screen becomes
slower due to the time-slicing operation, but if the user clicks on
an area that requires input such as a title box, the present
display mode is too slow for such a situation. Therefore, any click
on the parts of the web page displayed on the PDA screen 78 that
invoke a change to the image displayed on the screen (since the PDA
sends a message to the server as to where the click down occurred),
the priority of downloading information changes to give first
priority to filling up the display area on the PDA with any new
image, thereby receiving the full bandwidth for this operation
instead of time-slicing. After the display area is filled up, the
other priorities follow as previously described. This is
particularly useful for text entry so the user may see what text
was entered without any lengthy delay, and also for menu items on
the page.
To conserve on bandwidth consumption, the server monitors the
activities of the user from messages sent back from the PDA on user
activities such as scrolling, clicking or text input. Thus, after a
pre-set duration (which is user adjustable), the server may stop
sending new blocks of information (as a means of bandwidth
conservation) on any particular web page transmitted to the PDA if
there is no response from the PDA on these user activities. Unless
the user shows intentions of moving out of the display area from
such user activities, bandwidth conservation is engaged in
accordance with the present invention.
A similar mode exists to conserve on bandwidth for areas that
require animation changes, in accordance with the present
invention. The server will send no blocks of data after a time-out
period, which is user adjustable or preset at the server. This is
usually a few seconds, and is particularly useful in instances
where the animation changes are for an advertisement, in which the
user has no interest. Thus, in this mode of bandwidth conservation,
the only changes to the display image would occur if the user
scrolled outside the presently displayed image, or clicked or typed
in an area, which invokes a change to the displayed image.
* * * * *