U.S. patent application number 10/071172 was filed with the patent office on 2002-11-28 for method and device for browsing information on a display.
This patent application is currently assigned to Myorigo, L.L.C.. Invention is credited to Hannula, Manne, Vaananen, Johannes.
Application Number | 20020175896 10/071172 |
Document ID | / |
Family ID | 8561217 |
Filed Date | 2002-11-28 |
United States Patent
Application |
20020175896 |
Kind Code |
A1 |
Vaananen, Johannes ; et
al. |
November 28, 2002 |
Method and device for browsing information on a display
Abstract
The idea of the present invention is to browse information on a
display device of a hand-held device naturally and logically.
Characteristic of the invention is that information is browsed on
the display device essentially in a mirror-like way. In other
words, the portion of the virtual data object displayed on the
display device is moved in the same direction as the hand-held
device is tilted. In other words, the movements of the portion of
the virtual data object displayed on the display device depends on
the orientation of the hand-held device. An important feature of
the invention is also that a certain orientation of the hand-held
device always displays the same portion of the virtual data object
on the display device. The browsing method described above is
extremely logical, and the movements and responses to the movements
are natural.
Inventors: |
Vaananen, Johannes; (Oulu,
FI) ; Hannula, Manne; (Oulu, FI) |
Correspondence
Address: |
COHEN, PONTANI, LIEBERMAN & PAVANE
551 Fifth Avenue, Suite 1210
New York
NY
10176
US
|
Assignee: |
Myorigo, L.L.C.
Wilmington
DE
|
Family ID: |
8561217 |
Appl. No.: |
10/071172 |
Filed: |
February 8, 2002 |
Current U.S.
Class: |
345/158 |
Current CPC
Class: |
G06F 2200/1637 20130101;
G06F 2200/1614 20130101; G06F 1/1686 20130101; G06F 3/1407
20130101; G09G 2320/0261 20130101; G06F 1/1626 20130101; G06F
1/1694 20130101 |
Class at
Publication: |
345/158 |
International
Class: |
G09G 005/00 |
Foreign Application Data
Date |
Code |
Application Number |
May 16, 2001 |
FI |
20011039 |
Claims
We claim:
1. A method for browsing information on a display device of a
hand-held device, wherein the method comprises a virtual display
being the display device of the hand-held device, a viewpoint from
which the virtual display is viewed and a virtual data object
comprising information to be viewed on the virtual display, wherein
the method comprises the steps of: coupling the display device to a
digital processor; mapping information content generated by the
digital processor into the virtual data object suitable for
conveying information to the user of the hand-held device;
displaying a portion of the virtual data object at a time on the
display device, the virtual data object comprising characters,
pictures, lines, links, video or pixels that can be conveniently
displayed on the display device at a time; wherein information is
browsed on the display device essentially in a mirror-like way, the
method further comprising the step of: moving the portion of the
virtual data object displayed on the display device in the same
direction as the hand-held device is tilted, whereby a certain
orientation of the hand-held device always displays the same
portion of the virtual data object on the display device.
2. The method according to claim 1, wherein the method comprises
the steps of: setting a predefined xy-plane as a xy-plane;
determining a relation between the rotation degree around the
x-axis and the y-axis and the amount of the displacement of the
portion on the virtual data object displayed on the display device
at a time; displacing the position of the displayed portion of the
virtual data object to the right when the hand-held device is
rotated essentially towards the positive rotation direction around
the y-axis; displacing the position of the displayed portion of the
virtual data object to the left when the hand-held device is
rotated essentially towards the negative rotation direction around
the y-axis; displacing the position of the displayed portion of the
virtual data object upwards when the hand-held device is rotated
essentially towards the positive rotation direction around the
x-axis; displacing the position of the displayed portion of the
virtual data object downwards when the hand-held device is rotated
essentially towards the negative rotation direction around the
x-axis; and displaying the movement of the portion of the virtual
data object on the display device of the hand-held device according
to the set relation.
3. The method according to claim 2, wherein the method comprises
the step of: changing the relation between the rotation degree
around the x-axis and/or the y-axis and the amount of the
displacement of the portion on the virtual data object in
proportion to the distance between the viewpoint and the display
device.
4. The method according to claim 1, wherein the movement of the
portion of the virtual data object displayed on the display device
is proportional to the change amount and/or rate of the rotational
movement around the x-axis and/or y-axis.
5. The method according to claim 1, wherein the method comprises
the steps of: setting the display device into a zoom mode;
determining the distance between the viewpoint and the display
device; and zooming in or out the displayed information based on
the determined distance information.
6. The method according to claim 1, wherein the method comprises
the steps of: setting the display device into a zoom mode; and
zooming in or out the displayed information when rotating the
hand-held device around the axis being essentially perpendicular to
the predefined xy-plane.
7. The method according to claim 1, wherein the method comprises
the steps of: setting the display device into a zoom mode; and
zooming in or out the displayed information when the hand-held
device is tilted.
8. The method according to claim 1, wherein the information
displayed on the display device essentially depends on the location
and orientation of the virtual display, the viewpoint and the
virtual data object.
9. The method according to claim 1, wherein the method comprises
the steps of: setting the display device surface level as an
xy-plane; determining a relation between the x-axial and/or y-axial
movement of the hand-held device and the amount of the displacement
of the portion of the virtual data object displayed on the display
device at a time; and moving the portion of the virtual data object
displayed on the display device in the same direction as the
hand-held device is moved in the xy-plane according to the relation
information.
10. The method according to claim 1, wherein filtering the x-axial,
y-axial and/or tilting movements before displaying the movements on
the display device.
11. The method according to claim 1, wherein changing the relation
between the rotation degree around the x-axis and y-axis and the
amount of the displacement of the portion of the virtual data
object displayed on the display device at a time.
12. The method according to claim 1, wherein the method comprises
the step of: keeping the orientation of the information displayed
on the display device unchanged when rotating the hand-held device
around the axis being essentially perpendicular to the surface
level of the hand-held device.
13. A hand-held device for browsing information, wherein the
hand-held device comprises a virtual display being the display
device of the hand-held device, the hand-held device comprising: a
digital processor (30); a memory (60,70) coupled to the digital
processor (30), the memory (60,70) comprising a virtual data object
suitable for conveying information to the user of the hand-held
device; a display device (10) coupled to the digital processor
(30); means (30) for moving the portion of the virtual data object
displayed on the display device in the same direction as the
hand-held device is tilted, whereby a certain orientation of the
hand-held device always displays the same portion of the virtual
data object on the display device.
14. The hand-held device according to claim 13, wherein the
hand-held device comprises: means (30) for setting an xy-plane as a
default xy-plane; relation information (60) based on the rotation
degree around the x-axis and y-axis and the amount of the
displacement of the portion of the virtual display space displayed
on the display device at a time; means (30) for determining the
rotation amount around the x-axis and/or y-axis; and means (30) for
changing the location of the portion of the virtual data object
displayed on the display device (10) based on the rotational amount
around the x-axis and/or y-axis and the relation information
(REL).
15. The hand-held device according to claim 13, wherein the
hand-held device comprises means (30) for changing the relation
information (60).
16. The hand-held device according to claim 13, wherein the
hand-held device comprises: means (30) for setting the display
device into a zoom mode; means (20,50) for determining the distance
between the viewpoint and the display device; and means (30) for
zooming in or out the displayed information based on the distance
information.
17. The hand-held device according to claim 13, wherein the
hand-held device comprises means (30) for zooming in or out the
displayed information when rotating the hand-held device around the
axis being essentially perpendicular to the predefined
xy-plane.
18. The hand-held device according to claim 13, wherein the
hand-held device comprises: means (30) for setting the display
device surface level as an xy-plane; relation information (60)
between the x-axial and/or y-axial movement of the hand-held device
and the amount of the displacement of the portion of the virtual
data object displayed on the display device at a time; means (30)
for determining the amount of displacement in the xy-plane; and
means (30) for moving the portion of the virtual data object
displayed on the display device (10) in the same direction as the
hand-held device is moved in the xy-plane according to the relation
information (60).
19. The hand-held device according to claim 13, wherein the
hand-held device comprises means (30) for filtering the x-axial,
y-axial and/or tilting movements before displaying the movements on
the display device (10).
20. The hand-held device according to claim 13, wherein the
hand-held device comprises means (30) for changing the relation
(60) between the rotation degree around the x-axis and y-axis and
the amount of the displacement of the portion of the virtual data
object displayed on the display device (10) at a time.
21. The hand-held device according to claim 13, wherein the
hand-held device comprises means (30) for changing the relation
(60) between the x-axial and/or y-axial movement of the hand-held
device and the amount of the displacement of the portion of the
virtual data object displayed on the display device at a time.
22. A computer program embodied on a computer-readable medium,
wherein the computer program executes the program steps recorded in
a computer-readable medium to perform a method for browsing
information on a display device of a hand-held device, wherein the
method comprises a virtual display being the display device of the
hand-held device, a viewpoint from which the virtual display is
viewed and a virtual data object comprising information to be
viewed on the virtual display, wherein the method comprises the
steps of: coupling the display device to a digital processor;
mapping information content generated by the digital processor into
the virtual data object suitable for conveying information to the
user of the hand-held device; displaying a portion of the virtual
data object at a time on the display device, the virtual data
object comprising characters, pictures, lines, links, video or
pixels that can be conveniently displayed on the display device at
a time; wherein in the method information is browsed on the display
device essentially in a mirror-like way, the method further
comprising the step of: moving the portion of the virtual data
object displayed on the display device in the same direction as the
hand-held device is tilted, whereby a certain orientation of the
hand-held device always displays the same portion of the virtual
data object on the display device.
23. The computer program according to claim 22, wherein the
computer program executes the steps of: setting a predefined
xy-plane as a xy-plane; determining a relation between the rotation
degree around the x-axis and the y-axis and the amount of the
displacement of the portion on the virtual data object displayed on
the display device at a time; displacing the position of the
displayed portion of the virtual data object to the right when the
hand-held device is rotated essentially towards the positive
rotation direction around the y-axis; displacing the position of
the displayed portion of the virtual data object to the left when
the hand-held device is rotated essentially towards the negative
rotation direction around the y-axis; displacing the position of
the displayed portion of the virtual data object upwards when the
hand-held device is rotated essentially towards the positive
rotation direction around the x-axis; displacing the position of
the displayed portion of the virtual data object downwards when the
hand-held device is rotated essentially towards the negative
rotation direction around the x-axis; and displaying the movement
of the portion of the virtual data object on the display device of
the hand-held device according to the set relation.
24. The computer program according to claim 22, wherein the
computer program executes the step of: changing the relation
between the rotation degree around the x-axis and/or the y-axis and
the amount of the displacement of the portion on the virtual data
object in proportion to the distance between the viewpoint and the
display device.
25. The computer program according to claim 22, wherein the
movement of the portion of the virtual data object displayed on the
display device is proportional to the change amount and/or rate of
the rotational movement around the x-axis and/or y-axis.
26. The computer program according to claim 22, wherein the
computer program executes the steps of: setting the display device
into a zoom mode; determining the distance between the user of the
hand-held device to the display device; and zooming in or out the
displayed information based on the determined distance
information.
27. The computer program according to claim 22, wherein the
computer program executes the steps of: setting the display device
into a zoom mode; and zooming in or out the displayed information
when rotating the hand-held device around the axis being
essentially perpendicular to the surface level of the hand-held
device.
28. The computer program according to claim 22, wherein the
computer program executes the steps of: setting the display device
into a zoom mode; and zooming in or out the displayed information
when the hand-held device is tilted.
29. The computer program according to claim 22, wherein the
information displayed on the display device essentially depends on
the location and orientation of the virtual display, the viewpoint
and the virtual data object.
30. The computer program according to claim 22, wherein the
computer program executes the steps of: setting the display device
surface level as an xy-plane; determining a relation between the
x-axial and/or y-axial movement of the hand-held device and the
amount of the displacement of the portion of the virtual data
object displayed on the display device at a time; and moving the
portion of the virtual data object displayed on the display device
in the same direction as the hand-held device is moved in the
xy-plane according to the relation information.
31. The computer program according to claim 22, wherein filtering
the x-axial, y-axial and/or tilting movements before displaying the
movements on the display device.
32. The computer program according to claim 22, wherein changing
the relation between the rotation degree around the x-axis and
y-axis and the amount of the displacement of the portion of the
virtual data object displayed on the display device at a time.
33. The computer program according to claim 22, wherein the
computer program executes the step of: keeping the orientation of
the information displayed on the display device unchanged when
rotating the hand-held device around the axis being essentially
perpendicular to the surface level of the hand-held device.
Description
FIELD OF THE INVENTION
[0001] The present invention relates to display devices where
information can be browsed. In particular, the present invention
relates to a novel and improved method and system for browsing
information with hand-held devices with a display device.
BACKGROUND OF THE INVENTION
[0002] Various electronic mobile devices, e.g. mobile phones,
computers, Personal Digital Assistants (PDA, comprise displays. The
transfer of the information to be viewed on the display is executed
at least partially by a processor. A device typically comprises
also a keypad with which the user of the device enters various
commands. There are also touch-sensitive displays (touch screens).
There a separate keypad is not needed. A device is controlled by
touching the touch screen.
[0003] The display of a mobile device is capable of showing only
limited amount of information at a time. Because of the size of the
display, e.g. a large image must be viewed part by part. In order
to view such an image, the user of the device controls the display,
e.g. by scrolling the display with a mouse etc.
[0004] Devices equipped with a display have different kinds of user
interfaces with which the user interacts with the device. There are
graphical user interfaces and speech controlled user interfaces. A
graphical user interface can be controlled with various control
devices including, for example, keypad, touch screen, different
kinds of cursor controlling methods, etc.
[0005] There are, however, drawbacks in the prior-art devices in
the usability of the device, especially in the browsing of
information with the device. When the information to be viewed on
the display must be viewed by parts, it is difficult and slow to
browse the whole information part by part. It is, for example,
difficult to display a wide panorama picture on the display, while
at the same time quickly and easily browsing the picture.
[0006] For the user of a mobile hand-held device it is difficult to
perceive visual entireties that can not be displayed at a time on
the display. Therefore the browsing of the information should be
carried out as naturally and logically as possible. A user of a
mobile hand-held device must be able to learn and use the device
easily and efficiently.
[0007] From prior-art solutions it is known to use location
detectors for browsing information with a device. Reference
publication WO 9918495 (Telefonaktiebolaget L M Ericsson) describes
a method where the display device is moved essentially in the plane
of the display device, whereby different parts of a complete screen
image are shown on said display device. When the display device is
moved essentially in a direction perpendicular to the plane of the
display device, the magnification of the screen image changes. The
movement in the plane is a bit problematic. In the plane movement
the necessary movements may be quite remarkable/large, and it may
be difficult to maintain the display device in a proper position
for reading or browsing.
[0008] Another prior-art solution is to use tilt detectors for
moving, or to be more specific, for scrolling the view on the
display device. One solution of this kind is described in WO
9814863 (Philips). When the screen image is moved by scrolling
(tilting the display device), the result is better than in moving
the display device in the plane of the display device, as described
above. However, to move the screen image fluently and to return
from some point to the initial point of browsing is difficult
because controlling a discontinuous motion requires continuous and
precise handling of the display device. The controlling of the
scrolling movement can be compared to a movement of a ball on a
plane surface by tilting the plane. In order to stop the rolling of
the ball, the plane surface must be perpendicular against the
gravity of the earth. In other words, the control of the movements
and usability are not at an acceptable level so that the use of
such a device would be natural and logical.
[0009] There are also various kinds of motion and/or location
controlled display devices used in, e.g. in virtual helmets. There
the display device focuses like a virtual camera. The display
device displays an object to which the device (camera) points in
the modelled virtual environment. To use a virtual camera model in
a hand-held device is not so straightforward because displaying
peripheries of a large screen image results in a disadvantageous
viewing angle. Therefore, the adjustment and zooming of a display
image must be implemented in a most natural and logical manner. In
prior-art solutions the browsing of information on the display
device is slow and awkward because the solutions are based on
artificial logic.
THE PURPOSE OF THE INVENTION
[0010] The goal of the present invention is to adjust the view on
the display device in a manner as natural as possible so that the
user of the hand-held device can concentrate on the information
displayed on the display device and not on the adjustment of the
displayed information.
SUMMARY OF THE INVENTION
[0011] The present invention describes a method, hand-held device
and computer program for browsing information on a display device
of a hand-held device. In the present invention, the display device
is coupled to a processor mapping the information content generated
by the processor into the virtual data object suitable for
conveying the information to the user of the hand-held device. The
display device displays a portion of the virtual data object at a
time on the display device. The virtual data object comprises e.g.
characters, pictures, lines, links, video or pixels that can be
conveniently displayed on the display device at a time.
[0012] The idea of the present invention is to browse information
on the display device of a hand-held device naturally and
logically. Characteristic of the invention is that information is
browsed on the display device essentially in a mirror-like way. In
other words, the portion of the virtual data object displayed on
the display device is moved at the same direction as the hand-held
device is tilted. In other words, the movements of the portion of
the virtual data object displayed on the display device depends on
the orientation of the hand-held device. An important feature of
the invention is also that a certain orientation of the hand-held
device always displays the same portion of the virtual data object
on the display device. The browsing method described above is
extremely logical, and the movements and responses to the movements
are natural.
[0013] The core functions of the browsing can be explained by means
of the following example. The information is browsed with the
hand-held device essentially in the same way as looking at a view
from a hand mirror. The hand mirror is typically held in hand quite
close to the viewer. The hand mirror represents the display device
and the view behind the viewer the virtual data object. When the
hand mirror is tilted, the view behind the viewer moves in response
to the changes in the orientation of the hand mirror.
[0014] When approaching the functionality of a hand mirror the
browsing of information on a display device of a hand-held device
is made natural and logical.
[0015] The present invention is most applicable with hand-held
devices with a display when a large data object is displayed by
parts on the display. With the present invention, a large data
object can be browsed naturally and logically from the user's
perspective. The position memory of the muscles of a human body
makes it easier to return to previously browsed points and to the
starting point.
[0016] The present invention also reduces the need to use exterior
mechanical switches, keypad or other known control mechanisms for
browsing information on the display device. Therefore the use of a
hand-held device is easier and more simple. The basic
functionalities of the present invention can be implemented with
mass production components, and with moderate processing power.
Thus, the features described in the present invention can be taken
in use in consumer products without notable expense increase.
[0017] Other objects and features of the present invention will
become apparent from the following detailed description considered
in conjunction with the accompanying drawings. It is to be
understood, however, that the drawings are designed solely for
purposes of illustration and not as a definition of the limits of
the invention, for which reference should be made to the appended
claims. It should be further understood that the drawings are not
necessarily drawn to scale and that, unless otherwise indicated,
they are merely intended to conceptually illustrate the structures
and procedures described herein.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings, which are included to provide a
further understanding of the invention and constitute a part of
this specification, illustrate embodiments of the invention and
together with the description help to explain the principles of the
invention. In the drawings:
[0019] FIG. 1 illustrates how the hand-held device is operated
according to the present invention,
[0020] FIGS. 2a, 2b and 2c illustrate more specific examples of how
the hand-held device of FIG. 1 is handled,
[0021] FIG. 3 illustrates an exemplary viewing setup of the present
invention,
[0022] FIG. 4 illustrates an example of how a view on the display
device can be formed and calculated according to the viewing setup
of FIG. 3,
[0023] FIG. 5 is a block diagram illustrating an embodiment of the
hand-held device in accordance with the present invention,
[0024] FIG. 6 is a block diagram illustrating another embodiment of
the hand-held device in accordance with the present invention,
[0025] FIGS. 7a, 7b, 7c and 7d illustrate the view change of the
display of the hand-held device in response to user actions,
[0026] FIGS. 8a, 8b and 8c illustrate different ways of browsing
information, and
[0027] FIG. 9 is a flow diagram illustrating the operation of a
preferred embodiment of the present invention.
DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS
[0028] FIG. 1 illustrates a simplified portable hand-held device
according to the present invention. The hand-held device is e.g. a
mobile phone or a Personal Digital Assistant (PDA). The display
device of the hand-held device displays information stored on a
memory of the hand-held device. The hand-held device is explained
more specifically in later examples. FIG. 1 represents the basic
browsing functionality. Information is browsed on the display
device by tilting (rotating) the hand-held device 40 towards
directions 2,3,4 and 5 around the axis 6 and 7. The memory of the
hand-held device 40 comprises a virtual data object comprising
characters, pictures, lines, links, video or pixels that can be
conveniently displayed on the display device at a time. A portion
of the virtual data object displayed on the display device is moved
at the same direction as the hand-held device is tilted. Moreover,
a certain orientation of the hand-held device 40 always displays
the same portion of the virtual data object on the display
device.
[0029] FIGS. 2a, 2b and 2c represent a more specific example of
tilting the hand-held device 40. It can be said that a typical
starting situation is that the hand-held device 40 is in a 20-30
degree angle with the horizontal plane 8. This plane is in one
embodiment set as a default xy-plane from which the rotation angles
of the hand-held device 40 are measured. It can also be said that
this starting point is the most appropriate one for viewing
information with the display device. So when the user tilts the
hand-held device 40, the viewing angle changes. The view on the
display device changes in real time to correspond to the new
viewing angle. A very important feature of the invention is that
the view on the display device depends on the viewing angle, and
the same viewing angle displays always the same view on the display
device. This feature is very natural and logical.
[0030] In FIG. 2a, angle .alpha. corresponds to the aforementioned
20-30 degrees. FIG. 2a is regarded as a starting position when the
browsing begins. In FIG. 2b, the hand-held device 40 has been
tilted to an angle .beta..sub.1, which is smaller than angle
.alpha.. The view on the display device changes based on the
tilting movements essentially in real time, and the movement of the
information on the display device is towards the same direction as
the hand-held device 40 is tilted. In FIG. 2c, the hand-held device
40 is tilted to an angle .beta..sub.2, which is bigger than angle
.alpha..
[0031] In one embodiment, the angle (.alpha.) is a predetermined
angle, and it is determined by the manufacturer of the hand-held
device 40. In the determination process it is defined that the
display view plane is based on axis x_VD and y_VD, which are
perpendicular to each other. The hand-held device is then set to a
certain position (.alpha.), and that position is set as a default
xy-plane. In FIGS. 2a, 2b and 2c, the default plane is determined
based on angle .alpha.. In another embodiment, the default plane
can be freely determined based on any x-axis, y-axis and/or
z-axis.
[0032] From that moment on, the hand-held device 40 is tilted in
respective to this plane. When the default xy-plane is fixed, the
user of the hand-held device is always capable of returning to a
certain view by tilting the device back to the original orientation
when the sensors measuring the orientation of the hand-held device
do not cause any restrictions to the measured position. In another
embodiment, the angle .alpha. can be readjusted to a desired
value.
[0033] FIGS. 3 and 4 represent an exemplary embodiment of the setup
of a "mirroring system". It includes a viewpoint VP, a virtual
screen VS and a virtual display VD. The viewpoint VP represents the
location of a viewer of a hand-held device. The VD represents the
display device of the hand-held device. The virtual screen
represents the actual information browsed on the display
device.
[0034] For simplicity in the following the viewpoint VP is defined
to be at point [0 0 0]. Furthermore, the middle point of the
virtual display VD is defined to be at P_xyz wherein
P_xyz=[P_xyz.sub.1P_xyz.sub.2P_xyz.sub.- 3].sup.T, and the virtual
screen VS to be at plane x=kuva_shift.
[0035] The orientation of the virtual display VD is defined by
tilting angels .alpha..sub.x,.alpha..sub.y,.alpha..sub.z indicating
rotation angle over each coordinate axe. In FIG. 4, the virtual
display VD is a plane and has some size. Each coordinate in this VD
plane is defined using notation P=[P_xyz.sub.2+peili_y
P_xyz.sub.3+peili_z] when the orientation
[0036] of the VD is defined to be parallel with the x-plane.
[0037] It must be noted that FIGS. 3 and 4 represent only one
embodiment of the possible positions of the VS, VP and VD, and the
axes used.
[0038] In order to the determine the orientation of the VD, two
orthogonal vectors (in the x-plane) are defined as follows:
L=[0,1,-1].sup.T
M=[0,1,1].sup.T
[0039] Those vectors present the orthogonal direction vectors of
the VD. Next, the orientation of the virtual display VD is defined
using the rotation angles: 1 R x = [ 1 0 0 0 cos ( x ) - sin ( x )
0 sin ( x ) cos ( x ) ] R y = [ cos ( y ) 0 sin ( y ) 0 1 0 - sin (
y ) 0 cos ( y ) ] R z = [ cos ( z ) - sin ( z ) 0 sin ( z ) cos ( z
) 0 0 0 1 ]
[0040] Next the unit normal vector of the VD is calculated: 2 PT 1
= R x R y R z L PT 2 = R x R y R z M PNT = PT 1 .times. PT 2 (
cross product ) PN = PNT | PNT |
[0041] where PN is the unit normal vector of the VD-plane. The PN
defines the applicable orientation of the VD to be used in the
projection calculation.
[0042] Next, the "image" on the virtual display VD is calculated.
Let's assume that there is a vector beginning from the VP and being
reflected via the VD. The point where the reflected vector hits on
the plane VS defines the projection of the point on the VS to the
point on the VD-plane. Hence, if all points on VD are processed as
described above, the image on the VD can be defined.
[0043] The idea of calculation is presented using vectors in FIG.
4. Using the vectors the algorithm works as follows:
[0044] 1. The points P and VP define a vector A.
[0045] 2. The projection proj of the vector A on the normal vector
PN is calculated.
[0046] 3. The sum of the vector A and proj*PN defines a point
Q.
[0047] 4. The points Q and VP define a vector B.
[0048] 5. The point defined as sum of the VP and 2*B defines a
point R.
[0049] 6. The direction vector that goes via P and R defines a
direction vector that hits the plane VS at point S.
[0050] 7. The result of this process is that the image of point P
in VD is the image of point S in VS.
[0051] By repeating phases 1-7 for all points in the VD-plane the
whole image of the virtual display VD is defined. Using vector
calculation the same can be presented as follows:
[0052] First the point P is defined:
P=P.sub.--xyz+R.sub.xR.sub.yR.sub.z[0 peili.sub.--y
peili.sub.--z].sup.T
[0053] where P_xyz is the coordinate of the middle point of the VD,
peili_y is the y-coordinate on the VD plane-coordinate system and
peili_z is the z-coordinate on the VD plane-coordinate system.
[0054] Next, the projection on the normal vector is defined: 3 A =
P - V P proj = A P N | P N |
[0055] Hence the point Q can be defined:
Q=P-proj*PN
[0056] Further, the point R can be defined (the reason for the
factor 2 is that in mirror the arriving and departing light beam
have equal angles compared to the normal vector of the
surface).
B=Q-VP
R=VP+2*B
[0057] And finally the direction vector C is defined as
follows:
C=R-P.
[0058] Because the VS is located at plane x=kuva_shift, the vector
C hits that plane at the point
S=k*C+P
[0059] where 4 k = - P 1 + kuva_shift C 1
[0060] where P.sub.1 is the x-component of the point P and C.sub.1
is the x-component of the vector C. Note that in this calculation
the VP was defined to the origin to simplify the presentation of
the algorithm. However, in practice the VP can locate freely in the
coordinate space. It must be noted that the image on the virtual
screen VS is horizontally inversed when the virtual screen VS is
viewed from the viewpoint VP direction.
[0061] The system of FIG. 4 has several charasteristics:
[0062] 1. The view on the display device moves into the same
direction as it is tilted. In one embodiment, the movement of the
portion of the virtual data object displayed on the display device
is proportional to the change amount and/or rate of the rotational
movement.
[0063] 2. When the distance between the VP and VD increases, the
same tilting angle causes greater movements on the virtual screen
VS. In other words, the browsing speed of the information on the
display device increases as the distance between the VP and VD
increases. In one embodiment, this movement factor can be adjusted
by the user of the hand-held device.
[0064] 3. When rotating the display device, the view on the display
device remains unchanged in relative to the user.
[0065] 4. The view on the display device depends on the position
and orientation of the VS, VP and VD.
[0066] 5. A certain VS-VP-VD position/orientation combination
always constitute the same view on the display device.
[0067] 6. When the position of the VD alters, the viewing angle
between the VP and VD changes.
[0068] 7. Zooming can be implemented by changing the position of
the VS, VP and VD.
[0069] 8. Zooming can be implemented by enlarging the object on the
VS or altering the radius of curvature of the mirror (VD).
[0070] 9. If the figure on the VS is in the right way when viewed
from the VP, the view on the VD is mirrored (horizontally
inversed).
[0071] The present invention does not have to implement all the
aforementioned features, but the most appropriate ones can be
chosen. The ideal mirror-like functionality means that the
information on the display device changes when:
[0072] a) the location or orientation of the hand-held device in
proportion to the coordinates bound to the physical environment
changes,
[0073] b) the location of the user (VP) in proportion to the
coordinates bound to the hand-held device changes,
[0074] c) the virtual location of the data (virtual screen)
displayed on the display device in proportion to the coordinates
bound to the physical environment changes.
[0075] In order to simulate the operation of a mirror to the user,
the information on the display device is changed at least either
according to a) or b). If only a) or b) is taken into
consideration, the operation of the display is not so mirror-like
as if both a) and b) were implemented. In one embodiment, the
display device operates according to all a), b) and c).
[0076] FIG. 5 represents one example of a preferred hand-held
device 40. The hand-held device 40 is e.g. a mobile phone. The
hand-held device comprises a processor 30 and a display device 10
coupled to the processor 30. The data memory 60 and the program
memory 70 are also coupled to the processor 30. The program memory
70 contains e.g. the operation system. The sizes of the memories,
and the processing power of the processor 30 depend on the device
and application used. The program memory 60 can additionally
contain different kinds of software applications with which various
tasks can be executed. Application software comprise e.g. word
processing, graphical and spreadsheet software. The software
applications and data used by them are loaded into the data memory
60 in order to be able to use the software.
[0077] The display adapter 90 with the processor 30 controls the
display device 10. In order to not to use the data memory 60 for
storing display-related information, the display adapter 90
comprises a data buffer in which the information to be displayed on
the display device 10 is stored.
[0078] The hand-held device 40 comprises measuring means which in a
preferred embodiment of the invention refer to acceleration
sensor(s) 50. With the acceleration sensor(s) 50 it is possible to
measure tilting movements of the hand-held device 40. The processor
30 receives the measurement results and interprets them. The
acceleration sensor(s) 50 can be e.g. piezo-electric or capacitive
producing an analog voltage which is proportional to the
acceleration factor.
[0079] With the acceleration sensor(s) 50 it is possible to measure
one, two or three-dimensional accelerations. The measurement of
tilting movements is based on the fact that the highest
acceleration is parallel to the gravity of the earth. Therefore,
the orientation of the hand-held device 40 can be defined in
relation to the earth. It is also possible to use gyroscopes with
its various forms to measure the orientation of the hand-held
device 40. The quantities measured are e.g. tilting angle and
accelerations.
[0080] The relation information between the rotation degree of the
hand-held device and the memory address corresponding to the
displayed view is stored e.g. on the data memory 60. The processor
30 defines the orientation of the hand-held device 40 in relation
to the user or a reference position. The processor 30 may also
define the distance between the user and the hand-held device 40 or
the user orientation in relation to the hand-held device 40.
[0081] The most important point is not the way of how the
aforementioned definitions are made but the fact that the
orientation of the hand-held device 40 affects the information
displayed on the display device 10. The memory space can be
implemented logically, e.g. as a two-dimensional memory space. When
browsing starts, the processor 30 starts the definition process of
the new memory address from the current memory address so that
displacement in the memory space corresponds to the direction and
amount of change in orientation according to the relation
information.
[0082] The hand-held device 40 comprises also a browse lock 80 with
which it is signalled when the browsing is executed. The
orientation of the hand-held device 40 must remain in the same
position in order to keep the view on the display device unchanged.
In a preferred embodiment, the hand-held device 40 comprises a lock
feature, e.g. a push-button, with which the browsing can be locked.
The user can tilt the hand-held device back to an appropriate
viewing orientation in order to view the information on the display
device 10 properly. The browsing may then continue when the button
is released.
[0083] The hand-held device 40 in FIG. 6 is almost the same as the
hand-held device 40 in FIG. 5. In FIG. 5, the hand-held device
comprises also a locator 20. It is possible to control the view on
the display device 10 also by other means than acceleration
sensor(s) or equivalent means. The hand-held device 40 can comprise
e.g. a (video) camera measuring the orientation and location of the
hand-held device in relation to the user of the hand-held device 40
or to another reference point in the surroundings of the user. The
camera 20 may be set to recognise and measure distance to a certain
reference point, e.g. the eyes of the user. Therefore, when the
orientation and/or position of the hand-held device 40 changes, the
viewing angle measured by the camera also changes. Thus, it can be
concluded that the hand-held device 40 has been tilted and/or moved
towards some direction.
[0084] By analysing the video image it is possible to define the
orientation of the hand-held device 40 in proportion to the
reference point and the distance of the hand-held device 40 to the
reference point tens of times within a second. The browsing
functionality can be implemented merely using the video camera, so
that additional acceleration sensor(s) are not necessarily needed.
The measuring of the distance can also be implemented with an
ultrasonic radar connected through an analog-digital converter to
the processor 30 of the hand-held device 40. In one embodiment,
from the user's perspective the information on the display device
10 is essentially browsed in the same manner as when looking in a
mirror. In other words, the view on the display 10 depends on the
viewing angle in relation to the display device plane as the view
in a mirror depends on the viewing angle to the mirror.
[0085] In one embodiment of FIG. 5, the locator 20 comprises a
video camera seeking the location of the head and eyes of the user.
Heuristic algorithms and neural network seeking the location of the
head and eyes can be used. Acceleration sensors are more
appropriate to use in hand-held devices than a video camera,
because they are cheaper. The acceleration sensors may also be a
more appropriate solution in devices which do not have a built-in
video camera for a default feature, e.g. in the (third generation)
mobile phones. The advantage of the use of the video camera is that
the use of the hand-held device is not restricted to the position
of the hand-held device, e.g. when being on one's back the
hand-held device can be used without problems. Also the selection
of starting point of browsing is more free, and choice (of the
starting point) can be given to the user of the hand-held device.
In one embodiment of FIG. 5, the display device surface level is
set as an xy-plane. A certain relation between the x-axial and/or
y-axial movement of the hand-held device and the amount of the
displacement of the portion of the virtual data object displayed on
the display device at a time has been determined. So, when the
hand-held device 40 is moved along x- and/or y-axis, the portion of
the virtual data object displayed on the display device moves in
the same direction as the hand-held device is moved in the xy-plane
according to the relation information.
[0086] In a preferred embodiment of FIGS. 5 and 6 the processor 30
comprises also means for filtering the x-axial, y-axial and/or
tilting movements before displaying the movements on the display
device. Therefore, minor unintentional movements can be filtered
out.
[0087] In one embodiment of FIGS. 5 and 6, the relation between the
tilting movements and the amount of the displacement of the portion
of the virtual data object displayed on the display device at a
time can be changed. Therefore, a user may define e.g. that from
now on a 10 degree tilting causes the same effect on the display as
a 15 degree tilting earlier. In one embodiment, the relation is
linear. In other words, the relation between the tilting movements
and the amount of the displacement of the portion of the virtual
data object displayed on the display device at a time does not
depend on the amount of the tilting. In another embodiment, the
relation is linear, but e.g. exponential. In other words, the
amount of the displacement of the portion of the virtual data
object displayed on the display device at a time depends on the
amount of the tilting. For example, the value of the relation
factor changes (e.g. exponentially) as the tilting amount
increases.
[0088] FIGS. 7a-7d represent the situation where the size of the
information on the display device depends on the zoom factor in
addition to the orientation of the hand-held device. The zoom
factor can be controlled in different ways. In one embodiment, the
zoom factor depends on the distance between the user and the
hand-held device. FIG. 7a represent the display device 10, on which
graphical FIGS. 21, 22 and 23 are seen. The view on the display
device 10 depends on the orientation of the hand-held device or the
viewing angle from which the user of the hand-held views the
display device. When the user of the hand-held device sets FIG. 21
in the middle of the display device, and the zoom factor is
increased, FIG. 21 grows as depicted in FIGS. 7b and 7c. In FIG.
7d, the zoom factor has decreased, and also the viewing angle
between the user and the hand-held device has changed.
[0089] The zoom factor can be modified with several different ways.
In one embodiment, the zoom factor depends on the distance between
the reference point (e.g. the eyes of the user) and the hand-held
device. When the distance decreases, FIG. 21 grows, and vice versa.
The display device 10 may have to be set to a zoom mode before the
zoom factor changes. If the zoom factor was all the time dependent
on the distance between the reference point and the hand-held
device, the browsing operation would not necessarily be practical
because the view on the display 10 would change whenever the
aforementioned distance changes.
[0090] In another embodiment, the zoom factor changes when rotating
the hand-held device around the axis being essentially
perpendicular to a predefined xy-plane. The xy-plane may be the
present plane of the display device 10 or some other predetermined
plane. Yet in another embodiment, the zoom factor is changed by
tilting the hand-held device. Before this the display device must
be set into a zoom mode. When the hand-held device is tilted, e.g.
to the right the zoom factor increases, and when the hand-held
device is tilted to the left, the zoom factor decreases. It is not
important which predefined tilting directions are used but that the
two directions can be separated sufficiently from each other. The
aforementioned zoom mode is set on and off e.g. with a
predetermined button of the hand-held device.
[0091] FIGS. 8a-8c represent different ways to implement the user
interface. In FIG. 8a, the display device 10 of the hand-held
device 40 contains information to be viewed by the user. In FIG.
8a, an A letter is on the display device 10, In one embodiment, the
information on the display device 10 remains in the same position
with respect to the user when the hand-held device 40 is rotated
around the axis being perpendicular to the display surface plane,
as depicted in FIG. 8b. In other words, the information on the
display device 10 remains in the same position because the
information is attached to the real physical coordinates.
[0092] In another embodiment, the information on the display device
10 remains in the same position with respect to the hand-held
device 40 when the hand-held device 40 is rotated around the axis
being perpendicular to the display surface plane, as depicted in
FIG. 8c. In other words, the orientation of the information on the
display device 10 changes with respect to the user of the hand-held
device 40 because the information is not attached to the real
physical coordinates but to the display device.
[0093] FIG. 9 represents a flow diagram describing the
functionality of a method of the present invention. FIG. 9
describes a hand-held device 40 comprising means for measuring
acceleration 50 and a processor 30. Means for measuring
acceleration refer e.g. to a multiaxial acceleration sensor suited
for measuring changes in the orientation of the hand-held device
40.
[0094] The hand-held device is switched on, and it is ready for
browsing information on the display device, as represented in phase
100. When the hand-held device is functional, the acceleration
sensor 50 measures constantly acceleration readings. The processor
30 receives the acceleration readings and defines the orientation
of the hand-held device and also the change in the orientation
compared to the prior measurement(s), as represented in phases 101
and 102. In phase 103, it is tested whether the browsing is on or
off. If the browsing is off, the processor 30 examines if a
predetermined browsing startup condition is fulfilled (phase 104).
If it is not fulfilled, the method returns back to phase 101. It
means that the orientation of the hand-held device has not changed
sufficiently, which would indicate that the user wishes to browse
information on the display device of the hand-held device.
[0095] If the predetermined browsing startup condition is
fulfilled, the processor 30 sets the browsing as started (phase
106) and determines the browsing speed based on the current
acceleration value (phase 108). The processor 30 also changes the
information presented on the display device according to a relation
between the rotation degree and the amount of the displacement of
the portion on the virtual data object stored on the data memory 60
and the determined browsing speed (phase 108). A certain
orientation of the hand-held device always causes the same view
(the same portion on the virtual data object stored on the memory)
on the display device. If it is observed in phase 103 that the
browsing is already on, and the browsing stopping condition is
fulfilled (phase 105), the processor 30 stops the browsing and sets
the browsing as stopped (phases 107 and 109). If it is observed
that the browsing stopping condition is not fulfilled (phase 105),
the processor 30 returns back to phase 101.
[0096] It is obvious to a person skilled in the art that with the
advancement of technology, the basic idea of the invention may be
implemented in various ways. The invention and its embodiments are
thus not limited to the examples described above, instead they may
vary within the scope of the claims.
* * * * *