U.S. patent application number 15/883456 was filed with the patent office on 2018-08-09 for electronic device having force-based modifiable graphical elements and method of operating same.
This patent application is currently assigned to NANOPORT TECHNOLOGY INC.. The applicant listed for this patent is NANOPORT TECHNOLOGY INC.. Invention is credited to David Michael Lopez REYES, Timothy Jing Yin SZETO.
Application Number | 20180225030 15/883456 |
Document ID | / |
Family ID | 61525552 |
Filed Date | 2018-08-09 |
United States Patent
Application |
20180225030 |
Kind Code |
A1 |
SZETO; Timothy Jing Yin ; et
al. |
August 9, 2018 |
ELECTRONIC DEVICE HAVING FORCE-BASED MODIFIABLE GRAPHICAL ELEMENTS
AND METHOD OF OPERATING SAME
Abstract
There is described a method of operating an electronic device
having a housing and a user interface mounted to the housing. The
user interface includes force sensor(s) on an edge of the housing
and a display screen. The method includes displaying, on the
display screen, a graphical element having an edge portion;
receiving an input from the force sensor(s) on the edge, the input
being a measured value of a force applied on the force sensor(s);
and in response to said input received, modifying the graphical
element on the display screen in a manner to move, to a destination
position, the edge portion depending on a direction of movement of
the edge portion, the destination position of the edge portion
depending on an amplitude of the measured value.
Inventors: |
SZETO; Timothy Jing Yin;
(Mississauga, CA) ; REYES; David Michael Lopez;
(Toronto, CA) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
NANOPORT TECHNOLOGY INC. |
Markham |
|
CA |
|
|
Assignee: |
NANOPORT TECHNOLOGY INC.
Markham
ON
NANOPORT TECHNOLOGY INC.
Markham
ON
|
Family ID: |
61525552 |
Appl. No.: |
15/883456 |
Filed: |
January 30, 2018 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
15585309 |
May 3, 2017 |
9916073 |
|
|
15883456 |
|
|
|
|
62447406 |
Jan 17, 2017 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 3/0414 20130101;
G06F 3/041 20130101; G06F 2203/04105 20130101; G06F 3/0482
20130101; G06F 3/04845 20130101 |
International
Class: |
G06F 3/0484 20130101
G06F003/0484; G06F 3/041 20060101 G06F003/041 |
Claims
1-20. (canceled)
21. A computer-implemented method of operating an electronic device
having a housing and a user interface mounted to the housing, the
user interface including at least one force sensor on a first edge
of two opposite edges of the housing and a display screen, the
computer-implemented method comprising: displaying, on the display
screen, a graphical element having a first edge portion; receiving
a first input from the at least one force sensor on the first edge,
the first input resulting from a force applied on the at least one
force sensor on the first edge; and in response to said first input
received, modifying the graphical element displayed on the display
screen including moving the first edge portion of the graphical
element in the direction of the force applied and away from the
first edge, to a destination position.
22. The computer-implemented method of claim 21 wherein the
modifying further comprises deforming the first edge portion from a
straight edge into a curved edge, a peak of the curved edge
reaching the destination position.
23. The computer-implemented method of claim 22 wherein the
straight edge represents a relaxed state, and the curved edge
represents a stressed state, and the deformation resembles a shape
of a real-world elastic material receiving the force.
24. The computer-implemented method of claim 21 further comprising
modifying the graphical element back to an initial configuration
upon determining that the applied force has been released from the
at least one force sensor on the first edge.
25. The computer-implemented method of claim 21 wherein the user
interface further has at least a second force sensor on a second
edge of the two opposite edges of the housing, the second force
sensor being disposed at a position along the second edge opposite
a position of the first force sensor along the first edge; further
comprising receiving a second input from the second force sensor on
the second edge resulting from a corresponding second force, and
moving a second edge portion of the graphical element from an
initial position adjacent a second edge of the display screen, in
the direction of the second force and away from the second edge, to
a second destination position.
26. The computer-implemented method of claim 25 wherein the first
edge portion and the second edge portion are moved
simultaneously.
27. The computer-implemented method of claim 26 further comprising
activating a function of the electronic device after both the first
edge portion and the second edge portion have reached their
corresponding destination positions.
28. The computer-implemented method of claim 21 wherein said
receiving a first input includes measuring a first measured value
of a force applied on the at least one force sensor on the first
edge, and the destination position depends on an amplitude of the
first measured value.
29. The computer-implemented method of claim 28 wherein the
destination position of the first edge portion reaches a maximal
recessed destination position when the amplitude of the first
measured value exceeds a given amplitude threshold.
30. The computer-implemented method of claim 29 further comprising
activating a target function of the electronic device when the
amplitude of the first measured value exceeds the given amplitude
threshold.
31. The computer-implemented method of claim 21 wherein the user
interface includes a plurality of force sensors having associated
locations along the first edge, said receiving including receiving
a plurality of first inputs from the plurality of force sensors on
the first edge, the plurality of first inputs resulting from one or
more forces applied on at least one of the plurality of force
sensors on the first edge.
32. The computer-implemented method of claim 31 wherein the
plurality of first inputs are a plurality of first measured values
of the one or more forces applied on a corresponding one of the
plurality of force sensors on the first edge.
33. The computer-implemented method of claim 32 wherein the first
edge portion has a width extending along the first edge and
adjacent a location associated with a maximal one of amplitudes of
the plurality of first measured values.
34. The computer-implemented method of claim 33 further comprising
determining a location offset between the location associated with
the maximal one of the amplitudes of the plurality of first
measured values and a location of a graphical button displayed on
the display screen.
35. An electronic device comprising: a housing having a first edge
and a second edge opposite the first edge; a user interface mounted
to the housing, the user interface including at least one force
sensor on the first edge of the housing and a display screen; a
processor housed within the housing and in communication with the
user interface, the processor being configured to execute steps of:
displaying, on the display screen, a graphical element having a
first edge portion; receiving a first input from the at least one
force sensor on the first edge, the first input resulting from a
force applied on the at least one force sensor on the first edge;
and in response to said first input received, modifying the
graphical element displayed on the display screen including moving
the first edge portion of the graphical element in the direction of
the force applied and away from the first edge, to a destination
position.
36. The electronic device of claim 35 wherein the modifying further
comprises deforming the first edge portion from a straight edge
into a curved edge, a peak of the curved edge reaching the
destination position.
37. The electronic device of claim 36 wherein the straight edge
represents a relaxed state, and the curved edge represents a
stressed state, and the deformation resembles a shape of a
real-world elastic material receiving the force.
38. The electronic device of claim 35 further comprising modifying
the graphical element back to an initial configuration upon
determining that the applied force has been released from the at
least one force sensor on the first edge.
39. The electronic device of claim 35 wherein the user interface
further has at least a second force sensor on a second edge of the
two opposite edges of the housing, the second force sensor being
disposed at a position along the second edge opposite a position of
the first force sensor along the first edge; the processor being
further configured to execute steps of receiving a second input
from the second force sensor on the second edge resulting from a
corresponding second force, and moving a second edge portion of the
graphical element from an initial position adjacent a second edge
of the display screen, in the direction of the second force and
away from the second edge, to a second destination position.
40. The electronic device of claim 39 wherein the first edge
portion and the second edge portion are moved simultaneously.
Description
FIELD
[0001] The improvements generally relate to the field of electronic
devices and more particularly to electronic devices having force
sensitive user interfaces.
BACKGROUND
[0002] Conventional electronic devices such as mobile phones or
tablet computers typically have a user interface that includes
sensors (e.g., buttons) to receive user inputs. In some cases,
these sensors can receive accidental inputs (e.g., pocket dials)
that can cause the electronic device to perform an undesired
function.
[0003] There remains room for improvement.
SUMMARY
[0004] In at least some cases, electronic devices include a user
interface with one or more force sensors for measuring a force
being applied on a corresponding one of the force sensors by one or
more user touches, and a display screen to display graphical
elements.
[0005] This disclosure relates to the modification of a graphical
element displayed on the display screen in response to the
measurement of a force being applied on a corresponding force
sensor. The modification can include the movement of the graphical
element in a manner which deforms it and/or changes its shape. Such
modifiable graphical elements can help a user to confirm that the
one or more touches have in fact been received by the electronic
device, which may reduce the amount of accidental inputs and which
may provide an enhanced feeling of unity with the electronic
device. In some embodiments, the more the force measured increases,
the more the graphical element is modified, until a given force
threshold is exceeded, wherein a target function may be activated.
In some other embodiments, the modification of the graphical
element can be based on a confidence that the input received is
actually desired by the user, rather than simply being an
accidental input.
[0006] In accordance with one aspect, there is provided a
computer-implemented method of operating an electronic device
having a housing and a user interface mounted to the housing, the
user interface including at least one force sensor on a first edge
of two opposite edges of the housing and a display screen, the
computer-implemented method comprising: displaying, on the display
screen, a foreground graphical element having a first edge portion,
and a background graphical element protruding from the first edge
portion of the foreground graphical element; receiving a first
input from the at least one force sensor on the first edge, the
first input being a first measured value of a force applied on the
at least one force sensor on the first edge; and in response to
said first input received, modifying the foreground graphical
element on the display screen in a manner to move, to a destination
position, the first edge portion of the foreground graphical
element and thereby one of reveal and hide a first corresponding
portion of the background graphical element, depending on a
direction of movement of the first edge portion, the destination
position of the first edge portion depending on an amplitude of the
first measured value.
[0007] In accordance with another aspect, there is provided an
electronic device comprising: a housing having a first edge and a
second edge opposite the first edge; a user interface mounted to
the housing, the user interface including at least one force sensor
on the first edge of the housing and a display screen; a processor
housed within the housing and in communication with the user
interface, the processor being configured to execute steps of:
displaying, on the display screen, a foreground graphical element
having a first edge portion, and a background graphical element
protruding from the first edge portion of the foreground graphical
element; receiving a first input from the at least one force sensor
on the first edge, the first input being a first measured value of
a force applied on the at least one force sensor on the first edge;
and in response to said first input received, modifying the
foreground graphical element on the display screen in a manner to
move, to a destination position, the first edge portion of the
foreground graphical element and thereby one of reveal and hide a
first corresponding portion of the background graphical element,
depending on a direction of movement of the first edge portion, the
destination position of the first edge portion depending on an
amplitude of the first measured value.
[0008] In accordance with another aspect, there is provided a
computer-implemented method of operating an electronic device
having a housing and a user interface mounted to the housing, the
user interface including at least one force sensor sensitive at
different locations on a first edge of two opposite edges of the
housing and a display screen, the computer-implemented method
comprising: displaying, on the display screen, a graphical element
having a first edge portion; receiving a first input from the at
least one force sensor on the first edge, the first input being a
first measured value of one or more forces applied on the at least
one force sensor on the first edge; and in response to said first
input received, modifying the graphical element on the display
screen in a manner to move, to a destination position, the first
edge portion depending on an amplitude of the first measured value
and on a first input confidence, the first input confidence being
function of a first location offset between a location of the first
input received and a first expected input location.
[0009] In accordance with another aspect, there is provided a
computer-implemented method of operating an electronic device
having a housing and a user interface mounted to the housing, the
user interface including at least one force sensor on a first edge
of two opposite edges of the housing and a display screen, the
computer-implemented method comprising: displaying, on the display
screen, a graphical element having a first edge portion; receiving
a first input from the at least one force sensor on the first edge,
the first input resulting from a force applied on the at least one
force sensor on the first edge; and in response to said first input
received, modifying the graphical element displayed on the display
screen including moving the first edge portion of the graphical
element in the direction of the force applied and away from the
first edge, to a destination position.
[0010] In accordance with another aspect, there is provided an
electronic device comprising: a housing having a first edge and a
second edge opposite the first edge; a user interface mounted to
the housing, the user interface including at least one force sensor
on the first edge of the housing and a display screen; a processor
housed within the housing and in communication with the user
interface, the processor being configured to execute steps of:
displaying, on the display screen, a graphical element having a
first edge portion; receiving a first input from the at least one
force sensor on the first edge, the first input resulting from a
force applied on the at least one force sensor on the first edge;
and in response to said first input received, modifying the
graphical element displayed on the display screen including moving
the first edge portion of the graphical element in the direction of
the force applied and away from the first edge, to a destination
position.
[0011] Many further features and combinations thereof concerning
the present improvements will appear to those skilled in the art
following a reading of the instant disclosure.
DESCRIPTION OF THE FIGURES
[0012] In the figures,
[0013] FIG. 1 is a front elevation view of an example of an
electronic device shown held by a user, in accordance with an
embodiment;
[0014] FIG. 2 is a schematic view of an example of a graphical user
interface that can be displayed by the electronic device of FIG. 1,
in accordance with an embodiment;
[0015] FIG. 3 is a schematic view of a first portion of the
graphical user interface of FIG. 2 when a force of a magnitude
F.sub.1 is applied on a first edge of the electronic device of FIG.
1, in accordance with an embodiment;
[0016] FIG. 4 is a schematic view of the first portion of the
graphical user interface of FIG. 2 when a force of a magnitude
F.sub.2 is applied on the first edge of the electronic device of
FIG. 1, in accordance with an embodiment;
[0017] FIG. 5 is a schematic view of the first portion of the
graphical user interface of FIG. 2 when a force of a magnitude
F.sub.3 is applied on the first edge of the electronic device of
FIG. 1, in accordance with an embodiment;
[0018] FIG. 6 is a schematic view of the first portion of the
graphical user interface of FIG. 2 when a force of a magnitude
F.sub.4 is applied on the first edge of the electronic device of
FIG. 1, in accordance with an embodiment;
[0019] FIG. 7 is a schematic view of the graphical user interface
of FIG. 2 when a force of a magnitude F.sub.5 is applied on the
first edge of the electronic device of FIG. 1, in accordance with
an embodiment;
[0020] FIGS. 7A-B are graphs showing the first measured values
received from corresponding force sensors of the electronic device
of FIG. 1 when forces of magnitude F.sub.5 and F.sub.6 are applied
simultaneously on its first edge, in accordance with an
embodiment;
[0021] FIG. 8 is a schematic view of a second portion of the
graphical user interface of FIG. 2 when opposite forces of
magnitudes F.sub.7 and F.sub.8 are applied on both edges of the
electronic device of FIG. 1, in accordance with an embodiment;
[0022] FIG. 9 is a schematic view of the second portion of the
graphical user interface of FIG. 2 when two opposite forces of
magnitudes F.sub.9 and F.sub.10 are applied on both edges of the
electronic device of FIG. 1, in accordance with an embodiment;
and
[0023] FIG. 10 is a perspective view of the electronic device of
FIG. 1, showing the graphical user interface displayed when the two
opposite forces of magnitudes F.sub.9 and F.sub.10 are applied on
its both edges, in accordance with an embodiment.
DETAILED DESCRIPTION
[0024] FIG. 1 shows an example of an electronic device 100 having a
housing 110, a user interface 120 mounted to the housing 110, and a
processor 130 housed within the housing 110 and in communication
with the user interface 120. The electronic device 100 can be
provided in the form of a smartphone, an electronic tablet or any
other suitable electronic device.
[0025] As depicted, the housing 110 has two opposite edges 112, 114
which will be referred to as the first edge 112 and the second edge
114.
[0026] In this example, the user interface 120 includes one or more
force sensors 122 for receiving user inputs, and a display screen
126 for displaying a graphical user interface. As can be
understood, the term "force sensor" can be used interchangeably
with the term "pressure sensor". Force sensors can thus be force
sensitive, or equivalently, pressure sensitive.
[0027] More specifically, in this specific embodiment, force
sensors 122 are disposed proximate the first edge 112 and on the
second edge 114 of the housing 110. For example, force sensors 122
include force sensors 122a and 122b disposed proximate the first
edge 112 and force sensors 122c and 122d disposed proximate the
second edge 114. An example of an electronic device having such
force-sensitive sides is described in International Patent
Publication No. WO 2016/065482, the contents of which are
incorporated by reference herein. It is understood, however, that
in some other embodiments, only one of the first and second edges
112 and 114 can be provided with force sensor(s).
[0028] The force sensors 122 allow a user to enter inputs while
gripping the electronic device 100 (e.g., with one hand 10), by way
of forces applied at locations of any one of the force sensors 122
of the first and second edges 112 and 114. As depicted, the inputs
can be entered by the same hand 10 that grips the electronic device
100.
[0029] For instance, as shown in FIG. 1, the index finger 12 of
hand 10 applies a force on the force sensor 122a which is located
at an upper portion of the first edge 112 while the thumb 14 of
hand 10 applies a force on the force sensor 122c which is located
at an upper portion of the second edge 114.
[0030] In the embodiment shown, each of the first and second edges
112 and 114 of the electronic device 100 has a plurality of force
sensors 122. However, in some other embodiments, the electronic
device has a first force sensor sensitive along the first edge and
a second force sensor sensitive along the second edge. In these
embodiments, each of the first and second force sensors provides an
indication of measured force at particular locations along a
corresponding one of the first and second edges.
[0031] As will be described below, when a force is applied on any
one of the force sensors 122, the processor 130 is configured to
modify the graphical user interface being displayed on the display
screen 126 by moving a graphical element of the graphical user
interface. Examples of such moving can encompass deforming a
graphical element, changing the shape of a graphical element and/or
removing the graphical element.
[0032] Accordingly, the graphical user interface can respond to
forces applied by a user's hand by creating an animated motion
effect that resembles a piece of material moving in response to the
applied forces. In some embodiments, the animated motion effect can
resemble an elastic material deforming to transition between a
stressed state and a relaxed state in response to the applied
forces.
[0033] Examples described with reference to FIGS. 2-9 show a
graphical user interface 200 of the electronic device 100
responding to different forces applied on one or more of the force
sensors 122 of the electronic device 100 of FIG. 1. Reference to
components of the electronic device 100 of FIG. 1 will thus be made
throughout the following description.
[0034] As depicted in FIG. 2, the graphical user interface 200
includes a foreground graphical element 240 having a first edge
portion 242 and a background graphical element 250 which protrudes
from the first edge portion 242 of the foreground graphical element
240. In this way, one can see both the foreground graphical element
240 and a portion of the background graphical element 250. More
specifically, one can see the background graphical element extend
from underneath and beyond the front graphical element towards the
edge of the electronic device. In some embodiments, the first edge
portion 242 can be said to be a moveable graphical element or a
deformable graphical element.
[0035] During operation of the electronic device 100, graphical
buttons, wallpaper, applications, etc. can be displayed as part of
the foreground graphical element 240. In this example, the
foreground graphical element 240 spans the entire length of the
graphical user interface 200. However, in some other embodiment,
the foreground graphical element 240 spans a portion of the
graphical user interface 200.
[0036] The background graphical element 250 can include one or more
graphical buttons indicating regions where a user can apply a force
to provide user input. For instance, the background graphical
element 250 includes 3 graphical buttons, i.e. first and second
spaced-apart graphical buttons 252a and 252b on a left-hand side of
the graphical user interface 200, and a third graphical button 254
on a right-hand side of the graphical user interface 200. As can be
seen, the graphical buttons 252a, 252b and 254 are partially hidden
by the foreground graphical element 240.
[0037] In this example, the first graphical button 252a is
displayed at a location adjacent the force sensor 122a on the first
edge 112 of the electronic device 100, the second graphical button
252b is displayed at a location adjacent the force sensor 122b on
the first edge 112, and the third graphical button 254 is displayed
at a location adjacent the force sensor 122d on the second edge
114.
[0038] As can be understood, the graphical user interface 200 shown
in FIG. 2 is an example only. Other graphical user interfaces can
be used in other embodiments.
[0039] For example, the force sensor 122a is configured to
transmit, to the processor 130, a first input in response to a
force applied by the user at the location of the force sensor 122a.
In this case, the first input is a first measured value of the
force applied on the force sensor 122a.
[0040] In response to the reception of the first input, the
processor 130 is configured to modify the graphical user interface
200 being displayed on the display screen 126.
[0041] More specifically, when the processor 130 receives the first
input from the force sensor 122a, the processor 130 is configured
to modify the foreground graphical element 240 on the display
screen 126 in a manner to move, to a destination position, the
first edge portion 242 of the foreground graphical element 240 and
thereby reveal or hide a corresponding portion of the background
graphical element 250, depending on a direction of movement of the
first edge portion 242.
[0042] The destination position of the first edge portion 242
depends on an amplitude of the first measured value. Accordingly,
when the amplitude of the first measured value increases, the first
edge portion 242 moves away from the first edge 112, namely, in the
direction of the force applied, and reveals a previously hidden
portion of the corresponding graphical button of the background
graphical element 250. In this way, when such a force is applied,
the previously hidden portion is revealed to show a graphical icon,
which graphically represents the function associated with
activating the corresponding graphical button. In contrast, when
the amplitude of the first measured value decreases, the first edge
portion 242 moves towards the first edge 112 and hides the
corresponding portion of the background graphical element 250. In
this way, as a force applied on the first edge 112 increases, the
first edge portion 242 moves away from the first edge 112 such that
an increased portion of the first graphical button 252a becomes
visible, and vice-versa.
[0043] FIG. 2 shows the graphical user interface 200 when no force
is applied on the force sensor 122a. In this case, the processor
130 receives a first measured value having a null amplitude
f.sub.0, i.e. equal to zero, and displays the first edge portion
242 of the foreground graphical element 240 adjacent to the first
edge 112 of the housing 110 and parallel to the first edge 112. In
this case, the foreground graphical element 240 is not modified,
and the first edge portion 242 remains parallel to the first edge
112. Accordingly, the first edge portion 242 is not distinguishable
from the foreground graphical element 240 when no force is applied
in this example.
[0044] FIG. 3 shows an enlarged view of a first portion 200a of the
graphical user interface 200 of FIG. 2 when a first force of
magnitude F.sub.1 is applied on the force sensor 122a. In this
case, the processor 130 receives a first measure value having a
non-null, first amplitude f.sub.1. Accordingly, the processor 130
modifies the foreground graphical element 240 on the display screen
126 in a manner to move, to a first destination position d.sub.1,
the first edge portion 242 of the foreground graphical element 240
and thereby revealing a corresponding portion 256 of the background
graphical element 250. In this case, the magnitude of the movement
of the first edge portion 242 is proportional to the first
amplitude f.sub.1. A graphical animation can show the movement of
the first edge portion 242 between the first destination position
d.sub.1 and the second destination position d.sub.2.
[0045] FIG. 4 shows an enlarged view of the first portion 200a of
the graphical user interface 200 of FIG. 2 when a second force of
magnitude F.sub.2 is applied on the force sensor 122a, the
magnitude F.sub.2 being greater than the magnitude F.sub.1
(F.sub.2>F.sub.1). In this case, the processor 130 receives a
first measured value that has a non-null, second amplitude f.sub.2,
which is proportionally greater than the first amplitude f.sub.1.
Accordingly, the processor 130 modifies the foreground graphical
element 240 on the display screen 126 in a manner to move, to a
second destination position d.sub.2 farther away from the first
edge 112 than the first destination d.sub.1, the first edge portion
242 of the foreground graphical element 240 and thereby further
revealing the corresponding portion 256 of the background graphical
element 250.
[0046] In some embodiments, the destination position of the first
edge portion 242 reaches a maximal recessed destination position
d.sub.max when the amplitude of the first measured value exceeds a
given threshold amplitude f.sub.th. For instance, in the example
illustrated in FIG. 3, the second amplitude f.sub.2 of the first
measured value exceeds the given threshold amplitude
(f.sub.2>f.sub.th) so the second destination position d.sub.2
coincides with the maximal recessed destination position d.sub.max.
In this way, even if a force of greater magnitude than the
magnitude F.sub.2 is applied on the force sensor 122a on the first
edge 112, the first edge portion 242 will not be moved farther than
the second destination position d.sub.2.
[0047] In some embodiments, the processor 130 activates a target
function of the electronic device 100 when the amplitude of the
first measured value exceeds a given threshold amplitude
f.sub.th.
[0048] For instance, as depicted in FIG. 4, the portion 256 of the
background graphical element 250 that is revealed by the first edge
portion 242 includes the first graphical button 252a. In this
specific case, the first graphical button 252a is a camera icon
which is generally associated with a camera function of the
electronic device. Accordingly, the processor 130 activates a
camera function of the electronic device 100 in response to a first
measured value having an amplitude exceeding the given threshold
amplitude f.sub.th. Other functions can be activated in this
manner.
[0049] In some embodiments, when the function of the first
graphical button 252a is activated, the first graphical button 252a
slightly moves in a direction of the first edge 112 to confirm to
the user that the function has actually been activated. In other
embodiments, the first graphical button 252a can jiggle when it is
activated. Alternatively, the movement of the first graphical
button 252a can emulate the movement of a button which is pressed
against an elastic element such as a spring.
[0050] As exemplified in FIGS. 3 and 4, the first edge portion 242
moves away from the first edge 112 when the amplitude of the first
measured value increases. More specifically, in this example, the
first edge portion 242 does so while maintaining a shape which
resembles the shape of some real-world elastic material receiving a
force. In this case, the shape is provided in the form of a
Gaussian-shaped curve, but any other suitable curve (e.g., a
parabolic curve) can also be used. As depicted, the Gaussian-shaped
curve points away from the first edge 112 and has a magnitude that
is proportional to the amplitude of the first measured value.
Accordingly, the magnitude of the Gaussian-shape curve of the first
edge portion 242 increases when the amplitude of the first measured
value increases, and the magnitude of the Gaussian-shape curve of
the first edge portion 242 decreases when the amplitude of the
first measured value decreases. It can also be noticed that the
Gaussian-shape curve has a peak position which follows the
destination position of the first edge portion 242.
[0051] In the case where the movement of the first edge portion 242
is a deformation, each deformation has an associated degree of
deformation after the animated motion effect finishes. In this
example, the degree of deformation can be defined by a destination
position d.sub.i and a width w.sub.i as shown in FIGS. 3 and 4. In
some embodiments, the deformation recedes once the force being
applied by the user is released, having a graphical effect of the
depicted elastic material returning to a relaxed state and the
first edge portion 242 returning to its original destination
position as shown in FIG. 2. In this way, the first edge portion
242 can be moved from a relaxed state in which the first edge
portion 242 is at its original destination position to a stressed
state in which the first edge portion 242 is at the first
destination position d.sub.1. Meanwhile, the first edge portion 242
can be said to be in a transition state when transitioning between
the relaxed state and the stressed state. The transition state can
include a movement of the first edge portion 242 in response to a
force which can be similar to how a flexible material would change
its shape upon receiving such a force. For instance, the movement
can deform a straight edge into a curved edge, or vice versa, and
therefore change the shape of the initial, undeformed first edge
portion 242. Once deformed, the first edge portion 242 can be said
to be in a stressed state (e.g., compressed or stretched), and once
the force is released, the first edge portion 242 can revert to its
initial shape or relaxed state.
[0052] The location of the deformation is determined according to a
force applied by a user's hand 10 (e.g., index finger 12 and/or
thumb 14). Multiple deformations can be shown if forces are applied
at multiple locations along either one or both of the first and
second edges 112 and 114 of the electronic device 100.
[0053] As discussed, the degree of deformation may be determined
according to a magnitude F.sub.i of the force applied at a given
location. As depicted, the degree of deformation may also be
determined according to a confidence that the force corresponds to
an intended user input (e.g., rather than accidental user input or
forces associated with a user holding the electronic device 100
during operation), which can be referred to as an "input confidence
c.sub.i".
[0054] FIGS. 3 and 4 depict graphs of input confidence functions
260 for determining the input confidence c.sub.1 and c.sub.2,
respectively. Broadly described, the input confidence c.sub.i can
be determined according to the difference(s) between a location
y.sub.i of an input and one or more pre-defined expected input
locations y.sub.exp such as a location of any one of the graphical
buttons 252a, 252b or 254. Such difference(s) are referred to as
location offset .DELTA.y.sub.i, wherein
.DELTA.y.sub.i=y.sub.i-y.sub.ref. The input confidence c.sub.i is a
function which typically varies between 0 and 1. The y-axis of
these graphs represents location offset .DELTA.y.sub.i and the
x-axis intersects the center of a corresponding graphical button,
representing the expected input location y.sub.exp. The pre-defined
expected input locations y.sub.exp can be defined programatically,
and can change depending on which software of the electronic device
is being executed. For instance, different applications can
pre-define different graphical buttons and/or set the graphical
icon depicted on the graphical button.
[0055] As shown, the input confidence c.sub.i is highest (e.g.,
equal to 1) when the location offset .DELTA.y.sub.i is null, i.e.
when the location y.sub.i of the input is equal to the expected
input location Y.sub.exp (e.g., the center of a graphical button).
Moreover, the input confidence c.sub.i decreases as the location
offset .DELTA.y.sub.i deviates from zero, i.e. when the location of
the input deviates from the expected input location y.sub.exp. In
this example, the relationship between the input confidence c.sub.i
and the location offset .DELTA.y.sub.i is represented by a Gaussian
function. However, in practice, the input confidence c.sub.i can be
set to 0 when the location offset .DELTA.y.sub.i is greater than a
location offset threshold .DELTA.y.sub.thres. Other functions,
especially functions that have a maximum at its center and approach
zero at tails, can also be used.
[0056] For instance, the destination position d.sub.i can be
calculated to be proportional to the magnitude of the force, while
the width w.sub.i can be calculated to be proportional to the input
confidence c.sub.i as follows:
d.sub.i=A-f when f.sub.1<f.sub.th or d.sub.i=d.sub.max when
f.sub.i>f.sub.th, and (1)
w.sub.i=Bc.sub.i+C when c.sub.i<1 or w.sub.i=w.sub.max when
c.sub.i=1 , (2)
[0057] where f.sub.i represents the amplitude of the measured value
as measured by a corresponding one of the force sensors 122,
c.sub.i represents the input confidence as described above, and A,
B and C each represent a constant parameter, which can be
pre-defined and stored on a memory of the electronic device 100.
Each of d.sub.i and w.sub.i can have dimensional units such as
pixels or mm.
[0058] Parameter A represents a conversion rate from force (e.g.,
Newtons) to mm or pixels (e.g., 2 mm/1 N), parameter B represents a
scaling factor for conversion of input confidence to deformation
width (e.g., 5 mm), and parameter C represents a minimum
deformation width (e.g., 3 mm).
[0059] Accordingly, in the example shown in FIG. 3, the first
destination position d.sub.1 is given by d.sub.1=Af.sub.1 and the
first edge portion 242 has a first width w.sub.1 given by
w.sub.1=Bc.sub.1+C=w.sub.max, wherein the input confidence c.sub.1
is maximal (e.g., c.sub.1=1) as the location offset .DELTA.y is
null.
[0060] In the example shown in FIG. 4, the second destination
position d.sub.2 is given by d.sub.2=Af.sub.2 and the first edge
portion 242 has a second width w.sub.2 given by
w.sub.2=Bc.sub.2+C=w.sub.max, wherein the input confidence c.sub.2
is maximal (e.g., c.sub.2=1) as the location offset .DELTA.y is
null also in this example. As can be expected, the second
destination position d.sub.2 is greater than the first destination
position d.sub.1, and w.sub.1 is equal to w.sub.2. In this
embodiment, the second destination position d.sub.2 is the maximal
recessed destination position d.sub.max because the amplitude
f.sub.2 of the first measured value exceeds the given threshold
value f.sub.th.
[0061] FIG. 5 shows an enlarged view of a first portion 200a of the
graphical user interface 200 of FIG. 2 when a third force of
magnitude F.sub.3 is applied on one of the force sensors 122a and
122b on the first edge 112, but spaced from a location of the force
sensor 122a. In this example, the first edge portion 242 of the
foreground graphical element 240 is moved at a location where the
input is received based on the plurality of first measured values
received from the force sensors 122a and 122b on the first edge
112.
[0062] In this case, the processor 130 receives a first measured
value having a non-null, third amplitude f.sub.3, which is similar
to the second amplitude f.sub.2. Accordingly, the processor 130
modifies the foreground graphical element 240 on the display screen
126 in a manner to move, to a third destination position d.sub.3
away from the first edge 112, the first edge portion 242 of the
foreground graphical element 240 and thereby revealing a
corresponding portion 256 of the background graphical element 250
located where the first input has been received. As described
above, in this example, the third destination position d.sub.3 is
given by d.sub.3=Af.sub.3.
[0063] In this example, the first edge portion 242 has a third
width w.sub.3 given by w=Bc.sub.3+C=C, wherein the input confidence
c.sub.3 is minimal (e.g., c.sub.3=0) as the location offset
.DELTA.y is greater than a given location offset threshold
.DELTA.y.sub.thres, .DELTA.y>.DELTA.y.sub.thres.
[0064] FIG. 6 shows an enlarged view of a first portion 200a of the
graphical user interface 200 of FIG. 2 when a fourth force of
magnitude F.sub.4 is applied on one of the force sensors 122a and
122b on the first edge 112, but spaced from a location of the force
sensor 122a. In this example, the first edge portion 242 of the
foreground graphical element 240 is moved at a location where the
input is received based on the plurality of first measured values
received from the force sensors 122a and 122b on the first edge
112.
[0065] In this case, the processor 130 receives a first measured
value having a non-null, third amplitude f.sub.4. Accordingly, the
processor 130 modifies the foreground graphical element 240 on the
display screen 126 in a manner to move, to a fourth destination
position d.sub.4 away from the first edge 112, the first edge
portion 242 of the foreground graphical element 240 and thereby
revealing a previously hidden portion 256 of the background
graphical element 250 located where the first input has been
received. Again, the fourth destination position d.sub.4 is given
by d.sub.4=Af.sub.4 in this example. As shown in FIG. 6, revealing
only a top portion of the graphical button 252a provides a visual
cue to the user that the location of the force should be lowered if
the user intends to activate graphical button 252a.
[0066] In this example, the first edge portion 242 has a fourth
width w.sub.4 given by w.sub.4=Bc.sub.4+C, wherein the input
confidence c.sub.4 is between 0 and 1 (e.g., c.sub.4.apprxeq.0.5)
as the location offset .DELTA.y is between 0 and the location
offset threshold .DELTA.y.sub.thres.
[0067] FIGS. 3-6 show a simplified graphical user interface having
only the first graphical button 252a. However, in practice, a
graphical user interface is likely to have several graphical
buttons, such as the second and third graphical buttons 252b and
254 shown in FIG. 2.
[0068] FIG. 7 shows the graphical user interface 200 of FIG. 2 when
a force of magnitude F.sub.5 is applied on the force sensors 122a
and 122b on the first edge 112 of the electronic device 100. In
this example, each one of the first and second graphical buttons
252a and 252b has its corresponding input confidence functions
260.
[0069] In this case, the processor 130 determines input confidences
c.sub.5 and c.sub.6 using each of the input confidence functions
260 based on a location of the force of magnitude F.sub.5. In this
example, the processor 130 moves the first edge portion 242 where
the force is received. In this embodiment, the destination position
d.sub.5 is calculated based on the amplitude f.sub.5 of the first
measured value received in a manner similar than described above.
However, the width w.sub.5 is determined based on a maximal
c.sub.max of the input confidences c.sub.5 and c.sub.6, i.e.
c.sub.max=max(c.sub.5, c.sub.6)=c.sub.5. For instance, in the
illustrated example, the input confidence c.sub.5 is 0 while the
input confidence c.sub.6 is greater than 0. Accordingly, the width
w.sub.5 is calculated based on the input confidence c.sub.5, and
not based on the input confidence c.sub.6. Accordingly, the width
w.sub.5 is given by w.sub.5==Bc.sub.5+C. Although a maximizing
function is used in this embodiment, alternatively the width
w.sub.5 can be determined by summing the input confidence c.sub.5
and c.sub.6, averaging the input confidences c.sub.5 and c.sub.6,
etc.
[0070] In the embodiments of FIGS. 3-6, the processor 130 moves the
first edge portion 242 at a given location which coincides with the
location of the first input. However, in practice, the user may
grip the electronic device 100 and apply forces on force sensors
122 at different locations of the electronic device 100 in doing
so.
[0071] For instance, FIG. 7A shows a graph plotting the amplitude
of the first measured values received from the force sensors 122a
and 122b on the first edge 112 of the electronic device as function
of their corresponding locations. As can be seen, first curve 300
can be associated with forces applied by a user's index finger (see
force of magnitude F.sub.6 shown in dashed line in FIG. 7) while
second curve 302 can be associated with forces applied by a user's
middle finger (see force of magnitude F.sub.5 in FIG. 7).
[0072] After having received the first measured values shown in
FIG. 7A, the processor 130 can combine these data to obtain a net
deformation curve 304 such as the one shown in FIG. 7B. In some
embodiments, the processor 130 modifies the graphical user
interface based on a maximal one of the amplitudes f of the first
measured values, the maximal amplitude f.sub.max, and moves the
first edge portion 242 at a location of the maximal amplitude
f.sub.max, as shown in FIG. 7 for example. In some cases, if the
maximal amplitude f.sub.max of the first measured values fails to
exceed a corresponding amplitude threshold f.sub.thres, the
processor 130 ignores the first measured values and do not modify
the graphical user interface accordingly.
[0073] FIG. 8 shows an enlarged view of a second portion 200b of
the graphical user interface 200 of FIG. 2 when two opposite forces
of magnitudes F.sub.7 and F.sub.8 are applied simultaneously on the
first and second edges 112 and 114.
[0074] Indeed, in some embodiments, some functions can be activated
by applying multiple forces simultaneously at different locations
of the electronic device 100 (e.g., by multiple fingers or thumb
such as shown in FIG. 1). For example, a function associated with
the second and third graphical buttons 252b and 254 of the
graphical user interface 200 can be activated by applying forces on
their corresponding locations. In this example, in response to
applying forces on the second and third graphical buttons 252b and
254, the processor 130 activates a power on/off request.
Accordingly, in this specific case, the second graphical button
252b is a power icon which is generally associated with a power
function of the electronic device. Other function can be activated
in this way in other embodiments.
[0075] A deformation can be displayed for each force applied on the
force sensors 122 of the electronic device 100. The destination
position d.sub.7 of the first edge portion 242 and the destination
position d.sub.8 of the second edge portion 244 can be determined
in a manner similar to that described above. However, in this
example, the width w.sub.7 of the first edge portion 242 and the
width w.sub.8 of the second edge portion 244 are determined using
an aggregate input confidence c' determined based on all forces
applied, as follows.
[0076] As shown, for each graphical button 252b, 254, a
corresponding input confidence function 260 is defined. Given the
locations of the two forces applied on the first and second edges
112 and 114 of the electronic device 100, an input confidence may
be determined using each input confidence function 260, to arrive
at input confidences c.sub.7 and c.sub.8.
[0077] In the illustrated example, the input confidence c.sub.7 is
maximal, e.g., equal to 1, because the location of the force of
magnitude F.sub.7 corresponds to the expected input location of the
second graphical button 252b. In contrast, the input confidence
c.sub.8 is minimal, e.g., equal to 0, because the location of the
force of magnitude F.sub.8 is beyond the location offset threshold
y.sub.thres. The aggregate input confidence c' is the product of
c.sub.7 and c.sub.8, which yields 0, i.e. c.sub.7.times.c.sub.8=0.
According to equation (2) above, the width w.sub.7 of the first
edge portion 242 and the width w.sub.8 of the second edge portion
244 thus correspond to the minimum deformation width C.
[0078] FIG. 9 shows an enlarged view of the second portion 200b of
the graphical user interface 200 of FIG. 2 when two opposite forces
of magnitudes F.sub.9 and F.sub.10 are applied simultaneously on
the first and second edges 112 and 114.
[0079] Still in this example, a deformation is displayed for each
force applied on the force sensors 122 of the electronic device
100. The destination position d.sub.9 of the first edge portion 242
and the destination position d.sub.10 of the second edge portion
244 can be determined in a manner similar to that described above.
The width w.sub.9 of the first edge portion 242 and the width
w.sub.10 of the second edge portion 244 are determined based on the
aggregate input confidence c' as described above.
[0080] Given the locations of the two forces applied on the first
and second edges 112 and 114 of the electronic device 100, an input
confidence is determined using each input confidence function 260,
to arrive at input confidences c.sub.9 and c.sub.10.
[0081] In the illustrated example, both the input confidences
c.sub.9 and c.sub.10 are maximal, e.g., equal to 1, because the
location of the forces of magnitudes F.sub.9 and F.sub.10
corresponds to the expected input locations y.sub.exp associated
with the first graphical button 252b and the third graphical button
254. The aggregate input confidence c' is the product of c.sub.9
and c.sub.10 which yields 1, i.e. c.sub.9.times.c.sub.10=1.
According to equation (2) above, the width w.sub.9 of the first
edge portion 242 and the width w.sub.10 of the second edge portion
244 are both maximal, which reveals both the first and third
graphical buttons 252a and 254.
[0082] FIG. 10 shows a perspective view of the electronic device
100 with a graphical user interface including a deformable element
in a deformed state, as the one shown in FIG. 9. As depicted, the
first and second edges 112 and 114 are curved and the display
screen 126 is curved so as to the curved edges 112 and 114. In this
embodiment, it is intended that the background graphical element
150 can be displayed at least partially on the curved portions of
the first and second edges 112 and 114. However, it is understood
that, in some other embodiments, the electronic device can have
squared edges and a flat screen extending towards the squared
edges.
[0083] As can be understood, the examples described above and
illustrated are intended to be exemplary only. It is noted that the
modification of the edge portion of the foreground graphical
element can include animating the edge portion during its
modification from a first state to a second state. Is it intended
that the animation of the edge portion can be performed while the
foreground graphical element displays an image or a video. The
scope is indicated by the appended claims.
* * * * *