U.S. patent application number 11/823192 was filed with the patent office on 2009-01-01 for method and apparatus for multi-touch tactile touch panel actuator mechanisms.
This patent application is currently assigned to IMMERSION Corporation, a Delaware Corporation. Invention is credited to Daniel H. Gomez, Ryan Steger, Christopher J. Ullrich.
Application Number | 20090002328 11/823192 |
Document ID | / |
Family ID | 40159806 |
Filed Date | 2009-01-01 |
United States Patent
Application |
20090002328 |
Kind Code |
A1 |
Ullrich; Christopher J. ; et
al. |
January 1, 2009 |
Method and apparatus for multi-touch tactile touch panel actuator
mechanisms
Abstract
A method and apparatus of actuator mechanisms for a multi-touch
tactile touch panel are disclosed. The tactile touch panel includes
an electrical insulated layer and a tactile layer. The top surface
of the electrical insulated layer is capable of receiving an input
from a user. The tactile layer includes a grid or an array of
haptic cells. The top surface of the haptic layer is situated
adjacent to the bottom surface of the electrical insulated layer,
while the bottom surface of the haptic layer is situated adjacent
to a display. Each haptic cell further includes at least one
piezoelectric material, Micro-Electro-Mechanical Systems ("MEMS")
element, thermal fluid pocket, MEMS pump, resonant device, variable
porosity membrane, laminar flow modulation, or the like. Each
haptic cell is configured to provide a haptic effect independent of
other haptic cells in the tactile layer.
Inventors: |
Ullrich; Christopher J.;
(Santa Cruz, CA) ; Steger; Ryan; (Sunnyvale,
CA) ; Gomez; Daniel H.; (Newton, MA) |
Correspondence
Address: |
James M. Wu;JW Law Group
84 W. Santa Clara Street, Suite 820
San Jose
CA
95113
US
|
Assignee: |
IMMERSION Corporation, a Delaware
Corporation
San Jose
CA
|
Family ID: |
40159806 |
Appl. No.: |
11/823192 |
Filed: |
June 26, 2007 |
Current U.S.
Class: |
345/173 |
Current CPC
Class: |
G09B 21/004 20130101;
G06F 2203/04104 20130101; G06F 3/0414 20130101; G06F 3/016
20130101 |
Class at
Publication: |
345/173 |
International
Class: |
G06F 3/041 20060101
G06F003/041 |
Claims
1. A tactile touch panel comprising a plurality of piezoelectric
cells forming a piezoelectric layer having a first and a second
surfaces, wherein said first surface of said piezoelectric layer is
configured to receive inputs, wherein each one of said plurality of
piezoelectric cells includes at least one piezoelectric material,
wherein said piezoelectric material is configured to provide a
haptic effect independent of other piezoelectric cells.
2. The tactile touch panel of claim 1, wherein said plurality of
piezoelectric cells is capable of sensing said inputs.
3. The tactile touch panel of claim 1, further comprising an
electrical insulated layer having a third and a fourth surfaces,
wherein said fourth surface of said electrical insulated layer is
situated adjacent to said first surface of said plurality of
piezoelectric cells and said third surface of said electrical
insulated layer is configured to interface with said inputs.
4. The tactile touch panel of claim 3, wherein said third surface
of said electrical insulated layer is capable of transmitting said
inputs to said first surface of said plurality of piezoelectric
cells.
5. The tactile touch panel of claim 1, further comprising a display
coupled to said second surface of said piezoelectric layer, wherein
said display is capable of projecting images viewable from said
third surface of said electrical insulated layer.
6. The tactile touch panel of claim 5, wherein said display is a
flat panel display or a flexible display.
7. The tactile touch panel of claim 1, further comprising a
plurality of electrical wires connected to said plurality of
piezoelectric cells.
8. The tactile touch panel of claim 7, wherein said piezoelectric
material deforms in response to electrical potentials applied from
at least one of said electrical wires.
9. The tactile touch panel of claim 1, wherein each one of said
plurality of piezoelectric cells is configured to be less than 5
millimeters by 5 millimeters in size.
10. The tactile touch panel of claim 9, wherein multiple adjacent
piezoelectric cells of said plurality of piezoelectric cells are
capable of providing multiple haptic effects substantially
simultaneous.
11. A haptic touch panel comprising a plurality of
Micro-Electro-Mechanical Systems ("MEMS") cells formed a MEMS layer
having a first and a second surfaces, wherein said first surface is
capable of receiving inputs, wherein each one of said plurality of
MEMS cells includes at least one MEMS element, wherein said MEMS
element is configured to provide a haptic effect independent of
other MEMS cells.
12. The haptic touch panel of claim 11, further comprising an
insulated layer having a third and a fourth surfaces, wherein said
fourth surface of said insulated layer is situated adjacent to said
first surface of said MEMS layer.
13. The haptic touch panel of claim 11, further comprising a
display coupled to said second surface of said MEMS layer, wherein
said display is capable of projecting images viewable from said
third surface of said insulated layer.
14. The haptic touch panel of claim 13, wherein said display is a
flat panel display or a flexible display.
15. The haptic touch panel of claim 11, further comprising a
plurality of electrical wires wherein each one of said plurality of
MEMS cells is connected to at least one of said plurality of wires
for facilitating said haptic effect.
16. The haptic touch panel of claim 15, wherein said MEMS element
is a cantilever-spring.
17. The haptic touch panel of claim 15, wherein said MEMS element
is a shape memory alloy ("SMA").
18. The haptic touch panel of claim 15, wherein said MEMS element
is made of piezo materials.
19. The haptic touch panel of claim 15, wherein said MEMS element
is a resonant mechanical retractable device, wherein said resonant
mechanical retractable device vibrates in response to a unique
frequency.
20. The haptic touch panel of claim 19, wherein multiple adjacent
MEMS cells of said plurality of MEMS cells are capable of providing
multiple haptic effects in response to different frequencies.
21. The haptic touch panel of claim 15, wherein said MEMS element
deforms in response to electrical potentials provided by said
electrical wires.
22. A method for providing multiple simultaneous haptic effects
comprising: detecting a first deformation of a sensing layer;
detecting a second deformation of said sensing layer substantially
same time as said first deformation; generating a first input in
accordance with a location of said first deformation and a second
input in accordance with a location of said second deformation;
activating a first haptic cell with a first haptic effect in
response to said first input and a second haptic cell with a second
haptic effect in response to said second input.
23. The method of claim 22, further comprising: displaying an image
viewable through or on an insulated layer; and monitoring said
insulated layer in accordance with said image.
24. The method of claim 23, wherein said detecting a first
deformation of a sensing layer further includes sensing said first
deformation of said insulated layer.
25. The method of claim 22, wherein said detecting a first
deformation of a sensing layer further includes sensing said first
deformation of said first haptic cell.
26. The method of claim 22, wherein detecting a first deformation
of said sensing layer further includes detecting said first
deformation in response to a first depressing by a first
finger.
27. The method of claim 22, further comprising detecting said first
and said second deformations depressed by a same finger.
28. The method of claim 22, wherein said detecting a second
deformation of said sensing layer further includes sensing said
second deformation in response to a second depressing by a second
finger.
29. The method of claim 22, wherein detecting a first deformation
of said sensing layer further includes detecting said first
deformation in response to a first depressing by a stylus.
30. The method of claim 22, wherein said activating a first haptic
cell with a first haptic effect in response to said first input and
a second haptic cell with a second haptic effect in response to
said second input further includes initiating said first haptic
effect and said second haptic effect substantially same time.
31. The method of claim 22, wherein said activating a first haptic
cell with a first haptic effect in response to said first input and
a second haptic cell with a second haptic effect in response to
said second input further includes: activating a first
piezoelectric material of said first haptic cell for generating
said first haptic effect; and activating a second piezoelectric
material of said second haptic cell for generating said second
haptic effect.
32. The method of claim 22, wherein said activating a first haptic
cell with a first haptic effect in response to said first input and
a second haptic cell with a second haptic effect in response to
said second input further includes: activating a first
Micro-Electro Mechanical System ("MEMS") element of said first
haptic cell to generate said first haptic effect; and activating a
second MEMS element of said second haptic cell for generating said
second haptic effect.
33. The method of claim 22, wherein said activating a first haptic
cell with a first haptic effect in response to said first input and
a second haptic cell with a second haptic effect in response to
said second input further includes: activating a first
Micro-Electro Mechanical System ("MEMS") cell to create a first
turbulence of laminar flow for facilitating said first haptic
effect; and activating a second MEMS cell to create a second
turbulence of laminar flow for generating said second haptic
effect.
34. The method of claim 22, wherein said activating a first haptic
cell with a first haptic effect in response to said first input and
a second haptic cell with a second haptic effect in response to
said second input further includes: activating a first fluid filled
pocket of said first haptic cell for generating said first haptic
effect; and activating a second fluid filled pocket of said second
haptic cell for generating said second haptic effect.
Description
RELATED APPLICATIONS
[0001] This application is related to the following co-pending
application, which is assigned to the Assignee of the present
invention.
[0002] Application Ser. No. ______, filed ______, entitled "Method
and Apparatus for Multi-Touch Tactile Touch panel Actuator
Mechanisms," attorney docket no. IMM272 (1054.P003US).
FIELD OF THE INVENTION
[0003] The present invention relates to the field of electronic
interface devices. More specifically, the present invention relates
to a user interface device having haptic actuators.
BACKGROUND OF THE INVENTION
[0004] As computer-based systems, appliances, automated teller
machines (ATM), point of sale terminals and the like have become
more prevalent in recent years, the ease of use of the
human-machine interface is becoming more and more important. Such
interfaces should operate intuitively and require little or no
training so that they may be used by virtually anyone. Many
conventional user interface devices are available on the market,
such as the key board, the mouse, the joystick, and the touch
screen. One of the most intuitive and interactive interface devices
known is the touch panel, which can be a touch screen or a touch
pad. A touch screen includes a touch sensitive input panel and a
display device, usually in a sandwich structure and provides a user
with a machine interface through touching a panel sensitive to the
user's touch and displaying content that the user "touches." A
conventional touch pad is a small planar rectangular pad, which can
be installed near a display, on a computer, an automobile, ATM
machines, and the like.
[0005] A conventional touch-sensitive component of a touch panel
employs various types of touch sensing technology such as
capacitive sensors, pressure sensors and the like as known in the
art to detect locations being pressed on the panel. For example, a
user contacts a region of a touch screen commonly with a fingertip
to emulate a button press and/or moves his or her finger on the
panel according to the graphics displayed behind the panel on the
display device.
[0006] A problem associated with the conventional approach for
generating a haptic feedback is relying on global motion of a
mechanical carrier attached to the touch screen to produce haptic
or tactile feedback. Using the global motion approach typically
limits to one haptic feedback to one input at a given time.
[0007] Accordingly, there is a need for a touch panel or surface,
which is capable of providing multiple tactile or haptic feedbacks
in response to multiple touches simultaneously at a given time.
SUMMARY OF THE INVENTION
[0008] A method and apparatus of actuator mechanisms for a
multi-touch tactile touch panel are disclosed. The tactile touch
panel includes an electrical insulated layer and a tactile layer,
wherein the electrical insulated layer includes a top surface and a
bottom surface. The top surface of the electrical insulated layer
is capable of receiving an input from a user. The tactile layer,
which is also known as a haptic layer, a feedback layer, or the
like, includes a grid or an array of haptic cells. The top surface
of the haptic layer is situated adjacent to the bottom surface of
the electrical insulated layer, while the bottom surface of the
haptic layer is situated adjacent to a display. Each haptic cell
further includes at least one piezoelectric material,
Micro-Electro-Mechanical Systems ("MEMS") element, thermal fluid
pocket, MEMS pump, resonant device, variable porosity membrane,
laminar flow modulation, or the like. Each haptic cell is
configured to provide a haptic effect independent of other haptic
cells in the tactile layer.
[0009] Additional features and benefits of the present invention
will become apparent from the detailed description, figures and
claims set forth below.
BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The present invention will be understood more fully from the
detailed description given below and from the accompanying drawings
of various embodiments of the invention, which, however, should not
be taken to limit the invention to the specific embodiments, but
are for explanation and understanding only.
[0011] FIG. 1 illustrates an electronic interface device or system
capable of providing multiple tactile feedbacks in response to
multiple touches in accordance with one embodiment of the present
invention;
[0012] FIG. 2 is a top view of an interface device illustrating a
haptic touch panel having an array or a grid of haptic cells in
accordance with one embodiment of the present invention;
[0013] FIG. 3(a-b) illustrates a haptic cell using piezoelectric
materials to generate haptic effects in accordance with one
embodiment of the present invention;
[0014] FIG. 4(a-b) is a diagram illustrating another embodiment of
a haptic cell using Micro-Electro-Mechanical Systems ("MEMS")
device to generate haptic effects in accordance with one embodiment
of the present invention;
[0015] FIG. 5(a-b) illustrates a side view of an interface device
having an array of haptic cells with thermal fluid pockets in
accordance with one embodiment of the present invention;
[0016] FIG. 6(a-b) illustrates a haptic cell employing
Micro-Electro-Mechanical Systems pumps to generate haptic effects
in accordance with one embodiment of the present invention;
[0017] FIG. 7 illustrates a side view diagram for an interface
device having an array of haptic cells using variable porosity
membrane in accordance with one embodiment of the present
invention;
[0018] FIG. 8 is a side view of an interface device having an array
of haptic cells using various resonant devices in accordance with
one embodiment of the present invention;
[0019] FIG. 9(a-b) illustrates a top view diagram of a multi-touch
haptic display 900 having laminar flow of fluid in accordance with
one embodiment of the present invention; and
[0020] FIG. 10 is a flowchart illustrating a process of providing
multiple haptic effects in accordance with one embodiment of the
present invention.
DETAILED DESCRIPTION
[0021] Embodiments of the present invention are described herein in
the context of a method, system and apparatus of actuator
mechanisms for a multi-touch tactile touch panel. Those of ordinary
skilled in the art will realize that the following detailed
description of the present invention is illustrative only and is
not intended to be in any way limiting. Other embodiments of the
present invention will readily suggest themselves to such skilled
persons having the benefit of this disclosure.
[0022] Reference will now be made in detail to implementations of
the present invention as illustrated in the accompanying drawings.
The same reference indicators will be used throughout the drawings
and the following detailed description to refer to the same or like
parts.
[0023] In the interest of clarity, not all of the standard hardware
and routine features of the implementations described herein are
shown and described. It will, of course, be appreciated that in the
development of any such actual implementation, numerous
implementation-specific decisions must be made in order to achieve
the developer's specific goals, such as compliance with
application- and business-related constraints, and that these
specific goals will vary from one implementation to another and
from one developer to another. Moreover, it will be appreciated
that such a development effort might be complex and time-consuming,
but would nevertheless be a routine undertaking of engineering for
those of ordinary skilled in the art having the benefit of this
disclosure.
[0024] The present invention discloses an electronic interface
device using multi-touch actuator mechanisms for a touch panel. In
one embodiment, the interface device having a tactile touch panel
is capable of providing multiple haptic feedbacks in response to
multiple contacts simultaneously. The haptic feedback may also be
referred to as tactile effect, tactile feedback, haptic effect,
force feedback, or vibrotactile feedback. The tactile touch panel
can also be referred to as a haptic touch pad, vibrotactile touch
panel, force feedback touch panel, haptic touch panel, or the
like.
[0025] The tactile touch panel, in one embodiment, includes an
electrical insulated layer and a tactile layer, wherein the
electrical insulated layer includes a top surface and a bottom
surface. The top surface of the electrical insulated layer is
capable of receiving an input from a user. The tactile layer, which
is also known as a haptic layer, a feedback layer, or the like,
includes a grid or an array of haptic cells. The top surface of the
haptic layer is situated adjacent to the bottom surface of the
electrical insulated layer, while the bottom surface of the haptic
layer is situated adjacent to a display. Each haptic cell further
includes at least one piezoelectric material,
Micro-Electro-Mechanical Systems ("MEMS") element, thermal fluid
pocket, MEMS pump, resonant device, variable porosity membrane,
laminar flow modulation, or the like. Each haptic cell is
configured to provide a haptic effect independent of other haptic
cells in the tactile layer.
[0026] FIG. 1 illustrates an electronic interface device or system
100 capable of providing multiple tactile feedbacks in response to
multiple touches substantially simultaneous in accordance with one
embodiment of the present invention. System 100 includes a
touch-sensitive panel or touch panel 102, a display panel 104, and
a case 106. Touch-sensitive panel 102, in one embodiment, is made
of substantially transparent materials, and is capable of
transmitting light so that objects or images displayed in display
104 can be seen through the touch-sensitive panel 102. Display 104
can be any type of display such as a cathode ray tube ("CRT"),
liquid crystal display ("LCD"), plasma display, flat panel display,
flexible display or the like. Both touch-sensitive panel 102 and
display 104 may be installed together with case 106. It should be
noted that touch-sensitive panel 102 and display 104 can be
integrated into the same unit or device. In an alternative
embodiment, display 102 may be removed from system 100 when
displaying images are not necessary. For example, a touch pad used
on a laptop or on a vehicle dashboard, which does not require
displaying images, can be opaque.
[0027] Touch panel 102, in one embodiment, includes an insulated
layer and an array or a grid of haptic cells 120, wherein haptic
cells 120 are separated by borders 124. Each of haptic cells 120 is
capable of providing a haptic effect in response to an input
independent of other haptic cells 120 in touch panel 102. For
example, when multiple contacts are depressed on touch panel 102
substantially simultaneously, touch-sensitive panel or touch panel
102 activates haptic cells 120 to generate multiple haptic effects
in response to the multiple contacts. It should be noted that the
multiple contacts may be made by one finger or multiple fingers.
The dimension or size of each of the haptic cells 120 is configured
to be less than 5 millimeters.times.5 millimeters, although other
sizes may be used as appropriate. Touch panel 102 accepts a user's
selection(s) when one or more cells 120 are contacted, touched or
depressed by the user's finger(s). In one embodiment, touch panel
102 rejects a user's selection when a border 124 is touched.
[0028] Touch panel 102 further includes circuits 110 mounted at the
edge or otherwise attached to the panel via a cable or flexible
circuit. Circuits 110 are used to provide digital control signals
and/or a power source to haptic cells 120. In one embodiment, case
106 further includes a digital processing unit for data processing.
In another embodiment, touch panel 102 is capable of providing a
tactile overlay that includes a grid of haptic cells 120 wherein
each of the haptic cells 120 is approximately the size of half
(1/2) a fingertip. Each haptic cell 120 is capable of providing
vibrotactile or kinesthetic feedback through a localized strain. In
one embodiment, the grid cells can be hexagonal or any other type
of two-dimensional (2-D) configurations. Alternatively, it should
be noted that the grid of haptic cells 120 does not necessarily
cover the entire touch panel surface. The layout of haptic cells
120 can be selectively configured to meet the application's
requirements.
[0029] FIG. 2 illustrates a top view of an interface device 200
illustrating a haptic touch panel 206 having an array or a grid of
haptic cells 210 in accordance with one embodiment of the present
invention. Referring back to FIG. 2, device 200 further includes
circuit blocks 202-204, which are configured to perform various
functions such as maintaining power supplies, transmitting control
signals, and/or controlling fluid flow. In one embodiment, device
200 also includes a display, which is placed behind touch panel
206. In one embodiment, touch panel 206 is substantially
transparent thereby the images displayed by the display can be
viewed through touch panel 206. When the application does not
require displaying images, the surface of touch panel 206 is opaque
and blocks most of the light from passing through touch panel
206.
[0030] An array of haptic cells 210 of touch panel 206 is capable
of generating haptic effects in response to their control signals.
Control signals, in one aspect, are generated in accordance with
the inputs received. To provide multiple haptic effects in response
to multiple touches, each haptic cell 210 is capable of initiating
a haptic effect independent of other haptic cells 210 in touch
panel 206. In another embodiment, each of haptic cells 210 of touch
panel 206 is capable of generating a unique haptic effect in
response to a specific input. A unique haptic effect initiates a
specific haptic sensation to a user's input. It should be noted
that each cell 210 can be further divided into multiple sub cells
wherein each sub cell can generate its own haptic effect.
[0031] FIG. 3(a) illustrates a haptic cell 210 using piezoelectric
materials to generate haptic effects in accordance with one
embodiment of the present invention. Cell 210 includes an
electrical insulated layer 302, a piezoelectric material 304, and
wires 306. Electrical insulated layer 302 has a top surface and a
bottom surface, wherein the top surface is configured to receive
inputs. A grid or an array of piezoelectric materials 304, in one
embodiment, is constructed to form a piezoelectric or haptic layer,
which also has a top and a bottom surface. The top surface of the
piezoelectric layer is situated adjacent to the bottom surface of
electrical insulated layer 302. Each cell 210 includes at least one
piezoelectric material 304 wherein piezoelectric material 304 is
used to generate haptic effects independent of other piezoelectric
cells 210 in piezoelectric layer. In one embodiment, multiple
adjacent or neighboring cells 210 are capable of generating
multiple haptic effects in response to multiple substantially
simultaneous touches. In another embodiment, each of cells 210 has
a unique piezoelectric material thereby it is capable of initiating
a unique haptic sensation.
[0032] It should be noted that a tactile touch panel, which
includes an electrical insulated layer 302 and a piezoelectric
layer, in some embodiments further includes a display. This display
may be coupled to the bottom surface of the piezoelectric layer and
is capable of projecting images that are viewable from the top
surface of electrical insulated layer 302. It should be noted that
the display can be a flat panel display or a flexible display.
Piezoelectric materials 304, in one embodiment, are substantially
transparent and small. The dimension of a cell 210 having
piezoelectric material can be configured to be less than 5
millimeters by 5 millimeters. The shape of piezoelectric material
304, for example, deforms in response to electrical potentials
applied via electrical wires 306.
[0033] During a manufacturing process, a piezoelectric film is
printed to include an array or a grid of piezoelectric cells 210.
In one embodiment, a film of cells 210 containing piezoelectric
materials is printed on a sheet in a cell grid arrangement. The
film further includes wirings for directly addressing every cell
210 in the device using electrical control signals. Cells 210, for
example, can be stimulated using edge or back mounted electronics.
Piezoelectric materials may include crystals and/or ceramics such
as quartz (SiO.sub.2)
[0034] FIG. 3(b) illustrates a haptic cell 210 generating haptic
effects in accordance with an embodiment of the present invention.
During operation, when a voltage potential applies to piezoelectric
material 305 via wires 306, piezoelectric material 305 deforms from
its original shape of piezoelectric material 304, as shown in FIG.
3(a), to expanded shape of piezoelectric material 305. Deformation
of piezoelectric material 305 causes electrical insulated layer 303
to deform or strain from its original state of layer 302, as shown
in FIG. 3(a). In an alternative embodiment, piezoelectric materials
305 return to its original state as soon as the voltage potential
is removed. It should be noted that the underlying concept of the
present invention does not change if additional blocks (circuits or
mechanical devices) are added to the device illustrated in FIG.
3(a-b). If the piezoelectric material is replaced with other
materials such as shape memory alloys ("SMAs`), such material may
be capable of maintaining its deformed shape for a period of time
after the voltage potential is removed. It should be noted that the
underlying concept of the embodiments of the present invention does
not change if different materials other than piezoelectric
actuators are employed.
[0035] FIG. 4(a) is a diagram 400 illustrating another embodiment
of a haptic cell 210 using Micro-Electro-Mechanical Systems
("MEMS") device 402 to generate haptic effects in accordance with
one embodiment of the present invention. Diagram 400 depicts a
block 410, which shows a top view of cell 210. Cell 210 includes a
MEMS device 402. In one embodiment, MEMS device 402 is
substantially transparent thereby the image projection from a
display, not shown in FIG. 4(a), can be viewed through block 410.
It should be noted that each of haptic cells 210 is coupled to at
least one wire to facilitate and generate haptic effects.
[0036] MEMS can be considered as an integration of mechanical
devices, sensors, and electronics on a silicon or organic
semiconductor substrate, which can be manufactured through
conventional microfabrication process. For example, the electronic
devices may be manufactured using semiconductor fabrication process
and micromechanical devices may be fabricated using compatible
microfabrication process. In one embodiment, a grid or an array of
MEMS devices 402 are made of multiple cantilever-springs. A grid of
cantilever-springs can be etched using MEMS manufacturing
techniques. Also, electrical wirings for stimulating or driving
cantilever-springs can also be directly etched onto the surface of
the MEMS device 402 thereby every single MEMS device can be
correctly addressed. MEMS cantilevers can be stimulated using a
resonant drive (for vibrotactile) or direct actuation
(kinesthetic). In another embodiment, the MEMS are stimulated in
response to the energy generated by the display. For example, radio
frequency energy, light, or heat generated by the pixels of a
plasma display could provide an excitation source or activation
signal for a MEMS haptic cell.
[0037] FIG. 4(b) illustrates a side view of MEMS device 402,
wherein MEMS device 412 can be stimulated or deformed from its
original state of MEMS device 402 to deformed state of MEMS device
414 when a voltage potential across MEMS device is applied.
Displacement 404 between the original state and the deformed state
depends on the composition of materials used and the size of MEMS
device 402. Although smaller MEMS devices 402 are easier to
fabricate, they offer smaller displacement 404. In one embodiment,
cantilever-springs can be made of piezo materials. It should be
noted that the actuation of piezo material is generally
vibrotactile sensation. It should be further noted that piezo
material can be used as a sensor for sensing fingertip positions
and depressions.
[0038] MEMS device 402, in another embodiment, uses shape memory
alloy ("SMA") in place of cantilever-spring as mentioned above. The
actuation generated by MEMS device 402 using SMA provides
kinesthetic actuation. SMA, also known as memory metal, could be
made of copper-zinc-aluminum, copper-aluminum-nickel,
nickel-titanium alloys, or a combination of copper-zinc-aluminum,
copper-aluminum-nickel, and/or nickel-titanium alloys. Upon
deforming from SMA's original shape, SMA regains its original shape
in accordance with an ambient temperature and/or surrounding
environment. It should be noted that the present invention may
combine piezoelectric elements, cantilever-spring, and/or SMA to
achieve a specific haptic sensation.
[0039] FIG. 5(a) is a side view diagram of an interface device 500
illustrating an array of haptic cells 502 with thermal fluid
pockets 504 in accordance with one embodiment of the present
invention. Device 500 includes an insulated layer 506, a haptic
layer 512, and a display 508. While the top surface of insulated
layer 506 is capable of receiving inputs from a user, the bottom
surface of insulated layer 506 is placed adjacent to the top
surface of haptic layer 512. The bottom surface of haptic layer 512
is placed adjacent to display 508, wherein haptic layer 512 and
insulated layer 506 may be substantially transparent thereby
objects or images displayed in display 508 can be seen through
haptic layer 512 and insulated layer 506. It should be noted that
display 508 is not a necessary component in order for the interface
device to function.
[0040] Haptic layer 512, in one embodiment, includes a grid of
fluid filled cells 502, which further includes at least one thermal
fluid pocket 504 and an associated activating cell 510. It should
be noted that each of fluid filled cells 502 can include multiple
thermal fluid pockets 504 and associated activating cells 510. In
another embodiment, a fluid filled cell 502 includes multiple
associated or shared activating cells 510 thereby initiating a
different activating cell generates a different haptic
sensation(s).
[0041] Activating cell 510, in one embodiment, is a heater, which
is capable of heating an associated thermal fluid pocket 504.
Various electrical, optical, and mechanical techniques relating to
heating technology can be used to fabricate activating cells 510.
For example, various electrically controlled resistors can be used
for activating cells 510, wherein resistors can be implanted in
haptic layer 512 during the fabrication. Alternatively, optical
stimulators such as infrared lasers can be used as activating cells
510 to heat up thermal fluid pockets 504. Optical stimulator, for
example, can be mounted at the edge of the interface device. It
should be noted that activating cells 510 can be any types of
optical or radioactive stimulator as long as it can perform the
function of a heating device. Activating cells 510 may also use
rear mounted thermal stimulators, which are similar technologies
like hot plasma displays such as are commonly found in flat panel
plasma televisions.
[0042] Device 500 further includes a set of control wires, not
shown in FIG. 5(a), wherein each of activating cells 510 is coupled
to at least one pair of wires. The wires are configured to transmit
activating/deactivating control signals, which are used to drive
activating cells 510. It should be noted that each of fluid filled
cells 502 is addressable using signals from wires or wireless
networks. Display 508, in one aspect, can be a flat panel display
or a flexible display. In an alternative embodiment, the physical
location of display 508 is exchangeable with haptic layer 512.
Also, thermal fluid pockets 504, in one embodiment, can be
activated by a piezoelectric grid.
[0043] Thermal fluid pockets 504, in one embodiment, include fluid
with physical properties of low specific heat and high thermal
expansion. Examples of this fluid include glycerin, ethyl alcohol,
or the like. Thermal fluid pockets 504 are capable of producing
multiple localized strains in response to multiple touches received
by insulated layer 506. Each localized strain is created by a
heated thermal fluid pocket 504 wherein the heat is generated by an
associated activating cell 510. In one embodiment, a thermal fluid
pocket 504 changes its physical shape in accordance with the
temperature of the fluid in the pocket. In another embodiment,
fluid filled cell 502 has an active cooling system, which is used
to restore the expanded shape of thermal fluid pocket 504 to its
original shape after it is deactivated. The control of fluid
temperature affects haptic bandwidth. Rapid rising of fluid
temperature and fast heat dissipation of fluid enhance haptic
bandwidth of thermal fluid packets.
[0044] The physical size of each fluid cell 502 can also affect the
performance of the cell for generating haptic sensation(s). For
example, if the size of fluid cell 504 is smaller than 1/2
fingertip, the performance of cell 504 enhances because smaller
cell permits rapid heat dissipation as well as quick temperature
rising of fluid in the cell. In another embodiment, thermal plastic
pockets filled with plastic fluid are used in place of thermal
fluid pockets 504 filled with thermally sensitive fluid to enhance
the haptic effects. Using thermal plastic pockets filled with
plastic-like fluid can produce high thermal plastic strain. For
example, a type of plastic fluid is polyethylene. Thermal plastic
pockets can also provide different and unique haptic sensations to
the user. In another embodiment, some exotic fluids such as
electrorheological and/or magnetorheological fluid can be used in
place of thermal fluid in thermal fluid pockets 504. Thermal fluid
pockets 504 filled with electrorheological fluid can be stimulated
by a local or remote electrical field, while thermal fluid pockets
504 filled with magnetorheological fluid can be stimulated by a
local or remote magnetic field.
[0045] FIG. 5(b) is a side view diagram for an interface device 550
illustrating an array of haptic cells 502 using thermal fluid
pockets 554 in accordance with one embodiment of the present
invention. Device 550 also shows an activated thermal fluid pocket
554 and an activated activating cell 560. During the operation,
thermal fluid pocket 554 increases its physical volume (or size)
from its original state 556 to expanded thermal fluid pocket 554
when activating cell 560 is activated. When activating cell 560 is
activated, it provides heat 562 to thermal fluid pocket 554 or 556
to expand the size of thermal fluid pocket 554 or 556. Due to the
expansion of thermal fluid pocket 554, a localized portion 552 of
insulated layer 506 is created. As soon as the temperature of the
fluid in the thermal fluid pocket 554 cools down, the size of
thermal fluid pocket 554 returns to its original state 556. The
change of size between original size of a thermal fluid pocket 556
and expanded size of thermal fluid pocket 554 generates a haptic
effect. It should be noted that activating cell 560 could be an
electric heater or an optical heater such as an infrared
simulator.
[0046] FIG. 6(a) is a side view diagram of an interface device 600
illustrating an array of MEMS pumps 602 in accordance with one
embodiment of the present invention. Diagram 600 includes an
insulated layer 606 and a haptic layer 612. While the top surface
of insulated layer 606 is configured to receive a touch or touches
from a user, the bottom surface of insulated layer 606 is placed
adjacent to the top surface of haptic layer 612. The bottom surface
of haptic layer 612 is, in one embodiment, placed adjacent to a
display (not shown in FIG. 6(a)), wherein haptic layer 612 and
insulated layer 606 may be substantially transparent thereby
objects or images displayed in the display can be seen through
haptic layer 612 and insulated layer 606. It should be noted that
display is not a necessary component in order for the interface
device to function.
[0047] Haptic layer 612, in one embodiment, includes a grid of MEMS
pumps 602, which further includes at least one pocket 604. Each
MEMS pump 602 includes a pressurized valve 608 and a depressurized
valve 610. Pressurized valve 608 is coupled to an inlet tube 614
while depressurized valve 610 is coupled to an outlet tube 616. In
one embodiment, inlet tube 614, which is under high liquid
pressure, is used to pump liquid through pressurized valve 608 to
expand pocket 604. Similarly, outlet tube 616, which is under low
pressure, is used to release the liquid through depressurized valve
610 to release the pressure from pocket 604. It should be noted
that MEMS pumps 602 can be coupled to the same pressurized liquid
reservoir. It should be further noted that pressurized valve 608
and depressurized valve 610 can be combined into one single valve
for both inlet tube 614 and outlet tube 616. It should be further
noted that inlet tube 614 and outlet tube 616 can also be combined
into one tube.
[0048] A grid of MEMS pumps 602 includes an array of pressurized
valves 608 and depressurized valves 610, wherein pressurized valves
608 are coupled with a rear or a side mounted liquid reservoir
under pressure while depressurized vales 610 are coupled to a rear
or a side mounted depressurized liquid reservoir with low pressure.
Valves 608-610 control the filling and emptying the liquid pockets
604 in MEMS pumps 602 to produce localized strain. An advantage of
using pressurized liquid reservoir is to quickly deform the surface
of insulated layer 606 and to maintain the deformation with minimal
or no energy consumption (or expenditure). It should be noted that
MEMS pump 602 can also use pressurized air or other gases to
achieve similar results as liquid.
[0049] Device 600 further includes a set of control wires 617-618,
which can be used to control pressurized valve 608 and
depressurized valve 610, respectively. It should be noted that each
valve in haptic layer 612 is addressable using electrical signals
transmitted from wires or wireless network.
[0050] FIG. 6(b) illustrates two diagrams of an interface device
620 and 650 having an array of MEMS pumps 604 in accordance with
one embodiment of the present invention. Device 620 illustrates an
activated pocket 623, which includes an activated inlet valve 630
and a deactivated outlet valve 632. During an operation, pocket 623
increases its physical volume (or size) from its original state 624
to its expanded pocket 623 when inlet valve 630 is activated. When
inlet valve 630 is activated (or open) in response to electrical
signal from wire 628, inlet tube 625 pumps liquid 626 from
pressurized reservoir to pocket 623. Due to the expansion of pocket
623, a localized strain 622 of insulated layer 606 is created.
[0051] Device 650 illustrates an activated MEMS pump returns from
its expanded state of pocket 623 to the original state of pocket
653. When depressurized valve 660 is activated, depressurized valve
660 releases liquid 656 from pocket 653 to low pressurized outlet
654. It should be noted that depressurized valve 660 is controlled
by at least one control signal via wire 658. The changing in volume
between original size of pocket 604 and expanded size of pocket 623
generates haptic effects.
[0052] FIG. 7 illustrates a side view diagram for an interface
device 700 having an array of haptic cells 702 using variable
porosity membrane 710 in accordance with one embodiment of the
present invention. Device 700 includes an insulated layer 706 and a
haptic layer 712. While the top surface of insulated layer 706 is
configured to receive inputs from a user, the bottom surface of
insulated layer 706 is placed adjacent to the top surface of haptic
layer 712. The bottom surface of haptic layer 712 is, in one
embodiment, placed adjacent to a display (not shown in FIG. 7),
wherein haptic layer 712 and insulated layer 706 may be
substantially transparent thereby objects or images displayed in
the display can be seen through haptic layer 712 and insulated
layer 706. It should be noted that display is not a necessary
component in order for the interface device to function.
[0053] Haptic layer 712, in one embodiment, includes a grid of
haptic cells 702, inlet valves 703, and outlet valves 704. Haptic
cells 702, in one embodiment, are pockets capable of containing
fluid. Haptic layer 712 is similar to haptic layer 612 as shown in
FIG. 6(a) except that haptic layer 712 employs porosity membranes.
While each inlet valve 703 is controlled by control signal(s)
transmitted by wire 713, each outlet valve 704 is controlled by
electrical signals transmitted over a wire 714. Every inlet valve
703 or outlet valve 704 employs at least one porosity membrane 710.
Porosity membranes 710 are coupled (or face) to a liquid reservoir
wherein each membrane 710 is configured to control how much liquid
should enter and/or pass through membrane 710. An advantage of
using porosity membranes is to maintain the deformation of
insulated layer 706 with minimal or no energy consumption.
[0054] FIG. 8 is a side view of an interface device 800 having an
array of haptic cells 802 using various resonant devices in
accordance with one embodiment of the present invention. Device 800
includes an insulated layer 806 and a haptic layer 812. While the
top surface of insulated layer 806 is configured to receive an
input from a user, the bottom surface of insulated layer 806 is
placed adjacent to the top surface of haptic layer 812. The bottom
surface of haptic layer 812 is, in one embodiment, placed adjacent
to a display (not shown in FIG. 8), wherein haptic layer 812 and
insulated layer 806 may be substantially transparent thereby
objects or images displayed in the display can be seen through
haptic layer 812 and insulated layer 806. It should be noted that
display is not a necessary component in order for the interface
device to function.
[0055] Haptic layer 812, in one embodiment, includes a grid of
haptic cells 802, wherein each cell 802 further includes a
permanent magnet 804, an electro magnet 810, and two springs 808.
Haptic layer 812 is similar to haptic layer 612 shown in FIG. 6(a)
except that haptic layer 812 employs resonant devices while haptic
layer 612 uses MEMS pumps. Haptic cell 802, in one embodiment, uses
a resonant mechanical retractable device to generate haptic
effects. The resonant mechanical retractable device vibrates in
response to a unique frequency, which could be generated by a side
mounted resonant stimulator 816 or a rear mounted resonant
stimulator 814. A resonant grid, in one embodiment, is used to form
a haptic layer 812. Each cell 802 is constructed using resonant
mechanical elements such as Linear Resonant Actuator ("LRA") or
MEMS springs. Each cell 802, however, is configured to have a
slightly different resonant frequency and a high Q (high
amplification at resonance and a narrow resonant frequency band).
As such, each cell 802 can be stimulated using mechanical pressure
waves originating at the edges of the sheet. The haptic effects can
also be generated by a piezoelectric or other high bandwidth
actuator that induces acoustic waves.
[0056] Cell 802, in another embodiment, includes one spring 808. In
yet another embodiment, cell 802 includes more than two springs
808. Each spring 808 is configured to respond to a specific range
of frequencies thereby each spring 808 can produce a unique haptic
sensation.
[0057] FIG. 9(a) illustrates a top view diagram of a multi-touch
haptic display 900 having laminar flow of fluid in accordance with
one embodiment of the present invention. Display 900 includes a
fluid inlet reservoir 902, an array of MEMS cells 904, a fluid exit
reservoir 906, and a display 908. Fluid inlet reservoir 902 and
fluid exit reservoir 906 facilitate and guide laminar flow 910 from
fluid inlet reservoir 902 to fluid exit reservoir 906 as indicated
by large arrows. Laminar flow 910 is nonturbulent, streamline, or
smooth flow of a viscous fluid between layers. It should be noted
that laminar flow 910 and MEMS cells 904 may be substantially
transparent thereby objects or images displayed in display 908 can
be viewed through laminar flows 910 and MEMS cells 904. It,
however, should be noted that display 908 is not a necessary
component in order for the device to function.
[0058] Display 900 contains an array of individually addressable
MEMS cells or haptic elements 904. MEMS cells 904, also known as
MEMS turbulence inducing cells, are used to create multiple
asynchronous local haptic effects 912 across the display surface.
Each asynchronous local haptic effect 912 occurs when a local
turbulence is induced by an associated MEMS cell 914. When a MEMS
cell 914 is activated, it produces local turbulent flow and induces
a vibration or change of surface film texture of the cell 904,
which creates a haptic sensation or effect. When a MEMS cell 904 is
deactivated, laminar flow can flow through an associated MEMS cell
smoothly without any turbulence. Each MEMS cell 904 can be
activated independent of other MEMS cells 904 in display 900.
[0059] FIG. 9(b) illustrates a cross-section view diagram of a
multi-touch haptic display 950 having laminar flow of fluid in
accordance with one embodiment of the present invention. Display
950 includes a fluid inlet reservoir 902, an array of MEMS cells
904, a fluid exit reservoir 906, a display 908, and a thin flexible
transparent membrane 952. As shown in FIG. 9(b), smooth laminar
fluid flow 954 flows from fluid inlet reservoir 902 to fluid exit
reservoir 906 as indicated by many small arrows.
[0060] MEMS cells 904 includes multiple redundant deformable
structures wherein the deformable structures actuate out of the
plane of MEMS cell 904 when it is activated. When a MEMS cell 962
is activated, it activates the deformable structures, also known as
hairs, which cause a local patch of flowing fluid to transition
from laminar flow to turbulent flow 960. Fluid turbulence flow 960
causes local vibration of membrane and creates a localized haptic
sensation 958. Each MEMS cell 904 is addressable and can be
activated independent of other MEMS cells 904 in display 950.
[0061] The present invention includes various processing steps,
which will be described below. The steps of the present invention
may be embodied in machine or computer executable instructions. The
instructions can be used to cause a general purpose or special
purpose system, which is programmed with the instructions; to
perform the steps of the present invention. Alternatively, the
steps of the present invention may be performed by specific
hardware components that contain hard-wired logic for performing
the steps, or by any combination of programmed computer components
and custom hardware components. While embodiments of the present
invention will be described with reference to the Internet, the
method and apparatus described herein is equally applicable to
other network infrastructures or other data communications
environments.
[0062] FIG. 10 is a flowchart illustrating a process of providing
multiple haptic effects in accordance with one embodiment of the
present invention. At block 1002, a process displays an image,
which is viewable through an insulated layer. In one embodiment,
the display can be a flat panel screen or a flexible display.
Alternative, the process displays an opaque background without any
images. Also, the images may be projected onto the insulated layer
from the above or below. It should be noted that if the application
does not require displaying images, the display is not necessary
and may be removed. In an alternative embodiment, the insulated
layer is capable of interfacing with users and receiving inputs.
After block 1002, the process moves to the next block.
[0063] At block 1004, the process monitors the insulated layer. In
one embodiment, the process identifies the image and the possible
inputs can be detected in accordance with the image. The process is
capable of monitoring multiple contacts substantially same time.
The process proceeds to block 1006.
[0064] At block 1006, the process detects a first deformation of
the insulated layer in response to a first depressing by a first
finger. It should be noted that the finger can also be a stylus or
any finger-like pointed objects. The process can also detect the
first and the second deformations depressed by the same finger.
After block 1006, the process moves to the next block.
[0065] At block 1008, the process detects a second deformation of
the insulated layer substantially the same time as the first
deformation. The process is also capable of sensing the second
deformation in response to a second depressing by a second finger.
It should be noted that the second finger can be a stylus or any
kind of pointed objects. The process is capable of detecting more
deformations of the insulated layer if more depressions or contacts
are made. After block 1008, the process moves to the next
block.
[0066] At block 1010, the process generates a first input in
accordance with a location of the first deformation and a second
input in accordance with a location of the second deformation. The
process is capable of generating more inputs if more contacts or
depressions are detected.
[0067] At block 1012, the process activates a first haptic cell
with a first haptic effect in response to the first input and a
second haptic cell with a second haptic effect in response to the
second input. In one embodiment, the process initiates the first
haptic effect and the second haptic effect substantially the same
time. In another embodiment, the process activates a first
piezoelectric material of the first haptic cell to generate the
first haptic effect and activates a second piezoelectric material
of the second haptic cell to generate the second haptic effect. In
another embodiment, the process activates a first MEMS element of
the first haptic cell to generate the first haptic effect and
activates a second MEMS element of the second haptic cell to
generate the second haptic effect. In yet another embodiment, the
process activates a first fluid filled pocket of the first haptic
cell to generate the first haptic effect and activates a second
fluid filled pocket of the second haptic cell to generate the
second haptic effect.
[0068] While particular embodiments of the present invention have
been shown and described, it will be obvious to those skilled in
the art that, based upon the teachings herein, changes and
modifications may be made without departing from this invention and
its broader aspects. Therefore, the appended claims are intended to
encompass within their scope all such changes and modifications as
are within the true spirit and scope of this invention.
* * * * *