U.S. patent application number 15/641180 was filed with the patent office on 2017-10-26 for systems and methods for eliciting cutaneous sensations by electromagnetic radiation.
The applicant listed for this patent is Pine Development Corporation. Invention is credited to Alexander A. Brownell, William J. Yu.
Application Number | 20170308170 15/641180 |
Document ID | / |
Family ID | 50979435 |
Filed Date | 2017-10-26 |
United States Patent
Application |
20170308170 |
Kind Code |
A1 |
Yu; William J. ; et
al. |
October 26, 2017 |
SYSTEMS AND METHODS FOR ELICITING CUTANEOUS SENSATIONS BY
ELECTROMAGNETIC RADIATION
Abstract
The present disclosure provides various systems and methods for
inducing cutaneous sensations by delivering electromagnetic
radiation to directly or indirectly excite neural tissue. An
electromagnetic radiation source, such as one or more infrared
lasers, may be used to transcutaneously excite neural tissue. The
excited neural tissue may be interpreted by the user's nervous
system as cutaneous sensations. Accordingly, a system as described
herein may be used to induce sensations to allow actual cutaneous
sensations to be simulated. A system for inducing a cutaneous
sensation via transcutaneously focused electromagnetic radiation
may be incorporated in a display to provide cutaneous sensation
feedback or used as a separate accessory component associated with
a display. Numerous additional applications and variations are
provided herein.
Inventors: |
Yu; William J.; (Mountain
View, CA) ; Brownell; Alexander A.; (Bountiful,
UT) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Pine Development Corporation |
Mountain View |
CA |
US |
|
|
Family ID: |
50979435 |
Appl. No.: |
15/641180 |
Filed: |
July 3, 2017 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
14037290 |
Sep 25, 2013 |
9696804 |
|
|
15641180 |
|
|
|
|
13722844 |
Dec 20, 2012 |
8574280 |
|
|
14037290 |
|
|
|
|
61579776 |
Dec 23, 2011 |
|
|
|
61585741 |
Jan 12, 2012 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
A61N 2005/0643 20130101;
A61N 5/0622 20130101; A61N 5/06 20130101; G06F 3/016 20130101; A61N
5/00 20130101 |
International
Class: |
G06F 3/01 20060101
G06F003/01; A61N 5/06 20060101 A61N005/06; A61N 5/06 20060101
A61N005/06; A61N 5/00 20060101 A61N005/00 |
Claims
1. A system configured to induce a cutaneous sensation in a user of
an electronic device based upon a tactile application executable on
the electronic device, the system comprising: a stimulation system
configured to generate an output operable to excite neural tissue;
an interface component configured to direct the output of the
stimulation system onto a target area of skin of the user; and a
controller configured to generate a control signal to cause the
stimulation system to modify one or more characteristics of the
output of the stimulation system in order to induce a cutaneous
sensation based upon a tactile application executable on the
electronic device.
2. The system of claim 1, wherein the interface component is
further configured to direct the output of the stimulation system
onto the target area while the target area is in physical contact
with the interface component.
3. The system of claim 1, wherein the interface component is
further configured to direct the output of the stimulation system
onto the target area while the target area is physically separated
from the interface component.
4. The system of claim 1, wherein the electronic device comprises a
computer and the interface component comprises a mouse.
5. The system of claim 4, wherein the mouse comprises a control
surface, and the target area comprises a finger pad of a user that
interacts with the control surface.
6. The system of claim 4, wherein the control surface comprises one
of a button, a touch pad, and a track pad.
7. The system of claim 1, wherein the electronic device comprises a
computer and the interface component comprises a keyboard.
8. The system of claim 1, further comprising a display component,
the interface component being distinct from the display component,
and wherein the controller is configured to modify one or more
characteristics of the output of the stimulation system based on an
object on the display component.
9. The system of claim 8, wherein the sensation corresponds to an
object appearing on the display component.
10. The system of claim 1, wherein the electronic device comprises
one of a telepresence medicine device, a gaming device, a tablet
computer device, a telephone device, an electronic Braille display
device, an industrial control station, entertainment system, and an
electronic surgical control device.
11. The system of claim 1, wherein the system further comprises: a
thermal feedback system configured to measure a temperature
associated with the target area; and wherein the controller is
configured to dynamically control the stimulation system to
maintain the temperature below a threshold temperature.
12. The system of claim 1, wherein the output of the stimulation
system comprises one of infrared electromagnetic radiation and
visible electromagnetic radiation
13. The system of claim 1, wherein the stimulation system comprises
a laser emission system.
14. A method for inducing a cutaneous sensation in a user of an
electronic device, the method comprising: executing a tactile
application on an electronic device; generating, using an
stimulation system associated with the electronic device, an output
operable to excite neural tissue; directing the output of the
stimulation system onto a target area of skin of a user using an
interface component; and generating a control signal to cause the
stimulation system to modify one or more characteristics of the
output of the stimulation system in order to induce a cutaneous
sensation based upon the tactile application executing on the
electronic device.
15. The method of claim 14, wherein directing the output of the
stimulation system onto the target area occurs while the target
area is in physical contact with the interface component.
16. The method of claim 14, wherein directing the output of the
stimulation system onto the target area occurs while the target
area is physically separated from the interface component.
17. The method of claim 14, further comprising: displaying an
object on a display component; and wherein the sensation
corresponds to the object displayed on the display component.
18. The method of claim 14, wherein the electronic device comprises
one of a telepresence medicine device, a gaming device, a tablet
computer device, a telephone device, an electronic Braille display
device, an industrial control station, and an electronic surgical
control device.
19. The method of claim 14, further comprising: measuring a
temperature associated with the target area; and wherein the
controller is configured to dynamically control the stimulation
system to maintain the temperature below a threshold
temperature.
20. The system of claim 1, wherein the output of the stimulation
system comprises one of infrared electromagnetic radiation and
visible electromagnetic radiation.
Description
RELATED APPLICATIONS
[0001] The present application claims the benefit under 35 U.S.C.
.sctn.119(e) of U.S. Provisional Patent Application No. 61/579,776,
filed Dec. 23, 2011, and titled "SYSTEMS AND METHODS FOR OPTICALLY
EXCITING NEURAL TISSUE FOR HAPTICS APPLICATIONS;" Provisional
Patent Application No. 61/585,741, filed Jan. 12, 2012, and titled
"SYSTEMS AND METHODS FOR OPTICALLY EXCITING NEURAL TISSUE FOR
HAPTICS APPLICATIONS;" Utility Patent application Ser. No.
13/722,844, filed Dec. 20, 2012, and titled "SYSTEMS AND METHODS
FOR ELICITING CUTANEOUS SENSATIONS BY ELECTROMAGNETIC RADIATION;"
Utility patent application Ser. No. 14/037,290, filed Sep. 25,
2013, and titled "SYSTEMS AND METHODS FOR ELICITING CUTANEOUS
SENSATIONS BY ELECTROMAGNETIC RADIATION, all of which are
incorporated herein by reference in their entireties.
TECHNICAL FIELD
[0002] The present disclosure is directed to systems and methods
for directly or indirectly exciting neural tissue using
electromagnetic radiation. More particularly, the present
disclosure is related to stimulation of neural or other excitable
tissues using electromagnetic radiation for inducing cutaneous
sensations.
BRIEF SUMMARY
[0003] According to various embodiments, a system for
inducing--cutaneous sensations, may comprise an electromagnetic
radiation emission system configured to emit electromagnetic
radiation suitable for directly or indirectly exciting neural
tissue. The system may also include an electronic display
configured to display a graphical user interface and a detection
system configured to detect a point of contact (such as a finger
contact) with the display. A controller configured to transmit a
control signal to the electromagnetic radiation emission system to
cause the electromagnetic radiation emission system to direct
electromagnetic radiation at the contact detected by the detection
system to directly or indirectly excite neural tissue associated
with the contact in order to induce a cutaneous sensation.
According to some embodiments, the electromagnetic radiation
emission system may further comprise at least one focusing element
controllable for selectively focusing the electromagnetic radiation
emitted by the electromagnetic radiation emission system.
Transcutaneously focused electromagnetic radiation may include
single or multiple beams of electromagnetic radiation coincident at
a focal point. Alternatively, a splitting element may be utilized
in some embodiments in order to direct electromagnetic radiation
from one source of electromagnetic radiation to a plurality of
points of contact. Another embodiment would be a single source of
electromagnetic radiation emission that is redirected or switched
by the controller to individual fiber optic lines (e.g. an optical
switch), which are spatially arranged to allow for specific
illumination/irradiation at specific points on the user. Finally,
some embodiments may incorporate multiple sources of
electromagnetic radiation that may be selectively directed toward
multiple points of contact.
[0004] The system may further comprise a storage medium containing
a library of cutaneous sensations, each of which may be defined by
a set of characteristics of the electromagnetic radiation. The
controller may be configured to modulate the characteristics of the
electronic radiation emitted by the electromagnetic radiation
emission system to induce a specific cutaneous sensation. These
individual or pre-defined sensations can also be combined and
tailored via the controller to create unique cutaneous
sensations.
[0005] At least one of the cutaneous sensations may be defined by a
set of characteristics of the electromagnetic radiation in at least
two locations in the neural tissue separated by more than a
two-point discrimination region. The controller may be configured
to modify or modulate one or more characteristics of the
electromagnetic radiation emitted by the electromagnetic radiation
emission system to induce a cutaneous sensation corresponding to an
object displayed on the graphical user interface at the location of
the contact with the electronic display. According to some
embodiments, the characteristics of the electromagnetic radiation
modified or modulated by the controller may include pulse width,
pulse repetition rate, shape, amplitude, fluence, depth, frequency,
location(s), spot size, wave shape, duty cycle, rasterization
patterns, and the like.
[0006] The electromagnetic radiation emission system may comprise
Light-Emitted Diodes (LEDs) or various forms of laser sources,
including edge-emitting and surface emitting semiconductor lasers
for example, and nonlinear frequency conversion of these laser
sources. Of course, according to various embodiments, other types
of visible and electromagnetic radiation sources may also be
utilized.
[0007] The electronic display may be a touch screen electronic
display, and the detection system may utilize a touch screen
digitizer or the like of the touch screen electronic display to
detect the contact and determine the point of contact or area of
contact with the user. The electromagnetic radiation emission
system may be part of a moveable stage configured to move relative
to the plane of the electronic display, and the controller may be
configured to control the movement of the stage to direct the
electromagnetic radiation to the contact detected by the detection
system. The stage may comprise at least one magnet, and the
controller may be configured to control the movement of the stage
relative to the plane of the electronic display using a series of
electromagnets proximate at least two edges of the electronic
display. Alternatively, other forms of mechanical actuation may be
utilized to reposition the moveable stage. The moveable stage will
allow for mounting one of a mirror(s) and a focusing element(s)
such as a lens that can direct the incoming electromagnetic
radiation from the perimeter of the display to perpendicular to the
plane of the display and into the point of user contact.
[0008] Alternatively, the electromagnetic radiation emission system
may be configured to direct the electromagnetic radiation to the
point of contact detected by the detection system via the stage via
fiber optic cable mounted to a moveable stage. The controller may
be configured to cause the electromagnetic radiation emission
system to transcutaneously focus electromagnetic radiation at the
contact using a procession pattern bounded by a two-point
discrimination region.
[0009] The system may further comprise a sub-threshold electrical
stimulation system configured to electrically stimulate a portion
of the user. According to some embodiments, the electrical
stimulation system is used to elicit electrical stimulation to
achieve a subthreshold value that can later use electromagnetic
radiation to achieve the threshold and achieve sensation. The
controller may be further configured to adjust the fluence of the
electromagnetic radiation based on calibration results. For
example, the calibration results may define a minimum energy
density to induce a cutaneous sensation in the contact. The
calibration results may be obtained from a calibration phase,
performed by directing electromagnetic radiation of various
fluences at the point of contact, receiving feedback from a user
indicating which of the electromagnetic radiation pulses induced a
cutaneous sensation in the contact; and associating a minimum
energy density to induce a cutaneous sensation with the lowest
fluence indicated by the user as having induced a cutaneous
sensation.
[0010] The system may further comprise a thermal feedback system
configured to measure a temperature associated with the contact,
and the controller may be configured to dynamically control the
electromagnetic radiation emission system to only deliver the
appropriate fluence to achieve the desired stimulation. According
to one embodiment the temperature of the finger or other body part
is determined by a thermistor, or the like, to provide the control
system with an indication of the skin temperature. According to
another embodiment, the temperature of the glass is maintained at a
certain known temperature using feedback from an embedded
thermistor, or the like, to provide an indication of the glass
temperature.
[0011] The thermal feedback system may comprise a non-contact
infrared thermometer, a thermistor, and/or a thermocouple.
[0012] The system for inducing cutaneous sensations may be
implemented on a user interface component instead of or in addition
to a display. A discrete user interface component may be associated
with a display in some embodiments. The user interface component
may comprise a track pad or a keyboard or a mouse key or any
portion thereof. According to some embodiments the user interface
component may comprise an enclosure, and the enclosure may be
configured to receive at least one finger (or other portion of the
user, such as a hand) within the enclosure. The enclosure may
comprise a glove configured to enclose two or more fingers, a
finger wrap configured to receive a single finger, or a hand
enclosure configured to receive a hand. According to one
embodiment, a finger wrap or finger sleeve may include embedded
fiber optic lines. An optical switch may be used to deliver
electromagnetic radiation to a target area. Further, such
embodiments may be configured to deliver a rasterized pattern of
electromagnetic energy in order to stimulate multiple target
areas.
[0013] The user interface component may be associated with a
display, and the controller may be configured to modify one or more
characteristics of the electromagnetic radiation emitted by the
electromagnetic radiation emission system to induce a cutaneous
sensation corresponding to an object displayed on the display. The
system may be integrated into a peripheral computing device
configured to allow a user to provide input to a computing device,
and the user interface component may comprise a surface of the
peripheral device. The peripheral computing device may comprise one
of a computer mouse and a computer keyboard, and the user interface
component comprises a surface of a button.
[0014] In one embodiment, a system for communicating visual
information via cutaneous sensations may comprise an imaging device
configured to image at least one object; an electromagnetic
radiation emission system configured to emit electromagnetic
radiation suitable for directly or indirectly exciting neural
tissue. The system may further include an interface component
configured to deliver electromagnetic radiation to a target area of
a user's skin and a controller configured to control operation of
the electromagnetic radiation emission system. The controller may
map at least one object imaged by the imaging device to a cutaneous
sensation and transmit a control signal to the electromagnetic
radiation emission system to cause the electromagnetic radiation
emission system to deliver electromagnetic radiation at the point
of contact to directly or indirectly excite neural tissue and
thereby induce a cutaneous sensation at the point of contact.
[0015] In another embodiment, a multi-layer display configured to
induce cutaneous sensations may comprise a touch screen digitizer
layer or the like configured to detect a point of contact with a
user. The multi-layer display may also include an electronic
display layer configured to display objects; a spatial light
modulator (SLM) layer configured to dynamically focus and steer
electromagnetic radiation; a VCSEL and lenslet array layer
configured to selectively emit electromagnetic radiation suitable
for exciting neural tissue. A controller may be configured to
control the SLM layer and the VCSEL and lenslet array layer to
deliver electromagnetic radiation at the point of contact to
thereby directly or indirectly excite neural tissue to induce a
cutaneous sensation at the point of contact.
[0016] A method for inducing cutaneous sensations may comprise
displaying graphical information on an electronic display;
detecting a point of contact with a user on the display; and
transcutaneously focusing and steering electromagnetic radiation at
the contact of the finger to excite neural tissue in the finger to
induce a cutaneous sensation.
[0017] Various methods are also disclosed herein for inducing
cutaneous sensations. Such methods may include displaying a
graphical user interface on an electronic display and detecting a
point of contact of a finger on a haptic feedback surface
associated with the electronic display. Further the method may
include generating a control signal to cause an electromagnetic
radiation emission system to deliver electromagnetic radiation. In
response to the control signal, electromagnetic radiation may be
delivered at the point of contact to directly or indirectly excite
neural tissue in the finger and thereby induce a cutaneous
sensation.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Non-limiting and non-exhaustive embodiments of the
disclosure are described herein, including various embodiments of
the disclosure illustrated in the figures listed below.
[0019] FIG. 1 illustrates a block diagram of a system for exciting
tissue using electromagnetic radiation, according to certain
embodiments.
[0020] FIG. 2 illustrates a simplified embodiment of a system for
delivering electromagnetic radiation onto a finger of a user,
according to certain embodiments.
[0021] FIG. 3 illustrates a simplified embodiment of multiple
electromagnetic radiation beams transcutaneously coincident within
a finger of a user, according to certain embodiments.
[0022] FIG. 4 illustrates a display associated with a system for
inducing cutaneous sensations via transcutaneously focused
electromagnetic radiation, according to certain embodiments.
[0023] FIG. 5A illustrates a touch screen configured to induce
cutaneous sensations in a user's finger by delivering
electromagnetic radiation to a point of contact with the user,
according to certain embodiments.
[0024] FIG. 5B illustrates an accessory component configured to
induce cutaneous sensations in a user's finger while using a
display by delivering electromagnetic radiation to a point of
contact between the accessory and the user, according to certain
embodiments.
[0025] FIG. 5C illustrates a conceptual representation of an
electromagnetic radiation delivery system including a single
electromagnetic radiation source that may be incorporated into a
finger sleeve or other device, according to certain
embodiments.
[0026] FIG. 5D illustrates a conceptual representation of an
electromagnetic radiation delivery system 580 including a plurality
of electromagnetic radiation sources that may be incorporated into
a finger sleeve or other device, according to certain
embodiments.
[0027] FIG. 6A illustrates an embodiment of a moveable stage for
transcutaneously rastering electromagnetic radiation to excite
tissue, according to certain embodiments.
[0028] FIG. 6B illustrates another embodiment of a moveable stage
for transcutaneously rastering electromagnetic radiation to excite
tissue, according to certain embodiments.
[0029] FIGS. 7A-C illustrate examples of rasterization patterns for
inducing cutaneous sensations using electromagnetic radiation,
according to certain embodiments.
[0030] FIG. 8 illustrates an electro-optical system for inducing
cutaneous sensations, including an electrical stimulation system
and a system for transcutaneously focusing electromagnetic
radiation, according to certain embodiments.
[0031] FIG. 9 illustrates a schematic of an initial user
calibration procedure of a device including an electromagnetic
radiation stimulation system for inducing cutaneous sensations.
[0032] FIG. 10 illustrates a block diagram of a system for inducing
cutaneous sensations using electromagnetic radiation including a
thermal feedback system, according to certain embodiments.
[0033] FIG. 11 illustrates a system integrated within a peripheral
device of a computer for inducing cutaneous sensations using
electromagnetic radiation, according to certain embodiments.
[0034] FIGS. 12A-C illustrate three embodiments for directing
electromagnetic radiation to a point of contact using
electromagnetic radiation within a surface, according to certain
embodiments.
[0035] FIG. 13 illustrates an example of a display incorporating a
system for inducing cutaneous sensations using electromagnetic
radiation, according to certain embodiments.
[0036] FIG. 14A illustrates a schematic of a relatively thin fluid
layer configured to provide ocular protection from electromagnetic
radiation that may be used to induce haptic sensations, according
to certain embodiments.
[0037] FIG. 14B illustrates a finger depressing the relatively thin
fluid layer, thereby allowing electromagnetic radiation to
penetrate the fluid layer and induce a cutaneous sensation in the
finger of the user, according to certain embodiments.
[0038] FIG. 15A illustrates an embodiment of a display
incorporating a system for inducing cutaneous sensations using
electromagnetic radiation and an array of lenslets and/or VCSELs,
according to certain embodiments.
[0039] FIG. 15B illustrates an embodiment of a display, in which
collimated electromagnetic radiation traverses a transmissive
spatial light modulator layer.
[0040] FIG. 15C illustrates an embodiment of a display including a
spatial light modulating layer, which may impose a grating or dot
pattern for scanning stimulation spots and rasterization
schemas.
[0041] In the following description, numerous specific details are
provided for a thorough understanding of the various embodiments
disclosed herein. The systems and methods disclosed herein can be
practiced without one or more of the specific details, or with
other methods, components, materials, etc. In addition, in some
cases, well-known structures, materials, or operations may not be
shown or described in detail in order to avoid obscuring aspects of
the disclosure. Furthermore, the described features, structures, or
characteristics may be combined in any suitable manner in one or
more alternative embodiments.
DETAILED DESCRIPTION
[0042] According to various embodiments, electromagnetic radiation
may be used to induce apparent cutaneous sensations in a user.
Accordingly, the various embodiments of the systems described
herein are configured to induce cutaneous sensations through the
application of transcutaneous electromagnetic radiation. In some
embodiments, mechanical deformation of the skin is used to produce
tactile sensation. Challenges associated with using mechanical
devices for creating haptic sensations may include the inertia of
moving parts and the difficulty in miniaturization to create
sufficiently high resolution. In other embodiments, direct
electrical stimulation of tissue may be used. However, electrical
stimulation often has poor spatial resolution due to current
spreading between electrodes.
[0043] Electromagnetic radiation, such as light emitted in the
infrared or visible spectrum, may be applied to a user's skin in
order to stimulate neural tissue in the skin and thereby induce
action potentials, either directly or indirectly, at the site of
irradiation. Irradiation of the skin that induces, either directly
or indirectly, action potentials in the peripheral nervous system
which are highly spatially selective, and may thus achieve high
resolution and may be utilized in connection with many
applications.
[0044] Lasers may be used in ablative and non-ablative
applications. Ablative laser systems may impart sufficient energy
to the tissue so that some portion of the tissue architecture may
be destroyed or otherwise transformed. For example, ablative lasers
may be used in surgery to replace or supplement the use of scalpels
and cautery instruments. Ablative lasers may also be used in
aesthetic dermatology to encourage dermal remodeling. At somewhat
lower energies, both ablative and non-ablative, systems may be
adapted for port wine stain removal using photocoagulation
techniques that selectively destroy the excessive accumulation of
blood vessels.
[0045] Non-ablative laser technologies may apply lower energies or
fluences than ablative lasers. Non-ablative lasers may be used to
promote wound healing, relax sore muscles, and potentially alter
cellular function in various ways. Low-level light therapy (LLLT)
devices have applications in medical and veterinary uses, as well
as in the field of dentistry. According to various embodiments
described herein, electromagnetic radiation may be used at
wavelengths at energies that do not cause tissue damage. According
to some embodiments, the energy density utilized by some
embodiments may be higher than that of traditional LLLT
devices.
[0046] Electromagnetic radiation suitable for exciting neural
tissue, either directly or indirectly, may include specific
wavelengths or a range of wavelengths, according to various
embodiments. The application of electromagnetic radiation to
nervous tissue may elicit action potentials. The system may utilize
a wide range of one or more different wavelengths between
approximately 400 nanometers (nm) and 8000 nm, including but not
limited to 650, 808, 850, 860, 885, 915, 940, 980, 1064, 1120,
1310,1450, 1470, 1490, 1495, 1540, 1550, 1850, 1862, 1870, 2000,
2100, 2120, 3000, 4000, and 6000 nm. The present disclosure
contemplates several different embodiments of the device with
different electromagnetic radiation sources. For example, one
embodiment uses a laser as the electromagnetic radiation source. In
another embodiment, the electromagnetic radiation source may
utilize one or more LEDs. In another embodiment, a flash tube
broad-spectrum light source may be utilized. Any of a wide variety
of electromagnetic radiation sources capable of delivering
electromagnetic radiation at a sufficient power density may be
utilized. Some embodiments may utilize a single wavelength source,
filter all but a single wavelength of a broad-spectrum source,
and/or utilize a multi-wavelength source of electromagnetic
radiation. In some embodiments, a filter may be placed at any point
between the electromagnetic radiation source and the tissue to be
stimulated.
[0047] The present disclosure provides various embodiments of
systems and methods for inducing cutaneous sensations using
transcutaneously delivered electromagnetic radiation. As used
herein, the terms electromagnetic radiation represents the breadth
of the electromagnetic spectrum, as applicable to the present
disclosure. In various embodiments, tissue may be transcutaneously
irradiated for the purpose of simulating the sensations of
cutaneous touch. The cutaneous sensations may represent physical
traits of actual objects at a remote location, or can represent
simulated objects.
[0048] For example, a system for inducing cutaneous sensations
using transcutaneously delivered electromagnetic radiation may be
used for any number of tactile applications, such as, but not
limited to, telepresence medicine, compact electronic Braille
displays, virtual product online shopping, representing virtual and
physical objects and drawings in computer generated images and
computer aided drafting (CAD), for touch screen feedback, control
feedback, and/or entertainment and gaming devices. The use of
electromagnetic radiation for stimulating neural tissue may be more
responsive than a mechanical system and may provide higher spatial
resolution than a purely electrical system. In addition, the lack
of moving parts may result in higher reliability and lower
maintenance of the system.
[0049] In some embodiments, transcutaneously delivered
electromagnetic radiation may be used to induce cutaneous or
subcutaneous sensations for use in less-than-lethal weapons.
Less-than-lethal weaponry is widely used by military and police
forces for crowd control and other situations where slowing or
immobilizing a person is preferable to causing serious injury or
death. Transcutaneous application of electromagnetic radiation may
be used to cause a sensation on or beneath the skin. For example, a
less-than-lethal weapon utilizing transcutaneously applied
electromagnetic radiation may be used to cause sensations
associated with burning, pressure, scraping, cutting, and/or other
unpleasant or painful sensations that may deter a person from a
particular course of conduct. Such a system may be configured to
cause no damage, or minimal damage, to tissue. Rather, the system
may simply induce sensations in the brain as being extremely
unpleasant or injurious.
[0050] In some embodiments, a patch or plate, which is adhered to
the skin, may be used to transmit electromagnetic radiation
to/through the skin. Such a device may be adapted to communicate
tactile information to the wearer discretely and/or silently. Such
a system may be used in silent military applications.
[0051] In some embodiments, a system configured to induce cutaneous
sensations via transcutaneously focused electromagnetic radiation
may be used to induce pleasant sensations as well. For example, a
system may be adapted to comfort and/or calm premature infants in
incubators. The system may simulate human contact without exposing
them to the contamination that the incubator intends to avoid. In
some embodiments such a patch could provide comfort to older
patients as well. Those afflicted with depression, seasonal
affective disorder, or other mental illness that have shown
response to vagal nerve stimulation. Cutaneous stimulation on the
proper body parts may also help alleviate some of their symptoms.
Also, in autism and other developmental disorders many individuals
engage in autostimulation behaviors. In some of these cases the
autostimulation behavior can cause serious injury. Optical
stimulation of these patients may prove to satisfy the desire for
stimulation in a less injurious manner.
[0052] A system may utilize a control program to control the
application of electromagnetic radiation. For example, the
electromagnetic radiation may be directed to the tissue in such a
way that only a small portion of the tissue is irradiated. The
tissue may be excited in such a way that the brain perceives it as
mechanical stimulation. The amount of energy imparted to the tissue
may be the minimum necessary to reliably and reproducibly elicit
the desired response. In some embodiments, the control program may
be calibrated for a set of users and/or a specific user. In some
embodiments, a feedback mechanism may be used to dynamically adjust
the output. For example, the control program may be initially
calibrated and then dynamically adjust the amplitude, focus,
rasterization pattern, and/or other attributes of the
electromagnetic radiation based on a thermal sensor to protect the
skin from damage.
[0053] The control program may utilize an infrared imaging device
or other temperature probe to detect the surface temperature of the
skin and make appropriate adjustment to the stimulation protocol.
In another embodiment the feedback may be quasi-closed loop and may
be accomplished by incorporating calculations from a proprietary
computer simulation, and/or empirical data collected from various
human or phantom tissue testing in the form of a look-up table
where the stimuli delivered are known to change the tissue
temperatures and subsequent stimuli are adjusted accordingly.
[0054] Tactile and/or other cutaneous sensations may be created by
the activation of mechanoreceptors that are normally triggered.
These receptors are distributed unequally in different areas of the
skin. In order to selectively stimulate a different number of
receptors, neuronal axons, or other excitable tissues the
application of optical energy may be applied in a controlled
manner.
[0055] In some embodiments, a plurality of optical focusing devices
may be used to direct electromagnetic radiation to the tissue. This
may be accomplished through the use of any combination of lenses,
mirrors, fiber optics, and/or other electromagnetic manipulation
materials. The incident electromagnetic radiation may be focused to
provide a spot size, large enough to assure stimulation of
excitable tissues, while remaining small enough that collateral
heating of non-excitable tissues are minimized. The beam shape of
the electromagnetic radiation may be controlled to limit collateral
heating of non-excitable tissues. For example, a highly converging
beam with short focal region may be focused at or beneath the skin
surface. In other embodiments, the electromagnetic radiation may
comprise several beams of electromagnetic radiation focused
transcutaneously. Focusing electromagnetic radiation may include
the utilization of optical components such as lenses and/or
mirrors, and/or the usage of coincident beams of electromagnetic
radiation. The focal point(s) may be at a location(s) within the
tissue where electromagnetic radiation can be used to produce
neural excitation.
[0056] In some embodiments, to avoid overheating of a single area
of tissue, beam procession may be used within a small area. The
procession of the beam may be confined to an area where different
stimuli are spatially indistinguishable by the brain. In other
words, the area of confinement for the procession may be
experienced by the user as stimulation of the same point on the
skin. Accordingly, two-point discrimination may vary depending upon
which area of a user's body is irradiated.
[0057] Accordingly, by rasterizing the applied beam of
electromagnetic radiation, the system may stimulate many points on
the skin simultaneously, or nearly simultaneously, and/or may
reduce cutaneous and/or subcutaneous thermal buildup. In one
embodiment, the beam is scanned or rastered through the use of a
device, such as a galvanometer-based optical scanner. In another
embodiment, one or more prisms may split the beam and the split
beams may be shuttered, such as via a mechanical and/or liquid
crystal display (LCD) based spatial light modulation system. In
another embodiment, a Spatial Light Modulator (SLM) may be used to
dynamically modify the wave front of the electromagnetic radiation
in order to adaptively focus the beam inside the skin layer. A
grating structure can also be written on the same SLM in order to
scan the electromagnetic radiation over the skin.
[0058] In another embodiment, the beam may be split and transmitted
via a multi-bundle fiber array combined with shutter control at the
output of each of the fibers from the bundle. In one embodiment,
optical rastering can be implemented by the use of a fiber bundle
with N fibers and a 1.times.N optical switch. The stimulation light
can come from either a single LED/laser source, or it can be the
combined output from M LED/laser sources. By connecting a 1.times.N
optical switch to an N-fiber bundle, the light can be sequentially
directed to any one of the N-fibers in the bundle by the use of the
optical switch.
[0059] In another embodiment, digital light processing technology
may be used to split and direct multiple beams. In one embodiment,
a two-dimensional motion stage, described in terms of an X-Y
coordinate system, may be used to move the electromagnetic
radiation source. In some embodiments, a tilting mechanism may be
used to adjust the incident angle and allow for a greater area for
beam delivery. That is, the electromagnetic radiation source may be
moved within a limited two-dimensional array combined with a
tilting mechanism to widen the effective two-dimensional range of
the system.
[0060] Any of the variously described embodiments of systems for
inducing cutaneous sensations via transcutaneously focused
electromagnetic radiation may be integrated within a display, such
as a touch screen display. For example, a system may be integrated
within an LCD or organic LED (OLED) screen. For example, a system
may be integrated and associated with a pixel or cluster of pixels
adjacent to an electromagnetic radiation source or electromagnetic
radiation transmission element. The system may be configured to
provide tactile feedback associated with the display or touchscreen
display. In other embodiments, the electromagnetic radiation source
and/or electromagnetic radiation transmission element(s) may be
placed behind a display that is transparent to the electromagnetic
radiation so that the electromagnetic radiation passes through the
display or vias built into the display.
[0061] For example, the electromagnetic radiation may be directed
through the surface of a touch screen display into the finger,
fingers, and/or hand of a user to induce a cutaneous sensation. A
system, according to any of the embodiments described herein, may
be integrated into an interactive display such as on a smartphone,
tablet computer, computer monitor, or television. In such devices,
the location or placement of a finger or other object may be
determined by hardware built directly into the screen and/or
software. The location and contact area information may be utilized
by the system to irradiate only when and where tissue (e.g., a
finger) is present. In addition to location and contact area
information, movement attributes such as speed and direction may be
determined and used by the control system to dynamically adjust the
electromagnetic radiation transmission settings. For example, the
sensations induced by the transcutaneously focused electromagnetic
radiation may simulate a textured surface. The textured surface
felt may be dynamic and/or changeable. The sensations could also be
used as feedback for actions performed, such as a button press. The
stimulation surface may also be a track pad or other dedicated
non-display surface through which the electromagnetic stimulating
energy may pass.
[0062] In one embodiment, a system for inducing cutaneous
sensations using transcutaneously focused electromagnetic radiation
may be embodied as an off-display device associated with a second
device. For example, a system may interact with a user's single
finger, multiple fingers, or a full hand. In some embodiments,
users may insert a portion of their bodies, such as a finger, hand,
arm, etc. within the stimulating area of the device. The user, or
portion of the user inserted within the system, could be held
immobile or allowed to move. The system may induce sensations
associated with virtual objects such that they feel or provide
simulated sensations associated with corresponding physical
objects. In some embodiments, the portion of the user to be
irradiated with electromagnetic radiation may be decoupled from
other surfaces, so as to limit sensations other than those induced
via the transcutaneously focused electromagnetic radiation.
[0063] In some embodiments, the off-display embodiment may be
passive in the sense that the tissue to be stimulated cannot move
to interact with a display of the object being represented. In
other embodiments, the off-display embodiment may be partially
interactive by allowing the finger(s) and/or hand to move and/or
respond within the off-display system. In such an embodiment, the
system may track movement and make appropriate adjustments to the
stimulating beams for appropriate focus.
[0064] In another embodiment, a covering or housing may be secured
to a finger(s) or hand of a user that allows the user to interact
with a display. The covering may prevent the finger(s) or hand from
receiving mechanical stimulation, such as from the surface of the
touch screen display. Electromagnetic radiation may be directed
onto the finger pad(s) of the user via fiber optics and/or other
lenses or mirrors within the covering. The fiber optics may be
connected to a remote electromagnetic radiation source. The
covering may allow for interaction with a display or other
interface. The electromagnetic radiation may induce sensations
during the interaction and the covering may limit extraneous
sensations (such as the texture of the display). In some
embodiments, the display may be a computer or phone screen. The
display could also be a holographic or virtual reality display
presented in two or three-dimensions.
[0065] A library of stimulation protocols may dictate the various
sensations that can be induced by the system. Appropriately
stimulating different receptors, axons, dendrites or other
excitable tissues with electromagnetic radiation at the appropriate
place on/within the skin and with the appropriate repetition rates
may be used to effectively replicate tactile stimulation sensations
experienced by touching a physical object. A library may contain
the basic components of complex sensations that, when combined
appropriately, are capable of inducing a wide range, or even all,
of the cutaneous sensations, including, but not limited to, those
involving textures, pain, hot, cold, wet, dry, sticky, etc. Thus, a
controller may modify the characteristics of the electromagnetic
radiation to change the pulse width, shape, amplitude, energy
density, duty cycle, frequency, depth, location(s), spot size, wave
shape, modulation characteristics, rasterization patterns and/or
other characteristics of the electromagnetic radiation beam to
induce any of a wide variety of cutaneous sensations. In some
embodiments, these pre-defined cutaneous sensation effects can be
combined and mixed appropriately to create new sensations within
predetermined safety limits to prevent harm to the user.
[0066] The stimulation protocols may include waveforms of various
shapes and patterns. The various pulses delivered are combined into
trains consisting of, but not limited to, square, triangular,
trapezoidal, and sinusoidal shaped pulses. Electromagnetic
radiation may be continuously emitted, pulsed, electronically
shuttered, pulse-width modulated (PWM), and/or otherwise modulated
or pulsed. The spot size of the incident electromagnetic radiation
may also be varied to create different sensory effects. This spot
could be dynamically altered by movable lenses and/or by a variable
aperture.
[0067] Obtaining tactile information may be done by a number of
different means. In one embodiment, a system may be configured to
obtain tactile information for replication using an ultrasonic
probe. An ultrasonic probe may be used to gather topographical
information of an object, how compliant an object is, and/or
subsurface characteristics of an object. In some embodiments, a
laser may be used to determine the characteristics of a surface of
an object. In some embodiments, a 3-D camera system may also be
used to capture surface characteristics of an object. In another
embodiment, a series of probes mounted on calibrated springs may be
utilized to mechanically determine characteristics of a surface. As
the spring is compressed, its displacement gives the appropriate
information about the object's properties. Additionally, a
differential transformer may be used to measure linear or
translational displacements on a surface.
[0068] In one embodiment, a system, as described herein, may be
utilized by the user to measure physical characteristics of objects
and induce corresponding sensations for the user. For example, an
imaging device may be used to scan the surface of an object. The
system may translate the image into a series of tactile sensations
to be induced using transcutaneously focused electromagnetic
radiation. In some embodiments, characteristics such as color, grey
scale, line thickness, temperature, and the like, may be translated
into tactile sensations.
[0069] In one embodiment, a sub-threshold electrical stimulation
system may be combined with transcutaneously focused
electromagnetic radiation. For example, electrodes may be placed
near or at the location of the user where electromagnetic radiation
is to be received. The electrodes may be configured to provide
sub-threshold stimuli in the general area. Accordingly, the
electrodes themselves may not produce any action potential in the
mechanoreceptors or their afferent axons. The electrical stimuli
may be cyclic at high rates corresponding to the necessary
electromagnetic stimulation. Electromagnetic radiation may be
applied in conjunction with the cyclical electrical stimuli.
According to such an embodiment, since the electrical stimuli
provides a sub-threshold stimulation, the electromagnetic radiation
may be used to induce cutaneous sensations at lower energy
densities and may also provide for greater selectivity of action
potentials from A.beta. and A.delta. neural fibers that convey
tactile information over C fibers that carry pain and thermal
information.
[0070] In one embodiment, the tactile information conveyed by a
system as described herein may be associated with a tactile logo or
tactile signature. A static or dynamic sensation may be
incorporated into any number of applications. For example, a
tactile logo may be felt beneath a visually displayed web page. The
logo may not be visually displayed and only felt by the user. Such
functionality could be incorporated in both on-display and
off-display systems as described herein.
[0071] For example, the tactile logo could be generated constantly
beneath the finger every time the finger is in contact with the
screen. Alternatively, the tactile logo may be associated with
specific displayed content, such as, but not limited to, a text,
images, and/or animation delivered to the user, such that when a
specific object or text is touched by the user, the logo is felt. A
recognizable tactile logo could let the user know who is sponsoring
a certain web page, that a page is secure, or in another
application without taking up visual space on the screen. This may
be particularly valuable on a mobile phone or other device with
limited screen space. Customers could purchase tactile logos for
inclusion on personal or company web pages, applications, and/or
the like.
[0072] Tactical representations may be encoded similar to black and
white digital photographs. For example, a tactical representation
may be represented by x and y coordinates with amplitude or depth
information encoded at each point. Each point may be called a tixel
(tactile image element). The number of tixels and the range of
possible representations (e.g., bits) for amplitude or depth
information may define the resolution of a tactile representation.
Each surface of an object may be represented by a tactile image or
code. With a library of such images the surfaces of these objects
may be represented as a corresponding induced sensation to the
user. In another embodiment, thermal images that show temperature
fields or gradients can be represented as tixels and
representations (e.g., bits) for temperature or temperature
gradients may define the resolution of a tactile thermal
representation.
[0073] In many embodiments, the user interface may be a flat
surface on top of which cutaneous sensations are created. In such
embodiments, the surface textural information and an object's shape
and compliance may be conveyed to the user. In another embodiment,
an object's temperature and temperature gradient information may be
conveyed to the user. In embodiments in which a user's hand,
finger, or other portions of the body are free to move in three
dimensions, the surface information of a three-dimensional object
may be conveyed.
[0074] A library of cutaneous tactile sensations and effects may be
collected and stored in a control program or a separate memory
location. Additionally, custom or combinations of sensations may be
created. Sensations can be derived from a combination of pre-formed
sensations dynamically calculated in software or stored for
subsequent retrieval and use. Sensations may be based on empirical
data, based on physiological testing, algorithmic data, and/or
derived from initial calibration data. In one embodiment,
algorithms used to determine sensations may account for variables,
including but not limited to reflectance, temperature, finger
speed, finger pressure, and tixel data to appropriately deliver the
desired sensation(s).
[0075] In various embodiments, a controller or control system may
be implemented as any combination of hardware, firmware, and/or
software. For example, a controller may be implemented as a
field-programmable gate array (FPGA). In some embodiments, an
electronic controller may be distinct from other components of the
system for inducing sensations using transcutaneously focused
electromagnetic radiation. The system may include microprocessors
and other electronic components associated with displays, touch
screens, data storage, data connectivity, memory, non-transitory
computer readable media, etc.
[0076] Some of the infrastructure that can be used with embodiments
disclosed herein is already available, such as general-purpose
computers, computer programming tools and techniques, digital
storage media, and communication networks. A computing device or
other electronic controller may include a processor, such as a
microprocessor, a microcontroller, logic circuitry, and/or the
like. The processor may include a special-purpose processing device
such as application-specific integrated circuits (ASIC),
programmable array logic (PAL), programmable logic array (PLA), a
programmable logic device (PLD), FPGA, or another customizable
and/or programmable device. The computing device may also include a
machine-readable storage device, such as non-volatile memory,
static RAM, dynamic RAM, ROM, CD-ROM, disk, tape, magnetic storage,
optical storage, flash memory, or another machine-readable storage
medium. Various aspects of certain embodiments may be implemented
using hardware, software, firmware, or a combination thereof.
[0077] The embodiments of the disclosure may be understood with
reference to the drawings, wherein like parts are designated by
like numerals throughout. The components of the disclosed
embodiments, as generally described and illustrated, could be
arranged and designed in a wide variety of different
configurations. Furthermore, the features, structures, and
operations associated with one embodiment may be applicable to or
combined with the features, structures, or operations described in
conjunction with another embodiment. In other instances, well-known
structures, materials, or operations are not shown or described in
detail to avoid obscuring aspects of this disclosure.
[0078] Thus, the following detailed description of the embodiments
of the systems and methods of the disclosure is not intended to
limit the scope of the disclosure, as claimed, but is merely
representative of possible embodiments.
[0079] FIG. 1 illustrates a block diagram of a system 100 for
exciting tissue using electromagnetic radiation from a light source
130, according to one embodiment.
[0080] According to various embodiments, a system 100 may include a
control program, 110, a power source 120, a light source 130, one
or more optical components 140 for transcutaneously focusing
electromagnetic radiation from the light source 130 on excitable
tissue 150, and a feedback system 160.
[0081] As illustrated, the light source 130, or other
electromagnetic radiation source, may generate pulses of light at
appropriate energies and duration to stimulate excitable tissues,
mechanoreceptors, and/or innervating afferent axons. According to
various embodiments, the pulse duration of the light source 130 may
be in the range from 1 .mu.s to 500 ms and stimulation frequency
may be in the range from 0 Hz to 1000 Hz. Other pulse ranges and/or
frequency ranges capable of stimulating excitable tissues may be
utilized. In some embodiments, one or more optical components 140
may be used to focus the light on or within the excitable tissue
150. The optical components 140, in conjunction with the light
source 130, may be configured to minimize radiation exposure of
non-excitable tissue and/or avoid excessive heating of the
excitable tissue 150. Light emitted by the light source 130 may,
directly or indirectly, excite action potentials to induce
sensations corresponding to tactile sensations, as interpreted by
the central nervous system.
[0082] The feedback system 160 may measure skin temperature,
pressure from the user on the system, user movement relative to the
system, and/or to determine effectiveness of various incident
energies and points of stimulation. The feedback system may provide
information to the control program for dynamically adjusting the
inducement of cutaneous sensations via the transcutaneously focused
electromagnetic radiation emitted by the light source 130. The
control program 110 may be implemented in hardware, firmware,
and/or software. The control program may communicate with and/or
control the feedback system 160, the optical components 140, the
light source 130, and/or the power source 120.
[0083] FIG. 2 illustrates a simplified embodiment of a system 200
for delivering electromagnetic radiation 220 focused 245 on a
finger 210 of a user. As illustrated, a wide beam of
electromagnetic radiation 220 is focused by a lens 230 such that
the focused electromagnetic radiation converges at or below the
epidermal surface of the user's finger 210. Only at the focus point
240, is the radiation fluence sufficiently high to stimulate,
directly or indirectly, an action potential within the finger 210.
The electromagnetic radiation may diverge 255 and/or be
absorbed/scattered after stimulating the action potential within
the finger 210.
[0084] According to various embodiments, units of energy may be
expressed in terms of fluence or Joules per square centimeter. In
various embodiments, the electromagnetic radiation 220 used to
excite the action potential may be between 1 mJ/cm.sup.2 and 100
J/cm.sup.2. For example, the energy of individual pulses may be
between approximately 0.1 and 25 J/cm.sup.2. Outside of the focus
point 240, the fluence may be sub-threshold for action potential
initiation and of lower fluence, resulting in less tissue heating.
In some embodiments, an actuator may mechanically rotate, move,
vibrate, and/or otherwise direct the electromagnetic radiation 220.
In another embodiment, the electromagnetic radiation 220 and/or the
lens 230 may rotate, move and/or vibrate using beam steering
capabilities due to mirrors, spatial light modulators, or other
optical and/or electrical methods. The actuator may control the
procession of the electromagnetic beam to mitigate collateral
tissue heating.
[0085] FIG. 3 illustrates a simplified embodiment of a system 300
for delivering multiple electromagnetic radiation beams 320 and 330
transcutaneously coincident, at 340, on a finger of a user. Two
beams 320 and 330 are shown in the illustrated embodiment, but any
number of beams may be used. Each of the beams 320 and 330 may have
insufficient energy densities to excite tissue and, thus minimize
the energy imparted to non-excitable tissue. The point of
coincidence 340 may include the combined energy densities of each
of the beams 320 and 330 of electromagnetic radiation. Thus, at the
point of coincidence 340, the energy density may be sufficient to
initiate an action potential. In some embodiments, the size of the
focus may be adapted to create different sensory effects. In some
embodiments, the size of the focal spot may also be dynamically
adjusted.
[0086] FIG. 4 illustrates a display 430 associated with a system
420 for inducing haptic sensations via transcutaneously focused
electromagnetic radiation. In various embodiments, the system 420
may be in communication with the display 430. Accordingly, a
portion 415 of the user 410 within the system 420 may receive
cutaneous sensations induced by transcutaneously focused
electromagnetic radiation. As illustrated, the system 420 may be a
hand enclosure configured to receive a hand of a user. In such an
embodiment, a user may receive cutaneous sensation associated with
images, objects, icons, or the like on the display 430. In some
embodiments, the finger or hand of the user may be immobilized. In
other embodiments, the finger or hand of the user may move within
the system 420 and/or be able to provide responses to the received
cutaneous sensations induced by the system 420. The power supply,
light source, lensing system, and feedback systems may be all
housed in the single enclosure. According to other embodiments,
various components may be housed in multiple enclosures. Further, a
user's finger or hand could be suspended at a distance above a
stimulating surface rather than coming into direct contact with a
stimulating surface.
[0087] FIG. 5A illustrates a display screen 520 configured with a
system to induce cutaneous sensations in a user's finger 515 using
transcutaneously delivered electromagnetic radiation. The
illustrated embodiment is an example of an on-display
configuration. Display 520 could be part of a mobile device, such
as a smartphone or tablet computer, a stationary device such as a
desktop computer, interactive public display, industrial control
station, surgical control station, and/or other interactive display
device. In the illustrated embodiment, the finger 515 of a user 510
comes into contact with the display 520, upon which the image is
displayed. In some embodiments, the optical energy may be delivered
through the front of the display.
[0088] In one embodiment, a system for inducing cutaneous
sensations via transcutaneously focused electromagnetic radiation
may be in the form of a flat surface adjacent to or opposite a
display surface. For example, on a mobile phone or a tablet
computer, the system could be integrated into a flat surface that
is beside, beneath, and/or on the sides of a display surface. Such
an embodiment may allow for tactile interaction with the content
displayed without obscuring any portion of the visual display. A
light source according to any of the various embodiments described
herein may utilize various types of lasers, VCSELs, LEDs, and/or
other high-density focusable light sources.
[0089] FIG. 5B illustrates an accessory component 560 configured to
induce cutaneous sensations in a user's finger 515 via
transcutaneously focused electromagnetic radiation while using a
display 525. In the illustrated embodiment, the electromagnetic
radiation may be transmitted through the accessory device 560
(illustrated as a finger sleeve) into the finger 515 of the user
510. The electromagnetic radiation may originate from a remote
source and be transmitted via a fiber optic cable 565 to the
accessory component 560. In some embodiments, the accessory
component 560 may secure the finger 515 suspended away from the
walls thereof to avoid mechanical stimulation due to physical
contact with external objects, such as the display 525.
[0090] Optical components for focusing the electromagnetic
radiation and/or feedback sensors and components may be
incorporated into the accessory component 560. In some embodiments,
an interaction mechanism between the external wall of the accessory
component 560 and the display 525 may allow the user to interact
with the virtual or distant object shown on the display 525 and
experience the tactile sensations in a natural manner. For example,
the interaction mechanism may utilize a laser distance finder,
capacitive touch screen, an image sensor, a camera, a 3D or depth
camera, and/or an ultrasound echolocation system.
[0091] FIG. 5C illustrates a conceptual representation of an
electromagnetic radiation delivery system 500 including a single
electromagnetic radiation source 574 that may be incorporated into
a finger sleeve or other device, according to certain embodiments.
System 500 may include a switch controller 572 coupled to an
optical switch 570. Electromagnetic radiation source 574 may be
coupled to optical switch 570. A plurality of fiber optic cables
582 may be bundled in a cable 580.
[0092] An enlarged view of a distal end 578 of cable 580 shows the
plurality of fiber optic cables 582. Optical switch 570 may
selectively direct electromagnetic radiation generated from
electromagnetic radiation source 574 to any one of the plurality of
fiber optic cables 582.
[0093] FIG. 5D illustrates a conceptual representation of an
electromagnetic radiation delivery system 590 including a plurality
of electromagnetic radiation sources 574a-574d that may be
incorporated into a finger sleeve or other device, according to
certain embodiments. System 590 may include a number of components
that are similar to system 500, and accordingly, similar reference
numbers are utilized. System 590 also includes a fiber combiner
576. System 590 may utilize a plurality of electromagnetic
radiation sources 574a-574d in order to realize an increase in
power output, a decrease in the cost of the system or other
potential advantages.
[0094] FIG. 6A illustrates an embodiment of a stage 615 with a
moveable electromagnetic radiation source 617 for rastering
electromagnetic radiation to transcutaneously excite tissue. A
system, according to any of the various embodiments described
herein, may utilize a moveable electromagnetic radiation source 617
to control where the electromagnetic radiation is transcutaneously
focused. In some embodiments the stage 615 contains the
electromagnetic radiation source 617 and optics, while in others it
includes only the optics or only the electromagnetic radiation
source 617. In some embodiments, the stage 615 may be behind a
display 610 (or other user interface component such as a track pad
or dedicated haptic feedback surface), while in other embodiments
the stage 615 may be in front of the display 610. In the latter
embodiment, the stage 615 may be substantially transparent to
visible light, such that the display 610 is not or minimally
impeded. The mechanism for moving the stage 615 may be mechanically
and/or electromagnetically controlled.
[0095] For example, as illustrated in FIG. 6B, the stage 650 may
contain lightweight permanent magnets 652 that would be attracted
to or repelled from certain areas by a grid or array of
controllable electromagnets 655. These electromagnets 655 can
reside either on the rear of the display 640 or along a frame
around the display 640, so as to not occlude the display 640 for
the user. In an embodiment where the electromagnetic radiation
source is not incorporated into the stage 650 itself, the source
660 may be in the same plane as the stage, but off to the side of
the field. The electromagnetic radiation may be directed from the
source to the stage 650 where it is reflected and/or refracted by
optical components and focused into the intended tissue.
[0096] The ability to raster the electromagnetic radiation beam may
be useful for stimulating multiple points when simulating
mechanical stimuli. Any of a wide variety of rasterizing systems,
methods, and patterns may be used. For example, a galvanometer
based scanner or beam splitter with shuttering technologies may be
utilized. Microelectromechanical system (MEMS) based reflection and
direction of the beam may be used to rasterize the beam. In one
embodiment, the tip of one or more laser fibers with external
optical lens(es) (i.e., separate from the delivery fibers) may be
attached to the stage 650. The laser fiber and external optics may
be collectively called the laser head. The plan of the assembly and
stage 650 may be parallel to the plane on which a finger, fingers,
hand, or other portion of the body is to be stimulated. The stage
650 may be moveable in order to reach all areas of the finger or
full hand.
[0097] A controller may move the stage 650 based position
information in the x-y direction. The position of the stage 650 may
be determined by encoders or position sensors 665 on each axis of
the stage 650 and/or near the edges of the display 640. The
controller may also periodically move the beam, such that the
procession irradiates a certain location on the finger pad/hand for
only a certain amount of time. The procession of the beam can be in
multiple patterns, all contained within the area smaller than
two-point discrimination, as described above. FIGS. 7A-7C
illustrate three example procession patterns that may be used
within a two-point discrimination region to reduce thermal buildup
on the tissue. FIGS. 7A-7C may also be examples of rasterization
patterns used to induce various cutaneous sensations at desired
locations. In such embodiments, the various points in the patterns
illustrated in FIGS. 7A-7C may be separated by more than two-point
discrimination.
[0098] In some embodiments, the same tissue location may be
irradiated repeatedly using different light source parameters, such
as, but not limited to, pulse widths, frequencies, energies and/or
waveforms for each pulse or series of pulses. The control system
may track the energies and exposure time delivered to each spot on
the tissue and adjust the pulses based on the feedback data to
induce the appropriate sensations and avoid injury. If the tissue
is moved relative to the light source, there may be a need to
increase the power delivered to the previously unexposed tissues.
Tracking tissue placement relative to the area of irradiation may
allow the control program to deliver the appropriate power to the
tissue to induce the desired sensation.
[0099] FIG. 8 illustrates an electro-optical system 800 for
inducing cutaneous sensations, including an electrical
sub-threshold-inducing device 820 and a system for transcutaneously
focusing electromagnetic radiation according to any of the various
embodiments described herein (not shown). As illustrated,
electrodes 830 and 835 leading from an electrical stimulator 820
may be attached to a finger 815 of user 810. The electrodes 830 and
835 may be attached away from the area to be optically stimulated
to avoid mechanical stimulation. The sub-threshold electrical
stimulation may utilize any of a wide variety of waveform shapes,
such as square 825, or another waveform, such as sinusoidal,
triangular, trapezoidal, monopolar, and/or bipolar. The electrical
stimulation may be sub-threshold for all or most sensory
modalities. It may be used to reduce the activation threshold
necessary to induce cutaneous sensations using transcutaneously
focused electromagnetic radiation. If the transmembrane potential
of the mechanoreceptor, its afferent axon, or other excitable cells
are raised closer to the action potential threshold, then less
electromagnetic radiation may be required to directly or indirectly
initiate the action potential.
[0100] FIG. 9 illustrates a schematic 900 of a user calibration
procedure, according to one embodiment. The energies required to
transcutaneously induce a cutaneous sensation using electromagnetic
radiation may differ for each user. For example, the pigmentation
and other intrinsic characteristics, such as the finger print
pattern or skin optical properties, of each user's skin can be
different. Accordingly, in some embodiments a controller may
perform an initial calibration for each user and/or use.
[0101] A calibration procedure may include imparting energies that
should initially be sub-threshold, followed by successively higher
energy levels. The user may respond by indicating to the control
program whether or not a sensation was felt. FIG. 9 illustrates
progressively higher energy levels as peaks 910, 920, 930, and 940,
followed by troughs 915, 925, 935, and 945, respectively. In some
embodiments, the control program may deliver the next, higher
energy stimuli only after receiving some response or after a time
period has passed during which a response would have been expected.
A calibration procedure may also be used to determine the range of
fluences that may be used (e.g., the lowest energy that can be felt
and the highest energy density that won't cause harm or be
uncomfortable). The calibration procedure may be used for various
cutaneous sensations, such as tactile and temperature/heating
sensations.
[0102] According to some embodiments, a calibration procedure may
be based on continuously delivering groups of pulses that step up
to higher energy levels after predetermined periods of time. Each
group of pulses may include a priming pulse that precedes a train
of identical pulses. The priming pulse may reduce sensation latency
at the new energy level.
[0103] Part of the calibration procedure may account for feedback
variables, such as skin temperature, skin tone, incident pressure
on stimulation surface, finger speed, and duration of previous
exposure. In some instances, when using multiple fingers it is
possible that different fingers would have different calibration
results. In such cases, the program or controller may keep track of
the fingers individually, delivering the appropriate energies to
each finger (or other region of the body).
[0104] FIG. 10 illustrates a block diagram of a system 1000 for
inducing cutaneous sensations via transcutaneously focused
electromagnetic radiation, including a thermal feedback system
1050. As illustrated, an electromagnetic radiation source and/or
optics 1040 may impart energy via a beam 1020 to tissue 1015. The
energy may be the stimulus for direct or indirect excitation of
mechanoreceptors and/or neural or other excitable tissue. Output
from the electromagnetic radiation source may be adjusted based on
tissue temperature to deliver the appropriate energy to create
sensation. At certain wavelengths the byproduct of the light beam
incident on the tissues may be thermal energy buildup (heat). A
thermal feedback system 1050 may measure the radiation from the
tissues to determine the temperature of the tissues. If the tissues
become too hot, then the controller 1030 may lower the intensity of
the light beam output by the system.
[0105] Any type of temperature sensor or detector, such as a
thermistor, may be used to determine the temperature of the finger.
The feedback system 1050 may be in physical contact with the tissue
1015. In another embodiment, the thermal feedback system 1050 may
be a non-contact sensor. A sensor may be placed to the periphery of
the surface so as to not distort or impede the passage of the
stimulating light. Sensors may be integrated into the surface or
display and/or made from materials transparent to the necessary
wavelengths of stimulating light and, in the case of a visible
display, visible light. As with the thermal imager above, the
temperature data may feed the algorithm(s) that adjust the
stimulation output appropriately.
[0106] The temperature of the stimulating surface may be
controlled. For example, in an embodiment where the tissue is in
contact with a surface through which stimulation passes. Such an
embodiment may not have a temperature feedback system. The surface
temperature could be actively heated (or cooled) through any number
of mechanisms including, but not limited to, embedded electric
heating filaments, thermoelectric heat pump, IR radiation, or
directing the heat from other processes such as the computer or
graphics processor.
[0107] FIG. 11 illustrates a system 1100 for transcutaneously
inducing cutaneous sensations integrated within a peripheral device
1153 of a computer. A sensation-inducing system according to any of
the various embodiments described herein may be incorporated into
any of a wide variety of peripheral control devices, such as the
illustrated computer mouse 1153. The finger 1110 used to control
the mouse 1153 may have a finger pad 1115 on a control surface
1152. A system configured to induce cutaneous sensations using
transcutaneously focused electromagnetic radiation may be
integrated with the control surface 1152. A pressure sensitive
feedback mechanism may be used to modulate the imparted stimulation
in proportion to the pressure exerted by the user. In another
embodiment, the finger pad 1115 may be held away from any physical
surface similar to the concept described in conjunction with FIG.
5B.
[0108] FIGS. 12A-C illustrates three embodiments for incorporating
a system for inducing cutaneous sensations via transcutaneously
focused electromagnetic radiation within a surface 1220. As
illustrated in FIG. 12A an electromagnetic radiation source 1250
may be positioned beneath a surface 1220, such as a touch pad,
track pad, or display. For example, the surface 1220 may be part of
an off-display embodiment where there is no visual information
conveyed, or it could be a visible display that is interactive both
for vision and touch. In an embodiment in which the surface 1220 is
a display, the display could be any number of different display
types including, but not limited to, LCD, LED, OLED, AMOLED, e-ink,
array of controllable mirrors, or digital micromirror device. The
display may be substantially transparent to the electromagnetic
radiation used for stimulation.
[0109] Alternatively, the electromagnetic radiation from the source
1250 may be transmitted via channels or vias in the surface 1220.
FIG. 12A illustrates an embodiment in which a source 1250 is
directly beneath the surface 1220. FIG. 12B illustrates an
embodiment in which electromagnetic radiation from the source 1250
is directed by an optical scanner 1255 through the surface 1220 and
onto/into a finger 1210. FIG. 12C illustrates an embodiment in
which electromagnetic radiation from a source 1250 is directed
through an optical scanner 1255 to a series of reflective mirrors
1260 with angled surfaces 1265 in order to reflect the
electromagnetic radiation onto/into the finger 1210 of a user. The
optical scanner 1255 may be configured to irradiate multiple points
on the finger to induce cutaneous sensations.
[0110] FIG. 13 illustrates an example of a display 1300 with an
integrated system 1320 for inducing cutaneous sensations via
transcutaneously focused electromagnetic radiation. As illustrated,
a display surface 1310 may include a series of LEDs or other
visible light sources 1327 clustered on the display as pixels 1325.
Electromagnetic radiation sources for inducing cutaneous sensations
via transcutaneously focused electromagnetic radiation may be
integrated within the display surface. In some embodiments, the
electromagnetic radiation sources may be formed on the same
substrate. The electromagnetic radiation sources may include LEDs,
laser diodes, IR light sources, VSCELs, and/or other suitable
sources. The proximity of the sensation inducing sources to the
finger of a user may reduce the power requirements required to
induce sensations. A digitizer or other technology may be used to
sense the location and area of contact of a user's finger (or
fingers, hand, portion of the body, etc.). Limiting stimulating
emissions to those areas where the target tissue is touching the
screen may minimize energy waste.
[0111] It may be desirable to minimize or eliminate
sensation-inducing electromagnetic radiation in any location except
for a desired location and area of contact (i.e., where a finger
is). Certain wavelengths of electromagnetic radiation may cause
damage to the eye at high intensities, such as the corneal surface,
lens, or retina. FIG. 14A illustrates a schematic of a relatively
thin fluid layer 1445 on a surface 1440 configured to prevent stray
electromagnetic radiation emissions. The fluid layer 1445 may
absorb, reflect, or refract sensation-inducing electromagnetic
radiation and prevent it from negatively impacting a user. FIG. 14B
illustrates a finger 1410 depressing the relatively thin fluid
layer 1445 with a finger pad 1415. The depression 1450 may vacate a
sufficient amount of the fluid to allow the sensation-inducing
electromagnetic radiation to penetrate the finger pad 1415. The
fluid layer 1445 may allow visible light to pass with minimal
attenuation and optical distortion.
[0112] FIGS. 15A-15C illustrate an embodiment of a display 1510
with an integrated system for inducing cutaneous sensations via
transcutaneously focused electromagnetic radiation, utilizing an
array of lenslets and VCSELs 1513. As illustrated in FIG. 15A, a
touch screen display 1510 may include one or more functional layers
1511, 1512, and 1513 manufactured as a single physical layer or as
discrete physical layers. A touch sensitive layer 1511 may be
configured to receive touch inputs from a finger or fingers of a
user. A transmissive or reflective Spatial Light Modulator (SLM)
Layer 1512 may be configured to modulate visible light and/or other
electromagnetic radiation. A third layer 1513 may include an
N.times.N array of lenslets with integrated VCSELs. Each lenslet
may be a discrete lens aligned with a VCSEL in order to collimate
the upward emitting VCSEL output.
[0113] As illustrated in FIG. 15B, the collimated electromagnetic
radiation may then traverse the transmissive spatial light
modulator layer 1512 where the wave front 1550 could be arbitrarily
modulated prior to being transmitted through the touch sensitive
layer 1511. The electromagnetic radiation from the VCSELs may be
used to induce cutaneous sensations by transcutaneously focusing
the electromagnetic radiation on a user's finger(s) 1510. In
various embodiments, the electromagnetic radiation may be
transmitted through the display/touch screen layer(s) 1511. For
instance, if the wavelength of the sensation-inducing
electromagnetic radiation is about 1300 nm, the electromagnetic
radiation may be transmitted with minimal loss through silicon.
Furthermore, in various embodiments, the location of the finger
1510 on the touch screen 1511 may be detected by the touch screen
and used by the controller of the system for inducing cutaneous
sensations via transcutaneously focused electromagnetic
radiation.
[0114] In various embodiments, the SLM layer 1512 may be used to
optimize focusing on the surface or inside human tissue through
close-loop feedback control. For example, the wave front of the
electromagnetic radiation may be modulated in a systematic fashion.
After each alteration of the wave front, a user may provide
feedback based on the strength of the tactile sensation.
Convergence to an optimum wave front may be achievable after a
number of iterations during an initial calibration phase. An
adaptive focusing scheme may be used to increase focal intensity by
several folds. The focused/modulated electromagnetic radiation may
be scanned over the finger by imposing a dynamically changing phase
grating pattern using the SLM layer 1512. SLM based optical
scanning has the advantage of size and speed, eliminating the need
for mechanical scanner and the inertia associated with it.
[0115] Additionally, as illustrated in FIG. 15C, the SLM layer
1512, may impose a grating or dot pattern 1555 for scanning
stimulation spots and rasterization schemas. The grating 1555 may
be formed as a distinct layer in addition to the SLM layer 1512, or
in place of the SLM layer 1512. Additionally, it may be possible to
create a variety of static patterns using diffractive optical
elements and/or selectively activating pixels of the SLM.
[0116] The above description provides numerous specific details for
a thorough understanding of the embodiments described herein.
However, those of skill in the art will recognize that one or more
of the specific details may be omitted, modified, and/or replaced
by a similar process or system.
* * * * *