U.S. patent application number 17/549860 was filed with the patent office on 2022-04-07 for flexible touch sensing system and method with deformable material.
The applicant listed for this patent is Purdue Research Foundation. Invention is credited to Karthik Ramani, Sang Ho Yoon.
Application Number | 20220107709 17/549860 |
Document ID | / |
Family ID | 1000006028717 |
Filed Date | 2022-04-07 |
United States Patent
Application |
20220107709 |
Kind Code |
A1 |
Yoon; Sang Ho ; et
al. |
April 7, 2022 |
FLEXIBLE TOUCH SENSING SYSTEM AND METHOD WITH DEFORMABLE
MATERIAL
Abstract
A sensing system includes a stretchable base material, a
plurality of electrodes, a capacitive sensing channel and a
controller. The stretchable base material has a resistance
distribution that changes in response to being mechanically
deformed as a result of a human body contact. The base material has
a rebound elasticity. The electrodes are attached to a perimeter of
the base material, the capacitive sensing channel is attached to
the base material. The controller is operatively connected to the
plurality of electrodes and the capacitive sensing channel. The
controller is configured to measure instantaneous voltage
measurements from the plurality of electrodes, and determine
whether the base material is mechanically deformed based on the
instantaneous voltage measurements using a support vector machine
classifier.
Inventors: |
Yoon; Sang Ho; (West
Lafayette, IN) ; Ramani; Karthik; (West Lafayette,
IN) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
Purdue Research Foundation |
West Lafayette |
IN |
US |
|
|
Family ID: |
1000006028717 |
Appl. No.: |
17/549860 |
Filed: |
December 13, 2021 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
16544891 |
Aug 19, 2019 |
11199936 |
|
|
17549860 |
|
|
|
|
62719540 |
Aug 17, 2018 |
|
|
|
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G06F 3/0416 20130101;
G06F 3/0443 20190501; G06F 3/0447 20190501 |
International
Class: |
G06F 3/044 20060101
G06F003/044; G06F 3/041 20060101 G06F003/041 |
Claims
1. A sensing system, comprising: a stretchable base material having
a resistance distribution that changes in response to being
mechanically deformed as a result of a human body contact, the base
material having a rebound elasticity; a plurality of electrodes
attached to a perimeter of the base material; a capacitive sensing
channel attached to the base material; and a controller operatively
connected to the plurality of electrodes and the capacitive sensing
channel, the controller configured to, measure instantaneous
voltage measurements from the plurality of electrodes; determine
whether the base material is mechanically deformed based on the
instantaneous voltage measurements using a support vector machine
classifier.
2. The sensing system of claim 1, wherein the controller is further
configured to: while the base material is not mechanically
deformed, (i) determine voltage differences between the
instantaneous voltage measurements and a homogenous baseline
voltage and (ii) determine a location of the human body contact
using electrical impedance tomography (EIT).
3. The sensing system of claim 2, wherein the controller is further
configured to: determine the homogenous baseline voltage based on
instantaneous voltage measurements from the plurality of electrodes
while the base material is not mechanically deformed.
4. The sensing system of claim 2, wherein the controller is further
configured to: begin adaptively updating a baseline EIT measurement
when the human body contact with the base material begins; and stop
updating the baseline EIT measurement when the human body contact
with the base material ends.
5. The sensing system of claim 4, wherein the controller is further
configured to: measure capacitance measurements from the capacitive
sensing channel; and determine a beginning and ending of the human
body contact based on the capacitance measurements.
6. The sensing system of claim 1, wherein the base material
comprises a carbon filled elastomer.
7. The sensing system of claim 1, wherein the controller utilizes a
neighboring method to measure the instantaneous voltage
measurements from the plurality of electrodes, said neighboring
method comprising: a) measuring a first voltage differential
between a first adjacent pair of electrodes in the plurality of
electrodes; b) measuring a second voltage differential between a
second adjacent pair of electrodes in the plurality of electrodes,
the first and second adjacent pairs of electrodes having a common
electrode; and c) continuing to measure voltage differentials
between successive neighboring pairs of the electrodes until all
adjacent pairs of the electrodes have been evaluated for their
voltage differential.
8. The sensing system of claim 1, wherein the controller is further
configured to: determine an image reconstruction of the location
and shape of the human body contact using electrical impedance
tomography; and apply a color filter to the image reconstruction to
localize a contact coordinate of the location of the human body
contact.
9. The sensing system of claim 1, wherein the base material is
imprinted with graphics to indicate control buttons of a device
user control interface.
10. The sensing system of claim 1, wherein the plurality of
electrodes are evenly spaced along the perimeter of the base
material.
11. The sensing system of claim 1, further comprising: a current
source connected between the controller and the plurality of
electrodes.
12. The sensing system of claim 1, further comprising: an amplifier
connected in a return path from the plurality of electrodes to the
controller.
13. The sensing system of claim 1, the controller being further
configured to: while the base material is mechanically deformed,
determine a deformation level using a support vector machine
regression.
Description
CROSS REFERENCE TO RELATED APPLICATION
[0001] This application is a continuation of U.S. patent
application Ser. No. 16/544,891, filed Aug. 19, 2019, which in turn
claims priority to U.S. Provisional Patent Application No.
62/719,540 filed Aug. 17, 2018, both of which are hereby
incorporated by reference in their entireties.
BACKGROUND
[0002] Stretchable soft sensors have been explored as promising
input methods for adding interactions on both rigid and elastic
physical objects, smart textiles, shape-changing surfaces,
humanoids, and the human body. With a high flexibility and
stretchability of the sensors, a wide scope of natural applications
have been suggested. Still, the expensive and multi-step
fabrication processes hinder production of inexpensive, customized
soft sensors. Moreover, such sensors often cannot maintain
localization of a sensing point when the material is deformed
during a touch interaction. Therefore, improvements are needed in
the field.
SUMMARY
[0003] By jointly emphasizing fabrication, multi-modality and novel
computational methods, the present disclosure provides a
single-volume soft-matter sensor that provides multimodal sensing
and is able to support and restore contact localization upon and
after deformation of the sensing material. The presently disclosed
sensor and associated methods allow users to fabricate sensors
inexpensively, customize interfaces easily, and deploy them
instantly for continuous touch input.
[0004] In certain embodiments, the presently disclosed device
utilizes carbon-filled liquid silicone rubber, a non-toxic
piezoresistive elastomer material. The major hurdle in employing
the carbon-filled silicone as an interaction input is the lack of
real time sensing capability. This is mainly due to a rebound
elasticity of the material, which causes a slow-recovery of the
sensing signals after the material deformations that occur during
an input event. In the present disclosure, an adaptive
multi-sensing process is implemented using an electrical impedence
tomography (EIT) process to achieve real-time contact localization
and a learning-based support vector machine (SVM) to achieve
deformation awareness. The disclosed system is therefore able to
update contact localization in the presence of deformation of the
sensing material.
[0005] By employing the EIT and SVM technique, the presently
disclosed system enables a human touch to interface and interact
with the sensor via electrodes placed on the material boundary
only. In this way, the sensor can be fabricated in a single-volume
manner and implemented without invasive wirings or electronics or
other elements which have to be fabricated and placed in the
interior of the material boundary. No interior elements are
required, instead the material itself is used as a sensor. Using
the disclosed method provides sensing contact localization and
stretching within the sensor material. To this end, users are
allowed to perform interactions instantly after deployment without
any extra training processes.
[0006] According to various aspects, a system is provided,
comprising a single volume soft sensor capable of sensing real-time
continuous contact and stretching. A low-cost and an easy way to
fabricate such piezoresistive elastomer-based soft sensors for
instant interactions is also provided. An electrical impedance
tomography (EIT) technique with SVM learning is employed to
estimate changes of resistance distribution on the sensor caused by
fingertip contact and determine contact localization even during a
material deformation event. The EIT image reconstruction is
processed with a difference in resistance measurement (.delta.V)
which the difference between an instant measurement reading
(V.sub.i) and a homogeneous baseline reading (V.sub.H). A
deformation switch value is determined to maintain and restore the
contact localization during and after the deformation event. When
deformation occurs, the most recent .delta.V before the deformation
event occurred is maintained and used during the deformation event.
Upon release from the deformation, we updated the homogeneous
baseline using .delta.V, where V.sub.H=V.sub.i-.delta.V. Using the
presently disclosed method, the contact localization can be
maintained during the deformation and restored after the
deformation as shown in FIG. 2B.
[0007] This summary is provided to introduce the selection of
concepts in a form that is easy to understand the detailed
embodiments of the descriptions. The embodiments are then brought
together in a final embodiment which described an environment,
thereby stressing that each of the embodiments may be viewed in
isolation, but also the synergies among them are very significant.
This summary is not intended to identify key subject matter or key
features or essential features thereof.
BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The above and other objects, features, and advantages of
various examples will become more apparent when taken in
conjunction with the following description and drawings wherein
identical reference numerals have been used, where possible, to
designate identical features that are common to the figures, and
wherein:
[0009] FIG. 1A illustrates a change in sensor output values without
dynamic manipulation upon a deformation event.
[0010] FIG. 1B illustrates a change in sensor output values with
dynamic manipulation upon a deformation event.
[0011] FIG. 2A illustrates a contact localization output for a
stretching event without a deformation switch value
incorporated.
[0012] FIG. 2B illustrates a contact localization output for a
stretching event with a deformation switch value incorporated.
[0013] FIG. 3A illustrates a sensor activating the initial sensing
electrode pair.
[0014] FIG. 3B illustrates a sensor activating a subsequent sensing
electrode pair.
[0015] FIG. 4 is a process flowchart illustrating a touch sensing
process according to one embodiment.
[0016] FIG. 5 is a process flowchart illustrating a baseline update
process according to one embodiment.
[0017] FIG. 6 shows a schematic view of a 16-electrode system.
[0018] FIG. 7 illustrates a high-level diagram showing the
components of a sensing system.
DETAILED DESCRIPTION
[0019] The term "drawings" used herein refers to drawings attached
herewith and to sketches, drawings, illustrations, photographs, or
other visual representations found in this disclosure. The terms
"I," "we," "our" and the like throughout this disclosure do not
refer to any specific individual or group of individuals.
[0020] Sensing performed by the presently disclosed system is based
on an EIT technique with an SVM learning process which estimates
the resistance distribution of the conductive material using
inverse problem analysis based on measurements from the sensor
boundary. The difficulty of providing real-time sensing with
carbon-filled silicone rubber is due to the material's rebound
elasticity (>50%), which causes a long settling time (>10 s)
and small shifts in baseline values as shown in FIGS. 1A and 1B.
FIG. 1A illustrates a change in sensor output values without
dynamic manipulation upon a deformation event. FIG. 1B illustrates
a change in sensor output values with dynamic manipulation upon a
deformation event.
[0021] The presently disclosed sensing method is based on
carbon-filled liquid silicone rubber that changes its resistance
distribution upon mechanical deformations. In one example,
four-terminal sensing is used to measure resistance since this
method reduces the inaccuracy from contact resistances. Unlike
matrix tactile sensors where arrays of electrodes are required
within the sensing area, the presently disclosed system utilizes
sensing electrodes 204 and a capacitive channel 206 coupled to the
outer edge of the sensor 202. Then, a Neighboring Method is used
where DC current is fed through two adjacent electrodes 202 and the
voltage differential is measured successively throughout the
adjacent electrode pairs as shown in FIGS. 3A and 3B. FIG. 3A shows
the initial sensing electrode pair and FIG. 3B shows the next
successive electrode pair sensing.
[0022] According to one embodiment, EIT image reconstruction is
carried out by comparing the measurements at two different
instances. The update method comprises the following steps as shown
in FIG. 4:
[0023] Raw sensor readings from sensors 204 are fed into EIT
channel 404 and SVM channel 406.
[0024] Before feeding the sensor values to the SVM classifier
(block 408), differential dynamic manipulation (block 410) is
applied when the sensor settlement enters the quasi-steady state,
i.e., dV.sub.avg<dV.sub.avg,threshold when
V.sub.avg<V.sub.avg,threshold.
[0025] The deformation type is classified and assigned a
deformation identifier (block 413) using SVM with polynomial kernel
(block 412).
[0026] If there is "No Deformation," (block 414) the presence of
deformation is confirmed in the previous frame (block 416).
[0027] If the deformation event exists in the previous frame,
V.sub.H=V.sub.i-.delta.V is set to update the homogeneous baseline
(block 418). Otherwise, EIT localization (block 420) is processed
using the baseline process 500 of FIG. 5 to update .delta.V (block
422) with the current Vi to localize a contact coordinate location
in the sensor 202 (block 424).
[0028] If any deformation is detected, multiple channels are
activated: 1) .delta.V from the most recent localization during "No
Deformation" is used (block 426) and a contact coordinate is
outputted and 2) the system determines the level of the
corresponding deformation (block 430) using SVM regression (block
428) with a polynomial kernel, with the regressed values mapped to
corresponding deformation (block 432).
[0029] FIG. 5 shows a flowchart 500 which illustrates a baseline
update process. First, if a contact is not detected, i.e., the
capacitive sensing value cap, is less than a predetermined
threshold cap.sub.threshold (stage 502), instant measurement
readings (Vi) are set as a homogeneous baseline data (V.sub.H)
(stage 504). If cap.sub.i>=cap.sub.threshold, a movement
detection is evaluated (stage 506). If div Vavg,i>=div Vavg
threshold, the system sets the previous frame's data (Vi-1) as VH
(stage 508) and proceeds to perform an image reconstruction using
EIT (stage 510). If div Vavg,i<div Vavg threshold, the system
directly proceeds to stage 410 and performs an image reconstruction
using EIT. The system may optionally apply a color filter to the
reconstructed image for blob detection and localize a contact
coordinate from the center of the blob (stage 512) before
outputting the contact position (x, y).
[0030] Throughout this description, some aspects are described in
terms that would ordinarily be implemented as software programs.
Those skilled in the art will readily recognize that the equivalent
of such software can also be constructed in hardware, firmware, or
micro-code. Because data-manipulation algorithms and systems are
well known, the present description is directed in particular to
algorithms and systems forming part of, or cooperating more
directly with, systems and methods described herein. Other aspects
of such algorithms and systems, and hardware or software for
producing and otherwise processing signals or data involved
therewith, not specifically shown or described herein, are selected
from such systems, algorithms, components, and elements known in
the art. Given the systems and methods as described herein,
software not specifically shown, suggested, or described herein
that is useful for implementation of any aspect is conventional and
within the ordinary skill in such arts.
[0031] FIG. 7 is a high-level diagram showing the components of the
exemplary system 1000 for analyzing the EIT location data and
performing other analyses described herein, and related components.
The system 1000 includes a processor 1086, a peripheral system
1020, a user interface system 1030, and a data storage system 1040.
The peripheral system 1020, the user interface system 1030 and the
data storage system 1040 are communicatively connected to the
processor 1086. Processor 1086 can be communicatively connected to
network 1050 (shown in phantom), e.g., the Internet or a leased
line, as discussed below. The EIT data may be received using sensor
202 (via electrodes 204) and/or displayed using display units
(included in user interface system 1030) which can each include one
or more of systems 1086, 1020, 1030, 1040, and can each connect to
one or more network(s) 1050. Processor 1086, and other processing
devices described herein, can each include one or more
microprocessors, microcontrollers, field-programmable gate arrays
(FPGAs), application-specific integrated circuits (ASICs),
programmable logic devices (PLDs), programmable logic arrays
(PLAs), programmable array logic devices (PALs), or digital signal
processors (DSPs).
[0032] Processor 1086 can implement processes of various aspects
described herein. Processor 1086 can be or include one or more
device(s) for automatically operating on data, e.g., a central
processing unit (CPU), microcontroller (MCU), desktop computer,
laptop computer, mainframe computer, personal digital assistant,
digital camera, cellular phone, smartphone, or any other device for
processing data, managing data, or handling data, whether
implemented with electrical, magnetic, optical, biological
components, or otherwise. Processor 1086 can include
Harvard-architecture components, modified-Harvard-architecture
components, or Von-Neumann-architecture components.
[0033] The phrase "communicatively connected" includes any type of
connection, wired or wireless, for communicating data between
devices or processors. These devices or processors can be located
in physical proximity or not. For example, subsystems such as
peripheral system 1020, user interface system 1030, and data
storage system 1040 are shown separately from the data processing
system 1086 but can be stored completely or partially within the
data processing system 1086.
[0034] The peripheral system 1020 can include one or more devices
configured to provide digital content records to the processor
1086. For example, the peripheral system 1020 can include digital
still cameras, digital video cameras, cellular phones, or other
data processors. The processor 1086, upon receipt of digital
content records from a device in the peripheral system 1020, can
store such digital content records in the data storage system
1040.
[0035] The user interface system 1030 can include a mouse, a
keyboard, another computer (connected, e.g., via a network or a
null-modem cable), or any device or combination of devices from
which data is input to the processor 1086. The user interface
system 1030 also can include a display device, a
processor-accessible memory, or any device or combination of
devices to which data is output by the processor 1086. The user
interface system 1030 and the data storage system 1040 can share a
processor-accessible memory.
[0036] In various aspects, processor 1086 includes or is connected
to communication interface 1015 that is coupled via network link
1016 (shown in phantom) to network 1050. For example, communication
interface 1015 can include an integrated services digital network
(ISDN) terminal adapter or a modem to communicate data via a
telephone line; a network interface to communicate data via a
local-area network (LAN), e.g., an Ethernet LAN, or wide-area
network (WAN); or a radio to communicate data via a wireless link,
e.g., WiFi or GSM. Communication interface 1015 sends and receives
electrical, electromagnetic or optical signals that carry digital
or analog data streams representing various types of information
across network link 1016 to network 1050. Network link 1016 can be
connected to network 1050 via a switch, gateway, hub, router, or
other networking device.
[0037] Processor 1086 can send messages and receive data, including
program code, through network 1050, network link 1016 and
communication interface 1015. For example, a server can store
requested code for an application program (e.g., a JAVA applet) on
a tangible non-volatile computer-readable storage medium to which
it is connected. The server can retrieve the code from the medium
and transmit it through network 1050 to communication interface
1015. The received code can be executed by processor 1086 as it is
received, or stored in data storage system 1040 for later
execution.
[0038] Data storage system 1040 can include or be communicatively
connected with one or more processor-accessible memories configured
to store information. The memories can be, e.g., within a chassis
or as parts of a distributed system. The phrase
"processor-accessible memory" is intended to include any data
storage device to or from which processor 1086 can transfer data
(using appropriate components of peripheral system 1020), whether
volatile or nonvolatile; removable or fixed; electronic, magnetic,
optical, chemical, mechanical, or otherwise. Exemplary
processor-accessible memories include but are not limited to:
registers, floppy disks, hard disks, tapes, bar codes, Compact
Discs, DVDs, read-only memories (ROM), erasable programmable
read-only memories (EPROM, EEPROM, or Flash), and random-access
memories (RAMs). One of the processor-accessible memories in the
data storage system 1040 can be a tangible non-transitory
computer-readable storage medium, i.e., a non-transitory device or
article of manufacture that participates in storing instructions
that can be provided to processor 1086 for execution.
[0039] In an example, data storage system 1040 includes code memory
1041, e.g., a RAM, and disk 1043, e.g., a tangible
computer-readable rotational storage device such as a hard drive.
Computer program instructions are read into code memory 1041 from
disk 1043. Processor 1086 then executes one or more sequences of
the computer program instructions loaded into code memory 1041, as
a result performing process steps described herein. In this way,
processor 1086 carries out a computer implemented process. For
example, steps of methods described herein, blocks of the flowchart
illustrations or block diagrams herein, and combinations of those,
can be implemented by computer program instructions. Code memory
1041 can also store data, or can store only code.
[0040] Various aspects described herein may be embodied as systems
or methods. Accordingly, various aspects herein may take the form
of an entirely hardware aspect, an entirely software aspect
(including firmware, resident software, micro-code, etc.), or an
aspect combining software and hardware aspects These aspects can
all generally be referred to herein as a "service," "circuit,"
"circuitry," "module," or "system."
[0041] Furthermore, various aspects herein may be embodied as
computer program products including computer readable program code
stored on a tangible non-transitory computer readable medium. Such
a medium can be manufactured as is conventional for such articles,
e.g., by pressing a CD-ROM. The program code includes computer
program instructions that can be loaded into processor 1086 (and
possibly also other processors), to cause functions, acts, or
operational steps of various aspects herein to be performed by the
processor 1086 (or other processor). Computer program code for
carrying out operations for various aspects described herein may be
written in any combination of one or more programming language(s),
and can be loaded from disk 1043 into code memory 1041 for
execution. The program code may execute, e.g., entirely on
processor 1086, partly on processor 1086 and partly on a remote
computer connected to network 1050, or entirely on the remote
computer.
[0042] Those skilled in the art will recognize that numerous
modifications can be made to the specific implementations described
above. The implementations should not be limited to the particular
limitations described. Other implementations may be possible.
* * * * *