U.S. patent application number 16/297504 was filed with the patent office on 2020-09-10 for in-memory computation device with inter-page and intra-page data circuits.
This patent application is currently assigned to MACRONIX INTERNATIONAL CO., LTD.. The applicant listed for this patent is MACRONIX INTERNATIONAL CO., LTD.. Invention is credited to Chun-Hsiung HUNG, Shang-Chi YANG.
Application Number | 20200286553 16/297504 |
Document ID | / |
Family ID | 1000003987610 |
Filed Date | 2020-09-10 |
![](/patent/app/20200286553/US20200286553A1-20200910-D00000.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00001.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00002.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00003.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00004.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00005.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00006.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00007.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00008.png)
![](/patent/app/20200286553/US20200286553A1-20200910-D00009.png)
![](/patent/app/20200286553/US20200286553A1-20200910-M00001.png)
View All Diagrams
United States Patent
Application |
20200286553 |
Kind Code |
A1 |
HUNG; Chun-Hsiung ; et
al. |
September 10, 2020 |
IN-MEMORY COMPUTATION DEVICE WITH INTER-PAGE AND INTRA-PAGE DATA
CIRCUITS
Abstract
An in-memory computation device is described that comprises a
memory with a plurality of blocks B(n) of cells, where n ranges
from 0 to N-1. A page output circuit PO(n) and page input circuit
PI(n) are operatively coupled to block B(n) in the plurality of
sets. A data bus system for providing an external source of input
data and a destination for output data is provided. Data circuits
are configurable connect page input circuit PI(n) to one or more of
page output circuit PO(n), page output circuit PO(n-1), and the
data bus system to source the page input data in a sensing cycle.
This configuration can be done between each sensing cycle, or in
longer intervals, in order to support a variety of neural network
configurations and operations.
Inventors: |
HUNG; Chun-Hsiung; (HSINCHU,
TW) ; YANG; Shang-Chi; (Changhua, TW) |
|
Applicant: |
Name |
City |
State |
Country |
Type |
MACRONIX INTERNATIONAL CO., LTD. |
HSINCHU |
|
TW |
|
|
Assignee: |
MACRONIX INTERNATIONAL CO.,
LTD.
HSINCHU
TW
|
Family ID: |
1000003987610 |
Appl. No.: |
16/297504 |
Filed: |
March 8, 2019 |
Current U.S.
Class: |
1/1 |
Current CPC
Class: |
G11C 13/0026 20130101;
G06N 3/08 20130101; G11C 13/0028 20130101; G11C 13/004 20130101;
G06N 3/0635 20130101 |
International
Class: |
G11C 13/00 20060101
G11C013/00 |
Claims
1. An in-memory computation device, comprising: a memory including
a plurality of blocks B(n), where n ranges from 0 to N-1; a
plurality of page output circuits PO(n), where n ranges from 0 to
N-1, connected to the memory; a plurality of page input circuits
PI(n), where n ranges from 0 to N-1, connected to the memory and
responsive to a page input data to apply a bias to the memory in a
sensing cycle; the plurality of page output circuits, the plurality
of page input circuits and the plurality of blocks being
operatively coupled, so that a page output circuit PO(n) in the
plurality of page output circuits and a page input circuit PI(n) in
the plurality of page input circuits being operatively coupled to a
block B(n) in the plurality of blocks, where n ranges from 0 to
N-1; a data bus system; and data circuits configurable connect the
page input circuit PI(n) to one or more of the page output circuit
PO(n), the page output circuit PO(n-1), and the data bus system to
select a source for the page input data in the sensing cycle.
2. The device of claim 1, including word line drivers coupled to
the plurality of blocks in the memory to apply signals on word
lines in the blocks, and wherein the page output circuit PO(n) in
the plurality of page output circuits includes circuits that
generate an output indicating a sum-of-signals on a set of bit
lines in block B(n), responsive to a signal on a selected word
line.
3. The device of claim 2, wherein the page input circuit PI(n)
includes bias circuitry responsive to the input data for biasing
bit lines in the block B(n), wherein the signals on the bit lines
represent a product of the input data and thresholds of memory
cells coupled to the selected word line.
4. The device of claim 1, wherein the page input circuit PI(n)
includes bias circuitry responsive to the input data for biasing
bit lines in the block B(n).
5. The device of claim 1, wherein the page input circuit PI(n)
includes an input register connected to the data circuits, to store
an input vector VI(n) including the page input data, and circuits
to generate bias signals for bit lines in the block B(n) in
response to the input vector VI(n).
6. The device of claim 5, wherein the block B(n) includes a number
Z of bit lines, the input vector VI(n) includes Z chunks of M bits,
and the page input circuit PI(n) includes digital-to-analog
conversion circuits to convert chunks of M bits in the input vector
VI(n) to Z analog bias voltages for corresponding bit lines in the
block B(n).
7. The device of claim 6, wherein the memory cells in block B(n) on
a selected word line store coefficient vector W(n), and the signals
on the bit lines in the block B(n) represent a product of chunks in
the input vector VI(n) and coefficients in the coefficient vector
W(n) for the selected word line.
8. The device of claim 1, wherein the page input circuit PI(n)
includes an input register connected to the data circuits, to store
an input vector VI(n) including the page input data, and circuits
to generate bias signals for bit lines in the block B(n) in
response to the input vector VI(n), and page output circuit PO(n)
in the plurality of page output circuits generates an output vector
VO(n).
9. The device of claim 8, wherein the input vector VI(n) includes Z
chunks of M bits, and the page output circuit PO(n) in the
plurality of page output circuits includes sensing circuits to
generate an output including one or more chunks of M bits
indicating a sum-of-signals on the bit lines in block B(n) in
response to a word line signal on a selected word line, and an
output register to store all or part of output vector VO(n)
including the Z chunks of M bits.
10. The device of claim 1, wherein the data circuits transfer all
or part of output vector VO(n), including Z chunks of M bits as an
input vector VI(n+1) to a page input circuit PI(n+1).
11. The device of claim 1, wherein the memory comprises nonvolatile
memory cells.
12. The device of claim 11, wherein the nonvolatile memory cells
are charge trapping memory cells.
13. The device of claim 1, including word line drivers coupled to
the plurality of blocks in the memory to apply signals on word
lines in the blocks, and wherein the page output circuit PO(n) in
the plurality of page output circuits includes circuits that
generate an output indicating a sum-of-signals on one or more bit
lines in block B(n), responsive to a signal or signals on one or
more selected word lines; and wherein page input circuit PI(n)
includes bias circuitry responsive to the input data for biasing
word lines in the block B(n), wherein the signals on the bit lines
represent a product of the input data and thresholds of memory
cells coupled to the one or more selected word lines.
14. An in-memory computation device, comprising: a memory including
a plurality of blocks B(n), where n ranges from 0 to N-1; a
plurality of page output circuits connected to the memory, a page
output circuit PO(n) in the plurality of page output circuits
generating a page output vector VO(n), and being operatively
coupled to a block B(n) in the plurality of blocks; a plurality of
page input circuits connected to the memory, a page input circuit
PI(n) in the plurality of page input circuits receiving a page
input vector VI(n), and generating input voltages in response to
the page input vector VI(n), and being operatively coupled to the
block B(n); a plurality of bit line bias circuits connected to the
plurality of blocks, a bit line bias circuit Y(n) in the plurality
of bit line bias circuits being operatively coupled to the block
B(n) in the plurality of blocks, and to the page input circuit
PI(n) in the plurality of page input circuits, and biasing the bit
lines in the block B(n) in response to input voltages generated by
the page input circuit PI(n); a data bus system; and data circuits
configurable connect the page input circuit PI(n) to one or more of
the page output circuit PO(n), a page output circuit PO(n-1), and
the data bus system to select a source for the page input vector
VI(n) in a sensing cycle.
15. The device of claim 14, including word line drivers coupled to
a plurality of word lines in the memory to apply signals on
selected word lines in the plurality of blocks, and wherein the
page output circuit PO(n) in the plurality of page output circuits
includes circuits that generate page output vector VO(n) indicating
a sum-of-signals on the bit lines in block B(n), responsive to a
signal on a selected word line.
16. The device of claim 15, wherein the signals on the bit lines in
the block B(n) represent a product of the page input vector VI(n)
and thresholds of memory cells coupled to the selected word
line.
17. The device of claim 16, wherein the memory comprises
nonvolatile memory cells.
18. The device of claim 17, wherein the bit line bias circuit Y(n)
in the plurality of bit line bias circuits comprises a bit line
clamp transistor having a gate terminal connected to receive
corresponding input voltages generated by the page input circuit
PI(n).
19. The device of claim 14, wherein the page input vector VI(n)
includes a plurality of multi-bit chunks of data, each chunk in the
plurality corresponding to one bit line in the block B(n).
20. An in-memory computation device, comprising: a memory including
a plurality of blocks B(n), where n ranges from 0 to N-1; a
plurality of page output circuits connected to the memory, a page
output circuit PO(n) in the plurality of page output circuits
generating a page output vector VO(n), and being operatively
coupled to a block B(n) in the plurality of blocks; a plurality of
page input circuits connected to the memory, a page input circuit
PI(n) in the plurality of page input circuits receiving a page
input vector VI(n), and generating input voltages in response to
the page input vector VI(n), and being operatively coupled to the
block B(n); a plurality of word line drivers connected to the
plurality of blocks, a word line driver WD(n) in the plurality of
word line drivers being operatively coupled to the block B(n) in
the plurality of blocks, and to the page input circuit PI(n) in the
plurality of page input circuits, and biasing the word lines in
block B(n) in response to input voltages generated by the page
input circuit PI(n); a data bus system; and data circuits
configurable connect page input circuit PI(n) to one or more of
page output circuit PO(n), page output circuit PO(n-1), and the
data bus system to select a source for the page input vector VI(n)
in a sensing cycle.
Description
BACKGROUND
Field
[0001] The present invention relates to in-memory computing
devices, and more particularly, to in-memory computing devices
supporting efficient data sharing among multiple computational
stages.
Description of Related Art
[0002] A neural network is an information processing paradigm that
is inspired by the way biological nervous systems process
information. With the availability of large training datasets and
sophisticated learning algorithms, neural networks have facilitated
major advances in numerous domains such as computer vision, speech
recognition, and natural language processing.
[0003] The basic unit of computation in a neural network is often
referred to as a neuron. A neuron receives inputs from other
neurons, or from an external source, performs an operation, and
provides an output. FIG. 1 illustrates an example neural network
100. The neural network 100 contains multiple neurons arranged in
layers, which can be considered to be an example of a computational
stage including many parallel operations. The neural network 100
includes an input layer 102 of input neurons (i.e., neurons that
provide the input data), three hidden layers 106, 108 and 110 of
hidden neurons (i.e., neurons that perform computations and
transfer information from the input neurons to the output neurons),
and an output layer 104 of output neurons (i.e., neurons that
provide the output data). Neurons in adjacent layers have synaptic
layers of connections between them. For example, the synaptic layer
112 connects neurons in the input layer 102 and the hidden layer
106, the synaptic layer 114 connects neurons in the hidden layers
106 and 108, the synaptic layer 116 connects neurons in the hidden
layers 108 and 110, and the synaptic layer 118 connects the neurons
in the hidden layer 110 and the output layer 104. All these
connections have weights associated with them. For example, the
neurons 122, 124 and 126 in the hidden layer 106 are connected to a
neuron 128 in the hidden layer 108 by connections with weights
w.sub.1 132, w.sub.2 134 and w.sub.3 136, respectively. The output
for the neuron 128 in the hidden layer 108 can be calculated as a
function of the inputs (x.sub.1, x.sub.2, and x.sub.3) from the
neurons 122, 124 and 126 in the hidden layer 106 and the weights
w.sub.1 132, w.sub.2 134 and w.sub.3 136 in the connections. The
function can be expressed as follows:
f ( x i ) = i = 1 M w i X i ##EQU00001##
[0004] In the sum-of-products expression above, each product term
is a product of a variable input x.sub.1 and a weight w.sub.1. The
weight w.sub.1 can vary among the terms, corresponding for example
to coefficients of the variable inputs x.sub.1. Similarly, outputs
from the other neurons in the hidden layer can also be calculated.
The outputs of the two neurons in the hidden layer 110 act as
inputs to the output neuron in the output layer 104.
[0005] Neural networks are used to learn patterns that best
represent a large set of data. The hidden layers closer to the
input layer in general learn high level generic patterns, and the
hidden layers closer to the output layer in general learn more
data-specific patterns. Training is a phase in which a neural
network learns from training data. During training, the connections
in the synaptic layers are assigned weights based on the results of
the training session. Inference is a stage in which a trained
neural network is used to infer/predict using input data and to
produce output data based on the prediction.
[0006] In-memory computing is an approach in which memory cells,
organized in an in-memory computing device, can be used for both
data processing and memory storage. A neural network can be
implemented using an in-memory computing device for a number of
synaptic layers. The weights for the sum-of-products function can
be stored in memory cells of the in-memory computing device. The
sum-of-products function can be realized as a circuit operation in
the in-memory computing device in which the electrical
characteristics of the memory cells of the array effectuate the
function.
[0007] An engineering issue associated with neural networks relates
to movement of data among the synaptic layers. In some embodiments,
there can be thousands of neurons in each layer, and the routing of
the outputs of the neurons to the inputs of other neurons can be a
time consuming aspect of the execution of the neural network.
[0008] It is desirable to provide in-memory neural network
technology that supports efficient movement of data among the
computational components of the system.
SUMMARY
[0009] An in-memory computation device is described that comprises
a memory configured in a plurality of blocks useable for in-memory
computations. Blocks B(n), n going from 0 to N-1, in the plurality
of blocks have corresponding page input circuits PI(n) and page
output circuits PO(n) that are operatively coupled to sets of bit
lines in the blocks. For example, each block B(n) can include a set
S(n) of bit lines coupled to its corresponding page output circuit
and to its corresponding page input circuit. The device includes in
some embodiments a data bus system for providing an external source
of input data and a destination for output data. Data circuits are
configurable to connect page input circuit PI(n) to one or more of
page output circuit PO(n), page output circuit PO(n-1), and the
data bus system to source input data for a sensing cycle. This
configuration can be done between each sensing cycle, or in longer
intervals, in order to support a variety of neural network
configurations and operations.
[0010] In a device described herein, a plurality of bit line bias
circuits is connected to bit lines in the plurality of blocks. Bit
line bias circuit Y(n) in the plurality of bit line bias circuits
being operatively coupled to block B(n), and to page input circuit
PI(n) in the plurality of page input circuits. The bit line biasing
circuit can bias the bit lines in block B(n) in response to input
voltages generated by page input circuit PI(n).
[0011] Other aspects and advantages of the present invention can be
seen on review of the drawings, the detailed description and the
claims, which follow.
BRIEF DESCRIPTION OF THE DRAWINGS
[0012] FIG. 1 illustrates an example neural network.
[0013] FIG. 2 illustrates a NOR style memory array configured for
in-memory computation of a sum-of-products suitable for use in a
neural network.
[0014] FIG. 3A is a simplified diagram of an in-memory computation
device with inter-page and intra-page data circuits.
[0015] FIG. 3B is a simplified diagram of an alternative in-memory
computation device with inter-page and intra-page data
circuits.
[0016] FIG. 4 is a simplified diagram of an in-memory computation
device using a NOR style memory array with drain side bias
generated in response to input data.
[0017] FIG. 5 is a diagram of an embodiment of page input circuitry
usable in a device like that of FIG. 4.
[0018] FIG. 6 is a diagram of page output circuitry usable in a
device like that of FIG. 4.
[0019] FIG. 7 is a diagram of a digital-to-analog converter usable
to generate drain side bias voltages for use in an embodiment like
that of FIG. 4.
[0020] FIG. 8 is a diagram of an alternative embodiment of page
output circuitry usable in a device like that of FIG. 4.
DETAILED DESCRIPTION
[0021] A detailed description of embodiments of the present
invention is provided with reference to the FIGS. 2-8.
[0022] FIG. 2 illustrates a memory array 210 including a plurality
of memory cells (e.g. 211, 212) configured in a NOR style. The
memory cells in a given column of the array are connected between a
bit line and a source line. In the illustrated example, there are
columns of memory cells between bit line BL0 and source line SL0,
between bit line BL1 and source line SL1, between bit line BL2 and
source line SL2, and between bit line BL3 and source line SL3. To
execute a sum-of-products operation in-memory, a coefficient vector
(a set of coefficients W(i)) is stored in the memory cells in a
given row, such as on the row of cells connected to word line WL0.
Input data (X0-X3) is applied by biasing the bit lines BL0 to BL3
connecting a bias voltage to the drain side of the memory cells. A
row of memory cells is selected by applying a word line signal to a
selected word line (e.g. WL0). Unselected word lines are biased in
an off condition. Current is generated in the cells in the selected
row that is a function of the coefficient W(i) stored in the
accessed memory cell, and the input data X(i). The signals
generated on the bit lines or source lines in response to the word
line signal on the selected word line can be combined (215) to
generate the sum-of-products output SUM by sensing a sum of the
signals.
[0023] The memory cells in the array of FIG. 2 can be nonvolatile
memory cells, such as floating gate memory cells or dielectric
charge trapping memory cells. Also, programmable resistance memory
cells can be utilized, such as phase change memory, metal oxide
memory and others. In some embodiments, the memory cells in the
array of FIG. 2 can be volatile memory cells. The memory cells in
the array of FIG. 2 can store one bit per cell in some examples. In
other examples, the memory cells in the array of FIG. 2 can store
multiple bits per cell. In yet other examples, the memory cells in
the array of FIG. 2 can store analog values.
[0024] The NOR style array can be implemented with many bit lines
and word lines, storing thousands or millions of bits. The NOR
style array of FIG. 2 can be implemented in a three-dimensional
configuration, with many two-dimensional arrays stacked in many
levels.
[0025] In-memory computation circuits as described herein can use
memory arrays of other styles in some examples, including for
example AND style arrays or NAND style arrays.
[0026] FIG. 3A is a simplified diagram of an in-memory computation
device that includes a memory 300 that comprises an array of memory
cells storing coefficients W(i), and including a plurality of word
lines and a plurality of bit lines. The memory is configured in a
plurality of blocks, B(0), B(1), and so on. Each block includes a
plurality of bit lines and a plurality of word lines. The word
lines can be shared among more than one block in some embodiments,
so that he word lines for more than one block are driven and
decoded together, or constitute a shared single word line conductor
across more than one block. In other embodiments, the word lines
are driven and decoded for individual blocks.
[0027] In this example, the blocks B(n) in the plurality of blocks
comprise corresponding YPASS circuits 301, 302, which combine the
signals on the set S(n) of bit lines in the block into an output
signal on a data line DL(n). The memory cells coupled to the bit
lines in the set S(n) of bit line store a coefficient vector W(n)
represented by the threshold voltages of the memory cells in a row
or rows selected by word line signals.
[0028] Also, the YPASS circuits include a plurality of bit line
bias circuits (not shown in FIG. 3A) connected to the plurality of
sets of bit lines. Bit line bias circuit Y(n) in the plurality of
bit line bias circuits is operatively coupled to set S(n) of bit
lines in the corresponding block B(n), and to page input circuit
PI(n) in the plurality of page input circuits. The bit line bias
circuit biases the bit lines in set S(n) in response to input
voltages generated by page input circuit PI(n). The input signals
X[1:z] on line 360 are input voltages in this example, applied to
YPASS circuit 301, and X[z+1:2z] on line 361 are applied to YPASS
circuit 302, to be applied to the corresponding set of bit lines
having "z" members. In general, there can be a large number of
blocks including YPASS circuits, each of which includes N sets S(n)
of bit lines, where a YPASS circuit in block (n) including set S(n)
bit lines receives input signals, X[n*z+1:(n+1)*z)], for n going
from 0 to N-1.
[0029] In the illustrated embodiment, two blocks, B(0) and B(1),
are illustrated. In general, there can be a plurality of blocks
B(n), where "n" ranges from 0 to N-1, and N can be any positive
integer more than 1. In some embodiments, the number N can be 8 for
example, or 16. In other embodiments, N can be in much higher.
[0030] The memory 300 can comprise a single two-dimensional array,
in which the blocks are arranged side-by-side in sequence. In other
embodiments, the memory 300 can comprise a three-dimensional array,
including a plurality of stacked two-dimensional arrays. In this
case, each two-dimensional array in the stack can comprise one
block. Blocks arranged in sequence can reside on sequential levels
of the stack. In other arrangements, each two-dimensional array in
the stack can comprise more than one block configured as
illustrated.
[0031] A data bus system, represented by the input data block 320
and the input data block 321, is provided on the device. The data
bus system can be used as an input/output interface for data from
an external data source, or from other circuitry on the device that
generates data for use as the input vectors in some configurations.
The input data block 320 and the input data block 321 can comprise,
for examples, a data cache memory, a register or a latch coupled to
input/output circuitry on the device or other data bus
circuits.
[0032] The output signals on lines DL0 and DL1 from the YPASS
circuits 301, 302 represent an in-memory computation result,
combining the signals on the bit lines in the corresponding block
B(0) or B(1), that are produced in response to weights stored in
memory cells selected by a word line signal on a selected word line
in the memory array, and to the input signals X[n*z+1:(n+1)*z)]
from lines 360, 361 for the corresponding block. Word line drivers
398 and decoders are included to provide the word line signals on
selected word lines. In the simplified example shown in FIG. 3A,
YPASS circuits 301, 302 are coupled to corresponding page input
circuits PI(0), PI(1) which provide input signals X[n*z+1:(n+1)*z)]
on lines 360 and 361, respectively. Also, YPASS circuits 301, 302
are coupled to corresponding page output circuits PO(0), PO(1) to
receive in-memory computation result on corresponding data lines
DL0 and DL1, respectively.
[0033] In this illustration, the page input circuits (PI(0) and
PI(1)) include respective input registers 314, 316 which store
input data in the form of input vectors VI(n) received as input
from data circuits on the device for the corresponding block B(n).
The page input circuits can include circuits that convert the input
vector VI(n), which have a number of bits equal to a multiple M of
the number Z of bit lines in the block, to the input signals. Also,
the page output circuits (PO(0) and PO(1)) include respective
output registers 315, 317 which store output vectors VO(n)
generated in response to an in-memory computation using memory
cells in the corresponding block B(n) for output to the data
circuits on the device. The output vectors VO(n) can have the same
number of bits as the input vectors VI(n), or a different number of
bits.
[0034] Data circuits on the device are configurable to interconnect
the input register for a given page (e.g., PI(1)) singly or in any
combination, with sources of input data, including the output
register of a previous page (e.g., PO(0)), the output register of
the same page (e.g., PO(1) as feedback) and the data bus to source
input data for a given sensing cycle. An input vector including the
input data applied to the input register in a given sensing cycle
can be sourced by a single source, or a combination of multiple
sources. The data circuits are configurable to transfer an output
vector VO(0) of page output circuit PO(0) as all or part an input
vector VI(1) to the next page input circuit PI(1) as represented by
the line 340. Also, data circuits are configurable to feed back the
output vector VO(0) of the page output circuit PO(0) as all or part
of an input vector VI(0) for the page input circuit PI(0), as
represented by line 350. In addition, the data circuits are
configurable to connect the page input circuit PI(0) to the data
bus system including input data block 320 as represented by line
330, to receive all or part of an input vector VI(0) from another
source in the bus system.
[0035] Likewise, the data circuits on the device are configurable
to transfer an output vector VO(1) of page output circuit PO(1) as
all or part of an input vector VI(2) for the next page input
circuit as represented by the line 341. Also, data circuits are
configurable to feedback the output VO(1) of the page output
circuit PO(1) as all or part of an input vector VI(1) for the page
input circuit PI(1), as represented by line 351. In addition, the
data circuits are configurable to connect the page input circuit
PI(1) to the data bus system including input data block 321 as
represented by line 331, to receive all or part of an input vector
VI(1) from an another source on the bus system.
[0036] Configuration circuits 399 are included on the device. The
circuits 399 include logic, configuration parameters storage, or
both. The configuration circuits 399 control the configuration of
the data circuits for the routing of input vectors and output
vectors among the page input and page output circuits. The
configuration of the data circuits can be set dynamically for each
sensing cycle as suits the needs of a particular implementation,
using timed control signals delivered to switches in the data
circuits. Alternatively, the configuration circuit can use volatile
or nonvolatile configuration registers to set up the data
circuits.
[0037] Control circuits, not shown, are coupled to the circuit of
FIG. 3A, and include logic to control execution of in-memory
computation operations, data circuit configuration operations in
coordination with configuration circuits 399, and memory operations
like program and erase. The control circuits can comprise command
decoders, configuration registers, state machines and timing
circuits set up to control the operations. The control circuits can
include voltage regulators, charge pumps and other bias circuits to
provide appropriate bias voltages in support of the memory
operations.
[0038] FIG. 3B is a simplified diagram of an in-memory computation
device that includes a memory 900 that comprises an array of memory
cells storing coefficients W(i), and including a plurality of word
lines and a plurality of bit lines. The memory is configured in a
plurality of blocks, B(0), B(1), and so on. Each block includes a
plurality of bit lines and a plurality of word lines. The word
lines are driven and decoded for individual blocks in this
example.
[0039] In this example, the blocks B(n) in the plurality of blocks
comprise corresponding YPASS circuits 901, 902, which combine the
signals on the set S(n) of bit lines in the block into an output
signal on a data line DL(n). The memory cells coupled to the bit
lines in the set S(n) of bit line store a coefficient vector W(n)
represented by the threshold voltages of the memory cells in a row
or rows selected by one or more word line signals.
[0040] Also, the word line drivers include a plurality of bias
circuits (not shown in FIG. 3B) connected to the plurality of sets
of word lines. Word line driver WD(n) in the plurality of word line
drivers is operatively coupled to a set of word lines in the
corresponding block B(n), and to page input circuit PI(n) in the
plurality of page input circuits. The word line bias circuit biases
the word lines in corresponding block in response to input voltages
generated by page input circuit PI(n). The input signals X[1:z] on
line 960 are input voltages in this example, applied to a word line
driver 938 for block B(0), and X[z+1:2z] on line 961 are applied to
a word line driver for block B(1), to be applied to the
corresponding set of word lines having "z" members.
[0041] In the illustrated embodiment, two blocks, B(0) and B(1),
are illustrated. In general, there can be a plurality of blocks
B(n), where "n" ranges from 0 to N-1, and N can be any positive
integer more than 1. In some embodiments, the number N can be 8 for
example, or 16. In other embodiments, N can be in much higher.
[0042] The memory 900 can comprise a single two-dimensional array,
in which the blocks are arranged side-by-side in sequence. In other
embodiments, the memory 900 can comprise a three-dimensional array,
including a plurality of stacked two-dimensional arrays. In this
case, each two-dimensional array in the stack can comprise one
block. Blocks arranged in sequence can reside on sequential levels
of the stack. In other arrangements, each two-dimensional array in
the stack can comprise more than one block configured as
illustrated.
[0043] A data bus system, represented by the input data block 920
and the input data block 921, is provided on the device. The data
bus system can be used as an input/output interface for data from
an external data source, or from other circuitry on the device that
generates data for use as the input vectors in some configurations.
The input data block 920 and the input data block 921 can comprise,
for examples, a data cache memory, a register or a latch coupled to
input/output circuitry on the device or other data bus
circuits.
[0044] The output signals on lines DL0 and DL1 from the YPASS
circuits 901, 902 represent an in-memory computation result,
combining the signals on the bit lines in the corresponding block
B(0) or B(1), that are produced in response to weights stored in
memory cells selected by one or more word line signals on a
selected word line in the memory array, and to the input signals
X[n*z+1:(n+1)*z)] from lines 960, 961 for the corresponding block.
In the simplified example shown in FIG. 3B, the word line drivers
(e.g. 938) for the blocks B(0) and B(1) are coupled to
corresponding page input circuits PI(0), PI(1) which provide input
signals X[n*z+1:(n+1)*z)] on lines 960 and 961, respectively. Thus,
word line driver WD(n) in the plurality of word line drivers is
operatively coupled to block B(n) in the plurality of blocks, and
to page input circuit PI(n) in the plurality of page input
circuits, and biases the word lines in block B(n) in response to
input voltages generated by page input circuit PI(n). Also, YPASS
circuits 901, 902 are coupled to corresponding page output circuits
PO(0), PO(1) to receive in-memory computation result on
corresponding data lines DL0 and DL1, respectively.
[0045] In this illustration, the page input circuits (PI(0) and
PI(1)) include respective input registers 914, 916 which store
input data in the form of input vectors VI(n) received as input
from data circuits on the device for the corresponding block B(n).
The page input circuits can include circuits that convert the input
vector VI(n), which have a number of bits equal to a multiple M of
the number Z of bit lines in the block, to the input signals. Also,
the page output circuits (PO(0) and PO(1)) include respective
output registers 915, 917 which store output vectors VO(n)
generated in response to an in-memory computation using memory
cells in the corresponding block B(n) for output to the data
circuits on the device. The output vectors VO(n) can have the same
number of bits as the input vectors VI(n), or a different number of
bits.
[0046] Data circuits on the device are configurable to interconnect
the input register for a given page (e.g., PI(1)) singly or in any
combination, with sources of input data, including the output
register of a previous page (e.g., PO(0)), the output register of
the same page (e.g., PO(1) as feedback) and the data bus to source
input data for a given sensing cycle. An input vector including the
input data applied to the input register in a given sensing cycle
can be sourced by a single source, or a combination of multiple
sources. The data circuits are configurable to transfer an output
vector VO(0) of page output circuit PO(0) as all or part an input
vector VI(1) to the next page input circuit PI(1) as represented by
the line 940. Also, data circuits are configurable to feed back the
output vector VO(0) of the page output circuit PO(0) as all or part
of an input vector VI(0) for the page input circuit PI(0), as
represented by line 950. In addition, the data circuits are
configurable to connect the page input circuit PI(0) to the data
bus system including input data block 920 as represented by line
930, to receive all or part of an input vector VI(0) from another
source in the bus system.
[0047] Likewise, the data circuits on the device are configurable
to transfer an output vector VO(1) of page output circuit PO(1) as
all or part of an input vector VI(2) for the next page input
circuit as represented by the line 941. Also, data circuits are
configurable to feedback the output VO(1) of the page output
circuit PO(1) as all or part of an input vector VI(1) for the page
input circuit PI(1), as represented by line 951. In addition, the
data circuits are configurable to connect the page input circuit
PI(1) to the data bus system including input data block 921 as
represented by line 931, to receive all or part of an input vector
VI(1) from an another source on the bus system.
[0048] Configuration circuits 999 are included on the device. The
circuits 999 include logic, configuration parameters storage, or
both. The configuration circuits 999 control the configuration of
the data circuits for the routing of input vectors and output
vectors among the page input and page output circuits. The
configuration of the data circuits can be set dynamically for each
sensing cycle as suits the needs of a particular implementation,
using timed control signals delivered to switches in the data
circuits. Alternatively, the configuration circuit can use volatile
or nonvolatile configuration registers to set up the data
circuits.
[0049] Control circuits, not shown, are coupled to the circuit of
FIG. 3B, and include logic to control execution of in-memory
computation operations, data circuit configuration operations in
coordination with configuration circuits 999, and memory operations
like program and erase. The control circuits can comprise command
decoders, configuration registers, state machines and timing
circuits set up to control the operations. The control circuits can
include voltage regulators, charge pumps and other bias circuits to
provide appropriate bias voltages in support of the memory
operations.
[0050] FIG. 4 is a diagram of an in-memory computation device like
that of FIG. 3A with information about embodiments of the YPASS
circuits 401, 402, the page input circuits PI(0), PI(1) and the
page output circuits PO(0), PO(1). The in-memory computation device
includes a memory array 410 along with supporting circuitry as
mentioned with respect to FIG. 3A. The memory array 410 includes a
plurality of blocks B(0), B(1), . . . A set S(n) of bit lines in
each block is coupled to a corresponding YPASS circuit 401,
402.
[0051] The YPASS circuit 401 includes a bit line bias circuit. In
this embodiment, the bit line bias circuit includes a plurality of
clamp transistors 490-493 having one source/drain terminal coupled
to a bit line in the block B(0) of the array 410, and another
source/drain terminal coupled to a summing node on the data line
DL0. The gates of the clamp transistors 490-493 are connected to
corresponding input signals in the set of input signals X[1:z]
provided by the page input circuit 419.
[0052] The YPASS circuit 402 includes a bit line bias circuit. In
this embodiment, the bit line bias circuit includes a plurality of
clamp transistors 494-497, each having one source/drain terminal
coupled to a bit line in the block B(1) of the array 410, and
another source/drain terminal coupled to a summing node on the data
line DL1. The gates of the clamp transistors 494-497 are connected
to corresponding input signals in the set of input signals
X[z+1:2z] provided by the page input circuit 459.
[0053] In the example shown in FIG. 4, the page input circuit 419
(PI(0)) includes an input register 420 and supporting logic 421,
which converts the input vector stored in the input register 420 to
the input signals X[1:z] applied to the gates of the clamp
transistors. Likewise, the page input circuit 459 (PI(1)) includes
an input register 460 and supporting logic 461, which converts the
input vector stored in the input register 460 to the input signals
X[z+1:2z] applied to the gates of the clamp transistors.
[0054] In the example shown in FIG. 4, the page output circuit 429
(PO(0)) includes an N-bit sense amplifier 430, or other type of
multilevel sensing circuit or analog-to-digital converter. The
output of the sense amplifier 430 is coupled to an output register
431 or other type of register. The output register 431 is connected
to a compute unit 432 which can accumulate or otherwise process
data stored in the output register 431. In combination, the
circuits in the page output circuit 429 convert the signal on the
DL0 line into an output vector VO(n) for the computation based on a
sum-of-signals generated by memory cells on a selected word line on
the block B(n). The sum-of-signals can correspond to a
sum-of-products.
[0055] Likewise, the page output circuit 469 (PO(1)) includes an
N-bit sense amplifier 470, or other type of multilevel sensing
circuit or analog-to-digital converter. The output of the sense
amplifier 470 is coupled to an output register 471 or other type of
register. The output register 471 in this example is connected to a
compute unit 472 which can accumulate or otherwise process data
stored in the output register 471. In combination, the circuits in
the page output circuit 469 convert the signal on the DL1 line into
an output vector VO(n) for the computation based on a
sum-of-signals generated by memory cells on a selected word line on
the block B(n).
[0056] In this example, the cell current in each memory cell on the
selected word line coupled to a bit line in the set of bit lines
can be represented in one example memory system by the
equation:
I C E L L .apprxeq. .mu. n * Cox * ( W / L ) * ( V G S - V T H -
CELL ) * V D S .apprxeq. .mu. n * Cox * ( W / L ) * ( V W L - V
TH_CELL ) * ( Xi - V TH_ypass ) ##EQU00002##
[0057] Data circuits on the device are configurable to transfer an
output vector VO(0) of page output circuit 429 as an input vector
VI(1) for the next page input circuit 459 as represented by the
line 443. Also, data circuits are configurable to feedback the
output VO(0) of the page output circuit 429 as an input vector
VI(0) for the page input circuit 419 as represented by line 442. In
addition, the data circuits are configurable to connect the page
input circuit 419 to the data bus system 440 as represented by line
441, to receive an input vector VI(0) from an another source on the
bus system.
[0058] Likewise, data circuits on the device are configurable to
transfer an output vector VO(1) of page output circuit 469 as an
input vector VI(1) for the next page input circuit as represented
by the line 483. Also, data circuits are configurable to feedback
the output VO(1) of the page output circuit 469 as an input vector
VI(1) for the page input circuit 459 as represented by line 482. In
addition, the data circuits are configurable to connect the page
input circuit 459 to the data bus system 480 as represented by line
481 to receive an input vector VI(1) from an another source on the
bus system.
[0059] FIG. 5 illustrates one example of a page input circuit 500,
representing page input circuit PI(n), which is configured to
deliver the input signals on bus 505 to the YPASS circuits of the
block B(n) in the memory array. Also, portions of the data circuits
which are configured to deliver the input vector VI(n) are
illustrated.
[0060] In this example, the page input circuit 500 includes an
input register 501 that comprises a plurality of latches
520-523.
[0061] Data circuits include a set of switches 510, 511, 512, 513
operable as a multiplexer in response to the signal SW. The
switches 510, 511, 512, 513 connect and disconnect the register 501
to the previous page output circuit which provides output vector
VO(n-1) on lines 508 of the data circuits for use as input vector
VI(n). Also, the data circuits include a multiplexer 551 and input
bus register 550. The multiplexer 551 is responsive to the signal
SW_IN to connect and disconnect the input data from the input bus
register 550 to deliver an input vector VI(n) to the input register
501. Also the data circuits include a bus 509 which is coupled to a
multiplexer (see, MUX 642 in FIG. 6), to receive the output vector
VO(n) as feedback from the page output circuit PO(n) coupled to the
same block B(n), for use as input vector VI(n). The signal SW and
the signal SW_IN are provided by a configuration circuit such as
configuration circuits 399 and configuration circuits 999 shown in
FIGS. 3A and 3B. The signals SW and SW_IN are provided
independently for the page input circuits and page output circuits
operatively coupled with each block B(n).
[0062] In this example, for an embodiment in which block B(n)
includes a number Z of bit lines, the input vector VI(n) includes Z
chunks of M bits of data. Input register 501 applies the Z chunks
of M bits of input vector VI(n) to conversion circuits 502 that
provide the bias voltages X5 to X8 in the Figure to establish the
drain level of the selected memory cells in the corresponding block
B(n). In this example, the circuits 502 include one M bit
digital-to-analog converter 530, 531, 532, 533 for each of the Z
chunks in the input vector VI(n) to generate Z analog bias voltages
for corresponding bit lines in the block B(n). In this example, M
is 2 and Z is 4 with an input vector including 8 bits. In other
embodiments, the input vector can include 16 bits, or any number of
bits. Also, the number of chunks in the input vector used to
generate a bias voltage for one of the corresponding bit lines in
the set of bit lines can be determined by the number Z of bit lines
in the set.
[0063] FIG. 6 illustrates one example of a page output circuit 600,
representing page output circuit PO(n) which is coupled to the
block B(n) and to the page input circuit PI(n). The page output
circuit 600 is configured to receive the signal on data line DL(n)
from the YPASS circuits coupled to the block B(n).
[0064] The data line DL(n) is connected to a sense amplifier 601 or
other type of sensing circuit or analog-to-digital converter. The
sense amplifier 601 in this example includes an adjustable current
source I1 which is connected to node 610, which is also connected
to data line DL(n). A precharge transistor 612 is coupled to a
sensing node SEN. Also a capacitor 611 is coupled to the sensing
node SEN. Transistor M2 is connected in series between node 610 and
the sensing node SEN. The sensing node SEN is connected to the gate
of transistor M1. Transistor M1 is connected between ground and
output node A. The output node A is coupled to a set of switches
responsive to respective switch signals SWQ1 to SWQ8 in this
example for an 8-bit output. The switches apply data signals Q1 to
Q8 to corresponding latches in the output register 620. The output
register 620 stores the output vector VO(n) for the stage. In
operation, the adjustable current source I1 is operated in
coordination with the switch signals SWQ1 to SWQ8 to sense a
plurality of levels of the signal on DL(n) and store the resulting
sensing results in the output register 620.
[0065] Data circuits are coupled to the output register 620 for
transferring the output vector VO(n) on lines 630 to the page input
circuit in the next stage as the input vector VI(n+1), or in
feedback on lines 631 as the input vector VI(n) for the page input
circuit PI(n) of the same stage through a multiplexer 642, which is
controlled by the signal SW_FB. The signal SW_FB is provided by a
configuration logic/store such as configuration circuits 399 and
configuration circuits 999 shown in FIGS. 3A and 3B, for page input
circuits and page output circuits operatively coupled with each
block B(n).
[0066] FIG. 7 illustrates an example digital-to-analog converter
700 which can be utilized in a page input circuit like that shown
in FIG. 5. In this example, a chunk of 2 bits D1 and D2 of the
input vector register 705 are applied as input to a 4-to-1
multiplexer 704. A reference voltage generating circuit includes an
operational amplifier 701 which receives a reference voltage on its
"+" input and a feedback signal on its "-" input. The output of the
operational amplifier 701 drives the gate of a PMOS transistor 702,
controlling current flow through a string of resistors 703. Nodes
between the resistors provide output voltage levels O1 to O4 as
inputs to the multiplexer 704. In response to the chunk of input
data, the reference voltage is applied to the "-" input of the
operational amplifier 710, the output of which drives the gate of
an NMOS transistor 711 connected between the supply potential and a
resistor 712. The resistor is connected to ground. The node on the
resistor 712 is applied as feedback to the "+" input of the
operational amplifier 710. The operational amplifier 710 in this
example is configured in a unity gain mode, and its output is
applied as one of the signals X[i] used to bias a bit line in the
block B(n).
[0067] FIG. 8 illustrates an alternative embodiment of a page
output circuit, representing page output circuit PO(n) which is
coupled to the block B(n) and to the page input circuit PI(n). The
page output circuit PO(n) is configured to receive the signal on
data line DL(n) from the YPASS circuits coupled to the block
B(n).
[0068] The data line DL(n) is connected to a sense amplifier 820 or
other type of sensing circuit or analog-to-digital converter. The
sense amplifier 820 in this example includes an adjustable current
source I1 which is connected to node 810, which is also connected
to data line DL(n). A precharge transistor 812 is coupled to a
sensing node SEN. Also a capacitor 811 is coupled to the sensing
node SEN. Transistor M2 is connected in series between node 810 and
the sensing node SEN. The sensing node SEN is connected to the gate
of transistor M1. Transistor M1 is connected between ground and
output node A. The output node A is coupled to a set of switches
including two members in this example, responsive to respective
switch signals SWQ1 to SWQ2 in this example for an 2-bit output.
The switches apply data signals Q1 to Q2 to corresponding latches
(e.g. 821) in the output register 802. The output register 802
stores a chunk of output vector VO(n) for the stage. In operation,
the adjustable current source I1 is operated in coordination with
the switch signals SWQ1 to SWQ2 to sense a plurality of levels of
the signal on DL(n) and store the resulting sensing results in the
output register 802 in each sensing cycle. For example, in each
sensing cycle, the sum of the signals from the set of memory cells
on one selected word line in the selected set of bit lines can be
converted to a 2-bit chunk.
[0069] The output of the output register 802 is applied to a layer
interface block 803. In this example, the output vector VO(n) can
be generated using bit lines on a plurality of layers of a 3D
memory. In this example, the interface block 803 provides the
chunks of data generated by the sense amplifier 800, which are
combined with corresponding chunks generated by interface blocks on
other layers of the 3D memory to form an output vector VO(n) having
Z chunks of M bits.
[0070] In this example, the page output circuit PO(n) is described
as comprising chunk-wide portions on a plurality of layers of a 3D
memory. The page input circuit PI(n) can be configured in a similar
way for connection to a block B(n) coupled to memory cells on the
plurality of layers of the 3D memory.
[0071] In other embodiments, the interface block 803 can be used to
accumulate additional chunks of the output vector in a sequence of
sensing cycles using memory cells on a set of word lines in a
single layer or single 2D array.
[0072] Data circuits are coupled to the interface block 803 for
transferring the output vector VO(n) on lines 830 to the page input
circuit in the next stage as the input vector VI(n+1), or in
feedback on lines 831 as the input vector VI(n) for the page input
circuit PI(n) of the same stage through a multiplexer 842, which is
controlled by the signal SW_FB. The signal SW_FB is provided by a
configuration circuit such as configuration circuits 399 and
configuration circuits 999 shown in FIGS. 3A and 3B, for page input
circuits and page output circuits operatively coupled with each
block B(n).
[0073] An in-memory computing structure is described in which a
sum-of-products result latched in a page output circuit can
feedback as an input vector for the page input circuit in the same
block, or pass as an input vector to the page input circuit in the
next page. Also, the page input vector in each block can be coupled
to a bus system that provides input vectors from other sources on
the bus. Thus, the input data can be latched from outside the
in-memory computing device, from a feedback loop of the previous
state output from the same block, or from the output data in a
previous block. With this scheme, a large bandwidth with very
little routing capacitance can be achieved. Connection between two
blocks can be very short, so that the previous layer or previous
page output can be delivered to the next layer, or next block, fast
and with low power. The structure can be applied to artificial
intelligence processors use for both training and inference steps
in deep learning.
[0074] While the present invention is disclosed by reference to the
preferred embodiments and examples detailed above, it is to be
understood that these examples are intended in an illustrative
rather than in a limiting sense. It is contemplated that
modifications and combinations will readily occur to those skilled
in the art, which modifications and combinations will be within the
spirit of the invention and the scope of the following claims.
* * * * *