350 likes | 358 Vues
A Dynamic Analog Concurrently-Processed Adaptive Chip. Malcolm Stagg Grade 11. A Dynamic Analog Concurrently-Processed Adaptive Chip. Purpose: To design a reconfigurable analog neural network on a chip A chip that can learn over time For any application neural networks are used
E N D
A Dynamic Analog Concurrently-Processed Adaptive Chip Malcolm Stagg Grade 11
A Dynamic Analog Concurrently-Processed Adaptive Chip • Purpose: • To design a reconfigurable analog neural network on a chip • A chip that can learn over time • For any application neural networks are used • Improvement over previous designs: • High density • High routability • On-chip learning • Multiple learning algorithms
A Dynamic Analog Concurrently-Processed Adaptive Chip • Neuron and synapse circuits are created • For TSMC 0.35µm CMOS (Complementary Metal Oxide Semiconductor) implementation • Dense layout of transistors for analog arithmetic circuits • High Density • Neurons and Synapses are as small as possible without compromising performance • Supports both Backpropagation and Hebbian learning on each cell
A Dynamic Analog Concurrently-Processed Adaptive Chip • Neural networks: • Have been used for years in many applications, from business and finance to science and engineering • Ideally implemented as arrays of electronic cells • Many learning algorithms have been designed • The first electronic neural network was made in 1951, by Marvin Minsky (called “SNARC”) • Carver Mead pioneered chip implementation of neural cells, designing a retina-like neural circuit • Intel created a popular analog neural network chip in 1989 (called “ETANN”)
A Dynamic Analog Concurrently-Processed Adaptive Chip • Backpropagation: • Feedback-based learning algorithm (supervised learning) • Requires calculation of the correct output for an input pattern • Will learn to represent the correct output over time • Hebbian: • Non-feedback-based learning algorithm (unsupervised learning) • Calculates synapse weights based on correlation between input patterns • Will learn to identify and classify input patterns • Hopfield: • Advanced memory recall algorithm, using Hebbian learning
in out + - BP BP Hebb BPout Hebb BPin A Dynamic Analog Concurrently-Processed Adaptive Chip Neuron Block Diagram
A Dynamic Analog Concurrently-Processed Adaptive Chip Neuron Cell Diagram
A Dynamic Analog Concurrently-Processed Adaptive Chip • The neuron design: • Supports Backpropagation and Hebbian • Uses an SRAM cell to enable/disable algorithm-dependent circuits • Uses tanh sigmoid circuit for forwards-propagation (range: [-1, 1]) • Derivative as sech2 for backwards propagation (range: [0, 1])
in out BP Weight Update Parameters Hebb BPout BP BPin A Dynamic Analog Concurrently-Processed Adaptive Chip Synapse Block Diagram
A Dynamic Analog Concurrently-Processed Adaptive Chip Synapse Cell Diagram
A Dynamic Analog Concurrently-Processed Adaptive Chip • The synapse design: • Supports Backpropagation and Hebbian • Uses an SRAM cell to enable/disable algorithm-dependent circuits • Uses the Gilbert multiplier cell for forwards- and backwards- propagation • All inputs are ensured to be symmetrical (c+x, c-x) • Differential pair-based circuits are used • Input range is small for good linearity
A Dynamic Analog Concurrently-Processed Adaptive Chip • High routability • Multiple routing pathways between neurons and synapses • Using programmable wires and connections • Enables use of advanced learning algorithms • To evolve/change the layout of the network over time • This allows the network to become more dense and more efficient, while maintaining a low synapse count
SRAM SRAM Analog Switch Analog Switch A Dynamic Analog Concurrently-Processed Adaptive Chip Programmable Wire Programmable Connection
Neuron Synapse Synapse Synapse A Dynamic Analog Concurrently-Processed Adaptive Chip Routing Cell Diagram
A Dynamic Analog Concurrently-Processed Adaptive Chip • Circuit Design • Created and modified analog and mixed-signal circuits for each learning/propagation task • Most circuits are implemented using a small number of MOSFET transistors • Transistors are relatively large (5µm length) and interdigitated for good matching • Simulations have been completed for all cells • CMOS layout has been completed for some of the cells
A Dynamic Analog Concurrently-Processed Adaptive Chip Current Summation
A Dynamic Analog Concurrently-Processed Adaptive Chip Tanh and Sech2 (Sigmoid and Derivative)
A Dynamic Analog Concurrently-Processed Adaptive Chip Differential Pairs (for multiplier input)
A Dynamic Analog Concurrently-Processed Adaptive Chip Gilbert Multiplier Cell
A Dynamic Analog Concurrently-Processed Adaptive Chip • Layout: • Completed for tanh/sech2 • First a stick diagram was planned • Initial layout planned in Microsoft Visio • Drawn in Cadence Virtuoso using unmatched transistors • Drawn in Cadence Virtuoso using matched (interdigitated) transistors • Planned for other cells, but not yet complete
A Dynamic Analog Concurrently-Processed Adaptive Chip Initial Virtuoso Layout of tanh (no matching)
A Dynamic Analog Concurrently-Processed Adaptive Chip Virtuoso Layout of tanh with interdigitated matching
A Dynamic Analog Concurrently-Processed Adaptive Chip Initial Visio layout of tanh (no matching)
A Dynamic Analog Concurrently-Processed Adaptive Chip Simulation Graphs Tanh Sigmoid
A Dynamic Analog Concurrently-Processed Adaptive Chip • Sigmoid: • Input Common Mode Range: 0.5V – 1.5V • Good accuracy compared to ideal function of tanh
A Dynamic Analog Concurrently-Processed Adaptive Chip Simulation Graphs Sech2 Sigmoid
A Dynamic Analog Concurrently-Processed Adaptive Chip • Sigmoid Derivative: • Input Common Mode Range: 0.5V – 1.5V • Good accuracy compared to ideal function of sech2
A Dynamic Analog Concurrently-Processed Adaptive Chip Simulation Graphs Multiplier
A Dynamic Analog Concurrently-Processed Adaptive Chip • Multiplier: • Input Common Mode Range: 2V – 3V • Symmetrical inputs around center point • Good linearity for input swing of ±150mV • Performance is reduced as input range is extended • Pre-processing differential pair is required
A Dynamic Analog Concurrently-Processed Adaptive Chip • Results: • A neural network is simulated in C++ • OCR problem, of 5 training sets of randomly ordered numerical characters • 10x10 pixels • 5% noise • Position offset of ±1 pixel horizontally and vertically • Comparing: • 100%-connected network • 20%-connected network • Simulation of a dynamically re-routed network by removing 10% of the synapses over 1000 cycles
A Dynamic Analog Concurrently-Processed Adaptive Chip Results 0 to 1000 training cycles
A Dynamic Analog Concurrently-Processed Adaptive Chip Results 200 to 1000 training cycles
A Dynamic Analog Concurrently-Processed Adaptive Chip Results 800 to 1000 training cycles
A Dynamic Analog Concurrently-Processed Adaptive Chip • Conclusions: • The dynamic routing algorithm is the ideal method of optimizing performance in a hardware neural network • Reducing the number of synapses to increase density • The cell circuits perform very well with system-made restrictions on input ranges • Restrictions increase accuracy and linearity • The arrangement of routing cells between neurons and synapses allows the routing algorithm to perform very efficiently
A Dynamic Analog Concurrently-Processed Adaptive Chip • Special Thanks To: • Victoria Stagg • My mother, for all of her help and support • Andrew Stagg • My brother, for all of his help and support • Dr. Jim Haslett, University of Calgary • For use of the ATIPS laboratory, to use Cadence Virtuoso • John Carney, Cadence Design • For the demo license of Cadence Orcad Layout • Dr. Vance Tyree, MOSIS Corporation • For agreeing to accept a fabrication proposal under the MOSIS education program • David Wells, Auton Engineering, Ltd. • For printing my trifold