SNAP: An Efficient Sparse Neural Acceleration Processor for Unstructured Sparse Deep Neural Network Inference

Recent developments in deep neural network (DNN) pruning introduces data sparsity to enable deep learning applications to run more efficiently on resource- and energy-constrained hardware platforms. However, these sparse models require specialized hardware structures to exploit the sparsity for stor...

Full description

Saved in:
Bibliographic Details
Published in:IEEE journal of solid-state circuits 2021-02, Vol.56 (2), p.636-647
Main Authors: Zhang, Jie-Fang, Lee, Ching-En, Liu, Chester, Shao, Yakun Sophia, Keckler, Stephen W., Zhang, Zhengya
Format: Article
Language:English
Subjects:
Acceleration
Arrays
Artificial neural networks
Channel index matching
CMOS
Convolution
deep neural network (DNN)
energy-efficient accelerator
Hardware
Indexes
Machine learning
Microprocessors
Neural networks
Pipelines
Reduction
sparse neural network
Sparsity
Throughput
unstructured sparsity
Weight
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent developments in deep neural network (DNN) pruning introduces data sparsity to enable deep learning applications to run more efficiently on resource- and energy-constrained hardware platforms. However, these sparse models require specialized hardware structures to exploit the sparsity for storage, latency, and efficiency improvements to the full extent. In this work, we present the sparse neural acceleration processor (SNAP) to exploit unstructured sparsity in DNNs. SNAP uses parallel associative search to discover valid weight (W) and input activation (IA) pairs from compressed, unstructured, sparse W and IA data arrays. The associative search allows SNAP to maintain a 75% average compute utilization. SNAP follows a channel-first dataflow and uses a two-level partial sum (psum) reduction dataflow to eliminate access contention at the output buffer and cut the psum writeback traffic by 22 \times compared with state-of-the-art DNN accelerator designs. SNAP's psum reduction dataflow can be configured in two modes to support general convolution (CONV) layers, pointwise CONV, and fully connected layers. A prototype SNAP chip is implemented in a 16-nm CMOS technology. The 2.3-mm 2 test chip is measured to achieve a peak effectual efficiency of 21.55 TOPS/W (16 b) at 0.55 V and 260 MHz for CONV layers with 10% weight and activation densities. Operating on a pruned ResNet-50 network, the test chip achieves a peak throughput of 90.98 frames/s at 0.80 V and 480 MHz, dissipating 348 mW.
ISSN:0018-9200
1558-173X
DOI:10.1109/JSSC.2020.3043870