Design Exploration of ReRAM-Based Crossbar for AI Inference

Yasmin Halawani, Baker Mohammad, Hani Saleh

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

ReRAM-based crossbar designs utilizing mixed-signal implementation has gained importance due to their low power, small size, low cost, and high throughput especially for multiply-and-add operations in AI-related applications. This paper provides a framework with associated code for analyzing the impact of ReRAM device variation and post-training analog conductance quantization that has not been fully explored for pre-trained network accelerators. A detailed study with end-to-end implementation is presented, ranging from mapping the pre-trained DNN weights to quantized crossbar conductance values and into final classification with the presence of variation. Monte Carlo analysis was performed to better analyze the impact of the different parameters on the final accuracy without being device-specific. The work assumes different conductance value variations, different conductance dynamic ranges, and various device quantization levels (QLs). MNIST and CIFAR-10 data sets were used in this study for ANN and CNN, respectively. Results show that for simple ANN, the accuracy drop due to quantization was 2% at 64-QLs. While for CNN, the decrease in classification accuracy was around 10% with the same number of levels. Moreover, weight variation might cause a 5% and 8% drop in classification accuracy for ANN with 5% and CNN with 3% variation, respectively. The study confirms that increasing the number of levels with small variation results in near-optimal accuracy. However, the increase in accuracy saturates at an upper limit. The amount of distortion propagated through the layers is different in the two cases. It is dependent on the complexity of input data and network structure, such as the size of the neurons in each layer, the number of layers, and the number of channels and filters at each stage. This contribution is the first to provide the framework to explore the implication that emphasizes on device-independent post-training DNN quantization and weight variation on classification accuracy. This helps explore design trade-offs especially for edge devices in cases where there is no access to the training set to the end-user due to security or cost issues for pre-trained networks.

Original languageBritish English
Article number9419046
Pages (from-to)70430-70442
Number of pages13
JournalIEEE Access
Volume9
DOIs
StatePublished - 2021

Keywords

  • Device variation
  • in-memory computing
  • neural network
  • post-training
  • quantization
  • ReRAM

Fingerprint

Dive into the research topics of 'Design Exploration of ReRAM-Based Crossbar for AI Inference'. Together they form a unique fingerprint.

Cite this