Neuromorphic vision based contact-level classification in robotic grasping applications

Xiaoqian Huang, Rajkumar Muthusamy, Eman Hassan, Zhenwei Niu, Lakmal Seneviratne, Dongming Gan, Yahya Zweiri

Research output: Contribution to journalArticlepeer-review

14 Scopus citations


In recent years, robotic sorting is widely used in the industry, which is driven by necessity and opportunity. In this paper, a novel neuromorphic vision-based tactile sensing approach for robotic sorting application is proposed. This approach has low latency and low power consumption when compared to conventional vision-based tactile sensing techniques. Two Machine Learning (ML) methods, namely, Support Vector Machine (SVM) and Dynamic Time Warping-K Nearest Neighbor (DTW-KNN), are developed to classify material hardness, object size, and grasping force. An Event-Based Object Grasping (EBOG) experimental setup is developed to acquire datasets, where 243 experiments are produced to train the proposed classifiers. Based on predictions of the classifiers, objects can be automatically sorted. If the prediction accuracy is below a certain threshold, the gripper re-adjusts and re-grasps until reaching a proper grasp. The proposed ML method achieves good prediction accuracy, which shows the effectiveness and the applicability of the proposed approach. The experimental results show that the developed SVM model outperforms the DTW-KNN model in term of accuracy and efficiency for real time contact-level classification.

Original languageBritish English
Article number4724
Pages (from-to)1-15
Number of pages15
JournalSensors (Switzerland)
Issue number17
StatePublished - 1 Sep 2020


  • Contact-level classification
  • Dynamic vision sensor
  • Haptics
  • Machine learning
  • Neuromorphic vision
  • Robotics sorting


Dive into the research topics of 'Neuromorphic vision based contact-level classification in robotic grasping applications'. Together they form a unique fingerprint.

Cite this