Cognitive Human-Machine Interfaces and Interactions for Unmanned Aircraft

Yixiang Lim, Subramanian Ramasamy, Alessandro Gardi, Trevor Kistan, Roberto Sabatini

Research output: Contribution to journalArticlepeer-review

35 Scopus citations


This paper presents the concept of Cognitive Human-Machine Interfaces and Interactions (CHMI2) for Unmanned Aircraft System (UAS) Ground Control Stations (GCS). CHMI2 represents a new approach to aviation human factors engineering that introduces adaptive functionalities in the design of operators’ command, control and display functions. A CHMI2 system assesses human cognitive states based on measurement of key psycho-physiological observables. The cognitive states are used to predict and enhance operator performance in the accomplishment of aviation tasks, with the objective of improving the efficiency and effectiveness of the overall human-machine teaming. The CHMI2 system presented in this paper employs a four-layer architecture comprising sensing, extraction, classification and adaptation functionalities. An overview of each layer is provided along with the layer’s metrics, algorithms and functions. Two relevant case studies are presented to illustrate the interactions between the different layers, and the conceptual design of the associated display formats is described. The results indicate that specific eye tracking variables provide discrimination between different modes of control. Furthermore, results indicate that the higher levels of automation supported by the CHMI2 are beneficial in Separation Assurance and Collision Avoidance (SA&CA) scenarios involving low-detectability obstacles and stringent time constraints to implement recovery manoeuvres. These preliminary results highlight that the introduction of CHMI2 functionalities in future UAS can significantly reduce reaction time and enhance operational effectiveness of unmanned aircraft response to collision and loss of separation events, as well as improve the overall safety and efficiency of operations.

Original languageBritish English
Pages (from-to)755-774
Number of pages20
JournalJournal of Intelligent and Robotic Systems: Theory and Applications
Issue number3-4
StatePublished - 1 Sep 2018


  • Cognitive ergonomics
  • Ground control station
  • Human factors engineering
  • Human machine interactions
  • Human machine interfaces
  • Psycho-physiological sensing
  • Sense and avoid
  • Unmanned aircraft


Dive into the research topics of 'Cognitive Human-Machine Interfaces and Interactions for Unmanned Aircraft'. Together they form a unique fingerprint.

Cite this