αPOMDP: POMDP-based user-adaptive decision-making for social robots

In this work we present αPOMDP: a User-Adaptive Decision-Making technique for social robots. This technique is based on the classical POMDP formulation which we extend with novel aspects inspired by Reward Shaping and Model-Based Reinforcement Learning. Our technique innovates in two main ways: by applying a novel set of rewarding schemes based on the state of the user and by employing a novel execution loop that enables the system to learn the impact of its actions on the user on-the-fly. Our technique has been tested with multiple POMDP solvers and reward formulations in simulations and with real users through the GrowMu social robot. Results show that our technique is able to correctly decide which actions to take, maintaining the user in positive states which interacting with the robot and methodically exploring and learning their characteristics, activities and behaviors.