Zero-Shot Transfer in Reinforcement Learning for Quasi-Similar Systems via Compensation Dynamics

In this paper, we propose a novel approach for zero-shot embodiment transfer of deep reinforcement learning (Deep RL) agents across quasi-similar systems. The challenge of transferring these agents across different but quasi-similar dynamic systems remains a significant obstacle in realizing the full potential of Deep RL. Existing approaches often require extensive fine-tuning or re-training of the agent when transferring to a new system, which is time consuming and resource-intensive. Our proposed method introduces compensatory dynamics in the “Trained On System" to synchronize the response with the more complex “Transferred To System", enabling seamless transfer without any adjustments to the trained agent. This approach is based on the hypothesis that these compensatory dynamics can achieve congruency in system response, facilitating the zero-shot transfer of the RL agent, in contrast to existing methods that typically require extensive fine-tuning or re-training. We demonstrate our method by training an RL agent to move the driver to a target position while damping load swing using a tower crane, and then transferring the agent to a quadrotor UAV slung-load system to perform a similar task. The calculated similarity index of 0.893 indicates a strong similarity between the two system responses, supporting the effectiveness of our zero-shot transfer approach.