TY - JOUR
T1 - Seg-CURL
T2 - Segmented Contrastive Unsupervised Reinforcement Learning for Sim-to-Real in Visual Robotic Manipulation
AU - Xu, Binzhao
AU - Hassan, Taimur
AU - Hussain, Irfan
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2023
Y1 - 2023
N2 - Training image-based reinforcement learning (RL) agents are sample-inefficient, limiting their effectiveness in real-world manipulation tasks. Sim2Real, which involves training in simulations and transferring to the real world, effectively reduces the dependence on real data. However, the performance of the transferred agent degrades due to the visual difference between the two environments. This research presents a low-cost segmentation-driven unsupervised RL framework (Seg-CURL) to solve the Sim2Real problem. We transform the input RGB views to the proposed semantic segmentation-based canonical domain. Our method incorporates two levels of Sim2Real: task-level Sim2Real, which transfers the RL agent to the real world, and observation-level Sim2Real, which transfers the simulated U-nets to segment real-world scenes. Specifically, we first train contrastive unsupervised RL(CURL) with segmented images in the simulation environment. Next, we employ two U-Nets to segment robotic hand-view and side-view images during real robot control. These U-Net are pre-trained with synthetic RGB and segmentation masks in the simulation environment and fine-tuned with only 20 real images. We evaluate the robustness of the proposed framework in both simulation and real environments. Seg-CURL is robust to the texture, lighting, shadow, and camera position gap. Finally, our algorithm is tested on a real Baxter robot with a dark hand-view in the cube lifting task with a success rate of 16/20 in zero-shot transfer.
AB - Training image-based reinforcement learning (RL) agents are sample-inefficient, limiting their effectiveness in real-world manipulation tasks. Sim2Real, which involves training in simulations and transferring to the real world, effectively reduces the dependence on real data. However, the performance of the transferred agent degrades due to the visual difference between the two environments. This research presents a low-cost segmentation-driven unsupervised RL framework (Seg-CURL) to solve the Sim2Real problem. We transform the input RGB views to the proposed semantic segmentation-based canonical domain. Our method incorporates two levels of Sim2Real: task-level Sim2Real, which transfers the RL agent to the real world, and observation-level Sim2Real, which transfers the simulated U-nets to segment real-world scenes. Specifically, we first train contrastive unsupervised RL(CURL) with segmented images in the simulation environment. Next, we employ two U-Nets to segment robotic hand-view and side-view images during real robot control. These U-Net are pre-trained with synthetic RGB and segmentation masks in the simulation environment and fine-tuned with only 20 real images. We evaluate the robustness of the proposed framework in both simulation and real environments. Seg-CURL is robust to the texture, lighting, shadow, and camera position gap. Finally, our algorithm is tested on a real Baxter robot with a dark hand-view in the cube lifting task with a success rate of 16/20 in zero-shot transfer.
KW - Reinforcement learning
KW - robotic manipulation
KW - Sim2Real
KW - zero-short learning
UR - http://www.scopus.com/inward/record.url?scp=85160238809&partnerID=8YFLogxK
U2 - 10.1109/ACCESS.2023.3278208
DO - 10.1109/ACCESS.2023.3278208
M3 - Article
AN - SCOPUS:85160238809
SN - 2169-3536
VL - 11
SP - 50195
EP - 50204
JO - IEEE Access
JF - IEEE Access
ER -