We study sim-to-real skill transfer and discovery in the context of robotics control using representation learning. We draw inspiration from spectral decomposition of Markov decision processes. The spectral decomposition brings about representation that can linearly represent the state-action value function induced by any policies, thus can be regarded as skills. The skill representations are transferable across arbitrary tasks with the same transition dynamics. Moreover, to handle the sim-to-real gap in the dynamics, we propose a skill discovery algorithm that learns new skills caused by the sim-to-real gap from real-world data. We promote the discovery of new skills by enforcing orthogonal constraints between the skills to learn and the skills from simulators, and then synthesize the policy using the enlarged skill sets. We demonstrate our methodology by transferring quadrotor controllers from simulators to Crazyflie 2.1 quadrotors. We show that we can learn the skill representations from a single simulator task and transfer these to multiple different real-world tasks including hovering, taking off, landing and trajectory tracking. Our skill discovery approach helps narrow the sim-to-real gap and improve the real-world controller performance by up to 30.2%.

Quadcopter trajectory tracking control has been extensively investigated and implemented in the past. Available controls mostly use the Euler angle standards to describe the quadcopters rotational kinematics and dynamics. As a result, the same rotation can be translated into different roll, pitch, and yaw angles because there are multiple Euler angle standards for characterization of rotation in a 3-dimensional motion space. Additionally, it is computationally expensive to convert a quadcopters orientation to the associated roll, pitch, and yaw angles, which may make it difficult to track quick and aggressive trajectories. To address these issues, this paper will develop a flatness-based trajectory tracking control without using Euler angles. We assess and test the proposed controls performance in the Gazebo simulation environment and contrast its functionality with the existing Mellinger controller, which has been widely adopted by the robotics and unmanned aerial system (UAS) communities.