Humans have the remarkable ability to pick up unfamiliar objects and quickly understand their surface properties, such as friction, and dynamics. This knowledge enables us not only to reorient objects using our arms but also to manipulate them within our hands, extending our capabilities beyond what is typically seen in traditional robotics. In this paper, we introduce a custom parallel gripper equipped with commercial 6-degree-of-freedom (DoF) force-torque (F/T) sensors and custom relative velocity sensors (see Figure 1), for in-hand slip-aware control that relies solely on in-hand sensing. The ability to independently measure force and planar velocity introduces new opportunities for intricate robotic manipulation. This hardware combination enables rapid estimation of friction and contact surface properties without the need for external sensors, thus facilitating for precise in-hand manipulation of objects in both rotational and translational movements. Slip-aware control significantly enhances the functionality of robotic manipulators by enabling the object-end-effector relative pose to adapt during grasping, thereby extending the operational workspace. This adaptability is particularly valuable in constrained environments, where the manipulator's movement is limited, or for intelligent human-robot interaction, enabling for instance more intuitive and safe handovers. Furthermore, in-hand slippage control opens up new opportunities for multi-arm manipulation of single objects, allowing for the repositioning of grasps without releasing the object, thereby enabling more efficient and flexible handling of larger items. Our system has been rigorously tested across a wide range of experiments, demonstrating its effectiveness and versatility.

Deformable objects present several challenges to the field of robotic manipulation. One of the tasks that best encapsulates the difficulties arising due to non-rigid behavior is shape control, which requires driving an object to a desired shape. While shape-servoing methods have been shown successful in contexts with approximately linear behavior, they can fail in tasks with more complex dynamics. We investigate an alternative approach, using offline RL to solve a planar shape control problem of a Deformable Linear Object (DLO). To evaluate the effect of material properties, two DLOs are tested namely a soft rope and an elastic cord. We frame this task as a goal-conditioned offline RL problem, and aim to learn to generalize to unseen goal shapes. Data collection and augmentation procedures are proposed to limit the amount of experimental data which needs to be collected with the real robot. We evaluate the amount of augmentation needed to achieve the best results, and test the effect of regularization through behavior cloning on the TD3+BC algorithm. Finally, we show that the proposed approach is able to outperform a shape-servoing baseline in a curvature inversion experiment.

This article addresses the obstacle avoidance problem for setpoint stabilization and path-following tasks in complex dynamic 2D environments that go beyond conventional scenes with isolated convex obstacles. A combined motion planner and controller is proposed for setpoint stabilization that integrates the favorable convergence characteristics of closed-form motion planning techniques with the intuitive representation of system constraints through Model Predictive Control (MPC). The method is analytically proven to accomplish collision avoidance and convergence under certain conditions, and it is extended to path-following control. Various simulation scenarios using a non-holonomic unicycle robot are provided to showcase the efficacy of the control scheme and its improved convergence results compared to standard path-following MPC approaches with obstacle avoidance.

Humans are able to manipulate Deformable Linear Objects (DLOs) such as cables and wires, with little or no visual information, relying mostly on force sensing. In this work, we propose a reduced DLO model which enables such blind manipulation by keeping the object under tension. Further, an online model estimation procedure is also proposed. A set of elementary sliding and clipping manipulation primitives are defined based on our model. The combination of these primitives allows for more complex motions such as winding of a DLO. The model estimation and manipulation primitives are tested individually but also together in a real-world cable harness production task, using a dual-arm YuMi, thus demonstrating that force-based perception can be sufficient even for such a complex scenario.

Contact surfaces in planar motion exhibit a coupling between tangential and rotational friction forces. This paper proposes planar friction models grounded in the LuGre model and limit surface theory. First, distributed planar extended state models are proposed and the Elasto-Plastic model is extended for multi-dimensional friction. Subsequently, we derive a reduced planar friction model, coupled with a pre-calculated limit surface, that offers reduced computational cost. The limit surface approximation through an ellipsoid is discussed. The properties of the planar friction models are assessed in various simulations, demonstrating that the reduced planar friction model achieves comparable performance to the distributed model while exhibiting ~80 times lower computational cost.

This paper proposes a motion control scheme for robots operating in a dynamic environment with concave obstacles. A Model Predictive Controller (MPC) is constructed to drive the robot towards a goal position while ensuring collision avoidance without direct use of obstacle information in the optimization problem. This is achieved by guaranteeing tracking performance of an appropriately designed receding horizon path. The path is computed using a guiding vector field defined in a subspace of the free workspace where each point in the subspace satisfies a criteria for minimum distance to all obstacles. The effectiveness of the control scheme is illustrated by means of simulation.

Motion planning methods like navigation functions and harmonic potential fields provide (almost) global convergence and are suitable for obstacle avoidance in dynamically changing environments due to their reactive nature. A common assumption in the control design is that the robot operates in a disjoint star world, i.e. all obstacles are strictly starshaped and mutually disjoint. However, in real-life scenarios obstacles may intersect due to expanded obstacle regions corresponding to robot radius or safety margins. To broaden the applicability of aforementioned reactive motion planning methods, we propose a method to reshape a workspace of intersecting obstacles into a disjoint star world. The algorithm is based on two novel concepts presented here, namely admissible kernel and starshaped hull with specified kernel, which are closely related to the notion of starshaped hull. The utilization of the proposed method is illustrated with examples of a robot operating in a 2D workspace using a harmonic potential field approach in combination with the developed algorithm.

Deformable object manipulation tasks have long been regarded as challenging robotic problems. However, until recently very little work has been done on the subject, with most robotic manipulation methods being developed for rigid objects. Deformable objects are more difficult to model and simulate, which has limited the use of model-free Reinforcement Learning (RL) strategies, due to their need for large amounts of data that can only be satisfied in simulation. This paper proposes a new shape control task for Deformable Linear Objects (DLOs). More notably, we present the first study on the effects of elastoplastic properties on this type of problem. Objects with elastoplasticity such as metal wires, are found in various applications and are challenging to manipulate due to their nonlinear behavior. We first highlight the challenges of solving such a manipulation task from an RL perspective, particularly in defining the reward. Then, based on concepts from differential geometry, we propose an intrinsic shape representation using discrete curvature and torsion. Finally, we show through an empirical study that in order to successfully solve the proposed task using Deep Deterministic Policy Gradient (DDPG), the reward needs to include intrinsic information about the shape of the DLO.

Learning generative models and inferring latent trajectories have shown to be challenging for time series due to the intractable marginal likelihoods of flexible generative models. It can be addressed by surrogate objectives for optimization. We propose Monte Carlo filtering objectives (MCFOs), a family of variational objectives for jointly learning parametric generative models and amortized adaptive importance proposals of time series. MCFOs extend the choices of likelihood estimators beyond Sequential Monte Carlo in state-of-the-art objectives, possess important properties revealing the factors for the tightness of objectives, and allow for less biased and variant gradient estimates. We demonstrate that the proposed MCFOs and gradient estimations lead to efficient and stable model learning, and learned generative models well explain data and importance proposals are more sample efficient on various kinds of time series data.