Furthermore, whencontrollingcomplex high-dimensional morphologies, such as humanoid bodies, tasks often require coordination of multiple skills simultaneously.

Learning a control policy for a multi-phase, long-horizon task, such as basketball maneuvers, remains challenging for reinforcement learning approaches due to the need for seamless policy composition and transitions between skills. A long-horizon task typically consists of distinct subtasks with well-defined goals, separated by transitional subtasks with unclear goals but critical to the success of the entire task. Existing methods like the mixture of experts and skill chaining struggle with tasks where individual policies do not share significant commonly explored states or lack well-defined initial and terminal states between different phases. In this paper, we introduce a novel policy integration framework to enable the composition of drastically different motor skills in multi-phase long-horizon tasks with ill-defined intermediate states. Based on that, we further introduce a high-level soft router to enable seamless and robust transitions between the subtasks. We evaluate our framework on a set of fundamental basketball skills and challenging transitions. Policies trained by our approach can effectively control the simulated character to interact with the ball and accomplish the long-horizon task specified by real-time user commands, without relying on ball trajectory references.

Data analysis plays an increasingly important role in soccer, offering new ways to evaluate individual and team performance. One specific application is the evaluation of dribbles: one-on-one situations where an attacker attempts to bypass a defender with the ball. While previous research has primarily relied on 2D positional tracking data, this fails to capture aspects like balance, orientation, and ball control, limiting the depth of current insights. This study explores how pose tracking data-- capturing players' posture and movement in three dimensions--can improve our understanding of dribbling skills. We extract novel pose-based features from 1,736 dribbles in the 2022/23 Champions League season and evaluate their impact on dribble success. Our results indicate that features capturing the attacker's balance and the alignment of the orientation between the attacker and defender are informative for predicting dribble success. Incorporating these pose-based features on top of features derived from traditional 2D positional data leads to a measurable improvement in model performance.

The RoboCup competition was started in 1997, and is known as the oldest RoboCup league. The RoboCup 2D Soccer Simulation League is a stochastic, partially observable soccer environment in which 24 autonomous agents play on two opposing teams. In this paper, we detail the main strategies and functionalities of CYRUS, the RoboCup 2021 2D Soccer Simulation League champions. The new functionalities presented and discussed in this work are (i) Multi Action Dribble, (ii) Pass Prediction and (iii) Marking Decision. The Multi Action Dribbling strategy enabled CYRUS to succeed more often and to be safer when dribbling actions were performed during a game. The Pass Prediction enhanced our gameplay by predicting our teammate's passing behavior, anticipating and making our agents collaborate better towards scoring goals. Finally, the Marking Decision addressed the multi-agent matching problem to improve CYRUS defensive strategy by finding an optimal solution to mark opponents' players.

Using human and animal motions to teach robots to dribble a ball, and simulated humanoid characters to carry boxes and play football. Five years ago, we took on the challenge of teaching a fully articulated humanoid character to traverse obstacle courses. Here, we describe a solution to both challenges called neural probabilistic motor primitives (NPMP), involving guided learning with movement patterns derived from humans and animals, and discuss how this approach is used in our Humanoid Football paper, published today in Science Robotics. We also discuss how this same approach enables humanoid full-body manipulation from vision, such as a humanoid carrying an object, and robotic control in the real-world, such as a robot dribbling a ball. An NPMP is a general-purpose motor control module that translates short-horizon motor intentions to low-level control signals, and it's trained offline or via RL by imitating motion capture (MoCap) data, recorded with trackers on humans or animals performing motions of interest.

Humans are able to perform a myriad of sophisticated tasks by drawing upon skills acquired through prior experience. For autonomous agents to have this capability, they must be able to extract reusable skills from past experience that can be recombined in new ways for subsequent tasks. Furthermore, when controlling complex high-dimensional morphologies, such as humanoid bodies, tasks often require coordination of multiple skills simultaneously. Learning discrete primitives for every combination of skills quickly becomes prohibitive. Composable primitives that can be recombined to create a large variety of behaviors can be more suitable for modeling this combinatorial explosion. In this work, we propose multiplicative compositional policies (MCP), a method for learning reusable motor skills that can be composed to produce a range of complex behaviors. Our method factorizes an agent's skills into a collection of primitives, where multiple primitives can be activated simultaneously via multiplicative composition. This flexibility allows the primitives to be transferred and recombined to elicit new behaviors as necessary for novel tasks. We demonstrate that MCP is able to extract composable skills for highly complex simulated characters from pre-training tasks, such as motion imitation, and then reuse these skills to solve challenging continuous control tasks, such as dribbling a soccer ball to a goal, and picking up an object and transporting it to a target location.

It can't dribble, let alone slam dunk, but Toyota's basketball robot hardly ever misses a free throw or a 3-pointer. The 207-centimeter-tall (6 feet 10-inches) machine made five of eight 3-point shots in a demonstration in a Tokyo suburb Monday, a ratio its engineers say is worse than usual. Toyota Motor Corp.'s robot, called Cue 3, computes a three-dimensional image where the basket is, using sensors on its torso, and adjusts motors inside its arm and knees to give the shot the right angle and propulsion for a swish. Efforts in developing human-shaped robots underline a global shift in robotics use from pre-programmed mechanical arms in limited situations like factories to functioning in the real world with people. The 2017 version of the robot was designed to make free throws.

I'm not what you'd call a coordinated man, so basketball horrifies me. Basketball players have to be one with the laws of physics. I am not one with the laws of physics. Now imagine teaching the machines something as complicated as dribbling--which is exactly what researchers at Carnegie Mellon University and a startup called DeepMotion have done. Using motion-capture technology, they've shown an algorithm generally how humans move when they dribble.

I'm not what you'd call a coordinated man, so basketball horrifies me. Basketball players have to be one with the laws of physics. I am not one with the laws of physics. Now imagine teaching the machines something as complicated as dribbling--which is exactly what researchers at Carnegie Mellon University and a startup called DeepMotion have done. Using motion-capture technology, they've shown an algorithm generally how humans move when they dribble.

While this animated fellow looks like something out of NBA 2K18, it's really an AI that's learning how to dribble in real time. The AI starts out fumbling the ball a bit and by cycle 95 it is able to do some real Harlem Globetrotters stuff. In short, what you're watching is a human-like avatar learning a very specialized human movement. To do this researchers at Carnegie Mellon and DeepMotion, Inc. created a "physics-based, real-time method for controlling animated characters that can learn dribbling skills from experience." The system, which uses "deep reinforcement learning," can use motion capture date to learn basic movements.