Monteiro, Juarez
Explorative Imitation Learning: A Path Signature Approach for Continuous Environments
Gavenski, Nathan, Monteiro, Juarez, Meneguzzi, Felipe, Luck, Michael, Rodrigues, Odinaldo
Some imitation learning methods combine behavioural cloning with self-supervision to infer actions from state pairs. However, most rely on a large number of expert trajectories to increase generalisation and human intervention to capture key aspects of the problem, such as domain constraints. In this paper, we propose Continuous Imitation Learning from Observation (CILO), a new method augmenting imitation learning with two important features: (i) exploration, allowing for more diverse state transitions, requiring less expert trajectories and resulting in fewer training iterations; and (ii) path signatures, allowing for automatic encoding of constraints, through the creation of non-parametric representations of agents and expert trajectories. We compared CILO with a baseline and two leading imitation learning methods in five environments. It had the best overall performance of all methods in all environments, outperforming the expert in two of them.
Self-Supervised Adversarial Imitation Learning
Monteiro, Juarez, Gavenski, Nathan, Meneguzzi, Felipe, Barros, Rodrigo C.
Behavioural cloning is an imitation learning technique that teaches an agent how to behave via expert demonstrations. Recent approaches use self-supervision of fully-observable unlabelled snapshots of the states to decode state pairs into actions. However, the iterative learning scheme employed by these techniques is prone to get trapped into bad local minima. Previous work uses goal-aware strategies to solve this issue. However, this requires manual intervention to verify whether an agent has reached its goal. We address this limitation by incorporating a discriminator into the original framework, offering two key advantages and directly solving a learning problem previous work had. First, it disposes of the manual intervention requirement. Second, it helps in learning by guiding function approximation based on the state transition of the expert's trajectories. Third, the discriminator solves a learning issue commonly present in the policy model, which is to sometimes perform a `no action' within the environment until the agent finally halts.
Imitating Unknown Policies via Exploration
Gavenski, Nathan, Monteiro, Juarez, Granada, Roger, Meneguzzi, Felipe, Barros, Rodrigo C.
Behavioral cloning is an imitation learning technique that teaches an agent how to behave through expert demonstrations. Recent approaches use self-supervision of fully-observable unlabeled snapshots of the states to decode state-pairs into actions. However, the iterative learning scheme from these techniques are prone to getting stuck into bad local minima. We address these limitations incorporating a two-phase model into the original framework, which learns from unlabeled observations via exploration, substantially improving traditional behavioral cloning by exploiting (i) a sampling mechanism to prevent bad local minima, (ii) a sampling mechanism to improve exploration, and (iii) self-attention modules to capture global features. The resulting technique outperforms the previous state-of-the-art in four different environments by a large margin.
Classifying Norm Conflicts using Learned Semantic Representations
Aires, Joรฃo Paulo, Granada, Roger, Monteiro, Juarez, Barros, Rodrigo C., Meneguzzi, Felipe
As natural language uses a diverse and often vague way to express ideas, identifying a norm conflict and its causes While most social norms are informal, they are often in contracts is a challenging task. An ever larger number of formalized by companies in contracts to regulate contracts being currently generated necessitates a fast and reliable trades of goods and services. When poorly process to identify norm conflicts. However, since such written, contracts may contain normative conflicts contracts are written in natural language, traditional revision resulting from opposing deontic meanings or contradict methods involve contract makers reading the contract and specifications. As contracts tend to be identifying conflicting points between norms. Such a method long and contain many norms, manually identifying requires huge human-effort and may not guarantee a revision such conflicts requires human-effort, which is that eliminates all conflicts. In response, we provide three time-consuming and error-prone. Automating such contributions towards automatically identifying and classifying task benefits contract makers increasing productivity potential conflicts between norms in contracts.
A Deep Neural Architecture for Kitchen Activity Recognition
Granada, Roger Leitzke (Pontifรญcia Universidade Catรณlica do Rio Grande do Sul) | Monteiro, Juarez (Pontifรญcia Universidade Catรณlica do Rio Grande do Sul) | Barros, Rodrigo Coelho (Pontifรญcia Universidade Catรณlica do Rio Grande do Sul) | Meneguzzi, Felipe Rech (Pontifรญcia Universidade Catรณlica do Rio Grande do Sul)
Computer-based human activity recognition of daily living has recently attracted much interest due to its applicability to ambient assisted living. Such applications require the automatic recognition of high-level activities composed of multiple actions performed by human beings in a given environment. We propose a deep neural architecture for kitchen activity recognition, which uses an ensemble of machine learning models and hand-crafted features to extract more information of the data. Experiments show that our approach achieves the state-of-the-art for identifying cooking actions in a well-known kitchen dataset.
Hybrid Activity and Plan Recognition for Video Streams
Granada, Roger Leitzke (Pontifical Catholic University of Rio Grande do Sul) | Pereira, Ramon Fraga (Pontifical Catholic University of Rio Grande do Sul) | Monteiro, Juarez (Pontifical Catholic University of Rio Grande do Sul) | Barros, Rodrigo Coelho (Pontifical Catholic University of Rio Grande do Sul) | Ruiz, Duncan (Pontifical Catholic University of Rio Grande do Sul) | Meneguzzi, Felipe (Pontifical Catholic University of Rio Grande do Sul)
Computer-based human activity recognition of daily living has recently attracted much interest due to its applicability to ambient assisted living. Such applications require the automatic recognition of high-level activities composed of multiple actions performed by human beings in an environment. In this work, we address the problem of activity recognition in an indoor environment, focusing on a kitchen scenario. Unlike existing approaches that identify single actions from video sequences, we also identify the goal towards which the subject of the video is pursuing. Our hybrid approach combines a deep learning architecture to analyze raw video data and identify individual actions which are then processed by a goal recognition algorithm that uses a plan library describing possible overarching activities to identify the ultimate goal of the subject in the video. Experiments show that our approach achieves the state-of-the-art for identifying cooking activities in a kitchen scenario.