Neural Information Processing Systems http://nips.cc/

It detects attention weights from input event sequence and returns the sequence representation.

What sets timeseries analysis apart from other machine learning To elaborate on the various considerations that need to be addressed, exercises is that time representation becomes a primary aspect of first consider that one cannot assume fully observed, uniformly the experiment setup, as it must adequately represent the temporal sampled inputs as there might be gaps in the data, varying relations that are relevant for the application at hand. In the work sampling rates, and (for multivariate timeseries) misalignment described here we study wo different variations of the Transformer between the time steps of the different variables. This dictates a architecture: one where we use the fixed time representation proposed representation that allows time differences to be computed, so that in the literature and one where the time representation is (for example) September 2023 is'closer' to January 2024 than it is to learned from the data. Our experiments use data from predicting September 2022. Simple timestamps allow this but do not capture the energy output of solar panels, a task that exhibits known periodicities periodicity: Consider, for instance, an application with seasonal (daily and seasonal) that is straight-forward to encode in periodicity where September 2023 is'closer' to September 2022 the fixed time representation. Our results indicate that even in an than to January 2024.

Spatio-temporal forecasting of future values of spatially correlated time series is important across many cyber-physical systems (CPS). Recent studies offer evidence that the use of graph neural networks to capture latent correlations between time series holds a potential for enhanced forecasting. However, most existing methods rely on pre-defined or self-learning graphs, which are either static or unintentionally dynamic, and thus cannot model the time-varying correlations that exhibit trends and periodicities caused by the regularity of the underlying processes in CPS. To tackle such limitation, we propose Time-aware Graph Structure Learning (TagSL), which extracts time-aware correlations among time series by measuring the interaction of node and time representations in high-dimensional spaces. Notably, we introduce time discrepancy learning that utilizes contrastive learning with distance-based regularization terms to constrain learned spatial correlations to a trend sequence. Additionally, we propose a periodic discriminant function to enable the capture of periodic changes from the state of nodes. Next, we present a Graph Convolution-based Gated Recurrent Unit (GCGRU) that jointly captures spatial and temporal dependencies while learning time-aware and node-specific patterns. Finally, we introduce a unified framework named Time-aware Graph Convolutional Recurrent Network (TGCRN), combining TagSL, and GCGRU in an encoder-decoder architecture for multi-step spatio-temporal forecasting. We report on experiments with TGCRN and popular existing approaches on five real-world datasets, thus providing evidence that TGCRN is capable of advancing the state-of-the-art. We also cover a detailed ablation study and visualization analysis, offering detailed insight into the effectiveness of time-aware structure learning.

Sequential modelling with self-attention has achieved cutting edge performances in natural language processing. With advantages in model flexibility, computation complexity and interpretability, self-attention is gradually becoming a key component in event sequence models. However, like most other sequence models, self-attention does not account for the time span between events and thus captures sequential signals rather than temporal patterns. Without relying on recurrent network structures, self-attention recognizes event orderings via positional encoding. To bridge the gap between modelling time-independent and time-dependent event sequence, we introduce a functional feature map that embeds time span into high-dimensional spaces. By constructing the associated translation-invariant time kernel function, we reveal the functional forms of the feature map under classic functional function analysis results, namely Bochner's Theorem and Mercer's Theorem. We propose several models to learn the functional time representation and the interactions with event representation. These methods are evaluated on real-world datasets under various continuous-time event sequence prediction tasks. The experiments reveal that the proposed methods compare favorably to baseline models while also capturing useful time-event interactions.

This paper gives a short overview of time representations in current symbol grounding architectures. Furthermore we report on a recently developed embodied language acquisition system that acquires object words from a linguistically unconstrained human-robot dialogue. Conceptual issues in future development of the system towards the acquisition of action words will be discussed briefly.