One Transformer for All Time Series: Representing and Training with Time-Dependent Heterogeneous Tabular Data

Despite the success of Deep Learning methods in different areas of Artificial Intelligence (AI), such as, for instance, Natural Language Processing, Computer Vision, Audio Processing, Robotics, etc., the use of deep networks to represent tabular data is so far largely under explored. However, tabular data have a large application interest, since many public institutions or commercial/industrial companies represent their knowledge using datasets of "tables" [1]. For instance, bank data, clinical data, commercial data, etc., are often provided as a list of attributes (field names) and corresponding values (field values) for each represented entity (sample). As reported in [2], over 65% of the datasets in the Google Dataset Search platform contain tabular files in either CSV or XLS formats. Particularly interesting is the case of financial transactions, which, for instance, describe the sequence of (time dependent) transactions of a given bank client on her/his bank account.