smx
Fighter: Unveiling the Graph Convolutional Nature of Transformers in Time Series Modeling
Zhang, Chen, Bu, Weixin, Xu, Wendong, Yu, Runsheng, Wu, Yik-Chung, Wong, Ngai
Transformers have achieved remarkable success in time series modeling, yet their internal mechanisms remain opaque. This work demystifies the Transformer encoder by establishing its fundamental equivalence to a Graph Convolutional Network (GCN). We show that in the forward pass, the attention distribution matrix serves as a dynamic adjacency matrix, and its composition with subsequent transformations performs computations analogous to graph convolution. Moreover, we demonstrate that in the backward pass, the update dynamics of value and feed-forward projections mirror those of GCN parameters. Building on this unified theoretical reinterpretation, we propose \textbf{Fighter} (Flexible Graph Convolutional Transformer), a streamlined architecture that removes redundant linear projections and incorporates multi-hop graph aggregation. This perspective yields an explicit and interpretable representation of temporal dependencies across different scales, naturally expressed as graph edges. Experiments on standard forecasting benchmarks confirm that Fighter achieves competitive performance while providing clearer mechanistic interpretability of its predictions.
SMX: Sequential Monte Carlo Planning for Expert Iteration
Macfarlane, Matthew V, Toledo, Edan, Byrne, Donal, Singh, Siddarth, Duckworth, Paul, Laterre, Alexandre
Developing agents that can leverage planning abilities during their decision and learning processes is critical to the advancement of Artificial Intelligence. Recent works have demonstrated the effectiveness of combining tree-based search methods and self-play learning mechanisms. Yet, these methods typically face scaling challenges due to the sequential nature of their search. While practical engineering solutions can partly overcome this, they still demand extensive computational resources, which hinders their applicability. In this paper, we introduce SMX, a model-based planning algorithm that utilises scalable Sequential Monte Carlo methods to create an effective self-learning mechanism. Grounded in the theoretical framework of control as inference, SMX benefits from robust theoretical underpinnings. Its sampling-based search approach makes it adaptable to environments with both discrete and continuous action spaces. Furthermore, SMX allows for high parallelisation and can run on hardware accelerators to optimise computing efficiency. SMX demonstrates a statistically significant improvement in performance compared to AlphaZero, as well as demonstrating its performance as an improvement operator for a model-free policy, matching or exceeding top model-free methods across both continuous and discrete environments.
Machine learning for the selection of carbon-based materials for tetracycline and sulfamethoxazole adsorption
Antiobiotics adsorption on carbon-based materials was modeled by machine learning. Random forest showed best prediction accuracy than GBT and ANN. Impact tendencies of SBET, pHsol, C0 on adsorption were similar for TC and SMX. Chemical compositions and pHpzc of CBMs showed different influences on TC and SMX. Antibiotics as emerging pollutants have attracted extensive attention due to their ecotoxicity and persistence in the environment.