Dreaming is All You Need

In the current digital age, the ability to accurately classify large datasets has become of paramount importance across a myriad of fields, including computer vision (CV) [26, 37, 38, 11], natural language processing (NLP) [30, 33, 8, 4], bioinformatics [21], etc. The blossoming of artificial intelligence and deep learning has greatly facilitated the handling of complex classification tasks. Deep learning's capacity to sift through multitudes of variables, discern patterns, and extract key features has led to impressive breakthroughs in numerous applications, from image recognition and voice recognition to disease prediction [20]. The groundbreaking convolutional neural networks (ConvNets), such as ResNet[16] and EfficientNet[39], have emerged as dominant architectures in computer vision, with ResNet addressing the vanishing gradient issue through deep residual networks and enabling deeper models without performance loss, while EfficientNet introduced a compound scaling method that scales depth, width, and resolution, enhancing both efficiency and accuracy. These models have set new benchmarks across various datasets and have been pivotal in applications such as autonomous driving and advanced image recognition, reshaping how machines interpret visual data. Meanwhile, the success of pre-trained unsupervised Transformers [41] like ViT [11] for vision tasks and BERT [8] has shown that using primarily standard Transformer layers can achieve significant performance in downstream applications, reaching levels comparable to previous state-of-the-art neural networks and suggesting that Transformers may offer greater scalability across diverse domains. Transformers have demonstrated superior model capabilities but often suffer from poor generalization when compared to chain-like networks due to a lack of appropriate inductive bias [42]. Recent research has focused on hybrid methods that combine the structures of both to retain their respective advantages [10, 42, 7, 19].