mou
Mamba or Transformer for Time Series Forecasting? Mixture of Universals (MoU) Is All You Need
Peng, Sijia, Xiong, Yun, Zhu, Yangyong, Shen, Zhiqiang
Time series forecasting requires balancing short-term and long-term dependencies for accurate predictions. Existing methods mainly focus on long-term dependency modeling, neglecting the complexities of short-term dynamics, which may hinder performance. Transformers are superior in modeling long-term dependencies but are criticized for their quadratic computational cost. Mamba provides a near-linear alternative but is reported less effective in time series longterm forecasting due to potential information loss. Current architectures fall short in offering both high efficiency and strong performance for long-term dependency modeling. To address these challenges, we introduce Mixture of Universals (MoU), a versatile model to capture both short-term and long-term dependencies for enhancing performance in time series forecasting. MoU is composed of two novel designs: Mixture of Feature Extractors (MoF), an adaptive method designed to improve time series patch representations for short-term dependency, and Mixture of Architectures (MoA), which hierarchically integrates Mamba, FeedForward, Convolution, and Self-Attention architectures in a specialized order to model long-term dependency from a hybrid perspective. The proposed approach achieves state-of-the-art performance while maintaining relatively low computational costs. Extensive experiments on seven real-world datasets demonstrate the superiority of MoU. Code is available at https://github.com/lunaaa95/mou/.
Search and Learning for Unsupervised Text Generation New Faculty Highlights Extended Abstract
The following article is an extended abstract submitted as part of AAAI's New Faculty Highlights Program. With the advances of deep learning techniques, text generation is attracting increasing interest in the artificial intelligence (AI) commu- nity, because of its wide applications and because it is an essential component of AI. Traditional text generation systems are trained in a supervised way, requiring massive labeled parallel corpora. In this paper, I will introduce our recent work on search and learning ap- proaches to unsupervised text generation, where a heuristic objective function estimates the quality of a candidate sentence, and discrete search algorithms generate a sentence by maximizing the search objective. A machine learning model further learns from the search results to smooth out noise and improve efficiency.
When OT meets MoM: Robust estimation of Wasserstein Distance
Staerman, Guillaume, Laforgue, Pierre, Mozharovskyi, Pavlo, d'Alchรฉ-Buc, Florence
Issued from Optimal Transport, the Wasserstein distance has gained importance in Machine Learning due to its appealing geometrical properties and the increasing availability of efficient approximations. In this work, we consider the problem of estimating the Wasserstein distance between two probability distributions when observations are polluted by outliers. To that end, we investigate how to leverage Medians of Means (MoM) estimators to robustify the estimation of Wasserstein distance. Exploiting the dual Kantorovitch formulation of Wasserstein distance, we introduce and discuss novel MoM-based robust estimators whose consistency is studied under a data contamination model and for which convergence rates are provided. These MoM estimators enable to make Wasserstein Generative Adversarial Network (WGAN) robust to outliers, as witnessed by an empirical study on two benchmarks CIFAR10 and Fashion MNIST. Eventually, we discuss how to combine MoM with the entropy-regularized approximation of the Wasserstein distance and propose a simple MoM-based re-weighting scheme that could be used in conjunction with the Sinkhorn algorithm.
Chinese firm to help build artificial intelligence infrastructure in Ethiopia - Xinhua
A Chinese firm has signed a Memorandum of Understanding (MoU) with Ethiopia authorities on establishing a National Artificial Intelligence Infrastructure (NAIF) in Ethiopia, reported state media outlet Ethiopia News Agency (ENA) on Saturday. The MoU was signed between Ethiopia Innovation and Technology State Minister, Sisay Tola and Chen Kuan, the founder and CEO of Chinese firm Infervision Technology Corporation in Ethiopia's capital Addis Ababa on Friday evening, reported ENA. Ethiopia hopes the partnership with Infervision will boost the technological capacity of its education, health care and medical services. Ethiopia also hopes the partnership will facilitate a platform for exchange of ideas and investment opportunities between enterprises of both countries in various sectors including energy, textile, agriculture, construction and information technology. Ethiopia and China have recently signed various agreements in the Information Communication and Technology (ICT), as Ethiopia looks to modernize its largely agrarian economy.
Department of Energy puts AI to biomedical use - FedScoop
The Department of Energy wants to lend its world-class artificial intelligence technology for private sector biomedical and public health research to develop better treatments for brain diseases and disorders. Secretary of Energy Rick Perry and Weill Family Foundation founder Sandy Weill signed a memorandum of understanding this week establishing a public-private partnership seeking supercomputing breakthroughs addressing neurological disorders like traumatic brain injuries. Research could range from better understanding brain function to finding new ways to treat, prevent and repair damage done by brain diseases. "By signing this MOU, we're collaborating at the critical nexus of leading-edge technology, our own national labs, world class capabilities at [University of California, San Francisco] -- the research excellence and health care expertise that's there," Perry said. "I know [University of California, Berkeley is going to be involved with this as well, and there's just this vast potential of philanthropy from the private sector."
Going deep in clustering high-dimensional data: deep mixtures of unigrams for uncovering topics in textual data
Anderlucci, Laura, Viroli, Cinzia
They can be basically defined as a multi-layer stack of algorithms or modules able to gradually learn a huge number of parameters in an architecture composed by multiple nonlinear transformations (LeCun et al., 2015). Typically, and for historical reasons, a structure for deep learning is identified with advanced neural networks: deep Feed Forward, Recurrent, Auto-encoder, Convolution neural networks are very effective and used algorithms of deep learning (Schmidhuber, 2015). They demonstrated to be particularly successful in supervised classification problems arising in several fields such as image and speech recognition, gene expression data, topic classification. When the aim is uncovering unknown classes in a unsupervised classification perspective, important methods of deep learning have been developed along the lines of mixture modeling, because of their ability to decompose a heterogeneous collection of units into a finite number of subgroups with homogeneous structures (Fraley and Raftery, 2002; McLachlan and Peel, 2000). In this direction, van den Oord and Schrauwen (2014) proposed Multilayer Gaussian Mixture Models for modeling natural images; Tang et al. (2012) defined deep mixture of factor analyzers with a greedy layer-wise learning algorithm able to learn each layer at a time. Viroli and McLachlan (2019) developed a general framework for Deep Gaussian mixture models that generalizes and encompasses the previous strategies and several flexible model-based clustering methods such as mixtures of mixture models (Li, 2005), mixtures of Factor Analyzers (McLachlan et al., 2003), mixtures of factor analyzers with common factor loadings (Baek et al., 2010), heteroscedastic factor mixture analysis (Montanari and Viroli, 2010) and mixtures of factor mixture analyzers introduced by Viroli (2010). A general'take-home-message' coming from the existing deep clustering strategies is that deep methods vs shallow ones appear to be very efficient and powerful tools especially for complex high-dimensional data; on the contrary, for simple and small data structures, a deep learning strategy cannot improve performance of simpler and conventional methods or, to better say, it is like to use a'sledgehammer to crack a nut'. The motivating problem behind this work derives from ticket data (i.e.
Invest India and UAE Govt. To Jointly Work on Artificial Intelligence
Invest India and the UAE Minister for Artificial Intelligence (AI) signed a Memorandum of Understanding (MoU) for India โ UAE Artificial Intelligence Bridge in New Delhi. This partnership will generate an estimated USD 20 billion in economic benefits during the next decade for both countries. The MoU will spur development across areas like Blockchain, AI and Analytics as data and processing will be a catalyst for innovation and business growth and serve as the backbone of more effective and efficient service delivery systems. By 2035 AI can potentially add USD 957 billion to the Indian economy. The MoU was signed in the presence of Minister of Commerce & Industry and Civil Aviation, Suresh Prabhu and H.E. Ahmad Sultan Al Falahi, Minister Plenipotentiary โ Commercial Attache, UAE Embassy at the India leg of GovHack series of World Government Summit.
The increasing prevalence of artificial intelligence
YOU'VE heard of it in movies or in passing conversations. Maybe your workplace uses it, or you're considering using it yourself. As technology continues to make ripples across the workplace, AI has become increasingly prevalent. Through AI, companies are able to analyse large amounts of data, which will allow them to better engage with customers. Today, AI is easily accessible.