Substitute (S): the item is somewhat relevant: it fails to fulfill some aspects of the query, but the item can be used as In the Amazon KDD Cup 2022, we aim to apply natural language a functional substitute; processing methods to improve the quality of search results that can Complement (C): the item does not fulfill the query but significantly enhance user experience and engagement with search could be used in combination with an exact item; engines for e-commerce. We discuss our practical solution for this Irrelevant (I): the item is irrelevant, or it fails to fulfill a competition, ranking 6th in task one, 2nd in task two, and 2nd in central aspect of the query.

The jellyfish sighting app, MedusApp, recently incorporated artificial intelligence (AI) to automatically recognize different species of jellyfish. Until now, this app only required users to select the species of jellyfish from a catalog provided; now the user can upload photos and have the species automatically identified before uploading them to the app for publication. MedusApp, which is freely available in Spanish and English for both Android and iPhone, has been developed by researchers from the University of Alicante (UA) and two computer scientists from the Polytechnic University of Valencia (UPV), in collaboration with the CIBER of Diseases (CIBERES) and the Immunoallergy Laboratory of the Fundación Jiménez Díaz Health Research Institute (IIS-FJD). Since its launch in 2018, the platform has amassed more than 100,000 downloads and 6,000 jellyfish sightings. "Thanks to the collaboration of citizens and their sightings, we have been able to train the AI software with several thousand real photos to generate a mathematical model with a total of 25 species, that will ultimately help the app automatically recognize the most common jellyfish," a novelty update that the programmers from the UPV Eduardo Blasco and Ramón Palacios have highlighted.

Recently, numerous efficient Transformers have been proposed to reduce the quadratic computational complexity of standard Transformers caused by the Softmax attention. However, most of them simply swap Softmax with an efficient attention mechanism without considering the customized architectures specially for the efficient attention. In this paper, we argue that the handcrafted vanilla Transformer architectures for Softmax attention may not be suitable for efficient Transformers. To address this issue, we propose a new framework to find optimal architectures for efficient Transformers with the neural architecture search (NAS) technique. The proposed method is validated on popular machine translation and image classification tasks. We observe that the optimal architecture of the efficient Transformer has the reduced computation compared with that of the standard Transformer, but the general accuracy is less comparable. It indicates that the Softmax attention and efficient attention have their own distinctions but neither of them can simultaneously balance the accuracy and efficiency well. This motivates us to mix the two types of attention to reduce the performance imbalance. Besides the search spaces that commonly used in existing NAS Transformer approaches, we propose a new search space that allows the NAS algorithm to automatically search the attention variants along with architectures. Extensive experiments on WMT' 14 En-De and CIFAR-10 demonstrate that our searched architecture maintains comparable accuracy to the standard Transformer with notably improved computational efficiency.

In this work we introduce a new transformer architecture called SparseDistilBERT (SDBERT), which is a combination of sparse attention and knowledge distillantion (KD). We implemented sparse attention mechanism to reduce quadratic dependency on input length to linear. In addition to reducing computational complexity of the model, we used knowledge distillation (KD). We were able to reduce the size of BERT model by 60% while retaining 97% performance and it only took 40% of time to train.

The rapid spread of COVID-19 disease has had a significant impact on the world. In this paper, we study COVID-19 data interpretation and visualization using open-data sources for 351 cities and towns in Massachusetts from December 6, 2020 to September 25, 2021. Because cities are embedded in rather complex transportation networks, we construct the spatio-temporal dynamic graph model, in which the graph attention neural network is utilized as a deep learning method to learn the pandemic transition probability among major cities in Massachusetts. Using the spectral graph wavelet transform (SGWT), we process the COVID-19 data on the dynamic graph, which enables us to design effective tools to analyze and detect spatio-temporal patterns in the pandemic spreading. We design a new node classification method, which effectively identifies the anomaly cities based on spectral graph wavelet coefficients. It can assist administrations or public health organizations in monitoring the spread of the pandemic and developing preventive measures. Unlike most work focusing on the evolution of confirmed cases over time, we focus on the spatio-temporal patterns of pandemic evolution among cities. Through the data analysis and visualization, a better understanding of the epidemiological development at the city level is obtained and can be helpful with city-specific surveillance.

Kernel logistic regression (KLR) is a conventional nonlinear classifier in machine learning. With the explosive growth of data size, the storage and computation of large dense kernel matrices is a major challenge in scaling KLR. Even the nystr\"{o}m approximation is applied to solve KLR, it also faces the time complexity of $O(nc^2)$ and the space complexity of $O(nc)$, where $n$ is the number of training instances and $c$ is the sampling size. In this paper, we propose a fast Newton method efficiently solving large-scale KLR problems by exploiting the storage and computing advantages of multilevel circulant matrix (MCM). Specifically, by approximating the kernel matrix with an MCM, the storage space is reduced to $O(n)$, and further approximating the coefficient matrix of the Newton equation as MCM, the computational complexity of Newton iteration is reduced to $O(n \log n)$. The proposed method can run in log-linear time complexity per iteration, because the multiplication of MCM (or its inverse) and vector can be implemented the multidimensional fast Fourier transform (mFFT). Experimental results on some large-scale binary-classification and multi-classification problems show that the proposed method enables KLR to scale to large scale problems with less memory consumption and less training time without sacrificing test accuracy.

The performance of Automatic Speech Recognition systems (ASRs) has increased significantly with the development of modern neural network topologies and the use of massive amount of data to train the models [1]. Although the accuracy of recent models improved for high-resource languages, such as English, the development of ASR models in other languages is still a difficult task using the same technologies [2, 3]. In this scenario, Self-Supervised Learning (SSL), a method in which representations with semantic information are learned by using unlabelled data, emerged as an important advance, allowing the training of deeper models using less labelled data [4, 5]. In this line of work, this paper explores the use of the Wav2vec 2.0 [6], a framework for self-supervised learning of discrete representations from raw audio data. Wav2vec 2.0 (Figure 1) is inspired by previous works in unsupervised pre-training for speech recognition, that is, Wav2vec [7] and Vq-Wav2vec [4].

Advanced music recommendation systems are being introduced along with the development of machine learning. However, it is essential to design a music recommendation system that can increase user satisfaction by understanding users' music tastes, not by the complexity of models. Although several studies related to music recommendation systems exploiting negative preferences have shown performance improvements, there was a lack of explanation on how they led to better recommendations. In this work, we analyze the role of negative preference in users' music tastes by comparing music recommendation models with contrastive learning exploiting preference (CLEP) but with three different training strategies - exploiting preferences of both positive and negative (CLEP-PN), positive only (CLEP-P), and negative only (CLEP-N). We evaluate the effectiveness of the negative preference by validating each system with a small amount of personalized data obtained via survey and further illuminate the possibility of exploiting negative preference in music recommendations. Our experimental results show that CLEP-N outperforms the other two in accuracy and false positive rate. Furthermore, the proposed training strategies produced a consistent tendency regardless of different types of front-end musical feature extractors, proving the stability of the proposed method.

The main advantage of using biometric information over traditional methods is that instead of requiring information that the user should know or possess (password, codes, PIN, etc.), they use characteristics that univocally and biologically define the users (fingerprints, iris, face, etc.). In particular, these characteristics are universal (all users can be measured), singular (each user has its own measures), permanent in time and context, and can be quantitatively measured [33]. Soft biometrics can be divided into two groups: physical and behavioural biometrics. Techniques of the first category use physical characteristics like face, iris, and fingerprint for their tasks [4], whereas techniques of the second one, use information extracted from user behaviours such as signature, voice, and keyboard typing [39]. Among the physical biometrics, face [15] and fingerprint [2] methodology have been the most explored, and have already been used in many real-world applications such as airport scanners, banking, military access control, smartphones or forensics [7, 36]. However, in the last decade, the use of iris has begun to attract interest in applications such as gender classification [27], iris liveness detection [8], border control [45] and citizen confirmation [22]. In fact, iris biometric represents a secure biometric with low forgery and error rates due to its highly certain features [43]. Furthermore, this biometric information is usually combined with Artificial Intelligence (AI) and Machine Learning techniques (ML) in order to implement user identification and verification systems.

As enterprises move from running ad hoc machine learning (ML) models to using AI/ML to transform their business at scale, the adoption of ML Operations (MLOps) becomes inevitable. As shown in the following figure, the ML lifecycle begins with framing a business problem as an ML use case followed by a series of phases, including data preparation, feature engineering, model building, deployment, continuous monitoring, and retraining. For many enterprises, a lot of these steps are still manual and loosely integrated with each other. Therefore, it's important to automate the end-to-end ML lifecycle, which enables frequent experiments to drive better business outcomes. Data preparation is one of the crucial steps in this lifecycle, because the ML model's accuracy depends on the quality of the training dataset.