Oceania
MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularization
Duan, Yue, Zhao, Zhen, Qi, Lei, Wang, Lei, Zhou, Luping, Shi, Yinghuan, Gao, Yang
The core issue in semi-supervised learning (SSL) lies in how to effectively leverage unlabeled data, whereas most existing methods tend to put a great emphasis on the utilization of high-confidence samples yet seldom fully explore the usage of low-confidence samples. In this paper, we aim to utilize low-confidence samples in a novel way with our proposed mutex-based consistency regularization, namely MutexMatch. Specifically, the high-confidence samples are required to exactly predict "what it is" by conventional True-Positive Classifier, while the low-confidence samples are employed to achieve a simpler goal -- to predict with ease "what it is not" by True-Negative Classifier. In this sense, we not only mitigate the pseudo-labeling errors but also make full use of the low-confidence unlabeled data by consistency of dissimilarity degree. MutexMatch achieves superior performance on multiple benchmark datasets, i.e., CIFAR-10, CIFAR-100, SVHN, STL-10, mini-ImageNet and Tiny-ImageNet. More importantly, our method further shows superiority when the amount of labeled data is scarce, e.g., 92.23% accuracy with only 20 labeled data on CIFAR-10. Our code and model weights have been released at https://github.com/NJUyued/MutexMatch4SSL.
An Efficient Drug-Drug Interactions Prediction Technology for Molecularly Intelligent Manufacturing
Gao, Peng, Gao, Feng, Ni, Jian-Cheng
Drug-Drug interactions (DDIs) are considered as the unwanted side, causing patients irreparable side effects [1] due to the concurrent consumption of two or more drugs [2-4]. As a fundamental research issue in the field of molecular biology, DDIs has received great attention. The resources of molecular biology, such as research literature, structure diagrams, and interactive genes analyses, have the substantial increments [5-7]. Researchers are facing enormous amount of materials to review, which requires considerable energy to investigate and organize crucial information with difficulty. The achievements of Natural Language Processing (NLP) could accelerate this procedure and improve the reliability for their further research, in the meantime, the linguistics could take advantages of the large, well-curated resources as well [5], benefiting the both groups. Considering the importance of DDIs in humans health, industry and the economy, and the substantial amount of cost and time of experimental approaches [1, 8], it is appropriate to introduce the automatic methodologies into this field. In the work by Zhang et al. [9], they argue that Information Extraction (IE) applications, such as discovering DDIs, must move beyond a scenario where relevant information is coherently and explicitly stated within a single document.
Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models
Kim, Najoung, Linzen, Tal, Smolensky, Paul
Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by composing known parts. Several benchmarks exploit distributional control across training and test to gauge compositional generalization, where certain lexical items only occur in limited contexts during training. While recent work using these benchmarks suggests that pretrained models achieve impressive generalization performance, we argue that exposure to pretraining data may break the aforementioned distributional control. Using the COGS benchmark of Kim and Linzen (2020), we test two modified evaluation setups that control for this issue: (1) substituting context-controlled lexical items with novel character sequences, and (2) substituting them with special tokens represented by novel embeddings. We find that both of these setups lead to lower generalization performance in T5 (Raffel et al., 2020), suggesting that previously reported results have been overestimated due to uncontrolled lexical exposure during pretraining. The performance degradation is more extreme with novel embeddings, and the degradation increases with the amount of pretraining data, highlighting an interesting case of inverse scaling.
A Seven-Layer Model for Standardising AI Fairness Assessment
Agarwal, Avinash, Agarwal, Harsh
Problem statement: Standardisation of AI fairness rules and benchmarks is challenging because AI fairness and other ethical requirements depend on multiple factors such as context, use case, type of the AI system, and so on. In this paper, we elaborate that the AI system is prone to biases at every stage of its lifecycle, from inception to its usage, and that all stages require due attention for mitigating AI bias. We need a standardised approach to handle AI fairness at every stage. Gap analysis: While AI fairness is a hot research topic, a holistic strategy for AI fairness is generally missing. Most researchers focus only on a few facets of AI model-building. Peer review shows excessive focus on biases in the datasets, fairness metrics, and algorithmic bias. In the process, other aspects affecting AI fairness get ignored. The solution proposed: We propose a comprehensive approach in the form of a novel seven-layer model, inspired by the Open System Interconnection (OSI) model, to standardise AI fairness handling. Despite the differences in the various aspects, most AI systems have similar model-building stages. The proposed model splits the AI system lifecycle into seven abstraction layers, each corresponding to a well-defined AI model-building or usage stage. We also provide checklists for each layer and deliberate on potential sources of bias in each layer and their mitigation methodologies. This work will facilitate layer-wise standardisation of AI fairness rules and benchmarking parameters.
Novel Deep Learning Framework For Bovine Iris Segmentation
Yoon, Heemoon, Park, Mira, Lee, Sang-Hee
Iris segmentation is the initial step to identify biometric of animals to establish a traceability system of livestock. In this study, we propose a novel deep learning framework for pixel-wise segmentation with minimum use of annotation labels using BovineAAEyes80 public dataset. In the experiment, U-Net with VGG16 backbone was selected as the best combination of encoder and decoder model, demonstrating a 99.50% accuracy and a 98.35% Dice coefficient score. Remarkably, the selected model accurately segmented corrupted images even without proper annotation data. This study contributes to the advancement of the iris segmentation and the development of a reliable DNNs training framework.
Multi-branch Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction
Ekanayake, Mevan, Pawar, Kamlesh, Harandi, Mehrtash, Egan, Gary, Chen, Zhaolin
Global correlations are widely seen in human anatomical structures due to similarity across tissues and bones. These correlations are reflected in magnetic resonance imaging (MRI) scans as a result of close-range proton density and T1/T2 parameters. Furthermore, to achieve accelerated MRI, k-space data are undersampled which causes global aliasing artifacts. Convolutional neural network (CNN) models are widely utilized for accelerated MRI reconstruction, but those models are limited in capturing global correlations due to the intrinsic locality of the convolution operation. The self-attention-based transformer models are capable of capturing global correlations among image features, however, the current contributions of transformer models for MRI reconstruction are minute. The existing contributions mostly provide CNN-transformer hybrid solutions and rarely leverage the physics of MRI. In this paper, we propose a physics-based stand-alone (convolution free) transformer model titled, the Multi-head Cascaded Swin Transformers (McSTRA) for accelerated MRI reconstruction. McSTRA combines several interconnected MRI physics-related concepts with the transformer networks: it exploits global MR features via the shifted window self-attention mechanism; it extracts MR features belonging to different spectral components separately using a multi-head setup; it iterates between intermediate de-aliasing and k-space correction via a cascaded network with data consistency in k-space and intermediate loss computations; furthermore, we propose a novel positional embedding generation mechanism to guide self-attention utilizing the point spread function corresponding to the undersampling mask. Our model significantly outperforms state-of-the-art MRI reconstruction methods both visually and quantitatively while depicting improved resolution and removal of aliasing artifacts.
Automatic Semantic Modeling for Structural Data Source with the Prior Knowledge from Knowledge Base
Xu, Jiakang, Mayer, Wolfgang, Zhang, HongYu, He, Keqing, Feng, Zaiwen
A critical step in sharing semantic content online is to map the structural data source to a public domain ontology. This problem is denoted as the Relational-To-Ontology Mapping Problem (Rel2Onto). A huge effort and expertise are required for manually modeling the semantics of data. Therefore, an automatic approach for learning the semantics of a data source is desirable. Most of the existing work studies the semantic annotation of source attributes. However, although critical, the research for automatically inferring the relationships between attributes is very limited. In this paper, we propose a novel method for semantically annotating structured data sources using machine learning, graph matching and modified frequent subgraph mining to amend the candidate model. In our work, Knowledge graph is used as prior knowledge. Our evaluation shows that our approach outperforms two state-of-the-art solutions in tricky cases where only a few semantic models are known.
AIhub monthly digest: December 2022 โ AI around the world, teleoperation, and multilingual translation
Welcome to our December 2022 monthly digest, where you can catch up with any AIhub stories you may have missed, get the low-down on recent events, and much more. This month, we hear from best paper award winners at ICIP and NeurIPS, and find out more about teleoperation, multilingual translation, and quality-diversity algorithms. We also have exciting news, in the form of a new focus series. We're delighted to announce the launch of our new focus series on AI around the world, where we cover exciting applications of AI across the globe. To kick off the series, we spoke with Rose Nakasi.
Lead Data Scientist (Machine Learning) at Employment Hero - Australia - Remote
Employment Hero is an Australian tech unicorn - valued at over $1 billion. Our world-class software is the easiest way for small to medium-sized businesses to manage HR, payroll, employee engagement, and benefits. We've grown 100% year on year since inception, and expanded globally in late 2020. We now service over 80,000 businesses and we will continue to grow rapidly in 2022 and beyond. We believe in distributed employment and take a Remote First approach with our people.
Rumour detection using graph neural network and oversampling in benchmark Twitter dataset
Patel, Shaswat, Bansal, Prince, Kaur, Preeti
Recently, online social media has become a primary source for new information and misinformation or rumours. In the absence of an automatic rumour detection system the propagation of rumours has increased manifold leading to serious societal damages. In this work, we propose a novel method for building automatic rumour detection system by focusing on oversampling to alleviating the fundamental challenges of class imbalance in rumour detection task. Our oversampling method relies on contextualised data augmentation to generate synthetic samples for underrepresented classes in the dataset. The key idea exploits selection of tweets in a thread for augmentation which can be achieved by introducing a non-random selection criteria to focus the augmentation process on relevant tweets. Furthermore, we propose two graph neural networks(GNN) to model non-linear conversations on a thread. To enhance the tweet representations in our method we employed a custom feature selection technique based on state-of-the-art BERTweet model. Experiments of three publicly available datasets confirm that 1) our GNN models outperform the the current state-of-the-art classifiers by more than 20%(F1-score); 2) our oversampling technique increases the model performance by more than 9%;(F1-score) 3) focusing on relevant tweets for data augmentation via non-random selection criteria can further improve the results; and 4) our method has superior capabilities to detect rumours at very early stage.