igt
GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion
Luo, Kangyang, Bai, Yuzhuo, Gao, Cheng, Si, Shuzheng, Shen, Yingli, Liu, Zhu, Wang, Zhitong, Kong, Cunliang, Li, Wenhao, Huang, Yufei, Tian, Ye, Xiong, Xuantang, Han, Lei, Sun, Maosong
Knowledge Graph Completion (KGC), which aims to infer missing or incomplete facts, is a crucial task for KGs. However, integrating the vital structural information of KGs into Large Language Models (LLMs) and outputting predictions deterministically remains challenging. To address this, we propose a new method called GLTW, which encodes the structural information of KGs and merges it with LLMs to enhance KGC performance. Specifically, we introduce an improved Graph Transformer (iGT) that effectively encodes subgraphs with both local and global structural information and inherits the characteristics of language model, bypassing training from scratch. Also, we develop a subgraph-based multi-classification training objective, using all entities within KG as classification objects, to boost learning efficiency.Importantly, we combine iGT with an LLM that takes KG language prompts as input.Our extensive experiments on various KG datasets show that GLTW achieves significant performance gains compared to SOTA baselines.
GlossLM: Multilingual Pretraining for Low-Resource Interlinear Glossing
Ginn, Michael, Tjuatja, Lindia, He, Taiqi, Rice, Enora, Neubig, Graham, Palmer, Alexis, Levin, Lori
Language documentation projects often involve the creation of annotated text in a format such as interlinear glossed text (IGT), which captures fine-grained morphosyntactic analyses in a morpheme-by-morpheme format. However, there are few existing resources providing large amounts of standardized, easily accessible IGT data, limiting their applicability to linguistic research, and making it difficult to use such data in NLP modeling. We compile the largest existing corpus of IGT data from a variety of sources, covering over 450k examples across 1.8k languages, to enable research on crosslingual transfer and IGT generation. We normalize much of our data to follow a standard set of labels across languages. Furthermore, we explore the task of automatically generating IGT in order to aid documentation projects. As many languages lack sufficient monolingual data, we pretrain a large multilingual model on our corpus. We demonstrate the utility of this model by finetuning it on monolingual corpora, outperforming SOTA models by up to 6.6%. We will make our pretrained model and dataset available through Hugging Face, as well as provide access through a web interface for use in language documentation efforts.
Inductive Graph Transformer for Delivery Time Estimation
Zhou, Xin, Wang, Jinglong, Liu, Yong, Wu, Xingyu, Shen, Zhiqi, Leung, Cyril
Providing accurate estimated time of package delivery on users' purchasing pages for e-commerce platforms is of great importance to their purchasing decisions and post-purchase experiences. Although this problem shares some common issues with the conventional estimated time of arrival (ETA), it is more challenging with the following aspects: 1) Inductive inference. Models are required to predict ETA for orders with unseen retailers and addresses; 2) High-order interaction of order semantic information. Apart from the spatio-temporal features, the estimated time also varies greatly with other factors, such as the packaging efficiency of retailers, as well as the high-order interaction of these factors. In this paper, we propose an inductive graph transformer (IGT) that leverages raw feature information and structural graph data to estimate package delivery time. Different from previous graph transformer architectures, IGT adopts a decoupled pipeline and trains transformer as a regression function that can capture the multiplex information from both raw feature and dense embeddings encoded by a graph neural network (GNN). In addition, we further simplify the GNN structure by removing its non-linear activation and the learnable linear transformation matrix. The reduced parameter search space and linear information propagation in the simplified GNN enable the IGT to be applied in large-scale industrial scenarios. Experiments on real-world logistics datasets show that our proposed model can significantly outperform the state-of-the-art methods on estimation of delivery time. The source code is available at: https://github.com/enoche/IGT-WSDM23.
IGT coding and robotics camp give youth a tech boost - Trinidad and Tobago Newsday
The first staging of the IGT Coding and Robotics Rock! Camp engaged youth participants from the International Game Technology (IGT) After School Advantage (ASA) Centers in Trinidad and Tobago, who are now better equipped with introductory tech skills. The IGT-sponsored camp was done in collaboration with Mona Geoinformatics Institute (MGI) located at The University of the West Indies, Mona, Jamaica, through its flagship philanthropic initiative, the IGT After School Advantage Programme. Introductory lessons in various aspects of coding and robotics were presented by the highly-skilled MGI team, assisted by Dr Nalini Ramsawak-Jodha who is an education specialist and STEM educator at UWI, St Augustine campus. Ramsawak-Jodha structured and aligned the curriculum to suit the needs of participants ranging from 11-18 years, said a media release. The virtual camp was simultaneously held in August in five of the territories where IGT operates –Barbados, Jamaica, St Kitts, St Maarten and TT.
Interferometric Graph Transform: a Deep Unsupervised Graph Representation
We propose the Interferometric Graph Transform (IGT), which is a new class of deep unsupervised graph convolutional neural network for building graph representations. Our first contribution is to propose a generic, complex-valued spectral graph architecture obtained from a generalization of the Euclidean Fourier transform. We show that our learned representation consists of both discriminative and invariant features, thanks to a novel greedy concave objective. From our experiments, we conclude that our learning procedure exploits the topology of the spectral domain, which is normally a flaw of spectral methods, and in particular our method can recover an analytic operator for vision tasks. We test our algorithm on various and challenging tasks such as image classification (MNIST, CIFAR-10), community detection (Authorship, Facebook graph) and action recognition from 3D skeletons videos (SBU, NTU), exhibiting a new state-of-the-art in spectral graph unsupervised settings.
Enabling app security by design, with IBM The MSP Hub
When software is at the core of your business and your customers present a high-profile target for cybercrime, application security becomes a driving factor. IGT's products and solutions enable players to experience their favourite games across all channels and regulated segments. But innovating to stay ahead of changing player requirements means new code is continually being turned out – by thousands of developers. The challenge was how IGT would meet business needs for speed-to-market while maintaining thorough application security throughout the development process. By employing the help of IBM Business Partner HCL, and the machine learning and AI-based capabilities of IBM Security AppScan, IGT can now rule out hundreds of'false positive' security issues, enabling them to focus on critical sections of code that really matter.
Reducing the variance in online optimization by transporting past gradients
Arnold, Sébastien M. R., Manzagol, Pierre-Antoine, Babanezhad, Reza, Mitliagkas, Ioannis, Roux, Nicolas Le
Most stochastic optimization methods use gradients once before discarding them. While variance reduction methods have shown that reusing past gradients can be beneficial when there is a finite number of datapoints, they do not easily extend to the online setting. One issue is the staleness due to using past gradients. We propose to correct this staleness using the idea of implicit gradient transport (IGT) which transforms gradients computed at previous iterates into gradients evaluated at the current iterate without using the Hessian explicitly. In addition to reducing the variance and bias of our updates over time, IGT can be used as a drop-in replacement for the gradient estimate in a number of well-understood methods such as heavy ball or Adam. We show experimentally that it achieves state-of-the-art results on a wide range of architectures and benchmarks. Additionally, the IGT gradient estimator yields the optimal asymptotic convergence rate for online stochastic optimization in the restricted setting where the Hessians of all component functions are equal.