Goto

Collaborating Authors

 gtr


Understanding Oversquashing in GNNs through the Lens of Effective Resistance

Black, Mitchell, Wan, Zhengchao, Nayyeri, Amir, Wang, Yusu

arXiv.org Artificial Intelligence

Message passing graph neural networks (GNNs) are a popular learning architectures for graph-structured data. However, one problem GNNs experience is oversquashing, where a GNN has difficulty sending information between distant nodes. Understanding and mitigating oversquashing has recently received significant attention from the research community. In this paper, we continue this line of work by analyzing oversquashing through the lens of the effective resistance between nodes in the input graph. Effective resistance intuitively captures the ``strength'' of connection between two nodes by paths in the graph, and has a rich literature spanning many areas of graph theory. We propose to use total effective resistance as a bound of the total amount of oversquashing in a graph and provide theoretical justification for its use. We further develop an algorithm to identify edges to be added to an input graph to minimize the total effective resistance, thereby alleviating oversquashing. We provide empirical evidence of the effectiveness of our total effective resistance based rewiring strategies for improving the performance of GNNs.


Dr.ICL: Demonstration-Retrieved In-context Learning

Luo, Man, Xu, Xin, Dai, Zhuyun, Pasupat, Panupong, Kazemi, Mehran, Baral, Chitta, Imbrasaite, Vaiva, Zhao, Vincent Y

arXiv.org Artificial Intelligence

In-context learning (ICL), teaching a large language model (LLM) to perform a task with few-shot demonstrations rather than adjusting the model parameters, has emerged as a strong paradigm for using LLMs. While early studies primarily used a fixed or random set of demonstrations for all test queries, recent research suggests that retrieving semantically similar demonstrations to the input from a pool of available demonstrations results in better performance. This work expands the applicability of retrieval-based ICL approaches by demonstrating that even simple word-overlap similarity measures such as BM25 outperform randomly selected demonstrations. Furthermore, we extend the success of retrieval-based ICL to instruction-finetuned LLMs as well as Chain-of-Thought (CoT) prompting. For instruction-finetuned LLMs, we find that although a model has already seen the training data at training time, retrieving demonstrations from the training data at test time yields better results compared to using no demonstrations or random demonstrations. Last but not least, we train a task-specific demonstration retriever that outperforms off-the-shelf retrievers.


Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature

Thai, Katherine, Karpinska, Marzena, Krishna, Kalpesh, Ray, Bill, Inghilleri, Moira, Wieting, John, Iyyer, Mohit

arXiv.org Artificial Intelligence

Literary translation is a culturally significant task, but it is bottlenecked by the small number of qualified literary translators relative to the many untranslated works published around the world. Machine translation (MT) holds potential to complement the work of human translators by improving both training procedures and their overall efficiency. Literary translation is less constrained than more traditional MT settings since translators must balance meaning equivalence, readability, and critical interpretability in the target language. This property, along with the complex discourse-level context present in literary texts, also makes literary MT more challenging to computationally model and evaluate. To explore this task, we collect a dataset (Par3) of non-English language novels in the public domain, each aligned at the paragraph level to both human and automatic English translations. Using Par3, we discover that expert literary translators prefer reference human translations over machine-translated paragraphs at a rate of 84%, while state-of-the-art automatic MT metrics do not correlate with those preferences. The experts note that MT outputs contain not only mistranslations, but also discourse-disrupting errors and stylistic inconsistencies. To address these problems, we train a post-editing model whose output is preferred over normal MT output at a rate of 69% by experts. We publicly release Par3 at https://github.com/katherinethai/par3/ to spur future research into literary MT.


A Lightweight Graph Transformer Network for Human Mesh Reconstruction from 2D Human Pose

Zheng, Ce, Mendieta, Matias, Wang, Pu, Lu, Aidong, Chen, Chen

arXiv.org Artificial Intelligence

Existing deep learning-based human mesh reconstruction approaches have a tendency to build larger networks in order to achieve higher accuracy. Computational complexity and model size are often neglected, despite being key characteristics for practical use of human mesh reconstruction models (e.g. virtual try-on systems). In this paper, we present GTRS, a lightweight pose-based method that can reconstruct human mesh from 2D human pose. We propose a pose analysis module that uses graph transformers to exploit structured and implicit joint correlations, and a mesh regression module that combines the extracted pose feature with the mesh template to reconstruct the final human mesh. We demonstrate the efficiency and generalization of GTRS by extensive evaluations on the Human3.6M and 3DPW datasets. In particular, GTRS achieves better accuracy than the SOTA pose-based method Pose2Mesh while only using 10.2% of the parameters (Params) and 2.5% of the FLOPs on the challenging in-the-wild 3DPW dataset. Code will be publicly available.


Quantum computing: the next tech frontier for trade finance? Global Trade Review (GTR)

#artificialintelligence

The archaic, paper-based world of trade finance looks set to take a further leap into the digital future, as trade finance distribution platform Tradeteq begins a collaboration with the Singapore Management University (SMU) to explore quantum computing-based solutions for the industry. Supported by the Monetary Authority of Singapore (MAS) under its artificial intelligence and data analytics (AIDA) grant scheme, the research project, titled Exploring the advantages of a quantum system for machine learning applied to credit scoring, aims to build a predictive machine learning model implemented on a quantum computer in order to tackle inefficiencies in approving trade finance. "Currently, many small and-medium-sized businesses are unable to grow their companies due to a lack of funding as they are deemed'too risky' by current credit rating models," says Pang Hwee Hwa, dean of the SMU School of Information Systems. "With shorter processing time, more businesses could be scored and with greater accuracy thereby creating more trust and providing greater access to finance for companies than ever before." Quantum computing is still very much in its infancy, and the technology doesn't yet exist to build a large-scale quantum computer.


Singapore's national trade platform launches Tradeteq's AI-based credit rating system Global Trade Review (GTR)

#artificialintelligence

Singapore's digital national trade platform has launched Tradeteq's AI-based credit scoring system to enable its users to better assess counterparty risk in trade deals. The system is being used to leverage various data sources to provide thorough credit reports for users of Singapore's Networked Trade Platform (NTP), including data on each company in the supply chain as well as each receivable. The NTP is a digital national trade information management system backed by the Singaporean government, which aims to make trade flowing through Singapore more efficient. Launched in September last year, the NTP brings the entire trade ecosystem to a single online location, digitising the trade processing process. It replaced two existing trade facilitation platforms in Singapore, TradeXchange and TradeNet.


Regtech could save banks £2.7bn on AML compliance Global Trade Review (GTR)

#artificialintelligence

Banks are squandering £2.7bn a year because of outdated anti-money laundering (AML) systems – costs that could be saved by adopting machine learning and big data technology, new calculations show. According to FortyTwo Data, an AML technology company, financial institutions are wasting armies of staff on chasing millions of false leads – red flags that turn out to be innocent – generated every year by legacy systems that rely on stale rules and scenarios. It concludes that, on average, 55% of false positives can be eradicated by modern systems, accounting for 42% of banks' cost on AML compliance. FortyTwo Data refers to figures from WealthInsight, which predicts that global spending on AML compliance will hit £6.4bn billion this year. The potential savings thus equates to £2.7bn.