Unlike most neural language models, humans learn language in a rich, multi-sensory and, often, multi-lingual environment. Current language models typically fail to fully capture the complexities of multilingual language use. We train an LSTM language model on images and captions in English and Spanish from MS-COCO-ES. We find that the visual grounding improves the model's understanding of semantic similarity both within and across languages and improves perplexity. However, we find no significant advantage of visual grounding for abstract words. Our results provide additional evidence of the advantages of visually grounded language models and point to the need for more naturalistic language data from multilingual speakers and multilingual datasets with perceptual grounding.

Models for text-to-image synthesis, such as DALL-E~2 and Stable Diffusion, have recently drawn a lot of interest from academia and the general public. These models are capable of producing high-quality images that depict a variety of concepts and styles when conditioned on textual descriptions. However, these models adopt cultural characteristics associated with specific Unicode scripts from their vast amount of training data, which may not be immediately apparent. We show that by simply inserting single non-Latin characters in a textual description, common models reflect cultural stereotypes and biases in their generated images. We analyze this behavior both qualitatively and quantitatively, and identify a model's text encoder as the root cause of the phenomenon. Additionally, malicious users or service providers may try to intentionally bias the image generation to create racist stereotypes by replacing Latin characters with similarly-looking characters from non-Latin scripts, so-called homoglyphs. To mitigate such unnoticed script attacks, we propose a novel homoglyph unlearning method to fine-tune a text encoder, making it robust against homoglyph manipulations.

How can we augment a dynamic graph for improving the performance of dynamic graph neural networks? Graph augmentation has been widely utilized to boost the learning performance of GNN-based models. However, most existing approaches only enhance spatial structure within an input static graph by transforming the graph, and do not consider dynamics caused by time such as temporal locality, i.e., recent edges are more influential than earlier ones, which remains challenging for dynamic graph augmentation. In this work, we propose TiaRa (Time-aware Random Walk Diffusion), a novel diffusion-based method for augmenting a dynamic graph represented as a discrete-time sequence of graph snapshots. For this purpose, we first design a time-aware random walk proximity so that a surfer can walk along the time dimension as well as edges, resulting in spatially and temporally localized scores. We then derive our diffusion matrices based on the time-aware random walk, and show they become enhanced adjacency matrices that both spatial and temporal localities are augmented. Throughout extensive experiments, we demonstrate that TiaRa effectively augments a given dynamic graph, and leads to significant improvements in dynamic GNN models for various graph datasets and tasks.

Since ChatGPT took the world by storm last fall, people have been in a frenzy debating the impact artificial intelligence and other new automated technology will have on America's job market. The "robots are taking our jobs" narrative was further boosted by viral videos showing new, "fully automated" McDonald's and Taco Bell restaurants. The knee-jerk reaction to these videos is to say that robots are coming for our jobs, but while AI and other kinds of automation have progressed, that doesn't mean they're necessarily eliminating jobs. Instead, the new tech is simply changing how we work and what kinds of jobs exist. Automation technology has ushered in a fleet of secret workers behind screens, machines, and smiling robot faces.

To assist with advanced analytics and deep insight by being a proactive partner in providing customer centric data analytics, including alternative methods of aggregating raw data (internal and external), which will ultimately influence the way in which we view and act on customer behaviour and customer health, i.e. identifying risks and opportunities. Implementing the use of machine learning to challenge and improve predictive modelling techniques, the available characteristic universe across the customer life cycle and optimising segmentation to enhance model performance. Solutions should satisfy customer centricity and digitisation objectives. Including but not limited to Extracting meaningful insights from data, Predictive modelling and machine learning, Stakeholder Engagement, Leadership and People Management.

The COVID-19 pandemic has significantly impacted the construction sector, which is sensitive to economic cycles. In order to boost value and efficiency in this sector, the use of innovative exploration technologies such as ultrasonic and Artificial Intelligence techniques in building material research is becoming increasingly crucial. In this study, we developed two models for predicting the Los Angeles (LA) and Micro Deval (MDE) coefficients, two important geotechnical tests used to determine the quality of rock aggregates. These coefficients describe the resistance of aggregates to fragmentation and abrasion. The ultrasound velocity, porosity, and density of the rocks were determined and used as inputs to develop prediction models using multiple regression and an artificial neural network. These models may be used to assess the quality of rock aggregates at the exploration stage without the need for tedious laboratory analysis.

Text generation from Abstract Meaning Representation (AMR) has substantially benefited from the popularized Pretrained Language Models (PLMs). Myriad approaches have linearized the input graph as a sequence of tokens to fit the PLM tokenization requirements. Nevertheless, this transformation jeopardizes the structural integrity of the graph and is therefore detrimental to its resulting representation. To overcome this issue, Ribeiro et al. have recently proposed StructAdapt, a structure-aware adapter which injects the input graph connectivity within PLMs using Graph Neural Networks (GNNs). In this paper, we investigate the influence of Relative Position Embeddings (RPE) on AMR-to-Text, and, in parallel, we examine the robustness of StructAdapt. Through ablation studies, graph attack and link prediction, we reveal that RPE might be partially encoding input graphs. We suggest further research regarding the role of RPE will provide valuable insights for Graph-to-Text generation.

These demands increase the cost of training and deploying these architectures, and constrain the spectrum of devices that they can be used on. Although deep learning has been very promising in many areas in terms of technology in recent years, it has some problems that need improvement. Even though deep learning can distinguish changes and subtle differences in data with interconnected neural networks, it makes it very difficult to define hyperparameters and determine their values before training the data. For this reason, different pruning methods have been proposed in the literature to reduce the parameters of convolutional networks Han et al. (2016); Hanson & Pratt (1988); LeCun & Cortes (2010); Ström (1997). The common problem of deep learning and pruning algorithms that have been studied in recent years is the decision of the pruning percentage with the heuristic approach at the pruning stage, making the success percentage of the deep learning algorithm dependent on the pruning parameter. On the other hand, the optimization models proposed with zero-norm penalty to ensure sparsity ignore the diversity of the layers as they only take into account the percentage of success. The combination of the layers that are close to each other does not increase the percentage of accuracy. As the primal example of DNNs, Convolutional Neural Networks (CNNs) are feed-forward architectures originally proposed to perform image processing tasks, Li et al. (2021) but they offered such high versatility and capacity that allowed them to be used in many other tasks including time series prediction, signal identification and natural language processing. Inspired by a biological visual perceptron that displays local receptive fields Goodfellow et al. (2016), a CNN uses learnable kernels to extract the relevant features at each processing

Privacy-preserving machine learning enables the training of models on decentralized datasets without the need to reveal the data, both on horizontal and vertically partitioned data. However, it relies on specialized techniques and algorithms to perform the necessary computations. The privacy preserving scalar product protocol, which enables the dot product of vectors without revealing them, is one popular example for its versatility. Unfortunately, the solutions currently proposed in the literature focus mainly on two-party scenarios, even though scenarios with a higher number of data parties are becoming more relevant. For example when performing analyses that require counting the number of samples which fulfill certain criteria defined across various sites, such as calculating the information gain at a node in a decision tree. In this paper we propose a generalization of the protocol for an arbitrary number of parties, based on an existing two-party method. Our proposed solution relies on a recursive resolution of smaller scalar products. After describing our proposed method, we discuss potential scalability issues. Finally, we describe the privacy guarantees and identify any concerns, as well as comparing the proposed method to the original solution in this aspect.

Entropy is a fundamental concept in the field of information theory. During measurement, conventional entropy measures are susceptible to length and amplitude changes in time series. A new entropy metric, neural network entropy (NNetEn), has been developed to overcome these limitations. NNetEn entropy is computed using a modified LogNNet neural network classification model. The algorithm contains a reservoir matrix of N=19625 elements that must be filled with the given data. The contribution of this paper is threefold. Firstly, this work investigates different methods of filling the reservoir with time series (signal) elements. The reservoir filling method determines the accuracy of the entropy estimation by convolution of the study time series and LogNNet test data. The present study proposes 6 methods for filling the reservoir for time series. Two of them (Method 3 and Method 6) employ the novel approach of stretching the time series to create intermediate elements that complement it, but do not change its dynamics. The most reliable methods for short time series are Method 3 and Method 5. The second part of the study examines the influence of noise and constant bias on entropy values. Our study examines three different time series data types (chaotic, periodic, and binary) with different dynamic properties, Signal to Noise Ratio (SNR), and offsets. The NNetEn entropy calculation errors are less than 10% when SNR is greater than 30 dB, and entropy decreases with an increase in the bias component. The third part of the article analyzes real-time biosignal EEG data collected from emotion recognition experiments. The NNetEn measures show robustness under low-amplitude noise using various filters. Thus, NNetEn measures entropy effectively when applied to real-world environments with ambient noise, white noise, and 1/f noise.