FIGURE 1: LMU seal (left) style-transferred to Van Gogh's Sunflower painting (center) and blended with the prompt - Van Gogh, sunflowers - via CLIP+VGAN (right). In the last few years, there have been several breakthroughs in the methodologies used in Natural Language Processing (NLP) as well as Computer Vision (CV). Beyond these improvements on single-modality models, large-scale multimodal approaches have become a very active area of research. In this seminar, we reviewed these approaches and attempted to create a solid overview of the field, starting with the current state-of-the-art approaches in the two subfields of Deep Learning individually. Further, modeling frameworks are discussed where one modality is transformed into the other Chapter 3.1 and Chapter 3.2), as well as models in which one modality is utilized to enhance representation learning for the other (Chapter 3.3 and Chapter 3.4). To conclude the second part, architectures with a focus on handling both modalities simultaneously are introduced (Chapter 3.5). Finally, we also cover other modalities (Chapter 4.1 and Chapter 4.2) as well as general-purpose multi-modal models (Chapter 4.3), which are able to handle different tasks on different modalities within one unified architecture.

The Neural Architecture Search (NAS) problem is typically formulated as a graph search problem where the goal is to learn the optimal operations over edges in order to maximise a graph-level global objective. Due to the large architecture parameter space, efficiency is a key bottleneck preventing NAS from its practical use. In this paper, we address the issue by framing NAS as a multi-agent problem where agents control a subset of the network and coordinate to reach optimal architectures. We provide two distinct lightweight implementations, with reduced memory requirements (1/8th of state-of-the-art), and performances above those of much more computationally expensive methods. Theoretically, we demonstrate vanishing regrets of the form O(sqrt(T)), with T being the total number of rounds. Finally, aware that random search is an, often ignored, effective baseline we perform additional experiments on 3 alternative datasets and 2 network configurations, and achieve favourable results in comparison.

Motion planning is a mature area of research in robotics with many well-established methods based on optimization or sampling the state space, suitable for solving kinematic motion planning. However, when dynamic motions under constraints are needed and computation time is limited, fast kinodynamic planning on the constraint manifold is indispensable. In recent years, learning-based solutions have become alternatives to classical approaches, but they still lack comprehensive handling of complex constraints, such as planning on a lower-dimensional manifold of the task space while considering the robot's dynamics. This paper introduces a novel learning-to-plan framework that exploits the concept of constraint manifold, including dynamics, and neural planning methods. Our approach generates plans satisfying an arbitrary set of constraints and computes them in a short constant time, namely the inference time of a neural network. This allows the robot to plan and replan reactively, making our approach suitable for dynamic environments. We validate our approach on two simulated tasks and in a demanding real-world scenario, where we use a Kuka LBR Iiwa 14 robotic arm to perform the hitting movement in robotic Air Hockey.

In a bid to run with the tide of digital economy, fresh facts have emerged that seven Deposit Money Banks (DMBs) are already competing in not only adoption but performance efficiency and customer satisfaction in the use of Artificial Intelligence-powered chatbots. The banks are Zenith Bank with chatbot called Ziva; Fidelity bank plc's Ivy; First City Monument Bank's Temi; UBA Group with its Leo, an AI-powered Facebook Messenger bot that allow users perform banking transactions; Access Bank's Tamada; Heritage Bank's octopus chatbot and Keystone Bank's chatbot called oxygen. A chatbot is a computer programme or an artificial intelligence which conducts a conversation through audio or text. They are messaging apps which allow businesses and brands to remain online 24 hours, providing customer support by instant responses and complaint resolution. AI, chatbots and automated, self-service technologies free up call centre employees from routine tier-1 support requests so they can focus on more complex tasks.

A dystopian future for cybersecurity is presented by all good apocalyptic sci-fi films and books. Beyond the inclusion of open AI in many vendor slide decks in a way of positioning next-generation technology and a more automated, advanced, approach to technology, the concept is now becoming more mainstream. This advancement from a glorified Google search has sparked the interest of students seeking to expedite essay responses without consuming the time or need to read source materials, followed by teachers seeking to similarly automate marking. Anyone who has been frustrated by a website chatbot while attempting to get help or an answer to a vaguely complex question found solace in conversing with a computer. Without a doubt, it's opened up a world of possibilities for AI to influence how we interact with technology daily as individuals, rather than just being a mysterious black box powering systems ranging from weather forecasting to space rockets. Inevitably, the potential impact on cyber security will quickly become a key topic of discussion on both sides.

The number of women authoring scientific papers is increasing, but men still dominate overall and some fields won't reach gender equity until the next century, according to a major analysis. Cassidy Sugimoto at the Georgia Institute of Technology in Atlanta and Vincent Larivière at the University of Montreal in Canada analysed nearly 5.5 million scientific papers published between 2008 and 2020, using a machine-learning algorithm to estimate the likelihood that a person's name belonged to a man or woman.

Male and female lar gibbons sing duets with notes that are synchronised and occur at regular intervals. These are rhythmic qualities similar to those found in human songs, which could hint at an evolutionary basis for the origins of music. "I'm pretty sure the gibbon's isochronous capacities are better than mine," says Andrea Ravignani at the Max Planck Institute for Psycholinguistics in the Netherlands, referring to the capacity to sing notes that occur at regularly repeating intervals. This ability has previously been noted in indris (Indri indri), a type of lemur found in Madagascar and the only other primate whose calls exhibit distinct rhythms related to those found in human music. Male and female gibbons regularly sing duets to define territory and form social bonds.

Machine learning and statistical modeling methods were used to analyze the impact of climate change on financial wellbeing of fruit farmers in Tunisia and Chile. The analysis was based on face to face interviews with 801 farmers. Three research questions were investigated. First, whether climate change impacts had an effect on how well the farm was doing financially. Second, if climate change was not influential, what factors were important for predicting financial wellbeing of the farm. And third, ascertain whether observed effects on the financial wellbeing of the farm were a result of interactions between predictor variables. This is the first report directly comparing climate change with other factors potentially impacting financial wellbeing of farms. Certain climate change factors, namely increases in temperature and reductions in precipitation, can regionally impact self-perceived financial wellbeing of fruit farmers. Specifically, increases in temperature and reduction in precipitation can have a measurable negative impact on the financial wellbeing of farms in Chile. This effect is less pronounced in Tunisia. Climate impact differences were observed within Chile but not in Tunisia. However, climate change is only of minor importance for predicting farm financial wellbeing, especially for farms already doing financially well. Factors that are more important, mainly in Tunisia, included trust in information sources and prior farm ownership. Other important factors include farm size, water management systems used and diversity of fruit crops grown. Moreover, some of the important factors identified differed between farms doing and not doing well financially. Interactions between factors may improve or worsen farm financial wellbeing.

Multimodal knowledge graph link prediction aims to improve the accuracy and efficiency of link prediction tasks for multimodal data. However, for complex multimodal information and sparse training data, it is usually difficult to achieve interpretability and high accuracy simultaneously for most methods. To address this difficulty, a new model is developed in this paper, namely Interpretable Multimodal Knowledge Graph Answer Prediction via Sequence Modeling (IMKGA-SM). First, a multi-modal fine-grained fusion method is proposed, and Vgg16 and Optical Character Recognition (OCR) techniques are adopted to effectively extract text information from images and images. Then, the knowledge graph link prediction task is modelled as an offline reinforcement learning Markov decision model, which is then abstracted into a unified sequence framework. An interactive perception-based reward expectation mechanism and a special causal masking mechanism are designed, which "converts" the query into an inference path. Then, an autoregressive dynamic gradient adjustment mechanism is proposed to alleviate the insufficient problem of multimodal optimization. Finally, two datasets are adopted for experiments, and the popular SOTA baselines are used for comparison. The results show that the developed IMKGA-SM achieves much better performance than SOTA baselines on multimodal link prediction datasets of different sizes.

Contrastive deep clustering has recently gained significant attention with its ability of joint contrastive learning and clustering via deep neural networks. Despite the rapid progress, previous works mostly require both positive and negative sample pairs for contrastive clustering, which rely on a relative large batch-size. Moreover, they typically adopt a two-stream architecture with two augmented views, which overlook the possibility and potential benefits of multi-stream architectures (especially with heterogeneous or hybrid networks). In light of this, this paper presents a new end-to-end deep clustering approach termed Heterogeneous Tri-stream Clustering Network (HTCN). The tri-stream architecture in HTCN consists of three main components, including two weight-sharing online networks and a target network, where the parameters of the target network are the exponential moving average of that of the online networks. Notably, the two online networks are trained by simultaneously (i) predicting the instance representations of the target network and (ii) enforcing the consistency between the cluster representations of the target network and that of the two online networks. Experimental results on four challenging image datasets demonstrate the superiority of HTCN over the state-of-the-art deep clustering approaches. The code is available at https://github.com/dengxiaozhi/HTCN.