South America
Automated Reading Passage Generation with OpenAI's Large Language Model
Bezirhan, Ummugul, von Davier, Matthias
The widespread usage of computer-based assessments and individualized learning platforms has resulted in an increased demand for the rapid production of high-quality items. Automated item generation (AIG), the process of using item models to generate new items with the help of computer technology, was proposed to reduce reliance on human subject experts at each step of the process. AIG has been used in test development for some time. Still, the use of machine learning algorithms has introduced the potential to improve the efficiency and effectiveness of the process greatly. The approach presented in this paper utilizes OpenAI's latest transformer-based language model, GPT-3, to generate reading passages. Existing reading passages were used in carefully engineered prompts to ensure the AI-generated text has similar content and structure to a fourth-grade reading passage. For each prompt, we generated multiple passages, the final passage was selected according to the Lexile score agreement with the original passage. In the final round, the selected passage went through a simple revision by a human editor to ensure the text was free of any grammatical and factual errors. All AI-generated passages, along with original passages were evaluated by human judges according to their coherence, appropriateness to fourth graders, and readability.
Linking Representations with Multimodal Contrastive Learning
Arora, Abhishek, Yang, Xinmei, Jheng, Shao-Yu, Dell, Melissa
Many applications require grouping instances contained in diverse document datasets into classes. Most widely used methods do not employ deep learning and do not exploit the inherently multimodal nature of documents. Notably, record linkage is typically conceptualized as a string-matching problem. This study develops CLIPPINGS, (Contrastively Linking Pooled Pre-trained Embeddings), a multimodal framework for record linkage. CLIPPINGS employs end-to-end training of symmetric vision and language bi-encoders, aligned through contrastive language-image pre-training, to learn a metric space where the pooled image-text representation for a given instance is close to representations in the same class and distant from representations in different classes. At inference time, instances can be linked by retrieving their nearest neighbor from an offline exemplar embedding index or by clustering their representations. The study examines two challenging applications: constructing comprehensive supply chains for mid-20th century Japan through linking firm level financial records - with each firm name represented by its crop in the document image and the corresponding OCR - and detecting which image-caption pairs in a massive corpus of historical U.S. newspapers came from the same underlying photo wire source. CLIPPINGS outperforms widely used string matching methods by a wide margin and also outperforms unimodal methods. Moreover, a purely self-supervised model trained on only image-OCR pairs also outperforms popular string-matching methods without requiring any labels.
Classification of news spreading barriers
Sittar, Abdul, Mladenic, Dunja, Grobelnik, Marko
News media is one of the most effective mechanisms for spreading information internationally, and many events from different areas are internationally relevant. However, news coverage for some news events is limited to a specific geographical region because of information spreading barriers, which can be political, geographical, economic, cultural, or linguistic. In this paper, we propose an approach to barrier classification where we infer the semantics of news articles through Wikipedia concepts. To that end, we collected news articles and annotated them for different kinds of barriers using the metadata of news publishers. Then, we utilize the Wikipedia concepts along with the body text of news articles as features to infer the news-spreading barriers. We compare our approach to the classical text classification methods, deep learning, and transformer-based methods. The results show that the proposed approach using Wikipedia concepts based semantic knowledge offers better performance than the usual for classifying the news-spreading barriers.
On Robustness in Multimodal Learning
McKinzie, Brandon, Cheng, Joseph, Shankar, Vaishaal, Yang, Yinfei, Shlens, Jonathon, Toshev, Alexander
Multimodal learning is defined as learning over multiple heterogeneous input modalities such as video, audio, and text. In this work, we are concerned with understanding how models behave as the type of modalities differ between training and deployment, a situation that naturally arises in many applications of multimodal learning to hardware platforms. We present a multimodal robustness framework to provide a systematic analysis of common multimodal representation learning methods. Further, we identify robustness short-comings of these approaches and propose two intervention techniques leading to $1.5\times$-$4\times$ robustness improvements on three datasets, AudioSet, Kinetics-400 and ImageNet-Captions. Finally, we demonstrate that these interventions better utilize additional modalities, if present, to achieve competitive results of $44.2$ mAP on AudioSet 20K.
DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion
Cao, ZiHan, Cao, ShiQi, Wu, Xiao, Hou, JunMing, Ran, Ran, Deng, Liang-Jian
Denosing diffusion model, as a generative model, has received a lot of attention in the field of image generation recently, thanks to its powerful generation capability. However, diffusion models have not yet received sufficient research in the field of image fusion. In this article, we introduce diffusion model to the image fusion field, treating the image fusion task as image-to-image translation and designing two different conditional injection modulation modules (i.e., style transfer modulation and wavelet modulation) to inject coarse-grained style information and fine-grained high-frequency and low-frequency information into the diffusion UNet, thereby generating fused images. In addition, we also discussed the residual learning and the selection of training objectives of the diffusion model in the image fusion task. Extensive experimental results based on quantitative and qualitative assessments compared with benchmarks demonstrates state-of-the-art results and good generalization performance in image fusion tasks. Finally, it is hoped that our method can inspire other works and gain insight into this field to better apply the diffusion model to image fusion tasks. Code shall be released for better reproducibility.
Explicit and Implicit Semantic Ranking Framework
Zhu, Xiaofeng, Lin, Thomas, Anand, Vishal, Calderwood, Matthew, Clausen-Brown, Eric, Lueck, Gord, Yim, Wen-wai, Wu, Cheng
The core challenge in numerous real-world applications is to match an inquiry to the best document from a mutable and finite set of candidates. Existing industry solutions, especially latency-constrained services, often rely on similarity algorithms that sacrifice quality for speed. In this paper we introduce a generic semantic learning-to-rank framework, Self-training Semantic Cross-attention Ranking (sRank). This transformer-based framework uses linear pairwise loss with mutable training batch sizes and achieves quality gains and high efficiency, and has been applied effectively to show gains on two industry tasks at Microsoft over real-world large-scale data sets: Smart Reply (SR) and Ambient Clinical Intelligence (ACI). In Smart Reply, $sRank$ assists live customers with technical support by selecting the best reply from predefined solutions based on consumer and support agent messages. It achieves 11.7% gain in offline top-one accuracy on the SR task over the previous system, and has enabled 38.7% time reduction in composing messages in telemetry recorded since its general release in January 2021. In the ACI task, sRank selects relevant historical physician templates that serve as guidance for a text summarization model to generate higher quality medical notes. It achieves 35.5% top-one accuracy gain, along with 46% relative ROUGE-L gain in generated medical notes.
Intelligent humanoids in manufacturing to address worker shortage and skill gaps: Case of Tesla Optimus
Malik, Ali Ahmad, Masood, Tariq, Brem, Alexander
Technological evolution in the field of robotics is emerging with major breakthroughs in recent years. This was especially fostered by revolutionary new software applications leading to humanoid robots. Humanoids are being envisioned for manufacturing applications to form human-robot teams. But their implication in manufacturing practices especially for industrial safety standards and lean manufacturing practices have been minimally addressed. Humanoids will also be competing with conventional robotic arms and effective methods to assess their return on investment are needed. To study the next generation of industrial automation, we used the case context of the Tesla humanoid robot. The company has recently unveiled its project on an intelligent humanoid robot named Optimus to achieve an increased level of manufacturing automation. This article proposes a framework to integrate humanoids for manufacturing automation and also presents the significance of safety standards of human-robot collaboration. A case of lean assembly cell for the manufacturing of an open-source medical ventilator was used for human-humanoid automation. Simulation results indicate that humanoids can increase the level of manufacturing automation. Managerial and research implications are presented.
Personal Data in Large Language Models: The Removal Process and Its Impact
The ever-growing capabilities of large language models, such as OpenAI's GPT-4, have raised important questions about data privacy and the ethical use of personal information. As these models process and learn from vast amounts of text data, it is crucial to consider the implications of inadvertently including personal or sensitive information. I will discuss the use of personal data within large language models, the process of requesting the removal of that information, and the impact of such removal on the model's performance. Ultimately, will removing data from the training set make any difference to a large model? Large language models like GPT-4 are trained on massive datasets that include a wide range of text sources, from books and articles to social media and websites.
The man who unleashed AI on an unsuspecting Silicon Valley
As part of that job, he's planned a round-the-world goodwill tour to talk with politicians and people using OpenAI's technology. The month-long campaign -- which will take him to Canada, Brazil, Nigeria, Europe, Singapore, Japan, Indonesia and Australia, among other stops -- comes as debate over AI's impact on the world is heating up. The Italian government temporarily banned OpenAI in March, citing concerns about privacy and data collection.
Artificial Intelligence software used to spread misinformation in Venezuela
Artificial Intelligence software is being used to spread misinformation in Venezuela. Stefano Pozzebon takes a look at how it's being distributed, and how to spot fake video. Former Maryland Gov. Hogan's ex-chief of staff dies after confrontation with FBI agent'Lucky fire-breathing dragon': UConn's head coach admits to wearing this during games GOP lawmaker hands out'indict this!' ham sandwiches on Capitol Hill Listen to Trump's defiant message after being indicted Disney quietly takes power from Florida governor's board'Shark Tank' star reacts to Senate hearing on bank failures