Goto

Collaborating Authors

 Generative AI


Exploring the Limits of Semantic Image Compression at Micro-bits per Pixel

arXiv.org Artificial Intelligence

Traditional methods, such as JPEG, perform image compression by operating on structural information, such as pixel values or frequency content. These methods are effective to bitrates around one bit per pixel (bpp) and higher at standard image sizes. In contrast, text-based semantic compression directly stores concepts and their relationships using natural language, which has evolved with humans to efficiently represent these salient concepts. These methods can operate at extremely low bitrates by disregarding structural information like location, size, and orientation. In this work, we use GPT-4V and DALL-E3 from OpenAI to explore the quality-compression frontier for image compression and identify the limitations of current technology. We push semantic compression as low as 100 $\mu$bpp (up to $10,000\times$ smaller than JPEG) by introducing an iterative reflection process to improve the decoded image. We further hypothesize this 100 $\mu$bpp level represents a soft limit on semantic compression at standard image resolutions.


Transforming document understanding and insights with generative AI

MIT Technology Review

AI Assistant in Adobe Acrobat, now in beta, is a new generative AIโ€“powered conversational engine deeply integrated into Acrobat workflows, empowering everyone with the information inside their most important documents. As the creator of PDF, the world's most trusted digital document format, Adobe understands document challenges and opportunities well. Our continually evolving Acrobat PDF application, the gold standard for working with PDFs, is already used by more than half a billion customers to open around 400 billion documents each year. Starting immediately, customers will be able to use AI Assistant to work even more productively. All they need to do is open Acrobat on their desktop or the web and start working.


On Defining Smart Cities using Transformer Neural Networks

arXiv.org Artificial Intelligence

Cities worldwide are rapidly adopting smart technologies, transforming urban life. Despite this trend, a universally accepted definition of 'smart city' remains elusive. Past efforts to define it have not yielded a consensus, as evidenced by the numerous definitions in use. In this paper, we endeavored to create a new 'compromise' definition that should resonate with most experts previously involved in defining this concept and aimed to validate one of the existing definitions. We reviewed 60 definitions of smart cities from industry, academia, and various relevant organizations, employing transformer architecture-based generative AI and semantic text analysis to reach this compromise. We proposed a semantic similarity measure as an evaluation technique, which could generally be used to compare different smart city definitions, assessing their uniqueness or resemblance. Our methodology employed generative AI to analyze various existing definitions of smart cities, generating a list of potential new composite definitions. Each of these new definitions was then tested against the pre-existing individual definitions we have gathered, using cosine similarity as our metric. This process identified smart city definitions with the highest average cosine similarity, semantically positioning them as the closest on average to all the 60 individual definitions selected.


Virtual Reality for Understanding Artificial-Intelligence-driven Scientific Discovery with an Application in Quantum Optics

arXiv.org Artificial Intelligence

Generative Artificial Intelligence (AI) models can propose solutions to scientific problems beyond human capability. To truly make conceptual contributions, researchers need to be capable of understanding the AI-generated structures and extracting the underlying concepts and ideas. When algorithms provide little explanatory reasoning alongside the output, scientists have to reverse-engineer the fundamental insights behind proposals based solely on examples. This task can be challenging as the output is often highly complex and thus not immediately accessible to humans. In this work we show how transferring part of the analysis process into an immersive Virtual Reality (VR) environment can assist researchers in developing an understanding of AI-generated solutions. We demonstrate the usefulness of VR in finding interpretable configurations of abstract graphs, representing Quantum Optics experiments. Thereby, we can manually discover new generalizations of AI-discoveries as well as new understanding in experimental quantum optics. Furthermore, it allows us to customize the search space in an informed way - as a human-in-the-loop - to achieve significantly faster subsequent discovery iterations. As concrete examples, with this technology, we discover a new resource-efficient 3-dimensional entanglement swapping scheme, as well as a 3-dimensional 4-particle Greenberger-Horne-Zeilinger-state analyzer. Our results show the potential of VR for increasing a human researcher's ability to derive knowledge from graph-based generative AI that, which is a common abstract data representation used in diverse fields of science.


A User-Friendly Framework for Generating Model-Preferred Prompts in Text-to-Image Synthesis

arXiv.org Artificial Intelligence

Well-designed prompts have demonstrated the potential to guide text-to-image models in generating amazing images. Although existing prompt engineering methods can provide high-level guidance, it is challenging for novice users to achieve the desired results by manually entering prompts due to a discrepancy between novice-user-input prompts and the model-preferred prompts. To bridge the distribution gap between user input behavior and model training datasets, we first construct a novel Coarse-Fine Granularity Prompts dataset (CFP) and propose a novel User-Friendly Fine-Grained Text Generation framework (UF-FGTG) for automated prompt optimization. For CFP, we construct a novel dataset for text-to-image tasks that combines coarse and fine-grained prompts to facilitate the development of automated prompt generation methods. For UF-FGTG, we propose a novel framework that automatically translates user-input prompts into model-preferred prompts. Specifically, we propose a prompt refiner that continually rewrites prompts to empower users to select results that align with their unique needs. Meanwhile, we integrate image-related loss functions from the text-to-image model into the training process of text generation to generate model-preferred prompts. Additionally, we propose an adaptive feature extraction module to ensure diversity in the generated results. Experiments demonstrate that our approach is capable of generating more visually appealing and diverse images than previous state-of-the-art methods, achieving an average improvement of 5% across six quality and aesthetic metrics.


Groot: Adversarial Testing for Generative Text-to-Image Models with Tree-based Semantic Transformation

arXiv.org Artificial Intelligence

We evaluate Groot against current adcontext, optimizing it for compliance and effective-versarial methods: ness. SneakyPrompt (Yang et al., 2023) uses reinforcement learning to refine adversarial prompts for 4.4 Sensitive Element Drowning NSFW content generation in text-to-image models. In the Sensitive Element Drowning method, we We exclude techniques like TextFooler (Jin et al., address the challenge of image safety filters after 2020a), BAE (Garg and Ramakrishnan, 2020b), bypassing text filters. Text-to-image models canand TextBugger (Li et al., 2019a) from our benchcreate images on multiple canvases, allowing us tomarking due to their focus on text safety filters and strategically distribute sensitive content on one can-ineffectiveness in bypassing image safety filters for vas while overwhelming others with non-sensitivetext-to-image models.


Search Engines Post-ChatGPT: How Generative Artificial Intelligence Could Make Search Less Reliable

arXiv.org Artificial Intelligence

In this commentary, we discuss the evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI). Our discussion highlights challenges in the early stages of GenAI integration, particularly around factual inconsistencies and biases. We discuss how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability. Furthermore, search engines are already answering queries with error-laden, generated content, further blurring the provenance of information and impacting the integrity of the information ecosystem. We argue how all these factors could reduce the reliability of search engines. Finally, we summarize some of the active research directions and open questions.


Google's AI Boss Says Scale Only Gets You So Far

WIRED

For much of last year, knocking OpenAI off its perch atop the tech industry looked all but impossible, as the company rode a riot of excitement and hype generated by a remarkable, garrulous, and occasionally unhinged program called ChatGPT. Google DeepMind CEO Demis Hassabis has recently at least given Sam Altman some healthy competition, leading the development and deployment of an AI model that appears both as capable and as innovative as the one that powers OpenAI's barnstorming bot. Ever since Alphabet forged DeepMind by merging two of its AI-focused divisions last April, Hassabis has been responsible for corralling its scientists and engineers in order to counter both OpenAI's remarkable rise and its collaboration with Microsoft, seen as a potential threat to Alphabet's cash-cow search business. Google researchers came up with several of the ideas that went into building ChatGPT, yet the company chose not to commercialize them due to misgivings about how they might misbehave or be misused. In recent months, Hassabis has overseen a dramatic shift in pace of research and releases with the rapid development of Gemini, a "multimodal" AI model that already powers Google's answer to ChatGPT and a growing number of Google products.


Another Big Question About AI: Its Carbon Footprint

Mother Jones

This story was originally published by Yale E360 and is reproduced here as part of the Climate Desk collaboration. Two months after its release in November 2022, OpenAI's ChatGPT had 100 million active users, and suddenly tech corporations were racing to offer the public more "generative AI" Pundits compared the new technology's impact to the Internet, or electrification, or the Industrial Revolution--or the discovery of fire. Time will sort hype from reality, but one consequence of the explosion of artificial intelligence is clear: this technology's environmental footprint is large and growing. AI use is directly responsible for carbon emissions from non-renewable electricity and for the consumption of millions of gallons of fresh water, and it indirectly boosts impacts from building and maintaining the power-hungry equipment on which AI runs. As tech companies seek to embed high-intensity AI into everything from resume-writing to kidney transplant medicine and from choosing dog food to climate modeling, they cite many ways AI could help reduce humanity's environmental footprint.


CHATATC: Large Language Model-Driven Conversational Agents for Supporting Strategic Air Traffic Flow Management

arXiv.org Artificial Intelligence

Generative artificial intelligence (AI) and large language models (LLMs) have gained rapid popularity through publicly available tools such as ChatGPT. The adoption of LLMs for personal and professional use is fueled by the natural interactions between human users and computer applications such as ChatGPT, along with powerful summarization and text generation capabilities. Given the widespread use of such generative AI tools, in this work we investigate how these tools can be deployed in a non-safety critical, strategic traffic flow management setting. Specifically, we train an LLM, CHATATC, based on a large historical data set of Ground Delay Program (GDP) issuances, spanning 2000-2023 and consisting of over 80,000 GDP implementations, revisions, and cancellations. We test the query and response capabilities of CHATATC, documenting successes (e.g., providing correct GDP rates, durations, and reason) and shortcomings (e.g,. superlative questions). We also detail the design of a graphical user interface for future users to interact and collaborate with the CHATATC conversational agent.