Goto

Collaborating Authors

 Media


LEDITS: Real Image Editing with DDPM Inversion and Semantic Guidance

arXiv.org Artificial Intelligence

Recent large-scale text-guided diffusion models provide powerful image-generation capabilities. Currently, a significant effort is given to enable the modification of these images using text only as means to offer intuitive and versatile editing. However, editing proves to be difficult for these generative models due to the inherent nature of editing techniques, which involves preserving certain content from the original image. Conversely, in text-based models, even minor modifications to the text prompt frequently result in an entirely distinct result, making attaining one-shot generation that accurately corresponds to the users intent exceedingly challenging. In addition, to edit a real image using these state-of-the-art tools, one must first invert the image into the pre-trained models domain - adding another factor affecting the edit quality, as well as latency. In this exploratory report, we propose LEDITS - a combined lightweight approach for real-image editing, incorporating the Edit Friendly DDPM inversion technique with Semantic Guidance, thus extending Semantic Guidance to real image editing, while harnessing the editing capabilities of DDPM inversion as well. This approach achieves versatile edits, both subtle and extensive as well as alterations in composition and style, while requiring no optimization nor extensions to the architecture.


When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

arXiv.org Artificial Intelligence

Despite their impressive performance on diverse tasks, large language models (LMs) still struggle with tasks requiring rich world knowledge, implying the limitations of relying solely on their parameters to encode a wealth of world knowledge. This paper aims to understand LMs' strengths and limitations in memorizing factual knowledge, by conducting large-scale knowledge probing experiments of 10 models and 4 augmentation methods on PopQA, our new open-domain QA dataset with 14k questions. We find that LMs struggle with less popular factual knowledge, and that scaling fails to appreciably improve memorization of factual knowledge in the long tail. We then show that retrieval-augmented LMs largely outperform orders of magnitude larger LMs, while unassisted LMs remain competitive in questions about high-popularity entities. Based on those findings, we devise a simple, yet effective, method for powerful and efficient retrieval-augmented LMs, which retrieves non-parametric memories only when necessary. Experimental results show that this significantly improves models' performance while reducing the inference costs.


Convolutional Neural Network (CNN) to reduce construction loss in JPEG compression caused by Discrete Fourier Transform (DFT)

arXiv.org Artificial Intelligence

In recent decades, digital image processing has gained enormous popularity. Consequently, a number of data compression strategies have been put forth, with the goal of minimizing the amount of information required to represent images. Among them, JPEG compression is one of the most popular methods that has been widely applied in multimedia and digital applications. The periodic nature of DFT makes it impossible to meet the periodic condition of an image's opposing edges without producing severe artifacts, which lowers the image's perceptual visual quality. On the other hand, deep learning has recently achieved outstanding results for applications like speech recognition, image reduction, and natural language processing. Convolutional Neural Networks (CNN) have received more attention than most other types of deep neural networks. The use of convolution in feature extraction results in a less redundant feature map and a smaller dataset, both of which are crucial for image compression. In this work, an effective image compression method is purposed using autoencoders. The study's findings revealed a number of important trends that suggested better reconstruction along with good compression can be achieved using autoencoders.


Arnold Schwarzenegger claims AI future from 'Terminator' franchise is 'here today'

FOX News

Former California governor Arnold Schwarzenegger called for replacing fossil fuels and said clean energy projects across the world need to be fast-tracked during MSNBC's "Morning Joe" on Tuesday. Hollywood legend Arnold Schwarzenegger claimed that the future artificial intelligence technology that was predicted in the iconic "Terminator" franchise has "become a reality." During a Los Angeles event honoring the 75-year-old Austrian-born actor and his limited edition photo book, Schwarzenegger remarked on the similarities between today's real-world AI technology and the AI portrayed in the 80s action epic he starred in. "Now over the course of decades, it has become a reality," Schwarzenegger claimed. He also added praise for "Terminator" filmmaker James Cameron for predicting the future.


Single Sequence Prediction over Reasoning Graphs for Multi-hop QA

arXiv.org Artificial Intelligence

Recent generative approaches for multi-hop question answering (QA) utilize the fusion-in-decoder method~\cite{izacard-grave-2021-leveraging} to generate a single sequence output which includes both a final answer and a reasoning path taken to arrive at that answer, such as passage titles and key facts from those passages. While such models can lead to better interpretability and high quantitative scores, they often have difficulty accurately identifying the passages corresponding to key entities in the context, resulting in incorrect passage hops and a lack of faithfulness in the reasoning path. To address this, we propose a single-sequence prediction method over a local reasoning graph (\model)\footnote{Code/Models will be released at \url{https://github.com/gowtham1997/SeqGraph}} that integrates a graph structure connecting key entities in each context passage to relevant subsequent passages for each question. We use a graph neural network to encode this graph structure and fuse the resulting representations into the entity representations of the model. Our experiments show significant improvements in answer exact-match/F1 scores and faithfulness of grounding in the reasoning path on the HotpotQA dataset and achieve state-of-the-art numbers on the Musique dataset with only up to a 4\% increase in model parameters.


Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

arXiv.org Artificial Intelligence

Natural Language Feedback (NLF) is an increasingly popular avenue to align Large Language Models (LLMs) to human preferences. Despite the richness and diversity of the information it can convey, NLF is often hand-designed and arbitrary. In a different world, research in pedagogy has long established several effective feedback models. In this opinion piece, we compile ideas from pedagogy to introduce FELT, a feedback framework for LLMs that outlines the various characteristics of the feedback space, and a feedback content taxonomy based on these variables. Our taxonomy offers both a general mapping of the feedback space, as well as pedagogy-established discrete categories, allowing us to empirically demonstrate the impact of different feedback types on revised generations. In addition to streamlining existing NLF designs, FELT also brings out new, unexplored directions for research in NLF. We make our taxonomy available to the community, providing guides and examples for mapping our categorizations to future resources.


An Overwatch anime miniseries will debut on July 6th

Engadget

Blizzard has released a string of excellent Overwatch animated shorts over the years. While the shorts are sublimely rendered and help to sketch out the backstories of the cast, Blizzard hasn't neatly pulled together the sprawling narrative of this universe so far. The developers have pledged to do a better job of that in-game starting with Overwatch 2's sixth season, which gets under way in August. Before we get there, though, Blizzard is releasing an Overwatch anime. The three-episode miniseries is called Genesis.


Whose generated line is it anyway? AI tries to crack humour's DNA

The Guardian

I've seen some bad comedy acts over the years – but not, until now, one that is part of an existential threat to humanity. One of artificial intelligence's pre-eminent boffins, Geoffrey Hinton, sent out shock waves recently by arguing that, in relation to AI: "We're toast. This is the actual end of history." That's a hell of a backdrop to my visit to see Artificial Intelligence Improvisation, a show by the Improbotics troupe playing as part of an AI festival in London this week. You'll forgive me, I hope, for some hesitation in wielding the critical brickbat, given that the act under review boasts the capacity to wipe out all of us.


Baffled by the symbols on your car's dashboard? Your iPhone will soon tell you what they mean

Daily Mail - Science & tech

We've all been there - a new symbol pops up on your car's dashboard that you don't recognise, causing panic to set in. But the days of frantically rooting around for your car's handbook could soon be a thing of the past, thanks to Apple's next iPhone update. The tech giant is updating its Visual Look Up tool in the upcoming iOS 17 update. While the tool can already recognise popular landmarks, statues, art, plants, pets and more in photos, Apple has confirmed that it will soon also recognise symbols. 'Now users can identify food, storefronts, signs, and symbols, and lift individual subjects from photos and videos,' it explained.


Could AI movies like 'The Matrix' and 'Her' become a reality? Experts weigh in

FOX News

Veritone CEO Ryan Steelberg says the Writers Guild needs to make sure its writers are protected as AI becomes more popular. While watching a film, viewers might ponder its legitimacy, questioning if what they're seeing on screen can happen in real life. Artificial intelligence is no different. AI is being heavily developed and utilized now to edit and amplify films, but the depths of its use has also been explored in futuristic, science-fiction movies like "The Matrix" or "I, Robot." With the rise of AI platforms, including ChatGPT, AI appears to be infiltrating every industry.