Goto

Collaborating Authors

 sensei


SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models

Sancaktar, Cansu, Gumbsch, Christian, Zadaianchuk, Andrii, Kolev, Pavel, Martius, Georg

arXiv.org Artificial Intelligence

Exploration is a cornerstone of reinforcement learning (RL). Intrinsic motivation attempts to decouple exploration from external, task-based rewards. However, established approaches to intrinsic motivation that follow general principles such as information gain, often only uncover low-level interactions. In contrast, children's play suggests that they engage in meaningful high-level behavior by imitating or interacting with their caregivers. Recent work has focused on using foundation models to inject these semantic biases into exploration. However, these methods often rely on unrealistic assumptions, such as language-embedded environments or access to high-level actions. We propose SEmaNtically Sensible ExploratIon (SENSEI), a framework to equip model-based RL agents with an intrinsic motivation for semantically meaningful behavior. SENSEI distills a reward signal of interestingness from Vision Language Model (VLM) annotations, enabling an agent to predict these rewards through a world model. Using model-based RL, SENSEI trains an exploration policy that jointly maximizes semantic rewards and uncertainty. We show that in both robotic and video game-like simulations SENSEI discovers a variety of meaningful behaviors from image observations and low-level actions. SENSEI provides a general tool for learning from foundation model feedback, a crucial research direction, as VLMs become more powerful.


Input-sensitive dense-sparse primitive compositions for GNN acceleration

Lenadora, Damitha, Sathia, Vimarsh, Gerogiannis, Gerasimos, Yesil, Serif, Torrellas, Josep, Mendis, Charith

arXiv.org Artificial Intelligence

Graph neural networks (GNN) have become an important class of neural network models that have gained popularity in domains such as social and financial network analysis. Different phases of GNN computations can be modeled using both dense and sparse matrix operations. There have been many frameworks and optimization techniques proposed in the literature to accelerate GNNs. However, getting consistently high performance across many input graphs with different sparsity patterns and GNN embedding sizes has remained difficult. In this paper, we propose different algebraic reassociations of GNN computations that lead to novel dense and sparse matrix primitive selections and compositions. We show that the profitability of these compositions depends on the input graph, embedding size, and the target hardware. We developed SENSEi, a system that uses a data-driven adaptive strategy to select the best composition given the input graph and GNN embedding sizes. Our evaluations on a wide range of graphs and embedding sizes show that SENSEi achieves geomean speedups of $1.105\times$ (up to $2.959\times$) and $1.187\times$ (up to $1.99\times$) on graph convolutional networks and geomean speedups of $2.307\times$ (up to $35.866\times$) and $1.44\times$ (up to $5.69\times$) on graph attention networks on CPUs and GPUs respectively over the widely used Deep Graph Library. Further, we show that the compositions yield notable synergistic performance benefits on top of other established sparse optimizations such as sparse matrix tiling by evaluating against a well-tuned baseline.


AI-powered CRM platforms compared

#artificialintelligence

Beyond common features like ML and automation, CRM products can vary dramatically as each vendor takes AI in its own direction. The following AI-powered CRM platforms -- including Salesforce Einstein, IMB Watson and Azure Cognitive Services -- have their own strengths and weaknesses. Salesforce Einstein is the vendor's AI that powers many features in the Salesforce Customer Success Platform. Einstein's weaknesses include modest visualization features and limited or unproven utility beyond the sales and marketing domains. IBM Watson is an AI system that organizations can apply in various use cases, such as advertising, customer service, financial operations and sales.


7 Best Artificial Intelligence Stocks To Buy and Watch Now

#artificialintelligence

Savvy investors will want to check out stocks that capitalize on artificial intelligence -- using computers to simulate human thinking, reasoning and problem solving. Companies that use AI technology often do so by using software to process huge amounts of data. As the pandemic of 2020 forced lockdowns, many businesses took advantage of AI to interact with the public, as face-to-face contact was limited. That trend seems likely to continue. In a November market forecast, Gartner Inc. predicted the market for AI would grow 21.3%, to $62.5 billion, in 2022.


What Is Adobe Sensei? How This Artificial Intelligence Tool Helps Creators

#artificialintelligence

So you've heard of Adobe Sensei but don't really know what it is or how you can access it. Or maybe, you haven't even heard of it at all. With so much new software and tools constantly coming out, we know researching takes a long time and draws you away from your creative work. We've got all the answers right here to your questions about Adobe's artificial intelligence technology. Adobe Sensei is Adobe's artificial intelligence tool that integrates with Adobe software.


Top AI-Powered Design Tools in 2022

#artificialintelligence

There are approximately 20 million more enjoyable things to do than manually removing the background. Avocode allows you to share design files, make changes that are automatically updated, and generate code styles for your design projects. This is yet another design to code converter that enables you to create web, iOS, and Android apps exactly as you want them, without leaving out any minor details. It also allows you to generate production-ready code in a variety of languages, including CSS, SCSS, CSS in JS, Android, and React Native. In this article, we discussed AI design tools that will assist you in creating some truly amazing designs. The best part about all of these tools is that they are all free to use, so you can start creating some awesome designs right away.


AI might help edit the next generation of blockbusters

#artificialintelligence

The next few Tuesdays, The Verge's flagship podcast The Vergecast is showcasing a miniseries dedicated to the use of artificial intelligence in industries that are often overlooked, hosted by Verge senior reporter Ashley Carman. This week, the series focuses on AI for the video world. More specifically, we're looking at how AI is being used as a tool to help people streamline the process of creating video content. Yes, this might mean software taking on a bigger role in the very human act of creativity, but what if instead of replacing us, machine learning tools could be used to assist our work? That's what Scott Prevost, VP of Adobe Sensei -- Adobe's machine learning platform -- envisions for Adobe's AI products. "Sensei was founded on this firm belief that we have that AI is going to democratize and amplify human creativity, but not replace it," Prevost says.


Europe's answer to Amazon Go

#artificialintelligence

Helen, a stay-at-home mum living in the north of Lisbon, has just done a weekly grocery shop. But instead of paying for her items at the cash register, she's walked straight out of the store without going to a checkout. The 34-year-old is one of the early customers of Europe's first autonomous store, which uses computer vision and machine learning to enable customers to shop without queuing, paying with a cashier or even getting their wallet or phone out. "I'm glad this store opened in my neighbourhood," she told Sifted on a recent visit. "It's so much more convenient to shop when you have a baby stroller. The store, which is a partnership between technology provider Sensei and the physical retailer Continente, is an early example of what Sensei hopes will soon be used by retailers across the continent. It aims to be Europe's answer to Amazon Go, the checkoutless shop created by Amazon which launched in London this March after testing the water in the US over the past three years. Sensei's bet is that ultimately all shops will be compelled to adopt this technology in pursuit of improved customer experience. "There's no hassle, no friction in the experience: if you forgot to buy water, you just go in, buy your water, come out –– it's super fast!


Adobe's 'Liquid Mode' uses AI to automatically redesign PDFs for mobile devices – TechCrunch

#artificialintelligence

We've probably all been there: You've been poking around your phone for an hour, deep in some sort of Google research rabbit hole. You finally find a link that almost certainly has the info you've been looking for. Now you get to pinch and zoom your way through a document that's clearly not meant for a screen that fits in your hand. Given that the file format is approaching its 30th birthday, it makes sense that PDFs aren't exactly built for modern mobile devices. But neither PDFs nor smartphones are going away anytime soon, so Adobe has been working on a way to make them play nicely together.


Adobe's Liquid Mode Leverages AI to Reformat PDFs for Mobile Devices

#artificialintelligence

Emergent Insight: On August 13, 2020 Emergent Enterprise posted an article about the incredibly poor UX of the PDF file. Perhaps Adobe has provided some relief with a new AI-fueled tool that reformats PDF files for better readability on mobile devices and presumably other formats such as AR & VR. Kyle Wiggers explains Liquid Mode from Adobe in this post at VentureBeat and how it might make life easier for all of us who swipe, scroll, pinch and zoom PDF files. Perhaps the most striking statistic here is that 65% find PDF files frustrating. And yet we throw them at our audiences with regularity.