Goto

Collaborating Authors

computation


On the edge -- deploying deep applications on constrained devices

#artificialintelligence

So many AI advancements get to headlines: "AI is beating humans in Go!"; "Deep weather forecasting"; "Talking Mona Lisa painting"… And yet I do not feel too excited… Despite the appeal on the outlook, these results are achieved with models that are sound proof of concept but are still too far from the real world applications. And the reason for that is simple -- their size. Bigger models with bigger datasets get better results. But these are neither sustainable in terms of the physical resources they consume, such as memory and power, nor in inference times, which are very far from the real-time performance required for many applications. Real-life problems require smaller models that can run on constrained devices.


How We Accidentally Gave our Bots Their Personalities

#artificialintelligence

A couple months ago we noticed that the process of optimizing our computer models to evaluate text can produce pretty cool personalities for different bots so we figured we'd share what we've learned so far. We hope these bots can help with some of the challenges we are facing with getting persistent state out of natural language generation. We hope writing about how we developed them can provide some tips for our users who are helping us create new bots. So what do we mean by personalities? Here's a few examples of the intermediate step and the desired output (a score) that we generated when we played a game where we told some of our earlier bots that we were writing this blog post.


Brief Review -- Natural Image Denoising with Convolutional Networks

#artificialintelligence

A convolutional network is an alternating sequence of linear filtering and nonlinear transformation operations. The input and output layers include one or more images, while intermediate layers contain "hidden" units with images called feature maps that are the internal computations of the algorithm. A convolutional network is an alternating sequence of linear filtering and nonlinear transformation operations. The input and output layers include one or more images, while intermediate layers contain "hidden" units with images called feature maps that are the internal computations of the algorithm. The border of the image is explicitly encoded by padding an area surrounding the image with values of -1.


New hardware offers faster computation for artificial intelligence, with much less energy

#artificialintelligence

As scientists push the boundaries of machine learning, the amount of time, energy, and money required to train increasingly complex neural network models is skyrocketing. A new area of artificial intelligence called analog deep learning promises faster computation with a fraction of the energy usage. Programmable resistors are the key building blocks in analog deep learning, just like transistors are the core elements for digital processors. By repeating arrays of programmable resistors in complex layers, researchers can create a network of analog artificial "neurons" and "synapses" that execute computations just like a digital neural network. This network can then be trained to achieve complex AI tasks like image recognition and natural language processing.


What has quantum computing got to do with AI?

#artificialintelligence

Artificial intelligence (AI) is emerging as one of the key industry trends after decades of just being a researcher's dream. From conversations with Alexa and Siri to Waymo (Google) and Tesla's vehicles driving themselves, OpenAI's GPT-3 writing prose like a human, and DeepMind (Google)'s AlphaZero beating human chess grandmasters, it is becoming clear that AI is now mature enough to resolve real-life problems and is often faster and better at it than humans. Elsewhere in the tech industry, several visionaries are working towards developing quantum computers, which seek to leverage the properties of quantum physics to perform calculations much faster than today's computers. At this point, you cannot be blamed for wondering: what exactly has quantum computing got to do with AI? Algorithmic complexity is a somewhat obscure mathematical concept that connects the work being done by AI researchers and quantum computing pioneers. Computational complexity theory, a field sitting across mathematics and computer science, focuses on classifying computational problems according to their resource usages, such as space (memory) and time.


ISM330IS, 1st Sensor with Intelligent Sensor Processing Unit for Greater AI at the Edge

#artificialintelligence

The SENSOR TEST 2022 conference took place last month in Nuremberg, Germany, where attendees got to check out the ISM330IS, the first sensor with an intelligent sensor processing unit (ISPU). ST announced the technology in early 2022, and its demonstration is highly symbolic as it testifies to its soon availability. In a nutshell, the ISPU is a C-programmable embedded digital signal processor (DSP) capable of running machine learning and deep learning algorithms. It is thus the next evolution of AI at the Edge, or what ST calls "The Onlife Era". Indeed, the ISM330IS includes a floating point unit for single-bit precision computations, a first in a motion sensor.


Spike Pattern Association Neuron (SPAN) Learning Model

#artificialintelligence

There's a supervised learning algorithm for SNN that enables a single neuron to learn spike pattern associations of input-output spike sequences at the precise times of spikes. This algorithm is termed SPAN(Spike Pattern Association Neuron). Anyone can build SNN to associate the input to output temporal patterns of desired spike sequences using this SPAN neuron. Here the input, output, and desired spike trains are transformed into analog signals by convolving the spikes with a kernel function. This transformation simplifies the computation of the error signal and, therefore, allows the application of gradient descent to optimize the synaptic weights.


Why distributed AI is key to pushing the AI innovation envelope

#artificialintelligence

The future of AI is distributed, said Ion Stoica, co-founder, executive chairman and president of Anyscale on the first day of VB Transform. And that's because model complexity shows no signs of slowing down. "For the past couple of years, the compute requirements to train a state-of-the-art model, depending on the data set, grow between 10 times and 35 times every 18 months," he said. Just five years ago the largest models were fitting on a single GPU; fast forward to today and just to fit the parameters of the most advanced models, it takes hundreds or even thousands of GPUs. PaLM, or the Pathway Language Model from Google, has 530 billion parameters -- and that's only about half of the largest, at more than 1 trillion parameters.


The Difficulty of Estimating the Carbon Footprint of Machine Learning - KDnuggets

#artificialintelligence

Machine learning (ML) often mimics how human brains operate by attaching virtual neurons with virtual synapses. Deep learning (DL) is a subset of ML putting steroids into the virtual brain and growing it orders of magnitude larger. This neuron count has skyrocketed hand-in-hand with the advances in computational power. Most headlines about ML solving hard problems like self-driving cars or facial recognition use DL, but the steroids come with a cost. Global warming is arguably the most critical problem our generation has to solve in the following years.


The virtuous cycle of AI research

#artificialintelligence

Many recent research efforts seek to construct neural networks capable of executing algorithmic computation, primarily to endow them with reasoning capabilities – which neural networks typically lack. Critically, every one of these papers generates its own dataset, which makes it hard to track progress, and raises the barrier of entry into the field. The CLRS benchmark, with its readily exposed dataset generators, and publicly available code, seeks to improve on these challenges. We've already seen a great level of enthusiasm from the community, and we hope to channel it even further during ICML. The main dream of our research on algorithmic reasoning is to capture the computation of classical algorithms inside high-dimensional neural executors.