Goto

Collaborating Authors

ESP32 Machine Learning: ESP32 KNN classifier

#artificialintelligence

This tutorial describes how to use ESP32 Machine Learning. In more detail, it covers how to use an ESP32 KNN classifier to classify objects using their colors. To implement this ESP32 Machine Learning example, we will use a color sensor (TCS3200). This project derives from the Arduino Blog where it was used a KNN classifier to recognize different fruits. In this simple ESP32 KNN Machine Learning tutorial, we will replace the Arduino Nano 33 BLE with the ESP32 and we will add a color sensor because the ESP32 doesn't have a built-in sensor.


Simple Sensor Intentions for Exploration

arXiv.org Artificial Intelligence

Modern reinforcement learning algorithms can learn solutions to increasingly difficult control problems while at the same time reduce the amount of prior knowledge needed for their application. One of the remaining challenges is the definition of reward schemes that appropriately facilitate exploration without biasing the solution in undesirable ways, and that can be implemented on real robotic systems without expensive instrumentation. In this paper we focus on a setting in which goal tasks are defined via simple sparse rewards, and exploration is facilitated via agent-internal auxiliary tasks. We introduce the idea of simple sensor intentions (SSIs) as a generic way to define auxiliary tasks. SSIs reduce the amount of prior knowledge that is required to define suitable rewards. They can further be computed directly from raw sensor streams and thus do not require expensive and possibly brittle state estimation on real systems. We demonstrate that a learning system based on these rewards can solve complex robotic tasks in simulation and in real world settings. In particular, we show that a real robotic arm can learn to grasp and lift and solve a Ball-in-a-Cup task from scratch, when only raw sensor streams are used for both controller input and in the auxiliary reward definition.


Learning Sensor Multiplexing Design through Back-propagation

Neural Information Processing Systems

Recent progress on many imaging and vision tasks has been driven by the use of deep feed-forward neural networks, which are trained by propagating gradients of a loss defined on the final output, back through the network up to the first layer that operates directly on the image. We propose back-propagating one step further---to learn camera sensor designs jointly with networks that carry out inference on the images they capture. In this paper, we specifically consider the design and inference problems in a typical color camera---where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image. We learn the camera sensor's color multiplexing pattern by encoding it as layer whose learnable weights determine which color channel, from among a fixed set, will be measured at each location. These weights are jointly trained with those of a reconstruction network that operates on the corresponding sensor measurements to produce a full color image.


Raw Image Processing in Python

#artificialintelligence

Almost all modern-day cameras capture raw format images and process them in a format commonly known as sRGB, suitable for humans to see. However, one might wonder what all the techniques are used to convert the raw images into sRGB format are? Also, one might wonder how to use raw images or process them in a certain manner to get better performance on some machine learning tasks. This article attempts to answer all such questions in addition to step-by-step python code for each process. The machine learning algorithm behind those filters uses raw images to process the filter's image to give real-time results.


Learning Sensor Multiplexing Design through Back-propagation

Neural Information Processing Systems

Recent progress on many imaging and vision tasks has been driven by the use of deep feed-forward neural networks, which are trained by propagating gradients of a loss defined on the final output, back through the network up to the first layer that operates directly on the image. We propose back-propagating one step further---to learn camera sensor designs jointly with networks that carry out inference on the images they capture. In this paper, we specifically consider the design and inference problems in a typical color camera---where the sensor is able to measure only one color channel at each pixel location, and computational inference is required to reconstruct a full color image. We learn the camera sensor's color multiplexing pattern by encoding it as layer whose learnable weights determine which color channel, from among a fixed set, will be measured at each location. These weights are jointly trained with those of a reconstruction network that operates on the corresponding sensor measurements to produce a full color image. Our network achieves significant improvements in accuracy over the traditional Bayer pattern used in most color cameras. It automatically learns to employ a sparse color measurement approach similar to that of a recent design, and moreover, improves upon that design by learning an optimal layout for these measurements.