The advent of Internet of Things (IoT) technology has generated massive interest in voice-controlled smart homes. While many voice-controlled smart home systems are designed to understand and support widely spoken languages like English, speakers of low-resource languages like Luganda may need more support. This research project aimed to develop a Luganda speech intent classification system for IoT applications to integrate local languages into smart home environments. The project uses hardware components such as Raspberry Pi, Wio Terminal, and ESP32 nodes as microcontrollers. The Raspberry Pi processes Luganda voice commands, the Wio Terminal is a display device, and the ESP32 nodes control the IoT devices. The ultimate objective of this work was to enable voice control using Luganda, which was accomplished through a natural language processing (NLP) model deployed on the Raspberry Pi. The NLP model utilized Mel Frequency Cepstral Coefficients (MFCCs) as acoustic features and a Convolutional Neural Network (Conv2D) architecture for speech intent classification. A dataset of Luganda voice commands was curated for this purpose and this has been made open-source. This work addresses the localization challenges and linguistic diversity in IoT applications by incorporating Luganda voice commands, enabling users to interact with smart home devices without English proficiency, especially in regions where local languages are predominant.

Suppose you are alone at home or out for shopping or on vacations and someone breaks into your house. First thing comes into your mind is: if there is some gadget or home security system which can alert you or your neighbors. The home security camera does a good job but they may not work in complete dark. Also, you do not want your gadget to turn on false alarm if it is a cat. In this project I built a proof of concept which merely turn on an LED for demonstration purpose when it detects a person in light or dark using just a Raspberry Pi Pico and a low resolution thermal camera.

I have done my introduction to the Wio terminal at my previous post. This device can run TensorFlow lite micro. Meaning it can run a small neural network for to process data onboard. One use for this is processing sensor data, which the device does have several sensors that I have described in my earlier post.

Artificial intelligence used to happen almost exclusively in the cloud, but this introduces delays (latency) for the users and higher costs for the provider, so it's now very common to have on-device AI on mobile phones or other systems powered by application processors. But recently there's been a push to bring machine learning capabilities to even lower-end embedded systems powered by microcontrollers, as we've seen with GAP8 RISC-V IoT processor or Arm Cortex-M55 core and the Ethos-U55 micro NPU for Cortex-M microcontrollers, as well as Tensorflow Lite. Edge Impulse is another solution that aims to ease deployment of machine learning applications on Cortex-M embedded devices (aka Embedded ML or TinyML) by collecting real-world sensor data, training ML models on this data in the cloud, and then deploying the model back to the embedded device. The company collaborated with Arduino and announced support for the Arduino Nano 33 BLE Sense and other 32-bit Arduino boards last May. The solution supports motion sensing, computer vision, and audio recognition to detect glass breaking, hydraulic shocks, manufacturing defects, and so on.