Honey bees pollinate about one-third of the world's food supply, but bee colonies have alarmingly declined by nearly 40% over the past decade due to several factors, including pesticides and pests. Traditional methods for monitoring beehives, such as human inspection, are subjective, disruptive, and time-consuming. To overcome these limitations, artificial intelligence has been used to assess beehive health. However, previous studies have lacked an end-to-end solution and primarily relied on data from a single source, either bee images or sounds. This study introduces a comprehensive system consisting of bee object detection and health evaluation. Additionally, it utilized a combination of visual and audio signals to analyze bee behaviors. An Attention-based Multimodal Neural Network (AMNN) was developed to adaptively focus on key features from each type of signal for accurate bee health assessment. The AMNN achieved an overall accuracy of 92.61%, surpassing eight existing single-signal Convolutional Neural Networks and Recurrent Neural Networks. It outperformed the best image-based model by 32.51% and the top sound-based model by 13.98% while maintaining efficient processing times. Furthermore, it improved prediction robustness, attaining an F1-score higher than 90% across all four evaluated health conditions. The study also shows that audio signals are more reliable than images for assessing bee health. By seamlessly integrating AMNN with image and sound data in a comprehensive bee health monitoring system, this approach provides a more efficient and non-invasive solution for the early detection of bee diseases and the preservation of bee colonies.

The robotic system is shown in an experimental hive Artificial Life Lab/U. of Graz/Hiveopolis Honeybees are famously finicky when it comes to being studied. Research instruments and conditions and even unfamiliar smells can disrupt a colony's behavior. Now, a joint research team from the Mobile Robotic Systems Group in EPFL's School of Engineering and School of Computer and Communication Sciences and the Hiveopolis project at Austria's University of Graz have developed a robotic system that can be unobtrusively built into the frame of a standard honeybee hive. "Many rules of bee society – from collective and individual interactions to raising a healthy brood – are regulated by temperature, so we leveraged that for this study," explains EPFL PhD student Rafael Barmak, first author on a paper on the system recently published in Science Robotics. "The thermal sensors create a snapshot of the bees' collective behavior, while the actuators allow us to influence their movement by modulating thermal fields."

Honeybees are famously finicky when it comes to being studied. Research instruments and conditions and even unfamiliar smells can disrupt a colony's behavior. Now, a joint research team from the Mobile Robotic Systems Group in EPFL's School of Engineering and School of Computer and Communication Sciences and the Hiveopolis project at Austria's University of Graz have developed a robotic system that can be unobtrusively built into the frame of a standard honeybee hive. "Many rules of bee society--from collective and individual interactions to raising a healthy brood--are regulated by temperature, so we leveraged that for this study," explains EPFL Ph.D. student Rafael Barmak, first author on a paper on the system recently published in Science Robotics. "The thermal sensors create a snapshot of the bees' collective behavior, while the actuators allow us to influence their movement by modulating thermal fields."