The manufacturing sector like many other sectors was recently affected by workforce shortages, a problem that automation and robotics can heavily minimize Kugler (2022). Simultaneously, Reinforcement learning (RL) offers a promising solution where robots can learn to perform tasks through interaction and feedback from the environment Singh et al. (2022). However, despite their success in numerous simulation environments, we still don't see many real-world deployments of RL robotic solutions. In fact, many researchers either oversimplify the targeted real-world scenario such as Wu et al. (2023) or do not even evaluate their model in physical robots Lu et al. (2022); Na et al. (2022). It is known that training RL policies directly in real robots can be expensive, timeconsuming, labor-intensive, and maybe even dangerous, that is why it makes sense to try to leverage training in simulation.

Robotics can help address the growing worker shortage challenge of the manufacturing industry. As such, machine tending is a task collaborative robots can tackle that can also highly boost productivity. Nevertheless, existing robotics systems deployed in that sector rely on a fixed single-arm setup, whereas mobile robots can provide more flexibility and scalability. In this work, we introduce a multi-agent multi-machine tending learning framework by mobile robots based on Multi-agent Reinforcement Learning (MARL) techniques with the design of a suitable observation and reward. Moreover, an attention-based encoding mechanism is developed and integrated into Multi-agent Proximal Policy Optimization (MAPPO) algorithm to boost its performance for machine tending scenarios. Our model (AB-MAPPO) outperformed MAPPO in this new challenging scenario in terms of task success, safety, and resources utilization. Furthermore, we provided an extensive ablation study to support our various design decisions.

In the era of digital commerce, the surge in online shopping and the expectation for rapid delivery have placed unprecedented demands on warehouse operations. The traditional method of order fulfilment, where human order pickers traverse large storage areas to pick items, has become a bottleneck, consuming valuable time and resources. Robotic Mobile Fulfilment Systems (RMFS) offer a solution by using robots to transport storage racks directly to human-operated picking stations, eliminating the need for pickers to travel. This paper introduces'combi-stations'--a novel type of station that enables both item picking and replenishment, as opposed to traditional separate stations. We analyse the efficiency of combi-stations using queueing theory and demonstrate their potential to streamline warehouse operations. Our results suggest that combi-stations can reduce the number of robots required for stability and significantly reduce order turnover time, indicating a promising direction for future warehouse automation.

There is a growing interest in developing automated agents that can work alongside humans. In addition to completing the assigned task, such an agent will undoubtedly be expected to behave in a manner that is preferred by the human. This requires the human to communicate their preferences to the agent. To achieve this, the current approaches either require the users to specify the reward function or the preference is interactively learned from queries that ask the user to compare behavior. The former approach can be challenging if the internal representation used by the agent is inscrutable to the human while the latter is unnecessarily cumbersome for the user if their preference can be specified more easily in symbolic terms. In this work, we propose PRESCA (PREference Specification through Concept Acquisition), a system that allows users to specify their preferences in terms of concepts that they understand. PRESCA maintains a set of such concepts in a shared vocabulary. If the relevant concept is not in the shared vocabulary, then it is learned. To make learning a new concept more feedback efficient, PRESCA leverages causal associations between the target concept and concepts that are already known. In addition, we use a novel data augmentation approach to further reduce required feedback. We evaluate PRESCA by using it on a Minecraft environment and show that it can effectively align the agent with the user's preference.

In a robotic mobile fulfillment system, robots bring shelves, called pods, with storage items from the storage area to pick stations. At every pick station there is a person -- the picker -- who takes parts from the pod and packs them into boxes according to orders. Usually there are multiple shelves at the pick station. In this case, they build a queue with the picker at its head. When the picker does not need the pod any more, a robot transports the pod back to the storage area. At that time, we need to answer a question: "Where is the optimal place in the inventory to put this pod back?". It is a tough question, because there are many uncertainties to consider before answering it. Moreover, each decision made to answer the question influences the subsequent ones. The goal of this paper is to answer the question properly. We call this problem the Pod Repositioning Problem and formulate a deterministic model. This model is tested with different algorithms, including binary integer programming, cheapest place, fixed place, random place, genetic algorithms, and a novel algorithm called tetris.

Running out of charge for your smartphone could become a thing of the past, thanks to an outlandish plan from Amazon. A patent from the firm has revealed a scheme that would see'friendly' robots equipped with power sockets wandering through public places, like airports and shopping malls. The robots will come to your rescue when your battery is running low through an electronic call for help, most likely via WiFi, for a modest fee. An app could automate this process, sending out the distress signal once your reserves reach a certain limit. Running out of charge for your smartphone could become a thing of the past, thanks to an outlandish plan from Amazon.

Last week, Wal-Mart filed a patent for in-store service drones that could locate and deliver items to customers within the store. From the patent's description, the drones would be equipped with a number of sensors to be able to detect and grab the correct product and then drop it off at a designated landing area where consumers can grab the item. Just last fall, Wal-Mart also filed a patent for electronic self-driving shopping carts that can find the items on customers' shopping lists, and would also self-sort once a customer is finished with the cart, clearing aisles. Wal-Mart, which employs roughly 1.5 million people and is the 15th biggest public company in the world, makes about $482 billion in revenue a year. Adding drones to the mix could signal that Wal-Mart is looking to downsize its in-store employee number and replace them with robotic help.