We study the problem of programmatic reinforcement learning, in which policies are represented as short programs in a symbolic language. Programmatic policies can be more interpretable, generalizable, and amenable to formal verification than neural policies; however, designing rigorous learning approaches for such policies remains a challenge. Our approach to this challenge - a meta-algorithm called PROPEL - is based on three insights. First, we view our learning task as optimization in policy space, modulo the constraint that the desired policy has a programmatic representation, and solve this optimization problem using a form of mirror descent that takes a gradient step into the unconstrained policy space and then projects back onto the constrained space. Second, we view the unconstrained policy space as mixing neural and programmatic representations, which enables employing state-of-the-art deep policy gradient approaches. Third, we cast the projection step as program synthesis via imitation learning, and exploit contemporary combinatorial methods for this task. We present theoretical convergence results for PROPEL and empirically evaluate the approach in three continuous control domains. The experiments show that PROPEL can significantly outperform state-of-the-art approaches for learning programmatic policies.

This paper considers how to fuse Machine Learning (ML) and optimization to solve large-scale Supply Chain Planning (SCP) optimization problems. These problems can be formulated as MIP models which feature both integer (non-binary) and continuous variables, as well as flow balance and capacity constraints. This raises fundamental challenges for existing integrations of ML and optimization that have focused on binary MIPs and graph problems. To address these, the paper proposes PROPEL, a new framework that combines optimization with both supervised and Deep Reinforcement Learning (DRL) to reduce the size of search space significantly. PROPEL uses supervised learning, not to predict the values of all integer variables, but to identify the variables that are fixed to zero in the optimal solution, leveraging the structure of SCP applications. PROPEL includes a DRL component that selects which fixed-at-zero variables must be relaxed to improve solution quality when the supervised learning step does not produce a solution with the desired optimality tolerance. PROPEL has been applied to industrial supply chain planning optimizations with millions of variables. The computational results show dramatic improvements in solution times and quality, including a 60% reduction in primal integral and an 88% primal gap reduction, and improvement factors of up to 13.57 and 15.92, respectively.

We study the problem of programmatic reinforcement learning, in which policies are represented as short programs in a symbolic language. Programmatic policies can be more interpretable, generalizable, and amenable to formal verification than neural policies; however, designing rigorous learning approaches for such policies remains a challenge. Our approach to this challenge - a meta-algorithm called PROPEL - is based on three insights. First, we view our learning task as optimization in policy space, modulo the constraint that the desired policy has a programmatic representation, and solve this optimization problem using a form of mirror descent that takes a gradient step into the unconstrained policy space and then projects back onto the constrained space. Second, we view the unconstrained policy space as mixing neural and programmatic representations, which enables employing state-of-the-art deep policy gradient approaches.

Bacteria and other small creatures squirming inside bodies often have to propel themselves through thick, viscous environments. For a human, this would look like someone awkwardly trying to swim their way through a pool of honey. Nature has already come up with creative solutions to this sticky problem. E.coli, for example, uses a corkscrew motion to cut through the muck while flagella contort their frames and whip themselves forward. Now, using this natural adaptation as inspiration, researchers from Tampere University and Anhui Jianzhu University have created a new doughnut-shaped micro-robot capable of autonomously navigating its way through mucus and other goopy substances.

Scientists have discovered that the way sperms swim defies Newton's law of motion, which states there is an equal and opposite reaction Researchers at Kyoto University found the sperms' flagella, or tail, propels the agents forward by changing their shape to interact with the fluid. Sperms do so in a non-reciprocal way, which violates Newton's third law because they do not elicit an equal and opposite reaction from their surroundings. The flagellum's elasticity also suggests that there should be no movement at all, but instead, sperms whip their tails without releasing much energy into their surroundings. Researchers at Kyoto University found the sperms' flagella, or tail, propels the agents forward by changing their shape to interact with the fluid The team used human sperm cells and algae for the research because both have flagella that help them propel through the liquid, New Scientist reports. Men's bulging waistlines are blamed for the worrying trend and'everywhere chemicals' in the environment.

The global Artificial Intelligence (AI) in healthcare market size was valued at USD 10.54 billion in 2021. The market is projected to grow from USD 13.82 billion in 2022 to USD 164.10 billion by 2029, exhibiting a CAGR of 42.4% during the forecast period. The global COVID-19 pandemic has been unprecedented and staggering, experiencing higher-than-anticipated demand across all regions compared to pre-pandemic levels. Based on our analysis, the global AI in healthcare market exhibited a higher growth of 42.4% in 2020 as compared to 2019. Fortune Business Insights says that the global market size was USD 10.54 billion in 2021 and is projected to reach USD 164.10 billion by 2029.

In 2021, the UK has found itself under the bright lights of the world stage many times. A global audience has watched our pandemic response, the fruition of BREXIT and, most recently, the UK's COP26 presidency. So, why has the UK's AI and wider tech scene still not made it close to global superpower status that we see from China, Russia and the US? The results of the government's recent National AI Strategy are yet to be seen, but I predict there is deeper change needed. A thriving AI industry needs a combination of education, ambition, and nurtured innovation.

The British technology sector enjoyed its best year of investment yet, attracting £29.4 billion in 2021. So, why hasn't the UK's AI and tech scene made it to the same global superpower status that we see from China, Russia, and the US? We don't need to wait for the results of the government's recent National AI Strategy, to know that deeper change is needed to really propel the industry forward. To create an AI scene that can compete on the world stage, education, ambition, and innovation must be combined and accelerated. Here are three tactics I believe could propel the UK's AI industry forward this year: To capitalise on the promise of AI, businesses and universities must build stronger links between one another, to drive innovation that can more quickly and effectively reach the market.

This course is designed to provide an initial exposure on Artificial Intelligence for kids in an interactive and playful manner by engaging them in projects such as identifying celebrities in an image, making a face detector, recognizing handwritten and printed text, building Machine Learning Models, designing simple animations etc in simplified way. With clear explanations and a wide array of exciting activities for kids, this AI curriculum for schools is ideal for providing a basic clarity on what is AI to young children. This course will serve as a perfect welcome for them into the world of AI and Automation.

Most games lose relevance after a few years, but the indie rocket-building game Kerbal Space Program is a bit different. It's a glitchy, 10-year-old underdog of a game with a cult following of programmers, engineers, astronaut candidates, and your typical lay explosion enthusiasts, and it has a unique and active community of modders who've been fixing bugs, adding new features, and generally keeping the game fresh for nearly a decade. In the game, you are the omniscient director of a space program composed of literal little green men (and beloved little green woman Valentina Kerman--we see you trailblazer) that you send skyward in spacecraft of your own design. It often feels like watching those blurry old videos of rockets launching only to come straight back down in an explosion of fiery schadenfreude: You feel a little bit frightened, a little bit sadistic, and you really want to try it again. One of the most prolific Kerbal modders is Chris Adderley, Nertea in the game, who is an engineer at the Canadian space company MDA by day, designing ground-based systems that retrieve data from spacecraft.