Modern cities produce vast streams of heterogeneous data, from infrastructure maps to mobility logs and satellite imagery. However, integrating these sources into coherent spatial models for planning and prediction remains a major challenge. Existing agent-centric methods often rely on direct environmental sensing, limiting scalability and raising privacy concerns. This paper introduces CubeletWorld, a novel framework for representing and analyzing urban environments through a discretized 3D grid of spatial units called cubelets. This abstraction enables privacy-preserving modeling by embedding diverse data signals, such as infrastructure, movement, or environmental indicators, into localized cubelet states. CubeletWorld supports downstream tasks such as planning, navigation, and occupancy prediction without requiring agent-driven sensing. To evaluate this paradigm, we propose the CubeletWorld State Prediction task, which involves predicting the cubelet state using a realistic dataset containing various urban elements like streets and buildings through this discretized representation. We explore a range of modified core models suitable for our setting and analyze challenges posed by increasing spatial granularity, specifically the issue of sparsity in representation and scalability of baselines. In contrast to existing 3D occupancy prediction models, our cubelet-centric approach focuses on inferring state at the spatial unit level, enabling greater generalizability across regions and improved privacy compliance. Our results demonstrate that CubeletWorld offers a flexible and extensible framework for learning from complex urban data, and it opens up new possibilities for scalable simulation and decision support in domains such as socio-demographic modeling, environmental monitoring, and emergency response. The code and datasets can be downloaded from here.

Artificial intelligence (AI) has the potential to transform education with its power of uncovering insights from massive data about student learning patterns. However, ethical and trustworthy concerns of AI have been raised but are unsolved. Prominent ethical issues in high school AI education include data privacy, information leakage, abusive language, and fairness. This paper describes technological components that were built to address ethical and trustworthy concerns in a multi-modal collaborative platform (called ALLURE chatbot) for high school students to collaborate with AI to solve the Rubik's cube. In data privacy, we want to ensure that the informed consent of children, parents, and teachers, is at the center of any data that is managed. Since children are involved, language, whether textual, audio, or visual, is acceptable both from users and AI and the system can steer interaction away from dangerous situations. In information management, we also want to ensure that the system, while learning to improve over time, does not leak information about users from one group to another.

Half a year has passed since my book "Deep Reinforcement Learning Hands-On" has seen the light. It took me almost a year to write the book and after some time of rest from writing I've discovered that explaining RL methods and turning theoretical papers into working code is a lot of fun for me and I don't want to stop. Luckily, RL domain is evolving, so, there are lots of topics to write about. In mass perception, Deep Reinforcement Learning is a tool to be used mostly for game playing. This is not surprising, given the fact, that historically, the first success in the field was achieved in Atari game suite by Deep Mind in 2015. Atari benchmark suite turned out to be very successful for RL problems and, even now, lots of research papers are using it for demonstrating the efficiency of their methods. As the RL field progresses, the classical 53 Atari games continue to become less and less challenging (at the time of writing more than half of games are solved with super-human accuracy) and researches turn to more complex games, like StarCraft and Dota2. But this bias towards games creates a false impression "RL is about playing games'', which is very far from the truth. In my book, published in June 2018, I've tried to counterbalance this by accompanying Atari games with the examples from other domains, including stock trading (chapter 8), chatbots and NLP problems (chapter 12), web navigation automation (chapter 13), continuous control (chapters 14…16) and boards games (chapter 18). In fact RL having very flexible MDP model potentially could be applied to a wide variety of domains, where computer games is just one convenient and spectacular example of the complicated decision making. In this article I've tried to write a detailed description of the recent attempt to apply RL to a field of combinatorial optimisation. The paper discussed was published by the group of researchers from UCI (University of California, Irvine) and called "Solving the Rubik's Cube Without Human Knowledge''.

Robots with truly humanlike dexterity are far from becoming reality, but progress accelerated by AI has brought us closer to achieving this vision than ever before. In a research paper published in September, a team of scientists at Google detailed their tests with a robotic hand that enabled it to rotate Baoding balls with minimal training data. And at a computer vision conference in June, MIT researchers presented their work on an AI model capable of predicting the tactility of physical things from snippets of visual data alone. Now, OpenAI -- the San Francisco-based AI research firm cofounded by Elon Musk and others, with backing from luminaries like LinkedIn cofounder Reid Hoffman and former Y Combinator president Sam Altman -- says it's on the cusp of solving something of a grand challenge in robotics and AI systems: solving a Rubik's cube. Unlike breakthroughs achieved by teams at the University of California, Irvine and elsewhere, which leveraged machines tailor-built to manipulate Rubik's cubes with speed, the approach devised by OpenAI researchers uses a five-fingered humanoid hand guided by an AI model with 13,000 years of cumulative experience -- on the same order of magnitude as the 40,000 years used by OpenAI's Dota-playing bot.

Incredibly, the system learned to dominate the classic 3D puzzle in just 44 hours and without any human intervention. "A generally intelligent agent must be able to teach itself how to solve problems in complex domains with minimal human supervision," write the authors of the new paper, published online at the arXiv preprint server. Indeed, if we're ever going to achieve a general, human-like machine intelligence, we'll have to develop systems that can learn and then apply those learnings to real-world applications. Recent breakthroughs in machine learning have produced systems that, without any prior knowledge, have learned to master games like chess and Go. But these approaches haven't translated very well to the Rubik's Cube.

The complexity classes PPA and PPAD were introduced in a seminal paper of Papadimitriou [47] in 1994, in an attempt to classify several natural problems in the class TFNP [45]. TFNP is the class of total search problems in NP for which a solution exists for every instance and the solution can be efficiently verified. Various important problems were subsequently proven to be complete for the class PPAD, such as the complexity of many versions of Nash equilibrium [16, 11, 21, 46, 50, 12] and market equilibrium computation [15, 9, 57, 13, 52]. For details on this, and the significance of PPAcompleteness, see the related discussion in [23], but note the following basic points. As evidence of computational hardness, PPA-completeness is stronger than PPAD-completeness: PPAD PPA. In terms of expressive power, the distinction is that PPA-complete problems can embed a search for a guaranteed fixpoint in a non-oriented topological space, but PPAD problems restrict us to an oriented one. Complete problems for the class PPA seemed to be much more elusive than PPAD-complete ones, especially when one is interested in "natural" problems, where "natural" here has the very specific meaning of problems that do not explicitly contain a circuit in their definition.

A generally intelligent agent must be able to teach itself how to solve problems in complex domains with minimal human supervision. Recently, deep reinforcement learning algorithms combined with self-play have achieved superhuman proficiency in Go, Chess, and Shogi without human data or domain knowledge. In these environments, a reward is always received at the end of the game; however, for many combinatorial optimization environments, rewards are sparse and episodes are not guaranteed to terminate. We introduce Autodidactic Iteration: a novel reinforcement learning algorithm that is able to teach itself how to solve the Rubik's Cube with no human assistance. Our algorithm is able to solve 100% of randomly scrambled cubes while achieving a median solve length of 30 moves -- less than or equal to solvers that employ human domain knowledge.