Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, neural architecture search, and robotics. However, a closer examination reveals that the state-of-the-art methods for high-dimensional Bayesian optimization (HDBO) suffer from degrading performance as the number of dimensions increases, or even risk failure if certain unverifiable assumptions are not met. This paper proposes BAxUS that leverages a novel family of nested random subspaces to adapt the space it optimizes over to the problem. This ensures high performance while removing the risk of failure, which we assert via theoretical guarantees. A comprehensive evaluation demonstrates that BAxUS achieves better results than the state-of-the-art methods for a broad set of applications.

Large language models (LLMs) are used in chatbots or AI assistants to hold conversations with a human user. In such applications, the quality (e.g., user engagement, safety) of a conversation is important and can only be exactly known at the end of the conversation. To maximize its expected quality, conversation planning reasons about the stochastic transitions within a conversation to select the optimal LLM response at each turn. Existing simulation-based conversation planning algorithms typically select the optimal response by simulating future conversations with a large number of LLM queries at every turn. However, this process is extremely time-consuming and hence impractical for real-time conversations. This paper presents a novel approach called Semantic space COnversation Planning with improved Efficiency (SCOPE) that exploits the dense semantic representation of conversations to perform conversation planning efficiently. In particular, SCOPE models the stochastic transitions in conversation semantics and their associated rewards to plan entirely within the semantic space. This allows us to select the optimal LLM response at every conversation turn without needing additional LLM queries for simulation. As a result, SCOPE can perform conversation planning 70 times faster than conventional simulation-based planning algorithms when applied to a wide variety of conversation starters and two reward functions seen in the real world, yet achieving a higher reward within a practical planning budget. Our code can be found at: https://github.com/chenzhiliang94/convo-plan-SCOPE.

Speculative decoding is an effective method for accelerating inference of large language models (LLMs) by employing a small draft model to predict the output of a target model. However, when adapting speculative decoding to domain-specific target models, the acceptance rate of the generic draft model drops significantly due to domain shift. In this work, we systematically investigate knowledge distillation techniques for training domain draft models to improve their speculation accuracy. We compare white-box and black-box distillation approaches and explore their effectiveness in various data accessibility scenarios, including historical user queries, curated domain data, and synthetically generated alignment data. Our experiments across Function Calling, Biology, and Chinese domains show that offline distillation consistently outperforms online distillation by 11% to 25%, white-box distillation surpasses black-box distillation by 2% to 10%, and data scaling trends hold across domains. Additionally, we find that synthetic data can effectively align draft models and achieve 80% to 93% of the performance of training on historical user queries. These findings provide practical guidelines for training domain-specific draft models to improve speculative decoding efficiency.

Large language models (LLMs) deliver impressive performance but require large amounts of energy. In this work, we present a MatMul-free LLM architecture adapted for Intel's neuromorphic processor, Loihi 2. Our approach leverages Loihi 2's support for low-precision, event-driven computation and stateful processing. Our hardware-aware quantized model on GPU demonstrates that a 370M parameter MatMul-free model can be quantized with no accuracy loss. Based on preliminary results, we report up to 3x higher throughput with 2x less energy, compared to transformer-based LLMs on an edge GPU, with significantly better scaling. Further hardware optimizations will increase throughput and decrease energy consumption. These results show the potential of neuromorphic hardware for efficient inference and pave the way for efficient reasoning models capable of generating complex, long-form text rapidly and cost-effectively.

As robots are becoming increasingly intelligent and autonomous, from self-driving cars to assistive robots for vulnerable populations, important ethical questions inevitably emerge wherever and whenever such robots interact with humans and thereby impact human well-being. Questions that must be answered include whether such robots should be deployed in human societies in fairly unconstrained environments and what kinds of provisions are needed in robotic control systems to ensure that autonomous machines will not cause humans harms or at least minimize harm when it cannot be avoided. The goal of this specialty is to provide the first interdisciplinary forum for philosophers, psychologists, legal experts, AI researchers and roboticists to disseminate their work specifically targeting the ethical aspects of autonomous intelligent robots. Note that the conjunction of "AI and robotics" here indicates the journal's intended focus is on the ethics of intelligent autonomous robots, not the ethics of AI in general or the ethics of non-intelligent, non-autonomous machines. Examples of questions that we seek to address in this journal are: -- computational architectures for moral machines -- algorithms for moral reasoning, planning, and decision-making -- formal representations of moral principles in robots -- computational frameworks for robot ethics -- human perceptions and the social impact of moral machines -- legal aspects of developing and disseminating moral machines -- algorithms for learning and applying moral principles -- implications of robotic embodiment/physical presence in social space -- variance of ethical challenges across different contexts of human -robot interaction

Today we have all heard, at least once in our lives, about artificial intelligence. And although it seems like a term taken from a science fiction movie, the reality is that Artificial Intelligence is already part of our daily lives, you just are not aware of it. Artificial Intelligence (AI) has become an extremely powerful digital tool, which has been adopted by many organizations in almost all sectors, markets, and industries because this type of technology allows them to optimize work and profitability. You can see how AI is applied for a wide variety of purposes today, from medicine to the legal online casinos US industry. It is really surprising how this technology operates around us in most areas.

With the recent Intergovernmental Panel on Climate Change (IPPC) report painting a worrying picture of our battle against climate change, we will explore five ways analytics can help turn the tide. The UN Secretary-General, Antonio Guterres, called the report "a code red for humanity," adding that "the alarm bells are deafening and evidence irrefutable." U.S. President Joe Biden said about it, "The cost of inaction is mounting." In summary, without immediate action, the damage we've done may be irreversible. For this to change, we're going to have to rely on the latest tools and technologies, including big data, advanced analytics, modeling, and simulation techniques.

The tf$distribute$Strategy API provides an abstraction for distributing your training across multiple processing units. The goal is to allow users to enable distributed training using existing models and training code, with minimal changes. This tutorial uses the tf$distribute$MirroredStrategy, which does in-graph replication with synchronous training on many GPUs on one machine. Then, it uses all-reduce to combine the gradients from all processors and applies the combined value to all copies of the model. MirroredStategy is one of several distribution strategy available in TensorFlow core.

Tensorflow is not a Machine Learning specific library, instead, is a general purpose computation library that represents computations with graphs. Its core is implemented in C and there are also bindings for different languages. The bindings for the Go programming language, differently from the Python ones, are a useful tool not only for using Tensorflow in Go but also for understanding how Tensorflow is implemented under the hood. Officially, the Tensorflow's developers released: Being a Gopher and not a Java lover, I started looking at the Go bindings in order to understand what kind of tasks they were created for. The first thing to note is that the Go API, for admission of the maintainers itself, lacks the Variable support: this API is designed to use trained models and not for training models from scratch.