Neural Information Processing Systems http://nips.cc/

--An active approach to fault tolerance is essential for robot swarms to achieve long-term autonomy. Previous e fforts have focused on responding to spontaneous electro-mechanical faults and failures. However, many faults occur gradually over time. This work argues that the principles of predictive maintenance, in which potential faults are resolved before they hinder the operation of the swarm, o ffer a promising means of achieving long-term fault tolerance. This is a novel approach to swarm fault tolerance, which is shown to give a comparable or improved performance when tested against a reactive approach in almost all cases tested. However, a significant barrier to the deployment of autonomous robots in many real-world applications is the risk of failure or loss of autonomous control in the field.

We study a wide class of non-convex non-concave min-max games that generalizes over standard bilinear zero-sum games.

Algorithm 2 shows how to prune k percentage edges from PC C following heuristic h . In this section, we provide detailed proofs of Theorem 1 (Section B.1) and Theorem 2 (Section B.2). Similar to the proof of Lemma 1, we prove Theorem 2 by induction. Equation 6. 17 Inductive case #1: suppose for all children of a product unit Finally, we prove the approximation step in Equation 3. Let Hardware specifications All experiments are performed on a server with 32 CPUs, 126G Memory, and NVIDIA RTX A5000 GPUs with 26G Memory. For MNIST -family datasets, we split 5% of training set as validation set for early stopping. Tree Bank dataset, we follow the setting in Mikolov et al.

A.1 Diagonal State Spaces We restate Proposition 1 for convenience. Equation 4 we have K " p Ce (Equation 7). This proves part (a) and we now consider part (b). (Equation 7). " 0 and hence softmax p0, iπq is not defined.

In Section 3.2, we propose a shifting operation in eq. As presented in Section 3.2, for an As explained in Sec 4.1, two criteria for the input distribution to the Tab. 5 shows the detailed results of The exact learned policy return are listed in Tab. 6. A higher return indicates a better learned policy.

--Over the past few years, the size of language models has grown exponentially, as has the computational cost to train these large models. This rapid growth has motivated researchers to develop new techniques aimed at enhancing the efficiency of the training process. Despite these advancements, optimally predicting the model size or allocating optimal resources remains a challenge. Several efforts have addressed the challenge by proposing different scaling laws, but almost all of them are architecture-specific (dense or sparse). In this work we revisit existing scaling laws and propose a generalized scaling law to provide a unified framework that is applicable to both dense and sparse large language models. We evaluate and compare our proposed scaling law with existing scaling laws to demonstrate its effectiveness. In recent years, transformer architectures [1] have revolutionized the deep learning approach. These architectures are now the foundation for the majority of popular large language models (LLMs).

This paper considers model-free discrete-action reinforcement learning, with the agent learning with variants of stochastic Policy Gradient. The paper introduces and discusses the Bregman Divergence, then presents how it can be used to build a policy loss that allows stable and efficient learning. The core idea of the paper, that I found is best shown by Equation 7, is to optimize the policy by simultaneously minimizing the change between pi_t and pi_t 1 and following the policy gradient. The main contribution of the paper is the use of the Bregman Divergence for the "minimizing change between pi_t and pi_t 1" part of the algorithm. The paper is well-written and interesting to read.

Weaknesses: No Explanation of Transformations of Stochastic Processes: I was under the impression that transforming / reparameterizing a stochasic process is non-trivial. Thus, I was expecting Equation 7 to include a second derivative term. I'm not saying that Equation 7 is wrong, per se---transforming just the increments agrees with intuition. However, the problem is that the paper provides no explanation or mathematical references for stochastic processes and their transformations. There are *zero* citations in both Section 2.2 and Section 3.1.