Overestimation in single-agent reinforcement learning has been extensively studied. In contrast, overestimation in the multiagent setting has received comparatively little attention although it increases with the number of agents and leads to severe learning instability. Previous works concentrate on reducing overestimation in the estimation process of target Q-value. They ignore the follow-up optimization process of online Q-network, thus making it hard to fully address the complex multiagent overestimation problem. To solve this challenge, in this study, we first establish an iterative estimation-optimization analysis framework for multiagent value-mixing Q-learning. Our analysis reveals that multiagent overestimation not only comes from the computation of target Q-value but also accumulates in the online Q-network's optimization. Motivated by it, we propose the Dual Ensembled Multiagent Q-Learning with Hypernet Regularizer algorithm to tackle multiagent overestimation from two aspects. First, we extend the random ensemble technique into the estimation of target individual and global Q-values to derive a lower update target. Second, we propose a novel hypernet regularizer on hypernetwork weights and biases to constrain the optimization of online global Q-network to prevent overestimation accumulation. Extensive experiments in MPE and SMAC show that the proposed method successfully addresses overestimation across various tasks.

Moreover, when we tackle a K-progress by propagating the label information from way classification problem with a large K, the binary detectors labeled data to unlabeled data (Berthelot et al. 2019; Xu et al. are less robust to identify outliers from such a complex 2021; Wang et al. 2022b; Zheng et al. 2022). Despite the dataset that contains multi-class information (Carbonneau success, SSL methods are deeply rooted in the closed-set assumption et al. 2018). One advanced method, evidential deep learning that labeled data, unlabeled data and test data share (EDL) (Sensoy, Kaplan, and Kandemir 2018) can explicitly the same predefined label set. In reality (Yu et al. 2020), such quantify the classification uncertainty corresponding an assumption may not always hold as we can only accurately to the unknown class, by treating the network's output as evidence control the label set of labeled data, while unlabeled for parameterizing the Dirichlet distribution according and test data may include outliers that belong to the novel to subjective logic (Jøsang 2016). Compared with Softmax classes that are not seen in labeled data.

Uncertainty estimation is a key factor that makes deep learning reliable in practical applications. Recently proposed evidential neural networks explicitly account for different uncertainties by treating the network's outputs as evidence to parameterize the Dirichlet distribution, and achieve impressive performance in uncertainty estimation. However, for high data uncertainty samples but annotated with the one-hot label, the evidence-learning process for those mislabeled classes is over-penalized and remains hindered. To address this problem, we propose a novel method, Fisher Information-based Evidential Deep Learning ($\mathcal{I}$-EDL). In particular, we introduce Fisher Information Matrix (FIM) to measure the informativeness of evidence carried by each sample, according to which we can dynamically reweight the objective loss terms to make the network more focused on the representation learning of uncertain classes. The generalization ability of our network is further improved by optimizing the PAC-Bayesian bound. As demonstrated empirically, our proposed method consistently outperforms traditional EDL-related algorithms in multiple uncertainty estimation tasks, especially in the more challenging few-shot classification settings.