Large Language Models (LLMs) can correct their self-generated responses, but a decline in accuracy after self-correction is also witnessed. To have a deeper understanding of self-correction, we endeavor to decompose, evaluate, and analyze the self-correction behaviors of LLMs. By enumerating and analyzing answer correctness before and after self-correction, we decompose the self-correction capability into confidence (being confident to correct answers) and critique (turning wrong answers to correct) capabilities, and propose two metrics from a probabilistic perspective to measure these 2 capabilities, along with another metric for overall self-correction capability evaluation. Based on our decomposition and evaluation metrics, we conduct extensive experiments and draw some empirical conclusions. For example, we find different models can exhibit distinct behaviors: some models are confident while others are more critical. We also find the trade-off between the two capabilities (i.e. improving one can lead to a decline in the other) when manipulating model self-correction behavior by prompts or in-context learning. Further, we find a simple yet efficient strategy to improve self-correction capability by transforming Supervision Fine-Tuning (SFT) data format, and our strategy outperforms vanilla SFT in both capabilities and achieves much higher accuracy after self-correction. Our code will be publicly available on GitHub.

One of the key strengths of Large Language Models (LLMs) is their ability to interact with humans by generating appropriate responses to given instructions. This ability, known as instruction-following capability, has established a foundation for the use of LLMs across various fields and serves as a crucial metric for evaluating their performance. While numerous evaluation benchmarks have been developed, most focus solely on clear and coherent instructions. However, we have noted that LLMs can become easily distracted by instruction-formatted statements, which may lead to an oversight of their instruction comprehension skills. To address this issue, we introduce the Intention of Instruction (IoInst) benchmark. This benchmark evaluates LLMs' capacity to remain focused and understand instructions without being misled by extraneous instructions. The primary objective of this benchmark is to identify the appropriate instruction that accurately guides the generation of a given context. Our findings suggest that even recently introduced state-of-the-art models still lack instruction understanding capability. Along with the proposition of IoInst in this study, we also present broad analyses of the several strategies potentially applicable to IoInst.

We compare different methods for extracting The multimodal affective computing field has seen audio features as well as different fusion significant advances in feature extraction and multimodal network methods to combine audio and text signals fusion methodologies in recent years. By to identify the best-performing procedures. We find combining audio, text and visual signals, these that the addition of audio signals consistently improves models offer a more comprehensive, nuanced understanding performance and also that our transformer of human emotions. However, there fusion network further enhances results for most are still limitations: hand-crafted feature extraction metrics and achieves state-of-the-art results across algorithms often lack flexibility and generalization all datasets, indicating its efficacy in enhancing across diverse tasks. To overcome these limitations, cross-modality modeling and its potential for multimodal recent studies have proposed fully end-to-end models emotion detection. From multi-loss training, that optimize both feature extraction and learning we also observe that 1) using distinct labels for processes jointly (Dai et al., 2021). Our work each modality in multi-loss training significantly extracts feature representations from pre-trained benefits the models' performance, and 2) training models for different modalities and combines them on multimodal features improves not only the overall in an end-to-end manner, which provides a comprehensive model performance but also the model's accuracy and adaptable solution for multimodal on the single-modality subnet.