Throughout this project, we have attempted to develop an automatic evaluator that determines whether Basque language compositions meet the C1 level. To achieve our goal, we obtained 10,000 transcribed compositions through an agreement between HABE and HiTZ to train our system. We have developed different techniques to avoid data scarcity and system overfitting: EDA, SCL and regulation; We have also conducted tests with different Language Models to analyze their behavior. Finally, we have also performed analyses of different system behaviors to measure model calibration and the impact of artifacts. -- Proiektu honetan zehar euskarazko idazlanek C1 maila duten edo ez zehazten duen ebaluatzaile automatiko bat garatzen saiatu gara. Gure helburua betetzeko HABE eta HiTZ arteko hitzarmenaren bitartez 10.000 transkribatutako idazlan eskuratu ditugu gure sistema entrenatzeko. Datu eskasia eta sistemaren gaindoitzea ekiditeko teknika ezberdinak landu ditugu: EDA, SCL eta erregulazioa; Hizkuntza Eredu ezberdinekin ere probak egin ditugu duten portaera aztertzeko. Azkenik, sistema ezberdinen portaeren analisiak ere egin ditugu, ereduen kalibrazioa eta artefaktuen eragina neurtzeko.

Autonomous systems (AS) often use Deep Neural Network (DNN) classifiers to allow them to operate in complex, high-dimensional, non-linear, and dynamically changing environments. Due to the complexity of these environments, DNN classifiers may output misclassifications during operation when they face domains not identified during development. Removing a system from operation for retraining becomes impractical as the number of such AS increases. To increase AS reliability and overcome this limitation, DNN classifiers need to have the ability to adapt during operation when faced with different operational domains using a few samples (e.g. 2 to 100 samples). However, retraining DNNs on a few samples is known to cause catastrophic forgetting and poor generalisation. In this paper, we introduce Dynamic Incremental Regularised Adaptation (DIRA), an approach for dynamic operational domain adaption of DNNs using regularisation techniques. We show that DIRA improves on the problem of forgetting and achieves strong gains in performance when retraining using a few samples from the target domain. Our approach shows improvements on different image classification benchmarks aimed at evaluating robustness to distribution shifts (e.g.CIFAR-10C/100C, ImageNet-C), and produces state-of-the-art performance in comparison with other methods from the literature.

In this paper, we give an overview of a recently developed method for dynamic domain adaptation, named DIRA, which relies on a few samples in addition to a regularisation approach, named elastic weight consolidation, to achieve state-of-the-art (SOTA) domain adaptation results. DIRA has been previously shown to perform competitively with SOTA unsupervised adaption techniques. However, a limitation of DIRA is that it relies on labels to be provided for the few samples used in adaption. This makes it a supervised technique. In this paper, we propose a modification to the DIRA method to make it self-supervised i.e. remove the need for providing labels. Our proposed approach will be evaluated experimentally in future work.