Goto

Collaborating Authors

 ljubljana


Automated Assignment Grading with Large Language Models: Insights From a Bioinformatics Course

Poličar, Pavlin G., Špendl, Martin, Curk, Tomaž, Zupan, Blaž

arXiv.org Artificial Intelligence

Providing students with individualized feedback through assignments is a cornerstone of education that supports their learning and development. Studies have shown that timely, high-quality feedback plays a critical role in improving learning outcomes. However, providing personalized feedback on a large scale in classes with large numbers of students is often impractical due to the significant time and effort required. Recent advances in natural language processing and large language models (LLMs) offer a promising solution by enabling the efficient delivery of personalized feedback. These technologies can reduce the workload of course staff while improving student satisfaction and learning outcomes. Their successful implementation, however, requires thorough evaluation and validation in real classrooms. We present the results of a practical evaluation of LLM-based graders for written assignments in the 2024/25 iteration of the Introduction to Bioinformatics course at the University of Ljubljana. Over the course of the semester, more than 100 students answered 36 text-based questions, most of which were automatically graded using LLMs. In a blind study, students received feedback from both LLMs and human teaching assistants without knowing the source, and later rated the quality of the feedback. We conducted a systematic evaluation of six commercial and open-source LLMs and compared their grading performance with human teaching assistants. Our results show that with well-designed prompts, LLMs can achieve grading accuracy and feedback quality comparable to human graders. Our results also suggest that open-source LLMs perform as well as commercial LLMs, allowing schools to implement their own grading systems while maintaining privacy.


Sarcasm Detection in a Less-Resourced Language

Đoković, Lazar, Robnik-Šikonja, Marko

arXiv.org Artificial Intelligence

The sarcasm detection task in natural language processing tries to classify whether an utterance is sarcastic or not. It is related to sentiment analysis since it often inverts surface sentiment. Because sarcastic sentences are highly dependent on context, and they are often accompanied by various non-verbal cues, the task is challenging. Most of related work focuses on high-resourced languages like English. To build a sarcasm detection dataset for a less-resourced language, such as Slovenian, we leverage two modern techniques: a machine translation specific medium-size transformer model, and a very large generative language model. We explore the viability of translated datasets and how the size of a pretrained transformer affects its ability to detect sarcasm. We train ensembles of detection models and evaluate models' performance. The results show that larger models generally outperform smaller ones and that ensembling can slightly improve sarcasm detection performance. Our best ensemble approach achieves an $\text{F}_1$-score of 0.765 which is close to annotators' agreement in the source language.


ILiAD: An Interactive Corpus for Linguistic Annotated Data from Twitter Posts

Gonzalez, Simon

arXiv.org Artificial Intelligence

Social Media platforms have offered invaluable opportunities for linguistic research. The availability of up-to-date data, coming from any part in the world, and coming from natural contexts, has allowed researchers to study language in real time. One of the fields that has made great use of social media platforms is Corpus Linguistics. There is currently a wide range of projects which have been able to successfully create corpora from social media. In this paper, we present the development and deployment of a linguistic corpus from Twitter posts in English, coming from 26 news agencies and 27 individuals. The main goal was to create a fully annotated English corpus for linguistic analysis. We include information on morphology and syntax, as well as NLP features such as tokenization, lemmas, and n- grams. The information is presented through a range of powerful visualisations for users to explore linguistic patterns in the corpus. With this tool, we aim to contribute to the area of language technologies applied to linguistic research.


Bandits in Matching Markets: Ideas and Proposals for Peer Lending

Sarkar, Soumajyoti

arXiv.org Artificial Intelligence

Motivated by recent applications of sequential decision making in matching markets, in this paper we attempt at formulating and abstracting market designs for P2P lending. We describe a paradigm to set the stage for how peer to peer investments can be conceived from a matching market perspective, especially when both borrower and lender preferences are respected. We model these specialized markets as an optimization problem and consider different utilities for agents on both sides of the market while also understanding the impact of equitable allocations to borrowers. We devise a technique based on sequential decision making that allow the lenders to adjust their choices based on the dynamics of uncertainty from competition over time and that also impacts the rewards in return for their investments. Using simulated experiments we show the dynamics of the regret based on the optimal borrower-lender matching and find that the lender regret depends on the initial preferences set by the lenders which could affect their learning over decision making steps.


DYMOND: DYnamic MOtif-NoDes Network Generative Model

Zeno, Giselle, La Fond, Timothy, Neville, Jennifer

arXiv.org Artificial Intelligence

Motifs, which have been established as building blocks for network structure, move beyond pair-wise connections to capture longer-range correlations in connections and activity. In spite of this, there are few generative graph models that consider higher-order network structures and even fewer that focus on using motifs in models of dynamic graphs. Most existing generative models for temporal graphs strictly grow the networks via edge addition, and the models are evaluated using static graph structure metrics -- which do not adequately capture the temporal behavior of the network. To address these issues, in this work we propose DYnamic MOtif-NoDes (DYMOND) -- a generative model that considers (i) the dynamic changes in overall graph structure using temporal motif activity and (ii) the roles nodes play in motifs (e.g., one node plays the hub role in a wedge, while the remaining two act as spokes). We compare DYMOND to three dynamic graph generative model baselines on real-world networks and show that DYMOND performs better at generating graph structure and node behavior similar to the observed network. We also propose a new methodology to adapt graph structure metrics to better evaluate the temporal aspect of the network. These metrics take into account the changes in overall graph structure and the individual nodes' behavior over time.


Local Clustering in Contextual Multi-Armed Bandits

Ban, Yikun, He, Jingrui

arXiv.org Artificial Intelligence

We study identifying user clusters in contextual multi-armed bandits (MAB). Contextual MAB is an effective tool for many real applications, such as content recommendation and online advertisement. In practice, user dependency plays an essential role in the user's actions, and thus the rewards. Clustering similar users can improve the quality of reward estimation, which in turn leads to more effective content recommendation and targeted advertising. Different from traditional clustering settings, we cluster users based on the unknown bandit parameters, which will be estimated incrementally. In particular, we define the problem of cluster detection in contextual MAB, and propose a bandit algorithm, LOCB, embedded with local clustering procedure. And, we provide theoretical analysis about LOCB in terms of the correctness and efficiency of clustering and its regret bound. Finally, we evaluate the proposed algorithm from various aspects, which outperforms state-of-the-art baselines.


SMASH Open Call 1 - 2023 • SMASH

#artificialintelligence

SMASH is an innovative, intersectoral, career-development training program for outstanding postdoctoral researchers, co-funded by the Marie Skłodowska-Curie Actions COFUND program. SMASH is open to researchers around the world who are interested in developing cutting-edge machine learning applications for science and humanities. During the five years of the SMASH project (2023-28) and over three planned calls, SMASH aims to hire 50 individuals for 2-year full-time postdoctoral contracts with highly attractive conditions. Fellows will be hosted by five leading Slovenian research institutions: the University of Nova Gorica, the University of Ljubljana, the Jožef Stefan Institute, the Institute of Information Science*, and the Slovenian Environment Agency*. Applicants should propose ambitious research projects in one of SMASH's five key research areas, that are centered on the use of cutting-edge machine learning, or more broadly, artificial intelligence techniques, to address some of the world's most challenging questions in: Applicants should choose the SMASH host institution and supervisor with whom they will coordinate the project proposal preparation.


Classification of Cross-cultural News Events

Sittar, Abdul, Mladenic, Dunja

arXiv.org Artificial Intelligence

We present a methodology to support the analysis of culture from text such as news events and demonstrate its usefulness on categorizing news events from different categories (society, business, health, recreation, science, shopping, sports, arts, computers, games and home) across different geographical locations (different places in 117 countries). We group countries based on the culture that they follow and then filter the news events based on their content category. The news events are automatically labelled with the help of Hofstedes cultural dimensions. We present combinations of events across different categories and check the performances of different classification methods. We also presents experimental comparison of different number of features in order to find a suitable set to represent the culture.


Forecasting Sensor Values in Waste-To-Fuel Plants: a Case Study

Brecelj, Bor, Šircelj, Beno, Rožanec, Jože M., Fortuna, Blaž, Mladenić, Dunja

arXiv.org Artificial Intelligence

In this research, we develop machine learning models to predict future sensor readings of a waste-to-fuel plant, which would enable proactive control of the plant's operations. We developed models that predict sensor readings for 30 and 60 minutes into the future. The models were trained using historical data, and predictions were made based on sensor readings taken at a specific time. We compare three types of models: (a) a n\"aive prediction that considers only the last predicted value, (b) neural networks that make predictions based on past sensor data (we consider different time window sizes for making a prediction), and (c) a gradient boosted tree regressor created with a set of features that we developed. We developed and tested our models on a real-world use case at a waste-to-fuel plant in Canada. We found that approach (c) provided the best results, while approach (b) provided mixed results and was not able to outperform the n\"aive consistently.


Machine Beats Machine: Machine Learning Models to Defend Against Adversarial Attacks

Rožanec, Jože M., Papamartzivanos, Dimitrios, Veliou, Entso, Anastasiou, Theodora, Keizer, Jelle, Fortuna, Blaž, Mladenić, Dunja

arXiv.org Artificial Intelligence

We propose using a two-layered deployment of machine learning Artificial Intelligence (AI) solutions have penetrated the Industry models to prevent adversarial attacks. The first layer determines 4.0 domain by revolutionizing the rigid production lines enabling whether the data was tampered, while the second layer solves a innovative functionalities like mass customization, predictive maintenance, domain-specific problem. We explore three sets of features and zero defect manufacturing, and digital twins. However, three dataset variations to train machine learning models. Our results AI-fuelled manufacturing floors involve many interactions between show clustering algorithms achieved promising results. In the AI systems and other legacy Information and Communications particular, we consider the best results were obtained by applying Technology (ICT) systems, generating a new territory for malevolent the DBSCAN algorithm to the structured structural similarity index actors to conquer. Hence, the threat landscape of Industry 4.0 is measure computed between the images and a white reference expanded unpredictably if we also consider the emergence of adversary image.