Goto

Collaborating Authors

 government operation


Assessing the Reliability of Large Language Models for Deductive Qualitative Coding: A Comparative Study of ChatGPT Interventions

arXiv.org Artificial Intelligence

In this study, we investigate the use of large language models (LLMs), specifically ChatGPT, for structured deductive qualitative coding. While most current research emphasizes inductive coding applications, we address the underexplored potential of LLMs to perform deductive classification tasks aligned with established human-coded schemes. Using the Comparative Agendas Project (CAP) Master Codebook, we classified U.S. Supreme Court case summaries into 21 major policy domains. We tested four intervention methods: zero-shot, few-shot, definition-based, and a novel Step-by-Step Task Decomposition strategy, across repeated samples. Performance was evaluated using standard classification metrics (accuracy, F1-score, Cohen's kappa, Krippendorff's alpha), and construct validity was assessed using chi-squared tests and Cramer's V. Chi-squared and effect size analyses confirmed that intervention strategies significantly influenced classification behavior, with Cramer's V values ranging from 0.359 to 0.613, indicating moderate to strong shifts in classification patterns. The Step-by-Step Task Decomposition strategy achieved the strongest reliability (accuracy = 0.775, kappa = 0.744, alpha = 0.746), achieving thresholds for substantial agreement. Despite the semantic ambiguity within case summaries, ChatGPT displayed stable agreement across samples, including high F1 scores in low-support subclasses. These findings demonstrate that with targeted, custom-tailored interventions, LLMs can achieve reliability levels suitable for integration into rigorous qualitative coding workflows.


UAE: How artificial intelligence will help smoothen government operations

#artificialintelligence

Artificial intelligence (AI) will smoothen the government operations and also accelerate the pace of developments of all the governments as AI's role increases and becomes more mainstream, say public and private executives. "It's going to be interesting how artificial intelligence will react on smoothing the governments challenges through adopting the behaviours and with creative model. The ultimate goal of AI is to increase the value of the work, cut the cost, and save time. Therefore, it will accelerate the pace of the developments in all governments," said Dr. Ebrahim Al Alkeem Al Zaabi, director of cybersecurity and artificial intelligence at Government of Abu Dhabi. A study by Oliver Wyman, a global management consulting firm, has estimated that the efficiencies generated by AI technology can support Middle Eastern government budgets by up to $7 billion (Dh25.7 billion) annually.


(Summary) Demystifying artificial intelligence in government

#artificialintelligence

Artificial intelligence already helps run government, with cognitive applications doing everything from reducing backlogs and cutting costs to handling tasks we can't easily do on our own, such as predicting fraudulent transactions and identifying criminal suspects via facial recognition. Indeed, while we expect AI-based technology in the years ahead to fundamentally transform how public-sector employees get work done--eliminating some jobs, redesigning countless others, and even creating entirely new professions1--it's already changing the nature of many jobs and revolutionizing facets of government operations. Agencies today face new choices about whether some work should be fully automated, divided among people and machines, or performed by people but enhanced by machines. Our latest report, AI-augmented government, conservatively estimates that simply automating tasks that computers already routinely do could free up 96.7 million federal government working hours annually, potentially saving $3.3 billion. At the high end, we estimate that AI technology could free up as many as 1.2 billion working hours every year, saving $41.1 billion.


How artificial intelligence could transform government

#artificialintelligence

Let our Chatbot help--type your question above to explore AI topics. Artificial intelligence already helps run government, with cognitive applications doing everything from reducing backlogs and cutting costs to handling tasks we can't easily do on our own, such as predicting fraudulent transactions and identifying criminal suspects via facial recognition. Indeed, while we expect AI-based technology in the years ahead to fundamentally transform how public-sector employees get work done--eliminating some jobs, redesigning countless others, and even creating entirely new professions1--it's already changing the nature of many jobs and revolutionizing facets of government operations. Agencies today face new choices about whether some work should be fully automated, divided among people and machines, or performed by people but enhanced by machines. Our latest report, AI-augmented government, conservatively estimates that simply automating tasks that computers already routinely do could free up 96.7 million federal government working hours annually, potentially saving $3.3 billion.


How artificial intelligence could transform government

#artificialintelligence

Artificial intelligence already helps run government, with cognitive applications doing everything from reducing backlogs and cutting costs to handling tasks we can't easily do on our own, such as predicting fraudulent transactions and identifying criminal suspects via facial recognition. Indeed, while we expect AI-based technology in the years ahead to fundamentally transform how public-sector employees get work done--eliminating some jobs, redesigning countless others, and even creating entirely new professions1--it's already changing the nature of many jobs and revolutionizing facets of government operations. Agencies today face new choices about whether some work should be fully automated, divided among people and machines, or performed by people but enhanced by machines. Our latest report, AI-augmented government, conservatively estimates that simply automating tasks that computers already routinely do could free up 96.7 million federal government working hours annually, potentially saving $3.3 billion. At the high end, we estimate that AI technology could free up as many as 1.2 billion working hours every year, saving $41.1 billion.