NLP datasets annotated with human judgments are rife with disagreements between the judges. This is especially true for tasks depending on subjective judgments such as sentiment analysis or offensive language detection. Particularly in these latter cases, the NLP community has come to realize that the approach of 'reconciling' these different subjective interpretations is inappropriate. Many NLP researchers have therefore concluded that rather than eliminating disagreements from annotated corpora, we should preserve them-indeed, some argue that corpora should aim to preserve all annotator judgments. But this approach to corpus creation for NLP has not yet been widely accepted. The objective of the LeWiDi series of shared tasks is to promote this approach to developing NLP models by providing a unified framework for training and evaluating with such datasets. We report on the second LeWiDi shared task, which differs from the first edition in three crucial respects: (i) it focuses entirely on NLP, instead of both NLP and computer vision tasks in its first edition; (ii) it focuses on subjective tasks, instead of covering different types of disagreements-as training with aggregated labels for subjective NLP tasks is a particularly obvious misrepresentation of the data; and (iii) for the evaluation, we concentrate on soft approaches to evaluation. This second edition of LeWiDi attracted a wide array of participants resulting in 13 shared task submission papers.

The important Kemeny problem, which consists of computing median consensus rankings of an election with respect to the Kemeny voting rule, admits important applications in biology and computational social choice and was generalized recently via an interesting setwise approach by Gilbert et. al. Our first results establish optimal quantitative extensions of the Unanimity property and the well-known $3/4$-majority rule of Betzler et al. for the classical Kemeny median problem. Moreover, by elaborating an exhaustive list of quantified axiomatic properties (such as the Condorcet and Smith criteria, the $5/6$-majority rule, etc.) of the $3$-wise Kemeny rule where not only pairwise comparisons but also the discordance between the winners of subsets of three candidates are also taken into account, we come to the conclusion that the $3$-wise Kemeny voting scheme induced by the $3$-wise Kendall-tau distance presents interesting advantages in comparison with the classical Kemeny rule. For example, it satisfies several improved manipulation-proof properties. Since the $3$-wise Kemeny problem is NP-hard, our results also provide some of the first useful space reduction techniques by determining the relative orders of pairs of alternatives. Our works suggest similar interesting properties of higher setwise Kemeny voting schemes which justify and compensate for the more expensive computational cost than the classical Kemeny scheme.

Accurate, real-time measurements of price index changes using electronic records are essential for tracking inflation and productivity in today's economic environment. We develop empirical hedonic models that can process large amounts of unstructured product data (text, images, prices, quantities) and output accurate hedonic price estimates and derived indices. To accomplish this, we generate abstract product attributes, or ``features,'' from text descriptions and images using deep neural networks, and then use these attributes to estimate the hedonic price function. Specifically, we convert textual information about the product to numeric features using large language models based on transformers, trained or fine-tuned using product descriptions, and convert the product image to numeric features using a residual network model. To produce the estimated hedonic price function, we again use a multi-task neural network trained to predict a product's price in all time periods simultaneously. To demonstrate the performance of this approach, we apply the models to Amazon's data for first-party apparel sales and estimate hedonic prices. The resulting models have high predictive accuracy, with $R^2$ ranging from $80\%$ to $90\%$. Finally, we construct the AI-based hedonic Fisher price index, chained at the year-over-year frequency. We contrast the index with the CPI and other electronic indices.

Machine and deep learning methods for medical and healthcare applications have shown significant progress and performance improvement in recent years. These methods require vast amounts of training data which are available in the medical sector, albeit decentralized. Medical institutions generate vast amounts of data for which sharing and centralizing remains a challenge as the result of data and privacy regulations. The federated learning technique is well-suited to tackle these challenges. However, federated learning comes with a new set of open problems related to communication overhead, efficient parameter aggregation, client selection strategies and more. In this work, we address the step prior to the initiation of a federated network for model training, client recruitment. By intelligently recruiting clients, communication overhead and overall cost of training can be reduced without sacrificing predictive performance. Client recruitment aims at pre-excluding potential clients from partaking in the federation based on a set of criteria indicative of their eventual contributions to the federation. In this work, we propose a client recruitment approach using only the output distribution and sample size at the client site. We show how a subset of clients can be recruited without sacrificing model performance whilst, at the same time, significantly improving computation time. By applying the recruitment approach to the training of federated models for accurate patient Length of Stay prediction using data from 189 Intensive Care Units, we show how the models trained in federations made up from recruited clients significantly outperform federated models trained with the standard procedure in terms of predictive power and training time.

While conditional generation models can now generate natural language well enough to create fluent text, it is still difficult to control the generation process, leading to irrelevant, repetitive, and hallucinated content. Recent work shows that planning can be a useful intermediate step to render conditional generation less opaque and more grounded. We present a web browser-based demonstration for query-focused summarization that uses a sequence of question-answer pairs, as a blueprint plan for guiding text generation (i.e., what to say and in what order). We illustrate how users may interact with the generated text and associated plan visualizations, e.g., by editing and modifying the blueprint in order to improve or control the generated output. A short video demonstrating our system is available at https://goo.gle/text-blueprint-demo.

This was conducted through trialling and ranking various training patch selection sampling strategies, patch and batch sizes and data augmentations and scaling. We also compared model accuracy through producing the LULC classification using a single pass of a grid of patches and averaging multiple grid passes and three rotated version of each patch. Our results showed: a stratified random sampling approach for producing training patches improved the accuracy of classes with a smaller area while having minimal effect on larger classes; a smaller number of larger patches compared to a larger number of smaller patches improves model accuracy; applying data augmentations and scaling are imperative in creating a generalised model able to accurately classify LULC features in imagery from a different date and sensor; and producing the output classification by averaging multiple grids of patches and three rotated versions of each patch produced and more accurate and aesthetic result. Combining the findings from the trials, we fully trained five models on the 2018 training image and applied the model to the 2015 test image with the output LULC classifications achieving an average kappa of 0.84 user accuracy of 0.81 and producer accuracy of 0.87. This study has demonstrated the importance of data pre-processing for developing a generalised deep-learning model for LULC classification which can be applied to a different date and sensor. Future research using CNN and earth observation data should implement the findings of this study to increase LULC model accuracy and transferability.

This paper develops a trivalent semantics for the truth conditions and the probability of the natural language indicative conditional. Our framework rests on trivalent truth conditions first proposed by W. Cooper and yields two logics of conditional reasoning: (i) a logic C of inference from certain premises; and (ii) a logic U of inference from uncertain premises. But whereas C is monotonic for the conditional, U is not, and whereas C obeys Modus Ponens, U does not without restrictions. We show systematic correspondences between trivalent and probabilistic representations of inferences in either framework, and we use the distinction between the two systems to cast light, in particular, on McGee's puzzle about Modus Ponens. The result is a unified account of the semantics and epistemology of indicative conditionals that can be fruitfully applied to analyzing the validity of conditional inferences.

At the RSA security conference in San Francisco this week, there's been a feeling of inevitability in the air. At talks and panels across the sprawling Moscone convention center, at every vendor booth on the show floor, and in casual conversations in the halls, you just know that someone is going to bring up generative AI and its potential impacts on digital security and malicious hacking. NSA cybersecurity director Rob Joyce has been feeling it, too. "You can't walk around RSA without talking about AI and malware," he said on Wednesday afternoon during his now annual "State of the Hack" presentation. "I think we've all seen the explosion. I won't say it's delivered yet, but this truly is some game-changing technology."

Artificial intelligence was once something the average person described in the abstract. They had no tactile relationship with it that they were aware of, even if their devices were often utilizing it. That's all changed over the past year as people have started to engage with AI programs like OpenAI's DALL-E and ChatGPT, and the technology is rapidly advancing. As AI is democratized, democracy itself is falling under new pressures. There will likely be many exciting ways it will be deployed, but it may also start to distort reality and could become a major threat to the 2024 presidential election if AI-generated audio, images, and videos of candidates proliferate.

Twitter CEO Elon Musk provides insight on the consequences of developing artificial intelligence and the potential impact on elections on "Tucker Carlson Tonight." Congress is under increasing pressure from technology giants and others to find a way to regulate artificial intelligence, and a likely candidate for early action is a bill that both Republicans and Democrats supported in the last Congress under Democrat leadership. In 2022, the House Energy and Commerce Committee passed the American Data Privacy and Protection Act (ADPPA), a bill that's aimed at boosting data privacy rights but would also play a big role in regulating emerging AI systems. The ADPPA won almost unanimous support from both parties last year and continues to be supported by companies that are eager to build trust in their AI products, and they believe that a federal regulatory structure will help them get there. BSA/Software Alliance represents dozens of companies, including Microsoft, Okta, Salesforce and others, that build software and AI tools that companies use to run their businesses. BSA is working closely with the committee to get a version of that bill passed this year that it hopes can be approved in a full House vote.