Automatic evaluation metrics capable of replacing human judgments are critical to allowing fast development of new methods. Thus, numerous research efforts have focused on crafting such metrics. In this work, we take a step back and analyze recent progress by comparing the body of existing automatic metrics and human metrics altogether. As metrics are used based on how they rank systems, we compare metrics in the space of system rankings. Our extensive statistical analysis reveals surprising findings: automatic metrics -- old and new -- are much more similar to each other than to humans. Automatic metrics are not complementary and rank systems similarly. Strikingly, human metrics predict each other much better than the combination of all automatic metrics used to predict a human metric. It is surprising because human metrics are often designed to be independent, to capture different aspects of quality, e.g. content fidelity or readability. We provide a discussion of these findings and recommendations for future work in the field of evaluation.

Previous research has shown that the fairness and the legitimacy of a moral decision-maker are important for people's acceptance of and compliance with the decision-maker. As technology rapidly advances, there have been increasing hopes and concerns about building artificially intelligent entities that are designed to intervene against norm violations. However, it is unclear how people would perceive artificial moral regulators that impose punishment on human wrongdoers. Grounded in theories of psychology and law, we predict that the perceived fairness of punishment imposed by a robot would increase the legitimacy of the robot functioning as a moral regulator, which would in turn, increase people's willingness to accept and comply with the robot's decisions. We close with a conceptual framework for building a robot moral regulator that successfully can regulate norm violations.

Recording the dynamics of unscripted human interactions in the wild is challenging due to the delicate trade-offs between several factors: participant privacy, ecological validity, data fidelity, and logistical overheads. To address these, following a 'datasets for the community by the community' ethos, we propose the Conference Living Lab (ConfLab): a new concept for multimodal multisensor data collection of in-the-wild free-standing social conversations. For the first instantiation of ConfLab described here, we organized a real-life professional networking event at a major international conference. Involving 48 conference attendees, the dataset captures a diverse mix of status, acquaintance, and networking motivations. Our capture setup improves upon the data fidelity of prior in-the-wild datasets while retaining privacy sensitivity: 8 videos (1920x1080, 60 fps) from a non-invasive overhead view, and custom wearable sensors with onboard recording of body motion (full 9-axis IMU), privacy-preserving low-frequency audio (1250 Hz), and Bluetooth-based proximity. Additionally, we developed custom solutions for distributed hardware synchronization at acquisition and time-efficient continuous annotation of body keypoints and actions at high sampling rates. Our benchmarks showcase some of the open research tasks related to in-the-wild privacy-preserving social data analysis: keypoints detection from overhead camera views, skeleton-based no-audio speaker detection, and F-formation detection.

It might not be a surprise that some city governments are not only unnerved by it, they're regulating it. Some government officials are understandably worried about artificial intelligence programs taking away jobs -- but lately, some municipalities appear to be concerned that AI is being used to help people get jobs. For instance, New York City and the District of Columbia are among locales that are enacting or considering laws to restrict how employers utilize artificial intelligence programs in hiring and promoting decisions. If you're unaware of what is transpiring in the world of human resources, AI and city governments, here's what is at stake. Increasingly, recruiters and human resources departments have been using AI tools to help find job candidates by performing repetitive and time-consuming tasks like analyzing resumes, arranging interviews with job candidates, and scheduling job assessments.

"When, for God's sake, are we going to confront the gun lobby?", said Joe Biden, saying too he was "sick and tired" of the blockage of part of theUS political class, which refuses to regulate gun sales. I used to talk about artificial intelligence &é the ethics that must be associated with it… AI is created by humans… like weapons… we're on the same subject! We need to obviously regulate AI algorithms so that they are not harmful to us humans… but apparently in the US that doesn't seem to be the case when we replace AI with weapons… catastrophic!! Just some evidence: 1/ Research on the subject across US is clear: the greater the number of firearms in circulation, the greater the increase in firearm violence. States with a large proportion of their population owning firearms have homicide rates that are 114% higher than those with a less armed population. A review of the scientific literature from Harvard University's Violence Prevention Research Center indicates that the availability of access to firearms is a well-established risk factor in numerous studies. Both men and women are at greater risk of being victims of firearm homicide in places where there are more guns.

The Biden administration unveiled a set of far-reaching goals Tuesday aimed at averting harms caused by the rise of artificial intelligence systems, including guidelines for how to protect people's personal data and limit surveillance. Earlier this year, after the publication of an AP review of an algorithmic tool used in a Pennsylvania child welfare system, OSTP staffers reached out to sources quoted in the article to learn more, according to multiple people who participated in the call. "If a tool or an automated system is disproportionately harming a vulnerable community, there should be, one would hope, that there would be levers and opportunities to address that through some of the specific applications and prescriptive suggestions," said Nelson, who also serves as deputy assistant to President Joe Biden. The white paper also did not specifically address AI-powered technologies funded through the Department of Justice, whose civil rights division separately has been examining algorithmic harms, bias and discrimination, Nelson said. Tucked between the calls for greater oversight, the white paper also said when appropriately implemented, AI systems have the power to bring about lasting benefits to society, such as helping farmers grow food more efficiently or identifying diseases.

This project is currently in development. The library provides access to machine learning algorithms and models in the browser, building on top of TensorFlow.js. The library is supported by code examples, tutorials, and sample data sets with an emphasis on ethical computing. Bias in data, stereotypical harms, and responsible crowdsourcing are part of the documentation around data collection and usage. Please read our Code of Conduct, which establishes our commitment to make ml5.js a friendly and welcoming environment.

Every legal case sets a precedent by developing the law in one of the following two ways. It either expands its scope, in which case it sets positive precedent, or it narrows it, in which case it sets negative precedent. Legal outcome prediction, the prediction of positive outcome, is an increasingly popular task in AI. In contrast, we turn our focus to negative outcomes here, and introduce a new task of negative outcome prediction. We discover an asymmetry in existing models' ability to predict positive and negative outcomes. Where the state-of-the-art outcome prediction model we used predicts positive outcomes at 75.06 F1, it predicts negative outcomes at only 10.09 F1, worse than a random baseline. To address this performance gap, we develop two new models inspired by the dynamics of a court process. Our first model significantly improves positive outcome prediction score to 77.15 F1 and our second model more than doubles the negative outcome prediction performance to 24.01 F1. Despite this improvement, shifting focus to negative outcomes reveals that there is still much room for improvement for outcome prediction models.

On the other hand, these research insights are meant to intervene on platforms that are globally present, serving a global population from diverse societies, cultures and values, with their own forms of injustices. A core concern in this arrangement is that of value imposition, where local values, i.e., values that are local to the regions where the interventions are built, implicitly shape and inform global systems without any or much room for discussion or contestation from those affected by those interventions. More specifically, interventions designed to address FATE failures necessarily impart a normative value system, but the values that guide the proposed solutions are rarely recognized as sites of contestation. This is problematic because while there may be ethical principles for ML that garner a degree of consensus across different value systems, in a pluralistic world this consensus is not something that should be assumed. Instead, we need to be explicit about the values that underpin the quest for ethical and just AI, and to cultivate an active debate about those values, critically examining and evaluating claims about them[28]. Another shortcoming of not being explicit about what normative value systems shape the interventions is the vagueness it entails, making it harder to arrive at a common vocabulary and shared understanding between computer scientists and civil society. Such a shared understanding is crucial to bridge the gap between research and practice, especially in a way that effectively supports the priorities of the latter constituency.

People are regularly confronted with potentially deceptive statements (e.g., fake news, misleading product reviews, or lies about activities). Only few works on automated text-based deception detection have exploited the potential of deep learning approaches. A critique of deep-learning methods is their lack of interpretability, preventing us from understanding the underlying (linguistic) mechanisms involved in deception. However, recent advancements have made it possible to explain some aspects of such models. This paper proposes and evaluates six deep-learning models, including combinations of BERT (and RoBERTa), MultiHead Attention, co-attentions, and transformers. To understand how the models reach their decisions, we then examine the model's predictions with LIME. We then zoom in on vocabulary uniqueness and the correlation of LIWC categories with the outcome class (truthful vs deceptive). The findings suggest that our transformer-based models can enhance automated deception detection performances (+2.11% in accuracy) and show significant differences pertinent to the usage of LIWC features in truthful and deceptive statements.