High replicability in personality psychology


According to the Big Five theory of personality, personality traits can be organized into five primary dimensions, including extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience. These dimensions are associated with myriad life outcomes, such as job satisfaction or health. Soto conducted a replication study of 78 previously published personality–life outcome findings after high-profile failures by others to replicate studies in other areas of psychology. Of personality–life outcome effects, 87% replicated successfully with effect sizes that were 77% as large as those in the original studies. Replicability was predicted by features of the original studies and the replication studies.

Reproducibility Challenges in Machine Learning for Health


Last year the United States Food and Drug Administration (FDA) cleared a total of 12 AI tools that use machine learning for health (ML4H) algorithms to inform medical diagnosis and treatment for patients. The tools are now allowed to be marketed, with millions of potential users in the US alone.Because ML4H tools directly affect human health, their development from experiments in labs to deployment in hospitals progresses under heavy scrutiny. A critical component of this process is reproducibility. A team of researchers from MIT, University of Toronto, New York University, and Evidation Health have proposed a number of "recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward" in their new paper Reproducibility in Machine Learning for Health. Just as boxers show their strength in the ring by getting up again after being knocked to the canvas, researchers test their strength in the arena of science by ensuring their work's reproducibility.

The Integrity Of Artificial Intelligence Is Under Threat - Disruption Hub


Artificial Intelligence is growing up fast. Although modern computers were only invented in the mid 20th century, they have already evolved into the complex machines we rely on today. Artificial Intelligence now governs a large proportion of consumer and business behaviour: from the way we use the internet, manufacture goods, and even hire and fire our workforces. However, as with any technology, when things grow too quickly, problems can arise. Artificial Intelligence as a scientific discipline might be struggling to keep up with the pace of change.

Sad Face


Leonard Martin agrees with Strack's concerns and says the replicators didn't fully follow their procedure. The work was so sloppy, he argued via email, that "the real story here may not be about the replicability of the pen in the mouth study or the replicability of psychology research in general but about the current method of assessing replicability." Given that such efforts can alter established findings in the field and tarnish people's reputations, he said that "Project Replication" should be very careful: "If the current lack of rigor continues, then psychology may find itself in its own version of the McCarthy era." Strack had one more concern: "What I really find very deplorable is that this entire replication thing doesn't have a research question." It does "not have a specific hypothesis, so it's very difficult to draw any conclusions," he told me.

Beyond-Quantum Modeling of Question Order Effects and Response Replicability in Psychological Measurements

arXiv.org Artificial Intelligence

A general tension-reduction (GTR) model was recently considered to derive quantum probabilities as (universal) averages over all possible forms of non-uniform fluctuations, and explain their considerable success in describing experimental situations also outside of the domain of physics, for instance in the ambit of quantum models of cognition and decision. Yet, this result also highlighted the possibility of observing violations of the predictions of the Born rule, in those situations where the averaging would not be large enough, or would be altered because of the combination of multiple measurements. In this article we show that this is indeed the case in typical psychological measurements exhibiting question order effects, by showing that their statistics of outcomes are inherently non-Hilbertian, and require the larger framework of the GTR-model to receive an exact mathematical description. We also consider another unsolved problem of quantum cognition: response replicability. It is has been observed that when question order effects and response replicability occur together, the situation cannot be handled anymore by quantum theory. However, we show that it can be easily and naturally described in the GTR-model. Based on these findings, we motivate the adoption in cognitive science of a hidden-measurements interpretation of the quantum formalism, and of its GTR-model generalization, as the natural interpretational framework explaining the data of psychological measurements on conceptual entities.