Goto

Collaborating Authors

 ignatius


A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing

Gómez-Rodríguez, Carlos, Williams, Paul

arXiv.org Artificial Intelligence

We evaluate a range of recent LLMs on English creative writing, a challenging and complex task that requires imagination, coherence, and style. We use a difficult, open-ended scenario chosen to avoid training data reuse: an epic narration of a single combat between Ignatius J. Reilly, the protagonist of the Pulitzer Prize-winning novel A Confederacy of Dunces (1980), and a pterodactyl, a prehistoric flying reptile. We ask several LLMs and humans to write such a story and conduct a human evalution involving various criteria such as fluency, coherence, originality, humor, and style. Our results show that some state-of-the-art commercial LLMs match or slightly outperform our writers in most dimensions; whereas open-source LLMs lag behind. Humans retain an edge in creativity, while humor shows a binary divide between LLMs that can handle it comparably to humans and those that fail at it. We discuss the implications and limitations of our study and suggest directions for future research.


Workers of the Future, Unite

Huffington Post - Tech news and opinion

Automation will undoubtedly transform the workforce, the economy, and society as a whole. Robotics, sophisticated "AI" software, and other technologies will cause lasting and profound changes in the future. But that's no reason to ignore the problems we're facing right now. In fact, the best way to ensure that we have a more equitable economy tomorrow is by fighting for one today. Two recent commentaries addressed the "robot question."


Pentagon Eyes Deep Machine Learning in Fight Against ISIS

#artificialintelligence

There is huge potential for deep machine learning to become a valuable asset in the intelligence gathering space, according to Pentagon Deputy Secretary Robert Work -- it could ultimately allow U.S. forces to get an edge in the fight against the Islamic State of Iraq and Syria (ISIS, ISIL, IS) by providing greater insights into their networks and practices. Work made the statement during a roughly hour-long talk called Securing Tomorrow, held March 30 by the Washington Post, where he addressed some of the threat concerns facing the United States and the strategy the Department of Defense is deploying to overcome them. Moderated by Post columnist David Ignatius, the discussion also focused on how the behemoth agency is approaching new technologies and the perceived threats being seen from top international competitors, like Russia and China. "Without question, we are absolutely certain, that the use of deep learning machines is going to allow us to have a better understanding of ISIS as a network and a better understanding of how we can target it precisely and lead to its defeat." The evaluative capabilities and intelligence gathering promise of deep machine learning, Work said, has already shown great potential through the use of publicly available materials on social media, which paint a clearer picture of the events surrounding the downing of Malaysian passenger airliner MH17.