Large language models (LLMs) have revolutionized the field of NLP. Notably, their in-context learning capabilities also enable their use as evaluation metrics for natural language generation, making them particularly advantageous in low-resource scenarios and time-restricted applications. In this work, we introduce PrExMe, a large-scale prompt exploration for metrics, where we evaluate more than 720 prompt templates for open-source LLM-based metrics on machine translation (MT) and summarization datasets, totalling over 6.6M evaluations. This extensive comparison (1) serves as a benchmark of the performance of recent open-source LLMs as metrics and (2) explores the stability and variability of different prompting strategies. We discover that, on the one hand, there are scenarios for which prompts are stable. For instance, some LLMs show idiosyncratic preferences and favor to grade generated texts with textual labels while others prefer to return numeric scores. On the other hand, the stability of prompts and model rankings can be susceptible to seemingly innocuous changes. For example, changing the requested output format from "0 to 100" to "-1 to +1" can strongly affect the rankings in our evaluation. Our study contributes to understanding the impact of different prompting approaches on LLM-based metrics for MT and summarization evaluation, highlighting the most stable prompting patterns and potential limitations.

Calls to regulate artificial intelligence are growing on Capitol Hill following a dire warning from tech giants. But many lawmakers also admit they don't know much more about the technology than the average American. WASHINGTON, D.C. – Calls to regulate artificial intelligence are growing on Capitol Hill following a dire warning from tech giants. But many lawmakers also admit they don't know much more about the technology than the average American. "I've had ChatGPT demonstrated to me by a friend, and its capabilities are kind of intimidating," Sen. Cynthia Lummis told Fox News.

If you hadn't heard, Elon Musk is worried about the machines. Though that may seem a quixotic stance for the head of multiple tech companies to take, it seems that his proximity to the bleeding edge of technological development has given him the heebie-jeebies when it comes to artificial intelligence. He's shared his fears of AI running amok before, likening it to "summoning the demon," and Musk doubled down on his stance at a meeting of the National Governors Association this weekend, telling state leaders that AI poses an existential threat to humanity. "Until people see robots going down the street killing people, they don't know how to react because it seems so ethereal. AI is a rare case where I think we need to be proactive in regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it's too late," according to the MIT Tech Review.

The tech tyrant, who is the CEO of SpaceX and Tesla, called for the regulation of Artificial Intelligence (AI). Musk, 48, was speaking at the South by Southwest tech conference in Austin, Texas, last year. He used his own products as an example of the leaps that science has already taken in modern time.

The entire world's industry seems to be converting into a services business. As automation takes over and industrial production becomes dominated by robots and automated assembly lines, and every type of industry drives for efficiency, many of the largest companies in the world will be service companies, who are also creating the jobs which earlier may have been created by the manufacturing industry. This shift that has happened gradually over the years, has profound implications for how economies grow and how cities themselves will evolve. And particularly in countries like India, it can dramatically change the outlook for the economy and our competitive position in the world. Amongst all the dire warnings being sounded around the world regarding Artificial Intelligence and Automation and Machine Learning, and what that will do to jobs, I remain immensely optimistic.

Lord Martin Rees, Astronomer Royal and University of Cambridge Emeritus Professor of Cosmology and Astrophysics, believes that machines could surpass humans within a few hundred years, ushering in eons of domination. He also cautions that while we will certainly discover more about the origins of biological life in the coming decades, we should recognize that alien intelligence may be electronic. "Just because there's life elsewhere doesn't mean that there is intelligent life," Lord Rees told The Conversation. "My guess is that if we do detect an alien intelligence, it will be nothing like us. It will be some sort of electronic entity."

A full-scale figure of a "T-800" terminator robot used in the movie Terminator 2, is displayed at a preview of the Terminator Exhibition in Tokyo in 2009. A full-scale figure of a "T-800" terminator robot used in the movie Terminator 2, is displayed at a preview of the Terminator Exhibition in Tokyo in 2009. Artificial intelligence is the subject of great hopes, dire warnings, and now -- a congressional caucus. Alarms about AI have been raised in apocalyptic movies and by some of the most pioneering minds in science and technology. Elon Musk, the Tesla and SpaceX CEO, said in July that AI is a "fundamental existential risk for human civilization."

If you hadn't heard, Elon Musk is worried about the machines. Though that may seem a quixotic stance for the head of multiple tech companies to take, it seems that his proximity to the bleeding edge of technological development has given him the heebie-jeebies when it comes to artificial intelligence. He's shared his fears of AI running amok before, likening it to "summoning the demon," and Musk doubled down on his stance at a meeting of the National Governors Association this weekend, telling state leaders that AI poses an existential threat to humanity. "Until people see robots going down the street killing people, they don't know how to react because it seems so ethereal. AI is a rare case where I think we need to be proactive in regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it's too late," according to the MIT Tech Review.

If you hadn't heard, Elon Musk is worried about the machines. Though that may seem a quixotic stance for the head of multiple tech companies to take, it seems that his proximity to the bleeding edge of technological development has given him the heebie-jeebies when it comes to artificial intelligence. He's shared his fears of AI running amok before, likening it to "summoning the demon," and Musk doubled down on his stance at a meeting of the National Governors Association this weekend, telling state leaders that AI poses an existential threat to humanity. "Until people see robots going down the street killing people, they don't know how to react because it seems so ethereal. AI is a rare case where I think we need to be proactive in regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it's too late," according to the MIT Tech Review.

Mark Zuckerberg and Bill Gates built billion-dollar technology companies in two very different areas, but they both agree on the biggest threats to American jobs. At his Harvard University commencement speech on Thursday, Facebook FB, 0.11% chief executive Zuckerberg, had some tough words for the Class of 2017. "Our generation will have to deal with tens of millions of jobs replaced by automation like self-driving cars and trucks," he said, adding, "When our parents graduated, purpose reliably came from your job, your church, your community,. But today, technology and automation are eliminating many jobs. Membership in communities is declining. Many people feel disconnected and depressed, and are trying to fill a void."