The training process of large language models (LLMs) often involves varying degrees of test data contamination. Although current LLMs are achieving increasingly better performance on various benchmarks, their performance in practical applications does not always match their benchmark results. Leakage of benchmarks can prevent the accurate assessment of LLMs' true performance. However, constructing new benchmarks is costly, labor-intensive and still carries the risk of leakage. Therefore, in this paper, we ask the question, Can we reuse these leaked benchmarks for LLM evaluation? We propose Inference-Time Decontamination (ITD) to address this issue by detecting and rewriting leaked samples without altering their difficulties. ITD can mitigate performance inflation caused by memorizing leaked benchmarks. Our proof-of-concept experiments demonstrate that ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU. On MMLU, using Inference-time Decontamination can lead to a decrease in the results of Phi3 and Mistral by 6.7% and 3.6% respectively. We hope that ITD can provide more truthful evaluation results for large language models.

John Kerry is an accomplished diplomat and a highly skilled public servant who has devoted his professional life to serving the American people and furthering the interests of his country on the international stage. Do you wish to discover his most notable accomplishments and credentials? In Massachusetts, Kerry started his career in public service as a prosecutor, where he worked to reduce crime and maintain public safety. He subsequently went on to serve for more than two decades as a senator in the US Senate, where he presided over the committee overseeing foreign relations and was a key player in establishing US foreign policy. He played a key role in crafting the Comprehensive Anti-Apartheid Act, which placed economic restrictions on the South African government in opposition to its apartheid system.

Artificial intelligence (AI) and machine-learning (ML) have quickly grown beyond a few major tech companies and hardcore academic researchers. Every marketing organization can tap into the power of AI to streamline operations and grow the business. The new book The AI Marketing Canvas: A Five-Stage Road Map to Implementing Artificial Intelligence in Marketing provides a growth framework for business and marketing leaders to implement AI using a five-stage model called the "AI Marketing Canvas." On this episode of Marketing Smarts, I speak with co-author Rajkumar Venkatesan about how he and his co-writer developed those stages by studying leading global brands. We cover examples of brands―including Google, Lyft and Coca-Cola―that have successfully woven AI into their marketing strategies. This is not a conversation about coding for AI models. Raj and I talk about how marketing leaders can go from "zero to hero" with AI in marketing, and what that means for your team and your company culture. Listen to the entire show now from the link above, or download the mp3 and listen at your convenience.

An Irish student has claimed the second prize and €1,600 in a prestigious international science festival for using artificial intelligence (AI) to forecast the weather. Conor Casey (18), from Pobalscoil Inbhear Scéine, Kenmare, Co Kerry represented Ireland at the Regeneron International Science and Engineering Fair (ISEF), where he claimed second place in his category, earth and environmental sciences. The sixth year student gained the accolade for his project'Using AI to Improve Weather Prediction.' "The whole ISEF experience has been absolutely amazing. It was great that the competition got to go ahead virtually, it gave me the opportunity to interact with people from around the world who have done some incredible work across a huge variety of areas," he said. "I really enjoyed presenting my project and hearing from world famous scientists, who have really encouraged me to develop my love of STEM. "I'm immensely grateful to everyone at SciFest for this opportunity, as well as all my family and my teacher Ms Sarah Abbott, without whom none of this would have been possible." Using artificial intelligence, Conor developed a forecast model with a similar level of accuracy to current models but with "greater efficiency." Conor's goal was to make it cheaper to generate weather forecasts by reducing the amount of resources required. The young scientist secured his place at the international competition when he won the runner-up award at the SciFest 2020 National Final last November. The event is the largest, most inclusive STEM fair programme for second level students in Ireland. Get today's news headlines, opinion, sport and more direct to your inbox at 7.30am every morning, and every evening, with our free daily newsletter. Regeneron ISEF is the world's largest international pre-college science competition, involving over 1,800 students from 64 countries, regions and territories competing for a prize fund of over €4million. The competition is usually held in the United States but for the second year running this year's event took place virtually. Sheila Porter, SciFest Founder and CEO, said: "We are all very proud of Conor, he is a great example of the high calibre of entries that SciFest produces.

President-elect Joe Biden named John Kerry to the newly created role of climate czar, a move that underscores the incoming administration's commitment to an international-focused approach to the issue and recognition of its strategic importance. Kerry, the former secretary of state, is a diplomatic heavyweight who helped piece together the landmark Paris climate agreement during the Obama administration and pushed hard for domestic climate policies as a US senator. "I've asked him to return to government to get America back on track to address one of the most urgent national security threats we face--the climate crisis," Biden said in a statement released on Monday. "This role is the first of its kind: the first cabinet-level climate position, and the first time climate change has had a seat at the table on the National Security Council." Kerry's appointment as "special presidential envoy for climate" is among the first of six cabinet-level nominations that the Biden team announced on Monday, as it works to form a government in spite of President Donald Trump's refusal to accept the results of the election.

The RDI Hub, ADAPT Research Centre and Microsoft will this week investigate the societal benefits for AI at the inaugural John McCarthy AI Summer School. John McCarthy, who the RDI Hub building in Killorglin, Co Kerry is named in honour of, is considered one of the founding fathers of the discipline of AI when he coined the term in the mid-1950s. This week, over two days, experts and thought leaders in the field of AI will virtually gather to discuss how the constantly growing field of AI can be harnessed for the benefit of society, with a particular focus on how it will change ways of working and the impact of AI post COVID-19. The inaugural event is aiming to create a community which will share best practice and foster the development of the conversation around AI which, according to estimates by PwC, has the potential to boost Ireland's GDP by €48 billion in 2030. The John McCarthy AI Summer School will take place over two half days on Thursday 3rd and Friday 4th of September, from 1pm to 5pm, with a mix of plenary lectures and roundtable discussions.

Ireland-based company Standard Access, provider of secure building access solutions and expert in smart real estate management, has developed an Artificial Intelligence (AI) technology to help large organisations, entertainment venues and building owners to comply with new public health crowd restrictions in the wake of the Covid-19 pandemic. By monitoring building capacity in real-time, SAvant enables building operators to adhere to new pandemic-responsive rules on crowd capacity and social distancing. As people gradually return to workplaces, social spaces and public transport, reduced occupancy and physical distancing will be important for employers and venue owners in order to build trust and confidence. The company is currently rolling out enterprise level software that can be incorporated into existing CCTV systems to detect the number of people in a particular area at any one time. SAvant technology also works on 360 degree cameras.

Fexco has opened a €21 million research and development facility in Killorglin, Co Kerry. Dedicated to Stanford University computer scientist John McCarthy, the 57,000 sq ft building houses Fexco's Research Development and Innovation Centre, and digital innovation centre RDI Hub, a not-for-profit public private partnership focused on the commercialisation of innovation and research and backed by Fexco, IT Tralee and Kerry County Council. Some 125 Fexco staff will be located at the building, replacing a manufacturing facility that was closed in 2012. Fexco said the operation is aimed at driving its continuing development as a leading global fintech company. "This new building signals our commitment to providing world-class innovative fintech solutions to a global audience as well as supporting our local community, through job creation and local investment," said Denis McCarthy, chief executive of Fexco.

We continue our series on artificial intelligence (AI) in public relations with Kerry Sheehan, Artificial Intelligence in PR Vice-Chair with the CIPR. You can read the full articles on our Patreon page. While you're here, take a look at our new podcast! Patreon support keeps this podcast alive and free for all listeners! The pace of the development of software and machine learning is increasing and by 2023 artificial intelligence will impact areas of PR such as stakeholder mapping, risk analysis, auditing, and behavioural analysis.

Kerry is Vice Chair of the globally leadiing #AIinPR panel looking at the implications of AI and in the PR and communications industry and also advising the industry and business on ethical AI implementation and PR - the public's adoption of good AI to realise its benefits rests on the shoulders of ethical PR practitioners. Kerry is a member of the Chartered Institute of Public Relations Council and the International Data Science Foundation, and is a graduate of Dame Wendy Hall's Fundamentals of Data, AI and Machine Learning for Business university course. Kerry will take you through the CIPR's #AIinPR Humans Still Needed globally acclaimed research and what data and artificial intelligence means for the PR and communications industry and what you can do to upskill as a practitioner to remain relevant and have the competitive edge.