The training process of large language models (LLMs) often involves varying degrees of test data contamination. Although current LLMs are achieving increasingly better performance on various benchmarks, their performance in practical applications does not always match their benchmark results. Leakage of benchmarks can prevent the accurate assessment of LLMs' true performance. However, constructing new benchmarks is costly, labor-intensive and still carries the risk of leakage. Therefore, in this paper, we ask the question, Can we reuse these leaked benchmarks for LLM evaluation? We propose Inference-Time Decontamination (ITD) to address this issue by detecting and rewriting leaked samples without altering their difficulties. ITD can mitigate performance inflation caused by memorizing leaked benchmarks. Our proof-of-concept experiments demonstrate that ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU. On MMLU, using Inference-time Decontamination can lead to a decrease in the results of Phi3 and Mistral by 6.7% and 3.6% respectively. We hope that ITD can provide more truthful evaluation results for large language models.

John Kerry is an accomplished diplomat and a highly skilled public servant who has devoted his professional life to serving the American people and furthering the interests of his country on the international stage. Do you wish to discover his most notable accomplishments and credentials? In Massachusetts, Kerry started his career in public service as a prosecutor, where he worked to reduce crime and maintain public safety. He subsequently went on to serve for more than two decades as a senator in the US Senate, where he presided over the committee overseeing foreign relations and was a key player in establishing US foreign policy. He played a key role in crafting the Comprehensive Anti-Apartheid Act, which placed economic restrictions on the South African government in opposition to its apartheid system.

Artificial intelligence (AI) and machine-learning (ML) have quickly grown beyond a few major tech companies and hardcore academic researchers. Every marketing organization can tap into the power of AI to streamline operations and grow the business. The new book The AI Marketing Canvas: A Five-Stage Road Map to Implementing Artificial Intelligence in Marketing provides a growth framework for business and marketing leaders to implement AI using a five-stage model called the "AI Marketing Canvas." On this episode of Marketing Smarts, I speak with co-author Rajkumar Venkatesan about how he and his co-writer developed those stages by studying leading global brands. We cover examples of brands―including Google, Lyft and Coca-Cola―that have successfully woven AI into their marketing strategies. This is not a conversation about coding for AI models. Raj and I talk about how marketing leaders can go from "zero to hero" with AI in marketing, and what that means for your team and your company culture. Listen to the entire show now from the link above, or download the mp3 and listen at your convenience.

President-elect Joe Biden named John Kerry to the newly created role of climate czar, a move that underscores the incoming administration's commitment to an international-focused approach to the issue and recognition of its strategic importance. Kerry, the former secretary of state, is a diplomatic heavyweight who helped piece together the landmark Paris climate agreement during the Obama administration and pushed hard for domestic climate policies as a US senator. "I've asked him to return to government to get America back on track to address one of the most urgent national security threats we face--the climate crisis," Biden said in a statement released on Monday. "This role is the first of its kind: the first cabinet-level climate position, and the first time climate change has had a seat at the table on the National Security Council." Kerry's appointment as "special presidential envoy for climate" is among the first of six cabinet-level nominations that the Biden team announced on Monday, as it works to form a government in spite of President Donald Trump's refusal to accept the results of the election.

The RDI Hub, ADAPT Research Centre and Microsoft will this week investigate the societal benefits for AI at the inaugural John McCarthy AI Summer School. John McCarthy, who the RDI Hub building in Killorglin, Co Kerry is named in honour of, is considered one of the founding fathers of the discipline of AI when he coined the term in the mid-1950s. This week, over two days, experts and thought leaders in the field of AI will virtually gather to discuss how the constantly growing field of AI can be harnessed for the benefit of society, with a particular focus on how it will change ways of working and the impact of AI post COVID-19. The inaugural event is aiming to create a community which will share best practice and foster the development of the conversation around AI which, according to estimates by PwC, has the potential to boost Ireland's GDP by €48 billion in 2030. The John McCarthy AI Summer School will take place over two half days on Thursday 3rd and Friday 4th of September, from 1pm to 5pm, with a mix of plenary lectures and roundtable discussions.

Ireland-based company Standard Access, provider of secure building access solutions and expert in smart real estate management, has developed an Artificial Intelligence (AI) technology to help large organisations, entertainment venues and building owners to comply with new public health crowd restrictions in the wake of the Covid-19 pandemic. By monitoring building capacity in real-time, SAvant enables building operators to adhere to new pandemic-responsive rules on crowd capacity and social distancing. As people gradually return to workplaces, social spaces and public transport, reduced occupancy and physical distancing will be important for employers and venue owners in order to build trust and confidence. The company is currently rolling out enterprise level software that can be incorporated into existing CCTV systems to detect the number of people in a particular area at any one time. SAvant technology also works on 360 degree cameras.

Fexco has opened a €21 million research and development facility in Killorglin, Co Kerry. Dedicated to Stanford University computer scientist John McCarthy, the 57,000 sq ft building houses Fexco's Research Development and Innovation Centre, and digital innovation centre RDI Hub, a not-for-profit public private partnership focused on the commercialisation of innovation and research and backed by Fexco, IT Tralee and Kerry County Council. Some 125 Fexco staff will be located at the building, replacing a manufacturing facility that was closed in 2012. Fexco said the operation is aimed at driving its continuing development as a leading global fintech company. "This new building signals our commitment to providing world-class innovative fintech solutions to a global audience as well as supporting our local community, through job creation and local investment," said Denis McCarthy, chief executive of Fexco.

We continue our series on artificial intelligence (AI) in public relations with Kerry Sheehan, Artificial Intelligence in PR Vice-Chair with the CIPR. You can read the full articles on our Patreon page. While you're here, take a look at our new podcast! Patreon support keeps this podcast alive and free for all listeners! The pace of the development of software and machine learning is increasing and by 2023 artificial intelligence will impact areas of PR such as stakeholder mapping, risk analysis, auditing, and behavioural analysis.

Kerry is Vice Chair of the globally leadiing #AIinPR panel looking at the implications of AI and in the PR and communications industry and also advising the industry and business on ethical AI implementation and PR - the public's adoption of good AI to realise its benefits rests on the shoulders of ethical PR practitioners. Kerry is a member of the Chartered Institute of Public Relations Council and the International Data Science Foundation, and is a graduate of Dame Wendy Hall's Fundamentals of Data, AI and Machine Learning for Business university course. Kerry will take you through the CIPR's #AIinPR Humans Still Needed globally acclaimed research and what data and artificial intelligence means for the PR and communications industry and what you can do to upskill as a practitioner to remain relevant and have the competitive edge.

The past couple of decades have seen technology forge ahead relentlessly, enabling us to communicate in unprecedented ways, work more smartly and from any location, carry our phones in our pocket and much more. And artificial intelligence has evolved somewhat too, alongside all of the'usual' tech that we use daily, bringing with it a more urgent perceived need to consider the threat that it might pose. Bill Gates has come out to state that he's firmly in the'AI is a threat camp', so too has Stephen Hawking, one of the greatest thinkers of our time. But not everyone is in agreement with them. At the Dublin Web Summit, I attended a talk – the name now escapes me unfortunately when it comes to who it was by – and when the speaker, a high profile tech name (that much I do know) was asked about the threat posed by AI, he said something that I found interesting.