human spirit
BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge
Tong, Terry, Wang, Fei, Zhao, Zhe, Chen, Muhao
This paper proposes a novel backdoor threat attacking the LLM-as-a-Judge evaluation regime, where the adversary controls both the candidate and evaluator model. The backdoored evaluator victimizes benign users by unfairly assigning inflated scores to adversary. A trivial single token backdoor poisoning 1% of the evaluator training data triples the adversary's score with respect to their legitimate score. We systematically categorize levels of data access corresponding to three real-world settings, (1) web poisoning, (2) malicious annotator, and (3) weight poisoning. These regimes reflect a weak to strong escalation of data access that highly correlates with attack severity. Under the weakest assumptions - web poisoning (1), the adversary still induces a 20% score inflation. Likewise, in the (3) weight poisoning regime, the stronger assumptions enable the adversary to inflate their scores from 1.5/5 to 4.9/5. The backdoor threat generalizes across different evaluator architectures, trigger designs, evaluation tasks, and poisoning rates. By poisoning 10% of the evaluator training data, we control toxicity judges (Guardrails) to misclassify toxic prompts as non-toxic 89% of the time, and document reranker judges in RAG to rank the poisoned document first 97% of the time. LLM-as-a-Judge is uniquely positioned at the intersection of ethics and technology, where social implications of mislead model selection and evaluation constrain the available defensive tools. Amidst these challenges, model merging emerges as a principled tool to offset the backdoor, reducing ASR to near 0% whilst maintaining SOTA performance. Model merging's low computational cost and convenient integration into the current LLM Judge training pipeline position it as a promising avenue for backdoor mitigation in the LLM-as-a-Judge setting.
Interpersonal Intelligence Is Critical for Artificial Intelligence
In a world that is racing toward artificial intelligence, we should ensure that we do not compromise the fundamental characteristics of being human toward human beings in the workplace (and every place). Artificial intelligence needs interpersonal intelligence in order to be truly effective. Interpersonal intelligence is the ability to humanly decode how we work across real, digital and virtual platforms. Its purpose is to enhance the awareness and impact of the touch and pain points of human interaction in the workplace, ensuring the dignity of the human spirit is maintained. Interpersonal intelligence unites all aspects of soft skills, emotional intelligence and human differences so that artificial intelligence can serve its purpose with greater competence and conviction.
Being mindful in chaos - Suzanne Jewell [Interview]
Suzanne is a global expert and proven strategic marketing and communications lead on strategic initiatives for corporations, start-ups, non-profits, and the community. Suzanne's most recent project is Mindful Mornings Miami, the hottest new one-hour talk show on independent JoltRadio. With a reach of over 200,000, the show focuses on what it means to wake up and live in the world today. We have the pleasure of welcoming Suzanne Jewell to our interview series, I am Aishwarya Jain from the peopleHum team before we begin just a quick introduction of peopleHum, peopleHum is an end-to-end, one-view, integrated Human Capital Management automation platform, the winner of the 2019 global Codie Award for HCM that is specifically built for crafted employee experiences and the future of work with AI and automation technologies. We run the peopleHum blog and video channel which receives upwards of 200,000 visitors a year and publish around 2 interviews with well-known names globally, every month. We're thrilled to have you on our series. Awesome to be here and welcome to everyone, and my first wish is that everyone is awake, aware and well, today! Thank you so much for that. You know, my experience of living in one place and working on pretty much every other continent on the planet really made me aware that I was always in, kind of autopilot. The media tends to do that, both in the way that it bombards us. And I find that I personally was really being the same way, and so, in that regard for me, mindfulness became a way to really get present to what was going on wherever I happen to be on the planet for my GPS location, but also where my head and my mind and my body is located. The most recent Harvard study actually shows that 47% of the time that we're out of bed and awakened vertical, our mind and our body are actually not in the same place.
3 strategies for keeping your marketing job in the age of AI
Artificial intelligence-based marketing tools can now learn, predict, personalize, strategize, make decisions, take action, provide insights and create content. So, what can human marketers do to make sure they are still useful? To get some ideas about what skills humans might retain as the waves of AI-powered change roll in, we asked a veteran of marketing's ups and downs, Bonnie Crater. She's CEO and president of San Mateo, California-based marketing analytics firm Full Circle Insights, and she's held marketing exec positions at Realization Technologies, VoiceObjects, Salesforce, Genesys, Stratify and Netscape. While there is software to clean and optimize data, she said, it is likely there will be a long-term need for marketers with data management skills.
Want to get more from AI? Build trust in your machines
Did you know that the self-driving car market could reach $87 billion by 2030¹? How many of us are ready to sit in the passenger seat of a speeding, driverless taxi? It would require complete trust in the artificial intelligence manning the wheel and controlling the brakes. We are at an inflection point as AI proliferates across virtually every industry. Yet, according to PwC's Global Data and Analytics Survey 2016: Big Decisions, only 39% of companies are highly data-driven.