misuse
The Secrets of the US Spyware King
In an exclusive interview with WIRED, Paragon Solutions CEO Andrew Boyd reveals the limits of the company's promise to keep bad actors from abusing its powerful espionage tool. Spyware maker Paragon Solutions has long positioned itself as the good guy in an industry seemingly filled with bad ones, vowing to never sell its mobile spyware to authoritarian regimes or ones with poor human rights records. It also promises to cut off any customer caught misusing its products against journalists, dissidents, or other non-legitimate targets. Yet weeks after Paragon, then Israeli-owned, was acquired by the US equity firm AE Industrial Partners in December 2024 and merged with REDLattice--an American offensive cyber firm owned by AE that this week announced plans to go public --WhatsApp alleged that Paragon's Graphite spyware was used to infect the phones of more than 60 individuals in more than 20 countries, including journalists and activists. Most of the targets were not identified, but the University of Toronto's Citizen Lab named two journalists and two activists in Italy. Italian authorities denied misuse . Paragon and its new US owners, despite their zero-tolerance policy for customer abuse of their software, initially declined to comment on the allegations, and reportedly was exploring potential legal action against WhatsApp after the company sent a cease-and-desist letter to Paragon.
Anthropic's CEO proposes a three-step plan to curb AI development
At least one CEO of an AI company is calling for a slower pace when it comes to developing artificial intelligence. Anthropic's CEO Dario Amodei wrote a lengthy post detailing a goal of pacing the speed at which AI is built, instead of forging ahead at the rapid rate that AI is currently on. Amodei proposed a three-tiered approach to achieve this, with Anthropic already committing to the first step. The first measure calls for "frontier AI" companies to commit to "ongoing, employee-like access" for third-party evaluators who would focus on verifying compliance with certain safety standards, evaluating if AI model training is aligned with the goal of slowing down and reporting incidents. The second step requires these AI companies to establish "common safety standards" with the help of governments in order to limit the rate of unchecked AI progress.
Anthropic details bad actors' efforts to misuse its AI for bioweapons
'Biological misuse is one of the most serious risks of frontier AI models. Without the correct safeguards, such capabilities could have catastrophic consequences.' 'Biological misuse is one of the most serious risks of frontier AI models. Without the correct safeguards, such capabilities could have catastrophic consequences.' Report comes two days after former employee quit claiming company's models could cause human extinction by 2030 Criminals, state-sponsored groups, spyware vendors, scientists and propagandists have attempted to use Anthropic's powerful artificial intelligence models to design missiles and bombs, create deadly pathogens and surveil dissidents, according to a threat intelligence report the company published on Thursday.
Flock Highlighted Police Departments Using Its Tech. Now 4 Face Allegations of Misuse
Flock posted videos on its YouTube channel highlighting at least four police departments whose officers have faced allegations of misusing the company's tech. Flock, the increasingly controversial automatic license plate reader (ALPR) company, has previously highlighted at least four police departments on its YouTube page that have since faced allegations of misusing the company's technology. The company featured the Savannah Police Department in a March 2022 YouTube video . Four officers and two workers from the department were recently placed on administrative leave after an internal investigation found "potential misuses" of Flock's ALPR system. Also in Georgia, the Albany Police Department was the subject of a glossy 2024 video .
Anthropic investigating claim of unauthorised access to Mythos AI tool
Anthropic is investigating a claim that a small group of people gained access to its Claude Mythos model - the cyber-security tool which the AI firm says is too powerful to release to the public. We're investigating a report claiming unauthorized access to Claude Mythos Preview through one of our third-party vendor environments, the company said in a statement. It was in response to a Bloomberg report that users in a private forum managed to access the model without the normal permissions. There is deep unease about Mythos' capabilities - though the UK's top cyber official has said advanced AI tools could be a net positive if the technology was secured from misuse. There is currently no suggestion that malicious actors have managed to get hold of the model, and Anthropic says it does not have evidence its systems are affected.
India's outsourcing industry is worth 300bn. Can it survive AI?
India's outsourcing industry is worth $300bn. Indian technology stocks have seen an unprecedented rout over the past few weeks over fears of artificial intelligence upending the traditional outsourcing model that powers the country's $300bn (ยฃ223bn) back-office industry. The sell-off - part of a global correction in traditional software and IT stocks - preceded the market nervousness caused by recent geopolitical uncertainty, and is particularly significant for India. Over the past three-and-a-half decades, India's software industry has created millions of white-collar jobs, spawning a new middle class driven by high ambition and strong purchasing power. This, in turn, has fuelled demand for apartments, cars and restaurants across top-tier cities such as Bengaluru, Hyderabad and Gurugram over the past 30 years.
AI firm Anthropic seeks weapons expert to stop users from 'misuse'
AI firm Anthropic seeks weapons expert to stop users from'misuse' The US artificial intelligence (AI) firm Anthropic is looking to hire a chemical weapons and high-yield explosives expert to try to prevent catastrophic misuse of its software. In other words, it fears that its AI tools might tell someone how to make chemical or radioactive weapons, and wants an expert to ensure its guardrails are sufficiently robust. In the LinkedIn recruitment post, the firm says applicants should have a minimum of five years experience in chemical weapons and/or explosives defence as well as knowledge of radiological dispersal devices - also known as dirty bombs. The firm told the BBC the role was similar to jobs in other sensitive areas that it has already created. Anthropic is not the only AI firm adopting this strategy.
ChatGPT firm blames boy's suicide on 'misuse' of its technology
Adam Raine's family say the version of ChatGPT he used had'clear safety issues'. Adam Raine's family say the version of ChatGPT he used had'clear safety issues'. ChatGPT firm blames boy's suicide on'misuse' of its technology The maker of ChatGPT has said the suicide of a 16-year-old was down to his "misuse" of its system and was "not caused" by the chatbot. The comments came in OpenAI's response to a lawsuit filed against the San Francisco company and its chief executive, Sam Altman, by the family of California teenager Adam Raine. Raine killed himself in April after extensive conversations and "months of encouragement from ChatGPT", the family's lawyer has said.
Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoption
Liu, Yepeng, Zhao, Xuandong, Song, Dawn, Wornell, Gregory W., Bu, Yuheng
Despite progress in watermarking algorithms for large language models (LLMs), real-world deployment remains limited. We argue that this gap stems from misaligned incentives among LLM providers, platforms, and end users, which manifest as four key barriers: competitive risk, detection-tool governance, robustness concerns and attribution issues. We revisit three classes of watermarking through this lens. \emph{Model watermarking} naturally aligns with LLM provider interests, yet faces new challenges in open-source ecosystems. \emph{LLM text watermarking} offers modest provider benefit when framed solely as an anti-misuse tool, but can gain traction in narrowly scoped settings such as dataset de-contamination or user-controlled provenance. \emph{In-context watermarking} (ICW) is tailored for trusted parties, such as conference organizers or educators, who embed hidden watermarking instructions into documents. If a dishonest reviewer or student submits this text to an LLM, the output carries a detectable watermark indicating misuse. This setup aligns incentives: users experience no quality loss, trusted parties gain a detection tool, and LLM providers remain neutral by simply following watermark instructions. We advocate for a broader exploration of incentive-aligned methods, with ICW as an example, in domains where trusted parties need reliable tools to detect misuse. More broadly, we distill design principles for incentive-aligned, domain-specific watermarking and outline future research directions. Our position is that the practical adoption of LLM watermarking requires aligning stakeholder incentives in targeted application domains and fostering active community engagement.