LLM-integrated app systems extend the utility of Large Language Models (LLMs) with third-party apps that are invoked by a system LLM using interleaved planning and execution phases to answer user queries. These systems introduce new attack vectors where malicious apps can cause integrity violation of planning or execution, availability breakdown, or privacy compromise during execution. In this work, we identify new attacks impacting the integrity of planning, as well as the integrity and availability of execution in LLM-integrated apps, and demonstrate them against IsolateGPT, a recent solution designed to mitigate attacks from malicious apps. We propose Abstract-Concrete-Execute (ACE), a new secure architecture for LLM-integrated app systems that provides security guarantees for system planning and execution. Specifically, ACE decouples planning into two phases by first creating an abstract execution plan using only trusted information, and then mapping the abstract plan to a concrete plan using installed system apps. We verify that the plans generated by our system satisfy user-specified secure information flow constraints via static analysis on the structured plan output. During execution, ACE enforces data and capability barriers between apps, and ensures that the execution is conducted according to the trusted abstract plan. We show experimentally that ACE is secure against attacks from the InjecAgent and Agent Security Bench benchmarks for indirect prompt injection, and our newly introduced attacks. We also evaluate the utility of ACE in realistic environments, using the Tool Usage suite from the LangChain benchmark. Our architecture represents a significant advancement towards hardening LLM-based systems using system security principles.

Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN), known as experts. We present SteerMoE, a framework for steering MoE models by detecting and controlling behavior-linked experts. Our detection method identifies experts with distinct activation patterns across paired inputs exhibiting contrasting behaviors. By selectively (de)activating such experts during inference, we control behaviors like faithfulness and safety without retraining or modifying weights. Across 11 benchmarks and 6 LLMs, our steering raises safety by up to +20% and faithfulness by +27%. In adversarial attack mode, it drops safety by -41% alone, and -100% when combined with existing jailbreak methods, bypassing all safety guardrails and exposing a new dimension of alignment faking hidden within experts.

Semantic Textual Relatedness (STR) captures nuanced relationships between texts that extend beyond superficial lexical similarity. In this study, we investigate STR in the context of job title matching - a key challenge in resume recommendation systems, where overlapping terms are often limited or misleading. We introduce a self-supervised hybrid architecture that combines dense sentence embeddings with domain-specific Knowledge Graphs (KGs) to improve both semantic alignment and explainability. Unlike previous work that evaluated models on aggregate performance, our approach emphasizes data stratification by partitioning the STR score continuum into distinct regions: low, medium, and high semantic relatedness. This stratified evaluation enables a fine-grained analysis of model performance across semantically meaningful subspaces. We evaluate several embedding models, both with and without KG integration via graph neural networks. The results show that fine-tuned SBERT models augmented with KGs produce consistent improvements in the high-STR region, where the RMSE is reduced by 25% over strong baselines. Our findings highlight not only the benefits of combining KGs with text embeddings, but also the importance of regional performance analysis in understanding model behavior. This granular approach reveals strengths and weaknesses hidden by global metrics, and supports more targeted model selection for use in Human Resources (HR) systems and applications where fairness, explainability, and contextual matching are essential.

This technical report analyzes the challenge of "hallucinations" (false information) in LLMs applied to law. It examines their causes, manifestations, and the effectiveness of the RAG mitigation strategy, highlighting its limitations and proposing holistic optimizations. The paper explores the ethical and regulatory implications, emphasizing human oversight as an irreplaceable role. It concludes that the solution lies not in incrementally improving generative models, but in adopting a "consultative" AI paradigm that prioritizes veracity and traceability, acting as a tool to amplify, not replace, professional judgment. -- Este informe técnico analiza el desafío de las "alucinaciones" (información falsa) en los LLMs aplicados al derecho. Se examinan sus causas, manifestaciones y la efectividad de la estrategia de mitigación RAG, exponiendo sus limitaciones y proponiendo optimizaciones holísticas. Se exploran las implicaciones éticas y regulatorias, enfatizando la supervisión humana como un rol insustituible. El documento concluye que la solución no reside en mejorar incrementalmente los modelos generativos, sino en adoptar un paradigma de IA "consultiva" que priorice la veracidad y la trazabilidad, actuando como una herramienta para amplificar, y no sustituir, el juicio profesional.

The legal field already uses various large language models (LLMs) in actual applications, but their quantitative performance and reasons for it are underexplored. We evaluated several open-source and proprietary LLMs -- including GPT-series, Anthropic, Deepseek and Llama-3, variants -- on parts of the European Qualifying Examination (EQE) for future European Patent Attorneys. OpenAI o1 led with 0.82 accuracy and 0.81 F1 score, whereas (Amazon Web Services) AWS Llama 3.1 8B lagged at 0.50 accuracy, and a Python-deployed Llama 3.1 8B scored 0.55. The latter two are within the range of mere guessing for the two-answer forced-choice design. None of the evaluated models could have passed the examination fully, as accuracy never exceeded the average threshold of 0.90 required for professional-level standards -- also not models that are regularly promoted for their assumed beyond-PhD- and bar-admitted-lawyer-level performance. GPT-4o excelled at integrating text and graphics, while Claude 3 Opus often lost formatting coherence. Human patent experts evaluated the textual justifications and uncovered various critical shortcomings of each model. They valued clarity and legal rationale over the raw correctness of the answers, which revealed misalignment between automatic metrics and expert judgment. Model outputs were sensitive to modest temperature changes and prompt wording, which underscores the remaining necessity of expert oversight. Future work should target logical consistency, robust multimodality, and adaptive prompting to approach human-level patent proficiency. In summary, despite the outstanding performance of recent large models, the general public might overestimate their performance. The field has a long way to go to develop a virtual patent attorney. This paper wants to point out several specific limitations that need solutions.

Classifying patents by their relevance to the UN Sustainable Development Goals (SDGs) is crucial for tracking how innovation addresses global challenges. However, the absence of a large, labeled dataset limits the use of supervised learning. Existing methods, such as keyword searches, transfer learning, and citation-based heuristics, lack scalability and generalizability. This paper frames patent-to-SDG classification as a weak supervision problem, using citations from patents to SDG-tagged scientific publications (NPL citations) as a noisy initial signal. To address its sparsity and noise, we develop a composite labeling function (LF) that uses large language models (LLMs) to extract structured concepts, namely functions, solutions, and applications, from patents and SDG papers based on a patent ontology. Cross-domain similarity scores are computed and combined using a rank-based retrieval approach. The LF is calibrated via a custom positive-only loss that aligns with known NPL-SDG links without penalizing discovery of new SDG associations. The result is a silver-standard, soft multi-label dataset mapping patents to SDGs, enabling the training of effective multi-label regression models. We validate our approach through two complementary strategies: (1) internal validation against held-out NPL-based labels, where our method outperforms several baselines including transformer-based models, and zero-shot LLM; and (2) external validation using network modularity in patent citation, co-inventor, and co-applicant graphs, where our labels reveal greater thematic, cognitive, and organizational coherence than traditional technological classifications. These results show that weak supervision and semantic alignment can enhance SDG classification at scale.

Charlie Kirk's killer'identified' by law enforcement in manhunt after Trump indicated he knew assassin's motive Charlie Kirk's body arrives in hearse at Salt Lake City as devastated wife and family gather with JD Vance Trump's address to the nation hit by outrageous'fake' claim REVEALED: The truth about the'vanishing plane' five miles from Charlie Kirk's assassination... as private jet owner is unmasked And the armed militia mystery. FBI terror hunter blows the lid on search for Charlie Kirk's assassin... and the vital clue cops are desperate for'Shots fired' at UMass Boston: SWAT teams rush to hall of residence as terrified students sprint away from scene Elite sniper breaks down Charlie Kirk assassin's sick plot... and reveals tiny detail everyone's missed: The gun. Joe Rogan is speechless as he learns of Charlie Kirk's assassination on his podcast Fans claim NFL is'rigged' after bizarre Vikings play against Bears goes viral Biden insiders spew venom on Kamala Harris claiming her scathing critique of the former president is a'sloppy attempt at deflecting blame' for her failure Charlotte Tilbury reveals the secrets behind the Dallas Cowboys Cheerleaders' flawless look Americans are unknowingly eating endangered animals... as study finds grocery store meat is mislabeled Mom of Ukrainian stabbing suspect says'killer' is'NOT racist because he dates all kinds of girls' Dave Portnoy: Trump played a huge part in Charlie Kirk's death Charlie Kirk's assassin sniper's nest giving him a terrifying clear shot before he vanished into thin air Charlie Kirk's body to be flown home aboard Air Force Two in rare tribute Silenced by a bullet to the throat - then came the sneers. MAUREEN CALLAHAN's blistering rebuke as America faces greatest political crisis since the 60s Looking to level up your home entertainment setup this fall? Sony's newest budget smart TV is a strong contender Daily Mail journalists select and curate the products that feature on our site.

FBI under pressure over open airport five miles from Charlie Kirk assassination hit as private jet'vanishes' after shooting Dave Portnoy slams'insane' left in furious Charlie Kirk rant... but adds Donald Trump'played a part' in death Elon Musk unleashes explosive rant on Kirk assassination calling the left'the party of murder' And the armed militia mystery. FBI terror hunter blows the lid on search for Charlie Kirk's assassin... and the vital clue cops are desperate for Elite sniper breaks down Charlie Kirk assassin's sick plot... and reveals tiny detail everyone's missed: The gun. Jimmy Kimmel reacts to assassination of Charlie Kirk: 'No finger pointing' Charlie Kirk's'incredibly strong' wife is too heartbroken to tell her children that'daddy isn't coming home' Sweater weather starts here - the cozy, chic pieces from Soft Surroundings you'll actually wear all season Vile protester screams'f***' Charlie Kirk as he drives through vigil Jennifer Lopez STILL living in $68M mansion she shared with Ben Affleck one year after listing, here's why NBA's first openly-gay player Jason Collins reveals brain tumor as ex-Nets center starts cancer treatment Silenced by a bullet to the throat - then came the sneers. MAUREEN CALLAHAN's blistering rebuke as America faces greatest political crisis since the 60s Microsoft denies any link to man behind'evil' Charlie Kirk assassination post A post has gone viral on X, claiming to show a Microsoft employee making'evil' comments about Charlie Kirk's assassination. Benjamin Van der Zar commented on a post saying: 'Professional hit from a few hundred yards, good job soldier.' Kirk, 31, was shot while speaking to a crowd at Utah Valley University in Orem, Utah, at 12:20pm local time.

Revealed: The political activist charged for distracting cops after'saying HE shot Charlie Kirk' FBI under pressure over open airport five miles from Charlie Kirk assassination hit as private jet'vanishes' after shooting Elite sniper breaks down Charlie Kirk assassin's sick plot... and reveals tiny detail everyone's missed: The gun. Elon Musk unleashes explosive rant on Kirk assassination calling the left'the party of murder' Shocking moment left-wing politicians refuse to take part in minute's silence for Charlie Kirk in European Parliament - prompting screaming match with right-wing opponents Silenced by a bullet to the throat - then came the sneers. MAUREEN CALLAHAN's blistering rebuke as America faces greatest political crisis since the 60s Jimmy Kimmel reacts to assassination of Charlie Kirk: 'No finger pointing' Charlie Kirk's'incredibly strong' wife is too heartbroken to tell her children that'daddy isn't coming home' Sweater weather starts here - the cozy, chic pieces from Soft Surroundings you'll actually wear all season Trader Joe's continues USA takeover as it snaps up more land after opening new stores this year My last conversation with Charlie: MARK HALPERIN remembers a dedicated, political genius, who didn't hate anyone. TMZ forced to apologize after staff heard erupting in laughter as Charlie Kirk's death was announced Police tapes capture chilling details of Charlie Kirk gunman's appearance as FBI faces fury over his escape This Oscar-nominated actress, 68, will soon reunite with her ex in Spain for their daughter's wedding, can you guess who? MORE: America's most famous scientist sparks outrage among liberals after MAGA post Social media platform Bluesky has issued a warning to all of its users, saying it will remove posts or ban anyone who celebrates the assassination of Charlie Kirk.

The Download: Trump's impact on science, and meet our climate and energy honorees How Trump's policies are affecting early-career scientists--in their own words Every year MIT Technology Review celebrates accomplished young scientists, entrepreneurs, and inventors from around the world in our Innovators Under 35 list. We've just published the 2025 edition . This year, though, the context is different: The US scientific community is under attack. Since Donald Trump took office in January, his administration has fired top government scientists, targeted universities and academia, and made substantial funding cuts to the country's science and technology infrastructure. We asked our six most recent cohorts about both positive and negative impacts of the administration's new policies. Their responses provide a glimpse into the complexities of building labs, companies, and careers in today's political climate.