Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to "hallucinate," or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as "eliminating" (Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.

There has been a surge of interest in what generative AI can do. But what does this technology really mean for the legal sector? To find out we must navigate a path between'Death of the Lawyer 2.0' hysteria and those who dismiss the whole thing as a gimmick. Artificial Lawyer looks at what this tech can really do. Generative AI (gen AI), working via Large Language Models such as OpenAI's GPT-3, can do some amazing things.

As AI, image quality and cloud-based services improve, the insurance industry is slowly moving closer to an era where many claims may soon be processed without any human touch at all. One area where fully automated claims could bring new efficiencies and benefits is in commercial auto insurance. In a market where losses and claims are high, improving operational efficiencies is one of the most viable ways to gain profitability. Experts say most of the technology is already here, and while it must be refined to be ready for widespread adoption, it could be a mainstream reality within the next few years. Commercial auto insurance has been a tough market for many carriers in recent years.

Artificial intelligence and machine learning are increasingly embraced by U.S. carriers as they seek to remain competitive and modernize their operations, a new LexisNexis Risk Solutions study has found. Struggles remain, however, in terms of figuring out staffing and proper use of the technology to optimize its benefits. LexisNexis' look at how the top 100 U.S. carriers are using and benefiting from artificial intelligence and machine learning found a robust adoption of the technology and a strong belief in the benefits it will bring. Approximately 62 percent of respondents said they worked for insurance carriers that have already adopted artificial intelligence (AI) and machine learning (ML) initiatives. About 75 percent said they believe AI and ML can provide carriers with a competitive advantage through better decision-making.

Adoption of AI is highest among top-tier carriers, according to a LexisNexis survey of 300 executives at the top 100 U.S. insurers. The company's "Hype or Reality: The State of Artificial Intelligence and Machine Learning in the Insurance Industry" white paper reports that more than 80% of respondents from the top 20 insurance companies meet the definition of "adopter" for AI and machine learning. By line of business, auto insurance is leveraging AI and machine learning at a 68% clip, LexisNexis found, followed by life, commercial and home insurance. Despite the incursion of many AI-focused insurtechs, 65% of companies with active implementations, pilots or approved projects prefer to develop their applications internally. Marketing, underwriting and claims are the most common practice areas in which AI is being applied.

As Vice President and Chief Technology Officer of Nexis Solutions, Stephen Iddings has been closely involved in driving the Artificial Intelligence (AI) strategy at LexisNexis. In an interview, he shared how LexisNexis uses AI and Machine Learning (ML) to innovate for customers and find operational efficiencies. But he warned that AI is only as useful as the quality of the data going into it and questions it is trying to answer. What are the main ways in which LexisNexis is using AI technologies? I would break this down into three big areas.

Around 8,000 technologists, half of whom are software engineers, work at RELX. Annually, the company spends $1.4bn on technology. The combination of our rich data assets, technology infrastructure and knowledge of how to use next generation technologies, such as machine learning and natural language processing, allows us to create effective solutions for our customers. Helping research chemists with Elsevier's Reaxys Reaxys enables the shortest path to chemistry research answers, supporting the early stages of drug development in the pharmaceutical industry, exploratory chemistry research in academia, and product development in industries such as chemicals and oil & gas. The amount of chemical information published each year is increasing exponentially, making it more and more challenging for research chemists to quickly find targeted and actionable information to help support their research.

UPDATE: Check out below where Lexis outlines some issues they have with the quality of the results. The language around the technology has softened from the height of its hype cycle, but there's still a sense out there that AI is this "thing." As one legal tech leader put it to me last year, "a lot of lawyers act like'we need to get some AI' without trying to figure out how AI solutions might be different." To some extent, that still holds sway. It's a conclusion that's not entirely off base because some solutions use the same underlying AI algorithms.

This story was co-published with NPR. But dig deeper and the implications of what they're selling might give many patients pause: A future in which everything you do -- the things you buy, the food you eat, the time you spend watching TV -- may help determine how much you pay for health insurance. With little public scrutiny, the health insurance industry has joined forces with data brokers to vacuum up personal details about hundreds of millions of Americans, including, odds are, many readers of this story. The companies are tracking your race, education level, TV habits, marital status, net worth. Then they feed this information into complicated computer algorithms that spit out predictions about how much your health care could cost them. Are you a woman who recently changed your name? You could be newly married and have a pricey pregnancy pending. Or maybe you're stressed and anxious from a recent divorce.

Insurers and data brokers are predicting your health costs based on data about things like race, marital status, how much TV you watch, whether you pay your bills on time or even buy plus-size clothing.Scanrail/Getty Images This story was originally co-published by ProPublica and NPR. But dig deeper and the implications of what they're selling might give many patients pause: A future in which everything you do--the things you buy, the food you eat, the time you spend watching TV--may help determine how much you pay for health insurance. With little public scrutiny, the health insurance industry has joined forces with data brokers to vacuum up personal details about hundreds of millions of Americans, including, odds are, many readers of this story. The companies are tracking your race, education level, TV habits, marital status, net worth. Then they feed this information into complicated computer algorithms that spit out predictions about how much your health care could cost them. Are you a woman who recently changed your name? You could be newly married and have a pricey pregnancy pending. Or maybe you're stressed and anxious from a recent divorce.