Deep Learning
Meet the new biologists treating LLMs like aliens
By studying large language models as if they were living things instead of computer programs, scientists are discovering some of their secrets for the first time. How large is a large language model? Think about it this way. In the center of San Francisco there's a hill called Twin Peaks from which you can view nearly the entire city. Picture all of it--every block and intersection, every neighborhood and park, as far as you can see--covered in sheets of paper. Now picture that paper filled with numbers. LLMs contain a LOT of parameters. That's one way to visualize a large language model, or at least a medium-size one: Printed out in 14-point type, a 200-billion-parameter model, such as GPT4o (released by OpenAI in 2024), could fill 46 square miles of paper--roughly enough to cover San Francisco.
The Dangerous Paradox of A.I. Abundance
Silicon Valley envisions artificial intelligence ushering in an era of economic plenty. But what if the benefits are largely confined to corporations and investors that own the technology itself? In early 2024, Anish Acharya, a general partner at Andreessen Horowitz, a big venture-capital firm based in Menlo Park, posted an article online titled "How AI Will Usher in an Era of Abundance." Since then, and even before, various Silicon Valley types have been tossing the term around loosely. Last summer, Elon Musk even adopted the term "sustainable abundance" for a new Tesla mission statement.
Get PromptBuilder AI prompt engineer for a one-time 39
When you purchase through links in our articles, we may earn a small commission. PromptBuilder removes the guesswork from AI prompting --turning simple ideas into high-quality prompts instantly, forever, for $39. If you've ever thought,, the answer is usually the prompt. Writing good prompts takes time, trial-and-error, and more patience than it should. PromptBuilder exists to skip all of that .
Manifold limit for the training of shallow graph convolutional neural networks
Tengler, Johanna, Brune, Christoph, Iglesias, José A.
We study the discrete-to-continuum consistency of the training of shallow graph convolutional neural networks (GCNNs) on proximity graphs of sampled point clouds under a manifold assumption. Graph convolution is defined spectrally via the graph Laplacian, whose low-frequency spectrum approximates that of the Laplace-Beltrami operator of the underlying smooth manifold, and shallow GCNNs of possibly infinite width are linear functionals on the space of measures on the parameter space. From this functional-analytic perspective, graph signals are seen as spatial discretizations of functions on the manifold, which leads to a natural notion of training data consistent across graph resolutions. To enable convergence results, the continuum parameter space is chosen as a weakly compact product of unit balls, with Sobolev regularity imposed on the output weight and bias, but not on the convolutional parameter. The corresponding discrete parameter spaces inherit the corresponding spectral decay, and are additionally restricted by a frequency cutoff adapted to the informative spectral window of the graph Laplacians. Under these assumptions, we prove $Γ$-convergence of regularized empirical risk minimization functionals and corresponding convergence of their global minimizers, in the sense of weak convergence of the parameter measures and uniform convergence of the functions over compact sets. This provides a formalization of mesh and sample independence for the training of such networks.
On the use of case estimate and transactional payment data in neural networks for individual loss reserving
Avanzi, Benjamin, Lambrianidis, Matthew, Taylor, Greg, Wong, Bernard
The use of neural networks trained on individual claims data has become increasingly popular in the actuarial reserving literature. We consider how to best input historical payment data in neural network models. Additionally, case estimates are also available in the format of a time series, and we extend our analysis to assessing their predictive power. In this paper, we compare a feed-forward neural network trained on summarised transactions to a recurrent neural network equipped to analyse a claim's entire payment history and/or case estimate development history. We draw conclusions from training and comparing the performance of the models on multiple, comparable highly complex datasets simulated from SPLICE (Avanzi, Taylor and Wang, 2023). We find evidence that case estimates will improve predictions significantly, but that equipping the neural network with memory only leads to meagre improvements. Although the case estimation process and quality will vary significantly between insurers, we provide a standardised methodology for assessing their value.
'Dangerous and alarming': Google removes some of its AI summaries after users' health put at risk
Google has said AI Overviews, which use generative AI to provide snapshots of information on a topic or question, are'helpful and reliable'. Google has said AI Overviews, which use generative AI to provide snapshots of information on a topic or question, are'helpful and reliable'. 'Dangerous and alarming': Google removes some of its AI summaries after users' health put at risk Google has removed some of its artificial intelligence health summaries after a Guardian investigation found people were being put at risk of harm by false and misleading information. The company has said its AI Overviews, which use generative AI to provide snapshots of essential information about a topic or question, are " helpful " and " reliable ". But some of the summaries, which appear at the top of search results, served up inaccurate health information, putting users at risk of harm.
OpenAI Is Asking Contractors to Upload Work From Past Jobs to Evaluate the Performance of AI Agents
To prepare AI agents for office work, the company is asking contractors to upload projects from past jobs, leaving it to them to strip out confidential and personally identifiable information. OpenAI is asking third-party contractors to upload real assignments and tasks from their current or previous workplaces so that it can use the data to evaluate the performance of its next-generation AI models, according to records from OpenAI and the training data company Handshake AI obtained by WIRED. The project appears to be part of OpenAI's efforts to establish a human baseline for different tasks that can then be compared with AI models. In September, the company launched a new evaluation process to measure the performance of its AI models against human professionals across a variety of industries. OpenAI says this is a key indicator of its progress towards achieving AGI, or an AI system that outperforms humans at most economically valuable tasks. "We've hired folks across occupations to help collect real-world tasks modeled off those you've done in your full-time jobs, so we can measure how well AI models perform on those tasks," reads one confidential document from OpenAI.
AI's Memorization Crisis
Large language models don't "learn"--they copy. And that could change everything for the tech industry. O n Tuesday, researchers at Stanford and Yale revealed something that AI companies would prefer to keep hidden. Four popular large language models--OpenAI's GPT, Anthropic's Claude, Google's Gemini, and xAI's Grok--have stored large portions of some of the books they've been trained on, and can reproduce long excerpts from those books. In fact, when prompted strategically by researchers, Claude delivered the near-complete text of,,, and, in addition to thousands of words from books including and .
The Download: the case for AI slop, and helping CRISPR fulfill its promise
If I were to locate the moment AI slop broke through into popular consciousness, I'd pick the video of rabbits bouncing on a trampoline that went viral last summer. For many savvy internet users, myself included, it was the first time we were fooled by an AI video, and it ended up spawning a wave of almost identical generated clips. My first reaction was that, broadly speaking, all of this sucked. That's become a familiar refrain, in think pieces and at dinner parties. Everything online is slop now--the internet "enshittified," with AI taking much of the blame. But then friends started sharing AI clips in group chats that were compellingly weird, or funny.
'Physical AI' Is Coming for Your Car
'Physical AI' Is Coming for Your Car What the latest tech-marketing buzzword has to say about the future of automotive. The systems powering the autonomous features in the Afeela 1 and Afeela prototype, both announced at CES, are the embodiment of "physical AI." Courtesy of Sony Honda Mobility Physical AI sounds like a contradiction in terms. But for the marketing architects, it's the latest term of art, a buzzword meant to point us citizens toward a bright and promising technological future. Back here on earth, the term is maybe most useful as a way to understand how automotive companies are thinking about themselves right now: as tech pioneers. It's also a handy shortcut to understanding how appetizing the automotive industry is for the companies that make chips-- what could be a $123 billion opportunity by 2032, up some 85 percent from 2023. The giant CES consumer tech showcase that just took place in Las Vegas always has its share of goofy robot demos, but this year's presentations showed how the world of robots, cars, and chipsets are growing ever closer.