schaeffer
Optimizing Hard Thresholding for Sparse Model Discovery
Jollie, Derek W., McCalla, Scott G.
Many model selection algorithms rely on sparse dictionary learning to provide interpretable and physics-based governing equations. The optimization algorithms typically use a hard thresholding process to enforce sparse activations in the model coefficients by removing library elements from consideration. By introducing an annealing scheme that reactivates a fraction of the removed terms with a cooling schedule, we are able to improve the performance of these sparse learning algorithms. We concentrate on two approaches to the optimization, SINDy, and an alternative using hard thresholding pursuit. We see in both cases that annealing can improve model accuracy. The effectiveness of annealing is demonstrated through comparisons on several nonlinear systems pulled from convective flows, excitable systems, and population dynamics. Finally we apply these algorithms to experimental data for projectile motion.
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks (0.69)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.67)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Optimization (0.67)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Model-Based Reasoning (0.67)
A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions
Negrini, Elisa, Liu, Yuxuan, Yang, Liu, Osher, Stanley J., Schaeffer, Hayden
Neural networks are one tool for approximating non-linear differential equations used in scientific computing tasks such as surrogate modeling, real-time predictions, and optimal control. PDE foundation models utilize neural networks to train approximations to multiple differential equations simultaneously and are thus a general purpose solver that can be adapted to downstream tasks. Current PDE foundation models focus on either learning general solution operators and/or the governing system of equations, and thus only handle numerical or symbolic modalities. However, real-world applications may require more flexible data modalities, e.g. text analysis or descriptive outputs. To address this gap, we propose a novel multimodal deep learning approach that leverages a transformer-based architecture to approximate solution operators for a wide variety of ODEs and PDEs. Our method integrates numerical inputs, such as equation parameters and initial conditions, with text descriptions of physical processes or system dynamics. This enables our model to handle settings where symbolic representations may be incomplete or unavailable. In addition to providing accurate numerical predictions, our approach generates interpretable scientific text descriptions, offering deeper insights into the underlying dynamics and solution properties. The numerical experiments show that our model provides accurate solutions for in-distribution data (with average relative error less than 3.3%) and out-of-distribution data (average relative error less than 7.8%) together with precise text descriptions (with correct descriptions generated 100% of times). In certain tests, the model is also shown to be capable of extrapolating solutions in time.
- North America > United States > California > Los Angeles County > Los Angeles (0.28)
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.14)
- North America > United States > Tennessee (0.04)
- Asia > Singapore > Central Region > Singapore (0.04)
VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction
Cao, Yadi, Liu, Yuxuan, Yang, Liu, Yu, Rose, Schaeffer, Hayden, Osher, Stanley
In-Context Operator Networks (ICONs) are models that learn operators across different types of PDEs using a few-shot, in-context approach. Although they show successful generalization to various PDEs, existing methods treat each data point as a single token, and suffer from computational inefficiency when processing dense data, limiting their application in higher spatial dimensions. In this work, we propose Vision In-Context Operator Networks (VICON), incorporating a vision transformer architecture that efficiently processes 2D functions through patch-wise operations. We evaluated our method on three fluid dynamics datasets, demonstrating both superior performance (reducing scaled $L^2$ error by $40\%$ and $61.6\%$ for two benchmark datasets for compressible flows, respectively) and computational efficiency (requiring only one-third of the inference time per frame) in long-term rollout predictions compared to the current state-of-the-art sequence-to-sequence model with fixed timestep prediction: Multiple Physics Pretraining (MPP). Compared to MPP, our method preserves the benefits of in-context operator learning, enabling flexible context formation when dealing with insufficient frame counts or varying timestep values.
- North America > United States > California > Los Angeles County > Los Angeles (0.14)
- North America > United States > California > San Diego County > San Diego (0.04)
- Asia > Singapore (0.04)
- Energy (0.68)
- Government > Regional Government > North America Government > United States Government (0.46)
AI and Insurance
Machine learning and Artificial Intelligence (AI) have been used by the insurance industry for more than a decade, yet this fast-moving technology is ever-changing, continually reshaping the form and function of the insurance process. Insurance companies use AI "to make their business more efficient," says Kathleen Birrane, Maryland Insurance Commissioner and the chair of innovation, cybersecurity, and technology committee at the National Association of Insurance Commissioners. "They use it in the underwriting process and in evaluating where the risk is in the claims process. There is no part of the process that doesn't use this technology." AI is simply the technology of computers simulating human reasoning by analyzing masses of historical data.
- North America > United States > Maryland (0.25)
- North America > Canada > Alberta (0.15)
Study urges caution when comparing neural networks to the brain
Neural networks, a type of computing system loosely modeled on the organization of the human brain, form the basis of many artificial intelligence systems for applications such speech recognition, computer vision, and medical image analysis. In the field of neuroscience, researchers often use neural networks to try to model the same kind of tasks that the brain performs, in hopes that the models could suggest new hypotheses regarding how the brain itself performs those tasks. However, a group of researchers at MIT is urging that more caution should be taken when interpreting these models. In an analysis of more than 11,000 neural networks that were trained to simulate the function of grid cells -- key components of the brain's navigation system -- the researchers found that neural networks only produced grid-cell-like activity when they were given very specific constraints that are not found in biological systems. "What this suggests is that in order to obtain a result with grid cells, the researchers training the models needed to bake in those results with specific, biologically implausible implementation choices," says Rylan Schaeffer, a former senior research associate at MIT.
We Live in the World of "WandaVision"
If--like Wanda Maximoff--you've been living in your own reality, distant from all things in 2021, you may not have heard about "WandaVision," whose first and only season ended on March 5th. The immensely popular show, from Disney and Marvel Studios, follows Wanda, a.k.a. the Scarlet Witch, an Eastern European refugee with "chaos magic" powers, and her husband Vision, a synthezoid (android) who died in the events of the Marvel movie "Avengers: Infinity War." Nearly all nine episodes of "WandaVision" depict the pair in what appears to be domestic suburban bliss. Nearly all take plots and visual style from one of the sitcoms that Wanda watched for solace during her bleak wartime youth, from the black and white of "The Dick Van Dyke Show" to the faux-reality vibe of "The Office." These anachronistic, self-contained sitcom scenarios fall apart as people from the outside world break in.
- Leisure & Entertainment (0.90)
- Media > Film (0.49)
- Government > Regional Government > North America Government > United States Government (0.47)
Procedural Generation of Initial States of Sokoban
Bento, Dâmaris S., Pereira, André G., Lelis, Levi H. S.
Procedural generation of initial states of state-space search problems have applications in human and machine learning as well as in the evaluation of planning systems. In this paper we deal with the task of generating hard and solvable initial states of Sokoban puzzles. We propose hardness metrics based on pattern database heuristics and the use of novelty to improve the exploration of search methods in the task of generating initial states. We then present a system called Beta that uses our hardness metrics and novelty to generate initial states. Experiments show that Beta is able to generate initial states that are harder to solve by a specialized solver than those designed by human experts.
- South America > Chile > Santiago Metropolitan Region > Santiago Province > Santiago (0.04)
- South America > Brazil > Rio Grande do Sul (0.04)
I, Edmonton
Ever since computers were clunky, whirring machines that took up entire floors, humans have marvelled at their potential, envisioning all the ways they could help or even be like us. Tapping into our own dark nature, science fiction tends to reach what creepily feels like the natural conclusion of obscenely smart machines with human dispositions; our demise. There's no robot apocalypse on the horizon, but the revolution is well under way. It's been here, in some form, since the '60s, and it's poised to lead the city, and world, in to the future. On April 1, 1964, U of A built Canada's first Department of Computing Science around five academics, a small support staff and the LGP-30, an 800-pound, deep freeze-shaped digital computer.
- North America > Canada > Alberta (0.16)
- North America > United States > California (0.14)
- North America > United States > Ohio (0.04)
- (3 more...)
- Health & Medicine > Therapeutic Area (1.00)
- Leisure & Entertainment > Games (0.70)
- Education (0.69)
A Bot Backed by Elon Musk Has Made an AI Breakthrough in Video Game World
Artificial-intelligence research group OpenAI said it created software capable of beating teams of five skilled human players in the video game Dota 2, a milestone in computer science. The achievement puts San Francisco-based OpenAI, whose backers include billionaire Elon Musk, ahead of other artificial-intelligence researchers in developing software that can master complex games combining fast, real-time action, longer-term strategy, imperfect information and team play. The ability to learn these kinds of video games at human or super-human levels is important for the advancement of AI because they more closely approximate the uncertainties and complexity of the real world than games such as chess, which IBM's software mastered in the late 1990s, or Go, which was conquered in 2016 with software created by DeepMind, the London-based AI company owned by Alphabet Inc. Dota 2 is a multiplayer science-fiction fantasy video game created by Bellevue, Washington-based Valve Corp. Each team is assigned a base on opposing ends of a map that can only be learned through exploration. Each player controls a separate character with unique powers and weapons. Each team must battle to reach the opposing team's territory and destroy a structure called an Ancient.
- North America > United States > Washington > King County > Bellevue (0.25)
- North America > United States > California > San Francisco County > San Francisco (0.25)
- North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.05)
- North America > Canada > Alberta > Census Division No. 11 > Edmonton Metropolitan Region > Edmonton (0.05)
- Information Technology > Artificial Intelligence > Games (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.87)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.55)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning > Generative AI (0.55)
This AI robot just destroyed humans in complex computer game
Artificial-intelligence research group OpenAI said it created software capable of beating teams of five skilled human players in the video game Dota 2, a milestone in computer science. The achievement puts San Francisco-based OpenAI, whose backers include billionaire Elon Musk, ahead of other artificial-intelligence researchers in developing software that can master complex games combining fast, real-time action, longer-term strategy, imperfect information and team play. The ability to learn these kinds of video games at human or super-human levels is important for the advancement of AI because they more closely approximate the uncertainties and complexity of the real world than games such as chess, which IBM's software mastered in the late 1990s, or Go, which was conquered in 2016 with software created by DeepMind, the London-based AI company owned by Alphabet Inc. Dota 2 is a multiplayer science-fiction fantasy video game created by Bellevue, Washington-based Valve Corp. Each team is assigned a base on opposing ends of a map that can only be learned through exploration. Each player controls a separate character with unique powers and weapons. Each team must battle to reach the opposing team's territory and destroy a structure called an Ancient.
- North America > United States > Washington > King County > Bellevue (0.25)
- North America > United States > California > San Francisco County > San Francisco (0.25)
- North America > Canada > British Columbia > Metro Vancouver Regional District > Vancouver (0.05)
- North America > Canada > Alberta > Census Division No. 11 > Edmonton Metropolitan Region > Edmonton (0.05)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (0.87)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.55)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning > Generative AI (0.55)