Contemporary pipelines largely learn from preferences, often alongside scalable-oversight efforts ("superalignment" Burns et al. (2023); Kim et al. (2024); Köpf et al. (2023)), and a growing survey literature maps the practical trade-offs--from data collection and reward inference to evaluation and safety (e.g., Shen et al., 2023; Kaufmann et al., 2025). A common structure underlies many systems: models propose alternatives, people (or proxies) compare them, and those preferences guide the next training round (Shin et al., 2023; Lee et al., 2021; Munos et al., 2024). Within this landscape, two families dominate. Reinforcement Learning from Human Feedback (RLHF) first trains a reward model from comparisons, then improves the policy via reinforcement learning with KL regularization (typically Proximal Policy Optimization (PPO)). This accommodates rich, sequence-level signals, but it introduces extra moving parts--reward modeling, on-policy sampling, and tuning--that can make training complex and sometimes unstable at scale (Kirk et al., 2023).

The experimental model won't compete with the biggest and best, but it could tell us why they behave in weird ways--and how trustworthy they really are. ChatGPT maker OpenAI has built an experimental large language model that is far easier to understand than typical models. That's a big deal, because today's LLMs are black boxes: Nobody fully understands how they do what they do. Building a model that is more transparent sheds light on how LLMs work in general, helping researchers figure out why models hallucinate, why they go off the rails, and just how far we should trust them with critical tasks. "As these AI systems get more powerful, they're going to get integrated more and more into very important domains," Leo Gao, a research scientist at OpenAI, told in an exclusive preview of the new work. "It's very important to make sure they're safe."

When you purchase through links in our articles, we may earn a small commission. Become a more efficient and better writer with the help of AI. ChatGPT can help with many things--creating images, looking up information, role-playing, solving math problems, programming and much more. But at the heart of everything it does are so-called "large language models"--AI algorithms trained on unimaginable amounts of text. So it's not surprising that what it does best is working with text.

A smartphone shows the official website of ChatGPT Atlas. A smartphone shows the official website of ChatGPT Atlas. "It'd be really nice to have a service that was sort of just observing your life and proactively helping you when you needed it," said OpenAI CEO Sam Altman in a recent Q&A about OpenAI's plans. This vision is at the heart of a new crop of AI browsers, notably OpenAI's ChatGPT Atlas and Perplexity's Comet. AI browsers differ from traditional browsers in at least two important ways.

SIMA 2, which can figure out how to solve problems inside virtual worlds, could lead to more general-purpose agents and better robots. Google DeepMind has built a new video-game-playing agent called SIMA 2 that can navigate and solve problems in a wide range of 3D virtual worlds. The company claims it's a big step toward more general-purpose agents and better real-world robots. Google DeepMind first demoed SIMA (which stands for "scalable instructable multiworld agent") last year. But SIMA 2 has been built on top of Gemini, the firm's flagship large language model, which gives the agent a huge boost in capability. The researchers claim that SIMA 2 can carry out a range of more complex tasks inside virtual worlds, figure out how to solve certain challenges by itself, and chat with its users.

OpenAI's Open-Weight Models Are Coming to the US Military The gpt-oss models are being tested for use on sensitive military computers. But some defense insiders say that OpenAI is still behind the competition. When OpenAI unveiled its first open-weight models in years this August, it wasn't just tech companies that were paying attention. The release also excited US military and defense contractors, which saw a chance to use them for highly secure operations. Initial results show that OpenAI's tools lag behind competitors in desired capabilities, some military vendors tell WIRED.

Unlike single-objective optimization, which aggregates objectives into a scalar through weighted sums, MOPs focus on generating specific or diverse Pareto solutions and learning the entire Pareto set directly.