Yesterday in an announcement blog post, AI company Anthropic unveiled Claude 4, its new generation of AI models consisting of Claude 4 Opus and Claude 4 Sonnet with a range of new abilities. Both Claude 4 models are hybrid models, which means they're capable of giving you short-and-quick answers or thinking longer on their responses with deeper reasoning. Claude 4 Opus is excellent at solving complex problems and at programming. The model can maintain its performance in long tasks over several hours with thousands of different steps. Meanwhile, Anthropic says Claude 4 Sonnet is a huge upgrade over Claude 3.7 Sonnet's abilities.

Efficient exploration is crucial to achieving good performance in reinforcement learning. Existing systematic exploration strategies (R-MAX, MBIE, UCRL, etc.), despite being promising theoretically, are essentially greedy strategies that follow some predefined heuristics. When the heuristics do not match the dynamics of Markov decision processes (MDPs) well, an excessive amount of time can be wasted in travelling through already-explored states, lowering the overall efficiency. We argue that explicit planning for exploration can help alleviate such a problem, and propose a Value Iteration for Exploration Cost (VIEC) algorithm which computes the optimal exploration scheme by solving an augmented MDP.

This is a clear example of exploration-then-exploitation behaviour with exactly one phase change in the process.

Breakthroughs, discoveries, and DIY tips sent every weekday. After decades of being tortured, shoved, kicked, burned, and bludgeoned, robots are finally getting their chance to fight back. This weekend, Chinese robotics maker Unitree says it will livestream the world's first boxing match between two of its humanoid robots. The event, titled Unitree Iron Fist King: Awakening, will feature a face-off between two of Unitree's 4.3-foot-tall G1 robots. The robots will reportedly be remotely controlled by human engineers, though they are also expected to demonstrate some autonomous, pre-programmed actions as well.

Figure 9: Example showing how a single line of HTML code is rendered by a browser's renderer. In this example, we can see that the tags

delimit different blocks which are therefore spaced by line breaks while other tags, such as , are rendered on the same line of text that precedes and follows them.

Iteratively Reweighted Least Squares (IRLS) is an easy to implement family of algorithms for solving these problems that has been studied for over 50 years. However, these algorithms often diverge for p>3, and since the work of Osborne (1985), it has been an open problem whether there is an IRLS algorithm that is guaranteed to converge rapidly for p>3. We propose p-IRLS, the first IRLS algorithm that provably converges geometrically for any p 2 [2, 1). Our algorithm is simple to implement and is guaranteed to find a high accuracy solution in a sub-linear number of iterations. Our experiments demonstrate that it performs even better than our theoretical bounds, beats the standard Matlab/CVX implementation for solving these problems by 10-50x, and is the fastest among available implementations in the high-accuracy regime.

As of yesterday, Microsoft has begun rolling out a new update to Windows 11 Insiders on the Dev and Canary Channels. This update brings new AI features to Notepad, Paint, and the Snipping Tool. Notepad now has the ability to write text from scratch using generative AI, which is meant to aid you by quickly producing drafts based on your prompts and instructions. To use AI text generation, simply right-click anywhere in the document and select Write. Type in your instructions, then either click Keep Text or Discard on the results.

Reinforcement learning from human feedback (RLHF) is widely used to train large language models (LLMs). However, it is unclear whether LLMs accurately learn the underlying preferences in human feedback data. We coin the term Learned Feedback Pattern (LFP) for patterns in an LLM's activations learned during RLHF that improve its performance on the fine-tuning task. We hypothesize that LLMs with LFPs accurately aligned to the fine-tuning feedback exhibit consistent activation patterns for outputs that would have received similar feedback during RLHF. To test this, we train probes to estimate the feedback signal implicit in the activations of a fine-tuned LLM. We then compare these estimates to the true feedback, measuring how accurate the LFPs are to the fine-tuning feedback. Our probes are trained on a condensed, sparse and interpretable representation of LLM activations, making it easier to correlate features of the input with our probe's predictions. We validate our probes by comparing the neural features they correlate with positive feedback inputs against the features GPT-4 describes and classifies as related to LFPs. Understanding LFPs can help minimize discrepancies between LLM behavior and training objectives, which is essential for the safety and alignment of LLMs.

Quantum computers could crack a common data encryption technique once they have a million qubits, or quantum bits. While this is still well beyond the capabilities of existing quantum computers, this new estimate is 20 times lower than previously thought, suggesting the day encryption is cracked is closer than we think.

The pandemic changed how people worked, shifting most professionals to remote or hybrid models. For the software company Atlassian, this flexible, distributed approach persists to this day. "We have 13,000 employees spread across the globe, and individuals can choose their working location every day," said Annie Dean, Head of Team Anywhere, Atlassian's distributed work policy. "It's about how we work, not where we work." The implementation of the flexible model has produced positive effects for employees and the company alike. Internal data reveals that even though only 34% of employees have opted to work from home, 92% of Atlassian employees reported that the ability to work from anywhere allows them to perform their best, and 91% said it's an important reason for staying at the company.