Large Language Model
The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?
Sun, Yutao, Chen, Mingshuai, Zhao, Tiancheng, Xu, Ruochen, Zhang, Zilun, Yin, Jianwei
Self-improving large language models (LLMs) -- i.e., to improve the performance of an LLM by fine-tuning it with synthetic data generated by itself -- is a promising way to advance the capabilities of LLMs while avoiding extensive supervision. Existing approaches to self-improvement often rely on external supervision signals in the form of seed data and/or assistance from third-party models. This paper presents Crescent -- a simple yet effective framework for generating high-quality synthetic question-answer data in a fully autonomous manner. Crescent first elicits the LLM to generate raw questions via a bait prompt, then diversifies these questions leveraging a rejection sampling-based self-deduplication, and finally feeds the questions to the LLM and collects the corresponding answers by means of majority voting. We show that Crescent sheds light on the potential of true self-improvement with zero external supervision signals for math reasoning; in particular, Crescent-generated question-answer pairs suffice to (i) improve the reasoning capabilities of an LLM while preserving its general performance (especially in the 0-shot setting); and (ii) distil LLM knowledge to weaker models more effectively than existing methods based on seed-dataset augmentation.
Mira Murati Launches Thinking Machines Lab to Make AI More Accessible
Last September, Mira Murati unexpectedly left her job as chief technology officer of OpenAI, saying, "I want to create the time and space to do my own exploration." The rumor in Silicon Valley was that she was stepping down to start her own company. Today she announced that indeed she is the CEO of a new public benefit corporation called Thinking Machines Lab. Its mission is to develop top-notch AI with an eye toward making it useful and accessible. Murati believes there's a serious gap between rapidly advancing AI and the public's understanding of the technology.
Elon Musk's startup rolls out new Grok-3 chatbot as AI competition intensifies
Elon Musk's artificial intelligence startup xAI has introduced Grok-3, the latest iteration of its chatbot that integrates with X, formerly Twitter. Grok-3 debut comes at a critical moment in the AI arms race as Musk looks to compete with the Chinese AI firm DeepSeek, Microsoft-backed OpenAI and Google. Musk's bot has seen less widespread adoption than DeepSeek's namesake chatbot, which wowed the world weeks ago and caused panic in stock markets, as well as OpenAI's ChatGPT and Google's Gemini. Grok-3 is being rolled out immediately to Premium subscribers of X, the social media platform owned by Musk. The chatbot can generate texts and images without many of the common guardrails against sexually suggestive imagery, vulgarity or the reproduction of well-known people's likenesses. "Grok-3 across the board is in a league of its own," Musk said during a livestream alongside three xAI engineers late on Monday.
xAI launches new Grok-3 AI model with DeepSearch reasoning
Elon Musk's AI company xAI just launched Grok-3, a new version of the AI model that powers the Grok AI chatbot. The new AI model was unveiled during a live broadcast on the social media platform formerly known as Twitter, with the creators describing it as the smartest AI in the world. The AI model comes in two flavors -- Grok-3 and Grok-3 mini -- and according to figures from xAI, both models perform as well or better than rivals from Google (Gemini), OpenAI (ChatGPT), Anthropic (Claude), and DeepSeek when it comes to math, science, and programming. Grok-3 also has a new built-in reasoning engine called DeepSearch, which lets you see the AI chatbot's thought process as it generates answers to your queries. The AI model was apparently trained for a total of 200 million processor hours on 100,000 Nvidia H100 Tensor Core GPUs.
xAI launches Grok 3 AI, claiming it is capable of 'human reasoning'
Meanwhile, the Grok 3 Reasoning and Grok 3 mini Reasoning models are capable of mimicking human-like reasoning when it comes to analyzing information the user needs. Other examples of AI models capable of reasoning tasks are DeepSeek's R1 and OpenAI's o3-mini. According to TechCrunch, xAI claimed during the event that Grok 3 Reasoning performed better than the best version of o3-mini on several benchmarks. Grok 3's features will initially be available to subscribers paying for X's Premium tier, which now costs 40 a month in the US. They will also be available through an upcoming separate subscription option for the standalone Grok app and Grok on the web.
'Hopeless' to potentially handy: law firm puts AI to the test
This was the second time Linklaters had run its LinksAI benchmark tests, with the original exercise taking place in October 2023. In the first run, OpenAI's GPT 2, 3 and 4 were tested alongside Google's Bard. The exam has now been expanded to include o1, from OpenAI, and Google's Gemini 2.0, which was also released at the end of 2024. It did not involve DeepSeek's R1 - the apparently low cost Chinese model which astonished the world last month - or any other non-US AI tool. The test involved posing the type of questions which would require advice from a "competent mid-level lawyer" with two years' experience.
DeepSeek 'shared user data' with TikTok owner ByteDance
Since then, multiple countries have warned that user data may not be properly protected, and in February a US cybersecurity company alleged potential data sharing between DeepSeek and ByteDance. DeepSeek's apparent overnight impact saw it shoot to the top of App Store charts in the UK, US and many other countries around the world - although it now sits far below ChatGPT in UK rankings. In South Korea, it had been downloaded over a million times before being pulled from Apple and Google's App Stores on Saturday evening. Existing users can still access the app and use it on a web browser. The data regulator, the Personal Information Protection Commission (PIPC), told South Korea's Yonhap News Agency that despite finding a link between DeepSeek and ByteDance, it was "yet to confirm what data was transferred and to what extent".
Musk debuts Grok-3 AI chatbot to rival OpenAI, DeepSeek
Elon Musk's artificial intelligence startup, xAI, showed off the updated Grok-3 model, showcasing a version of the chatbot technology that the billionaire has said is the "smartest AI on Earth." Across math, science and coding benchmarks, Grok-3 beats Alphabet's Google Gemini, DeepSeek's V3 model, Anthropic's Claude and OpenAI's GPT-4o, the company said via a live stream on Monday. Grok-3 has "more than 10 times" the computing power of its predecessor and completed pretraining in early January, Musk said in a presentation alongside three of xAI's engineers. "We're continually improving the models every day, and literally within 24 hours, you'll see improvements," Musk said.
Agentic Deep Graph Reasoning Yields Self-Organizing Knowledge Networks
We present an agentic, autonomous graph expansion framework that iteratively structures and refines knowledge in situ. Unlike conventional knowledge graph construction methods relying on static extraction or single-pass learning, our approach couples a reasoning-native large language model with a continually updated graph representation. At each step, the system actively generates new concepts and relationships, merges them into a global graph, and formulates subsequent prompts based on its evolving structure. Through this feedback-driven loop, the model organizes information into a scale-free network characterized by hub formation, stable modularity, and bridging nodes that link disparate knowledge clusters. Over hundreds of iterations, new nodes and edges continue to appear without saturating, while centrality measures and shortest path distributions evolve to yield increasingly distributed connectivity. Our analysis reveals emergent patterns, such as the rise of highly connected 'hub' concepts and the shifting influence of 'bridge' nodes, indicating that agentic, self-reinforcing graph construction can yield open-ended, coherent knowledge structures. Applied to materials design problems, we present compositional reasoning experiments by extracting node-specific and synergy-level principles to foster genuinely novel knowledge synthesis, yielding cross-domain ideas that transcend rote summarization and strengthen the framework's potential for open-ended scientific discovery. We discuss other applications in scientific discovery and outline future directions for enhancing scalability and interpretability.
Evaluating and Enhancing Out-of-Domain Generalization of Task-Oriented Dialog Systems for Task Completion without Turn-level Dialog Annotations
Mosharrof, Adib, Fereidouni, Moghis, Siddique, A. B.
Traditional task-oriented dialog (ToD) systems rely heavily on labor-intensive turn-level annotations, such as dialogue states and policy labels, for training. This work explores whether large language models (LLMs) can be fine-tuned solely on natural language dialogs to perform ToD tasks, without requiring such annotations. We evaluate their ability to generalize to unseen domains and compare their performance with models trained on fully annotated data. Through extensive experiments with three open-source LLMs of varying sizes and two diverse ToD datasets, we find that models fine-tuned without turn-level annotations generate coherent and contextually appropriate responses. However, their task completion performance - measured by accurate execution of API calls - remains suboptimal, with the best models achieving only around 53% success in unseen domains. To improve task completion, we propose ZeroToD, a framework that incorporates a schema augmentation mechanism to enhance API call accuracy and overall task completion rates, particularly in out-of-domain settings. We also compare ZeroToD with fine-tuning-free alternatives, such as prompting off-the-shelf LLMs, and find that our framework enables smaller, fine-tuned models that outperform large-scale proprietary LLMs in task completion. Additionally, a human study evaluating informativeness, fluency, and task completion confirms our empirical findings. These findings suggest the feasibility of developing cost-effective, scalable, and zero-shot generalizable ToD systems for real-world applications.