Recent studies have shown that large language models (LLMs), especially smaller ones, often lack robustness in grade school math (GSM) reasoning. In particular, they tend to experience performance drops when faced with distribution shifts, such as changes to numerical or nominal variables, or insertions of distracting clauses. A possible strategy to address this involves generating synthetic data to further "instantiate" reasoning problems on potential variations. In this work, we instead focuses on the strategy of "abstracting" reasoning problems. This not only helps counteract distribution shifts but also facilitates the connection to symbolic tools for deriving solutions. Focusing on GSM, we find that this abstraction process is better acquired through reinforcement learning (RL) than just supervised fine-tuning, which often fails to produce faithful abstractions. Our method, AbstRaL -- which promotes abstract reasoning in LLMs using RL on granular abstraction data -- significantly mitigates performance degradation on recent GSM perturbation benchmarks. Besides, improving GSM robustness via AbstRaL is shown to also implicitly benefit LLMs' capabilities on OOD mathematical and general reasoning tasks, indicating that abstract thinking broadly enables better generalizability.

Fine-tuning all parameters of large language models (LLMs) necessitates substantial computational power and extended time. Latest advancements in parameter-efficient fine-tuning (PEFT) techniques, such as Adapter tuning and LoRA, allow for adjustments to only a minor fraction of the parameters of these LLMs. Concurrently, it has been noted that the issue of over-smoothing diminishes the effectiveness of these Transformer-based LLMs, resulting in suboptimal performances in downstream tasks. In this paper, we present SIBO, which is a SImple BOoster to enhance PEFT, by injecting an initial residual. SIBO is straightforward and readily extensible to a range of state-of-the-art PEFT techniques to alleviate over-smoothing and enhance performance. Extensive experiments on 22 benchmark datasets demonstrate that SIBO significantly enhances the performance of various strong baselines, achieving up to 15.7% and 23.5% improvement over existing PEFT methods on the arithmetic and commonsense reasoning tasks, respectively.

Recent advancements highlight the success of instruction tuning with large language models (LLMs) utilizing Chain-of-Thought (CoT) data for mathematical reasoning tasks. Despite the fine-tuned LLMs, challenges persist, such as incorrect, missing, and redundant steps in CoT generation leading to inaccuracies in answer predictions. To alleviate this problem, we propose a dual instruction tuning strategy to meticulously model mathematical reasoning from both forward and reverse directions. This involves introducing the Intermediate Reasoning State Prediction task (forward reasoning) and the Instruction Reconstruction task (reverse reasoning) to enhance the LLMs' understanding and execution of instructions. Training instances for these tasks are constructed based on existing mathematical instruction tuning datasets. Subsequently, LLMs undergo multi-task fine-tuning using both existing mathematical instructions and the newly created data. Comprehensive experiments validate the effectiveness and domain generalization of the dual instruction tuning strategy across various mathematical reasoning tasks.

Decoding the core of procedural texts, exemplified by cooking recipes, is crucial for intelligent reasoning and instruction automation. Procedural texts can be comprehensively defined as a sequential chain of steps to accomplish a task employing resources. From a cooking perspective, these instructions can be interpreted as a series of modifications to a food preparation, which initially comprises a set of ingredients. These changes involve transformations of comestible resources. For a model to effectively reason about cooking recipes, it must accurately discern and understand the inputs Figure 1: A graphical depiction of the PizzaCommonsense and outputs of intermediate steps within the underlying motivation. Models are required to recipe. Aiming to address this, we present a learn knowledge about the input and output of each intermediate new corpus of cooking recipes enriched with step and predict the correct sequencing of descriptions of intermediate steps of the recipes these comestibles given the corresponding instructions that explicate the input and output for each step.

When it comes to writing a dating profile, it may seem hard to stand out from the crowd. But experts have worked out the best way to come across as more attractive – by getting creative with your words. Researchers asked users of online dating sites to rate dating profiles, and found that those who used metaphors and more concrete information were rated as more attractive, intelligent and funny. For example, the team suggests that instead of writing'I am a very good cook', you could use a metaphor and write'I am a star in the kitchen.' Alternatively, you could jazz up'Food is essential for me' by writing'Coffee and a cracker with cheese or jam are essential in my morning ritual'.

"Former President Donald J. Trump said on Monday that the F.B.I. had searched his Palm Beach, Fla., home and had broken open a safe." A Motorola Razr with last voice mail ex-girlfriend Cindy left. Haven't been able to bring myself to listen, but assume she says she loves me, I'm a great guy, and we can get back together anytime. Twelve hundred dollars in savings bonds my grandmother gave me for my twelfth birthday, due to mature in 2035. Will probably use money to buy a robot.

The AI could be able to work with a person to write a smart, funny christmas joke for the next christmas dinner. Well, I never really expected this article to turn into an attempt at AI in the first place. I was very excited to see a topic that had the potential to be fun, I just never imagined that it would turn into a massive joke on machines. Title: AI's Christmas Cracker Jokes (That Really Should Have Been Jokes) This is a new year's prediction. The world has got to the point where it can produce joke-writing AI, and some companies will develop those to sell them for christmas.

Just over a year has passed since the assassination of prime minister Olof Palme, a brutal and still unsolved handgun murder taking place in the heart of the nation's capital, Stockholm. Head of the ongoing police investigation, Hans Holmér, has just been forced to resign after cooking up increasingly incompetent and theatrical policing methods, lies and conspiracy theories. Cold war activity in the Baltic region is at peak levels. The navy regularly carry out large scale submarine hunts within Swedish territorial waters and the air force routinely scramble JA-37 Viggen interceptors in response to both Soviet and NATO counterparts scouting just outside national airspace. To fill the ranks of the armed forces, military service is mandated by universal conscription of all men aged 18 and above. Those who refuse to partake are punished by jail.

Picking the perfect password comes down to a battle between two competing priorities: creating safe passwords that are lengthy and unique, and creating ones you can remember. You might think to yourself, I already have more passwords than I need! I've created passwords for years! But with the rise of password breaches, and with more passwords exposed that are linked to usernames, a solid password strategy is becoming more essential every day. We'll start out with the basics: the best ways to store passwords, and how to avoid using popular, easily-guessed passwords.