AI-powered question-answering (Q&A) systems have emerged as important tools, alongside established search technologies, to enable quick access to relevant information and knowledge from large digital sources that are complex and time-consuming for humans to navigate. Advancements in large language models (LLMs) have revolutionized the field of Q&A, with models like GPT-3 (Brown et al. 2020), BERT (Devlin et al. 2018), and RoBERTa (Liu et al. 2019) demonstrating remarkable abilities in understanding and generating human-like text. However, the effectiveness of such models in handling domain-specific questions that require specialized knowledge is limited. Retrieval-augmented generation (RAG) techniques, which combine information retrieval and generative models (Lewis et al. 2021), have shown promise in boosting the quality of LLM output in Q&A tasks. RAG systems leverage the strengths of both retrieval and generation components to provide contextually relevant and informative responses. While there is a lack of established quantification of RAG accuracy, early findings suggest that generic RAG does not perform well in complex domains such as finance. In one instance, RAG based on generic LLMs such as GPT-4-Turbo fails to answer 81% of the questions derived from Securities and Exchange Commission (SEC) financial filings (Islam et al. 2023). Aitomatic, Inc. (except as noted, all authors are from Aitomatic)

The Socratic method is a way of guiding students toward solving a problem independently without directly revealing the solution to the problem. Although this method has been shown to significantly improve student learning outcomes, it remains a complex labor-intensive task for instructors. Large language models (LLMs) can be used to augment human effort by automatically generating Socratic questions for students. However, existing methods that involve prompting these LLMs sometimes produce invalid outputs, e.g., those that directly reveal the solution to the problem or provide irrelevant or premature questions. To alleviate this problem, inspired by reinforcement learning with AI feedback (RLAIF), we first propose a data augmentation method to enrich existing Socratic questioning datasets with questions that are invalid in specific ways. Next, we propose a method to optimize open-source LLMs such as LLama 2 to prefer ground-truth questions over generated invalid ones, using direct preference optimization (DPO). Our experiments on a Socratic questions dataset for student code debugging show that a DPO-optimized 7B LLama 2 model can effectively avoid generating invalid questions, and as a result, outperforms existing state-of-the-art prompting methods.

As full AI-based automation remains out of reach in most real-world applications, the focus has instead shifted to leveraging the strengths of both human and AI agents, creating effective collaborative systems. The rapid advances in this area have yielded increasingly more complex systems and frameworks, while the nuance of their characterization has gotten more vague. Similarly, the existing conceptual models no longer capture the elaborate processes of these systems nor describe the entire scope of their collaboration paradigms. In this paper, we propose a new unified set of dimensions through which to analyze and describe human-AI systems. Our conceptual model is centered around three high-level aspects - agency, interaction, and adaptation - and is developed through a multi-step process. Firstly, an initial design space is proposed by surveying the literature and consolidating existing definitions and conceptual frameworks. Secondly, this model is iteratively refined and validated by conducting semi-structured interviews with nine researchers in this field. Lastly, to illustrate the applicability of our design space, we utilize it to provide a structured description of selected human-AI systems.

Reinforcement Learning From Human Feedback (RLHF) has been a critical to the success of the latest generation of generative AI models. In response to the complex nature of the classical RLHF pipeline, direct alignment algorithms such as Direct Preference Optimization (DPO) have emerged as an alternative approach. Although DPO solves the same objective as the standard RLHF setup, there is a mismatch between the two approaches. Standard RLHF deploys reinforcement learning in a specific token-level MDP, while DPO is derived as a bandit problem in which the whole response of the model is treated as a single arm. In this work we rectify this difference, first we theoretically show that we can derive DPO in the token-level MDP as a general inverse Q-learning algorithm, which satisfies the Bellman equation. Using our theoretical results, we provide three concrete empirical insights. First, we show that because of its token level interpretation, DPO is able to perform some type of credit assignment. Next, we prove that under the token level formulation, classical search-based algorithms, such as MCTS, which have recently been applied to the language generation space, are equivalent to likelihood-based search on a DPO policy. Empirically we show that a simple beam search yields meaningful improvement over the base DPO policy. Finally, we show how the choice of reference policy causes implicit rewards to decline during training. We conclude by discussing applications of our work, including information elicitation in multi-tun dialogue, reasoning, agentic applications and end-to-end training of multi-model systems.

The quadratic complexity and weak length extrapolation of Transformers limits their ability to scale to long sequences, and while sub-quadratic solutions like linear attention and state space models exist, they empirically underperform Transformers in pretraining efficiency and downstream task accuracy. We introduce Megalodon, a neural architecture for efficient sequence modeling with unlimited context length. Megalodon inherits the architecture of Mega (exponential moving average with gated attention), and further introduces multiple technical components to improve its capability and stability, including complex exponential moving average (CEMA), timestep normalization layer, normalized attention mechanism and pre-norm with two-hop residual configuration. In a controlled head-to-head comparison with Llama2, Megalodon achieves better efficiency than Transformer in the scale of 7 billion parameters and 2 trillion training tokens. Megalodon reaches a training loss of 1.70, landing mid-way between Llama2-7B (1.75) and 13B (1.67). Code: https://github.com/XuezheMax/megalodon

Former Secretary of State Hillary Clinton took a swipe at voters "upset" by the forthcoming rematch between President Biden and former President Trump during her appearance on "The Tonight Show." Hillary Clinton reacted to a recent ruling in Arizona, which bans abortion in nearly all circumstances, calling it "cruelty" during an interview with Kelly Clarkson and encouraging Americans to vote in a way that would "make life better" for the largest number of people. "I feared it would happen but I hoped it wouldn't happen. Now here we are in the middle of this very difficult period for women in about half the states in our country, who cannot get the care that they need. And the old law in Arizona is without exceptions and the danger to women's lives as well as to our right to make our own decisions about our bodies and ourselves is so profound," Clinton said during the interview with Clarkson on "The Kelly Clarkson Show."

How to Do It is Slate's sex advice column. Send it to Jessica and Rich here. I'm (35F) very interested in having a threesome and have been working the apps to try to find the right person to help make this happen. I've had a few bites. I was sexting with one guy for days on end about our joint fantasy of making this happen, and I found a second guy, who said he'd like to join us.

Sen. Fetterman said he disagreed with President Biden's decision to keep the U.S. out of any offensive response to the Iran attacks on Sunday during an interview with CNN. Sen. John Fetterman, D-Pa., said he didn't agree with President Biden on his stance that the U.S. wouldn't join in an offensive operation against Iran during an interview on Sunday, saying he would never "capitulate to the fringe" of his party. CNN host Jake Tapper asked Fetterman to respond to reports that Biden told Israeli Prime Minister Benjamin Netanyahu that the U.S. wouldn't participate in any offensive operations against Iran during a conversation on Saturday. "Do you think that's the right call or should direct U.S. military action, as some of your colleagues in the Senate are suggesting, should that be on the table?" he asked. "I don't agree with that, I just think we should follow and have Israel's back in the situation. I don't agree with the president. I'm proud to stand with him and campaign for him and vote for him," he responded.

In this study, we employ a classification approach to show that different categories of literary "quality" display unique linguistic profiles, leveraging a corpus that encompasses titles from the Norton Anthology, Penguin Classics series, and the Open Syllabus project, contrasted against contemporary bestsellers, Nobel prize winners and recipients of prestigious literary awards. Our analysis reveals that canonical and so called high-brow texts exhibit distinct textual features when compared to other quality categories such as bestsellers and popular titles as well as to control groups, likely responding to distinct (but not mutually exclusive) models of quality. We apply a classic machine learning approach, namely Random Forest, to distinguish quality novels from "control groups", achieving up to 77\% F1 scores in differentiating between the categories. We find that quality category tend to be easier to distinguish from control groups than from other quality categories, suggesting than literary quality features might be distinguishable but shared through quality proxies.

In London last night, the 20th Bafta games awards celebrated a year that was stacked with critically acclaimed games. Taking place against the backdrop of an unprecedented year of layoffs and studio closures in the gaming industry, acknowledged by Bafta chair Sara Putt in her speech at the beginning of the evening, it was a much-needed night of recognition of the creative efforts of the video game development community. The sprawling Dungeons & Dragons-inspired role-playing game Baldur's Gate 3 won five awards, including the public voted EE players' choice award and best game, alongside music, narrative and best performer in a supporting role (won by Andrew Wincott for his role at the devilish Raphael). Nintendo picked up the family and multiplayer awards for the exuberant Super Mario Bros Wonder, and technical achievement for The Legend of Zelda: Tears of the Kingdom. Alan Wake 2, the arresting, idiosyncratic horror game from Finnish studio Remedy, won artistic achievement and audio achievement.