Pre-trained and frozen large language models (LLMs) can effectively map simple scene rearrangement instructions to programs over a robot's visuomotor functions through appropriate few-shot example prompting. To parse open-domain natural language and adapt to a user's idiosyncratic procedures, not known during prompt engineering time, fixed prompts fall short. In this paper, we introduce HELPER, an embodied agent equipped with an external memory of language-program pairs that parses free-form human-robot dialogue into action programs through retrieval-augmented LLM prompting: relevant memories are retrieved based on the current dialogue, instruction, correction, or VLM description, and used as in-context prompt examples for LLM querying. The memory is expanded during deployment to include pairs of user's language and action plans, to assist future inferences and personalize them to the user's language and routines. HELPER sets a new state-of-the-art in the TEACh benchmark in both Execution from Dialog History (EDH) and Trajectory from Dialogue (TfD), with a 1.7x improvement over the previous state-of-the-art for TfD. Our models, code, and video results can be found in our project's website: https://helper-agent-llm.github.io.

This work proposes to measure the scope of a patent claim as the reciprocal of the self-information contained in this claim. A probability of occurrence of the claim is obtained from a language model and this probability is used to compute the self-information. Grounded in information theory, this approach is based on the assumption that an unlikely concept is more informative than a usual concept, insofar as it is more surprising. In turn, the more surprising the information required to defined the claim, the narrower its scope. Five language models are considered, ranging from simplest models (each word or character is assigned an identical probability) to intermediate models (using average word or character frequencies), to a large language model (GPT2). Interestingly, the scope resulting from the simplest language models is proportional to the reciprocal of the number of words or characters involved in the claim, a metric already used in previous works. Application is made to multiple series of patent claims directed to distinct inventions, where each series consists of claims devised to have a gradually decreasing scope. The performance of the language models is assessed with respect to several ad hoc tests. The more sophisticated the model, the better the results. I.e., the GPT2 probability model outperforms models based on word and character frequencies, which themselves outdo the simplest models based on word or character counts. Still, the character count appears to be a more reliable indicator than the word count.

Large language models (LLMs) have emerged as a new paradigm for Text-to-SQL task. However, the absence of a systematical benchmark inhibits the development of designing effective, efficient and economic LLM-based Text-to-SQL solutions. To address this challenge, in this paper, we first conduct a systematical and extensive comparison over existing prompt engineering methods, including question representation, example selection and example organization, and with these experimental results, we elaborate their pros and cons. Based on these findings, we propose a new integrated solution, named DAIL-SQL, which refreshes the Spider leaderboard with 86.6% execution accuracy and sets a new bar. To explore the potential of open-source LLM, we investigate them in various scenarios, and further enhance their performance with supervised fine-tuning. Our explorations highlight open-source LLMs' potential in Text-to-SQL, as well as the advantages and disadvantages of the supervised fine-tuning. Additionally, towards an efficient and economic LLM-based Text-to-SQL solution, we emphasize the token efficiency in prompt engineering and compare the prior studies under this metric. We hope that our work provides a deeper understanding of Text-to-SQL with LLMs, and inspires further investigations and broad applications.

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.

A goal of artificial intelligence is to construct an agent that can solve a wide variety of tasks. Recent progress in text-guided image synthesis has yielded models with an impressive ability to generate complex novel images, exhibiting combinatorial generalization across domains. Motivated by this success, we investigate whether such tools can be used to construct more general-purpose agents. Specifically, we cast the sequential decision making problem as a text-conditioned video generation problem, where, given a text-encoded specification of a desired goal, a planner synthesizes a set of future frames depicting its planned actions in the future, after which control actions are extracted from the generated video. By leveraging text as the underlying goal specification, we are able to naturally and combinatorially generalize to novel goals. The proposed policy-as-video formulation can further represent environments with different state and action spaces in a unified space of images, which, for example, enables learning and generalization across a variety of robot manipulation tasks. Finally, by leveraging pretrained language embeddings and widely available videos from the internet, the approach enables knowledge transfer through predicting highly realistic video plans for real robots.

According to tech news site the Information, Mr Altman and Greg Brockman - another co-founder who quit on Friday as the company's president - were invited to the firm's San Francisco headquarters for talks on Sunday.

This copy is for your personal, non-commercial use only. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.

When Sam Altman, Elon Musk, and other investors formed the startup behind ChatGPT as a US not-for-profit organization in 2015, Altman told Vanity Fair he had very little experience with nonprofits. "So I'm just not sure how it's going to go," he said. He couldn't have imagined the drama of this week, with four directors on OpenAI's nonprofit board unexpectedly firing him as CEO and removing the company's president as chairman of the board. But the bylaws Altman and his cofounders initially established and a restructuring in 2019 that opened the door to billions of dollars in investment from Microsoft gave a handful of people with no financial stake in the company the power to upend the project on a whim. An attempt to restore Altman as CEO and replace the board ran into difficulty Sunday over the role of existing directors in choosing their replacements, Bloomberg reported.

Newly ousted OpenAI CEO Sam Altman and former president Greg Brockman are meeting with executives at the company's San Francisco headquarters now as discussions about possibly reinstating their positions continue, The Information reports. Per The Information, interim CEO Mira Murati and others have been leading the push to get Altman reinstated as CEO, and invited the two to HQ on Sunday. Altman and Brockman showed up for talks this afternoon, sources told The Information. Around the time of the report's publication, Altman tweeted a photo of himself wearing a guest badge for entry into the building, writing, "first and last time i ever wear one of these (sic)" -- which could be interpreted several different ways, at this point. Sources told The Verge that Altman has set a 5PM PT deadline for board members to reach an agreement that could ultimately determine whether he walks away from OpenAI, or they do. After Altman was fired without warning on Friday, Brockman stepped down in solidarity, along with a slew of senior researchers.

Sam Altman is being lined up for a surprise return as the chief executive of the ChatGPT developer OpenAI amid pressure from investors to reverse his surprise ousting. Altman was fired by the company board on Friday, citing a failure to be "candid in his communications", in a move that shocked Silicon Valley. However, OpenAI's investors – who include Microsoft – are pushing for his reinstatement, according to reports. On Saturday, the Information, a tech news website, reported that OpenAI was "optimistic" it could bring back Altman. The report quoted a memo from the company's chief strategy officer, Jason Kwon, telling staff that an effort was under way to bring back Altman and other senior colleagues who had left. "We are still working towards a resolution and we remain optimistic," Kwon wrote, according to the Information.