Large language models (LLMs) are increasingly adopted for a variety of tasks with implicit graphical structures, such as planning in robotics, multi-hop question answering or knowledge probing, structured commonsense reasoning, and more. While LLMs have advanced the state-of-the-art on these tasks with structure implications, whether LLMs could explicitly process textual descriptions of graphs and structures, map them to grounded conceptual spaces, and perform structured operations remains underexplored. To this end, we propose NLGraph (Natural Language Graph), a comprehensive benchmark of graph-based problem solving designed in natural language. NLGraph contains 29,370 problems, covering eight graph reasoning tasks with varying complexity from simple tasks such as connectivity and shortest path up to complex problems such as maximum flow and simulating graph neural networks. We evaluate LLMs (GPT-3/4) with various prompting approaches on the NLGraph benchmark and find that 1) language models do demonstrate preliminary graph reasoning abilities, 2) the benefit of advanced prompting and in-context learning diminishes on more complex graph problems, while 3) LLMs are also (un)surprisingly brittle in the face of spurious correlations in graph and problem settings. We then propose Build-a-Graph Prompting and Algorithmic Prompting, two instruction-based approaches to enhance LLMs in solving natural language graph problems. Build-a-Graph and Algorithmic prompting improve the performance of LLMs on NLGraph by 3.07% to 16.85% across multiple tasks and settings, while how to solve the most complicated graph reasoning tasks in our setup with language models remains an open research question.

Many real-world applications of language models (LMs), such as writing assistance and code autocomplete, involve human-LM interaction. However, most benchmarks are non-interactive in that a model produces output without human involvement. To evaluate human-LM interaction, we develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems and dimensions to consider when designing evaluation metrics. Compared to standard, non-interactive evaluation, HALIE captures (i) the interactive process, not only the final output; (ii) the first-person subjective experience, not just a third-party assessment; and (iii) notions of preference beyond quality (e.g., enjoyment and ownership). We then design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. With four state-of-the-art LMs (three variants of OpenAI's GPT-3 and AI21 Labs' Jurassic-1), we find that better non-interactive performance does not always translate to better human-LM interaction. In particular, we highlight three cases where the results from non-interactive and interactive metrics diverge and underscore the importance of human-LM interaction for LM evaluation.

At the turn of the century, when the modern web was just emerging and Microsoft was king, a small but growing technology movement posed an existential threat to the company. Steve Ballmer, Microsoft's CEO at the time, called one of its core elements "a cancer that attaches itself" to "everything it touches." The disease was a competing operating system, Linux, and the open-source software it represented: programs that were free for anyone to download, modify, and use, in contrast to expensive, proprietary software such as Microsoft Windows and Office. Open-source software did eventually attach itself to much of the internet--Mozilla Firefox, the Android operating system, and Wikipedia are all "open" projects--but the tech industry managed to turn the egalitarian philosophy into a business opportunity. Trillion-dollar companies use free open-source software to build or enhance their own products.

Google Bard Advanced is coming, and it may represent the company's first attempt to charge for an AI chatbot. The code on Bard's website shows the opportunity to get three months free of Bard Advanced -- implying that it will come at a cost after that. X (formerly Twitter) user Bedros Pamboukian shared the discovered code, which also includes a defunct Google One link. This could mean that users will be required to get a Google One subscription to access Bard Advanced. Google first announced Bard Advanced in December 2023 alongside the new AI large language model Gemini and an AI hypercomputer from Google Cloud.

While the Google Books case might work in the tech firms' favor, the fair-use picture was muddied by the Supreme Court's recent decision in a case involving artist Andy Warhol's use of a photograph of the rock star Prince, said Daniel Gervais, a professor at Vanderbilt Law and director of its intellectual property program. The court found that if the copying is done to compete with the original work, "that weighs against fair use" as a defense. So the Times's case may hinge in part on its ability to show that products like ChatGPT and Bing compete with and harm its business.

Alter3 is a humanoid robot first introduced in 2016. Imagine what it would be like to have a robot friend that can do things like take selfies, toss a ball, eat popcorn and play air guitar? Well, you might not have to wait too long. Researchers at the University of Tokyo have created a robot that can do all that and more, thanks to the power of GPT-4, the latest and most advanced large language model (LLM) in the world. CLICK TO GET KURT'S FREE CYBERGUY NEWSLETTER WITH SECURITY ALERTS, QUICK VIDEO TIPS, TECH REVIEWS, AND EASY HOW-TO'S TO MAKE YOU SMARTER A researcher gives Alter3, a humanoid robot, verbal instructions.

Coming soon to a Windows 11 PC near you: A dedicated keyboard key for Microsoft's Copilot AI. The company calls this "the first significant change to the Windows PC keyboard in nearly three decades" in a blog post ahead of CES 2024 in Las Vegas. Just like how the Windows key serves as a quick way to open up the Windows Start menu, the Copilot key immediately summons Microsoft's AI companion to do your bidding. While it's not a huge surprise to see Copilot manifest itself in the real world, especially given just how much Microsoft has been pushing AI over the past year, it's a bit of a shock to see it happen so quickly. The company claims the Copilot key is "another transformative moment in our journey with Windows where Copilot will be the entry point into the world of AI on the PC." Sure, let's go with that.

A humanoid robot that can perform actions based on text prompts could pave the way for machines that behave more like us and communicate using gestures. Large language models (LLMs) like GPT-4, the artificial intelligence behind ChatGPT, are proficient at writing many kinds of computer code, but they can struggle when it comes to doing this for robot movement. This is because almost every robot has very different physical forms and software to control its parts.

Poker, also known as Texas Hold'em, has always been a typical research target within imperfect information games (IIGs). IIGs have long served as a measure of artificial intelligence (AI) development. Representative prior works, such as DeepStack and Libratus heavily rely on counterfactual regret minimization (CFR) to tackle heads-up no-limit Poker. However, it is challenging for subsequent researchers to learn CFR from previous models and apply it to other real-world applications due to the expensive computational cost of CFR iterations. Additionally, CFR is difficult to apply to multi-player games due to the exponential growth of the game tree size. In this work, we introduce PokerGPT, an end-to-end solver for playing Texas Hold'em with arbitrary number of players and gaining high win rates, established on a lightweight large language model (LLM). PokerGPT only requires simple textual information of Poker games for generating decision-making advice, thus guaranteeing the convenient interaction between AI and humans. We mainly transform a set of textual records acquired from real games into prompts, and use them to fine-tune a lightweight pre-trained LLM using reinforcement learning human feedback technique. To improve fine-tuning performance, we conduct prompt engineering on raw data, including filtering useful information, selecting behaviors of players with high win rates, and further processing them into textual instruction using multiple prompt engineering techniques. Through the experiments, we demonstrate that PokerGPT outperforms previous approaches in terms of win rate, model size, training time, and response speed, indicating the great potential of LLMs in solving IIGs.

In this paper, we design a real-time question-answering system specifically targeted for helping sellers get relevant material/documentation they can share live with their customers or refer to during a call. Taking the Seismic content repository as a relatively large scale example of a diverse dataset of sales material, we demonstrate how LLM embeddings of sellers' queries can be matched with the relevant content. We achieve this by engineering prompts in an elaborate fashion that makes use of the rich set of meta-features available for documents and sellers. Using a bi-encoder with cross-encoder re-ranker architecture, we show how the solution returns the most relevant content recommendations in just a few seconds even for large datasets. Our recommender system is deployed as an AML endpoint for real-time inferencing and has been integrated into a Copilot interface that is now deployed in the production version of the Dynamics CRM, known as MSX, used daily by Microsoft sellers.