Figure 1: AttentionViz, our interactive visualization tool, allows users to explore transformer self-attention at scale by creating a joint embedding space for queries and keys. Each point in the scatterplot represents the query or key version of a word, as denoted by point color. Users can explore individual attention heads (left) or zoom out for a "global" view of attention (right). Abstract--Transformer models are revolutionizing machine learning, but their inner workings remain mysterious. In this work, we present a new visualization technique designed to help researchers understand the self-attention mechanism in transformers that allows these models to learn rich, contextual relationships between elements of a sequence. The main idea behind our method is to visualize a joint embedding of the query and key vectors used by transformer models to compute attention. Unlike previous attention visualization techniques, our approach enables the analysis of global patterns across multiple input sequences. We create an interactive visualization tool, AttentionViz (demo: http://attentionviz.com), based on these joint query-key embeddings, and use it to study attention mechanisms in both language and vision transformers. We demonstrate the utility of our approach in improving model understanding and offering new insights about query-key interactions through several application scenarios and expert feedback. The transformer neural network architecture [52] is having a major impact In this work, we describe a new visualization technique aimed at on fields ranging from natural language processing (NLP) [13, 42] better comprehending how transformers operate. Indeed, transformers are now deployed in introduction to transformers in Sec. However, the mechanisms these models to learn and use a rich set of relationships between input behind this success remain somewhat mysterious, especially as elements.

The performance of Large Language Models (LLMs) in reasoning tasks depends heavily on prompt design, with Chain-of-Thought (CoT) and self-consistency being critical methods that enhance this ability. However, these methods do not fully exploit the answers generated by the LLM to guide subsequent responses. This paper proposes a new prompting method, named Progressive-Hint Prompting (PHP), that enables automatic multiple interactions between users and LLMs by using previously generated answers as hints to progressively guide toward the correct answers. PHP is orthogonal to CoT and self-consistency, making it easy to combine with state-of-the-art techniques to further improve performance. We conducted extensive and comprehensive experiments on seven benchmarks. The results show that PHP significantly improves accuracy while remaining highly efficient. For instance, with text-davinci-003, we observed a 4.2% improvement on GSM8K with greedy decoding compared to Complex CoT, and a 46.17% reduction in sample paths with self-consistency.

In 1950, the British computer scientist Alan Turing disputed whether human reasoning can be matched by computers: "Can machines think?" (TURING, 1950). Subsequently, he proposed the Turing Test to measure computer or artificial intelligence. In a Turing test, a human interrogator is presented with responses from a human and a computer (with the ability to generate written texts in real-time). If the interrogator cannot distinguish between the answers, the computer system passes the Turing Test. Although several computer programs and chatbots like Eliza demonstrated success in the Turing test ((Weizenbaum, 1966) (Güzeldere & Franchi, 1995)), these programs arguably used certain tricks to pass the test (Pinar Saygin et al., 2000) rather than demonstrating any significant intelligence. With the advancement in machine learning and natural language processing (NLP), chatbots have gained significant research attention and have been used for a variety of commercial and non-commercial applications ((Luo et al., 2022), (Adamopoulou & Moussiades, 2020), (Ranoliya et al., 2017), (Rahman et al., 2017), (Zhou et al., 2020)). Despite their vast adoption, most chatbots do not have personalization, and user satisfaction remains questionable (Følstad & Brandtzaeg, 2020). This limitation prompted researchers and developers to focus on chatbot engagement in making chatbots more conversational.

Automaton-based representations of task knowledge play an important role in control and planning for sequential decision-making problems. However, obtaining the high-level task knowledge required to build such automata is often difficult. Meanwhile, large-scale generative language models (GLMs) can automatically generate relevant task knowledge. However, the textual outputs from GLMs cannot be formally verified or used for sequential decision-making. We propose a novel algorithm named GLM2FSA, which constructs a finite state automaton (FSA) encoding high-level task knowledge from a brief natural-language description of the task goal. GLM2FSA first sends queries to a GLM to extract task knowledge in textual form, and then it builds an FSA to represent this text-based knowledge. The proposed algorithm thus fills the gap between natural-language task descriptions and automaton-based representations, and the constructed FSA can be formally verified against user-defined specifications. We accordingly propose a method to iteratively refine the queries to the GLM based on the outcomes, e.g., counter-examples, from verification. We demonstrate GLM2FSA's ability to build and refine automaton-based representations of everyday tasks (e.g., crossing a road), and also of tasks that require highly-specialized knowledge (e.g., executing secure multi-party computation).

I remember the first time I saw the floaty rock. It was the middle of night, and I was facing the insomniac's dilemma: to reach for the phone or not. I reached and opened Twitter--this was two weeks ago; the new name hadn't yet sunk in--on the theory that a scroll through my feed might achieve some hypnotic effect, creating an opening for sleep to take hold. That's when I saw the blurry video. In it, a scrap of material, small and misshapen like a pencil's broken lead tip, hovers mystically above a thick wafer of polished metal.

CEO says OpenAI CEO Sam Altman said language and cultural inclusivity is "very important" to his company's mission as it builds and trains powerful artificial intelligence systems. OpenAI has launched web crawler GPTBot to improve artificial intelligence models. "Web pages crawled with the GPTBot user agent may potentially be used to improve future models and are filtered to remove sources that require paywall access, are known to gather personally identifiable information (PII) or have text that violates our policies," the company said in a post on its website. "Allowing GPTBot to access your site can help AI models become more accurate and improve their general capabilities and safety," OpenAI wrote. A web crawler is a type of bot.

The beta version of Spotify's AI-enhanced DJ feature is coming to 50 new countries, after soft-launching in the US and Canada back in February. In recent months, it's rolled out in the UK and Ireland, but now the robotic Wolfman Jack is headed to more countries in Europe, Asia and Africa, in addition to Australia and New Zealand. There's a caveat, but it depends on some initial understanding of what this tool actually does. The Spotify DJ is available to premium subscription members and provides algorithmic recommendations of what to listen to, just like any music streaming app. However, these recommendations are accompanied by AI-generated DJ commentary on what you're listening to. The DJ, based on Spotify's Xavier Jernigan, only speaks English, no matter where you live.

Peter Paccone, a social studies teacher in San Marino, Calif., has a new teacher's aid helping him in the classroom this year. He plans to defer to his helper to explain some simpler topics to his class of high schoolers, like the technical aspects of how a cotton gin worked, in order to free up time for him to discuss more analytical concepts, like the effects of the first industrial revolution. "What I feel that I don't have to do any longer is cover all the content," Paccone told a group of more than 40 educators in a May Zoom workshop, which he organized. If artificial intelligence is on the cusp of reshaping entire aspects of our society--from healthcare to warfare--the first realm that leaps to many minds is education: Asked a question online, the ChatGPT chatbot will produce an answer that reads like an essay. So as students and teachers prepare for a new school year, they are also grappling with AI's implications for learning, homework, and integrity.

An unbiased, purely fact-based AI chatbot is a cute idea, but it's technically impossible. To understand why, it's worth reading a story I just published on new research that sheds light on how political bias creeps into AI language systems. Researchers conducted tests on 14 large language models and found that OpenAI's ChatGPT and GPT-4 were the most left-wing libertarian, while Meta's LLaMA was the most right-wing authoritarian. "We believe no language model can be entirely free from political biases," Chan Park, a PhD researcher at Carnegie Mellon University, who was part of the study, told me. One of the most pervasive myths around AI is that the technology is neutral and unbiased.

Large Language Models (LLMs) are advanced Artificial Intelligence (AI) systems that have undergone extensive training using large datasets in order to understand and produce language that closely resembles that of humans. These models have reached a level of proficiency where they are capable of successfully completing university exams across several disciplines and generating functional code to handle novel problems. This research investigates the coding proficiency of ChatGPT 3.5, a LLM released by OpenAI in November 2022, which has gained significant recognition for its impressive text generating and code creation capabilities. The skill of the model in creating code snippets is evaluated across 10 various programming languages and 4 different software domains. Based on the findings derived from this research, major unexpected behaviors and limitations of the model have been identified. This study aims to identify potential areas for development and examine the ramifications of automated code generation on the evolution of programming languages and on the tech industry.