Large Language Model
Google I/O 2024: Everything revealed including Gemini AI, Android 15 and more
At the end of I/O, Google's annual developer conference at the Shoreline Amphitheater in Mountain View, Google CEO Sundar Pichai revealed that the company had said "AI" 121 times. That, essentially, was the crux of Google's two-hour keynote -- stuffing AI into every Google app and service used by more than two billion people around the world. Here are all the major updates from Google's big event, along with some additional announcements that came after the keynote. Google announced a brand new AI model called Gemini 1.5 Flash, which it says is optimised for speed and efficiency. Flash sits between Gemini 1.5 Pro and Gemini 1.5 Nano, which its the company's smallest model that runs locally on device.
OpenAI and Google are launching supercharged AI assistants. Here's how you can try them out.
On Tuesday, Google announced its own new tools, including a conversational assistant called Gemini Live, which can do many of the same things. It also revealed that it's building a sort of "do-everything" AI agent, which is currently in development but will not be released until later this year. Soon you'll be able to explore for yourself to gauge whether you'll turn to these tools in your daily routine as much as their makers hope, or whether they're more like a sci-fi party trick that eventually loses its charm. Here's what you should know about how to access these new tools, what you might use them for, and how much it will cost. What it's capable of: The model can talk with you in real time, with a response delay of about 320 milliseconds, which OpenAI says is on par with natural human conversation.
OpenAI overtakes Google in race to build the future, but who wants it?
When OpenAI released its ChatGPT tool in November 2022, it was a shot across the bows of Google, with generative artificial intelligence promising a new way to access the world's information beyond search engines. Since then, the rivalry between these firms has only grown, with both announcing new services this week. While there are signs that OpenAI is winning this race, is either company aiming for a future anyone actually wants?
OpenAI co-founder who had key role in attempted firing of Sam Altman departs
OpenAI's co-founder and chief scientist, Ilya Sutskever, is leaving the startup at the center of today's artificial intelligence boom. "After almost a decade, I have made the decision to leave OpenAI," Sutskever said in a post on X. Sutskever played a key role in the dramatic firing and rehiring in November last year of OpenAI's CEO, Sam Altman. At the time, Sutskever was on the board of OpenAI and helped to orchestrate Altman's firing. Days later, he reversed course, signing on to an employee letter demanding Altman's return and expressing regret for his "participation in the board's actions". After Altman returned, Sutskever was removed from the board, and his position at the company became unclear.
The Morning After: The biggest news from Google's I/O keynote
Google boss, Sundar Pichai, wrapped up the company's I/O developer conference by noting its almost-two-hour presentation had mentioned AI 121 times. Google's newest AI model, Gemini 1.5 Flash, is built for speed and efficiency. The company said it created Flash because developers wanted a lighter, less expensive model than Gemini Pro to build AI-powered apps and services. Google says it'll double Gemini's context window to two million tokens, enough to process two hours of video, 22 hours of audio, more than 60,000 lines of code or 1.4 million-plus words at the same time. But the bigger news is how the company is sewing AI into all the things you're already using.
OpenAI co-founder and Chief Scientist Ilya Sutskever is leaving the company
Ilya Sutskever has announced on X, formerly known as Twitter, that he's leaving OpenAI almost a decade after he co-founded the company. He's confident that OpenAI "will build [artificial general intelligence] that is both safe and beneficial" under the leadership of CEO Sam Altman, President Greg Brockman and CTO Mira Murati, he continued. In his own post about Sutskever's departure, Altman called him "one of the greatest minds of our generation" and credited him for his work with the company. Jakub Pachocki, OpenAI's previous Director of Research who headed the development of GPT-4 and OpenAI Five, has taken Sutskever's role as Chief Scientist. After almost a decade, I have made the decision to leave OpenAI.
OpenAI's Co-Founder and Chief Scientist Ilya Sutskever Is Leaving the Company
OpenAI Chief Scientist and co-founder Ilya Sutskever is leaving the artificial intelligence company, a departure that ends months of speculation in Silicon Valley about the future of a top AI researcher who played a key role in the brief ouster of Sam Altman last year. Sutskever will be replaced by Research Director Jakub Pachocki, OpenAI said on its blog Tuesday. In a post on X, Sutskever called trajectory of OpenAI "miraculous" and said that he was confident the company will build AI that is "both safe and beneficial" under its current leadership. The exit removes an executive and renowed researcher who has played a pivotal role in the company since its earliest days, helping guide discussions over the safety of AI technology and at times differing with Altman over strategy. When OpenAI was founded in 2015, he served as its research director after being recruited to join the company by Elon Musk.
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery
Ma, Pingchuan, Wang, Tsun-Hsuan, Guo, Minghao, Sun, Zhiqing, Tenenbaum, Joshua B., Rus, Daniela, Gan, Chuang, Matusik, Wojciech
Large Language Models have recently gained significant attention in scientific discovery for their extensive knowledge and advanced reasoning capabilities. However, they encounter challenges in effectively simulating observational feedback and grounding it with language to propel advancements in physical scientific discovery. Conversely, human scientists undertake scientific discovery by formulating hypotheses, conducting experiments, and revising theories through observational analysis. Inspired by this, we propose to enhance the knowledge-driven, abstract reasoning abilities of LLMs with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework: LLMs act as knowledgeable and versatile thinkers, proposing scientific hypotheses and reason about discrete components, such as physics equations or molecule structures; meanwhile, simulations function as experimental platforms, providing observational feedback and optimizing via differentiability for continuous parts, such as physical parameters. We conduct extensive experiments to demonstrate our framework's efficacy in constitutive law discovery and molecular design, unveiling novel solutions that differ from conventional human expectations yet remain coherent upon analysis.
Optimization Techniques for Sentiment Analysis Based on LLM (GPT-3)
Zhan, Tong, Shi, Chenxi, Shi, Yadong, Li, Huixiang, Lin, Yiyu
With the rapid development of natural language processing (NLP) technology, large-scale pre-trained language models such as GPT-3 have become a popular research object in NLP field. This paper aims to explore sentiment analysis optimization techniques based on large pre-trained language models such as GPT-3 to improve model performance and effect and further promote the development of natural language processing (NLP). By introducing the importance of sentiment analysis and the limitations of traditional methods, GPT-3 and Fine-tuning techniques are introduced in this paper, and their applications in sentiment analysis are explained in detail. The experimental results show that the Fine-tuning technique can optimize GPT-3 model and obtain good performance in sentiment analysis task. This study provides an important reference for future sentiment analysis using large-scale language models.
A Survey on Transformers in NLP with Focus on Efficiency
Ansar, Wazib, Goswami, Saptarsi, Chakrabarti, Amlan
The advent of transformers with attention mechanisms and associated pre-trained models have revolutionized the field of Natural Language Processing (NLP). However, such models are resource-intensive due to highly complex architecture. This limits their application to resource-constrained environments. While choosing an appropriate NLP model, a major trade-off exists over choosing accuracy over efficiency and vice versa. This paper presents a commentary on the evolution of NLP and its applications with emphasis on their accuracy as-well-as efficiency. Following this, a survey of research contributions towards enhancing the efficiency of transformer-based models at various stages of model development along with hardware considerations has been conducted. The goal of this survey is to determine how current NLP techniques contribute towards a sustainable society and to establish a foundation for future research.