Multi-modal large language models are regarded as a crucial step towards Artificial General Intelligence (AGI) and have garnered significant interest with the emergence of ChatGPT. However, current speech-language models typically adopt the cascade paradigm, preventing inter-modal knowledge transfer. In this paper, we propose SpeechGPT, a large language model with intrinsic cross-modal conversational abilities, capable of perceiving and generating multi-model content. With discrete speech representations, we first construct SpeechInstruct, a large-scale cross-modal speech instruction dataset. Additionally, we employ a three-stage training strategy that includes modality-adaptation pre-training, cross-modal instruction fine-tuning, and chain-of-modality instruction fine-tuning. The experimental results demonstrate that SpeechGPT has an impressive capacity to follow multi-modal human instructions and highlight the potential of handling multiple modalities with one model. Demos are shown in https://0nutation.github.io/SpeechGPT.github.io/.

This copy is for your personal, non-commercial use only. For non-personal use or to order multiple copies, please contact Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.

The UK will lead on limiting the dangers of artificial intelligence, Rishi Sunak has said, after calls from some tech experts and business leaders for a moratorium. Sunak said AI could bring benefits and prove transformative for society, but it had to be introduced "safely and securely with guard rails in place". The prime minister's comments sound a more cautious approach than in the past, after tech leaders including Twitter's owner, Elon Musk, and Apple's co-founder Steve Wozniak added their names to nearly 30,000 signatures on a letter urging a pause in significant projects. The letter called for a moratorium while the capabilities and dangers of systems such as ChatGPT-4 are properly studied and mitigated in response to fears about the creation of digital minds, fraud, disinformation and the risk to jobs. Sunak has been an advocate of AI, emphasising its benefits rather than risks, and in March the government unveiled a light-touch regulatory programme that did not appear to include proposals for any new laws or enforcement bodies.

ChatGPT is now a smartphone app, which could be good news for people who like to use the artificial intelligence chatbot and bad news for all the clone apps that have tried to profit off the technology. The free app started to become available on iPhones in the United States on Thursday and will later be coming to Android phones. Unlike the web version, you can also ask it questions using your voice. The company that makes it, OpenAI, said it will remain ad-free but "syncs your history across devices". "We're starting our rollout in the US and will expand to additional countries in the coming weeks," said a blog post announcing the new app, which is described in the App Store as the "official app" by OpenAI.

It's the first official smartphone app for the viral language model, joining a crowded field of third-party mobile AI software vying for your attention -- many of which tap into the GPT-3.5 and GPT-4 APIs powering ChatGPT. It's only available in the US for now, but the company says it will expand to additional countries "in the coming weeks." Feature-wise, OpenAI's app looks and behaves much like the ChatGPT website -- with the addition of voice input using OpenAI's Whisper speech recognition. It also allows switching between standard and GPT-4 language models for ChatGPT Plus subscribers, as well as conversation history (synced from the desktop if you sign in with the same account) and the ability to export data and delete or rename conversations. However, the company's recently launched plugins, including live web access, are absent.

This article was co-published with Understanding AI, a newsletter that explores how A.I. works and how it's changing our world. When Silicon Valley executives testify before Congress, they normally get raked over the coals. But OpenAI CEO Sam Altman's Tuesday appearance before the Senate Judiciary Committee went differently. Senators asked Altman probing questions and listened respectfully to his answers. Afterward, the committee's chairman, Sen. Richard Blumenthal of Connecticut praised Altman.

If you've searched for "ChatGPT" in Apple's App Store since the chatbot launched six months ago, you may have discovered some of the dozens of apps with names like Genie, Genius, and AI Writer claiming to be powered by OpenAI's technology. Or you might have found Microsoft's Bing app with the company's own chatbot inside, powered by OpenAI's GPT-4 text generator. But ChatGPT itself hasn't had an official iPhone app released by its own developer--until now. As with the original web model of the chatbot, the free-to-use version is built on GPT-3.5, and its most capable persona built on GPT-4 is accessible only if you're paying $20 a month for ChatGPT Plus. OpenAI says the mobile app syncs your history of chats with its bot across devices and will be expanding to other countries "in the coming weeks."

This week, US senators heard alarming testimony suggesting that unchecked AI could steal jobs, spread misinformation, and generally "go quite wrong," in the words of OpenAI CEO Sam Altman (whatever that means). He and several lawmakers agreed that the US may now need a new federal agency to oversee the development of the technology. But the hearing also saw agreement that no one wants to kneecap a technology that could potentially increase productivity and give the US a lead in a new technological revolution. Worried senators might consider talking to Missy Cummings, a onetime fighter pilot and engineering and robotics professor at George Mason University. She studies use of AI and automation in safety critical systems including cars and aircraft, and earlier this year returned to academia after a stint at the National Highway Traffic Safety Administration, which oversees automotive technology, including Tesla's Autopilot and self-driving cars.

At a Senate hearing on Tuesday, the CEO of OpenAI Sam Altman received a warm welcome from lawmakers, many of whom expressed surprise at his main argument: that AI should be regulated, and fast. It was a far cry from the grueling ordeals that tech CEOs have previously faced on Capitol Hill. Mark Zuckerberg, Jack Dorsey and Shou Zi Chew have all endured antagonistic Senate hearings in recent years about the wide-ranging impacts of their platforms--Facebook, Twitter and TikTok, respectively--on American democracy and the lives of their users. "I think what's happening today in this hearing room is historic," said Senator Dick Durbin (D., Ill.) during the Senate judiciary subcommittee hearing about oversight of AI. "I can't recall when we've had people representing large corporations or private sector entities come before us and plead with us to regulate them." But in calling for legal guardrails to govern the tech his company is building, Altman is not unlike the other Silicon Valley leaders who have testified before Congress in the past.

We've collected some of the articles, opinion pieces, videos and resources relating to large language models (LLMs). Some of these links also cover other generative models. We will periodically update this list to add any further resources of interest. This article represents the first update.