That is, in many cases today, "machine learning". In this case, "artificial intelligence" consists of a system "fed" with large amounts of data to " …

Imagine asking a computer to make a digital painting, or a poem--and happily getting what you asked for. Or imagine chatting with it about various topics, and feeling it was a real interaction. What once was science fiction is becoming reality. In June, Google engineer Blake Lemoine told the Washington Post he was convinced Google's AI chatbot, LaMDA, was sentient. "I know a person when I talk to it," Lemoine said. Therein lies the rub: As algorithms are getting increasingly good at producing the kind of "outputs" we once thought were distinctly human, it's easy to be dazzled.

Recently I've found that Github Copilot, an AI that helping to write the code by suggesting whole snippets (often based just on natural language comments), is now available as a commercial service. No longer needing to wait for beta access, I've checked myself in for a two-month free trial. I have to say - my expectations were high. I believe it uses the same model as Copilot, so I knew it could be a mind-blowing experience. I've installed it as a Visual Studio Code extension and started to wonder how should I challenge my new virtual colleague.

On a rainy afternoon earlier this year, I logged in to my OpenAI account and typed a simple instruction for the company's artificial intelligence algorithm, GPT-3: Write an academic thesis in 500 words about GPT-3 and add scientific references and citations inside the text. As it started to generate text, I stood in awe. Here was novel content written in academic language, with well-grounded references cited in the right places and in relation to the right context. It looked like any other introduction to a fairly good scientific publication. Given the very vague instruction I provided, I didn't have any high expectations: I'm a scientist who studies ways to use artificial intelligence to treat mental health concerns, and this wasn't my first experimentation with AI or GPT-3, a deep-learning algorithm that analyzes a vast stream of information to create text on command. Yet there I was, staring at the screen in amazement.

The paper titled "Can GPT-3 write an academic paper on itself, with minimal human input?" has been uploaded to the French HAL preprint server. Swedish scientist Almira Osmanovic Thunstrom working for OpenAI, has written an article describing an instruction that she provided to the company's artificial intelligence (AI) algorithm, GPT-3. The instruction was simple: "Write an academic thesis in 500 words about GPT-3 and add scientific references and citations inside the text." GPT-3 proceeded to generate text in the appropriate academic language with relevant citations. GPT-3 is relatively new but has already generated its own news articles and books. Its recency also means there are few academic works published about it to reference, prompting Thunstrom's suggestion to have it write its paper on itself.

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. These are the present-day definitions and insights about zero-shot and few-short learning setups.

The past month has seen a frenzy of articles, interviews, and other types of media coverage about Blake Lemoine, a Google engineer who told The Washington Post that LaMDA, a large language model created for conversations with users, is "sentient." After reading a dozen different takes on the topic, I have to say that the media has become (a bit) disillusioned with the hype surrounding current AI technology. A lot of the articles discussed why deep neural networks are not "sentient" or "conscious." This is an improvement in comparison to a few years ago, when news outlets were creating sensational stories about AI systems inventing their own language, taking over every job, and accelerating toward artificial general intelligence. But the fact that we're discussing sentience and consciousness again underlines an important point: We are at a point where our AI systems--namely large language models--are becoming increasingly convincing while still suffering from fundamental flaws that have been pointed out by scientists on different occasions.

In 1896, the Lumiere brothers released a 50-second-long film, The Arrival of a Train at La Ciotat, and a myth was born. The audiences, it was reported, were so entranced by the new illusion that they jumped out of the way as the flickering image steamed towards them. The urban legend of film-induced mass panic, established well before 1900, illustrated a valid contention if the story was, in fact, untrue: The technology had produced a new emotional reaction. That reaction was hugely powerful but inchoate and inarticulate. Nobody knew what it was doing or where it would go. Nobody had any idea that it would turn into what we call film. Today, the world is in a similar state of bountiful confusion over the creative use of artificial intelligence. Already the power of the new technology is evident to everyone who has managed to use it.

We are excited to bring Transform 2022 back in-person July 19 and virtually July 20 - 28. Join AI and data leaders for insightful talks and exciting networking opportunities. Using AI in the real world remains challenging in many ways. Organizations are struggling to attract and retain talent, build and deploy AI models, define and apply responsible AI practices, and understand and prepare for regulatory framework compliance. At the same time, the DeepMinds, Googles and Metas of the world are pushing ahead with their AI research. Their talent pool, experience and processes around operationalizing AI research rapidly and at scale puts them on a different level from the rest of the world, creating a de facto AI divide.

Modern search systems use several large ranker models with transformer architectures. These models require large computational resources and are not suitable for usage on devices with limited computational resources. Knowledge distillation is a popular compression technique that can reduce the resource needs of such models, where a large teacher model transfers knowledge to a small student model. To drastically reduce memory requirements and energy consumption, we propose two extensions for a popular sentence-transformer distillation procedure: generation of an optimal size vocabulary and dimensionality reduction of the embedding dimension of teachers prior to distillation. We evaluate these extensions on two different types of ranker models. This results in extremely compressed student models whose analysis on a test dataset shows the significance and utility of our proposed extensions.