Large Language Model
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Ostendorff, Malte, Rehm, Georg
Most Transformer language models are primarily pretrained on English text, limiting their use for other languages. As the model sizes grow, the performance gap between English and other languages with fewer compute and data resources increases even further. Consequently, more resource-efficient training methods are needed to bridge the gap for languages with fewer resources available. To address this problem, we introduce a cross-lingual and progressive transfer learning approach, called CLP-Transfer, that transfers models from a source language, for which pretrained models are publicly available, like English, to a new target language. As opposed to prior work, which focused on the cross-lingual transfer between two languages, we extend the transfer to the model size. Given a pretrained model in a source language, we aim for a same-sized model in a target language. Instead of training a model from scratch, we exploit a smaller model that is in the target language but requires much fewer resources. Both small and source models are then used to initialize the token embeddings of the larger model based on the overlapping vocabulary of the source and target language. All remaining weights are reused from the model in the source language. This approach outperforms the sole cross-lingual transfer and can save up to 80% of the training steps compared to the random initialization.
AI Passes U.S. Medical Licensing Exam
Two artificial intelligence (AI) programs -- including ChatGPT -- have passed the U.S. Medical Licensing Examination (USMLE), according to two recent papers. The papers highlighted different approaches to using large language models to take the USMLE, which is comprised of three exams: Step 1, Step 2 CK, and Step 3. ChatGPT is an artificial intelligence (AI) search tool that mimics long-form writing based on prompts from human users. It was developed by OpenAI, and became popular after several social media posts showed potential uses for the tool in clinical practice, often with mixed results. The first paper, published on medRxiv in December, investigated ChatGPT's performance on the USMLE without any special training or reinforcement prior to the exams. According to Victor Tseng, MD, of Ansible Health in Mountain View, California, and colleagues, the results showed "new and surprising evidence" that this AI tool was up to the challenge.
South Australian universities to allow use of artificial intelligence in assignments, if disclosed
Universities should stop panicking and embrace students' use of artificial intelligence, AI experts say. South Australia's three main universities have updated their policies to allow the use of AI as long as it is disclosed. The advent of ChatGPT, a language processing chatbot that can produce very human-like words, sparked fears students would use it to write essays. Anti-plagiarism software wouldn't pick it up because ChatGPT isn't plagiarising anything, it's producing new work in response to prompts from users. Flinders University, the University of Adelaide and the University of South Australia have adjusted their policies to allow AI use under strict controls.
University of Calgary AI project asks students, teachers about the use of ChatGPT - Calgary
The AI can write essays and poems and answer questions about many topics. "If I thought about using ChatGPT for creating an essay, the answer could be ok – I might use it, but I will also be using these other resources because my interest as a student is to create something that can be for the benefit of others," said Moya who is a research assistant with a new University of Calgary research project investigating the ethical use of AI in post-secondary learning and teaching. "I want to be creative and create opportunities for others." Known as ChatGPT and created by a company called OpenAI, the software is designed to generate human-like responses to a wide range of inputs by using algorithms. "It was interesting to see that this tool could provide some particular insights that could become the starting point of something," Moya said.
ChatGPT listed as author on research papers: many scientists disapprove
The artificial-intelligence (AI) chatbot ChatGPT that has taken the world by storm has made its formal debut in the scientific literature -- racking up at least four authorship credits on published papers and preprints. Journal editors, researchers and publishers are now debating the place of such AI tools in the published literature, and whether it's appropriate to cite the bot as an author. Publishers are racing to create policies for the chatbot, which was released as a free-to-use tool in November by tech company OpenAI in San Francisco, California. AI bot ChatGPT writes smart essays -- should professors worry? ChatGPT is a large language model (LLM), which generates convincing sentences by mimicking the statistical patterns of language in a huge database of text collated from the Internet. The bot is already disrupting sectors including academia: in particular, it is raising questions about the future of university essays and research production.
Use OpenAI with Google Spreadsheets
This article explains how you can integrate OpenAI GPT-3 with Google Spreadsheets. This allows you to complete spreadsheet tasks with the use of AI. Tip: Make sure to subscribe to above Gist since all future revisions with improvements will be made to this file. Then you can refer to this file later and update your functions. Note: When there are revisions to functions in the Gist file we discussed above, this is the same place you need to update the new revised code as well.
Google vs. ChatGPT told by aglio e olio
I don't really cook often, but I do love making pasta aglio e olio. It's a simple recipe that I discovered many years ago on YouTube. Yet, being the extremely irregular cook I am, I have to look up that simple recipe every once in a while. And every time I have to search for it, which is every six months or go, I feel like I have a worse experience on the internet than the last time. And by internet, I mean the Google monopoly.
How Can ChatGPT Help Twitch Streamers Get More Followers? – The Noname Nerd
I've been playing around with AI systems for about a year now. Lately, I've started playing with ChatGPT and, while I think that our society still needs to wrestle with some of the ethical implications of tools like this, I think that AI can be powerful if we understand it as a tool--one that is useful, but also imperfect, can be misused, and requires human supervision and editing. With the assistance of ChatGPT itself, here are some ways in which ChatGPT can help Twitch streamers get more followers. ChatGPT can assist Twitch streamers in a variety of ways to help them gain more followers. Providing engaging and informative chatbot responses to viewers during live streams, which can help keep viewers engaged and encourage them to follow the streamer.
Top AI startup news of the week: AI21 Labs, Mad Street Den, aiOla, and more
Check out all the on-demand sessions from the Intelligent Security Summit here. Announcements around generative AI-powered writing assistance show no signs of slowing this week, while a few sizable funding rounds fill out the top AI startup news. AI21 Labs launched Wordtune Spices this week, an add-on to its popular Wordtune editor platform, which offers a challenge to rival OpenAI's ChatGPT by providing users with links back to web-based sources -- something ChatGPT does not do. Spices is also meant to work alongside a writer to assist in the writing process. For example, it can generate a range of text options to add to and enhance sentences, or suggest statistics to strengthen an argument or sharpen a detail.