Generative AI models have shown impressive ability to produce images with text prompts, which could benefit creativity in visual art creation and self-expression. However, it is unclear how precisely the generated images express contexts and emotions from the input texts. We explored the emotional expressiveness of AI-generated images and developed RePrompt, an automatic method to refine text prompts toward precise expression of the generated images. Inspired by crowdsourced editing strategies, we curated intuitive text features, such as the number and concreteness of nouns, and trained a proxy model to analyze the feature effects on the AI-generated image. With model explanations of the proxy model, we curated a rubric to adjust text prompts to optimize image generation for precise emotion expression. We conducted simulation and user studies, which showed that RePrompt significantly improves the emotional expressiveness of AI-generated images, especially for negative emotions.

Text embeddings are commonly evaluated on a small set of datasets from a single task not covering their possible applications to other tasks. It is unclear whether state-of-the-art embeddings on semantic textual similarity (STS) can be equally well applied to other tasks like clustering or reranking. This makes progress in the field difficult to track, as various models are constantly being proposed without proper evaluation. To solve this problem, we introduce the Massive Text Embedding Benchmark (MTEB). MTEB spans 8 embedding tasks covering a total of 58 datasets and 112 languages. Through the benchmarking of 33 models on MTEB, we establish the most comprehensive benchmark of text embeddings to date. We find that no particular text embedding method dominates across all tasks. This suggests that the field has yet to converge on a universal text embedding method and scale it up sufficiently to provide state-of-the-art results on all embedding tasks. MTEB comes with open-source code and a public leaderboard at https://github.com/embeddings-benchmark/mteb.

Pre-trained Large Language Models (LLMs) are an integral part of modern AI that have led to breakthrough performances in complex AI tasks. Major AI companies with expensive infrastructures are able to develop and train these large models with billions and millions of parameters from scratch. Third parties, researchers, and practitioners are increasingly adopting these pre-trained models and fine-tuning them on their private data to accomplish their downstream AI tasks. However, it has been shown that an adversary can extract/reconstruct the exact training samples from these LLMs, which can lead to revealing personally identifiable information. The issue has raised deep concerns about the privacy of LLMs. Differential privacy (DP) provides a rigorous framework that allows adding noise in the process of training or fine-tuning LLMs such that extracting the training data becomes infeasible (i.e., with a cryptographically small success probability). While the theoretical privacy guarantees offered in most extant studies assume learning models from scratch through many training iterations in an asymptotic setting, this assumption does not hold in fine-tuning scenarios in which the number of training iterations is significantly smaller. To address the gap, we present \ewtune, a DP framework for fine-tuning LLMs based on Edgeworth accountant with finite-sample privacy guarantees. Our results across four well-established natural language understanding (NLU) tasks show that while \ewtune~adds privacy guarantees to LLM fine-tuning process, it directly contributes to decreasing the induced noise to up to 5.6\% and improves the state-of-the-art LLMs performance by up to 1.1\% across all NLU tasks. We have open-sourced our implementations for wide adoption and public testing purposes.

The scaling of large language models has greatly improved natural language understanding, generation, and reasoning. In this work, we develop a system that trained a trillion-parameter language model on a cluster of Ascend 910 AI processors and MindSpore framework, and present the language model with 1.085T parameters named PanGu-{\Sigma}. With parameter inherent from PanGu-{\alpha}, we extend the dense Transformer model to sparse one with Random Routed Experts (RRE), and efficiently train the model over 329B tokens by using Expert Computation and Storage Separation(ECSS). This resulted in a 6.3x increase in training throughput through heterogeneous computing. Our experimental findings show that PanGu-{\Sigma} provides state-of-the-art performance in zero-shot learning of various Chinese NLP downstream tasks. Moreover, it demonstrates strong abilities when fine-tuned in application data of open-domain dialogue, question answering, machine translation and code generation.

Intent-detection and slot-filling are the two main tasks in natural language understanding. In this study, we propose CTRAN, a novel encoder-decoder CNN-Transformer-based architecture for intent-detection and slot-filling. In the encoder, we use BERT, followed by several convolutional layers, and rearrange the output using window feature sequence. We use stacked Transformer encoders after the window feature sequence. For the intent-detection decoder, we utilize self-attention followed by a linear layer. In the slot-filling decoder, we introduce the aligned Transformer decoder, which utilizes a zero diagonal mask, aligning output tags with input tokens. We apply our network on ATIS and SNIPS, and surpass the current state-of-the-art in slot-filling on both datasets. Furthermore, we incorporate the language model as word embeddings, and show that this strategy yields a better result when compared to the language model as an encoder.

Professors at Utah Tech University are worried about AI technologies being used for classwork. Launched on Nov. 30 by OpenAI, ChatGPT is an artificially intelligent chat box that gives human-like, computer generated responses to any prompt it is given. ChatGPT, Moonbeam and Jasper are just a few websites where members can log in, input a prompt or question, and receive human-like artificially generated speech, marketing messages or even full essays. Professors are concerned that this could lead to cheating that is virtually impossible to detect, as well as a decrease in critical thinking among students. Randy Jasmine and Jim Haendiges, English professors at Utah Tech University, addressed the topic in an episode on their podcast, "Being Human UTU Podcast."

Despite their impressive capabilities, both GPT-4 and GPT-3.5 have limitations. These models can occasionally generate incorrect or nonsensical information, especially when dealing with ambiguous or complex queries. Additionally, both models may produce text that is overly verbose, repeating the same information in different ways. It's important to keep these limitations in mind when using these models for critical applications.

TL;DR: If you have expertise and vision in your field, possess a keen sense of judgment, and embrace AI as a tool to increase productivity, your job should be safe. Like the invention of the camera, an artist with a great mind and taste still thrives. And also as the old saying, you cannot beat the market if you just follow the crowd. Some jobs may become obsolete with the advent of new technologies, however, it's important to remember that these models have limitations and are not capable of all forms of logical deduction or induction. While some researchers are actively working on areas where ChatGPT may fall short, such as reasoning and multimodal capabilities, this shouldn't be the sole reason to feel safe in your job. It's possible that a model could outperform a human in certain areas of reasoning today, and even solve complex problems like the Riemann Hypothesis in the future.

When covering investment and personal finance stories, we aim to inform our readers rather than recommend specific financial product or asset classes. While we may highlight certain positives of a financial product or asset class, there is no guarantee that readers will benefit from the product or investment approach and may, in fact, make a loss if they acquire the product or adopt the approach. To the extent any recommendations or statements of opinion or fact made in a story may constitute financial advice, they constitute general information and not personal financial advice in any form. As such, any recommendations or statements do not take into account the financial circumstances, investment objectives, tax implications, or any specific requirements of readers. Readers of our stories should not act on any recommendation without first taking appropriate steps to verify the information in the stories consulting their independent financial adviser in order to ascertain whether the recommendation (if any) is appropriate, having regard to their investment objectives, financial situation and particular needs.

This week: TikTok challenges Google and Microsoft with search ads, GPT-4 is on the way, and social media engagement rates are dropping. Here's what happened this week in digital marketing. OpenAI hasn't been in the news enough lately so it's time for a fresh update. The next version of GPT, unimaginatively called GPT-4, will go live soon. In fact, it might already be live by the time you read this. As far as the updates that make it more worthwhile than GPT-3, it's got multimodal functionality. That means it supports text, speech, images, and even video. GPT-4 also works across multiple languages. If you've noticed that your social media engagement rates are on the decline, you're not alone.