Large Language Models (LLMs) have demonstrated remarkable performance across diverse tasks but are constrained by their small context window sizes. Various efforts have been proposed to expand the context window to accommodate even up to 200K input tokens. Meanwhile, building high-quality benchmarks with much longer text lengths and more demanding tasks to provide comprehensive evaluations is of immense practical interest to facilitate long context understanding research of LLMs. However, prior benchmarks create datasets that ostensibly cater to long-text comprehension by expanding the input of traditional tasks, which falls short to exhibit the unique characteristics of long-text understanding, including long dependency tasks and longer text length compatible with modern LLMs' context window size. In this paper, we introduce a benchmark for extremely long context understanding with long-range dependencies, XL$^2$Bench, which includes three scenarios: Fiction Reading, Paper Reading, and Law Reading, and four tasks of increasing complexity: Memory Retrieval, Detailed Understanding, Overall Understanding, and Open-ended Generation, covering 27 subtasks in English and Chinese. It has an average length of 100K+ words (English) and 200K+ characters (Chinese). Evaluating six leading LLMs on XL$^2$Bench, we find that their performance significantly lags behind human levels. Moreover, the observed decline in performance across both the original and enhanced datasets underscores the efficacy of our approach to mitigating data contamination.

The Space-Air-Ground-Sea integrated network calls for more robust and secure transmission techniques against jamming. In this paper, we propose a textual semantic transmission framework for robust transmission, which utilizes the advanced natural language processing techniques to model and encode sentences. Specifically, the textual sentences are firstly split into tokens using wordpiece algorithm, and are embedded to token vectors for semantic extraction by Transformer-based encoder. The encoded data are quantized to a fixed length binary sequence for transmission, where binary erasure, symmetric, and deletion channels are considered for transmission. The received binary sequences are further decoded by the transformer decoders into tokens used for sentence reconstruction. Our proposed approach leverages the power of neural networks and attention mechanism to provide reliable and efficient communication of textual data in challenging wireless environments, and simulation results on semantic similarity and bilingual evaluation understudy prove the superiority of the proposed model in semantic transmission.

I daily get requests (since not everybody still has DALL·E access) to create images with specific prompts. Q. Can DALL-E create an illustration to a text if you input this text as PROMPT? But things are more complicated than they seem. Sure, DALL·E is Transformer-driven, so every single part of a prompt you put in will be taken with focussed attention by the model to create a coherent image with inner logic. Sometimes you get some unusual pictures.

Just like me," says the frail man. His companion nods its head gently in agreement. The man smiles at it and turns his gaze back to the glowing horizon. The Artificial Intelligence Manifestation, known affectionately as "Aimy" to its human creators, is housed in a human-like body with head, arms and legs. It sits next to the bed of the ailing man as they look through an enormous window. Machines softly hum around them; small ultra-sleek tubes connect the life-saving medical device to the man's body. The machine has been keeping him alive for months as his age finally catches up with him. They gaze at the sunset. Soon the old man's breathing becomes labored and he coughs violently for a moment. Aimy leans in to help him through the coughing spell. Aimy's white metal body is fitted with heating elements that make it feel warm to the touch, like a human body. It props up Jason as his coughing continues. Aimy carefully moves the blanket a little, unbunching it from under Jason as he shifts uncomfortably. Finally his body stops wracking and he relaxes back into the bed. Aimy reaches over to adjust his pillow. They resume the silence for a while. Finally, it asks a question. "Jason," says Aimy in its soft voice. "Are you frightened of death?" it asks. Do you feel any differently now that it is about to actually happen?"

Growing up, I always saw playing video games as a natural extension of my interest in reading. To me, the fantastical worlds I explored in games mirrored those of my favorite children's books like Where the Wild Things Are and The Lorax. Many of the games I played and the stories I read shared a similar sense of whimsy and adventure, and piqued my interest with intriguing art styles. And that makes sense, given that some video games evoke the feeling of reading a great piece of children's literature. This is especially true for the point-and-click genre (named after the way you play), which can make you feel like you're turning the pages in a book as you progress from scene to scene, moving your character across a static, 2D illustrated background.

As we know, 'Data is the new oil.' It means that just like oil. If one knows the value of data, we can learn to extract and use, it can solve many problems. Now, data can be explained by two things, Model and Error. In this article, we are going to dive into the linear model.

Many tasks aim to measure machine reading comprehension (MRC), often focusing on question types presumed to be difficult. Rarely, however, do task designers start by considering what systems should in fact comprehend. In this paper we make two key contributions. First, we argue that existing approaches do not adequately define comprehension; they are too unsystematic about what content is tested. Second, we present a detailed definition of comprehension -- a "Template of Understanding" -- for a widely useful class of texts, namely short narratives. We then conduct an experiment that strongly suggests existing systems are not up to the task of narrative understanding as we define it.

Welcome back to Gaming IRL, a monthly segment where several editors talk about what they've been playing in their downtime. Gaming IRL is part of a broader series in which you'll find stories from all of the areas we cover: gadgets we use every day, the apps and services we adore, what we're watching and the music and podcasts we can't live without. Today is all about gaming. E3 is done and dusted for another year, but every year there are dozens of great games released, all of which are available right now. Fittingly, our picks this month range from a 1997 sim all the way up to a game that was released just today.

A futuristic novel, "2084" centers on a self-driving car named Winston that lives in a world where humans have reversed global warming. His owner is a math teacher named Julia. Children from every state are invited to participate in a national art competition as a result of the incredibly well-endowed federal arts budget. The winners travel to Washington, D.C., and meet the President of the United States, who is a very kind woman and also a doctor. Written in Seussian rhyme, this children's book is filled with many cute animals that live wherever they like, not necessarily on a farm!

Hearing that Bob Dylan just won the Nobel Prize for Literature, how could I not begin this talk with his famous line, "Something is happening here, but you don't know what it is, do you, Mr. Jones?" The future is full of amazing things. On my way here, I spoke out loud to a $200 device in my kitchen, and asked it to call a Lyft to take me to the airport. And in a few years, that car might well be driving itself. Someone seeing this for the first time would have every excuse to say "WTF?"