Sequence-to-sequence tasks often benefit from long contexts, but the quadratic complexity of self-attention in standard Transformers renders this non-trivial. During generation, temporary representations -stored in the so-called KV cache-account for a large portion of GPU memory usage and scale linearly with context length. We introduce KV-Distill, a Transformer compression framework that distills long context KV caches into significantly shorter representations in a question-independent fashion. KV-Distill can be trained as a parameter-efficient adaptor for pretrained models, and enables the compression of arbitrary spans of a context while preserving pre-trained model capabilities. We treat a compressed-uncompressed cache as a student-teacher pairing and apply a KL-type divergence to match the generated outputs. KV-Distill outperforms other compression techniques in worst-case extractive tasks and approaches uncompressed performance in long context question answering and summarization, and it can be fine-tuned on domain-specific contexts to reduce lengths by up to 99% while preserving downstream performance. We demonstrate the generalizability of KV-Distill across various model sizes and architectures.

Prediction of dynamic environmental variables in unmonitored sites remains a long-standing challenge for water resources science. The majority of the world's freshwater resources have inadequate monitoring of critical environmental variables needed for management. Yet, the need to have widespread predictions of hydrological variables such as river flow and water quality has become increasingly urgent due to climate and land use change over the past decades, and their associated impacts on water resources. Modern machine learning methods increasingly outperform their process-based and empirical model counterparts for hydrologic time series prediction with their ability to extract information from large, diverse data sets. We review relevant state-of-the art applications of machine learning for streamflow, water quality, and other water resources prediction and discuss opportunities to improve the use of machine learning with emerging methods for incorporating watershed characteristics into deep learning models, transfer learning, and incorporating process knowledge into machine learning models. The analysis here suggests most prior efforts have been focused on deep learning learning frameworks built on many sites for predictions at daily time scales in the United States, but that comparisons between different classes of machine learning methods are few and inadequate. We identify several open questions for time series predictions in unmonitored sites that include incorporating dynamic inputs and site characteristics, mechanistic understanding and spatial context, and explainable AI techniques in modern machine learning frameworks.

Aspect or query-based summarization has recently caught more attention, as it can generate differentiated summaries based on users' interests. However, the current dataset for aspect or query-based summarization either focuses on specific domains, contains relatively small-scale instances, or includes only a few aspect types. Such limitations hinder further explorations in this direction. In this work, we take advantage of crowd-sourcing knowledge on Wikipedia.org and automatically create a high-quality, large-scale open-domain aspect-based summarization dataset named OASum, which contains more than 3.7 million instances with around 1 million different aspects on 2 million Wikipedia pages. We provide benchmark results on OASum and demonstrate its ability for diverse aspect-based summarization generation. To overcome the data scarcity problem on specific domains, we also perform zero-shot, few-shot, and fine-tuning on seven downstream datasets. Specifically, zero/few-shot and fine-tuning results show that the model pre-trained on our corpus demonstrates a strong aspect or query-focused generation ability compared with the backbone model. Our dataset and pre-trained checkpoints are publicly available.

There's nothing magical about artificial intelligence, it's simply code designed by fallible humans using fallible data. The magic comes from the humans working with or seeing the benefits of AI. So the questions are: are we expecting too much from AI? Too what extent should companies and their executives rely on the output delivered by AI? This was the subject of debate at a panel hosted at AI Summit in New York, held in early December, focusing on risks in the emerging role of AI in the financial services sector, but the discussion had wide-ranging implications across all industries. "We think AI is telling us something, but it's not," cautioned Rod Butters, chief technology officer for Aible.

There's nothing magical about artificial intelligence, it's simply code designed by fallible humans using fallible data. The magic comes from the humans working with or seeing the benefits of AI. So the questions are: are we expecting too much from AI? Too what extent should companies and their executives rely on the output delivered by AI? This was the subject of debate at a panel hosted at AI Summit in New York, held in early December, which focused on risks in the emerging role of AI in the financial services sector, but the discussion had wide-ranging implications across all industries. "We think AI is telling us something, but it's not," cautioned Rod Butters, chief technology officer for Aible.

Palo Alto-based Cobalt Robotics Inc. today introduced a new line of robot security guards for indoor use. The roving robots use the same kind of components you'd expect in a self-driving car to sense people and problems in a building. Cofounders Travis Deyle and Erik Schluntz, who are former GoogleX and SpaceX engineers, say they designed the robots to complement, not replace, human security guards. Altogether, Cobalt's robo-guards pack 60 sensors, including lidar, ultrasound, depth sensors and cameras, as well as wide angle day and night cameras to detect people around them. They also include mics and two-way video chat screens that allow human security guards or building managers to remotely interact with a person who the robot approaches.

Here's the weirdest bit of gaming news I've seen in a while: Francis Ford Coppola's film studio American Zoetrope is making a video game. Weirder: It's a video game based on Coppola's classic 1979 film Apocalypse Now. "Forty years ago, I set out to make a personal art picture that could hopefully influence generations of viewers for years to come. Today, I'm joined by new daredevils, a team who want to make an interactive version of Apocalypse Now, where you are Captain Benjamin Willard amidst the harsh backdrop of the Vietnam War. I've been watching videogames grow into a meaningful way to tell stories, and I'm excited to explore the possibilities for Apocalypse Now for a new platform and a new generation."

Apocalypse Now is one of the greatest films ever made, and now it's going to be a video game. The game's creators -- including filmmaker Francis Ford Coppola -- are seeking $900,000 on Kickstarter to get development of the game moving. The plan is for a 2020 release. SEE ALSO: 3.5 million people supported Kickstarter projects in 2016 "Forty years ago, I set out to make a personal art picture that could hopefully influence generations of viewers for years to come," Coppola -- who appears to be serving in a creative advisory role -- said in a statement. "Today, I'm joined by new daredevils, a team who wants to make an interactive version of Apocalypse Now, where you are Captain Benjamin Willard amidst the harsh backdrop of the Vietnam War. I've been watching videogames grow into a meaningful way to tell stories, and I'm excited to explore the possibilities for Apocalypse Now for a new platform and a new generation."

A thunderstorm is considered severe if any one of the following phenomena accompanies the thunderstorm (and is reported): - tornadoes (intense, small-scale cyclones); - hailstones cm (in.) in diameter; - surface wind gusts in excess of 93 km h-1 (50 knots) and/or significant wind damage.