Recent advances in deep learning have relied heavily on the use of large Transformers due to their ability to learn at scale. However, the core building block of Transformers, the attention operator, exhibits quadratic cost in sequence length, limiting the amount of context accessible. Existing subquadratic methods based on low-rank and sparse approximations need to be combined with dense attention layers to match Transformers, indicating a gap in capability. In this work, we propose Hyena, a subquadratic drop-in replacement for attention constructed by interleaving implicitly parametrized long convolutions and data-controlled gating. In recall and reasoning tasks on sequences of thousands to hundreds of thousands of tokens, Hyena improves accuracy by more than 50 points over operators relying on state-spaces and other implicit and explicit methods, matching attention-based models. We set a new state-of-the-art for dense-attention-free architectures on language modeling in standard datasets (WikiText103 and The Pile), reaching Transformer quality with a 20% reduction in training compute required at sequence length 2K. Hyena operators are twice as fast as highly optimized attention at sequence length 8K, and 100x faster at sequence length 64K.

Decadal temperature prediction provides crucial information for quantifying the expected effects of future climate changes and thus informs strategic planning and decision-making in various domains. However, such long-term predictions are extremely challenging, due to the chaotic nature of temperature variations. Moreover, the usefulness of existing simulation-based and machine learning-based methods for this task is limited because initial simulation or prediction errors increase exponentially over time. To address this challenging task, we devise a novel prediction method involving an information tracking mechanism that aims to track and adapt to changes in temperature dynamics during the prediction phase by providing probabilistic feedback on the prediction error of the next step based on the current prediction. We integrate this information tracking mechanism, which can be considered as a model calibrator, into the objective function of our method to obtain the corrections needed to avoid error accumulation. Our results show the ability of our method to accurately predict global land-surface temperatures over a decadal range. Furthermore, we demonstrate that our results are meaningful in a real-world context: the temperatures predicted using our method are consistent with and can be used to explain the well-known teleconnections within and between different continents.

Graph representation learning aims to effectively encode high-dimensional sparse graph-structured data into low-dimensional dense vectors, which is a fundamental task that has been widely studied in a range of fields, including machine learning and data mining. Classic graph embedding methods follow the basic idea that the embedding vectors of interconnected nodes in the graph can still maintain a relatively close distance, thereby preserving the structural information between the nodes in the graph. However, this is sub-optimal due to: (i) traditional methods have limited model capacity which limits the learning performance; (ii) existing techniques typically rely on unsupervised learning strategies and fail to couple with the latest learning paradigms; (iii) representation learning and downstream tasks are dependent on each other which should be jointly enhanced. With the remarkable success of deep learning, deep graph representation learning has shown great potential and advantages over shallow (traditional) methods, there exist a large number of deep graph representation learning techniques have been proposed in the past decade, especially graph neural networks. In this survey, we conduct a comprehensive survey on current deep graph representation learning algorithms by proposing a new taxonomy of existing state-of-the-art literature. Specifically, we systematically summarize the essential components of graph representation learning and categorize existing approaches by the ways of graph neural network architectures and the most recent advanced learning paradigms. Moreover, this survey also provides the practical and promising applications of deep graph representation learning. Last but not least, we state new perspectives and suggest challenging directions which deserve further investigations in the future.

Russell Wald, director of the Stanford Institute for Human-Centered AI, sounds off on'The Story.' OpenAI CEO Sam Altman says the age of the giant artificial intelligence model is already over. "I think we're at the end of the era where it's going to be these, like, giant, giant models," he told an audience at the Massachusetts Institute of Technology over Zoom last week. "We'll make them better in other ways." During the same event, Altman also confirmed that his company is not developing Chat GPT-5. "An earlier version of the letter claimed OpenAI is training GPT-5 right now," he said, referencing a letter from billionaire Elon Musk and Apple co-founder Steve Wozniak.

Unlike satellite images, which are typically acquired and processed in near-real-time, global land cover products have historically been produced on an annual basis, often with substantial lag times between image processing and dataset release. We developed a new automated approach for globally consistent, high resolution, near real-time (NRT) land use land cover (LULC) classification leveraging deep learning on 10 m Sentinel-2 imagery. We utilize a highly scalable cloud-based system to apply this approach and provide an open, continuous feed of LULC predictions in parallel with Sentinel-2 acquisitions. This first-of-its-kind NRT product, which we collectively refer to as Dynamic World, accommodates a variety of user needs ranging from extremely up-to-date LULC data to custom global composites representing user-specified date ranges. Furthermore, the continuous nature of the product’s outputs enables refinement, extension, and even redefinition of the LULC classification. In combination, these unique attributes enable unprecedented flexibility for a diverse community of users across a variety of disciplines.

Fox News correspondent Grady Trimble has the latest on fears the technology will spiral out of control on'Special Report.' The European Union (EU) broke with other Western governments as it looks to empower regulators to govern emerging artificial intelligence (AI) technology and convene a summit on subject as fears over employment impacts continue to simmer across Europe. "Recent advances in the field of artificial intelligence have demonstrated that the speed of technological progress is faster and more unpredictable than policy-makers around the world have anticipated," a group of European lawmakers wrote in an open letter Monday. "We are moving very fast." The group released the letter almost in direct response to the Future of Life Institute's open letter, signed by experts and leaders including Elon Musk, Apple co-founder Steve Wozniak and former presidential candidate Andrew Yang, that called for a six-month pause on the development of "powerful" AI systems.

Originally published on the Distant Whispers blog. From the coverage that ChatGPT, developed by OpenAI, has been receiving since its launch in November 2022, you would be forgiven for thinking that is the only technology story around. And it deserves the spotlight. Few had expected the jaw-dropping rapid strides that this technology has made in the last few years, and it will continue to wow us this year. It has opened a bottle, and a genie with unsurpassed powers has emerged.

Fox News host Tucker Carlson provides insight on the consequences of the development of artificial intelligence and why he spoke to Elon Musk on'Tucker Carlson Tonight.' TUCKER CARLSON – Is artificial intelligence dangerous to humanity?.… Continue reading… TROUBLED'TIMES' – New York Times staffers are whining again and you won't believe why… Continue reading… GREGG JARRETT – Latest Hunter bombshell: Here's the kind of access his partners had to VP Joe Biden… Continue reading… WOKE STATE DEPARTMENT – America's State Department was seized by one political party. DAVE RAMSEY – I've helped Americans with money problems for decades. Parents should not pay their adult children's bills… Continue reading… NO MORE IRS – Hate Tax Day? Here's one way the IRS won't bother us ever again… Continue reading… TAIWAN IN A TIGHT SPOT – Here's why Biden needs to clarify his Taiwan policy ASAP… Continue reading… VIDEO OF THE DAY – Fox News host Sean Hannity says this week's New York City crime hearing revealed the ugly truth about House Democrats… Watch now… DR. MARC SIEGEL – There is a huge red flag in the rush to use ChatGPT in your doctor's office… Continue reading… CARTOON OF THE DAY – Will He or Won't He? Check out all of our political cartoons...

The Israeli author and historian said a lack of safety measures in new AI tech could cause the West to lose to China. Israeli historian and "Sapiens" author Yuval Noah Harari claimed there is no kill switch for artificial intelligence (AI) and urged for the implementation of safety checks and guardrails, or else risk the possibility of societal collapse. During a March interview with ABC News, OpenAI CEO Sam Altman was asked if ChatGPT had a "kill switch" in the event their AI went rogue. Altman's responded with a quick "yes." "What really happens is that any engineer can just say we're going to disable this for now. Or we're going to deploy this new version of the model," he added.

Scene: A U.S. adversary is at work on a new type of drone, ship, or aircraft and it's your job to find it, wherever it is. Not long ago, that task would take a massive effort of human, signals, and open-source intelligence collection. But a researcher from AI company Synthetaic has created a tool that will allow users to find virtually any large object that exists in any satellite photo of the Earth within just one day. It's also the sort of capability the National Geospatial-Intelligence Agency is also looking to develop, and it could radically shift strategic advantage on the battlefield. Corey Jaskolski, founder and CEO of Synthetaic, dubbed his satellite image scanning tool Rapid Automatic Image Categorization, or RAIC.