Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in computational complexity inherent in traditional attention mechanisms presents a prohibitive overhead. Existing approaches either impose strongly biased structures, such as sink or window attention which are task-specific, or radically modify the attention mechanism into linear approximations, whose performance in complex reasoning tasks remains inadequately explored. In this work, we propose a solution that adheres to the "less structure" principle, allowing the model to determine where to attend autonomously, rather than introducing predefined biases. We introduce Mixture of Block Attention (MoBA), an innovative approach that applies the principles of Mixture of Experts (MoE) to the attention mechanism. This novel architecture demonstrates superior performance on long-context tasks while offering a key advantage: the ability to seamlessly transition between full and sparse attention, enhancing efficiency without the risk of compromising performance. MoBA has already been deployed to handle actual production workloads with long-context requirements, demonstrating significant advancements in efficient attention computation for LLMs. Our code is available at https://github.com/MoonshotAI/MoBA.

Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in computational complexity inherent in traditional attention mechanisms presents a prohibitive overhead. Existing approaches either impose strongly biased structures, such as sink or window attention which are task-specific, or radically modify the attention mechanism into linear approximations, whose performance in complex reasoning tasks remains inadequately explored. In this work, we propose a solution that adheres to the ``less structure'' principle, allowing the model to determine where to attend autonomously, rather than introducing predefined biases. We introduce Mixture of Block Attention (MoBA), an innovative approach that applies the principles of Mixture of Experts (MoE) to the attention mechanism. This novel architecture demonstrates superior performance on long-context tasks while offering a key advantage: the ability to seamlessly transition between full and sparse attention, enhancing efficiency without the risk of compromising performance. MoBA has already been deployed to handle actual production workloads with long-context requirements, demonstrating significant advancements in efficient attention computation for LLMs. Our code is available at https://github.com/MoonshotAI/MoBA.

Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI). However, the quadratic increase in computational complexity inherent in traditional attention mechanisms presents a prohibitive overhead. Existing approaches either impose strongly biased structures, such as sink or window attention which are task-specific, or radically modify the attention mechanism into linear approximations, whose performance in complex reasoning tasks remains inadequately explored. In this work, we propose a solution that adheres to the ``less structure'' principle, allowing the model to determine where to attend autonomously, rather than introducing predefined biases. We introduce Mixture of Block Attention (MoBA), an innovative approach that applies the principles of Mixture of Experts (MoE) to the attention mechanism. This novel architecture demonstrates superior performance on long-context tasks while offering a key advantage: the ability to seamlessly transition between full and sparse attention, enhancing efficiency without the risk of compromising performance. MoBA has already been deployed to support Kimi's long-context requests and demonstrates significant advancements in efficient attention computation for LLMs. Our code is available at https://github.com/MoonshotAI/MoBA.

Current mobile assistants are limited by dependence on system APIs or struggle with complex user instructions and diverse interfaces due to restricted comprehension and decision-making abilities. To address these challenges, we propose MobA, a novel Mobile phone Agent powered by multimodal large language models that enhances comprehension and planning capabilities through a sophisticated two-level agent architecture. The high-level Global Agent (GA) is responsible for understanding user commands, tracking history memories, and planning tasks. The low-level Local Agent (LA) predicts detailed actions in the form of function calls, guided by sub-tasks and memory from the GA. Integrating a Reflection Module allows for efficient task completion and enables the system to handle previously unseen complex tasks. MobA demonstrates significant improvements in task execution efficiency and completion rate in real-life evaluations, underscoring the potential of MLLM-empowered mobile assistants.

Riot Games is expanding into every video game genre imaginable. Greg Street, the VP of IP and Entertainment at Riot Games, revealed on Twitter that the company is working on a massively multiplayer online (MMO) game set in the League of Legends universe. "My new job is to kick off a big (some might say massive) game that many of you, and many Rioters, have been asking us to create," he tweeted. When someone asked for confirmation of the genre, Street replied: "It is an MMO." No further details were given.

The e-commerce giant has a foothold in audiobooks, fresh groceries, Netflix-style video streaming and oh-so-much-more. It's no surprise, therefore, that the company wants to widen its influence in the video game industry. The Jeff Bezos empire already owns Twitch, the biggest game live-streaming service, and supports developers with its CryEngine-based Lumberyard platform and AWS server infrastructure. But it's never been a heavyweight game publisher. Multiplayer brawler Breakaway was canceled and The Grand Tour Game was a forgettable TV show tie-in. A lot is riding on Crucible, then.

As hygiene is becoming more important, Moba takes the next step in its hygienic program with the development of a far-reaching new vision crack detection system using Artificial Intelligence. The existing Crack Detector and Shell Strength Detector will become standard features in Moba's Shell Inspector. Shell strength becomes a standard feature of crack detection Moba is committed to adding new and valuable functions to its existing portfolio. For markets where optimising flock efficiency is key, the monitoring and efficient handling of different shell qualities can have a surprisingly positive result for the business. The shell strength detection function is such a valuable addition.

In previous articles, I've looked at a variety of video games that have proven useful test-beds for AI research, with the likes of Ms. Pac-Man, Super Mario Bros. and more recently StarCraft. But in this instance I want to look at a genre that is still relatively new whilst presenting exciting opportunities for AI research: Multiplayer Online Battle Arena's (MOBA). The MOBA genre is undoubtedly one of the most popular in gaming today, but what impact could this have upon AI research? I'm going to provide an overview of MOBA's as a genre, what aspects of their design can prove interesting to AI research and look at some projects that are now bearing fruit both in academia and in corporate research labs. Multiplayer Online Battle Arena's are an offshoot of Real-time Strategy (RTS) games, originating with the Aeon of Strife map for Blizzards StarCraft, followed by the'Defence of the Ancients' mod for WarCraft III: Reign of Chaos and its expansion The Frozen Throne.

AI and Games is a crowdfunded show and needs your support. You can help fund this series on Paypal, KoFi and Patreon (where you can get access to additional content). You can follow AI and Games on Facebook and Twitter: http://www.facebook.com/AIandGames I take a look at the potential MOBAs have to become the next big thing in AI research and some of the work that's already been achieved in academia and corporate R&D. "Varus As We Fall" from the League of Legends Soundtrack "Pre-Game" from the DOTA2 Soundtrack "Legends Never Die" from the League of Legends Soundtrack

PITTSBURGH (November 5, 2018) ... Expectations for artificial intelligences are very real and very high. An analysis in Forbes projects revenues from A.I. will skyrocket from $1.62 billion in 2018 to $31.2 billion in 2025. The report also included a survey revealing 84 percent of enterprises believe investing in A.I. will lead to competitive advantages. "It is exciting to see the tremendous successes and progress made in recent years," says Daniel Jiang, assistant professor of industrial engineering at the University of Pittsburgh Swanson School of Engineering. "To continue this trend, we are looking to develop more sophisticated methods for algorithms to learn strategies for optimal decision making."