Goto

Collaborating Authors

 manga


M2M-Gen: A Multimodal Framework for Automated Background Music Generation in Japanese Manga Using Large Language Models

Sharma, Megha, Haseeb, Muhammad Taimoor, Xia, Gus, Tsuruoka, Yoshimasa

arXiv.org Artificial Intelligence

This paper introduces M2M Gen, a multi modal framework for generating background music tailored to Japanese manga. The key challenges in this task are the lack of an available dataset or a baseline. To address these challenges, we propose an automated music generation pipeline that produces background music for an input manga book. Initially, we use the dialogues in a manga to detect scene boundaries and perform emotion classification using the characters faces within a scene. Then, we use GPT4o to translate this low level scene information into a high level music directive. Conditioned on the scene information and the music directive, another instance of GPT 4o generates page level music captions to guide a text to music model. This produces music that is aligned with the mangas evolving narrative. The effectiveness of M2M Gen is confirmed through extensive subjective evaluations, showcasing its capability to generate higher quality, more relevant and consistent music that complements specific scenes when compared to our baselines.


Ghost in the Shell's rad PS1 soundtrack is finally coming to the West

Engadget

The soundtrack to the spider-bot-crawling 1997 Ghost in the Shell game adaptation is coming to the West for the first time. Titled Ghost in the Shell: Megatech Body (as an ode to the Fuchikoma mech you pilot in the game), the soundtrack was produced by Takkyu Ishino. The PS1 game adaptation had late-90s gamers piloting a spider-like mech (first appearing in the 1991 manga), blasting enemies to smithereens with twin machine guns and guided missiles. Masamune Shirow, the original manga's author, wrote and illustrated its story and art design. But as 90s shooters often figured out, firing guns nonstop for hours on end is much better with a badass techno soundtrack pumping in the background like an energy drink for your ears. In addition to Ishino, it includes "warehouse-shaking bangers" from Mijk Van Dijk, The Advent, Joey Beltram and Brother from Another Planet (among others).


One Piece: From 'niche within a niche' to global phenomenon

BBC News

In the mid-1990s, manga (a term used for a range of Japanese comic books and graphic novels) was at its peak, with 1.34 billion manga collections sold in 1995. Popular titles of the time included Dragon Ball (about a martial artist on the search for magical orbs), Slam Dunk (about a basketball team) and Doraemon (about a time-travelling robotic cat). For Nakono, however, the One Piece comic series changed the industry. "Instead of relying on a haphazard, week-by-week method," he says, "it carefully built up characters, creating a story structure that leads to an emotional climax at the end." "There was a strong emphasis on cliffhangers in manga before One Piece," he continues.

  Country:
  Industry: Leisure & Entertainment > Sports > Basketball (0.60)

Sand Land, a game adaptation of Akira Toriyama's manga, drops on April 26

Engadget

Bandai Namco's Sand Land finally has a release date of April 26. This is a video game adaptation of a classic manga by artist Akira Toriyama. That's the same Akira Toriyama who created Dragon Ball, and also created the character designs for Chrono Trigger and many entries in the Dragon Quest series. Sand Land is a manga dating back to the mystical year of 2000 and it follows the adventures of the literal Devil's son, Beelzebub, as he explores a desert world accompanied by a human sheriff and a demon thief. Interestingly, the game seems like a beat-for-beat recreation of the anime, only in the form of a fast-paced action RPG.


CPST: Comprehension-Preserving Style Transfer for Multi-Modal Narratives

Chen, Yi-Chun, Jhala, Arnav

arXiv.org Artificial Intelligence

We investigate the challenges of style transfer in multi-modal visual narratives. Among static visual narratives such as comics and manga, there are distinct visual styles in terms of presentation. They include style features across multiple dimensions, such as panel layout, size, shape, and color. They include both visual and text media elements. The layout of both text and media elements is also significant in terms of narrative communication. The sequential transitions between panels are where readers make inferences about the narrative world. These feature differences provide an interesting challenge for style transfer in which there are distinctions between the processing of features for each modality. We introduce the notion of comprehension-preserving style transfer (CPST) in such multi-modal domains. CPST requires not only traditional metrics of style transfer but also metrics of narrative comprehension. To spur further research in this area, we present an annotated dataset of comics and manga and an initial set of algorithms that utilize separate style transfer modules for the visual, textual, and layout parameters. To test whether the style transfer preserves narrative semantics, we evaluate this algorithm through visual story cloze tests inspired by work in computational cognition of narrative systems. Understanding the connection between style and narrative semantics provides insight for applications ranging from informational brochure designs to data storytelling.


AI image generator Midjourney bans deepfakes of China's Xi Jinping 'to minimize drama'

FOX News

Midjourney, an AI image generator that creates realistic deepfakes, has been scrutinized recently for having a policy showing deference to China's communist government. The company enforces a rule that users can generate fake images of world leaders from President Biden to Vladimir Putin, but not Chinese President Xi Jinping. In a year-old message on the chat service Discord, the CEO of Midjourney, Inc. explained why the company has that rule. "I think we want to minimize drama," Midjourney CEO David Holz wrote last summer. He explained that the company did not immediately ban images of Xi, but it was triggered by abuse from users.


"Demon Slayer": The Viral Blockbuster from Japan

The New Yorker

One of the seismic cultural shifts of the pandemic era has been a migration into fantasies. Some of them are troubling, such as the conspiratorial prejudice that has fuelled QAnon and the recent surge in violence against those of Asian descent. Others are restorative: the immersive worlds of books, the virtual realities of video games, the hypnotic lull of binge-streamed television series. Many of the escapes that we use to nourish ourselves originated in Japan. The stunning success of Nintendo's Animal Crossing: New Horizons, which sold thirty-one million copies worldwide last year, is a striking example.


New Osamu Tezuka-inspired manga created by AI to be published

The Japan Times

A new manga plotted and designed by artificial intelligence that learned the artistic style of "Astro Boy" manga creator Osamu Tezuka will be published this week, a project sponsor said Wednesday. The manga "Paidon" to be released Thursday in the weekly comic magazine "Morning" was created by AI, which analyzed 65 works by Tezuka, including such classics as "Phoenix" and "Black Jack," according to Kioxia Holdings Corp., a memory chip maker that launched the project. By analyzing Tezuka's works, the AI generated character designs and basic storylines before professional creators added such elements as clothing and dialogue to complete the work. "I always felt sad whenever Osamu Tezuka fans said they could no longer enjoy new works by him. AI creating his new work … that's exactly the kind of (technologically advanced) world depicted in Tezuka's manga," the late author's son and video creator Makoto Tezuka, who contributed to the project, told a news conference in Tokyo.


10 Best Robot Sci-Fi Movies (According To IMDb)

#artificialintelligence

Sci-fi is a large and interesting genre for anyone who gets curious about what the future may hold. From flying cars to dystopian corporations, nothing is outside its range. RELATED: 10 2000s Sci-Fi Masterpieces You've Probably Never Seen There are tons of robot movies, B-grade schlock-fests like Chopping Mall, and big budget productions like Blade Runner: 2049. There's no such thing as an objective film rating, but IMDb is great for getting a consensus from the public. Let's see what they have to tell us about robots!


MANGA: Method Agnostic Neural-policy Generalization and Adaptation

Bharadhwaj, Homanga, Yamaguchi, Shoichiro, Maeda, Shin-ichi

arXiv.org Artificial Intelligence

MANGA: Method Agnostic Neural-policy Generalization and Adaptation Homanga Bharadhwaj 1, Shoichiro Y amaguchi 2, and Shin-ichi Maeda 2 Abstract -- In this paper we target the problem of transferring policies across multiple environments with different dynamics parameters and motor noise variations, by introducing a framework that decouples the processes of policy learning and system identification. Efficiently transferring learned policies to an unknown environment with changes in dynamics configurations in the presence of motor noise is very important for operating robots in the real world, and our work is a novel attempt in that direction. We introduce MANGA: Method Agnostic Neural-policy Generalization and Adaptation, that trains dynamics conditioned policies and efficiently learns to estimate the dynamics parameters of the environment given off-policy state-transition rollouts in the environment. Our scheme is agnostic to the type of training method used - both reinforcement learning (RL) and imitation learning (IL) strategies can be used. We demonstrate the effectiveness of our approach by experimenting with four different MuJoCo agents and comparing against previously proposed transfer baselines. I NTRODUCTION One of the most well recognized goals of robotics research is to develop autonomous agents that can perform a wide variety of tasks in various complex environments. Recently numerous deep reinforcement learning (RL) and imitation learning (IL) based approaches have sought to achieve good performance in complex robotic tasks through minimal supervision. However, a major concern in experimenting with the real environment directly is safety, both of the robot and of the environment. Safety concerns and also the issue of reproducibility has drawn robotics research extensively to simulation environments.