jelly
JELLY: Joint Emotion Recognition and Context Reasoning with LLMs for Conversational Speech Synthesis
Cha, Jun-Hyeok, Kim, Seung-Bin, Oh, Hyung-Seok, Lee, Seong-Whan
Recently, there has been a growing demand for conversational speech synthesis (CSS) that generates more natural speech by considering the conversational context. To address this, we introduce JELLY, a novel CSS framework that integrates emotion recognition and context reasoning for generating appropriate speech in conversation by fine-tuning a large language model (LLM) with multiple partial LoRA modules. We propose an Emotion-aware Q-former encoder, which enables the LLM to perceive emotions in speech. The encoder is trained to align speech emotions with text, utilizing datasets of emotional speech. The entire model is then fine-tuned with conversational speech data to infer emotional context for generating emotionally appropriate speech in conversation. Our experimental results demonstrate that JELLY excels in emotional context modeling, synthesizing speech that naturally aligns with conversation, while mitigating the scarcity of emotional conversational speech datasets.
- Asia > South Korea > Seoul > Seoul (0.05)
- North America > Canada > Ontario > Toronto (0.04)
The real-life Flubber? Glob of jelly can play Pong thanks to a basic kind of memory, bizarre study reveals
In the 1997 Robin Williams flick Flubber, an absent-minded professor creates a sentient ball of goo with incredible capabilities. Now, more than 25 years later, scientists have made a surprising discovery that could bring Flubber into the real world. Researchers from the University of Reading have created a non-living'hydrogel brain' which is capable of playing the video game Pong. Using a plate of electrodes hooked up to the classic game, the water-based jelly even managed to get 10 per cent better as it practised. While it might not be quite as bouncy as Robin Williams' invention, the researchers believe this breakthrough could change the future of artificial intelligence.
A glob of jelly can play Pong thanks to a basic kind of memory
An inanimate glob of ion-laced jelly can play the computer game Pong and even improve over time. Researchers plan further experiments to explore whether it can handle more complex computations and hope it could eventually be used to control robots. Inspired by previous research that used brain cells in a dish to play Pong, Vincent Strong and his colleagues at the University of Reading, UK, decided to try playing the tennis-like game with an even simpler material. They took a polymer material containing water and laced it with ions to make it responsive to electrical stimuli. When electricity is passed through the material, those ions move to the source of the current, dragging water with them and causing the gel to swell.
Materials: 'Super jelly' made from 80 per cent water can survive being run over by a CAR
No, it's'super jelly' -- a bizarre new material that can survive being run over by a car even though it's composed of 80 per cent water. The'glass-like hydrogel' may look and feel like a squishy jelly, but when compressed it acts like shatterproof glass, its University of Cambridge developers said. It is formed using a network of polymers held together by a series of reversible chemical interactions that can be tailored to control the gel's mechanical properties. This is the first time that a soft material has been produced that is capable of such significant resistance to compressive forces. Super jelly could find various applications, the team added, from use for building soft robotics and bioelectronics through to replacement for damaged cartilage.
OutRanked -- The Biz Stone Collection
At a tech conference recently, I was only partly joking when I said, "Everyone is working on Artificial Intelligence, what about just, Intelligence?" The true promise for the future of technology is for it to find a way to work with humanity such that the two are made better. Ideally, our best traits are amplified by technology. The best of technology is unlocked when humanity is woven into its DNA. I worked at Google when it was still a "startup," before it became a public company.