Goto

Collaborating Authors

 Media


Digital tech can offer rich opportunities for child development, study says

The Guardian

Although it has been argued that under-threes should not have any screen time at all, research has found that digital tech can offer "rich opportunities" for young children's development. A two-year study, Toddlers, Tech and Talk, funded by the Economic and Social Research Council and led by researchers from Manchester Metropolitan University (MMU), working with Lancaster, Queen's Belfast, Strathclyde and Swansea universities, looked at children's interactions with everything from Amazon Alexa to Ring doorbells, in diverse communities across the UK, to find out how tech was influencing 0- to three-year-olds' early talk and literacy. It examined how children use technology with parents or by themselves, whether taking photos and videos, using learning apps and playing games, listening and singing to songs, talking about favourite characters, or chatting on video calls. The researchers found that children were not only interacting with smart devices and appliances when very young, but also that digital tech could have benefits for language development and other skills. "The evidence generated through this study suggests that young children's digital activity often involves sensory exploration through touch, vision, hearing, movement and embodied cognition," the report said.


Urgent warning to iPhone users over troubling 'glitch' that exposes browser history when you hold your device a certain way

Daily Mail - Science & tech

An embarrassing glitch has reemerged on iPhone, judging from complaints online -- a bug that dredges up old web searches many would like to keep private. Some Apple users are discovering that their devices are displaying old adult content at unwanted moments, despite their best efforts to clear their browser history, tabs and cookies, even after making sure to view risqué content in'incognito mode' only. 'I have a fairly new iPhone and this is happening,' one Apple fan confessed. 'Old porn vid stuck in Firefox preview mode, only happens in landscape [mode] though.' Another user told horror stories of their iPhone displaying old adult content while'people have been looking over my shoulder as I am showing them something.'


The rise of AI: When will Congress regulate it?

FOX News

Fox News chief political anchor Bret Baier has the latest on the pros and cons of the bombshell developments on'Special Report.' It is said that predicting the future isn't magic. If that's the case, perhaps we should ask AI when Congress might pass a bill to regulate the emerging technology – before it spirals out of control. There's a push by Congressional leaders to approve a bill regulating AI when lawmakers return to Washington after the election. But the path to passage - and developing a consensus on establishing guardrails for AI - is far from certain.


AI Horizon Scanning -- White Paper p3395, IEEE-SA. Part III: Technology Watch: a selection of key developments, emerging technologies, and industry trends in Artificial Intelligence

arXiv.org Artificial Intelligence

Generative Artificial Intelligence (AI) technologies are in a phase of unprecedented rapid development following the landmark release of Chat-GPT, which brought the phenomenon to wide public attention. As the deployment of AI products rises geometrically, considerable attention is being given to the threats and opportunities that AI technologies offer, and to the need for regulatory and standards initiatives to ensure that use of the technology aligns with societal needs and generates broad benefits while mitigating risks and threats. This manuscript is the third of a series of White Papers informing the development of IEEE-SA's p3995 {\it `Standard for the Implementation of Safeguards, Controls, and Preventive Techniques for Artificial Intelligence Models'} \cite{P3395}, Chair Marina Cort\^{e}s. This part focuses on assessing calmly and objectively, as far as is possible, the current state of Artificial Intelligence (AI) technology development and identifying predominant trends, prospects, and ensuing risks. It necessarily forms a snapshot of the current instant of a rapidly-evolving landscape, with new products and innovations emerging continuously. While our main focus is on software and hardware developments and their corporate context, we also briefly review progress on robotics within the AI context and describe some implications of the substantial and growing AI energy demand.


Personalized Video Summarization by Multimodal Video Understanding

arXiv.org Artificial Intelligence

Video summarization techniques have been proven to improve the overall user experience when it comes to accessing and comprehending video content. If the user's preference is known, video summarization can identify significant information or relevant content from an input video, aiding them in obtaining the necessary information or determining their interest in watching the original video. Adapting video summarization to various types of video and user preferences requires significant training data and expensive human labeling. To facilitate such research, we proposed a new benchmark for video summarization that captures various user preferences. Also, we present a pipeline called Video Summarization with Language (VSL) for user-preferred video summarization that is based on pre-trained visual language models (VLMs) to avoid the need to train a video summarization system on a large training dataset. The pipeline takes both video and closed captioning as input and performs semantic analysis at the scene level by converting video frames into text. Subsequently, the user's genre preference was used as the basis for selecting the pertinent textual scenes. The experimental results demonstrate that our proposed pipeline outperforms current state-of-the-art unsupervised video summarization models. We show that our method is more adaptable across different datasets compared to supervised query-based video summarization models. In the end, the runtime analysis demonstrates that our pipeline is more suitable for practical use when scaling up the number of user preferences and videos.


StreamingBench: Assessing the Gap for MLLMs to Achieve Streaming Video Understanding

arXiv.org Artificial Intelligence

The rapid development of Multimodal Large Language Models (MLLMs) has expanded their capabilities from image comprehension to video understanding. However, most of these MLLMs focus primarily on offline video comprehension, necessitating extensive processing of all video frames before any queries can be made. This presents a significant gap compared to the human ability to watch, listen, think, and respond to streaming inputs in real time, highlighting the limitations of current MLLMs. In this paper, we introduce StreamingBench, the first comprehensive benchmark designed to evaluate the streaming video understanding capabilities of MLLMs. The benchmark consists of 18 tasks, featuring 900 videos and 4,500 human-curated QA pairs. Each video features five questions presented at different time points to simulate a continuous streaming scenario. We conduct experiments on StreamingBench with 13 open-source and proprietary MLLMs and find that even the most advanced proprietary MLLMs like Gemini 1.5 Pro and GPT-4o perform significantly below human-level streaming video understanding capabilities. We hope our work can facilitate further advancements for MLLMs, empowering them to approach human-level video comprehension and interaction in more realistic scenarios. The rapid evolution of Multimodal Large Language Models (MLLMs) has significantly reshaped the field of Artificial Intelligence (Yang et al., 2023; Reid et al., 2024; Liu et al., 2024c;a). Current advanced MLLMs (Reid et al., 2024; Wang et al., 2024a; Yao et al., 2024) have already demonstrated exceptional performance in video understanding tasks, excelling on existing video benchmarks (Fu et al., 2024; Wang et al., 2024b; Zhou et al., 2024; Ataallah et al., 2024). Moreover, several pioneering studies (Chen et al., 2024a; Zhang et al., 2024a; Wu et al., 2024) have focused on improving the ability of MLLMs to comprehend real-time online video streams, pushing the boundaries of their applicability and efficiency in dynamic environments. In the industry, streaming video understanding has also attracted significant attention, with OpenAI's GPT-4o (OpenAI, 2024) as a prominent example that demonstrates human-like perception and understanding of streaming inputs. Despite the recognition of the importance of streaming video understanding for MLLMs, most existing video understanding benchmarks (Fu et al., 2024; Wang et al., 2024b; Zhou et al., 2024) are In offline video benchmarks, questions are designed based on the entire video being visible. In contrast, StreamingBench presents questions at specific moments, with three main task categories specifically designed to evaluate fundamental capabilities in streaming video understanding.


Long Context RAG Performance of Large Language Models

arXiv.org Artificial Intelligence

Retrieval Augmented Generation (RAG) has emerged as a crucial technique for enhancing the accuracy of Large Language Models (LLMs) by incorporating external information. With the advent of LLMs that support increasingly longer context lengths, there is a growing interest in understanding how these models perform in RAG scenarios. Can these new long context models improve RAG performance? This paper presents a comprehensive study of the impact of increased context length on RAG performance across 20 popular open source and commercial LLMs. We ran RAG workflows while varying the total context length from 2,000 to 128,000 tokens (and 2 million tokens when possible) on three domain-specific datasets, and report key insights on the benefits and limitations of long context in RAG applications. Our findings reveal that while retrieving more documents can improve performance, only a handful of the most recent state of the art LLMs can maintain consistent accuracy at long context above 64k tokens. We also identify distinct failure modes in long context scenarios, suggesting areas for future research.


Growing a Tail: Increasing Output Diversity in Large Language Models

arXiv.org Artificial Intelligence

For large groups, use the name of the group or consortium and include a full list of the authors and affiliations at the end of the main manuscript or in the Supplementary Materials. Abstract: How diverse are the outputs of large language models when diversity is desired? We examine the diversity of responses of various models to questions with multiple possible answers, comparing them with human responses. Our findings suggest that models' outputs are highly concentrated, reflecting a narrow, mainstream'worldview', in comparison to humans, whose responses exhibit a much longer-tail. We examine three ways to increase models' output diversity: 1) increasing generation randomness via temperature sampling; 2) prompting models to answer from diverse perspectives; 3) aggregating outputs from several models. A combination of these measures significantly increases models' output diversity, reaching that of humans. We discuss implications of these findings for AI policy that wishes to preserve cultural diversity, an essential building block of a democratic social fabric. Conversely, a lack of diversity can result in extremism and exclusion (e.g., 1, 2).


Continual Audio-Visual Sound Separation

arXiv.org Artificial Intelligence

In this paper, we introduce a novel continual audio-visual sound separation task, aiming to continuously separate sound sources for new classes while preserving performance on previously learned classes, with the aid of visual guidance. This problem is crucial for practical visually guided auditory perception as it can significantly enhance the adaptability and robustness of audio-visual sound separation models, making them more applicable for real-world scenarios where encountering new sound sources is commonplace. The task is inherently challenging as our models must not only effectively utilize information from both modalities in current tasks but also preserve their cross-modal association in old tasks to mitigate catastrophic forgetting during audio-visual continual learning. To address these challenges, we propose a novel approach named ContAV-Sep (Continual Audio-Visual Sound Separation). ContAV-Sep presents a novel Cross-modal Similarity Distillation Constraint (CrossSDC) to uphold the cross-modal semantic similarity through incremental tasks and retain previously acquired knowledge of semantic similarity in old models, mitigating the risk of catastrophic forgetting. The CrossSDC can seamlessly integrate into the training process of different audio-visual sound separation frameworks. Experiments demonstrate that ContAV-Sep can effectively mitigate catastrophic forgetting and achieve significantly better performance compared to other continual learning baselines for audio-visual sound separation.


GenXD: Generating Any 3D and 4D Scenes

arXiv.org Artificial Intelligence

Figure 1: GenX D is a unified model for high-quality 3D and 4D generation from any number of condition images. By controlling the motion strength and condition masks, GenX D can support various application without any modification. The condition images are shown with star icon and the time dimension is illustrated with dash line. Recent developments in 2D visual generation have been remarkably successful. However, 3D and 4D generation remain challenging in real-world applications due to the lack of large-scale 4D data and effective model design. In this paper, we propose to jointly investigate general 3D and 4D generation by leveraging camera and object movements commonly observed in daily life. Due to the lack of real-world 4D data in the community, we first propose a data curation pipeline to obtain camera poses and object motion strength from videos. Based on this pipeline, we introduce a large-scale real-world 4D scene dataset: CamVid-30K. By leveraging all the 3D and 4D data, we develop our framework, GenX D, which allows us to produce any 3D or 4D scene. We propose multiview-temporal modules, which disentangle camera and object movements, to seamlessly learn from both 3D and 4D data. Additionally, GenX D employs masked latent conditions to support a variety of conditioning views. We perform extensive evaluations across various real-world and synthetic datasets, demonstrating GenX D's effectiveness and versatility compared to previous methods in 3D and 4D generation. The dataset and code will be made publicly available.