Goto

Collaborating Authors

 tian


StreamForest: Efficient Online Video Understanding with Persistent Event Memory

Neural Information Processing Systems

Multimodal Large Language Models (MLLMs) have recently achieved remarkable progress in video understanding. However, their effectiveness in real-time streaming scenarios remains limited due to storage constraints of historical visual features and insufficient real-time spatiotemporal reasoning. To address these challenges, we propose StreamForest, a novel architecture specifically designed for streaming video understanding. Central to StreamForest is the Persistent Event Memory Forest, a memory mechanism that adaptively organizes video frames into multiple event-level tree structures. This process is guided by penalty functions based on temporal distance, content similarity, and merge frequency, enabling efficient long-term memory retention under limited computational resources.


TheDilemmaofTriHardLossandan Element-WeightedTriHardLossforPerson Re-Identification

Neural Information Processing Systems

Features ofthese characteristics should beclustered between anchors and positive samples while are also utilized to repel between anchors and hard negative samples. It is harmful for learning mutual features within classes.


0c4bc137edaf0eb7f66a87275a8be706-Paper-Conference.pdf

Neural Information Processing Systems

Recent efforts for developing general-purpose estimators with broader coverage, incorporating thefront-door adjustment (FD) (Pearl, 2000) andothers, are not scalable due to the high computational cost of summing over a highdimensional set of variables.


SciCode: A Research Coding Benchmark Curated by Scientists

Neural Information Processing Systems

Since language models (LMs) now outperform average humans on many challenging tasks, it is becoming increasingly difficult to develop challenging, high-quality, and realistic evaluations. We address this by examining LM capabilities to generate code for solving real scientific research problems. Incorporating input from scientists and AI researchers in 16 diverse natural science sub-fields, including mathematics, physics, chemistry, biology, and materials science, we create a scientist-curated coding benchmark, SciCode.


Google is still aiming for its "moonshot" 2030 energy goals

MIT Technology Review

Google is still aiming for its "moonshot" 2030 energy goals The company's electricity demand has doubled since 2020, making its end-of-decade target more of a challenge. Last week, we hosted EmTech MIT, MIT Technology Review's annual flagship conference in Cambridge, Massachusetts. Over the course of three days of main-stage sessions, I learned about innovations in AI, biotech, and robotics. But as you might imagine, some of this climate reporter's favorite moments came in the climate sessions. I was listening especially closely to my colleague James Temple's discussion with Lucia Tian, head of advanced energy technologies at Google. They spoke about the tech giant's growing energy demand and what sort of technologies the company is looking to to help meet it.



Multi-Worker Selection based Distributed Swarm Learning for Edge IoT with Non-i.i.d. Data

arXiv.org Artificial Intelligence

Recent advances in distributed swarm learning (DSL) offer a promising paradigm for edge Internet of Things. Such advancements enhance data privacy, communication efficiency, energy saving, and model scalability. However, the presence of non-independent and identically distributed (non-i.i.d.) data pose a significant challenge for multi-access edge computing, degrading learning performance and diverging training behavior of vanilla DSL. Further, there still lacks theoretical guidance on how data heterogeneity affects model training accuracy, which requires thorough investigation. To fill the gap, this paper first study the data heterogeneity by measuring the impact of non-i.i.d. datasets under the DSL framework. This then motivates a new multi-worker selection design for DSL, termed M-DSL algorithm, which works effectively with distributed heterogeneous data. A new non-i.i.d. degree metric is introduced and defined in this work to formulate the statistical difference among local datasets, which builds a connection between the measure of data heterogeneity and the evaluation of DSL performance. In this way, our M-DSL guides effective selection of multiple works who make prominent contributions for global model updates. We also provide theoretical analysis on the convergence behavior of our M-DSL, followed by extensive experiments on different heterogeneous datasets and non-i.i.d. data settings. Numerical results verify performance improvement and network intelligence enhancement provided by our M-DSL beyond the benchmarks.


SciCode: A Research Coding Benchmark Curated by Scientists

Neural Information Processing Systems

Since language models (LMs) now outperform average humans on many challenging tasks, it is becoming increasingly difficult to develop challenging, high-quality, and realistic evaluations. We address this by examining LM capabilities to generate code for solving real scientific research problems. Incorporating input from scientists and AI researchers in 16 diverse natural science sub-fields, including mathematics, physics, chemistry, biology, and materials science, we create a scientist-curated coding benchmark, SciCode. In total, SciCode contains 338 subproblems decomposed from 80 challenging main problems, and it offers optional descriptions specifying useful scientific background information and scientist-annotated gold-standard solutions and test cases for evaluation. OpenAI o1-preview, the best-performing model among those tested, can solve only 7.7\% of the problems in the most realistic setting.


Review for NeurIPS paper: Counterfactual Data Augmentation using Locally Factored Dynamics

Neural Information Processing Systems

Weaknesses: Theoretical: The authors provide little formal justification for their approach. One of the main contributions seems to be the increase of effective sample size by performing data augmentation. What is unclear is why the increase actually happens. In remark 3.1, the authors attempt to answer the question "How much data can we generate using model-free CoDA?", claiming an exponential increase in data. This fact is not immediately clear.


Medical Manifestation-Aware De-Identification

arXiv.org Artificial Intelligence

Face de-identification (DeID) has been widely studied for common scenes, but remains under-researched for medical scenes, mostly due to the lack of large-scale patient face datasets. In this paper, we release MeMa, consisting of over 40,000 photo-realistic patient faces. MeMa is re-generated from massive real patient photos. By carefully modulating the generation and data-filtering procedures, MeMa avoids breaching real patient privacy, while ensuring rich and plausible medical manifestations. We recruit expert clinicians to annotate MeMa with both coarse- and fine-grained labels, building the first medical-scene DeID benchmark. Additionally, we propose a baseline approach for this new medical-aware DeID task, by integrating data-driven medical semantic priors into the DeID procedure. Despite its conciseness and simplicity, our approach substantially outperforms previous ones. Dataset is available at https://github.com/tianyuan168326/MeMa-Pytorch.