nerv
NeRV: Neural Representations for Videos
We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by $\textbf{25}\times$ to $\textbf{70}\times$, the decoding speed by $\textbf{38}\times$ to $\textbf{132}\times$, while achieving better video quality. With such a representation, we can treat videos as neural networks, simplifying several video-related tasks. For example, conventional video compression methods are restricted by a long and complex pipeline, specifically designed for the task. In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H.264, HEVC \etc). Besides compression, we demonstrate the generalization of NeRV for video denoising.
COLI: A Hierarchical Efficient Compressor for Large Images
Wang, Haoran, Pei, Hanyu, Lyu, Yang, Zhang, Kai, Li, Li, Fan, Feng-Lei
The escalating adoption of high-resolution, large-field-of-view imagery amplifies the need for efficient compression methodologies. Conventional techniques frequently fail to preserve critical image details, while data-driven approaches exhibit limited generalizability. Implicit Neural Representations (INRs) present a promising alternative by learning continuous mappings from spatial coordinates to pixel intensities for individual images, thereby storing network weights rather than raw pixels and avoiding the generalization problem. However, INR-based compression of large images faces challenges including slow compression speed and suboptimal compression ratios. To address these limitations, we introduce COLI (Compressor for Large Images), a novel framework leveraging Neural Representations for Videos (NeRV). First, recognizing that INR-based compression constitutes a training process, we accelerate its convergence through a pretraining-finetuning paradigm, mixed-precision training, and reformulation of the sequential loss into a parallelizable objective. Second, capitalizing on INRs' transformation of image storage constraints into weight storage, we implement Hyper-Compression, a novel post-training technique to substantially enhance compression ratios while maintaining minimal output distortion. Evaluations across two medical imaging datasets demonstrate that COLI consistently achieves competitive or superior PSNR and SSIM metrics at significantly reduced bits per pixel (bpp), while accelerating NeRV training by up to 4 times.
- Information Technology > Sensing and Signal Processing > Image Processing (1.00)
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.67)
NeRV: Neural Representations for Videos Supplementary Material Hao Chen
We provide the architecture details in Table 1. A.3 Implementation Details of Baselines Following prior works, we used ffmpeg [2] to produce the evaluation metrics for H.264 and HEVC. Then we use the following commands to compress videos with H.264 or HEVC codec under medium We also explore NeRV for video temporal interpolation task. We provide denoising results on'ig buck bunny' video in Figure 3. NeRV can reconstruct the original video with high fidelity. We test a smaller model on "Bosphorus" video, and it also has As the most popular media format nowadays, videos are generally viewed as frames of sequences.
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Sensing and Signal Processing > Image Processing (0.94)
- Information Technology > Artificial Intelligence > Representation & Reasoning (0.93)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.69)
NeRV: Neural Representations for Videos
We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by \textbf{25}\times to \textbf{70}\times, the decoding speed by \textbf{38}\times to \textbf{132}\times, while achieving better video quality.
em Evangelion /em 's Final Finale Does What Its Other Endings Couldn't
For being one of the most iconic and influential anime series of all time, Neon Genesis Evangelion is also one of the most confusing; as of the franchise's most recent film, released on Amazon Prime earlier this month, the series has officially ended four times. But the new--and truly final--movie, Evangelion: 3.0 1.0 Thrice Upon a Time, delivers a real capstone to the series, as well as a new argument for how to watch the series as a whole. In case you're totally unfamiliar with the series, the gist is as such: Three teenagers, Shinji Ikari (Megumi Ogata), Asuka Langley Shikinami (Yūko Miyamura), and Rei Ayanami (Megumi Hayashibara), serve as the pilots of giant robots known as Evangelions. Though their initial function was to fight against mysterious beings known as Angels, they know serve as pawns between the organization NERV, led by Shinji's father Gendo (Fumihiko Tachiki), who seeks to cause a mass extinction in order to reunite with his late wife, and WILLE, a group of former NERV employees who are now NERV's only opponents. The Rebuild of Evangelion tetralogy, of which Thrice Upon a Time is the last, serves as a sort of re-telling of the events of the original TV series.
- Leisure & Entertainment (0.71)
- Media > Film (0.51)