As a dominant paradigm, fine-tuning a pre-trained model on the target data is widely used in many deep learning applications, especially for small data sets. However, recent studies have empirically shown that training from scratch has the final performance that is no worse than this pre-training strategy once the number of training samples is increased in some vision tasks. In this work, we revisit this phenomenon from the perspective of generalization analysis by using excess risk bound which is popular in learning theory. The result reveals that the excess risk bound may have a weak dependency on the pre-trained model. The observation inspires us to leverage pre-training data for fine-tuning, since this data is also available for fine-tuning.

We seem to have more video footage and still images than ever before, thanks to smartphones, GoPro cameras and the backlog of older ones collected across a lifetime. Managing all these formats, as well as making sure they look their best, can be a frightening proposition. Thankfully, Winxvideo AI is a powerful all-in-one solution that not only uses advanced Artificial Intelligence software to upgrade the quality of your content but can rescue old photos and footage too. The newly updated version 4.0 also brings huge improvements to speed, plus a special price offer, so you can save both time and money while you upgrade your photo and video library. Winxvideo AI comes with an impressive array of features that can turn tired, old, blurry videos into something far more professional.

We propose a model-based lifelong reinforcement-learning approach that estimates a hierarchical Bayesian posterior distilling the common structure shared across different tasks. The learned posterior combined with a sample-based Bayesian exploration procedure increases the sample efficiency of learning across a family of related tasks. We first derive an analysis of the relationship between the sample complexity and the initialization quality of the posterior in the finite MDP setting. We next scale the approach to continuous-state domains by introducing a Variational Bayesian Lifelong Reinforcement Learning algorithm that can be combined with recent model-based deep RL methods, and that exhibits backward transfer. Experimental results on several challenging domains show that our algorithms achieve both better forward and backward transfer performance than state-of-the-art lifelong RL methods.

We present a simple, yet powerful data-augmentation technique to enable data-efficient learning from parametric experts for reinforcement and imitation learning. We focus on what we call the policy cloning setting, in which we use online or offline queries of an expert or expert policy to inform the behavior of a student policy. This setting arises naturally in a number of problems, for instance as variants of behavior cloning, or as a component of other algorithms such as DAGGER, policy distillation or KL-regularized RL. Our approach, augmented policy cloning (APC), uses synthetic states to induce feedback-sensitivity in a region around sampled trajectories, thus dramatically reducing the environment interactions required for successful cloning of the expert. We achieve highly data-efficient transfer of behavior from an expert to a student policy for high-degrees-of-freedom control problems.

While there has been progress in developing non-vacuous generalization bounds for deep neural networks, these bounds tend to be uninformative about why deep learning works. In this paper, we develop a compression approach based on quantizing neural network parameters in a linear subspace, profoundly improving on previous results to provide state-of-the-art generalization bounds on a variety of tasks, including transfer learning. We use these tight bounds to better understand the role of model size, equivariance, and the implicit biases of optimization, for generalization in deep learning. Notably, we find large models can be compressed to a much greater extent than previously known, encapsulating Occam's razor.

While language tasks are naturally expressed in a single, unified, modeling framework, i.e., generating sequences of tokens, this has not been the case in computer vision. As a result, there is a proliferation of distinct architectures and loss functions for different vision tasks. In this work we show that a diverse set of "core" computer vision tasks can also be unified if formulated in terms of a shared pixel-to-sequence interface. We focus on four tasks, namely, object detection, instance segmentation, keypoint detection, and image captioning, all with diverse types of outputs, e.g., bounding boxes or dense masks. Despite that, by formulating the output of each task as a sequence of discrete tokens with a unified interface, we show that one can train a neural network with a single model architecture and loss function on all these tasks, with no task-specific customization.

And in so many ways, it kinda sucks. A new graphics card costs more than a mortgage payment because billionaires are sucking up all the GPUs to boil the planet and make Hayao Miyazaki cry at the same time, and I still don't have a Marty McFly hoverboard. But at least I can order fast food that literally flies to my door. In fact, I could order a flying curry delivery if I lived in Charlotte, North Carolina--specifically, within four miles of the Arboretum Shopping Center--where DoorDash is now offering food deliveries via drone. You can choose from a limited selection of local eateries, including Panera Bread, Matcha Cafe Maiko, and Joa Korean.

We leverage the Neural Tangent Kernel and its equivalence to training infinitely-wide neural networks to devise \infty -AE: an autoencoder with infinitely-wide bottleneck layers. The outcome is a highly expressive yet simplistic recommendation model with a single hyper-parameter and a closed-form solution. Leveraging \infty -AE's simplicity, we also develop Distill-CF for synthesizing tiny, high-fidelity data summaries which distill the most important knowledge from the extremely large and sparse user-item interaction matrix for efficient and accurate subsequent data-usage like model training, inference, architecture search, etc. This takes a data-centric approach to recommendation, where we aim to improve the quality of logged user-feedback data for subsequent modeling, independent of the learning algorithm. We particularly utilize the concept of differentiable Gumbel-sampling to handle the inherent data heterogeneity, sparsity, and semi-structuredness, while being scalable to datasets with hundreds of millions of user-item interactions.

Apple users--specifically those who use Siri through products such as Macbooks, iPhones, and Apple TVs--may be entitled to make a claim after Apple's class action lawsuit settlement, worth 95 million dollars, regarding the voice-activated assistant. The settlement comes from a lawsuit filed in 2021 by Californian Fumiko Lopez, who claimed that Apple, via Siri, conducted "unlawful and intentional interception and recording of individuals' confidential communications without their consent and subsequent unauthorized disclosure of those communications." "Apple intentionally, willfully, and knowingly violated consumers' privacy rights, including within the sanctity of consumers' own homes where they have the greatest expectation of privacy," the lawsuit stated. "Plaintiffs and Class Members would not have bought their Siri Devices, or would have paid less for them, if they had known Apple was intercepting, recording, disclosing, and otherwise misusing their conversations without consent or authorization." In 2019, Apple published a statement titled "Improving Siri's privacy protections," in which they said they hadn't "been fully living up" to their "high ideals" and vowed to issue improvements.

Transformer architectures have been widely adopted in foundation models. Due to their high inference costs, there is renewed interest in exploring the potential of efficient recurrent architectures (RNNs). In this paper, we analyze the differences in the representational capabilities of Transformers and RNNs across several tasks of practical relevance, including index lookup, nearest neighbor, recognizing bounded Dyck languages, and string equality. For the tasks considered, our results show separations based on the size of the model required for different architectures. For example, we show that a one-layer Transformer of logarithmic width can perform index lookup, whereas an RNN requires a hidden state of linear size.