practical application
Disaster response an 'urgent issue,' says new land minister
Disaster response an'urgent issue,' says new land minister Land, Infrastructure, Transport and Tourism Minister Tatsunori Ibayashi answers questions during an interview in Tokyo's Chiyoda Ward on Wednesday. New Land, Infrastructure, Transport and Tourism Minister Tatsunori Ibayashi on Wednesday pledged to respond to heavy rain and other natural disasters with a sense of urgency. Disaster response is an "urgent issue," Ibayashi said in an interview with media organizations, adding that Prime Minister Sanae Takaichi has strongly instructed Cabinet ministers to prioritize countermeasures. Large-scale natural disasters, including Typhoon Dujuan, have caused significant damage recently in Japan, he said. "The types of disasters have diversified considerably."
Generalization Analysis of Transformers in Distribution Regression
In recent years, models based on the Transformer architecture have seen widespread applications and have become one of the core tools in the field of deep learning. Numerous successful techniques, such as parameter-efficient fine-tuning and efficient scaling, have been proposed surrounding their applications to further enhance performance. However, the success of these strategies has always lacked the support of rigorous mathematical theory. To study the underlying mechanisms behind Transformers and related techniques, we first propose a Transformer learning framework motivated by distribution regression, with distributions being inputs, connect a two-stage sampling process with natural language processing, and present a mathematical formulation of the attention mechanism called attention operator. We demonstrate that by the attention operator, Transformers can compress distributions into function representations without loss of information. Moreover, with the advantages of our novel attention operator, Transformers exhibit a stronger capability to learn functionals with more complex structures than convolutional neural networks and fully connected networks. Finally, we obtain a generalization bound within the distribution regression framework. Through the aforementioned theoretical results, we further discuss some successful techniques emerging with large language models (LLMs), such as prompt tuning, parameter-efficient fine-tuning, and efficient scaling. We also provide theoretical insights behind these techniques within our novel analysis framework.
Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining
Data attribution methods, which quantify the influence of individual training data points on a machine learning model, have gained increasing popularity in data-centric applications in modern AI. Despite a recent surge of new methods developed in this space, the impact of hyperparameter tuning in these methods remains under-explored. In this work, we present the first large-scale empirical study to understand the hyperparameter sensitivity of common data attribution methods. Our results show that most methods are indeed sensitive to certain key hyperparameters. However, unlike typical machine learning algorithms---whose hyperparameters can be tuned using computationally-cheap validation metrics---evaluating data attribution performance often requires retraining models on subsets of training data, making such metrics prohibitively costly for hyperparameter tuning.
Towards Differentiating Between Failures and Domain Shifts in Industrial Data Streams
Wojak-Strzelecka, Natalia, Bobek, Szymon, Nalepa, Grzegorz J., Stefanowski, Jerzy
Anomaly and failure detection methods are crucial in identifying deviations from normal system operational conditions, which allows for actions to be taken in advance, usually preventing more serious damages. Long-lasting deviations indicate failures, while sudden, isolated changes in the data indicate anomalies. However, in many practical applications, changes in the data do not always represent abnormal system states. Such changes may be recognized incorrectly as failures, while being a normal evolution of the system, e.g. referring to characteristics of starting the processing of a new product, i.e. realizing a domain shift. Therefore, distinguishing between failures and such ''healthy'' changes in data distribution is critical to ensure the practical robustness of the system. In this paper, we propose a method that not only detects changes in the data distribution and anomalies but also allows us to distinguish between failures and normal domain shifts inherent to a given process. The proposed method consists of a modified Page-Hinkley changepoint detector for identification of the domain shift and possible failures and supervised domain-adaptation-based algorithms for fast, online anomaly detection. These two are coupled with an explainable artificial intelligence (XAI) component that aims at helping the human operator to finally differentiate between domain shifts and failures. The method is illustrated by an experiment on a data stream from the steel factory.
Planning with General Objective Functions: Going Beyond Total Rewards
Standard sequential decision-making paradigms aim to maximize the cumulative reward when interacting with the unknown environment., i.e., maximize $\sum_{h = 1}^H r_h$ where $H$ is the planning horizon. However, this paradigm fails to model important practical applications, e.g., safe control that aims to maximize the lowest reward, i.e., maximize $\min_{h= 1}^H r_h$. In this paper, based on techniques in sketching algorithms, we propose a novel planning algorithm in deterministic systems which deals with a large class of objective functions of the form $f(r_1, r_2, ... r_H)$ that are of interest to practical applications. We show that efficient planning is possible if $f$ is symmetric under permutation of coordinates and satisfies certain technical conditions. Complementing our algorithm, we further prove that removing any of the conditions will make the problem intractable in the worst case and thus demonstrate the necessity of our conditions.
Conditional Diffusion Process for Inverse Halftoning
Inverse halftoning is a technique used to recover realistic images from ancient prints (\textit{e.g.}, photographs, newspapers, books). The rise of deep learning has led to the gradual incorporation of neural network designs into inverse halftoning methods. Most of existing inverse halftoning approaches adopt the U-net architecture, which uses an encoder to encode halftone prints, followed by a decoder for image reconstruction. However, the mainstream supervised learning paradigm with element-wise regression commonly adopted in U-net based methods has poor generalization ability in practical applications. Specifically, when there is a large gap between the dithering patterns of the training and test halftones, the reconstructed continuous-tone images have obvious artifacts.
most DSNs, covering various practical applications, such as camera networks for sports game videos capturing and
We thank the reviewers for all of these valuable comments. We provide point-by-point responses below. Re: generalize to other applications. Cooperative Navigation problem (Lowe et al. '17) and achieved a competitive mean reward (-4.8) against MADDPG Specifically, the stochastic target selection will make the executor inefficient to learn. We will further discuss the factors of each component in the next revision.
The study of the generalization of 2
We thank the reviewers for their constructive and positive comments. They will improve the quality of the paper. As an instance in RL, we mention the problem of "active exploration in MDPs" (see [28]), where the Reiterating the discussion in Section 2.3, let us consider the small-budget regime, and We will provide a footnote in page 7 to clarify this. This is indeed a nice remark. As a result, the theorem is valid even if irreducibility and aperiodicity are dropped.
A common concern from all reviewers is
We kindly thank the reviewers for their detailed reviews, valuable feedback and suggestions for improvement. Indeed, our proof of the new SW theorem relies on an "ordering" of the coordinates of arbitrary equivariant SW theorem under arbitrary finite group action would be desirable, however the proof is out of our reach as of today. In a way, this limitation is similar to the distinction between "point clouds" (which in We will add this discussion in the paper, and mention it in the abstract. In its "deep" original version, it covers all type of "Message-Passing" GNNs, but not spectral GNNs which use powers of the adjacency matrix. We will clarify this in the final version.
Adjustable AprilTags For Identity Secured Tasks
--Special tags such as AprilT ags that facilitate image processing and pattern recognition are useful in practical applications. In close and private environments, identity security is unlikely to be an issue because all involved AprilT ags can be completely regulated. However, in open and public environments, identity security is no longer an issue that can be neglected. T o handle potential harm caused by adversarial attacks, this note advocates utilization of adjustable AprilT ags instead of fixed ones. Special tags that facilitate image processing and pattern recognition are useful in practical applications.