Dealing with multi-task trade-offs during inference can be addressed via Pareto Front Learning (PFL) methods that parameterize the Pareto Front with a single model, contrary to traditional Multi-Task Learning (MTL) approaches that optimize for a single trade-off which has to be decided prior to training. However, recent PFL methodologies suffer from limited scalability, slow convergence and excessive memory requirements compared to MTL approaches while exhibiting inconsistent mappings from preference space to objective space. In this paper, we introduce PaLoRA, a novel parameter-efficient method that augments the original model with task-specific low-rank adapters and continuously parameterizes the Pareto Front in their convex hull. Our approach dedicates the original model and the adapters towards learning general and task-specific features, respectively. Additionally, we propose a deterministic sampling schedule of preference vectors that reinforces this division of labor, enabling faster convergence and scalability to real world networks. Our experimental results show that PaLoRA outperforms MTL and PFL baselines across various datasets, scales to large networks and provides a continuous parameterization of the Pareto Front, reducing the memory overhead $23.8-31.7$ times compared with competing PFL baselines in scene understanding benchmarks.

Deep learning algorithms - typically consisting of a class of deep neural networks trained by a stochastic gradient descent (SGD) optimization method - are nowadays the key ingredients in many artificial intelligence (AI) systems and have revolutionized our ways of working and living in modern societies. For example, SGD methods are used to train powerful large language models (LLMs) such as versions of ChatGPT and Gemini, SGD methods are employed to create successful generative AI based text-to-image creation models such as Midjourney, DALL-E, and Stable Diffusion, but SGD methods are also used to train DNNs to approximately solve scientific models such as partial differential equation (PDE) models from physics and biology and optimal control and stopping problems from engineering. It is known that the plain vanilla standard SGD method fails to converge even in the situation of several convex optimization problems if the learning rates are bounded away from zero. However, in many practical relevant training scenarios, often not the plain vanilla standard SGD method but instead adaptive SGD methods such as the RMSprop and the Adam optimizers, in which the learning rates are modified adaptively during the training process, are employed. This naturally rises the question whether such adaptive optimizers, in which the learning rates are modified adaptively during the training process, do converge in the situation of non-vanishing learning rates. In this work we answer this question negatively by proving that adaptive SGD methods such as the popular Adam optimizer fail to converge to any possible random limit point if the learning rates are asymptotically bounded away from zero. In our proof of this non-convergence result we establish suitable pathwise a priori bounds for a class of accelerated and adaptive SGD methods, which are also of independent interest.

Soft robots are distinguished by their flexible and adaptable, allowing them to perform tasks that are nearly impossible for rigid robots. However, controlling their configuration is challenging due to their nonlinear material response and infinite deflection degrees of freedom. A potential solution is to discretize the infinite-dimensional configuration space of soft robots into a finite but sufficiently large number of functional shapes. This study explores a co-design strategy for pneumatically actuated soft grippers with multiple encoded stable states, enabling desired functional shape and stiffness reconfiguration. An energy based analytical model for soft multistable grippers is presented, mapping the robots' infinite-dimensional configuration space into discrete stable states, allowing for prediction of the systems final state and dynamic behavior. Our approach introduces a general method to capture the soft robots' response with the lattice lumped parameters using automatic relevance determination regression, facilitating inverse co-design. The resulting computationally efficient model enables us to explore the configuration space in a tractable manner, allowing the inverse co-design of our robots by setting desired targeted positions with optimized stiffness of the set targets. This strategy offers a framework for controlling soft robots by exploiting the nonlinear mechanics of multistable structures, thus embodying mechanical intelligence into soft structures.

In this study, we conduct a comprehensive review of smart grid security, exploring system architectures, attack methodologies, defense strategies, and future research opportunities. We provide an in-depth analysis of various attack vectors, focusing on new attack surfaces introduced by advanced components in smart grids. The review particularly includes an extensive analysis of coordinated attacks that incorporate multiple attack strategies and exploit vulnerabilities across various smart grid components to increase their adverse impact, demonstrating the complexity and potential severity of these threats. Following this, we examine innovative detection and mitigation strategies, including game theory, graph theory, blockchain, and machine learning, discussing their advancements in counteracting evolving threats and associated research challenges. In particular, our review covers a thorough examination of widely used machine learning-based mitigation strategies, analyzing their applications and research challenges spanning across supervised, unsupervised, semi-supervised, ensemble, and reinforcement learning. Further, we outline future research directions and explore new techniques and concerns. We first discuss the research opportunities for existing and emerging strategies, and then explore the potential role of new techniques, such as large language models (LLMs), and the emerging threat of adversarial machine learning in the future of smart grid security.

As fifth-generation (5G) and upcoming sixth-generation (6G) communications exhibit tremendous demands in providing high data throughput with a relatively low latency, millimeter-wave (mmWave) technologies manifest themselves as the key enabling components to achieve the envisioned performance and tasks. In this context, mmWave integrated circuits (IC) have attracted significant research interests over the past few decades, ranging from individual block design to complex system design. However, the highly nonlinear properties and intricate trade-offs involved render the design of analog or RF circuits a complicated process. The rapid evolution of fabrication technology also results in an increasingly long time allocated in the design process due to more stringent requirements. In this thesis, 28-GHz transceiver circuits are first investigated with detailed schematics and associated performance metrics. In this case, two target systems comprising heterogeneous individual blocks are selected and demonstrated on both the transmitter and receiver sides. Subsequently, some conventional and large-scale machine learning (ML) approaches are integrated into the design pipeline of the chosen systems to predict circuit parameters based on desired specifications, thereby circumventing the typical time-consuming iterations found in traditional methods. Finally, some potential research directions are discussed from the perspectives of circuit design and ML algorithms.

This work describes the development and validation of a digital twin for a semi-autogenous grinding (SAG) mill controlled by an expert system. The digital twin consists of three modules emulating a closed-loop system: fuzzy logic for the expert control, a state-space model for regulatory control, and a recurrent neural network for the SAG mill process. The model was trained with 68 hours of data and validated with 8 hours of test data. It predicts the mill's behavior within a 2.5-minute horizon with a 30-second sampling time. The disturbance detection evaluates the need for retraining, and the digital twin shows promise for supervising the SAG mill with the expert control system. Future work will focus on integrating this digital twin into real-time optimization strategies with industrial validation.

Building on the unprecedented capabilities of large language models for command understanding and zero-shot recognition of multi-modal vision-language transformers, visual language navigation (VLN) has emerged as an effective way to address multiple fundamental challenges toward a natural language interface to robot navigation. However, such vision-language models are inherently vulnerable due to the lack of semantic meaning of the underlying embedding space. Using a recently developed gradient based optimization procedure, we demonstrate that images can be modified imperceptibly to match the representation of totally different images and unrelated texts for a vision-language model. Building on this, we develop algorithms that can adversarially modify a minimal number of images so that the robot will follow a route of choice for commands that require a number of landmarks. We demonstrate that experimentally using a recently proposed VLN system; for a given navigation command, a robot can be made to follow drastically different routes. We also develop an efficient algorithm to detect such malicious modifications reliably based on the fact that the adversarially modified images have much higher sensitivity to added Gaussian noise than the original images.

Soft robotics has emerged as the standard solution for grasping deformable objects, and has proven invaluable for mobile robotic exploration in extreme environments. However, despite this growth, there are no widely adopted computational design tools that produce quality, manufacturable designs. To advance beyond the diminishing returns of heuristic bio-inspiration, the field needs efficient tools to explore the complex, non-linear design spaces present in soft robotics, and find novel high-performing designs. In this work, we investigate a hierarchical design optimization methodology which combines the strengths of topology optimization and quality diversity optimization to generate diverse and high-performance soft robots by evolving the design domain. The method embeds variably sized void regions within the design domain and evolves their size and position, to facilitating a richer exploration of the design space and find a diverse set of high-performing soft robots. We demonstrate its efficacy on both benchmark topology optimization problems and soft robotic design problems, and show the method enhances grasp performance when applied to soft grippers. Our method provides a new framework to design parts in complex design domains, both soft and rigid.

Data imputation, the process of filling in missing feature elements for incomplete data sets, plays a crucial role in data-driven learning. A fundamental belief is that data imputation is helpful for learning performance, and it follows that the pursuit of better classification can guide the data imputation process. While some works consider using label information to assist in this task, their simplistic utilization of labels lacks flexibility and may rely on strict assumptions. In this paper, we propose a new framework that effectively leverages supervision information to complete missing data in a manner conducive to classification. Specifically, this framework operates in two stages. Firstly, it leverages labels to supervise the optimization of similarity relationships among data, represented by the kernel matrix, with the goal of enhancing classification accuracy. To mitigate overfitting that may occur during this process, a perturbation variable is introduced to improve the robustness of the framework. Secondly, the learned kernel matrix serves as additional supervision information to guide data imputation through regression, utilizing the block coordinate descent method. The superiority of the proposed method is evaluated on four real-world data sets by comparing it with state-of-the-art imputation methods. Remarkably, our algorithm significantly outperforms other methods when the data is missing more than 60\% of the features

In this work, we introduce the Resilient Projected Push-Pull (RP3) algorithm designed for distributed optimization in multi-agent cyber-physical systems with directed communication graphs and the presence of malicious agents. Our algorithm leverages stochastic inter-agent trust values and gradient tracking to achieve geometric convergence rates in expectation even in adversarial environments. We introduce growing constraint sets to limit the impact of the malicious agents without compromising the geometric convergence rate of the algorithm. We prove that RP3 converges to the nominal optimal solution almost surely and in the $r$-th mean for any $r\geq 1$, provided the step sizes are sufficiently small and the constraint sets are appropriately chosen. We validate our approach with numerical studies on average consensus and multi-robot target tracking problems, demonstrating that RP3 effectively mitigates the impact of malicious agents and achieves the desired geometric convergence.