Artificial Neural Networks (ANNs) can be viewed as nonlinear sieves that can approximate complex functions of high dimensional variables more effectively than linear sieves. We investigate the performance of various ANNs in nonparametric instrumental variables (NPIV) models of moderately high dimensional covariates that are relevant to empirical economics. We present two efficient procedures for estimation and inference on a weighted average derivative (WAD): an orthogonalized plug-in with optimally-weighted sieve minimum distance (OP-OSMD) procedure and a sieve efficient score (ES) procedure. Both estimators for WAD use ANN sieves to approximate the unknown NPIV function and are root-n asymptotically normal and first-order equivalent. We provide a detailed practitioner's recipe for implementing both efficient procedures. We compare their finite-sample performances in various simulation designs that involve smooth NPIV function of up to 13 continuous covariates, different nonlinearities and covariate correlations. Some Monte Carlo findings include: 1) tuning and optimization are more delicate in ANN estimation; 2) given proper tuning, both ANN estimators with various architectures can perform well; 3) easier to tune ANN OP-OSMD estimators than ANN ES estimators; 4) stable inferences are more difficult to achieve with ANN (than spline) estimators; 5) there are gaps between current implementations and approximation theories. Finally, we apply ANN NPIV to estimate average partial derivatives in two empirical demand examples with multivariate covariates.

Detecting anomalies at the time of happening is vital in environments like buildings and homes to identify potential cyber-attacks. This paper discussed the various mechanisms to detect anomalies as soon as they occur. We shed light on crucial considerations when building machine learning models. We constructed and gathered data from multiple self-build (DIY) IoT devices with different in-situ sensors and found effective ways to find the point, contextual and combine anomalies. We also discussed several challenges and potential solutions when dealing with sensing devices that produce data at different sampling rates and how we need to pre-process them in machine learning models. This paper also looks at the pros and cons of extracting sub-datasets based on environmental conditions.

Recently, there has been a trend of shifting the execution of deep learning inference tasks toward the edge of the network, closer to the user, to reduce latency and preserve data privacy. At the same time, growing interest is being devoted to the energetic sustainability of machine learning. At the intersection of these trends, we hence find the energetic characterization of machine learning at the edge, which is attracting increasing attention. Unfortunately, calculating the energy consumption of a given neural network during inference is complicated by the heterogeneity of the possible underlying hardware implementation. In this work, we hence aim at profiling the energetic consumption of inference tasks for some modern edge nodes and deriving simple but realistic models. To this end, we performed a large number of experiments to collect the energy consumption of convolutional and fully connected layers on two well-known edge boards by NVIDIA, namely Jetson TX2 and Xavier. From the measurements, we have then distilled a simple, practical model that can provide an estimate of the energy consumption of a certain inference task on the considered boards. We believe that this model can be used in many contexts as, for instance, to guide the search for efficient architectures in Neural Architecture Search, as a heuristic in Neural Network pruning, or to find energy-efficient offloading strategies in a Split computing context, or simply to evaluate the energetic performance of Deep Neural Network architectures.

Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian Optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models' predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

Renewable Energies (RE) have gained more attention in recent years since they offer clean and sustainable energy. One of the major sustainable development goals (SDG-7) set by the United Nations (UN) is to achieve affordable and clean energy for everyone. Among the world's all renewable resources, solar energy is considered as the most abundant and can certainly fulfill the target of SDGs. Solar energy is converted into electrical energy through Photovoltaic (PV) panels with no greenhouse gas emissions. However, power generated by PV panels is highly dependent on solar radiation received at a particular location over a given time period. Therefore, it is challenging to forecast the amount of PV output power. Predicting the output power of PV systems is essential since several public or private institutes generate such green energy, and need to maintain the balance between demand and supply. This research aims to forecast PV system output power based on weather and derived features using different machine learning models. The objective is to obtain the best-fitting model to precisely predict output power by inspecting the data. Moreover, different performance metrics are used to compare and evaluate the accuracy under different machine learning models such as random forest, XGBoost, KNN, etc.

Fostered by novel analytical techniques, digitalization and automation, modern bioprocess development provides high amounts of heterogeneous experimental data, containing valuable process information. In this context, data-driven methods like machine learning (ML) approaches have a high potential to rationally explore large design spaces while exploiting experimental facilities most efficiently. The aim of this review is to demonstrate how ML methods have been applied so far in bioprocess development, especially in strain engineering and selection, bioprocess optimization, scale-up, monitoring and control of bioprocesses. For each topic, we will highlight successful application cases, current challenges and point out domains that can potentially benefit from technology transfer and further progress in the field of ML.

Cloud removal is an essential task in remote sensing data analysis. As the image sensors are distant from the earth ground, it is likely that part of the area of interests is covered by cloud. Moreover, the atmosphere in between creates a constant haze layer upon the acquired images. To recover the ground image, we propose to use scattering model for temporal sequence of images of any scene in the framework of low rank and sparse models. We further develop its variant, which is much faster and yet more accurate. To measure the performance of different methods {\em objectively}, we develop a semi-realistic simulation method to produce cloud cover so that various methods can be quantitatively analysed, which enables detailed study of many aspects of cloud removal algorithms, including verifying the effectiveness of proposed models in comparison with the state-of-the-arts, including deep learning models, and addressing the long standing problem of the determination of regularisation parameters. The latter is companioned with theoretic analysis on the range of the sparsity regularisation parameter and verified numerically.

Every day, observation satellites capture terabytes of imagery of the Earth's surface that feed into a wide variety of civil and military applications. This stream of data has grown so large that only automated methods can feasibly analyze it. One critical component of remote sensing analysis is object detection: locating objects of interest on the Earth's surface in overhead imagery. Automated object detection algorithms have advanced by leaps and bounds over the last decade, but they still require vast amounts of labeled data for training, which is expensive and tedious to produce. Any technique that can reduce the resources needed to label objects in overhead imagery is therefore desirable. Most existing datasets for training overhead object detectors are labeled with horizontal bounding boxes [1][2][3][4][5], object-aligned bounding boxes [6][7][8][9][10], or segmentation masks [11][12].

An intelligent control strategy is proposed to eliminate the actuator saturation problem that exists in the trajectory tracking process of unmanned underwater vehicles (UUV). The control strategy consists of two parts: for the kinematic modeling part, a fuzzy logic-refined backstepping control is developed to achieve control velocities within acceptable ranges and errors of small fluctuations; on the basis of the velocities deducted by the improved kinematic control, the sliding mode control (SMC) is introduced in the dynamic modeling to obtain corresponding torques and forces that should be applied to the vehicle body. With the control velocities computed by the kinematic model and applied forces derived by the dynamic model, the robustness and accuracy of the UUV trajectory without actuator saturation can be achieved.

Compared to conventional robots, flexible manipulators offer many advantages, such as faster end-effector velocities and less energy consumption. However, their flexible structure can lead to undesired oscillations. Therefore, the applied control strategy should account for these elasticities. A feedforward controller based on an inverse model of the system is an efficient way to improve the performance. However, unstable internal dynamics arise for many common flexible robots and stable inversion must be applied. In this contribution, an approximation of the original stable inversion approach is proposed. The approximation simplifies the problem setup, since the internal dynamics do not need to be derived explicitly for the definition of the boundary conditions. From a practical point of view, this makes the method applicable to more complex systems with many unactuated degrees of freedom. Flexible manipulators modeled by the absolute nodal coordinate formulation (ANCF) are considered as an application example.