We are currently in a great era for researchers and scientists studying and developing in the field of complex systems. Half of the physics Nobel prize of 2021 was awarded to the physicist Giorgio Parisi for his contributions to the theory of complex systems [9] and the other half to two meteorologists Syukuro Manabe and Klaus Hasselmann to the modeling of the Earth's climate [10]. Parisi has made significant contributions to the literature on complex systems, including areas such as spin glass [11, 12, 13], stochastic resonance [14], surface growth [15], multifractality [16], and bird flocking [17].

In this blog, you will discover why machine learning practitioners should study linear algebra to improve their skills and capabilities as practitioners. After reading this blog, you will understand how can linear algebra be applied in machine learning. Linear algebra is the study of vector spaces, lines and planes, and mappings that are used for linear transforms. It was initially formalized in the 1800s to find the unknowns in linear equations systems, and hence it is relatively a young field of study. Linear Algebra is an essential field of mathematics that can also be called the mathematics of data.

Value iteration can find the optimal replenishment policy for a perishable inventory problem, but is computationally demanding due to the large state spaces that are required to represent the age profile of stock. The parallel processing capabilities of modern GPUs can reduce the wall time required to run value iteration by updating many states simultaneously. The adoption of GPU-accelerated approaches has been limited in operational research relative to other fields like machine learning, in which new software frameworks have made GPU programming widely accessible. We used the Python library JAX to implement value iteration and simulators of the underlying Markov decision processes in a high-level API, and relied on this library's function transformations and compiler to efficiently utilize GPU hardware. Our method can extend use of value iteration to settings that were previously considered infeasible or impractical. We demonstrate this on example scenarios from three recent studies which include problems with over 16 million states and additional problem features, such as substitution between products, that increase computational complexity. We compare the performance of the optimal replenishment policies to heuristic policies, fitted using simulation optimization in JAX which allowed the parallel evaluation of multiple candidate policy parameters on thousands of simulated years. The heuristic policies gave a maximum optimality gap of 2.49%. Our general approach may be applicable to a wide range of problems in operational research that would benefit from large-scale parallel computation on consumer-grade GPU hardware.

Safety certification of data-driven control techniques remains a major open problem. This work investigates backward reachability as a framework for providing collision avoidance guarantees for systems controlled by neural network (NN) policies. Because NNs are typically not invertible, existing methods conservatively assume a domain over which to relax the NN, which causes loose over-approximations of the set of states that could lead the system into the obstacle (i.e., backprojection (BP) sets). To address this issue, we introduce DRIP, an algorithm with a refinement loop on the relaxation domain, which substantially tightens the BP set bounds. Furthermore, we introduce a formulation that enables directly obtaining closed-form representations of polytopes to bound the BP sets tighter than prior work, which required solving linear programs and using hyper-rectangles. Furthermore, this work extends the NN relaxation algorithm to handle polytope domains, which further tightens the bounds on BP sets. DRIP is demonstrated in numerical experiments on control systems, including a ground robot controlled by a learned NN obstacle avoidance policy.

Blind source separation (BSS) aims to recover an unobserved signal $S$ from its mixture $X=f(S)$ under the condition that the effecting transformation $f$ is invertible but unknown. As this is a basic problem with many practical applications, a fundamental issue is to understand how the solutions to this problem behave when their supporting statistical prior assumptions are violated. In the classical context of linear mixtures, we present a general framework for analysing such violations and quantifying their impact on the blind recovery of $S$ from $X$. Modelling $S$ as a multidimensional stochastic process, we introduce an informative topology on the space of possible causes underlying a mixture $X$, and show that the behaviour of a generic BSS-solution in response to general deviations from its defining structural assumptions can be profitably analysed in the form of explicit continuity guarantees with respect to this topology. This allows for a flexible and convenient quantification of general model uncertainty scenarios and amounts to the first comprehensive robustness framework for BSS. Our approach is entirely constructive, and we demonstrate its utility with novel theoretical guarantees for a number of statistical applications.

The era of graph theory began with Euler in the year 1735 to solve the well-known problem of the Königsberg Bridge. In the modern age, graph theory is an integral component of computer science, artificial engineering, machine learning, deep learning, data science, and social networks. A graph G(V, E) is a non-linear data structure, which consists of pair of sets (V, E) where V is the non-empty set of vertices (points or nodes). E is the set of edges (lines or branches) such that there is a mapping f: E V i.e., from the set E to the set of ordered or unordered pairs of elements of V. The number of called the order of the graphs and the number of edges is called the size of graph G (V, E).

Abstract: Two landmark results in combinatorial random matrix theory, due to Komlós and Costello-Tao-Vu, show that discrete random matrices and symmetric discrete random matrices are typically nonsingular. In particular, in the language of graph theory, when p is a fixed constant, the biadjacency matrix of a random Erdős-Rényi bipartite graph G(n,n,p) and the adjacency matrix of an Erdős-Rényi random graph G(n,p) are both nonsingular with high probability. However, very sparse random graphs (i.e., where p is allowed to decay rapidly with n) are typically singular, due to the presence of "local" dependencies such as isolated vertices and pairs of degree-1 vertices with the same neighbour. In this paper we give a combinatorial description of the rank of a sparse random graph G(n,n,c/n) or G(n,c/n) in terms of such local dependencies, for all constants c e (and we present some evidence that the situation is very different for c e). This gives an essentially complete answer to a question raised by Vu at the 2014 International Congress of Mathematicians.

Becoming a machine learning engineer requires a combination of skills and knowledge in various areas such as mathematics, programming, data analysis, and machine learning algorithms. Learn the basics of mathematics and statistics: Machine learning requires a strong foundation in mathematics and statistics. You should be familiar with calculus, linear algebra, probability, and statistics. Master a programming language: You should learn a programming language such as Python or R, which are commonly used for machine learning. You should also be familiar with data structures, algorithms, and object-oriented programming.

We study supervised learning problems for predicting properties of individuals who belong to one of two demographic groups, and we seek predictors that are fair according to statistical parity. This means that the distributions of the predictions within the two groups should be close with respect to the Kolmogorov distance, and fairness is achieved by penalizing the dissimilarity of these two distributions in the objective function of the learning problem. In this paper, we showcase conceptual and computational benefits of measuring unfairness with integral probability metrics (IPMs) other than the Kolmogorov distance. Conceptually, we show that the generator of any IPM can be interpreted as a family of utility functions and that unfairness with respect to this IPM arises if individuals in the two demographic groups have diverging expected utilities. We also prove that the unfairness-regularized prediction loss admits unbiased gradient estimators if unfairness is measured by the squared $\mathcal L^2$-distance or by a squared maximum mean discrepancy. In this case, the fair learning problem is susceptible to efficient stochastic gradient descent (SGD) algorithms. Numerical experiments on real data show that these SGD algorithms outperform state-of-the-art methods for fair learning in that they achieve superior accuracy-unfairness trade-offs -- sometimes orders of magnitude faster. Finally, we identify conditions under which statistical parity can improve prediction accuracy.

Abstract: n varying environments it is beneficial for organisms to utilize available cues to infer the conditions they may encounter and express potentially favorable traits. However, external cues can be unreliable or too costly to use. We consider an alternative strategy where organisms exploit internal sources of information. Even without sensing environmental cues, their internal states may become correlated with the environment as a result of selection, which then form a memory that helps predict future conditions. To demonstrate the adaptive value of such internal memory in varying environments, we revisit the classic example of seed dormancy in annual plants.