Machine intelligence is that last invention that humanity will ever need to make.

The rapid development of artificial intelligence and deep learning technology has provided many opportunities to further enhance the safety, stability, and accuracy of industrial Cyber-Physical Systems (CPS). As indispensable components to many mission-critical CPS assets and equipment, mechanical bearings need to be monitored to identify any trace of abnormal conditions. Most of the data-driven approaches applied to bearing anomaly detection up-to-date are trained using a large amount of fault data collected a priori. In many practical applications, however, it can be unsafe and time-consuming to collect sufficient data samples for each fault category, making it challenging to train a robust classifier. In this paper, we propose a few-shot learning approach for bearing anomaly detection based on model-agnostic meta-learning (MAML), which targets for training an effective fault classifier using limited data.

Many important problems in the real world don't have unique solutions. It is thus important for machine learning models to be capable of proposing different plausible solutions with meaningful probability measures. In this work we introduce such a deep learning framework that learns the one-to-many mappings between the inputs and outputs, together with faithful uncertainty measures. We call our framework modal uncertainty estimation since we model the one-to-many mappings to be generated through a set of discrete latent variables, each representing a latent mode hypothesis that explains the corresponding type of input-output relationship. The discrete nature of the latent representations thus allows us to estimate for any input the conditional probability distribution of the outputs very effectively.

Back on October 12, 2019, the world witnessed a previously unimaginable accomplishment- the first sub-two-hour marathon was run in an incredible time of 1:59:40 by Kenyan native Eliud Kipchoge. He would later say in regards to the amazing achievement that he "expected more people all over the world to run under 2 hours after today" [1]. While Kipchoge set new records in long distance running, across the world a team of natural language processing (NLP) experts at OpenAI, the Elon Musk-backed AI firm, published a new transformer-based language model with 1.5 billion parameters that achieved previously unthinkable performance in nearly every language task it faced [2]. The main takeaway from the paper by many experts was that bigger is better-the intelligence of transformer models can dramatically increase with the scale of parameters. In March of 2020, this theory gained support with OpenAI's release of version three of the model or GPT-3 which encapsulates a staggering 175 billion parameters and achieved even more remarkable performance than version 2, despite sharing, quite literally, the same architecture [3].

Facebook today announced that Microsoft has expanded its participation in PyTorch, the social network's machine learning framework, to take ownership of the development and maintenance of the PyTorch build for Windows. The intent is to bring the experience on Windows in line with other platforms, like Linux; historically, PyTorch on Windows has lagged behind due to a lack of test coverage, a convoluted installation experience, and missing functionality. PyTorch, which Facebook publicly released in January 2017, is an open source machine learning library based on Torch, a scientific computing framework and script language that in turn is based on the Lua programming language. While TensorFlow has been around slightly longer (since November 2015), PyTorch continues to see rapid uptake in the data science and developer community. It claimed one of the top spots for fastest-growing open source projects last year, according to GitHub's 2018 Octoverse report, and Facebook recently revealed that in 2019 the number of contributors to the platform grew more than 50% year-over-year to nearly 1,200.

The increased sophistication of artificial neural networks (ANNs) coupled with the availability of AI-powered chips have driven am unparalleled enterprise interest in computer vision (CV). This exciting new technology will find myriad applications in several industries, and according to GlobalData forecasts, it would reach a market size of $28bn by 2030. The increasing adoption of AI-powered computer vision solutions, consumer drones; and the rising Industry 4.0 adoption will drive this phenomenal change. Deep learning has bought a new change in the role of machine vision used for smart manufacturing and industrial automation. The integration of deep learning propels machine vision systems to adapt itself to manufacturing variations.

Now, Alphabet's DeepMind is taking this automation further by developing deep learning algorithms that can handle programming tasks which have been, to date, the sole domain of the world's top computer scientists (and take them years to write). In a paper recently published on the pre-print server arXiv, the DeepMind team described a new deep reinforcement learning algorithm that was able to discover its own value function--a critical programming rule in deep reinforcement learning--from scratch. Surprisingly, the algorithm was also effective beyond the simple environments it trained in, going on to play Atari games--a different, more complicated task, achieving superhuman levels of play in 14 games. DeepMind says the approach could accelerate the development of reinforcement learning algorithms and even lead to a shift in focus, where instead of spending years writing the algorithms themselves, researchers work to perfect the environments in which they train. Move by move, game by game, an algorithm combines experience and value function to learn which actions bring greater rewards and improves its play, until eventually, engineers may shift from manually developing the algorithms themselves to building the environments where they learn.

If you can't explain it simply, you don't understand it well enough. Disclaimer: This article draws and expands upon material from (1) Christoph Molnar's excellent book on Interpretable Machine Learning which I definitely recommend to the curious reader, (2) a deep learning visualization workshop from Harvard ComputeFest 2020, as well as (3) material from CS282R at Harvard University taught by Ike Lage and Hima Lakkaraju, who are both prominent researchers in the field of interpretability and explainability. This article is meant to condense and summarize the field of interpretable machine learning to the average data scientist and to stimulate interest in the subject. Machine learning systems are becoming increasingly employed in complex high-stakes settings such as medicine (e.g. Despite this increased utilization, there is still a lack of sufficient techniques available to be able to explain and interpret the decisions of these deep learning algorithms. This can be very problematic in some areas where the decisions of algorithms must be explainable or attributable to certain features due to laws or regulations (such as the right to explanation), or where accountability is required. The need for algorithmic accountability has been highlighted many times, the most notable cases of which are Google's facial recognition algorithm that labeled some black people as gorillas, and Uber's self-driving car which ran a stop sign. Due to the inability of Google to fix the algorithm and remove the algorithmic bias that resulted in this issue, they solved the problem by removing words relating to monkeys from Google Photo's search engine. This illustrates the alleged black box nature of many machine learning algorithms. The black box problem is predominantly associated with the supervised machine learning paradigm due to its predictive nature. Accuracy alone is no longer enough. Academics in deep learning are acutely aware of this interpretability and explainability problem, and whilst some argue that these models are essentially black boxes, there have been several developments in recent years which have been developed for visualizing aspects of deep neural networks such the features and representations they have learned. The term info-besity has been thrown around to refer to the difficulty of providing transparency when decisions are made on the basis of many individual features, due to an overload of information.

As someone with a neuroscience background, it's interesting to see the parallels between the parameter size & number of neurons in the recent GPT-3 unveiling versus the cortical regions of the brain responsible for language processing. Estimates put cortical neurons at 25B, and there is somewhere on the order of 7K connections between each neuron (further pruned over time), putting the total number of cortical parameters at somewhere close to 175T. Let's say language processing areas are 1/5th of this, the rest being dedicated to vision, higher order processing, executive function, etc. This puts the language generating portion of the human brain at maybe 35T parameters - yet we're capable of producing such fantastic results with GPT-3's 175B, a number which is 1/200th of the size. Of course, the two are not directly equatable - GPT-3, for example, has'read' several million books' worth of content at this point, whereas the average human has read just a few dozen by the time they're able and language-producing, but it's still interesting to reason about.

Is Deep Learning now leading the charge for innovation in finance? Computational Finance, Machine Learning, and Deep Learning have been essential components of the finance sector for many years. The development of these techniques, technologies, and skills have enabled the financial industry to achieve explosive growth over the decades and become more efficient, sharp, and lucrative for its participants. Will this continue to be what drives the future of the financial industry? With the newer deep learning focus, people driving the financial industry have had to adapt by branching out from an understanding of theoretical financial knowledge.