Major disruptions in tokamak pose a serious threat to the vessel and its surrounding pieces of equipment. The ability of the systems to detect any behavior that can lead to disruption can help in alerting the system beforehand and prevent its harmful effects. Many machine learning techniques have already been in use at large tokamaks like JET and ASDEX, but are not suitable for ADITYA, which is comparatively small. Through this work, we discuss a new real-time approach to predict the time of disruption in ADITYA tokamak and validate the results on an experimental dataset. The system uses selected diagnostics from the tokamak and after some pre-processing steps, sends them to a time-sequence Long Short-Term Memory (LSTM) network. The model can make the predictions 12 ms in advance at less computation cost that is quick enough to be deployed in real-time applications.

Each year, almost 10% of claims are denied by payers (i.e., health insurance plans). With the cost to recover these denials and underpayments, predicting payer response (likelihood of payment) from claims data with a high degree of accuracy and precision is anticipated to improve healthcare staffs' performance productivity and drive better patient financial experience and satisfaction in the revenue cycle (Barkholz, 2017). However, constructing advanced predictive analytics models has been considered challenging in the last twenty years. That said, we propose a (low-level) context-dependent compact representation of patients' historical claim records by effectively learning complicated dependencies in the (high-level) claim inputs. Built on this new latent representation, we demonstrate that a deep learning-based framework, Deep Claim, can accurately predict various responses from multiple payers using 2,905,026 de-identified claims data from two US health systems. Deep Claim's improvements over carefully chosen baselines in predicting claim denials are most pronounced as 22.21% relative recall gain (at 95% precision) on Health System A, which implies Deep Claim can find 22.21% more denials than the best baseline system.

Neuron death is a complex phenomenon with implications for model trainability, but until recently it was measured only empirically. Recent articles have claimed that, as the depth of a rectifier neural network grows to infinity, the probability of finding a valid initialization decreases to zero. In this work, we provide a simple and rigorous proof of that result. Then, we show what happens when the width of each layer grows simultaneously with the depth. We derive both upper and lower bounds on the probability that a ReLU network is initialized to a trainable point, as a function of model hyperparameters. Contrary to previous claims, we show that it is possible to increase the depth of a network indefinitely, so long as the width increases as well. Furthermore, our bounds are asymptotically tight under reasonable assumptions: first, the upper bound coincides with the true probability for a single-layer network with the largest possible input set. Second, the true probability converges to our lower bound when the network width and depth both grow without limit. Our proof is based on the striking observation that very deep rectifier networks concentrate all outputs towards a single eigenvalue, in the sense that their normalized output variance goes to zero regardless of the network width. Finally, we develop a practical sign flipping scheme which guarantees with probability one that for a $k$-layer network, the ratio of living training data points is at least $2^{-k}$. We confirm our results with numerical simulations, suggesting that the actual improvement far exceeds the theoretical minimum. We also discuss how neuron death provides a theoretical interpretation for various network design choices such as batch normalization, residual layers and skip connections, and could inform the design of very deep neural networks.

Today's available datasets in the wild, e.g., from social media and open platforms, present tremendous opportunities and challenges for deep learning, as there is a significant portion of tagged images, but often with noisy, i.e. erroneous, labels. Recent studies improve the robustness of deep models against noisy labels without the knowledge of true labels. In this paper, we advocate to derive a stronger classifier which proactively makes use of the noisy labels in addition to the original images - turning noisy labels into learning features. To such an end, we propose a novel framework, ExpertNet, composed of Amateur and Expert, which iteratively learn from each other. Amateur is a regular image classifier trained by the feedback of Expert, which imitates how human experts would correct the predicted labels from Amateur using the noise pattern learnt from the knowledge of both the noisy and ground truth labels. The trained Amateur and Expert proactively leverage the images and their noisy labels to infer image classes. Our empirical evaluations on noisy versions of CIFAR-10, CIFAR-100 and real-world data of Clothing1M show that the proposed model can achieve robust classification against a wide range of noise ratios and with as little as 20-50% training data, compared to state-of-the-art deep models that solely focus on distilling the impact of noisy labels.

Graph representation learning nowadays becomes fundamental in analyzing graph-structured data. Inspired by recent success of contrastive methods, in this paper, we propose a novel framework for unsupervised graph representation learning by leveraging a contrastive objective at the node level. Specifically, we generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. To provide diverse node contexts for the contrastive objective, we propose a hybrid scheme for generating graph views on both structure and attribute levels. Besides, we provide theoretical justification behind our motivation from two perspectives, mutual information and the classical triplet loss. We perform empirical experiments on both transductive and inductive learning tasks using a variety of real-world datasets. Experimental experiments demonstrate that despite its simplicity, our proposed method consistently outperforms existing state-of-the-art methods by large margins. Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.

Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning change, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. The changes in goals or data are referred to as new tasks in a continual learning model. Most of the continual learning methods have a task-known setup in which the task identities are known in advance to the learning model. We propose Multi-view Task Conditional Neural Networks (Mv-TCNN) that does not require to known the reoccurring tasks in advance. We evaluate our model on standard datasets using MNIST, CIFAR10, CIFAR100, and also a real-world dataset that we have collected in a remote healthcare monitoring study (i.e. TIHM dataset). The proposed model outperforms the state-of-the-art solutions in continual learning and adapting to new tasks that are not defined in advance.

As a basis for modelling brain function, deep learning has in recent years been used to model systems in vision, audition, motor control, navigation, and cognitive control. In a new paper, DeepMind researchers call attention to another "fundamentally novel" development in AI research -- deep reinforcement learning (deep RL) -- which they believe also has vital implications for neuroscience and deserves more attention from neuroscientists. The first neuroscience applications of supervised deep learning can be traced back to the 1980s. The increasing availability of more powerful computers over the past decade has renewed research efforts in applying AI approaches -- especially supervised deep learning -- to neuroscience.Deep RL unites deep learning and reinforcement learning, a computational framework that has already had a substantial impact on neuroscience research. The DeepMind team proposes deep RL as a comprehensive framework for studying the interplay between learning, representation, and decision-making that can bring new set of research tools and a wide range of novel hypotheses to the brain sciences.Although deep neural networks have proven an impressive model for neural representation, the team notes that related research has mostly utilized supervised training and has therefore provided little direct leverage on the big-picture problem of understanding motivated, goal-directed …

Lately, I've been thinking and reading a lot about consciousness and how the human mind works. A question that emerges all the time is whether machines can emulate human thought. An even more interesting one is whether consciousness (a subjective experience) can arise from a machine, but I'll leave that discussion for a future post (I'll need 20 more years to think about that before I can write about it). So, how far are we from _behaviorally _imitating a human? Truth is, we achieved a lot in the past 5 years (see AlphaGo, OpenGPT-2, OpenAI Jukebox, Tesla Autopilot, Alphastar, OpenAI Dota2 Team, OpenAI API), but we're still quite not there.

Building a model is not always the goal of a deep learning field. The model may be working fine but it can still be improved with higher accuracy on both training and test sets. This can also include speeding up the model. Some of the features like Regularization, Batch normalization, and Hyperparameter tuning can help in improving our deep learning network with higher accuracy and speed. Let us see how regularization, which is one of these features, is used to improve our neural network.

Different algorithms have been introduced to segment objects based on the criteria like uniform colors, regular shapes, and nearby shadows. Since these algorithms are whether hard-coded or learned from selected features, have some limitations in terms of generalization. Furthermore, these algorithms' accuracies are low, and results are dependent on parameter selection due to manual feature extraction and the lack of big datasets and capable computers. For these reasons, the field of automatic feature extraction has been extensively studied for a long time. In addition, by emerging the neural networks and computers, which can process a huge amount of data in per seconds, a new approach was presented which stacks each layer of neural networks on top of each other to extract features automatically.