Education
Deep ensembles in bioimage segmentation
Nanni, Loris, Cuza, Daniela, Lumini, Alessandra, Loreggia, Andrea, Brahnam, Sheryl
Semantic segmentation consists in classifying each pixel of an image by assigning it to a specific label chosen from a set of all the available ones. During the last few years, a lot of attention shifted to this kind of task. Many computer vision researchers tried to apply autoencoder structures to develop models that can learn the semantics of the image as well as a low-level representation of it. In an autoencoder architecture, given an input, an encoder computes a low dimensional representation of the input that is then used by a decoder to reconstruct the original data. In this work, we propose an ensemble of convolutional neural networks (CNNs). In ensemble methods, many different models are trained and then used for classification, the ensemble aggregates the outputs of the single classifiers. The approach leverages on differences of various classifiers to improve the performance of the whole system. Diversity among the single classifiers is enforced by using different loss functions. In particular, we present a new loss function that results from the combination of Dice and Structural Similarity Index. The proposed ensemble is implemented by combining different backbone networks using the DeepLabV3+ and HarDNet environment. The proposal is evaluated through an extensive empirical evaluation on two real-world scenarios: polyp and skin segmentation. All the code is available online at https://github.com/LorisNanni.
Does MAML Only Work via Feature Re-use? A Data Centric Perspective
Miranda, Brando, Wang, Yu-Xiong, Koyejo, Sanmi
Recent work has suggested that a good embedding is all we need to solve many few-shot learning benchmarks. Furthermore, other work has strongly suggested that Model Agnostic Meta-Learning (MAML) also works via this same method - by learning a good embedding. These observations highlight our lack of understanding of what meta-learning algorithms are doing and when they work. In this work, we provide empirical results that shed some light on how meta-learned MAML representations function. In particular, we identify three interesting properties: 1) In contrast to previous work, we show that it is possible to define a family of synthetic benchmarks that result in a low degree of feature re-use - suggesting that current few-shot learning benchmarks might not have the properties needed for the success of meta-learning algorithms; 2) meta-overfitting occurs when the number of classes (or concepts) are finite, and this issue disappears once the task has an unbounded number of concepts (e.g., online learning); 3) more adaptation at meta-test time with MAML does not necessarily result in a significant representation change or even an improvement in meta-test performance - even when training on our proposed synthetic benchmarks. Finally, we suggest that to understand meta-learning algorithms better, we must go beyond tracking only absolute performance and, in addition, formally quantify the degree of meta-learning and track both metrics together. Reporting results in future work this way will help us identify the sources of meta-overfitting more accurately and help us design more flexible meta-learning algorithms that learn beyond fixed feature re-use. Finally, we conjecture the core challenge of re-thinking meta-learning is in the design of few-shot learning data sets and benchmarks - rather than in the algorithms, as suggested by previous work.
The Curse of Zero Task Diversity: On the Failure of Transfer Learning to Outperform MAML and their Empirical Equivalence
Miranda, Brando, Wang, Yu-Xiong, Koyejo, Sanmi
It has been recently observed that a transfer learning solution might be all we needed to solve many few-shot learning benchmarks. This raises important questions about when and how meta-learning algorithms should be deployed. In this paper, we make a first step in clarifying these questions by first formulating a computable metric for a few-shot learning benchmark that we hypothesize is predictive of whether meta-learning solutions will succeed or not. We name this metric the diversity coefficient of a few-shot learning benchmark. Using the diversity coefficient, we show that the MiniImagenet benchmark has zero diversity - according to twenty-four different ways to compute the diversity. We proceed to show that when making a fair comparison between MAML learned solutions to transfer learning, both have identical meta-test accuracy. This suggests that transfer learning fails to outperform MAML - contrary to what previous work suggests. Together, these two facts provide the first test of whether diversity correlates with meta-learning success and therefore show that a diversity coefficient of zero correlates with a high similarity between transfer learning and MAML learned solutions - especially at meta-test time. We therefore conjecture meta-learned solutions have the same meta-test performance as transfer learning when the diversity coefficient is zero.
A Survey on Interpretable Reinforcement Learning
Glanois, Claire, Weng, Paul, Zimmer, Matthieu, Li, Dong, Yang, Tianpei, Hao, Jianye, Liu, Wulong
Although deep reinforcement learning has become a promising machine learning approach for sequential decision-making problems, it is still not mature enough for high-stake domains such as autonomous driving or medical applications. In such contexts, a learned policy needs for instance to be interpretable, so that it can be inspected before any deployment (e.g., for safety and verifiability reasons). This survey provides an overview of various approaches to achieve higher interpretability in reinforcement learning (RL). To that aim, we distinguish interpretability (as a property of a model) and explainability (as a post-hoc operation, with the intervention of a proxy) and discuss them in the context of RL with an emphasis on the former notion. In particular, we argue that interpretable RL may embrace different facets: interpretable inputs, interpretable (transition/reward) models, and interpretable decision-making. Based on this scheme, we summarize and analyze recent work related to interpretable RL with an emphasis on papers published in the past 10 years. We also discuss briefly some related research areas and point to some potential promising research directions.
Dual Hierarchical Attention Networks for Bi-typed Heterogeneous Graph Learning
Zhao, Yu, Wei, Shaopeng, Du, Huaming, Chen, Xingyan, Li, Qing, Zhuang, Fuzhen, Liu, Ji, Kou, Gang
Abstract--Bi-type multi-relational heterogeneous graph (BMHG) is one of the most common graphs in practice, for example, academic networks, e-commerce user behavior graph and enterprise knowledge graph. It is a critical and challenge problem on how to learn the numerical representation for each node to characterize subtle structures. However, most previous studies treat all node relations in BMHG as the same class of relation without distinguishing the different characteristics between the intra-class relations and inter-class relations of the bi-typed nodes, causing the loss of significant structure information. To address this issue, we propose a novel Dual Hierarchical Attention Networks (DHAN) based on the bi-typed multi-relational heterogeneous graphs to learn comprehensive node representations with the intra-class and inter-class attention-based encoder under a hierarchical mechanism. Moreover, to sufficiently model node multi-relational information in BMHG, we adopt a newly proposed hierarchical mechanism.
An Investigation on Learning, Polluting, and Unlearning the Spam Emails for Lifelong Learning
Parne, Nishchal, Puppaala, Kyathi, Bhupathi, Nithish, Patgiri, Ripon
Machine unlearning for security is studied in this context. Several spam email detection methods exist, each of which employs a different algorithm to detect undesired spam emails. But these models are vulnerable to attacks. Many attackers exploit the model by polluting the data, which are trained to the model in various ways. So to act deftly in such situations model needs to readily unlearn the polluted data without the need for retraining. Retraining is impractical in most cases as there is already a massive amount of data trained to the model in the past, which needs to be trained again just for removing a small amount of polluted data, which is often significantly less than 1%. This problem can be solved by developing unlearning frameworks for all spam detection models. In this research, unlearning module is integrated into spam detection models that are based on Naive Bayes, Decision trees, and Random Forests algorithms. To assess the benefits of unlearning over retraining, three spam detection models are polluted and exploited by taking attackers' positions and proving models' vulnerability. Reduction in accuracy and true positive rates are shown in each case showing the effect of pollution on models. Then unlearning modules are integrated into the models, and polluted data is unlearned; on testing the models after unlearning, restoration of performance is seen. Also, unlearning and retraining times are compared with different pollution data sizes on all models. On analyzing the findings, it can be concluded that unlearning is considerably superior to retraining. Results show that unlearning is fast, easy to implement, easy to use, and effective.
Forecast Future Demand of Phone Using Predictive Analytics
This is step by step course on how to create predictive model using machine learning. It covers Numpy, Pandas, Matplotlib, Scikit learn and Django and at the end predictive model is deployed on Django. Most of things machine learning beginner do not know is how they can deploy a created model. How to put created model into application? Training model and getting 80%, 85% or 90% accuracy does not matter. As Artificial Intelligence Engineer you should be able to put created model into application.
Developing an RL agent for cognitive impact
I'm sure the parents amongst us are familiar with the dilemma of handing your kid an iPad to watch YouTube and get a few minutes of rest, knowing it might not be the best pedagogical experience (even if we won't admit it publicly). To put this in a more serious manner, a study by Prof. Horowitz-Kraus seeks to examine the cognitive impact of this dilemma. An experiment was conducted in a number of kindergartens to determine whether a child's cognitive development is affected when a person reads him a story as opposed to when he watches the same story on a tablet. I would recommend reading the full articles, but in brief, unsurprisingly the child shows more cognitive improvement in his social and language skills when there is a social interaction compared to the interaction with the tablet. While most of these situations don't necessarily come from laziness, sometimes there are situations where we are just not able to have one-on-one experiences with the children, for instance in hospitals.
Data Science Books You Should Start Reading in 2021
Aside from the real fact that Data Science is one of the highest-paid, hottest and most popular fields today, it's also somehow worth noting that it will certainly remain kind of innovative and also challenging for another decade or more as well. Data science is unquestionably one of the most in-demand professions right now. Data Science job openings abound in the global market, with enticing compensation packages from reputable employers. Companies are hiring data scientists across the board (many of which have data science departments). For ambitious data scientists all across the world, prestigious educational institutes are offering exclusive curriculum, including online diploma courses.
SANSKRIT TECH
SANSKRIT IS NO LONGER JUST A RELIGIOUS LANGUAGE, BUT IS THOUGHT TO BE THE BEST LANGUAGE FOR THE CURRENT DRIFT OF ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING! AS INDIA IS THE ORIGIN OF THE SANSKRIT LANGUAGE, SO WHY NOT BLESS THE WORLD WITH YET ANOTHER IDEA OF REVOLUTION, SANSKRIT APP. We can see the future of the world heading to Artificial Intelligence and Machine Learning. Not a single sector is thought to be left untouched by this futuristic growth. Making a machine learn normal human speech is a task, but is possible.