Systems & Languages

Recently, Neural Architecture Search is coming back to the research spotlight. For example, there is Weight Agnostic Neural Network (WANN) https://arxiv.org/abs/1906.04358 that demonstrates that Neural Architectures can be more significant than the weights of the network. Are researchers just making up new Neural Architecture Search methods for publication, or is there really a big difference? Are there any work that focused on a detailed comparison for Neural Architecture Search.

Deep Networks have been shown to provide state-of-the-art performance in many machine learning challenges. Unfortunately, they are susceptible to various types of noise, including adversarial attacks and corrupted inputs. In this work we introduce a formal definition of robustness which can be viewed as a localized Lipschitz constant of the network function, quantified in the domain of the data to be classified. We compare this notion of robustness to existing ones, and study its connections with methods in the literature. We evaluate this metric by performing experiments on various competitive vision datasets.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

A neural architecture, which is the structure and connectivity of the network, is typically either hand-crafted or searched by optimizing some specific objective criterion (e.g., classification accuracy). Since the space of all neural architectures is huge, search methods are usually heuristic and do not guarantee finding the optimal architecture, with respect to the objective criterion. In addition, these search methods might require a large number of supervised training iterations and use a high amount of computational resources, rendering the solution infeasible for many applications. Moreover, optimizing for a specific criterion might result in a model that is suboptimal for other useful criteria such as model size, representation of uncertainty and robustness to adversarial attacks. Thus, the resulting architectures of most strategies used today, whether hand crafting or heuristic searches, are densely connected networks, which are not an optimal solution for the objective they were created to achieve, let alone other objectives.

XAIN, the AI startup that specializes in privacy-oriented Federated Machine Learning (FedML), is developing an infrastructure to train artificial intelligence applications through FedML technology, a mechanism that emphasizes data privacy. XAIN's distributed approach to machine learning, which intends to comply with the European Commission's General Data Protection Regulations (GDPR), also provides greater efficiency in the way data is trained, marking a major breakthrough in a field otherwise burdened by costly and onerous processes. When you download facial recognition software onto your phone, your data is usually stored on the central database of the app providing the service. FaceApp, for instance, infuriated the public recently for storing data centrally, though they're far from the first AI-based app to lack privacy protection measures. Data aggregation is essential for AI technology to work -- the question is how to preserve privacy throughout the process.

Let's start from the broader classification. Distributed Artificial Intelligence (DAI) is a class of technologies and methods that span from swarm intelligence to multi-agent technologies and that basically concerns the development of distributed solutions for a specific problem. It can mainly be used for learning, reasoning, and planning, and it is one of the subsets of AI where simulation has a way greater importance than point-prediction. In this class of systems, autonomous learning processing agents (distributed at large scale and independent) reach conclusions or a semi-equilibrium through interaction and communication (even asynchronously). One of the big benefits of those with respect to neural networks is that they do not require the same amount of data to work -- far to say though these are simple systems.

Abstract: Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this work we propose to reexamine the methodology for comparing network architectures. In particular, we introduce a new comparison paradigm of distribution estimates, in which network design spaces are compared by applying statistical techniques to populations of sampled models, while controlling for confounding factors like network complexity. Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape. As a case study, we examine design spaces used in neural architecture search (NAS).

Energy does mean a thing. You are never creating energy, you are only transforming it. Meaning you are still taking energy from somewhere. Having enough energy for everyone to light their house is actually a rising problem since with less atom energy it gets harder to manage and distribute. While it gets hard to distribute that energy we are wasting tons of energy on ML.

We present a novel technique for pre-processing files that can improve file compression rates of existing general purpose lossless file compression algorithms, particularly for files that these algorithms perform poorly on. The elementary cellular automata (CA) pre-processing technique involves finding an optimal CA state that can be used to transform a file into a format that is more amenable to compression than the original file format. This technique is applicable to multiple file types and may be used to enhance multiple compression algorithms. Evaluation on files that we generated, as well as samples selected from online text repositories, finds that the CA pre-processing technique improves compression rates by up to 4% and shows promising results for assisting in compressing data that typically induce worst-case behavior in standard compression algorithms.

Currently, the text document retrieval systems have many challenges in exploring the semantics of queries and documents. Each query implies information which does not appear in the query but the documents related with the information are also expected by user. The disadvantage of the previous spreading activation algorithms could be many irrelevant concepts added to the query. In this paper, a proposed novel algorithm is only activate and add to the query named entities which are related with original entities in the query and explicit relations in the query.