Despite the empirical success of deep learning, it still lacks theoretical understandings to explain why randomly initialized neural network trained by first-order optimization methods is able to achieve zero training loss, even though its landscape is non-convex and non-smooth. Recently, there are some works to demystifies this phenomenon under over-parameterized regime. In this work, we make further progress on this area by considering a commonly used momentum optimization algorithm: Nesterov accelerated method (NAG). We analyze the convergence of NAG for two-layer fully connected neural network with ReLU activation. Specifically, we prove that the error of NAG converges to zero at a linear convergence rate $1-\Theta(1/\sqrt{\kappa})$, where $\kappa > 1$ is determined by the initialization and the architecture of neural network. Comparing to the rate $1-\Theta(1/\kappa)$ of gradient descent, NAG achieves an acceleration. Besides, it also validates NAG and Heavy-ball method can achieve a similar convergence rate.

Relational databases are the de facto standard for storing and querying structured data, and extracting insights from structured data requires advanced analytics. Deep neural networks (DNNs) have achieved super-human prediction performance in particular data types, e.g., images. However, existing DNNs may not produce meaningful results when applied to structured data. The reason is that there are correlations and dependencies across combinations of attribute values in a table, and these do not follow simple additive patterns that can be easily mimicked by a DNN. The number of possible such cross features is combinatorial, making them computationally prohibitive to model. Furthermore, the deployment of learning models in real-world applications has also highlighted the need for interpretability, especially for high-stakes applications, which remains another issue of concern to DNNs. In this paper, we present ARM-Net, an adaptive relation modeling network tailored for structured data, and a lightweight framework ARMOR based on ARM-Net for relational data analytics. The key idea is to model feature interactions with cross features selectively and dynamically, by first transforming the input features into exponential space, and then determining the interaction order and interaction weights adaptively for each cross feature. We propose a novel sparse attention mechanism to dynamically generate the interaction weights given the input tuple, so that we can explicitly model cross features of arbitrary orders with noisy features filtered selectively. Then during model inference, ARM-Net can specify the cross features being used for each prediction for higher accuracy and better interpretability. Our extensive experiments on real-world datasets demonstrate that ARM-Net consistently outperforms existing models and provides more interpretable predictions for data-driven decision making.

Transfer-based adversarial attacks can effectively evaluate model robustness in the black-box setting. Though several methods have demonstrated impressive transferability of untargeted adversarial examples, targeted adversarial transferability is still challenging. The existing methods either have low targeted transferability or sacrifice computational efficiency. In this paper, we develop a simple yet practical framework to efficiently craft targeted transfer-based adversarial examples. Specifically, we propose a conditional generative attacking model, which can generate the adversarial examples targeted at different classes by simply altering the class embedding and share a single backbone. Extensive experiments demonstrate that our method improves the success rates of targeted black-box attacks by a significant margin over the existing methods -- it reaches an average success rate of 29.6\% against six diverse models based only on one substitute white-box model in the standard testing of NeurIPS 2017 competition, which outperforms the state-of-the-art gradient-based attack methods (with an average success rate of $<$2\%) by a large margin. Moreover, the proposed method is also more efficient beyond an order of magnitude than gradient-based methods.

Hey Welcome back! Want to wish everyone in the US a happy 4th of July🎆🎇! Also, want to quickly mention that the NLP Index has doubled in size (since its inception) with now housing over 6,000…

In his book'The Master Algorithm', Pedro Domingos dissects the ML trains of thought into 5 tribes/groups. Symbolists believe in inductive logic, using data and rules to model intelligent systems. This can be computationally intensive at times, subjected to over-fitting and the'Bias-Variance' paradigm. However, experimentation is the key in most of the methods employed here, for e.g. Connectionists are interested in learning how the brain works and mimic the same using neural networks. Deep learning and mapping the brain (Neurosciences) is endogenous to this tribe.

Deep learning has revolutionized the way we design solution to complex problem. Just like humans learn by experience, machine learning algirithms has enabled to copy the learning mecanism to "train" a network capable of learning complex tasks. The performance of learning is directly proportional to the quality of the training dataset. If the dataset is dominated by a class the learner will be biased towords that majority class and may ignore the minority class. Since high class imbalance is naturally inherent in many real-world applications, e.g., fraud detection and cancer detection.

If you're a software developer who's used to flying solo, GitHub has some news: Get ready to welcome AI into the coding cockpit. Yesterday, the Microsoft-owned code repository and software platform announced GitHub Copilot, a tool created in partnership with Microsoft partner OpenAI that can make suggestions as programmers write code in real time. It's built atop OpenAI Codex, an AI model that learned from billions of lines of code. How it works: The tool analyzes your previous lines of code to suggest new code, and even functions, that developers can then accept or ignore. The current version works best with languages like Python, JavaScript, TypeScript, Ruby, and Go.

Artificial intelligence (AI) remains a loosely-defined and often misunderstood term. We, humans, are full of biases and prejudices, like the so-called anthropomorphic mentality: "the attribution of distinctively human-like feelings, mental states, and behavioral characteristics to inanimate objects, animals, and in general to natural phenomena and supernatural entities". We like everything around us to be like us, anthropomorphizing religious figures, animals, the environment, and technological artifacts (from computational artifacts to robots), including AI, which is developed as replicating humans in the body, mind and behavior. The human body, brain, intelligence, mind and behavior, all are the (privileged) sources of inspiration for AI, both as models to emulate and goals to achieve. Thus, the human body, brain and behavior are projected on human-like AI models, algorithms and applications, or robots, with all the consequences, like a highly probable future of "Extinction", synthetics vs. humans. It is clear and plain that such an AI/ML/DL is the Highway Road to Technological Unemployment and Omnicide (Anthropogenic Human Extinction), the termination of Homo Sapiens as a species. Of all possible scenarios of omnicide as climate change, global nuclear annihilation, biological warfare, ecological collapse, and emerging technologies, as biotechnology or self-replicating nanobots, the most real one is a human-replicating machine intelligence and learning (MIL) and cognitive technologies imitating cognitive functions/skills, capacities/capabilities.

Where does your enterprise stand on the AI adoption curve? Take our AI survey to find out. What is the technology stack you need to create fully autonomous vehicles? Companies and researchers are divided on the answer to that question. Approaches to autonomous driving range from just cameras and computer vision to a combination of computer vision and advanced sensors.

In the future, AI will be very close to replicating human intelligent behavior. So, after seeing the miracle in the advancements in the technology of AI in every field from agriculture to the industry/business mostly people want to learn AI. Therefore, Suggesting some programming language that will greatly help in creating an AI system. Python is a very easy programming language undoubtedly especially for machine learning, Deep learning, Natural language processing, and neural network. According to Forbe's article, ML and DL become very easy after coming of python.