Goto

Collaborating Authors

 Education


Non-Determinism and the Lawlessness of Machine Learning Code

arXiv.org Artificial Intelligence

Legal literature on machine learning (ML) tends to focus on harms, and thus tends to reason about individual model outcomes and summary error rates. This focus has masked important aspects of ML that are rooted in its reliance on randomness -- namely, stochasticity and non-determinism. While some recent work has begun to reason about the relationship between stochasticity and arbitrariness in legal contexts, the role of non-determinism more broadly remains unexamined. In this paper, we clarify the overlap and differences between these two concepts, and show that the effects of non-determinism, and consequently its implications for the law, become clearer from the perspective of reasoning about ML outputs as distributions over possible outcomes. This distributional viewpoint accounts for randomness by emphasizing the possible outcomes of ML. Importantly, this type of reasoning is not exclusive with current legal reasoning; it complements (and in fact can strengthen) analyses concerning individual, concrete outcomes for specific automated decisions. By illuminating the important role of non-determinism, we demonstrate that ML code falls outside of the cyberlaw frame of treating ``code as law,'' as this frame assumes that code is deterministic. We conclude with a brief discussion of what work ML can do to constrain the potentially harm-inducing effects of non-determinism, and we indicate where the law must do work to bridge the gap between its current individual-outcome focus and the distributional approach that we recommend.


All you need is feedback: Communication with block attention feedback codes

arXiv.org Artificial Intelligence

Deep neural network (DNN)-based channel code designs have recently gained interest as an alternative to conventional coding schemes, particularly for channels where existing codes do not provide satisfactory performance. Coding in the presence of feedback is one such problem, for which promising results have recently been obtained by various DNN-based coding architectures. In this paper, we introduce a novel learning-aided feedback code design, dubbed generalized block attention feedback (GBAF) codes, that achieves order-of-magnitude improvements in block error rate (BLER) compared to existing solutions. Sequence-to-sequence encoding and block-by-block processing of the message bits in GBAF codes not only reduce the communication overhead due to reduced number of interactions between the transmitter and receiver, but also enable flexible coding rates. More importantly, GBAF codes provide a modular structure that can be implemented using different neural network architectures. In this work, we employ the transformer architecture, which outperforms all the prior DNN-based code designs in terms the block error rate in the low signal-to-noise ratio regime when the feedback channel is noiseless. Reliable communication in the presence of noise has been a long-standing challenge. E. Ozfatura, Y. Shao and D. Gündüz are with Information Processing and Communications Lab, Department of Electrical and Electronic Engineering, Imperial College London. Information storage and communication are two core technologies that underpin the information age, and the success of both hinges on error correction codes, such as BCH, Reed-Muller, convolution, turbo, low-density parity-check (LDPC), and polar codes. While these codes can approach the fundamental Shannon capacity limit over an additive white Gaussian noise (AWGN) channel in the large blocklength regime, there are many scenarios where we do not have practical codes that approach the fundamental theoretical boundaries. Coding in the presence of feedback is one such challenging, yet practical scenario. The classical feedback channel model was introduced and studied by Shannon [1].


BaseTransformers: Attention over base data-points for One Shot Learning

arXiv.org Artificial Intelligence

Few shot classification aims to learn to recognize novel categories using only limited samples per category. Most current few shot methods use a base dataset rich in labeled examples to train an encoder that is used for obtaining representations of support instances for novel classes. Since the test instances are from a distribution different to the base distribution, their feature representations are of poor quality, degrading performance. In this paper we propose to make use of the well-trained feature representations of the base dataset that are closest to each support instance to improve its representation during meta-test time. To this end, we propose BaseTransformers, that attends to the most relevant regions of the base dataset feature space and improves support instance representations. Experiments on three benchmark data sets show that our method works well for several backbones and achieves state-of-the-art results in the inductive one shot setting. Code is available at github.com/mayug/BaseTransformers


[100%OFF] Step-by-Step SEO For Beginners: Boost Your Website

#artificialintelligence

Udemy is the biggest website in the world that offer courses in many categories, all the skills that you would be looking for are offered in Udemy, including languages, design, marketing and a lot of other categories, so when you ever want to buy a courses and pay for a new skills, Udemy would be the best forum for you. You can find payment courses, 100 free courses and coupons also, more than 12 categories are offered, and that what makes sure you will find the domain and the skill you are looking for. Our duty is to search for 100 off courses and free coupons. Would you like to learn how to rank your website higher with search engines? I will lay down very straightforward instructions to achieve this, after a short introduction to the theory of Search Engine Optimization ( SEO).


Machine Learning for Everybody! - KDnuggets

#artificialintelligence

Who is machine learning for? Machine learning is for everybody! Or, at least, that's the name of a new video course from feeCodeCamp, put together by instructor Kylie Ying. The course aims to bring machine learning fundamentals to complete beginners. Learn Machine Learning in a way that is accessible to absolute beginners.


The Supervised Machine Learning Bootcamp

#artificialintelligence

The supervised machine learning algorithms you will learn here are some of the most powerful data science tools you need to solve regression and classification tasks. These are invaluable skills anyone who wants to work as a machine learning engineer and data scientist should have in their toolkit. In this course, you will learn the theory behind all 6 algorithms, and then apply your skills to practical case studies tailored to each one of them, using Python's sci-kit learn library. First, we cover naïve Bayes – a powerful technique based on Bayesian statistics. Its strong point is that it's great at performing tasks in real-time.


Hyperbolic Deep Reinforcement Learning

arXiv.org Artificial Intelligence

We propose a new class of deep reinforcement learning (RL) algorithms that model latent representations in hyperbolic space. Sequential decision-making requires reasoning about the possible future consequences of current behavior. Consequently, capturing the relationship between key evolving features for a given task is conducive to recovering effective policies. To this end, hyperbolic geometry provides deep RL models with a natural basis to precisely encode this inherently hierarchical information. However, applying existing methodologies from the hyperbolic deep learning literature leads to fatal optimization instabilities due to the non-stationarity and variance characterizing RL gradient estimators. Hence, we design a new general method that counteracts such optimization challenges and enables stable end-to-end learning with deep hyperbolic representations. We empirically validate our framework by applying it to popular on-policy and off-policy RL algorithms on the Procgen and Atari 100K benchmarks, attaining near universal performance and generalization benefits. Given its natural fit, we hope future RL research will consider hyperbolic representations as a standard tool.


Domain Discrepancy Aware Distillation for Model Aggregation in Federated Learning

arXiv.org Artificial Intelligence

Knowledge distillation has recently become popular as a method of model aggregation on the server for federated learning. It is generally assumed that there are abundant public unlabeled data on the server. However, in reality, there exists a domain discrepancy between the datasets of the server domain and a client domain, which limits the performance of knowledge distillation. How to improve the aggregation under such a domain discrepancy setting is still an open problem. In this paper, we first analyze the generalization bound of the aggregation model produced from knowledge distillation for the client domains, and then describe two challenges, server-to-client discrepancy and client-to-client discrepancy, brought to the aggregation model by the domain discrepancies. Following our analysis, we propose an adaptive knowledge aggregation algorithm FedD3A based on domain discrepancy aware distillation to lower the bound. FedD3A performs adaptive weighting at the sample level in each round of FL. For each sample in the server domain, only the client models of its similar domains will be selected for playing the teacher role. To achieve this, we show that the discrepancy between the server-side sample and the client domain can be approximately measured using a subspace projection matrix calculated on each client without accessing its raw data. The server can thus leverage the projection matrices from multiple clients to assign weights to the corresponding teacher models for each server-side sample. We validate FedD3A on two popular cross-domain datasets and show that it outperforms the compared competitors in both cross-silo and cross-device FL settings.


NeuDep: Neural Binary Memory Dependence Analysis

arXiv.org Artificial Intelligence

Determining whether multiple instructions can access the same memory location is a critical task in binary analysis. It is challenging as statically computing precise alias information is undecidable in theory. The problem aggravates at the binary level due to the presence of compiler optimizations and the absence of symbols and types. Existing approaches either produce significant spurious dependencies due to conservative analysis or scale poorly to complex binaries. We present a new machine-learning-based approach to predict memory dependencies by exploiting the model's learned knowledge about how binary programs execute. Our approach features (i) a self-supervised procedure that pretrains a neural net to reason over binary code and its dynamic value flows through memory addresses, followed by (ii) supervised finetuning to infer the memory dependencies statically. To facilitate efficient learning, we develop dedicated neural architectures to encode the heterogeneous inputs (i.e., code, data values, and memory addresses from traces) with specific modules and fuse them with a composition learning strategy. We implement our approach in NeuDep and evaluate it on 41 popular software projects compiled by 2 compilers, 4 optimizations, and 4 obfuscation passes. We demonstrate that NeuDep is more precise (1.5x) and faster (3.5x) than the current state-of-the-art. Extensive probing studies on security-critical reverse engineering tasks suggest that NeuDep understands memory access patterns, learns function signatures, and is able to match indirect calls. All these tasks either assist or benefit from inferring memory dependencies. Notably, NeuDep also outperforms the current state-of-the-art on these tasks.


Sketched Reality: Sketching Bi-Directional Interactions Between Virtual and Physical Worlds with AR and Actuated Tangible UI

arXiv.org Artificial Intelligence

This paper introduces Sketched Reality, an approach that combines AR sketching and actuated tangible user interfaces (TUI) for bidirectional sketching interaction. Bi-directional sketching enables virtual sketches and physical objects to "affect" each other through physical actuation and digital computation. In the existing AR sketching, the relationship between virtual and physical worlds is only one-directional -- while physical interaction can affect virtual sketches, virtual sketches have no return effect on the physical objects or environment. In contrast, bi-directional sketching interaction allows the seamless coupling between sketches and actuated TUIs. In this paper, we employ tabletop-size small robots (Sony Toio) and an iPad-based AR sketching tool to demonstrate the concept. In our system, virtual sketches drawn and simulated on an iPad (e.g., lines, walls, pendulums, and springs) can move, actuate, collide, and constrain physical Toio robots, as if virtual sketches and the physical objects exist in the same space through seamless coupling between AR and robot motion. This paper contributes a set of novel interactions and a design space of bi-directional AR sketching. We demonstrate a series of potential applications, such as tangible physics education, explorable mechanism, tangible gaming for children, and in-situ robot programming via sketching.