Goto

Collaborating Authors

 Computational Learning Theory


Solving Weighted Abduction via Max-SAT Solvers

AAAI Conferences

Abduction is a form of inference that seeks the best explanation for the given observation. Because it provides a reasoning process based on background knowledge, it is used in applications that need convincing explanations. In this study, we consider weighted abduction, which is one of the commonly used mathematical models for abduction. The main difficulty associated with applying weighted abduction to real problems is its computational complexity. A state-of-the-art method formulates weighted abduction as an integer linear programming (ILP) problem and solves it using efficient ILP solvers; however, it is still limited to solving problems that include at most 100 rules of background knowledge and observations. In this study, we first formulate the weighted abduction problem as a Max-SAT problem whose hard clauses are mostly Horn clauses. Then, we propose to solve the problem using modern Max-SAT solvers. In our experiments, the proposed method solved the problems much faster than the state-of-the-art ILP-based weighted abduction.


Efficiently Learning Adversarially Robust Halfspaces with Noise

arXiv.org Machine Learning

We study the problem of learning adversarially robust halfspaces in the distribution-independent setting. In the realizable setting, we provide necessary and sufficient conditions on the adversarial perturbation sets under which halfspaces are efficiently robustly learnable. In the presence of random label noise, we give a simple computationally efficient algorithm for this problem with respect to any $\ell_p$-perturbation.


On Learnability under General Stochastic Processes

arXiv.org Machine Learning

Statistical learning theory under independent and identically distributed (iid) sampling and online learning theory for worst case individual sequences are two of the best developed branches of learning theory. Statistical learning under general non-iid stochastic processes is less mature. We provide two natural notions of learnability of a function class under a general stochastic process. We are able to sandwich the first one between iid and online learnability. We show that the second one is in fact equivalent to online learnability. Our results are sharpest in the binary classification setting but we also show that similar results continue to hold in the regression setting.


Search for developments of a box having multiple ways of folding by SAT solver

arXiv.org Artificial Intelligence

A polyomino is a two-dimensional shape formed by joining unit squares edge to edge. A polyomino is called a development if it can make a box by folding edges of unit squares forming the polyomino. As in Figure 1, there are developments that can fold into two incongruent boxes. Many such developments have been discovered. For example, for the surface area 22, it was shown by an exhaustive computer search that there are 2,263 common developments of two boxes of size 1 1 5 and 1 2 3 [1].


Machine Learning Theory - Underfitting vs Overfitting

#artificialintelligence

Sign in to report inappropriate content. Oversimplifying the problem Does not do well in the training set Error due to bias What is Overfitting? High Variance complicates the problem more than necessary to perform well on the training set.


Point Location and Active Learning: Learning Halfspaces Almost Optimally

arXiv.org Machine Learning

Given a finite set $X \subset \mathbb{R}^d$ and a binary linear classifier $c: \mathbb{R}^d \to \{0,1\}$, how many queries of the form $c(x)$ are required to learn the label of every point in $X$? Known as \textit{point location}, this problem has inspired over 35 years of research in the pursuit of an optimal algorithm. Building on the prior work of Kane, Lovett, and Moran (ICALP 2018), we provide the first nearly optimal solution, a randomized linear decision tree of depth $\tilde{O}(d\log(|X|))$, improving on the previous best of $\tilde{O}(d^2\log(|X|))$ from Ezra and Sharir (Discrete and Computational Geometry, 2019). As a corollary, we also provide the first nearly optimal algorithm for actively learning halfspaces in the membership query model. En route to these results, we prove a novel characterization of Barthe's Theorem (Inventiones Mathematicae, 1998) of independent interest. In particular, we show that $X$ may be transformed into approximate isotropic position if and only if there exists no $k$-dimensional subspace with more than a $k/d$-fraction of $X$, and provide a similar characterization for exact isotropic position.



Private Query Release Assisted by Public Data

arXiv.org Machine Learning

We study the problem of differentially private query release assisted by access to public data. In this problem, the goal is to answer a large class $\mathcal{H}$ of statistical queries with error no more than $\alpha$ using a combination of public and private samples. The algorithm is required to satisfy differential privacy only with respect to the private samples. We study the limits of this task in terms of the private and public sample complexities. First, we show that we can solve the problem for any query class $\mathcal{H}$ of finite VC-dimension using only $d/\alpha$ public samples and $\sqrt{p}d^{3/2}/\alpha^2$ private samples, where $d$ and $p$ are the VC-dimension and dual VC-dimension of $\mathcal{H}$, respectively. In comparison, with only private samples, this problem cannot be solved even for simple query classes with VC-dimension one, and without any private samples, a larger public sample of size $d/\alpha^2$ is needed. Next, we give sample complexity lower bounds that exhibit tight dependence on $p$ and $\alpha$. For the class of decision stumps, we give a lower bound of $\sqrt{p}/\alpha$ on the private sample complexity whenever the public sample size is less than $1/\alpha^2$. Given our upper bounds, this shows that the dependence on $\sqrt{p}$ is necessary in the private sample complexity. We also give a lower bound of $1/\alpha$ on the public sample complexity for a broad family of query classes, which by our upper bound, is tight in $\alpha$.


Diffusion Map for Manifold Learning, Theory and Implementation - KDnuggets

#artificialintelligence

'Curse of dimensionality' is a well-known problem in Data Science, which often causes poor performance, inaccurate results, and, most importantly, a similarity measure break-down. The primary cause of this is because high dimensional datasets are typically sparse, and often a lower-dimensional structure or'Manifold' would embed this data. So there is a non-linear relationship among the variables (or features or dimensions), which we need to learn to compute better similarity. Manifold learning is an approach to non-linear dimensionality reduction. The basis for algorithms in manifold learning is that the dimensionality of many data sets is only artificially high 1.


A fast and effective MIP-based heuristic for a selective and periodic inventory routing problem in reverse logistics

arXiv.org Artificial Intelligence

We consider an NP-hard selective and periodic inventory routing problem (SPIRP) in a waste vegetable oil collection environment. This SPIRP arises in the context of reverse logistics where a biodiesel company has daily requirements of oil to be used as raw material in its production process. These requirements can be fulfilled by using the available inventory, collecting waste vegetable oil or purchasing virgin oil. The problem consists in determining a period (cyclic) planning for the collection and purchasing of oil such that the total collection, inventory and purchasing costs are minimized, while meeting the company's oil requirements and all the operational constraints. We propose a MIP-based heuristic which solves a relaxed model without routing, constructs routes taking into account the relaxation's solution and then improves these routes by solving the capacitated vehicle routing problem associated to each period. Following this approach, an a posteriori performance guarantee is ensured, as the approach provides both a lower bound and a feasible solution. The performed computational experiments show that the MIP-based heuristic is very fast and effective as it is able to encounter near optimal solutions with low gaps within seconds, improving several of the best known results using just a fraction of the time spent by a state-of-the-art heuristic. A remarkable fact is that the proposed MIP-based heuristic improves over the best known results for all the large instances available in the literature.