Goto

Collaborating Authors

 Constraint-Based Reasoning


Unsupervised Machine Learning Hybrid Approach Integrating Linear Programming in Loss Function: A Robust Optimization Technique

arXiv.org Artificial Intelligence

Since its formal introduction by Dantzig in 1947, LP has been widely applied across various fields, including operations research, economics, and engineering, due to its ability to optimize objectives subject to linear constraints (Dantzig, 1951; Bazaraa et al., 2013). However, traditional LP approaches have certain limitations, particularly in dealing with non-linear, high-dimensional, and dynamic environments where relationships among variables are complex and non-linear (Bertsimas & Tsitsiklis, 1997). By contrast, machine learning (ML) methods, especially deep learning, have demonstrated remarkable success in modeling complex patterns and making predictions based on large datasets (LeCun et al., 2015; Goodfellow et al., 2016). Despite these strengths, ML models often lack the explicit interpretability and rigorous constraint satisfaction that LP offers (Rudin, 2019). This has motivated researchers to explore hybrid approaches that combine the strengths of LP and ML, aiming to develop models that are both interpretable and powerful in their predictive capabilities. This paper proposes a novel hybrid method that integrates LP within the loss function of an unsupervised machine learning model. By embedding LP constraints directly into the ML framework, this approach not only maintains the interpretability and constraint satisfaction of LP but also leverages the flexibility and learning capacity of ML. This integration is particularly beneficial in unsupervised or semi-supervised settings, where traditional LP methods may struggle to provide robust solutions due to the lack of labeled data (Amos & Kolter, 2017).


Realtime Generation of Streamliners with Large Language Models

arXiv.org Artificial Intelligence

Streamliners are certain constraints added to a constraint model to reduce the search space, thereby improving the feasibility and speed of finding solutions to complex constraint satisfaction problems. By incorporating domain-specific knowledge, streamliners can guide the constraint solver, allowing it to bypass less promising areas of the search space. Gomes and Sellmann (2004a) introduced streamliners to speed up the constrained-based search for hard combinatorial design problems. Today, streamliners are a standard tool for speeding up constrained-based search. Streamliners are closely related to implied/redundant constraints, symmetry-breaking constraints, and dominance-breaking constraints; however, adding a streamliner may even cause the constraint model to become inconsistent. Originally, streamliners were hand-crafted by researchers who used their theoretical insight to analyze the constrained model. However, progress has also been made on the automated generation of streamliners (Spracklen et al. 2023) by systematically trying the effect of some atomic constraints, such as imposing specific constraints on integer and function domains, like enforcing odd or even values, monotonicity, and properties like commutativity, as well as facilitating specific attributes in binary relations. These atomic restrictions are tested on thousands of problem instances, and those that show a good streamlining effect are systematically combined.


A General Framework for Constraint-based Causal Learning

arXiv.org Artificial Intelligence

By representing any constraint-based causal learning algorithm via a placeholder property, we decompose the correctness condition into a part relating the distribution and the true causal graph, and a part that depends solely on the distribution. This provides a general framework to obtain correctness conditions for causal learning, and has the following implications. We provide exact correctness conditions for the PC algorithm, which are then related to correctness conditions of some other existing causal discovery algorithms. We show that the sparsest Markov representation condition is the weakest correctness condition resulting from existing notions of minimality for maximal ancestral graphs and directed acyclic graphs. We also reason that additional knowledge than just Pearl-minimality is necessary for causal learning beyond faithfulness.


Reasoning about Study Regulations in Answer Set Programming

arXiv.org Artificial Intelligence

We are interested in automating reasoning with and about study regulations, catering to various stakeholders, ranging from administrators, over faculty, to students at different stages. Our work builds on an extensive analysis of various study programs at the University of Potsdam. The conceptualization of the underlying principles provides us with a formal account of study regulations. In particular, the formalization reveals the properties of admissible study plans. With these at end, we propose an encoding of study regulations in Answer Set Programming that produces corresponding study plans. Finally, we show how this approach can be extended to a generic user interface for exploring study plans.


Decomposition Strategies and Multi-shot ASP Solving for Job-shop Scheduling

arXiv.org Artificial Intelligence

The Job-shop Scheduling Problem (JSP) is a well-known and challenging combinatorial optimization problem in which tasks sharing a machine are to be arranged in a sequence such that encompassing jobs can be completed as early as possible. In this paper, we investigate problem decomposition into time windows whose operations can be successively scheduled and optimized by means of multi-shot Answer Set Programming (ASP) solving. From a computational perspective, decomposition aims to split highly complex scheduling tasks into better manageable subproblems with a balanced number of operations such that good-quality or even optimal partial solutions can be reliably found in a small fraction of runtime. We devise and investigate a variety of decomposition strategies in terms of the number and size of time windows as well as heuristics for choosing their operations. Moreover, we incorporate time window overlapping and compression techniques into the iterative scheduling process to counteract optimization limitations due to the restriction to window-wise partial schedules. Our experiments on different JSP benchmark sets show that successive optimization by multi-shot ASP solving leads to substantially better schedules within tight runtime limits than single-shot optimization on the full problem. In particular, we find that decomposing initial solutions obtained with proficient heuristic methods into time windows leads to improved solution quality.


Positive-Unlabeled Constraint Learning (PUCL) for Inferring Nonlinear Continuous Constraints Functions from Expert Demonstrations

arXiv.org Artificial Intelligence

Planning for a wide range of real-world robotic tasks necessitates to know and write all constraints. However, instances exist where these constraints are either unknown or challenging to specify accurately. A possible solution is to infer the unknown constraints from expert demonstration. This paper presents a novel Positive-Unlabeled Constraint Learning (PUCL) algorithm to infer a continuous arbitrary constraint function from demonstration, without requiring prior knowledge of the true constraint parameterization or environmental model as existing works. Within our framework, we treat all data in demonstrations as positive (feasible) data, and learn a control policy to generate potentially infeasible trajectories, which serve as unlabeled data. In each iteration, we first update the policy and then a two-step positive-unlabeled learning procedure is applied, where it first identifies reliable infeasible data using a distance metric, and secondly learns a binary feasibility classifier (i.e., constraint function) from the feasible demonstrations and reliable infeasible data. The proposed framework is flexible to learn complex-shaped constraint boundary and will not mistakenly classify demonstrations as infeasible as previous methods. The effectiveness of the proposed method is verified in three robotic tasks, using a networked policy or a dynamical system policy. It successfully infers and transfers the continuous nonlinear constraints and outperforms other baseline methods in terms of constraint accuracy and policy safety.


Fair Risk Minimization under Causal Path-Specific Effect Constraints

arXiv.org Machine Learning

In the realm of machine learning and artificial intelligence, ensuring fairness in algorithmic decisionmaking has become an imperative challenge. This concern is driven by the growing awareness of the ethical and societal consequences stemming from automated systems. Unlike the traditional focuses that solely enhance statistical and machine learning algorithms across various performance metrics, the emphasis on fairness injects a nuanced layer of complexity into predictive modeling. It mandates a conscientious evaluation of how algorithmic predictions may perpetuate or mitigate existing biases, thereby influencing real-world outcomes across diverse societal sectors. The notion of fairness within the machine learning community includes a diverse array of definitions for what it means for an algorithm to be fair, with each definition informed by distinct ethical considerations and operational implications [Mitchell et al., 2021, Plecko and Bareinboim, 2022, Barocas et al., 2023]. This diversity often leads to a challenging paradox for practitioners and theorists alike, as adhering to one notion of fairness can sometimes contradict the requirements or objectives of another [Kleinberg et al., 2017, Corbett-Davies and Goel, 2018, Friedler et al., 2021]. The task of ensuring algorithmic fairness is made more complex by its inherently contextspecific nature, which demands careful consideration in selecting fairness criteria that are not only theoretically sound but also practically applicable and aligned with prevailing societal values.


Analyzing the Effectiveness of Quantum Annealing with Meta-Learning

arXiv.org Artificial Intelligence

The field of Quantum Computing has gathered significant popularity in recent years and a large number of papers have studied its effectiveness in tackling many tasks. We focus in particular on Quantum Annealing (QA), a meta-heuristic solver for Quadratic Unconstrained Binary Optimization (QUBO) problems. It is known that the effectiveness of QA is dependent on the task itself, as is the case for classical solvers, but there is not yet a clear understanding of which are the characteristics of a problem that makes it difficult to solve with QA. In this work, we propose a new methodology to study the effectiveness of QA based on meta-learning models. To do so, we first build a dataset composed of more than five thousand instances of ten different optimization problems. We define a set of more than a hundred features to describe their characteristics, and solve them with both QA and three classical solvers. We publish this dataset online for future research. Then, we train multiple meta-models to predict whether QA would solve that instance effectively and use them to probe which are the features with the strongest impact on the effectiveness of QA. Our results indicate that it is possible to accurately predict the effectiveness of QA, validating our methodology. Furthermore, we observe that the distribution of the problem coefficients representing the bias and coupling terms is very informative to identify the probability of finding good solutions, while the density of these coefficients alone is not enough. The methodology we propose allows to open new research directions to further our understanding of the effectiveness of QA, by probing specific dimensions or by developing new QUBO formulations that are better suited for the particular nature of QA. Furthermore, the proposed methodology is flexible and can be extended or used to study other quantum or classical solvers.


Prompt2DeModel: Declarative Neuro-Symbolic Modeling with Natural Language

arXiv.org Artificial Intelligence

This paper presents a conversational pipeline for crafting domain knowledge for complex neuro-symbolic models through natural language prompts. It leverages large language models to generate declarative programs in the DomiKnowS framework. The programs in this framework express concepts and their relationships as a graph in addition to logical constraints between them. The graph, later, can be connected to trainable neural models according to those specifications. Our proposed pipeline utilizes techniques like dynamic in-context demonstration retrieval, model refinement based on feedback from a symbolic parser, visualization, and user interaction to generate the tasks' structure and formal knowledge representation. This approach empowers domain experts, even those not well-versed in ML/AI, to formally declare their knowledge to be incorporated in customized neural models in the DomiKnowS framework.


Machine Learning for Equitable Load Shedding: Real-time Solution via Learning Binding Constraints

arXiv.org Artificial Intelligence

Timely and effective load shedding in power systems is critical for maintaining supply-demand balance and preventing cascading blackouts. To eliminate load shedding bias against specific regions in the system, optimization-based methods are uniquely positioned to help balance between economical and equity considerations. However, the resulting optimization problem involves complex constraints, which can be time-consuming to solve and thus cannot meet the real-time requirements of load shedding. To tackle this challenge, in this paper we present an efficient machine learning algorithm to enable millisecond-level computation for the optimization-based load shedding problem. Numerical studies on both a 3-bus toy example and a realistic RTS-GMLC system have demonstrated the validity and efficiency of the proposed algorithm for delivering equitable and real-time load shedding decisions.