Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone $K$, a norm $\|\cdot\|$ and a smooth convex function $f$, we want either 1) to minimize the norm over the intersection of the cone and a level set of $f$, or 2) to minimize over the cone the sum of $f$ and a multiple of the norm. We focus on the case where (a) the dimension of the problem is too large to allow for interior point algorithms, (b) $\|\cdot\|$ is "too complicated" to allow for computationally cheap Bregman projections required in the first-order proximal gradient algorithms. On the other hand, we assume that {it is relatively easy to minimize linear forms over the intersection of $K$ and the unit $\|\cdot\|$-ball}. Motivating examples are given by the nuclear norm with $K$ being the entire space of matrices, or the positive semidefinite cone in the space of symmetric matrices, and the Total Variation norm on the space of 2D images. We discuss versions of the Conditional Gradient algorithm capable to handle our problems of interest, provide the related theoretical efficiency estimates and outline some applications.

Many data sets contain rich information about objects, as well as pairwise relations between them. For instance, in networks of websites, scientific papers, and other documents, each node has content consisting of a collection of words, as well as hyperlinks or citations to other nodes. In order to perform inference on such data sets, and make predictions and recommendations, it is useful to have models that are able to capture the processes which generate the text at each node and the links between them. In this paper, we combine classic ideas in topic modeling with a variant of the mixed-membership block model recently developed in the statistical physics community. The resulting model has the advantage that its parameters, including the mixture of topics of each document and the resulting overlapping communities, can be inferred with a simple and scalable expectation-maximization algorithm. We test our model on three data sets, performing unsupervised topic classification and link prediction. For both tasks, our model outperforms several existing state-of-the-art methods, achieving higher accuracy with significantly less computation, analyzing a data set with 1.3 million words and 44 thousand links in a few minutes.

After experimenting with a number of non-probabilistic methods for dealing with uncertainty many researchers reaffirm a preference for probability methods [1] [2], although this remains controversial. The importance of being able to form decisions from incomplete data in diagnostic problems has highlighted probabilistic methods [5] which compute posterior probabilities from prior distributions in a way similar to Bayes Rule, and thus are called Bayesian methods. This paper documents the use of a Bayesian method in a real time problem which is similar to medical diagnosis in that there is a need to form decisions and take some action without complete knowledge of conditions in the problem domain. This particular method has a limitation which is discussed.

A primary motivation for reasoning under uncertainty is to derive decisions in the face of inconclusive evidence. However, Shafer's theory of belief functions, which explicitly represents the underconstrained nature of many reasoning problems, lacks a formal procedure for making decisions. Clearly, when sufficient information is not available, no theory can prescribe actions without making additional assumptions. Faced with this situation, some assumption must be made if a clearly superior choice is to emerge. In this paper we offer a probabilistic interpretation of a simple assumption that disambiguates decision problems represented with belief functions. We prove that it yields expected values identical to those obtained by a probabilistic analysis that makes the same assumption. In addition, we show how the decision analysis methodology frequently employed in probabilistic reasoning can be extended for use with belief functions.

We introduce a graceful approach to probabilistic inference called bounded conditioning. Bounded conditioning monotonically refines the bounds on posterior probabilities in a belief network with computation, and converges on final probabilities of interest with the allocation of a complete resource fraction. The approach allows a reasoner to exchange arbitrary quantities of computational resource for incremental gains in inference quality. As such, bounded conditioning holds promise as a useful inference technique for reasoning under the general conditions of uncertain and varying reasoning resources. The algorithm solves a probabilistic bounding problem in complex belief networks by breaking the problem into a set of mutually exclusive, tractable subproblems and ordering their solution by the expected effect that each subproblem will have on the final answer. We introduce the algorithm, discuss its characterization, and present its performance on several belief networks, including a complex model for reasoning about problems in intensive-care medicine.

Considerable attention has been given to the problem of non-monotonic reasoning in a belief function framework. Earlier work (M. Ginsberg) proposed solutions introducing meta-rules which recognized conditional independencies in a probabilistic sense. More recently an e-calculus formulation of default reasoning (J. Pearl) shows that the application of Dempster's rule to a non-monotonic situation produces erroneous results. This paper presents a new belief function interpretation of the problem which combines the rules in a way which is more compatible with probabilistic results and respects conditions of independence necessary for the application of Dempster's combination rule. A new general framework for combining conflicting evidence is also proposed in which the normalization factor becomes modified. This produces more intuitively acceptable results.

This paper describes a generalization of previous methods for constructing tree-structured belief network with hidden variables. The major new feature of the described method is the ability to produce a tree decomposition even when there are errors in the correlation data among the input variables. This is an important extension of existing methods since the correlational co efficients usually cannot be measured with precision. The technique involves using a greedy search algorithm that locally minimizes an error function.

A. Abstract Control Strategies for hierachical treelike probabilistic inference networks are formulated and investigated. Strategies that utilize staged look-ahead and temporary focus on subgoals are formalized and refined using the Depth Vector concept that serves as a tool for defining the'virtual tree' regarded by the control strategy. The concept is illustrated by four types of control strategies for three-level trees that are characterized according to their Depth Vector, and according to the way they consider intermediate nodes and the role that they let these nodes play. INFERENTl is a computerized inference system written in Prolog, which provides tools for exercising a variety of control strategies. The system also provides tools for simulating test data and for comparing the relative average performance under different strategies.

Bayesian networks provide a probabilistic semantics for qualitative assertions about likelihood. A qualitative reasoner based on an algebra over these assertions can derive further conclusions about the influence of actions. While the conclusions are much weaker than those computed from complete probability distributions, they are still valuable for suggesting potential actions, eliminating obviously inferior plans, identifying important tradeoffs, and explaining probabilistic models.

Duda, Hart, and Nilsson have set forth a method for rule-based inference systems to use in updating the probabilities of hypotheses on the basis of multiple items of new evidence. Pednault, Zucker, and Muresan claimed to give conditions under which independence assumptions made by Duda et al. preclude updating-that is, prevent the evidence from altering the probabilities of the hypotheses. Glymour refutes Pednault et al.'s claim with a counterexample of a rather special form (one item of evidence is incompatible with all but one of the hypotheses); he raises, but leaves open, the question whether their result would be true with an added assumption to rule out such special cases. We show that their result does not hold even with the added assumption, but that it can nevertheless be largely salvaged. Namely, under the conditions assumed by Pednault et al., at most one of the items of evidence can alter the probability of any given hypothesis; thus, although updating is possible, multiple updating for any of the hypotheses is precluded.