We will make the distinction clearer by adding "serial" to the legend.

We consider the problem of estimating the input and hidden variables of a stochastic multi-layer neural network from an observation of the output.

Importantly, an item cannot be recommended twice to the same user. The probabilities that a user likes each item are unknown.

Importantly, an item cannot be recommended twice to the same user. The probabilities that a user likes each item are unknown.

We organize the Supplementary Material (Supp. We replace the categorical labels's' and'b' with regression There are two special cases where a weaker Gaussianity property applies, i.e. that the These two special cases are: 1. 's are close to the Stieltjes transform's around m, we have the following result: Lemma 2. F or any N,s N and any z H The second inequality will be proven while proving the first one. The second bound is a direct consequence of the first one, Lemma 2 and convexity. Proposition 5. F or any λ > 0, we have λ < ϑ(λ,N) λ + 1 N Tr[T Let λ > 0. 1. Recall that ϑ (λ) is the unique positive real number such that ϑ ( λ) = λ + ϑ (λ) N Tr null T This yields the lower bound on N ( t) . This allows us to conclude.

A.2 Proof of Theorem 1 We restate the theorem for completeness: Theorem 1. Assume Any ODE's solution, if it exists and converges, converges to an's estimate of the conditional effect is We now bound the remaining term. 's computation of the surrogate intervention involved Thus, such error does not accumulate even with large step sizes. Theorem 4. Effect Connectivity is necessary for nonparametric effect estimation in Let Effect Connectivity be violated, i.e. there exists a Thus, nonparametric effect estimation is impossible. The effect threshold here is 0.1.Figure 7: True positive vs. False negative rate as we vary the threshold on average

A.2 Proof of Theorem 1 We restate the theorem for completeness: Theorem 1. Assume Any ODE's solution, if it exists and converges, converges to an's estimate of the conditional effect is We now bound the remaining term. 's computation of the surrogate intervention involved Thus, such error does not accumulate even with large step sizes. Theorem 4. Effect Connectivity is necessary for nonparametric effect estimation in Let Effect Connectivity be violated, i.e. there exists a Thus, nonparametric effect estimation is impossible. The effect threshold here is 0.1.Figure 7: True positive vs. False negative rate as we vary the threshold on average

Neural Information Processing Systems http://nips.cc/

Importantly, an item cannot be recommended twice to the same user. The probabilities that a user likes each item are unknown.

In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition probability matrix in the policy evaluation step. This rank-one approximation is closely related to the stationary distribution of the corresponding transition probability matrix, which is approximated using the power method. We provide theoretical guarantees for the convergence of the proposed algorithm to optimal (action-)value function with the same rate and computational complexity as the value iteration algorithm in the planning problem and as the Q-learning algorithm in the learning problem. Through our extensive numerical simulations, however, we show that the proposed algorithm consistently outperforms first-order algorithms and their accelerated versions for both planning and learning problems.