We refer exclusively, throughout, to the more common implementation Skipgram with negative sampling.

The interval is computed from a single finite-length sample path from the Markov chain, and does not require the knowledge of any parameters of the chain. This stands in contrast to previous approaches, which either only provide point estimates, or require a reset mechanism, or additional prior knowledge.

Neural Information Processing Systems http://nips.cc/

First we start with the reparametrized Projected Gradient Descent algorithm. The update rule for g follows directly. Suppose that the loss does not converge to zero. Now, in general, to avoid converging to this set, we must make some additional assumptions on the initialization. However, it is also more general.

We thank for the reviews and will resolve the main concerns. We sincerely ask the reviewers to re-evaluate the rating. Linear regression is a fundamental theoretical problem. Moreover, the most important part in our setting is "under-determined". We apologize for our word "proposed" in the paper and will remove this word.

Neural Information Processing Systems http://nips.cc/

Neural Information Processing Systems http://nips.cc/

Nonlinear component analysis such as kernel Principle Component Analysis (KPCA) and kernel Canonical Correlation Analysis (KCCA) are widely used in machine learning, statistics and data analysis, but they cannot scale up to big datasets. Recent attempts have employed random feature approximations to convert the problem to the primal form for linear computational complexity. However, to obtain high quality solutions, the number of random features should be the same order of magnitude as the number of data points, making such approach not directly applicable to the regime with millions of data points. We propose a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD. Despite the non-convex nature of these problems, our method enjoys theoretical guarantees that it converges at the rate O (1 /t) to the global optimum, even for the top k eigen subspace. Unlike many alternatives, our algorithm does not require explicit orthogonaliza-tion, which is infeasible on big datasets. We demonstrate the effectiveness and scalability of our algorithm on large scale synthetic and real world datasets.

This paper proposes two efficient algorithms for computing approximate second-order stationary points (SOSPs) of problems with generic smooth non-convex objective functions and generic linear constraints. While finding (approximate) SOSPs for the class of smooth non-convex linearly constrained problems is computationally intractable, we show that generic problem instances in this class can be solved efficiently. Specifically, for a generic problem instance, we show that certain strict complementarity (SC) condition holds for all Karush-Kuhn-Tucker (KKT) solutions. Based on this condition, we design an algorithm named S uccessive N egative-curvature grA dient P rojection (SNAP), which performs either conventional gradient projection or some negative curvature based projection steps to find SOSPs.