This paper solves the task of knowledge base completion i.e. filling the missing relations between two entities by combining Statistical Relational Model like Markov Logic, and knowledge graph embedding method like TransE. Authors define a set of rules to be used in MLNs and then define a joint probability distribution over the observed and hidden triplets. Similarly, they define a joint probability distribution using KGE approaches (specifically they chose transE model). Then they employ the variational EM algorithm to learn the MLN weights and finally predicting the probabilities of hidden triplets. Originality: I really liked the paper, and enjoyed thoroughly reading it.

The submission proposes a model for time-series clustering. The model is a novel combination of several existing components: a) a deep recurrent auto-encoder using dilated RNNs, b) a spectral relaxation of the K-means objective and c) a self-supervision loss to discriminate time-series corrupted by random shuffling from the original ones. The model is evaluated on a common benchmark for time-series clustering and achieves superior performance to existing methods. Overall I feel positive about the proposed method as the quantitative results look promising and using the spectral relaxation of K-means for deep clustering is novel and original. Nevertheless I do have some concerns about the submission in its current form: 1.)

Limited novelty: The proposed approach is closely related to two lines of related work: 1) sg2im [4] which generates images from scene graph representations, and 2) semi-parametric image synthesis [3], which leverages semantic layouts and training images to generate novel images. The key difference to sg2im is the use of image crops in order to perform semi-parametric synthesis; however, in comparison to prior work on semi-parametric methods [3], as suggested by the authors (Line 82-83) the primary difference is the use of graph convolution architecture, where a similar graph convolution method has been introduced in [4]. I'd like to see more justifications from the authors regarding the technical novelty of this approach in presence of these two lines of work. Limited resolution: My concern about the limited novelty is exacerbated by the fact that the generated images are still in low-resolution (64x64) as prior work [4], even though high-resolution image crops are used to aid the image generation process. In contrast, related work [3] is able to generate images of much higher resolutions, e.g., 512x1024, using their semi-parametric method (which was not compared in the experiment).

This submission received borderline positive reviews. While the reviewers ultimately did not reach consensus during the discussion period, one did step forward to'champion' the paper, and another was supportive of this decision. This submission is a'systems paper,' and should be evaluated as such. The paper does not focus on new algorithmic results, but rather on building a nontrivial system to achieve impressive results on an important problem, and it justifies its design decisions (e.g. with an ablation study). There is some concern about the output images being low-resolution.

This paper presents SWAG, a method that uses the iterates of a Polyak-averaging-like stochastic gradient descent to approximate the posterior distribution of a neural network. It is presented as a simple baseline for uncertainty in large deep neural networks and the authors demonstrate its effectiveness on a variety of large scale tasks including residual networks on CIFAR and Imagenet. The strengths of this paper are: - it is indeed a simple baseline for a promising area of research that is really lacking good baselines - experiments are thorough and on benchmarks that are large and interesting to the wider deep learning community - the authors empirically evaluate the quality of their approximation and provide some analysis The main criticism of this paper is that it is not really Bayesian from a purist perspective. R3 is correct to point out that the presented approximation can not actually capture the true posterior as shown by Mandt et al. (Stochastic Gradient Descent as Approximate Bayesian Inference). The language of the paper at times implies otherwise and R3 is right to point this out (e.g.

Later in the paper this is somewhat explained, but it would be good to give additional references on this from learning theory. It is only based on [26] which is more related to approximation theory.

How can local-search methods such as stochastic gradient descent (SGD) avoid bad local minima in training multi-layer neural networks? Why can they fit random labels even given non-convex and non-smooth architectures? Most existing theory only covers networks with one hidden layer, so can we go deeper? In this paper, we focus on recurrent neural networks (RNNs) which are multi-layer networks widely used in natural language processing. They are harder to analyze than feedforward neural networks, because the \emph{same} recurrent unit is repeatedly applied across the entire time horizon of length L, which is analogous to feedforward networks of depth L .

The paper was proofread, well-structured, and very clear. The experiments were clearly described in detail, and provided relevant results. Below we outline some detailed comments of the results. In particular, Chizat and Bach prove that the training of an NTK parameterized network is closely modeled by "lazy training" (their terminology for a linearized model). This paper is not referenced in the related work section.

This paper studies deep neural networks in the regime where the layer widths grow to infinity. Its main contribution is to show that the dynamics of gradient descent for optimizing an infinite width neural network can be explained by the first-order Taylor expansion of the network around its initial parameters, given by the NTK of Jacot et al. Reviewers all agreed this is a valuable contribution which helps the current efforts on understanding the inner workings of gradient descent on large neural networks and its role with regards to generalisation. Despite some concerns about the applicability of this regime to explain the empirical performance of large deep nets and some concurrent work (Chizat and Bach), the authors successfully addressed these concerns in the rebuttal and therefore the AC recommends acceptance.

Weaknesses: There are a lot similar results in slightly different regime, which makes this work looks incremental. In the case of GD, this Morse assumption can be resolved by using a stronger stable manifold theorem in "Michael Shub. I suspect a similar combination might go through here? Usually one view asymptotic results (this paper) weaker than non-asymptotic results (earlier papers), it is also not clear from this paper if one can obtain probability 1 result by modifying the existing high probability result with Borel Cantelli lemma and a bit extra work.