We sincerely thank all the reviewers for their insightful comments to help us improve the paper. To Reviewer #2. 1. Are multiple sources more beneficial? This is largely due to the fact that domain gap also exists among different source domains. We will reorganize the layout of Figure 1 in the main paper to make it more clear. We thank the reviewer for pointing this out.

They also show how accurate their keypoint prediction is with an object tracking task using ground truth coordinates. Lastly, they use these keypoints effectively in two downstream RL tasks: model-free RL (neural fitted q iteration) with keypoint-indexed features as input and sample-efficient exploration by defining an intrinsic reward based on maximizing each keypoints movement in x,-x, y,-y directions.

The trustworthiness of f_p In spite of the clear assumption statements, I have concerns of utilizing f_p in the given setting. I am comfortable up to the derivation of Theorem 6 and Eq 6. However, the authors use Eq 7 to optimize the KL divergence, and Eq 7 uses the expectation with the distribution of f_p. While the paper asserts that the distribution function, f_p, can be approximated by the positive dataset. However, Algorithm 1 uses the sample minibatch of B P to empirically estimate f_p.

Learning binary classifiers only from positive and unlabeled (PU) data is an important and challenging task in many real-world applications, including web text classification, disease gene identification and fraud detection, where negative samples are difficult to verify experimentally. Most recent PU learning methods are developed based on the misclassification risk of the supervised learning type, and they may suffer from inaccurate estimates of class prior probabilities. In this paper, we introduce a variational principle for PU learning that allows us to quantitatively evaluate the modeling error of the Bayesian classifier directly from given data. This leads to a loss function which can be efficiently calculated without involving class prior estimation or any other intermediate estimation problems, and the variational learning method can then be employed to optimize the classifier under general conditions. We illustrate the effectiveness of the proposed variational method on a number of benchmark examples.

This paper presents an improved method for learning binary classifiers from positive and unlabeled data. Prior work has required the specification of the proportion of positive data in the unlabeled data set. This parameter is difficult to estimate and the resulting classifier is sensitive to it. While this paper is not the first to attempt to do away with the class prior estimation problem, this paper reports better empirical performance with theoretical results on consistency. As noted by all of the reviewers, the paper is very clearly written and helpfully provides a summary table comparing and contrasting prior work with the current work.

Originality: The main contribution of the paper is to propose a structured representation for video prediction models based on extracting keypoints from images. Models that extract keypoints from images had been proposed before, and here the authors propose an extension of those ideas to video. The paper also has experiments to empirically analyze this representation, which is often lacking in other video prediction papers, despite the fact that learning representations is one of the main motivations for video prediction. Clarity: The paper is well organized and clearly written. Quality and significance: The experiments are sound and properly assess some of the points made by the authors. I believe there are some issues/typos with the model formulation.

The paper proposes a new model for video prediction with a structured representation based on object keypoints. It is a novel approach and also experiment methodology is interesting and generalizable. Reviewers initially asked many questions and the rebuttal was convincing, at least for the majority of reviewers.

In this section, we present the formal proof of Theorem 1. To this end, we interpret DARP as a coordinate ascent algorithm of the Lagrangian dual of its original objective (1), and discuss the necessary and sufficient condition of correct convergence of DARP, i.e., convergence to the optimal solution of (1). Now, we will show that DARP is indeed a coordinate ascent algorithm for the dual of the above optimization. To this end, we formulate the Lagrangian dual of (3). In addition, the optimal objective value of (3) is equivalent to that of (4), i.e., the strong duality holds.

Summary and Contributions: Distribution Aligning Refinery of Pseudo-label (DARP) For semi-supervised learning (SSL), DARP is proposed to match the pseudo-labels with the underlying class distribution of the unlabeled data. The objective function is to minimize the KL divergence of the "aligned" pseudo-labels with the original pseudo-labels subject to the constraints that the "aligned" pseudo-labels are consistent with desired class/label distribution for the unlabeled data. To speed up the process, DARP uses a coordinate ascent algorithm for the Largrangian dual of the objective function. The evaluation was conducted with the CIFAR10 dataset with various artificially degrees of imbalance. DARP was used with a few existing algorithms for imbalanced SSL.