LetAbean nHermitian matrixandletBbea(n 1) (n 1)matrixwhich is constructed by deleting thei-th row andi-th column ofA. Denote thatΦ = [ϕ(x1),...,ϕ(xn)] Rn D, where D is the dimension of feature spaceH. Performing rank-n singular value decomposition (SVD) onΦ, we have Φ = HΣV, where H Rn n, Σ Rn n is a diagonal matrix whose diagonal elements are the singular values of Φ,andV RD n. F(α) in Eq.(21) is proven differentiable and thep-th component of the gradient is F(α) αp = Then, a reduced gradient descent algorithm [26] is adopted to optimize Eq.(21). The three deep neural networks are pre-trained on the ImageNet[5].

The first lemma is Lemma 3 in [24]. Hermitian matrix and let B be a Hermitian perturbation. To apply Lemma A.1, we must study the relationship between minimum eigenvalue gap of By Lemma A.2, we have (p 1) (p 1) ( p 1) Then, by the proof of Theorem 5.2, we have null null null l null x, y, null H (p 1) (p 1) ( p 1) (p 1) ( p 1) (p 1) (p 1) (p 1) ( p 1) ( p 1) ( p 1) (p 1) (p 1) (p 1) (p 1) (p 1) A.5 The Optimization of SimpleMKKM SimpleMKKM aims to solve the following kernel alignment-based optimization problem: min Assume that the number of iterations is T . Table 4: Large-scale datasets used in the experiments Dataset Samples View Clusters NUSWIDE 30000 5 31 A wA 30475 6 50 MNIST 60000 3 10 YtVideo 101499 5 31 B.2 Clustering Performance with Different Numbers of Landmarks As seen, as the number of landmarks increases, the ACC of the proposed method is approaching SimpleMKKM, and tends to be stable. It shows that we don't need too many landmarks To verify the assumptions about the eigenvalues of the empirical kernel matrix in Theorem 5.2, we To give more empirical studies of the proposed method, we conduct additional experiments on three classic algorithms, i.e., average multiple kernel The results are reported in the following three tables.

Multiple kernel clustering (MKC) is an important research topic that has been widely studied for decades.

We propose a simple yet effective multiple kernel clustering algorithm, termed simple multiple kernel k-means (SimpleMKKM). It extends the widely used supervised kernel alignment criterion to multi-kernel clustering. Our criterion is given by an intractable minimization-maximization problem in the kernel coefficient and clustering partition matrix. To optimize it, we re-formulate the problem as a smooth minimization one, which can be solved efficiently using a reduced gradient descent algorithm. We theoretically analyze the performance of SimpleMKKM in terms of its clustering generalization error. Comprehensive experiments on 11 benchmark datasets demonstrate that SimpleMKKM outperforms state of the art multi-kernel clustering alternatives.