If there is a bingo on mode-k, the m-th row of the mode-k expansion of P is a constant multiple of the (m 1)-th row, where mis a number determined by the bingo position. When a row is a constant multiple of another row, the rank of the matrix is reduced by a maximum of one, which means Rank(P(k)) Ik 1. In the same way, if there are bk bingos, then bk rows are constant multiple of the other rows, which means Rank(P(k)) Ik bk. For any positive tensor P, rank(P) = 1 if and only if its all many-body θparameters are 0. Proof. First, we show that rank(P) = 1 implies all many-body θ-parameters are 0. From the assumption of rank(P) = 1, the m-th row of the mode-k expansion of P have to be a constant multiple of the (m 1)-th row for all m= {2,...,Ik}and k [d].

We only prove fori as proof forj is analogous. Node identifierP Rn dp is an orthonormal matrix withn rows, and type identifier is a trainable matrix E Rbell(k) de with bell(k) rows Eγ1,...,Eγbell(k), each designated for an order-k We now letwin = [I,0], where I R(d+kdp+de) (d+kdp+de) is an identity matrix and0 R(d+kdp+de) (dT (d+kdp+de)) is a matrix filled with zeros. We now let the type identifiersEγ1,...,Eγbell(k) be radially equispaced unit vectors on any twodimensional subspace (Figure 6). For a given query indexi, let us assume there exists at least one key indexjsuch that(i,j) µ3. Therefore, with Eq. (42), we are simply duplicating each output entryFi = L With batch size 1024 on 8 RTX 3090 GPUs, fine-tuning takes 12hours.

Non-negative matrix factorization with fixed row and columnsums.