We only prove fori as proof forj is analogous. Node identifierP Rn dp is an orthonormal matrix withn rows, and type identifier is a trainable matrix E Rbell(k) de with bell(k) rows Eγ1,...,Eγbell(k), each designated for an order-k We now letwin = [I,0], where I R(d+kdp+de) (d+kdp+de) is an identity matrix and0 R(d+kdp+de) (dT (d+kdp+de)) is a matrix filled with zeros. We now let the type identifiersEγ1,...,Eγbell(k) be radially equispaced unit vectors on any twodimensional subspace (Figure 6). For a given query indexi, let us assume there exists at least one key indexjsuch that(i,j) µ3. Therefore, with Eq. (42), we are simply duplicating each output entryFi = L With batch size 1024 on 8 RTX 3090 GPUs, fine-tuning takes 12hours.

Non-negative matrix factorization with fixed row and columnsums.