kcur kcurX i=1

Out of the box, these models take as input a sequence of vectors in embedding space and output asequence ofvectors inthe same space. We treat the prediction of the model at the position corresponding toxi (that is absolute position 2i 1)asthepredictionof f(xi). A.2 Training Each training prompt is produced by sampling a random functionf from the function class we are training on, then sampling inputsxi from the isotropic Gaussian distributionN(0,Id) and constructing apromptas(x1,f(x1),...,xk,f(xk)). For the class of decision trees, the random functionf is represented by a decision tree of depth4 (with16leafnodes),with20dimensionalinputs. Minimum norm least squares is the optimal estimator for the linear regression problem.