A Experimental Details

Gym tasks are shown below in Table 8. Hyperparameter V alue Number of layers 3 Number of attention heads 1 Embedding dimension 128 Nonlinearity function ReLU Batch size 64 Context length K 20 HalfCheetah, Hopper, Walker 5 Reacher Return-to-go conditioning 6000 HalfCheetah 3600 Hopper 5000 Walker 50 Reacher Dropout 0 . 1 Learning rate 10 As briefly mentioned in Section 4.2, we found previously reported behavior cloning baselines to be The percentile behavior cloning experiments use the same hyperparameters. We give details of the illustrative example discussed in the introduction. The action is the integer index of the graph node to move to next. In this environment, we use the GPT model as described in Section 3 to generate both actions and return-to-go tokens.