Using Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks

Figure 2: Real data predicted vs. true results and category size distribution Python 3.8 Numpy + Pandas suite, Keras and Tensorflow Code is fully available in the lmmnn package on Github Running code: see details in package README file 3 n = 100, 000, σ At each run 80% (80,000) of the simulated data is used as training set, of which 10% (8,000) is used as validation set which the network only uses to check for early stopping. Embedding layer which maps q levels to a d = 0 .1 q vector, so input dimension is p + d - Physical activity (P A) definition: Subjects wore an accelerometer on their wrist for 7 days. ENMO in m-g was summarised across valid wear-time. ETL: We follow instructions by Pearce et al. (2020), implemented in R. At high level, we "once a week" is converted to 1 and "every day" is converted to 7. Finally the P A dependent variable is standardized to have a Baseline DNN architecture: Pearce et al. did not use DNNs, but two separate linear regressions, for men and women. ReLU activation of 10 and 5 neurons, followed by a single output neuron with no activation.