ForecastingFutureWorldEvents withNeuralNetworks SupplementaryMaterial

Finally,tomaketrainingmorestable,we average the loss over the sequence of predictions for each question to weigh the questions evenly. Is = [0.5, 0.55, ..., 0.95] num_intervals = len(Is) def low_containment_mask(lowers, uppers, labels, Is): # lowers, uppers: Predicted lower and upper bounds of intervals # Is: Target confidence levels # Returns: A list of boolean values indicating which confidence level # has containment ratio below the target level within batch contained = (lowers <= labels) * (labels <= uppers) ratio_contained = contained.mean(dim=0) In total, there are nearly 10,000 questions. Gray text indicates the number of questions after augmenting true/false questions with theirnegations,aprocedureweusetobalancethe dataset. Animportanttaskfor numerical forecasting is outputting calibrated uncertainty estimates.