Forecasting Future World Events with Neural Networks Supplementary Material

FiD Temporal models, we reduce the calibration error to 17%, showing potential for improvements. The top 10 articles are used for the FiD-Static model. Is = [0.5, 0.55, ..., 0.95] num_intervals = len(Is) def low_containment_mask(lowers, uppers, labels, Is): # lowers, uppers: Predicted lower and upper bounds of intervals # Is: Target confidence levels # Returns: A list of boolean values indicating which confidence level # has containment ratio below the target level within batch contained = (lowers <= labels) * (labels <= uppers) ratio_contained = contained.mean(dim=0) Figure 1: A reference implementation of the baseline training loss for outputting calibrated confidence intervals. In total, there are nearly 10,000 questions.