How did our global TC forecast stack up to observations over the past 12 months? - Reask

The Southern hemisphere tropical cyclone season is now officially over, concluding a first full year of forecasting for us at reask. In the past 12 months we have issued three forecasts for each of the six active basins using our automated Machine Learning (ML) approach – how good were these? Given the probabilistic nature of our forecasts choosing a framework to judge performance is critical. Having spent considerable time and effort modeling complete risk distributions we believe that simply comparing mean predictions with observed occurrences is a huge waste of useful information. Instead, and following a recent article from Nate Silver's FiveThirtyEight blog, we here look at how well calibrated the model appeared in its first year.