Kazmi, Najeeb
An ensemble of data-driven weather prediction models for operational sub-seasonal forecasting
Weyn, Jonathan A., Kumar, Divya, Berman, Jeremy, Kazmi, Najeeb, Klocek, Sylwester, Luferenko, Pete, Thambiratnam, Kit
We present an operations-ready multi-model ensemble weather forecasting system which uses hybrid data-driven weather prediction models coupled with the European Centre for Medium-range Weather Forecasts (ECMWF) ocean model to predict global weather at 1-degree resolution for 4 weeks of lead time. For predictions of 2-meter temperature, our ensemble on average outperforms the raw ECMWF extended-range ensemble by 4-17%, depending on the lead time. However, after applying statistical bias corrections, the ECMWF ensemble is about 3% better at 4 weeks. For other surface parameters, our ensemble is also within a few percentage points of ECMWF's ensemble. We demonstrate that it is possible to achieve near-state-of-the-art subseasonal-to-seasonal forecasts using a multi-model ensembling approach with data-driven weather prediction models.
Machine Learning at Microsoft with ML .NET
Ahmed, Zeeshan, Amizadeh, Saeed, Bilenko, Mikhail, Carr, Rogan, Chin, Wei-Sheng, Dekel, Yael, Dupre, Xavier, Eksarevskiy, Vadim, Erhardt, Eric, Eseanu, Costin, Filipi, Senja, Finley, Tom, Goswami, Abhishek, Hoover, Monte, Inglis, Scott, Interlandi, Matteo, Katzenberger, Shon, Kazmi, Najeeb, Krivosheev, Gleb, Luferenko, Pete, Matantsev, Ivan, Matusevych, Sergiy, Moradi, Shahab, Nazirov, Gani, Ormont, Justin, Oshri, Gal, Pagnoni, Artidoro, Parmar, Jignesh, Roy, Prabhat, Shah, Sarthak, Siddiqui, Mohammad Zeeshan, Weimer, Markus, Zahirazami, Shauheen, Zhu, Yiwen
Machine Learning is transitioning from an art and science into a technology available to every developer. In the near future, every application on every platform will incorporate trained models to encode data-based decisions that would be impossible for developers to author. This presents a significant engineering challenge, since currently data science and modeling are largely decoupled from standard software development processes. This separation makes incorporating machine learning capabilities inside applications unnecessarily costly and difficult, and furthermore discourage developers from embracing ML in first place. In this paper we present ML .NET, a framework developed at Microsoft over the last decade in response to the challenge of making it easy to ship machine learning models in large software applications. We present its architecture, and illuminate the application demands that shaped it. Specifically, we introduce DataView, the core data abstraction of ML .NET which allows it to capture full predictive pipelines efficiently and consistently across training and inference lifecycles. We close the paper with a surprisingly favorable performance study of ML .NET compared to more recent entrants, and a discussion of some lessons learned.