Yee, Charles
Maneuver Identification Challenge
Samuel, Kaira, Gadepally, Vijay, Jacobs, David, Jones, Michael, McAlpin, Kyle, Palko, Kyle, Paulk, Ben, Samsi, Sid, Siu, Ho Chit, Yee, Charles, Kepner, Jeremy
AI algorithms that identify maneuvers from trajectory data could play an important role in improving flight safety and pilot training. AI challenges allow diverse teams to work together to solve hard problems and are an effective tool for developing AI solutions. AI challenges are also a key driver of AI computational requirements. The Maneuver Identification Challenge hosted at maneuver-id.mit.edu provides thousands of trajectories collected from pilots practicing in flight simulators, descriptions of maneuvers, and examples of these maneuvers performed by experienced pilots. Each trajectory consists of positions, velocities, and aircraft orientations normalized to a common coordinate system. Construction of the data set required significant data architecture to transform flight simulator logs into AI ready data, which included using a supercomputer for deduplication and data conditioning. There are three proposed challenges. The first challenge is separating physically plausible (good) trajectories from unfeasible (bad) trajectories. Human labeled good and bad trajectories are provided to aid in this task. Subsequent challenges are to label trajectories with their intended maneuvers and to assess the quality of those maneuvers.
The MIT Supercloud Dataset
Samsi, Siddharth, Weiss, Matthew L, Bestor, David, Li, Baolin, Jones, Michael, Reuther, Albert, Edelman, Daniel, Arcand, William, Byun, Chansup, Holodnack, John, Hubbell, Matthew, Kepner, Jeremy, Klein, Anna, McDonald, Joseph, Michaleas, Adam, Michaleas, Peter, Milechin, Lauren, Mullen, Julia, Yee, Charles, Price, Benjamin, Prout, Andrew, Rosa, Antonio, Vanterpool, Allan, McEvoy, Lindsey, Cheng, Anson, Tiwari, Devesh, Gadepally, Vijay
Artificial intelligence (AI) and Machine learning (ML) workloads are an increasingly larger share of the compute workloads in traditional High-Performance Computing (HPC) centers and commercial cloud systems. This has led to changes in deployment approaches of HPC clusters and the commercial cloud, as well as a new focus on approaches to optimized resource usage, allocations and deployment of new AI frame- works, and capabilities such as Jupyter notebooks to enable rapid prototyping and deployment. With these changes, there is a need to better understand cluster/datacenter operations with the goal of developing improved scheduling policies, identifying inefficiencies in resource utilization, energy/power consumption, failure prediction, and identifying policy violations. In this paper we introduce the MIT Supercloud Dataset which aims to foster innovative AI/ML approaches to the analysis of large scale HPC and datacenter/cloud operations. We provide detailed monitoring logs from the MIT Supercloud system, which include CPU and GPU usage by jobs, memory usage, file system logs, and physical monitoring data. This paper discusses the details of the dataset, collection methodology, data availability, and discusses potential challenge problems being developed using this data. Datasets and future challenge announcements will be available via https://dcc.mit.edu.