AGOCS -- Accurate Google Cloud Simulator Framework

This is the accepted author's version of the paper. The final published version is available in the Proceedings of the 2016 I EEE International Conferences on Ubiquitous Intelligence and Computing (UIC), Advanced and Trusted Computing (ATC), Scalable Compu ting and Communications (ScalCom), Cloud and Big Data Computing (CBDCom), Internet of People (IoP), and Smart World Congress (SmartWorld). Distributed and Intelligent Systems Research Group University of Westminster London, United Kingdom Leszek.Sliwko@my.westminster.ac.uk Distributed and Intelligent Systems Rese arch Group University of Westminster London, United Kingdom V.S.Getov@ westminster.ac.uk Abstract -- This paper presents the Accurate Google Cloud Simulator (AGOCS) - a novel high - fidelity Cloud workload simulator based on parsing real workload traces, whic h can be conveniently used on a desktop machine for day - to - day research. Our simulation is based on real - world workload traces from a Google Cluster with 12.5K nodes, over a period of a calendar month. The framework is able to reveal very precise and detai led parameters of the executed jobs, tasks and nodes as well as to provide actual resource usage statistics. The system has been implemented in Scala language with focus on parallel execution and an easy - to - extend design concept. The paper presents the det ailed structural framework for AGOCS and discusses our main design decisions, whilst also suggesting alternative and possibly performance enhancing future approaches. The framework is available via the Open Source GitHub repository. Correctly characterizing user behavior is of utmost importance when modeling Cloud workloads [14, 19] . Cloud workloads have been researched thoughtfully and are relativ ely wel l understood [15, 16, 24, 27]; however there have been limited attempts to accurately simulate Cloud workloads with consideration of detailed t ask parameters and constraints [6, 7, 10, 13], especially with consideration of workload scheduling [25]. Evaluat ing the performance of distributed applications and services without unrestricted access to existing Cloud environments is a very difficult task, which can also be addressed via simulation. Existing Cloud simulators do succeed in representing high - view inf rastructure parameters (i.e.