Introducing the Team Data Science Process from Microsoft

TDSP provides recommendations for managing shared analytics and storage infrastructure, including cloud file systems for storing datasets, databases, Big Data clusters (Hadoop, Spark), machine learning services, etc., both on the cloud and on-premises. This is where raw and processed datasets are stored, enabling reproducible analysis. It also avoids duplication, which could lead to inconsistencies and additional infrastructure costs. Scripts are provided to provision the shared resources, track them and allow each team member to connect to those resources securely. Our data science team uses the Microsoft Data Science Virtual Machine as our cloud development environment.