How to choose the best training instance on SageMaker

Like EC2 instances, Amazon offers a variety of instances for training in SageMaker. Based on CPU cores, memory size and presence of GPUs, they come with different on-demand prices. A complete list can be found at Amazon SageMaker Pricing Page. From my point of view, too many choices is as bad as having none or limited choices, when you don't have a clear idea how to choose. How do you handle the overwhelm of choosing an appropriate instance type for your training task on SageMaker?