Can't Decide On Batch Size For Deep Learning Experiments? Yann LeCun Has A Funny Answer.

Batch Size has a massive impact on your model training and performance! Although large batch size can improve the available computational parallelism, it can cause degradation in the performance of the model. Small batch size is found to enhance the overall generalization of the model and also uses less memory. If a smaller batch size improves the model generalization and reduces the memory footprint, it is logical to use smaller batch sizes for model training. Yann LeCun, in his tweet, suggests that a batch size of 32 is the best option for optimum model training and performance.