Time to Choose TensorFlow Data over ImageDataGenerator
Generating training and validation batches with tf.data is way faster than ImageDataGenerator. Let's compare; First, we use ImageDataGenerator without using any augmentation -- Now we will use tf.data -- So tf.data is about 34 times faster than ImageDataGenerator and one of the main reasons for that is a technique called'Prefetching'. In the TensorFlow documentation series of examples are given with excellent explanations but here in brief what is happening. The data pipeline can be thought of as a combination of'producer' (generating batches) and'consumer' (batches that are used to train the neural net) and prefetch transformation provides benefits whenever there is an opportunity to overlap the work of a'producer' with the work of a'consumer'.
Sep-5-2021, 03:43:56 GMT
- Technology: