mini-batch
- 2024-10-21
A small, randomly selected subset of a batch processed in one iteration. The batch size of a mini-batch is usually between 10 and 1,000 examples.1
For example, suppose the entire training set (the full batch) consists of 1,000 examples. Further suppose that you set the batch size of each mini-batch to 20. Therefore, each iteration determines the loss on a random 20 of the 1,000 examples and then adjusts the weights and biases accordingly.1
It is much more efficient to calculate the loss on a mini-batch than the loss on all the examples in the full batch.1