A small, randomly selected subset of a batch processed in one iteration . The batch size of a mini-batch is usually between 10 and 1,000 examples.1
For example, suppose the entire training set (the full batch) consists of 1,000 examples. Further suppose that you set the batch size of each mini-batch to 20. Therefore, each iteration determines the loss on a random 20 of the 1,000 examples and then adjusts the weights and biases accordingly.1
It is much more efficient to calculate the loss on a mini-batch than the loss on all the examples in the full batch.1
목차 목차 관련 글 역전파 “The system aggregates the losses for all the examples to compute the total loss for the current batch.” ML crash course - Linear regression “Mean squared error: The average of L2 losses across a set of examples.” Epoch (machine learning) “An epoch represents N/batch size training iterations, where N is the total number of examples.” Learning rate “For example, a learning rate of 0.3 would adjust weights and biases three times more powerfully than a learning…” Training loss “A metric representing a model's loss during a particular training iteration.” Introduction to Machine Learning 관련 절: Definition of [[Machine learning|ML]] 경사하강법 “Gradient descent is older--much, much older--than machine learning.” Self-training (machine learning) “Notice that each iteration of Step 2 adds more labeled examples for Step 1 to train on.” 손실 함수 “During training or testing, a mathematical function that calculates the loss on a batch of examples.” Machine Learning Systems “Readers will develop the ability to reason about ML system architectures and apply enduring engineering princip…” 모델 (기계학습) “Unsupervised machine learning also generates models, typically a function that can map an input example to the …” Generalization (machine learning) “You train a model on the examples in the training set.” Example (machine learning) “Examples in supervised learning fall into two general categories:” Downsampling (machine learning) “Training on a disproportionately low percentage of over-represented class examples in order to improve model tr…” Parameter (machine learning) “The weights and biases that a model learns during training.” Hyperparameter “You could set the learning rate to 0.01 before one training session.” 수렴 (기계학습) “A state reached when loss values change very little or not at all with each iteration.” 지도 학습 “Supervised machine learning is analogous to learning a subject by studying a set of questions and their corresp…” Mean squared error “Divide the L2 loss by the number of examples in the batch.” Test set “For instance, a single example shouldn't belong to both the training set and the test set.”