Log loss
The loss function used in binary logistic regression.1
where
- is the dataset containing many labeled examples, wich are pairs.
- is the label in a labeled example. Since this is logistic regression, every value of must either be 0 or 1.
- is the predicted value (somewhere between 0 and 1, exclusive), given the set of features in .
Rationale
Squared loss works well for a linear regression where the rate of change of the output values is constant. However, the rate of change of a logistic regression model is not constant.
If you used squared loss to calculate errors for the sigmoid function, as the output got closer and closer to 0 and 1, you would need more memory to preserve the precision needed to track these values.
Instead, the loss function for logistic regression is log loss. The Log Loss equation returns the logarithm of the magnitude of the change, rather than just the distance from data to prediction.2