Loss (Cost) Function
A loss function or cost function calculates the difference between true and estimated values.
Machine Learning models are trained to minimize a loss function.
Stochastic gradient descent and backpropagation are examples of techniques used to reduce loss.
During model training, loss is reduced to an optimal level taking into account measures such as bias and variance.
Loss Function Examples
Examples include:
Bayesian Expected Loss - using a posterior probability
Classification Loss Functions - used in classification models
Discounted Maximum Loss - aka worst-case risk measure
Hinge Loss - used for "maximum-margin" classification such as Support Vector Machines
Mean Squared Error - the average squared difference between the estimated values and the actual value
0-1 Loss Function - for binary loss values