Choosing a loss function

Regression Problem

A problem where you predict a real-value quantity.

  • Output Layer Configuration: One node with a linear activation unit.
  • Loss Function: Mean Squared Error (MSE).

Binary Classification Problem

A problem where you classify an example as belonging to one of two classes.

The problem is framed as predicting the likelihood of an example belonging to class one, e.g. the class that you assign the integer value 1, whereas the other class is assigned the value 0.

  • Output Layer Configuration: One node with a sigmoid activation unit.
  • Loss Function: Cross-Entropy, also referred to as Logarithmic loss.

Multi-Class Classification Problem

A problem where you classify an example as belonging to one of more than two classes.

The problem is framed as predicting the likelihood of an example belonging to each class.

  • Output Layer Configuration: One node for each class using the softmax activation function.
  • Loss Function: Cross-Entropy, also referred to as Logarithmic loss.

Source: Loss and Loss Functions for Training Deep Learning Neural Networks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.