Binary Cross-Entropy Loss

On this page
Code implementation

PyTorch
TensorFlow

Further Resources
Report issue
Get help

Binary Cross-Entropy loss is a special case of Cross-Entropy loss used for multilabel classification (taggers). It is the cross entropy loss when there are only two classes involved. It is reliant on Sigmoid activation functions.

Mathematically, it is given as,

$$$BinaryC.E=-\sum_i^2 t_i log(p_i)$$$

Where $€€t_i€€$ is the true label and $€€p_i€€$ is the probability of the $€€i^{th}€€$ label.

Code implementation

PyTorch

    # importing the library
import torch
import torch.nn as nn

input = torch.randn(3, 5, requires_grad=True)

# Binary Cross-Entropy Loss

target = torch.ones([10, 64], dtype=torch.float32)  # 64 classes, batch size = 10
output = torch.full([10, 64], 1.5)  # A prediction (logit)

pos_weight = torch.ones([64])  # All weights are equal to 1
criterion = torch.nn.BCEWithLogitsLoss(pos_weight=pos_weight)
criterion(output, target)  # -log(sigmoid(1.5))
    python

TensorFlow

    # importing the library
import tensorflow as tf

y_true = [[0., 1.], [0., 0.]]
y_pred = [[0.6, 0.4], [0.4, 0.6]]

# Using 'auto'/'sum_over_batch_size' reduction type.
bce = tf.keras.losses.BinaryCrossentropy()
bce(y_true, y_pred).numpy()


# Calling with 'sample_weight'.
bce(y_true, y_pred, sample_weight=[1, 0]).numpy()
    python

Further Resources

Last modified 6mo ago

On this page
Code implementation

PyTorch
TensorFlow

Further Resources
Report issue
Get help