this one helps. If your targets are onehot encoded, use categorical_crossentropy. Examples of onehot encodings: code :
[1,0,0]
[0,1,0]
[0,0,1]
1
2
3
Share :

How is the categorical_crossentropy implemented in keras?
By : Ugyen Rinzin
Date : March 29 2020, 07:55 AM
it helps some times I see that you used the tensorflow tag, so I guess this is the backend you are using? code :
def categorical_crossentropy(output, target, from_logits=False):
"""Categorical crossentropy between an output tensor and a target tensor.
# Arguments
output: A tensor resulting from a softmax
(unless `from_logits` is True, in which
case `output` is expected to be the logits).
target: A tensor of the same shape as `output`.
from_logits: Boolean, whether `output` is the
result of a softmax, or is a tensor of logits.
# Returns
Output tensor.
# Note: tf.nn.softmax_cross_entropy_with_logits
# expects logits, Keras expects probabilities.
if not from_logits:
# scale preds so that the class probas of each sample sum to 1
output /= tf.reduce_sum(output,
reduction_indices=len(output.get_shape())  1,
keep_dims=True)
# manual computation of crossentropy
epsilon = _to_tensor(_EPSILON, output.dtype.base_dtype)
output = tf.clip_by_value(output, epsilon, 1.  epsilon)
return  tf.reduce_sum(target * tf.log(output),
reduction_indices=len(output.get_shape())  1)

Keras: binary_crossentropy & categorical_crossentropy confusion
By : Austin Lin
Date : March 29 2020, 07:55 AM
This might help you You are right by defining areas where each of these losses are applicable: binary_crossentropy (and tf.nn.sigmoid_cross_entropy_with_logits under the hood) is for binary multilabel classification (labels are independent). categorical_crossentropy (and tf.nn.softmax_cross_entropy_with_logits under the hood) is for multiclass classification (classes are exclusive).

Keras: Big onehotencoding: binary_crossentropy or categorical_crossentropy
By : Hanyin Fang
Date : March 29 2020, 07:55 AM
will help you In short: the (high) accuracy reported when you use loss='binary_crossentropy' is not the correct one, as you already have guessed. For your problem, the recommended loss is categorical_crossentropy. code :
model.compile(loss='categorical_crossentropy',
optimizer=keras.optimizers.Adam(),
metrics=['accuracy']
model.compile(loss='binary_crossentropy',
optimizer=keras.optimizers.Adam(),
metrics=['accuracy'])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) # WRONG way
model.fit(x_train, y_train,
batch_size=batch_size,
epochs=2, # only 2 epochs, for demonstration purposes
verbose=1,
validation_data=(x_test, y_test))
# Keras reported accuracy:
score = model.evaluate(x_test, y_test, verbose=0)
score[1]
# 0.9975801164627075
# Actual accuracy calculated manually:
import numpy as np
y_pred = model.predict(x_test)
acc = sum([np.argmax(y_test[i])==np.argmax(y_pred[i]) for i in range(10000)])/10000
acc
# 0.98780000000000001
score[1]==acc
# False
from keras.metrics import categorical_accuracy
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=[categorical_accuracy])
# Keras reported accuracy:
score = model.evaluate(x_test, y_test, verbose=0)
score[1]
# 0.98580000000000001
# Actual accuracy calculated manually:
y_pred = model.predict(x_test)
acc = sum([np.argmax(y_test[i])==np.argmax(y_pred[i]) for i in range(10000)])/10000
acc
# 0.98580000000000001
score[1]==acc
# True
Python version 3.5.3
Tensorflow version 1.2.1
Keras version 2.0.4

Is it scaled twice in keras code categorical_crossentropy?
By : LucidN21
Date : March 29 2020, 07:55 AM
this one helps. Because you need to make sure that each probability is between 0 and 1, else the crossentropy computation will be incorrect. Its a way to also prevent user errors when they make (unnormalized) probabilities outside that range.

Does tf.keras.losses.categorical_crossentropy return an array or a single value?
By : tinyman23
Date : March 29 2020, 07:55 AM
I hope this helps you . If you have a prediction shape of (samples of batch, classes) tf.keras.losses.categorical_crossentropy returns the losses in the shape of (samples of batch,). So, if your labels are: code :
[[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
[[0.9 0.05 0.05]
[0.5 0.89 0.6 ]
[0.05 0.01 0.94]]
[0.10536055 0.8046684 0.06187541]
loss = tf.keras.backend.mean(losses)

