logo
down
shadow

Keras: Use categorical_crossentropy without one-hot encoded array of targets


Keras: Use categorical_crossentropy without one-hot encoded array of targets

By : Harvard Hustler
Date : November 21 2020, 07:01 PM
this one helps. If your targets are one-hot encoded, use categorical_crossentropy. Examples of one-hot encodings:
code :
[1,0,0]
[0,1,0]
[0,0,1]
1
2
3


Share : facebook icon twitter icon
How is the categorical_crossentropy implemented in keras?

How is the categorical_crossentropy implemented in keras?


By : Ugyen Rinzin
Date : March 29 2020, 07:55 AM
it helps some times I see that you used the tensorflow tag, so I guess this is the backend you are using?
code :
def categorical_crossentropy(output, target, from_logits=False):
"""Categorical crossentropy between an output tensor and a target tensor.
# Arguments
    output: A tensor resulting from a softmax
        (unless `from_logits` is True, in which
        case `output` is expected to be the logits).
    target: A tensor of the same shape as `output`.
    from_logits: Boolean, whether `output` is the
        result of a softmax, or is a tensor of logits.
# Returns
    Output tensor.
 # Note: tf.nn.softmax_cross_entropy_with_logits
# expects logits, Keras expects probabilities.
if not from_logits:
    # scale preds so that the class probas of each sample sum to 1
    output /= tf.reduce_sum(output,
                            reduction_indices=len(output.get_shape()) - 1,
                            keep_dims=True)
    # manual computation of crossentropy
    epsilon = _to_tensor(_EPSILON, output.dtype.base_dtype)
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    return - tf.reduce_sum(target * tf.log(output),
                          reduction_indices=len(output.get_shape()) - 1)
Keras: binary_crossentropy & categorical_crossentropy confusion

Keras: binary_crossentropy & categorical_crossentropy confusion


By : Austin Lin
Date : March 29 2020, 07:55 AM
This might help you You are right by defining areas where each of these losses are applicable:
binary_crossentropy (and tf.nn.sigmoid_cross_entropy_with_logits under the hood) is for binary multi-label classification (labels are independent). categorical_crossentropy (and tf.nn.softmax_cross_entropy_with_logits under the hood) is for multi-class classification (classes are exclusive).
Keras: Big one-hot-encoding: binary_crossentropy or categorical_crossentropy

Keras: Big one-hot-encoding: binary_crossentropy or categorical_crossentropy


By : Hanyin Fang
Date : March 29 2020, 07:55 AM
will help you In short: the (high) accuracy reported when you use loss='binary_crossentropy' is not the correct one, as you already have guessed. For your problem, the recommended loss is categorical_crossentropy.
code :
model.compile(loss='categorical_crossentropy',
          optimizer=keras.optimizers.Adam(),
          metrics=['accuracy']
model.compile(loss='binary_crossentropy',
          optimizer=keras.optimizers.Adam(),
          metrics=['accuracy'])
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])  # WRONG way

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=2,  # only 2 epochs, for demonstration purposes
          verbose=1,
          validation_data=(x_test, y_test))

# Keras reported accuracy:
score = model.evaluate(x_test, y_test, verbose=0) 
score[1]
# 0.9975801164627075

# Actual accuracy calculated manually:
import numpy as np
y_pred = model.predict(x_test)
acc = sum([np.argmax(y_test[i])==np.argmax(y_pred[i]) for i in range(10000)])/10000
acc
# 0.98780000000000001

score[1]==acc
# False    
from keras.metrics import categorical_accuracy
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=[categorical_accuracy])
# Keras reported accuracy:
score = model.evaluate(x_test, y_test, verbose=0) 
score[1]
# 0.98580000000000001

# Actual accuracy calculated manually:
y_pred = model.predict(x_test)
acc = sum([np.argmax(y_test[i])==np.argmax(y_pred[i]) for i in range(10000)])/10000
acc
# 0.98580000000000001

score[1]==acc
# True    
Python version 3.5.3
Tensorflow version 1.2.1
Keras version 2.0.4
Is it scaled twice in keras code categorical_crossentropy?

Is it scaled twice in keras code categorical_crossentropy?


By : LucidN21
Date : March 29 2020, 07:55 AM
this one helps. Because you need to make sure that each probability is between 0 and 1, else the cross-entropy computation will be incorrect. Its a way to also prevent user errors when they make (unnormalized) probabilities outside that range.
Does tf.keras.losses.categorical_crossentropy return an array or a single value?

Does tf.keras.losses.categorical_crossentropy return an array or a single value?


By : tinyman23
Date : March 29 2020, 07:55 AM
I hope this helps you . If you have a prediction shape of (samples of batch, classes) tf.keras.losses.categorical_crossentropy returns the losses in the shape of (samples of batch,).
So, if your labels are:
code :
[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
[[0.9  0.05 0.05]
 [0.5  0.89 0.6 ]
 [0.05 0.01 0.94]]
[0.10536055 0.8046684  0.06187541]
loss = tf.keras.backend.mean(losses)
Related Posts Related Posts :
  • access django session from a decorator
  • Multi-Threaded data insertion in MySQL using python
  • Making all variables accessible to namespace
  • What are the differences among sqlite3 from python2.5, pysqlite and apsw
  • Detect marker with opencv and python
  • Python ctypes and dynamic linking
  • Downloading a directory tree with ftplib
  • ImportError: No Module named simplejson
  • Pre-generating GUIDs for use in python?
  • `ipython` tab autocomplete does not work on imported module
  • Matching blank entries in django queryset for optional field with corresponding ones in a required field
  • Control VLC from Python in Windows
  • Return unicode string from python via ajax
  • Dynamically customize django admin columns?
  • Any way to add tabbed forms in django administration site?
  • Python / SQLite - database locked despite large timeouts
  • Encoding in XML declaration python
  • deletion of folders
  • Python Mindstorms RCX
  • Regex to split on successions of newline characters
  • Unicode filename to python subprocess.call()
  • Removing non-breaking spaces from strings using Python
  • Is there a way to backup everything in an app-engine blobstore?
  • Process a set of files from a source directory to a destination directory in Python
  • How to know if optparse option was passed in the command line or as a default
  • Rewriting a for loop in pure NumPy to decrease execution time
  • Comparing a time delta in python
  • Find next lower item in a sorted list
  • MySQLdb not INSERTING, _mysql does fine
  • Scipy Negative Distance? What?
  • What are the common patterns in web programming?
  • Percent signs in windows path
  • How to add a random number to a subsection of a numpy array?
  • How to generate all the values of an iterable besides the last few?
  • Searching by both class and range in XPath
  • Python code execution in Perl interpreter
  • Best Way to Include Variable in Python3
  • Serialize the @property methods in a Python class
  • What is the most platform- and Python-version-independent way to make a fast loop for use in Python?
  • Good way to edit the previous defined class in ipython
  • Bounced email on Google App Engine
  • Search jpeg files using python
  • Dynamically create class attributes
  • python unichr problem
  • Python beginner, strange output problem
  • Python: Finding a value in 1 list and finding that corresponding index in another list
  • can't install mysqlclient on mac os x mojave
  • Error indicates flattened dimensions when loading pre-trained network
  • how to replace underlines with words?
  • Adding through iteration
  • Use OpenCV on deployed Flask app (Heroku)
  • How to skip interstitial in a django view if a user hits the back button?
  • Any Naive Bayesian Classifier in python?
  • Python 2.5.2: remove what found between two lines that contain two concrete strings
  • Python 2.5.2 script that add "The function starts here" to all the functions of the files of a directory
  • HttpResponseRedirect question
  • Python socket error on UDP data receive. (10054)
  • Encoding issues with cloud ml
  • numpy.where - Weird behaviour: new elements spawning from nowhere?
  • I can't move my player in pygame, can you figure why?
  • shadow
    Privacy Policy - Terms - Contact Us © 35dp-dentalpractice.co.uk