I'm trying to make a neural network that can identify if there is a cat in a picture. I found this tutorial on tensorflow website and tried to adapt it to my problem. The tutorial is for classifying cats and dogs, but since I only want to detect cats, I changed the categories to cats and non-cats.
For non-cats I downloaded a dataset of random images.
I added two generators to the code from the tutorial:
test_data_gen = test_image_generator.flow_from_directory(batch_size=batch_size, directory=test_dir, target_size=(IMG_HEIGHT, IMG_WIDTH), class_mode='binary') pred_data_gen = pred_image_generator.flow_from_directory(batch_size=batch_size, directory=pred_dir, target_size=(IMG_HEIGHT, IMG_WIDTH), class_mode='binary') And then tested the model like this:
print('\n# Evaluate on test data') results = model.evaluate_generator(test_data_gen) print('test loss, test acc:', results) print('\n# Generate predictions') predictions = model.predict(pred_data_gen) print(len(predictions)) print(predictions) That's the output:
# Evaluate on test data test loss, test acc: [0.45212748232815003, 0.9324082] # Generate predictions for custom samples 256 [[ -8.023465 ] [ -7.781438 ] [ 50.281197 ] [-10.172492 ] [ -5.1096087 ] [ 43.0299 ] [ 21.416649 ] ... [-10.866359 ] [-14.797473 ] [ 84.72212 ] [ 23.712345 ] [ -6.4916744 ] [-18.384903 ] [ 33.10642 ]] The test accuracy is very high, but I have no idea what these results mean. I thought they should be between 0 and 1, but these even have negative values. How should I interpret these results?
EDIT:
This is my model (before adding the sigmoid activation function to the last layer):
model = Sequential([ Conv2D(16, 3, padding='same', activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH, 3)), MaxPooling2D(), Conv2D(32, 3, padding='same', activation='relu'), MaxPooling2D(), Conv2D(64, 3, padding='same', activation='relu'), MaxPooling2D(), Flatten(), Dense(512, activation='relu'), Dense(1) ]) I changed the last layer to Dense(1, activation='sigmoid') and the output looks like this:
# Evaluate on test data test loss, test acc: [0.714477022488912, 0.5949367] # Generate predictions for custom samples 256 [[1.] [1.] [1.] ... [1.] [1.] [1.]] All predicted values are ones, even though only half of the images are cats in the test set.
EDIT2:
Here is how I compile and fit the model:
model.compile(optimizer='adam', loss=tf.keras.losses.BinaryCrossentropy(from_logits=True), metrics=['accuracy']) model.summary() history = model.fit_generator( train_data_gen, steps_per_epoch=total_train // batch_size, epochs=epochs, validation_data=val_data_gen, validation_steps=total_val // batch_size ) 3 Answers
The last layer in your model is a Dense layer with a single neuron. Since no parameters have been passed to it, by default it has linear activation. User @Code Pope indicated that it is a "failure" (mistake in Tensorflow docs?). It isn't, the code is perfectly fine.
Your loss function computes binary crossentropy and it can easily work with linear activation. In fact computing sigmoid is just a little extra that is not needed. The ANN will output negative values for one class and positive for the other one. They are not normalised, but are so-called logits - that's why you are saying in your loss function from_logits=True.
How to get predictions:
from sklearn.metrics import accuracy_score images, actual = next(train_data_gen) predictions = model.predict(images) predictions = (predictions > 0).flatten() accuracy_score(results, pred) You need to use sigmoid function for scale outputs to [0, 1]
2In the sample you have linked (here, there is failure. In the last Dense layer, there must be a sigmoid function to have a value between 0 and 1. The original sample is from the book Deep Learning for Python from Francois Chollet and the sample is from chapter 5.2. The sample is available on github and I recommend you to use it: here. I have used the sample and it works fine. Your edited model looks fine, but I am not sure if you have created the right folder structure for your images and therefore for your ImageDataGenerator. I have trained the model and then loaded an image to predict:
from keras.preprocessing import image from keras.applications.vgg16 import preprocess_input, decode_predictions import numpy as np img_path = '/Users/CodePope/Downloads/catsdogs/cats_and_dogs_smalla/test/dogs/dog.1500.jpg' img = image.load_img(img_path, target_size=(150,150)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x /=255. model.predict(x) The result is the following:
array([[0.8290838]], dtype=float32) which looks fine, as 0 is for cat and 1 is for dog. For a cat image we have:
from keras.preprocessing import image from keras.applications.vgg16 import preprocess_input, decode_predictions import numpy as np img_path = '/Users/CodePope/Downloads/catsdogs/cats_and_dogs_smalla/test/cats/cat.1500.jpg' img = image.load_img(img_path, target_size=(150,150)) x = image.img_to_array(img) x = np.expand_dims(x, axis=0) x /=255. model.predict(x) Result:
array([[0.15701984]], dtype=float32) And this is in spite of the fact that I have just trained the model for five epochs with a validation accuracy of 0.67.
4
