xbe.at

Python / Python for ML and AI

Understanding Keras Model with Python Examples

6 min read, 15 sections

Contents
  1. Introduction to Keras
  2. Sequential Model
  3. Compiling the Model
  4. Training the Model
  5. Model Evaluation
  6. Making Predictions
  7. Saving and Loading Models
  8. Custom Layers
  9. Callbacks
  10. Functional API
  11. Data Preprocessing
  12. Transfer Learning
  13. Real-Life Example: Image Classification
  14. Real-Life Example: Sentiment Analysis
  15. Additional Resources

Introduction to Keras

Keras is a high-level neural network API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation and ease of use, making it an excellent choice for both beginners and experienced deep learning practitioners.

import keras
print(keras.__version__)

Sequential Model

The Sequential model is the simplest type of model in Keras. It's a linear stack of layers where you can add one layer at a time. This model is suitable for a wide range of problems and is particularly easy to use for beginners.

from keras.models import Sequential
from keras.layers import Dense

model = Sequential([
    Dense(64, activation='relu', input_shape=(10,)),
    Dense(64, activation='relu'),
    Dense(1, activation='sigmoid')
])

model.summary()

Compiling the Model

Before training, we need to compile the model. This step configures the learning process by specifying the optimizer, loss function, and metrics to track during training.

model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

# Output:
# Model compiled successfully

Training the Model

Training the model involves feeding it with data, allowing it to learn patterns and adjust its weights. We use the fit() method for this purpose, specifying the number of epochs and batch size.

import numpy as np

# Generate dummy data
data = np.random.random((1000, 10))
labels = np.random.randint(2, size=(1000, 1))

# Train the model
history = model.fit(data, labels, epochs=10, batch_size=32, validation_split=0.2)

print(history.history['accuracy'][-1])

Model Evaluation

After training, it's crucial to evaluate the model's performance on unseen data. Keras provides the evaluate() method for this purpose.

# Generate test data
test_data = np.random.random((100, 10))
test_labels = np.random.randint(2, size=(100, 1))

# Evaluate the model
loss, accuracy = model.evaluate(test_data, test_labels)
print(f"Test accuracy: {accuracy}")

Making Predictions

Once the model is trained and evaluated, we can use it to make predictions on new data using the predict() method.

# Generate new data for prediction
new_data = np.random.random((5, 10))

# Make predictions
predictions = model.predict(new_data)
print("Predictions:")
print(predictions)

Saving and Loading Models

Keras allows you to save your trained models for later use or deployment. You can save both the model architecture and weights.

# Save the model
model.save('my_model.h5')

# Load the model
from keras.models import load_model
loaded_model = load_model('my_model.h5')

# Verify the loaded model
loaded_model.summary()

Custom Layers

Keras allows you to create custom layers by subclassing the Layer class. This is useful when you need functionality not provided by the built-in layers.

from keras import backend as K
from keras.layers import Layer

class MyLayer(Layer):
    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(MyLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        self.kernel = self.add_weight(name='kernel', 
                                      shape=(input_shape[1], self.output_dim),
                                      initializer='uniform',
                                      trainable=True)

    def call(self, x):
        return K.dot(x, self.kernel)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], self.output_dim)

# Use the custom layer
model = Sequential([MyLayer(64, input_shape=(10,))])
model.compile(optimizer='rmsprop', loss='mse')
model.summary()

Callbacks

Callbacks in Keras are powerful tools that allow you to customize the behavior of the model during training. They can be used for early stopping, model checkpointing, and more.

from keras.callbacks import EarlyStopping, ModelCheckpoint

early_stop = EarlyStopping(monitor='val_loss', patience=3)
checkpoint = ModelCheckpoint('best_model.h5', save_best_only=True)

history = model.fit(data, labels, epochs=50, batch_size=32, 
                    validation_split=0.2,
                    callbacks=[early_stop, checkpoint])

print(f"Training stopped after {len(history.history['loss'])} epochs")

Functional API

While the Sequential model is great for linear stacks of layers, the Functional API allows for more complex model architectures, including models with multiple inputs or outputs, shared layers, and non-linear topology.

from keras.layers import Input, Dense
from keras.models import Model

# Define inputs
input_a = Input(shape=(32,))
input_b = Input(shape=(32,))

# Define shared layers
shared_layer = Dense(16, activation='relu')

# Use the shared layer
x = shared_layer(input_a)
y = shared_layer(input_b)

# Output layer
output = Dense(1, activation='sigmoid')(keras.layers.concatenate([x, y]))

# Create and compile model
model = Model(inputs=[input_a, input_b], outputs=output)
model.compile(optimizer='rmsprop', loss='binary_crossentropy')

model.summary()

Data Preprocessing

Keras provides utilities for data preprocessing, which can be crucial for model performance. Here's an example of using the Tokenizer for text preprocessing.

from keras.preprocessing.text import Tokenizer

texts = ['The cat sat on the mat.', 'The dog ate my homework.']

tokenizer = Tokenizer(num_words=100)
tokenizer.fit_on_texts(texts)

sequences = tokenizer.texts_to_sequences(texts)
print("Sequences:", sequences)

word_index = tokenizer.word_index
print("Word Index:", word_index)

Transfer Learning

Transfer learning allows you to use pre-trained models for your own tasks. This is particularly useful when you have limited data or computational resources.

from keras.applications import VGG16
from keras.layers import Dense, GlobalAveragePooling2D
from keras.models import Model

# Load pre-trained VGG16 model
base_model = VGG16(weights='imagenet', include_top=False)

# Add custom layers
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(10, activation='softmax')(x)

# Create new model
model = Model(inputs=base_model.input, outputs=predictions)

# Freeze base model layers
for layer in base_model.layers:
    layer.trainable = False

model.compile(optimizer='rmsprop', loss='categorical_crossentropy')
model.summary()

Real-Life Example: Image Classification

Let's use Keras to build a simple Convolutional Neural Network (CNN) for classifying images of handwritten digits using the MNIST dataset.

from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPooling2D, Flatten
from keras.utils import to_categorical

# Load and preprocess data
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train = x_train.reshape(x_train.shape[0], 28, 28, 1).astype('float32') / 255
x_test = x_test.reshape(x_test.shape[0], 28, 28, 1).astype('float32') / 255
y_train = to_categorical(y_train)
y_test = to_categorical(y_test)

# Build model
model = Sequential([
    Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1)),
    MaxPooling2D(pool_size=(2, 2)),
    Flatten(),
    Dense(128, activation='relu'),
    Dense(10, activation='softmax')
])

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train model
history = model.fit(x_train, y_train, batch_size=128, epochs=5, validation_split=0.1)

# Evaluate model
score = model.evaluate(x_test, y_test)
print(f'Test accuracy: {score[1]}')

Real-Life Example: Sentiment Analysis

In this example, we'll use Keras to build a simple recurrent neural network for sentiment analysis on movie reviews.

from keras.datasets import imdb
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Embedding, LSTM, Dense

# Load data
max_features = 20000
maxlen = 80
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)

# Pad sequences
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)

# Build model
model = Sequential([
    Embedding(max_features, 128),
    LSTM(128, dropout=0.2, recurrent_dropout=0.2),
    Dense(1, activation='sigmoid')
])

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Train model
history = model.fit(x_train, y_train, batch_size=32, epochs=5, validation_split=0.2)

# Evaluate model
score = model.evaluate(x_test, y_test)
print(f'Test accuracy: {score[1]}')

Additional Resources

For those interested in diving deeper into Keras and its applications, here are some valuable resources:

  1. Keras Official Documentation: https://keras.io/
  2. "Deep Learning with Python" by François Chollet (creator of Keras)
  3. ArXiv paper: "Keras: Deep Learning for humans" by François Chollet (https://arxiv.org/abs/1806.01091)
  4. TensorFlow's Keras guide: https://www.tensorflow.org/guide/keras
  5. Keras GitHub repository: https://github.com/keras-team/keras

These resources provide comprehensive guides, tutorials, and research papers to further your understanding of Keras and its capabilities in deep learning.

More in Python for ML and AI

  1. Understanding LSTMs with Python10 min

    Long Short-Term Memory (LSTM) networks are a type of recurrent neural network designed to overcome the vanishing gradient problem in traditional RNNs. LSTMs...

  2. Transforming Document Summarization with Python8 min

    Document summarization is the process of distilling the most important information from a text while maintaining its core meaning. This technique is crucial...

  3. Understanding Optimizers in DSPy with Python7 min

    DSPy is a framework for building declarative language models and prompting-based systems. Optimizers play a crucial role in fine-tuning these models to...

  4. Top 10 Python AI Libraries and Their Uses11 min

    The field of Artificial Intelligence (AI) has been revolutionized by Python libraries that simplify complex tasks and accelerate development. This...

  5. Understanding Robust Scaler with Python8 min

    Robust Scaler is a preprocessing technique used in machine learning to scale features that may contain outliers. It uses statistics that are robust to...

  6. Time Series Regression Models in Python10 min

    Time series regression is a statistical method used to analyze and predict time-dependent data. In Python, various regression techniques can be applied to...