xbe.at

Machine learning / Fundamentals and overview

Building a Machine Learning Model The Complete Workflow in Python

4 min read, 15 sections

Contents
  1. The Machine Learning Workflow
  2. Data Collection and Importing
  3. Data Exploration and Visualization
  4. Data Preprocessing
  5. Feature Selection and Engineering
  6. Splitting the Data
  7. Model Selection
  8. Model Training
  9. Model Evaluation
  10. Hyperparameter Tuning
  11. Model Interpretation
  12. Model Deployment
  13. Real-Life Example: Predicting Customer Churn
  14. Real-Life Example: Image Classification for Plant Disease Detection
  15. Additional Resources

The Machine Learning Workflow

Machine learning is a powerful tool for solving complex problems. This slideshow will guide you through the complete workflow of building a machine learning model using Python, from data preparation to model deployment.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix

# This code sets up the basic libraries we'll use throughout the workflow

Data Collection and Importing

The first step in any machine learning project is gathering and importing data. We'll use the popular Iris dataset as an example.

from sklearn.datasets import load_iris

# Load the Iris dataset
iris = load_iris()
X, y = iris.data, iris.target

# Create a DataFrame for easier data manipulation
df = pd.DataFrame(X, columns=iris.feature_names)
df['target'] = y

print(df.head())

Data Exploration and Visualization

Understanding your data is crucial. Let's visualize the relationships between features using a scatter plot matrix.

pd.plotting.scatter_matrix(df.iloc[:, :4], figsize=(10, 10))
plt.tight_layout()
plt.show()

# This creates a matrix of scatter plots for all pairs of features

Data Preprocessing

Data preprocessing involves handling missing values, encoding categorical variables, and scaling numerical features.

# Check for missing values
print(df.isnull().sum())

# Scale the features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

print("Original data:\n", X[:2])
print("\nScaled data:\n", X_scaled[:2])

Feature Selection and Engineering

Selecting relevant features and creating new ones can significantly improve model performance. Let's create a new feature as an example.

# Create a new feature: petal area
df['petal_area'] = df['petal length (cm)'] * df['petal width (cm)']

# Visualize the new feature
plt.scatter(df['petal_area'], df['target'])
plt.xlabel('Petal Area')
plt.ylabel('Species')
plt.show()

Splitting the Data

Before training our model, we need to split our data into training and testing sets.

X = df.drop('target', axis=1)
y = df['target']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

print("Training set shape:", X_train.shape)
print("Testing set shape:", X_test.shape)

Model Selection

Choosing the right model depends on your problem and data. We'll use logistic regression for this example.

model = LogisticRegression(random_state=42)

# Train the model
model.fit(X_train, y_train)

print("Model coefficients:", model.coef_)
print("Model intercept:", model.intercept_)

Model Training

During training, the model learns patterns from the data to make predictions.

# Make predictions on the test set
y_pred = model.predict(X_test)

# Calculate accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Model accuracy: {accuracy:.2f}")

Model Evaluation

Evaluating your model helps you understand its performance and identify areas for improvement.

# Create a confusion matrix
cm = confusion_matrix(y_test, y_pred)

plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)
plt.title('Confusion Matrix')
plt.colorbar()
plt.xlabel('Predicted label')
plt.ylabel('True label')
plt.show()

Hyperparameter Tuning

Optimizing model parameters can lead to better performance. We'll use GridSearchCV for this task.

from sklearn.model_selection import GridSearchCV

param_grid = {'C': [0.1, 1, 10, 100], 'penalty': ['l1', 'l2']}
grid_search = GridSearchCV(LogisticRegression(random_state=42), param_grid, cv=5)
grid_search.fit(X_train, y_train)

print("Best parameters:", grid_search.best_params_)
print("Best cross-validation score:", grid_search.best_score_)

Model Interpretation

Understanding how your model makes decisions is crucial for building trust and improving it.

import shap

# Create a SHAP explainer
explainer = shap.LinearExplainer(model, X_train)

# Calculate SHAP values
shap_values = explainer.shap_values(X_test)

# Visualize feature importance
shap.summary_plot(shap_values, X_test, plot_type="bar")

Model Deployment

Once satisfied with your model's performance, you can deploy it to make predictions on new data.

import joblib

# Save the model
joblib.dump(model, 'iris_model.joblib')

# Load the model (in a new session or application)
loaded_model = joblib.load('iris_model.joblib')

# Make predictions with the loaded model
new_data = [[5.1, 3.5, 1.4, 0.2]]  # Example: features of a new flower
prediction = loaded_model.predict(new_data)
print("Predicted species:", iris.target_names[prediction[0]])

Real-Life Example: Predicting Customer Churn

Let's apply our workflow to predict customer churn for a telecom company.

# Assuming we have a DataFrame 'telecom_df' with customer data
X = telecom_df.drop('Churn', axis=1)
y = telecom_df['Churn']

# Preprocess data (handle categorical variables, scale features, etc.)
# Split data, train model, evaluate performance

# Example: Feature importance analysis
feature_importance = pd.DataFrame({'feature': X.columns, 'importance': model.coef_[0]})
feature_importance = feature_importance.sort_values('importance', ascending=False)
print(feature_importance.head())

Real-Life Example: Image Classification for Plant Disease Detection

Machine learning can be used to identify plant diseases from leaf images, helping farmers take timely action.

from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense

# Assuming we have a directory structure with images of healthy and diseased leaves
datagen = ImageDataGenerator(rescale=1./255, validation_split=0.2)

train_generator = datagen.flow_from_directory(
    'plant_images',
    target_size=(224, 224),
    batch_size=32,
    class_mode='binary',
    subset='training')

# Build and train a simple CNN model
model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(224, 224, 3)),
    MaxPooling2D(2, 2),
    Flatten(),
    Dense(1, activation='sigmoid')
])

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(train_generator, epochs=10)

Additional Resources

For those interested in diving deeper into machine learning, here are some valuable resources:

  1. "Deep Learning" by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (arXiv:1602.03929)
  2. "Machine Learning: A Probabilistic Perspective" by Kevin P. Murphy (not available on arXiv)
  3. "Practical Machine Learning with Python" tutorial series on ArXiv (arXiv:2006.16632)

Remember to always refer to the official documentation of the libraries used in this workflow for the most up-to-date information.

More in Fundamentals and overview

  1. Comparing Gini Impurity and Entropy in Decision Trees14 min

    The Gini impurity metric quantifies the probability of incorrect classification when randomly choosing an element from a dataset based on the class...

  2. Beginner's Guide to Python for Data Science and Machine Learning8 min

    Pandas is a powerful data manipulation library that provides essential tools for data analysis. The first step in any data science project is loading and...

  3. Decoding the Decoder Understanding its Role in Machine Learning Models8 min

    The Decoder is a crucial component in many machine learning models, particularly in sequence-to-sequence architectures like transformers. It takes encoded...

  4. Beginner's Guide to Machine Learning with Python12 min

    Understanding fundamental data structures is crucial for efficient machine learning implementation. NumPy arrays and Pandas DataFrames form the backbone of...

  5. Demystifying Machine Learning When Theory Meets Practice13 min

    Linear algebra's foundation in ML starts with vectors. Vectors represent features, parameters, or data points in n-dimensional space, forming the basic...

  6. Balancing Accuracy and Interpretability in Machine Learning with Python11 min

    In machine learning, achieving high accuracy is often the primary goal. However, interpretability is equally important, especially in critical domains like...