xbe.at

Machine learning / Advanced ML topics

Manim for Machine Learning in Python

9 min read, 16 sections

Contents
  1. Introduction to Manim for Machine Learning
  2. Setting Up Manim for ML Projects
  3. Visualizing a Simple Linear Regression
  4. Animating Gradient Descent
  5. Visualizing Decision Boundaries
  6. Animating Neural Network Architecture
  7. Visualizing K-Means Clustering
  8. Animating Principal Component Analysis (PCA)
  9. Visualizing Convolutional Neural Networks (CNN)
  10. Visualizing Recurrent Neural Networks (RNN)
  11. Visualizing t-SNE for Dimensionality Reduction
  12. Visualizing Support Vector Machines (SVM)
  13. Real-life Example: Image Classification
  14. Real-life Example: Image Classification
  15. Real-life Example: Natural Language Processing
  16. Additional Resources

Introduction to Manim for Machine Learning

Manim is a powerful animation library in Python, originally created by Grant Sanderson (3Blue1Brown) for mathematical animations. It has since evolved to become a versatile tool for creating high-quality animations, particularly useful in visualizing machine learning concepts. This slideshow will explore how Manim can be used to illustrate and explain various machine learning algorithms and concepts.

from manim import *

class IntroScene(Scene):
    def construct(self):
        title = Text("Manim for Machine Learning")
        subtitle = Text("Visualizing ML concepts with animations")
        
        self.play(Write(title))
        self.wait(1)
        self.play(title.animate.to_edge(UP))
        self.play(Write(subtitle))
        self.wait(2)

Setting Up Manim for ML Projects

Before diving into machine learning visualizations, it's crucial to set up Manim properly. This involves installing Manim and its dependencies, as well as importing necessary libraries for machine learning tasks.

# Install Manim (run in terminal or command prompt)
# pip install manim

# Import required libraries
from manim import *
import numpy as np
import sklearn.datasets as datasets

class MLSetup(Scene):
    def construct(self):
        code = Code(
            """
            from manim import *
            import numpy as np
            import sklearn.datasets as datasets
            """,
            language="python",
            font_size=24
        )
        self.play(Create(code))
        self.wait(2)

Visualizing a Simple Linear Regression

Linear regression is a fundamental machine learning algorithm. Let's use Manim to visualize how it works with a simple 2D example.

class LinearRegressionVisualization(Scene):
    def construct(self):
        # Generate sample data
        X = np.linspace(0, 10, 20)
        y = 2 * X + 1 + np.random.normal(0, 1, 20)
        
        # Create Axes
        axes = Axes(
            x_range=[0, 11],
            y_range=[0, 25],
            axis_config={"color": BLUE},
        )
        
        # Plot data points
        dots = VGroup(*[Dot(axes.c2p(x, y)) for x, y in zip(X, y)])
        
        # Create line of best fit
        line = axes.get_line_graph(
            x_values=X,
            y_values=2 * X + 1,
            line_color=RED
        )
        
        self.play(Create(axes), Create(dots))
        self.wait(1)
        self.play(Create(line))
        self.wait(2)

Animating Gradient Descent

Gradient descent is a crucial optimization algorithm in machine learning. Let's visualize how it works in finding the minimum of a simple 2D function.

class GradientDescentAnimation(Scene):
    def construct(self):
        def f(x, y):
            return x**2 + y**2
        
        axes = ThreeDAxes()
        surface = Surface(
            lambda u, v: axes.c2p(u, v, f(u, v)),
            u_range=(-2, 2),
            v_range=(-2, 2)
        )
        
        self.set_camera_orientation(phi=75 * DEGREES, theta=-30 * DEGREES)
        self.play(Create(axes), Create(surface))
        
        dot = Sphere(radius=0.05).move_to(axes.c2p(1.5, 1.5, f(1.5, 1.5)))
        
        def update_dot(dot, dt):
            x, y, _ = axes.p2c(dot.get_center())
            grad_x, grad_y = 2*x, 2*y
            x -= 0.1 * grad_x
            y -= 0.1 * grad_y
            dot.move_to(axes.c2p(x, y, f(x, y)))
        
        dot.add_updater(update_dot)
        self.play(Create(dot))
        self.wait(5)
        dot.remove_updater(update_dot)

Visualizing Decision Boundaries

Decision boundaries are crucial in classification problems. Let's use Manim to visualize how a simple decision boundary separates two classes in a 2D space.

class DecisionBoundaryVisualization(Scene):
    def construct(self):
        # Generate sample data
        X, y = datasets.make_blobs(n_samples=100, centers=2, random_state=42)
        
        # Create Axes
        axes = Axes(
            x_range=[-4, 4],
            y_range=[-4, 4],
            axis_config={"color": BLUE},
        )
        
        # Plot data points
        dots = VGroup(*[Dot(axes.c2p(x[0], x[1]), color=RED if y_==0 else BLUE) for x, y_ in zip(X, y)])
        
        # Create decision boundary (simplified as a straight line)
        line = Line(axes.c2p(-4, -3), axes.c2p(4, 3), color=GREEN)
        
        self.play(Create(axes), Create(dots))
        self.wait(1)
        self.play(Create(line))
        self.wait(2)

Animating Neural Network Architecture

Neural networks are a cornerstone of modern machine learning. Let's use Manim to create an animation of a simple feedforward neural network architecture.

class NeuralNetworkAnimation(Scene):
    def construct(self):
        layers = [3, 4, 4, 2]
        network = VGroup()
        
        for i, layer_size in enumerate(layers):
            layer = VGroup(*[Circle(radius=0.2) for _ in range(layer_size)])
            layer.arrange(DOWN, buff=0.5)
            network.add(layer)
        
        network.arrange(RIGHT, buff=1)
        
        edges = VGroup()
        for layer1, layer2 in zip(network[:-1], network[1:]):
            for neuron1 in layer1:
                for neuron2 in layer2:
                    edge = Line(neuron1.get_center(), neuron2.get_center(), stroke_opacity=0.5)
                    edges.add(edge)
        
        self.play(Create(network))
        self.wait(1)
        self.play(Create(edges))
        self.wait(2)

Visualizing K-Means Clustering

K-Means clustering is a popular unsupervised learning algorithm. Let's use Manim to visualize how it works on a 2D dataset.

class KMeansVisualization(Scene):
    def construct(self):
        # Generate sample data
        X, _ = datasets.make_blobs(n_samples=100, centers=3, random_state=42)
        
        # Create Axes
        axes = Axes(
            x_range=[-4, 4],
            y_range=[-4, 4],
            axis_config={"color": BLUE},
        )
        
        # Plot data points
        dots = VGroup(*[Dot(axes.c2p(x[0], x[1])) for x in X])
        
        # Initialize centroids
        centroids = VGroup(*[Dot(axes.c2p(np.random.uniform(-3, 3), np.random.uniform(-3, 3)), color=RED) for _ in range(3)])
        
        self.play(Create(axes), Create(dots))
        self.wait(1)
        self.play(Create(centroids))
        
        # Animate centroid updates (simplified)
        for _ in range(3):
            new_centroids = VGroup(*[Dot(axes.c2p(np.mean(X[:, 0]), np.mean(X[:, 1])), color=RED) for _ in range(3)])
            self.play(Transform(centroids, new_centroids))
            self.wait(1)

Animating Principal Component Analysis (PCA)

PCA is a dimensionality reduction technique widely used in machine learning. Let's visualize how PCA finds the principal components of a 2D dataset.

class PCAVisualization(Scene):
    def construct(self):
        # Generate correlated data
        X = np.random.multivariate_normal([0, 0], [[2, 1.5], [1.5, 2]], size=100)
        
        # Create Axes
        axes = Axes(
            x_range=[-5, 5],
            y_range=[-5, 5],
            axis_config={"color": BLUE},
        )
        
        # Plot data points
        dots = VGroup(*[Dot(axes.c2p(x[0], x[1])) for x in X])
        
        # Calculate principal components
        _, eigenvectors = np.linalg.eig(np.cov(X.T))
        pc1 = eigenvectors[:, 0]
        pc2 = eigenvectors[:, 1]
        
        # Create arrows for principal components
        arrow1 = Arrow(start=axes.c2p(0, 0), end=axes.c2p(pc1[0]*3, pc1[1]*3), color=RED)
        arrow2 = Arrow(start=axes.c2p(0, 0), end=axes.c2p(pc2[0]*3, pc2[1]*3), color=GREEN)
        
        self.play(Create(axes), Create(dots))
        self.wait(1)
        self.play(Create(arrow1), Create(arrow2))
        self.wait(2)

Visualizing Convolutional Neural Networks (CNN)

CNNs are crucial for image processing tasks in machine learning. Let's create a simplified visualization of how a convolutional layer works.

class CNNVisualization(Scene):
    def construct(self):
        # Create input image
        input_image = Rectangle(height=3, width=3, fill_color=BLUE, fill_opacity=0.5)
        input_image.to_edge(LEFT)
        
        # Create kernel
        kernel = Square(side_length=1, fill_color=RED, fill_opacity=0.5)
        kernel.next_to(input_image, RIGHT)
        
        # Create output feature map
        output = Rectangle(height=2, width=2, fill_color=GREEN, fill_opacity=0.5)
        output.to_edge(RIGHT)
        
        self.play(Create(input_image), Create(kernel), Create(output))
        self.wait(1)
        
        # Animate convolution operation
        for i in range(2):
            for j in range(2):
                self.play(kernel.animate.move_to(input_image.get_center() + np.array([i-0.5, j-0.5, 0])))
                self.wait(0.5)
        
        self.wait(2)

Visualizing Recurrent Neural Networks (RNN)

RNNs are essential for sequence data in machine learning. Let's create a simple visualization of an RNN unrolled over time.

class RNNVisualization(Scene):
    def construct(self):
        def create_cell():
            return Circle(radius=0.5, fill_color=BLUE, fill_opacity=0.5)
        
        cells = VGroup(*[create_cell() for _ in range(4)])
        cells.arrange(RIGHT, buff=1.5)
        
        arrows = VGroup(*[Arrow(start=c1.get_right(), end=c2.get_left()) for c1, c2 in zip(cells[:-1], cells[1:])])
        
        loop_arrows = VGroup(*[CurvedArrow(start_point=c.get_top(), end_point=c.get_top()+UP*0.5, angle=-TAU/4) for c in cells])
        
        input_arrows = VGroup(*[Arrow(start=c.get_bottom()+DOWN*0.5, end=c.get_bottom()) for c in cells])
        output_arrows = VGroup(*[Arrow(start=c.get_top(), end=c.get_top()+UP*0.5) for c in cells])
        
        self.play(Create(cells))
        self.wait(1)
        self.play(Create(arrows), Create(loop_arrows))
        self.wait(1)
        self.play(Create(input_arrows), Create(output_arrows))
        self.wait(2)

Visualizing t-SNE for Dimensionality Reduction

t-SNE (t-Distributed Stochastic Neighbor Embedding) is a popular technique for visualizing high-dimensional data. Let's create an animation showing how t-SNE might transform a dataset.

class TSNEVisualization(Scene):
    def construct(self):
        # Create initial high-dimensional representation
        dots_initial = VGroup(*[Dot(np.array([np.random.uniform(-4, 4), np.random.uniform(-4, 4), 0])) for _ in range(50)])
        
        # Create final 2D embedding
        dots_final = VGroup(*[Dot(np.array([np.random.uniform(-4, 4), np.random.uniform(-4, 4), 0])) for _ in range(50)])
        
        # Color dots based on clusters
        colors = [RED, BLUE, GREEN]
        for i, dot in enumerate(dots_final):
            dot.set_color(colors[i % 3])
        
        self.play(Create(dots_initial))
        self.wait(1)
        self.play(Transform(dots_initial, dots_final))
        self.wait(2)

Visualizing Support Vector Machines (SVM)

SVMs are powerful classifiers that find the optimal hyperplane to separate classes. Let's visualize how SVM works in 2D space.

class SVMVisualization(Scene):
    def construct(self):
        # Create Axes
        axes = Axes(
            x_range=[-5, 5],
            y_range=[-5, 5],
            axis_config={"color": BLUE},
        )
        
        # Generate sample data
        X, y = datasets.make_classification(n_samples=50, n_features=2, n_informative=2, n_redundant=0, n_clusters_per_class=1, random_state=42)
        
        # Plot data points
        dots = VGroup(*[Dot(axes.c2p(x[0], x[1]), color=RED if y_==0 else BLUE) for x, y_ in zip(X, y)])
        
        # Create decision boundary
        line = Line(axes.c2p(-5, -3), axes.c2p(5, 3), color=GREEN)
        
        # Create margin lines
        margin1 = Line(axes.c2p(-5, -4), axes.c2p(5, 2), color=YELLOW, stroke_opacity=0.5)
        margin2 = Line(axes.c2p(-5, -2), axes.c2p(5, 4), color=YELLOW, stroke_opacity=0.5)
        
        self.play(Create(axes), Create(dots))
        self.wait(1)
        self.play(Create(line))
        self.wait(1)
        self.play(Create(margin1), Create(margin2))
        self.wait(2)

Real-life Example: Image Classification

Let's visualize how a Convolutional Neural Network (CNN) might process an image for classification.

class ImageClassificationExample(Scene):
    def construct(self):
        # Create input image
        input_image = Rectangle(height=3, width=3, fill_color=BLUE, fill_opacity=0.5)
        input_image.to_edge(LEFT)
        
        # Create convolutional

Real-life Example: Image Classification

Let's visualize how a Convolutional Neural Network (CNN) processes an image for classification, such as identifying objects in photographs.

class ImageClassificationExample(Scene):
    def construct(self):
        # Create input image
        input_image = ImageMobject("cat.jpg").scale(0.5)
        input_image.to_edge(LEFT)
        
        # Create convolutional layers
        conv_layers = VGroup(*[Rectangle(height=0.5, width=0.5) for _ in range(3)])
        conv_layers.arrange(RIGHT, buff=0.5)
        conv_layers.next_to(input_image, RIGHT)
        
        # Create fully connected layers
        fc_layers = VGroup(*[Circle(radius=0.2) for _ in range(10)])
        fc_layers.arrange_in_grid(rows=2, cols=5)
        fc_layers.next_to(conv_layers, RIGHT)
        
        # Create output layer
        output = Text("Cat", font_size=36)
        output.next_to(fc_layers, RIGHT)
        
        self.play(FadeIn(input_image))
        self.play(Create(conv_layers), Create(fc_layers))
        self.play(Write(output))
        self.wait(2)

Real-life Example: Natural Language Processing

Let's visualize a simple sentiment analysis task using a Recurrent Neural Network (RNN) for processing text data.

class SentimentAnalysisExample(Scene):
    def construct(self):
        # Create input text
        input_text = Text("This movie was great!", font_size=24)
        input_text.to_edge(LEFT)
        
        # Create RNN cells
        rnn_cells = VGroup(*[Circle(radius=0.3) for _ in range(5)])
        rnn_cells.arrange(RIGHT, buff=0.5)
        rnn_cells.next_to(input_text, RIGHT)
        
        # Create arrows between cells
        arrows = VGroup(*[Arrow(start=c1.get_right(), end=c2.get_left()) for c1, c2 in zip(rnn_cells[:-1], rnn_cells[1:])])
        
        # Create output
        output = Text("Positive", color=GREEN, font_size=36)
        output.next_to(rnn_cells, RIGHT)
        
        self.play(Write(input_text))
        self.play(Create(rnn_cells), Create(arrows))
        self.play(Write(output))
        self.wait(2)

Additional Resources

For those interested in diving deeper into Manim for machine learning visualizations, here are some valuable resources:

  1. Manim Community Documentation: https://docs.manim.community/
  2. "Visualizing Machine Learning Models with Python" by Jay Alammar (ArXiv:1909.01066)
  3. "Interactive Visualizations for Machine Learning Explanability" by Gomez et al. (ArXiv:2104.11455)
  4. Manim tutorials on YouTube by 3Blue1Brown
  5. GitHub repositories with Manim examples for machine learning concepts

These resources provide a wealth of information on both Manim usage and advanced machine learning visualization techniques. Remember to verify the most current versions of these resources, as the field of ML visualization is rapidly evolving.

More in Advanced ML topics

  1. Neural Scaling in Python Optimizing Model Performance7 min

    Neural scaling refers to the practice of increasing the size and complexity of neural networks to improve their performance. This concept has gained...

  2. Location Parameter in Machine Learning with Python8 min

    The location parameter is a crucial concept in machine learning, particularly in statistical modeling. It represents the central tendency of a distribution...

  3. Popular Distance Measures in Machine Learning6 min

    Distance measures play a crucial role in various machine learning algorithms, helping to quantify the similarity or dissimilarity between data points. These...

  4. Levels of Measurement in Machine Learning with Python8 min

    Levels of measurement, also known as scales of measurement, are fundamental concepts in statistics and machine learning. They categorize data based on the...

  5. Power Series and Sequences in Machine Learning Using Python8 min

    Power series are mathematical tools that represent functions as infinite sums of terms. In machine learning, they're used to approximate complex functions...

  6. Lagrange Multipliers in Machine Learning and AI:9 min

    Lagrange Multipliers are a powerful mathematical tool used in optimization problems, particularly in machine learning and AI. They help us find the optimal...