xbe.at

Deep learning / CNN and computer vision

Surprising Vulnerability of Neural Networks to One-Pixel Attacks

9 min read, 14 sections

Contents
  1. Understanding One-Pixel Attacks
  2. Differential Evolution Framework
  3. Implementing the Attack Function
  4. Neural Network Prediction Analysis
  5. Differential Evolution Optimization
  6. Image Preprocessing for Attack
  7. Implementing Success Rate Evaluation
  8. Advanced Mutation Strategies
  9. Real-world Implementation
  10. Performance Metrics and Visualization
  11. Advanced Search Strategies
  12. Results Visualization and Analysis
  13. Real-world Example with CIFAR-10
  14. Additional Resources

Understanding One-Pixel Attacks

Neural networks are surprisingly vulnerable to minimal perturbations in input images. One-pixel attacks represent a fascinating subset of adversarial attacks where modifying just a single pixel can cause state-of-the-art neural networks to misclassify images with high confidence.

import numpy as np
from PIL import Image

def modify_single_pixel(image_array, x, y, rgb_values):
    """
    Modifies a single pixel in the image
    
    Args:
        image_array: numpy array of image (height, width, 3)
        x, y: coordinates of pixel to modify
        rgb_values: tuple of (r,g,b) values
    """
    modified = image_array.copy()
    modified[y, x] = rgb_values
    return modified

# Example usage
image = np.array(Image.open('horse.jpg'))
modified = modify_single_pixel(image, 100, 100, (255, 0, 0))
# Modified image now has one red pixel at (100,100)

Differential Evolution Framework

Differential Evolution is an optimization algorithm that doesn't require gradient information, making it ideal for one-pixel attacks. It works by evolving a population of candidate solutions through mutation and selection operations to find optimal pixel modifications.

import numpy as np

class DifferentialEvolution:
    def __init__(self, bounds, population_size=20, mutation_factor=0.8, crossover_rate=0.7):
        self.bounds = bounds
        self.population_size = population_size
        self.mutation_factor = mutation_factor
        self.crossover_rate = crossover_rate
        
    def initialize_population(self, dim):
        """Initialize random population within bounds"""
        population = np.random.rand(self.population_size, dim)
        for i, (low, high) in enumerate(self.bounds):
            population[:, i] = low + population[:, i] * (high - low)
        return population

    def mutate(self, population):
        """Create mutant vectors using current population"""
        return population + self.mutation_factor * (
            population[np.random.permutation(self.population_size)] -
            population[np.random.permutation(self.population_size)]
        )

Implementing the Attack Function

The core attack function combines image processing with differential evolution to systematically search for the most effective pixel modification that causes misclassification in the target neural network.

def one_pixel_attack(image, model, target_class, max_iterations=100):
    """
    Performs one-pixel attack on the given image
    
    Args:
        image: Input image array (height, width, channels)
        model: Target neural network model
        target_class: Desired misclassification class
        max_iterations: Maximum optimization iterations
    """
    image_dims = image.shape
    bounds = [(0, image_dims[0]), (0, image_dims[1]), (0, 255), (0, 255), (0, 255)]
    
    def objective_function(x):
        pixel_x, pixel_y, r, g, b = x.astype(int)
        perturbed = image.copy()
        perturbed[pixel_y, pixel_x] = [r, g, b]
        pred = model.predict(np.expand_dims(perturbed, 0))
        return -pred[0][target_class]  # Negative because we maximize confidence
    
    optimizer = DifferentialEvolution(bounds)
    best_pixel = optimizer.optimize(objective_function, max_iterations)
    return best_pixel

Neural Network Prediction Analysis

Understanding how neural networks process and classify images is crucial for implementing effective one-pixel attacks. We'll create a function to analyze prediction confidence scores before and after pixel modifications.

import tensorflow as tf

def analyze_predictions(original_image, modified_image, model, class_names):
    """
    Analyzes and compares model predictions for original and modified images
    
    Returns dictionary of prediction confidences
    """
    orig_pred = model.predict(np.expand_dims(original_image, 0))[0]
    mod_pred = model.predict(np.expand_dims(modified_image, 0))[0]
    
    analysis = {
        'original_class': class_names[orig_pred.argmax()],
        'original_confidence': float(orig_pred.max()),
        'modified_class': class_names[mod_pred.argmax()],
        'modified_confidence': float(mod_pred.max()),
        'confidence_delta': float(mod_pred.max() - orig_pred.max())
    }
    return analysis

Differential Evolution Optimization

Differential Evolution requires careful implementation of selection and crossover operations to effectively find the optimal pixel modification. This implementation focuses on the core optimization loop with adaptive parameter control.

def optimize(self, objective_func, max_iterations):
    """
    Main optimization loop for differential evolution
    
    Args:
        objective_func: Function to optimize
        max_iterations: Maximum number of iterations
    """
    dim = len(self.bounds)
    population = self.initialize_population(dim)
    fitness = np.array([objective_func(ind) for ind in population])
    
    for iteration in range(max_iterations):
        # Generate trial vectors
        trial_pop = self.mutate(population)
        
        # Ensure bounds
        for i, (low, high) in enumerate(self.bounds):
            trial_pop[:, i] = np.clip(trial_pop[:, i], low, high)
        
        # Selection
        trial_fitness = np.array([objective_func(ind) for ind in trial_pop])
        better_indices = trial_fitness < fitness
        
        # Update population
        population[better_indices] = trial_pop[better_indices]
        fitness[better_indices] = trial_fitness[better_indices]
    
    return population[fitness.argmin()]

Image Preprocessing for Attack

Proper image preprocessing is crucial for successful one-pixel attacks. This implementation handles various image formats and ensures compatibility with the target neural network's input requirements.

def preprocess_image(image_path, target_size=(224, 224)):
    """
    Preprocesses images for attack implementation
    
    Args:
        image_path: Path to input image
        target_size: Required input size for the model
    Returns:
        Preprocessed image array
    """
    # Load and resize image
    image = tf.keras.preprocessing.image.load_img(
        image_path, target_size=target_size
    )
    image_array = tf.keras.preprocessing.image.img_to_array(image)
    
    # Normalize pixel values
    image_array = image_array.astype('float32')
    image_array /= 255.0
    
    # Add batch dimension
    image_array = np.expand_dims(image_array, axis=0)
    
    return image_array

def create_adversarial_pattern(input_image, target_class, model):
    """
    Creates adversarial pattern for one-pixel attack
    """
    with tf.GradientTape() as tape:
        input_image = tf.convert_to_tensor(input_image)
        tape.watch(input_image)
        prediction = model(input_image)
        loss = tf.keras.losses.sparse_categorical_crossentropy(
            [target_class], prediction
        )
    
    gradient = tape.gradient(loss, input_image)
    signed_grad = tf.sign(gradient)
    return signed_grad

Implementing Success Rate Evaluation

A comprehensive evaluation framework is essential to measure the effectiveness of one-pixel attacks across different images and target classes. This implementation tracks success rates and confidence changes.

class AttackEvaluator:
    def __init__(self, model, class_names):
        self.model = model
        self.class_names = class_names
        self.results = []
    
    def evaluate_attack(self, original_image, modified_image, target_class):
        """
        Evaluates success of one-pixel attack
        
        Returns dict with attack metrics
        """
        orig_pred = self.model.predict(original_image)[0]
        mod_pred = self.model.predict(modified_image)[0]
        
        success = (np.argmax(mod_pred) == target_class and 
                  np.argmax(orig_pred) != target_class)
        
        return {
            'success': success,
            'original_confidence': float(np.max(orig_pred)),
            'attack_confidence': float(mod_pred[target_class]),
            'confidence_drop': float(np.max(orig_pred) - mod_pred[target_class])
        }
    
    def run_evaluation(self, test_images, target_classes):
        """Evaluates attack success on multiple images"""
        for image, target in zip(test_images, target_classes):
            modified = one_pixel_attack(image, self.model, target)
            result = self.evaluate_attack(image, modified, target)
            self.results.append(result)
        
        return {
            'success_rate': sum(r['success'] for r in self.results) / len(self.results),
            'avg_confidence_drop': np.mean([r['confidence_drop'] for r in self.results])
        }

Advanced Mutation Strategies

Different mutation strategies can significantly impact the success rate of one-pixel attacks. This implementation provides multiple mutation variants for optimal pixel selection.

class AdvancedDifferentialEvolution(DifferentialEvolution):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.mutation_strategies = {
            'rand1': self._rand1,
            'best1': self._best1,
            'current_to_best1': self._current_to_best1
        }
    
    def _rand1(self, population, fitness):
        """Random mutation strategy"""
        r1, r2, r3 = np.random.choice(len(population), 3, replace=False)
        return population[r1] + self.mutation_factor * (
            population[r2] - population[r3]
        )
    
    def _best1(self, population, fitness):
        """Best individual based mutation"""
        best_idx = np.argmin(fitness)
        r1, r2 = np.random.choice(len(population), 2, replace=False)
        return population[best_idx] + self.mutation_factor * (
            population[r1] - population[r2]
        )
    
    def _current_to_best1(self, population, fitness, current_idx):
        """Current-to-best mutation strategy"""
        best_idx = np.argmin(fitness)
        r1, r2 = np.random.choice(len(population), 2, replace=False)
        return population[current_idx] + self.mutation_factor * (
            population[best_idx] - population[current_idx]
        ) + self.mutation_factor * (
            population[r1] - population[r2]
        )

Real-world Implementation

A complete implementation of one-pixel attack against a pre-trained ResNet50 model, demonstrating the practical application of the attack on real-world image classification systems.

import tensorflow as tf
from tensorflow.keras.applications import ResNet50
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions

def perform_real_world_attack():
    """
    Complete implementation of one-pixel attack on ResNet50
    """
    # Load pre-trained model
    model = ResNet50(weights='imagenet')
    
    # Load and preprocess target image
    image_path = 'horse.jpg'
    original_image = preprocess_image(image_path)
    
    # Initial prediction
    initial_pred = model.predict(original_image)
    decoded_pred = decode_predictions(initial_pred, top=1)[0]
    print(f"Original prediction: {decoded_pred[0][1]} ({decoded_pred[0][2]:.2f})")
    
    # Perform attack
    target_class = 6  # 'frog' in ImageNet
    attack_result = one_pixel_attack(
        original_image[0], 
        model, 
        target_class,
        max_iterations=100
    )
    
    # Create attacked image
    attacked_image = modify_single_pixel(
        original_image[0],
        int(attack_result[0]),
        int(attack_result[1]),
        attack_result[2:5]
    )
    
    # Evaluate attack success
    final_pred = model.predict(np.expand_dims(attacked_image, 0))
    decoded_final = decode_predictions(final_pred, top=1)[0]
    print(f"After attack: {decoded_final[0][1]} ({decoded_final[0][2]:.2f})")
    
    return attacked_image, attack_result

Performance Metrics and Visualization

Implementation of comprehensive metrics and visualization tools to analyze the effectiveness of one-pixel attacks across different scenarios and target classes.

import matplotlib.pyplot as plt
import seaborn as sns

class AttackAnalyzer:
    def __init__(self):
        self.metrics = {}
    
    def analyze_attack(self, original_image, attacked_image, 
                      original_pred, attacked_pred, pixel_location):
        """
        Analyzes attack performance and generates visualizations
        """
        # Calculate perturbation metrics
        l2_dist = np.linalg.norm(original_image - attacked_image)
        confidence_change = np.max(original_pred) - np.max(attacked_pred)
        
        # Store metrics
        self.metrics = {
            'l2_distance': l2_dist,
            'confidence_change': confidence_change,
            'pixel_location': pixel_location,
            'attack_success': np.argmax(original_pred) != np.argmax(attacked_pred)
        }
        
        # Generate visualization
        fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 6))
        
        ax1.imshow(original_image)
        ax1.set_title(f'Original: {np.max(original_pred):.2%} confidence')
        
        ax2.imshow(attacked_image)
        ax2.set_title(f'Attacked: {np.max(attacked_pred):.2%} confidence')
        
        # Mark modified pixel
        ax2.plot(pixel_location[0], pixel_location[1], 'rx', markersize=10)
        
        return fig, self.metrics

Advanced Search Strategies

Implementation of sophisticated search strategies to optimize the pixel selection process, incorporating both local and global search mechanisms for improved attack success rates.

class AdvancedPixelSearch:
    def __init__(self, image_shape, model):
        self.image_shape = image_shape
        self.model = model
        self.search_history = []
    
    def region_based_search(self, regions=9):
        """
        Implements region-based pixel search strategy
        """
        h, w = self.image_shape[:2]
        region_h, region_w = h // 3, w // 3
        
        best_pixel = None
        best_score = float('inf')
        
        for i in range(regions):
            # Calculate region boundaries
            start_h = (i // 3) * region_h
            start_w = (i % 3) * region_w
            end_h = start_h + region_h
            end_w = start_w + region_w
            
            # Search within region
            pixel = self._search_region(
                (start_h, end_h),
                (start_w, end_w)
            )
            
            score = self._evaluate_pixel(pixel)
            if score < best_score:
                best_score = score
                best_pixel = pixel
                
            self.search_history.append({
                'region': i,
                'pixel': pixel,
                'score': score
            })
        
        return best_pixel
    
    def _search_region(self, h_bounds, w_bounds):
        """Local search within region"""
        population = np.random.randint(
            low=[h_bounds[0], w_bounds[0], 0, 0, 0],
            high=[h_bounds[1], w_bounds[1], 256, 256, 256],
            size=(20, 5)
        )
        return self._optimize_population(population)

Results Visualization and Analysis

A comprehensive implementation for visualizing attack results, including confidence mapping and pixel influence analysis across different regions of the target image.

class ResultsVisualizer:
    def __init__(self):
        self.attack_results = []
    
    def generate_confidence_map(self, image, model, pixel_results):
        """
        Creates heatmap of pixel influence on classification
        """
        h, w = image.shape[:2]
        confidence_map = np.zeros((h, w))
        
        for px, py, r, g, b, conf in pixel_results:
            confidence_map[int(px), int(py)] = conf
            
        plt.figure(figsize=(10, 8))
        sns.heatmap(confidence_map, cmap='viridis')
        plt.title('Pixel Influence Heatmap')
        
        return confidence_map
    
    def plot_attack_progression(self, history):
        """
        Visualizes attack progress over iterations
        """
        iterations = range(len(history))
        confidences = [h['confidence'] for h in history]
        
        plt.figure(figsize=(10, 6))
        plt.plot(iterations, confidences, 'b-')
        plt.xlabel('Iteration')
        plt.ylabel('Target Class Confidence')
        plt.title('Attack Progression')
        
        return {'min_conf': min(confidences),
                'max_conf': max(confidences),
                'iterations': len(history)}

Real-world Example with CIFAR-10

Implementation of a complete attack scenario using the CIFAR-10 dataset, demonstrating practical application and effectiveness measurement.

def cifar10_attack_example():
    """
    Complete example of one-pixel attack on CIFAR-10
    """
    # Load CIFAR-10 data
    (x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()
    x_test = x_test.astype('float32') / 255.0
    
    # Load pre-trained model
    model = tf.keras.models.load_model('cifar10_model.h5')
    
    # Select test image
    test_idx = 0
    original_image = x_test[test_idx]
    original_class = y_test[test_idx][0]
    
    # Define target class (different from original)
    target_class = (original_class + 1) % 10
    
    # Perform attack
    attack_params = {
        'population_size': 400,
        'max_iterations': 100,
        'mutation_factor': 0.5,
        'crossover_rate': 0.7
    }
    
    optimizer = AdvancedDifferentialEvolution(
        bounds=[(0, 32), (0, 32), (0, 1), (0, 1), (0, 1)],
        **attack_params
    )
    
    result = optimize_attack(original_image, model, target_class, optimizer)
    
    # Evaluate results
    modified_image = apply_attack(original_image, result)
    success = evaluate_attack(model, modified_image, target_class)
    
    return {
        'original_class': original_class,
        'target_class': target_class,
        'success': success,
        'pixel_modification': result
    }

Additional Resources

More in CNN and computer vision

  1. Response:11 min

    Kolmogorov-Arnold Networks (KANs) are a class of neural networks inspired by the Kolmogorov-Arnold representation theorem. These networks aim to approximate...

  2. Posterior Collapse in VAEs vs. Diffusion Models10 min

    Posterior collapse is a phenomenon that can occur in Variational Autoencoders (VAEs) but is less common in diffusion models. This presentation will explore...

  3. Optimizing Diffusion Models for Faster Generation7 min

    The DPM-Solver framework revolutionizes diffusion model sampling by implementing a high-order numerical solver based on differential equations theory. It...

  4. Matrix Orientation, Hidden-state Alignment, and Knowledge Distillation in Python9 min

    Matrix orientation is a crucial concept in linear algebra and machine learning. It refers to how matrices are arranged and interpreted, affecting...

  5. Matrix Multiplication in Deep Learning4 min

    Matrix multiplication is a fundamental operation in deep learning, crucial for processing inputs, applying weights, and propagating information through...

  6. Introduction to Padding, Strides, and Pooling in Deep Learning with Python9 min

    Padding, strides, and pooling are fundamental concepts in deep learning, particularly in convolutional neural networks (CNNs). These techniques help manage...