xbe.at

Deep learning / CNN and computer vision

Behind the Scenes of Generative Adversarial Networks (GANs)

10 min read, 15 sections

Contents
  1. Introduction to GANs
  2. The Generator Network
  3. The Discriminator Network
  4. The Adversarial Training Process
  5. Challenges in GAN Training
  6. Evaluating GAN Performance
  7. Real-life Example: Image Generation
  8. Real-life Example: Text Generation
  9. Mode Collapse in GANs
  10. Strategies to Mitigate Mode Collapse
  11. Conditional GANs (cGANs)
  12. Progressive Growing of GANs
  13. Wasserstein GAN (WGAN)
  14. CycleGAN for Image-to-Image Translation
  15. Additional Resources

Introduction to GANs

Generative Adversarial Networks (GANs) are a class of machine learning models consisting of two neural networks: a generator and a discriminator. The generator creates new data from random noise, while the discriminator evaluates whether the data is real or generated. This adversarial process leads to the creation of highly realistic synthetic data.

import random

class Generator:
    def generate(self, noise):
        # Simplified generator logic
        return [n + random.uniform(-0.5, 0.5) for n in noise]

class Discriminator:
    def discriminate(self, data):
        # Simplified discriminator logic
        return sum(data) / len(data) > 0.5

class GAN:
    def __init__(self):
        self.generator = Generator()
        self.discriminator = Discriminator()

    def train(self, iterations):
        for _ in range(iterations):
            noise = [random.uniform(-1, 1) for _ in range(10)]
            generated_data = self.generator.generate(noise)
            is_real = self.discriminator.discriminate(generated_data)
            print(f"Generated data deemed real: {is_real}")

gan = GAN()
gan.train(5)

The Generator Network

The generator network in a GAN takes random noise as input and transforms it into data that resembles the training set. It learns to map from a latent space to a data distribution of interest, such as images, text, or audio.

import random

class Generator:
    def __init__(self, input_size, output_size):
        self.weights = [[random.uniform(-1, 1) for _ in range(input_size)] for _ in range(output_size)]
        self.biases = [random.uniform(-1, 1) for _ in range(output_size)]

    def generate(self, noise):
        output = []
        for i in range(len(self.weights)):
            neuron_output = sum(w * n for w, n in zip(self.weights[i], noise)) + self.biases[i]
            output.append(max(0, neuron_output))  # ReLU activation
        return output

# Example usage
generator = Generator(input_size=10, output_size=5)
noise = [random.uniform(-1, 1) for _ in range(10)]
generated_data = generator.generate(noise)
print("Generated data:", generated_data)

The Discriminator Network

The discriminator network in a GAN is trained to distinguish between real data from the training set and fake data produced by the generator. It acts as a binary classifier, learning to identify the characteristics that separate real and generated samples.

import random

class Discriminator:
    def __init__(self, input_size):
        self.weights = [random.uniform(-1, 1) for _ in range(input_size)]
        self.bias = random.uniform(-1, 1)

    def discriminate(self, data):
        weighted_sum = sum(w * d for w, d in zip(self.weights, data)) + self.bias
        return 1 / (1 + pow(2.718, -weighted_sum))  # Sigmoid activation

# Example usage
discriminator = Discriminator(input_size=5)
real_data = [0.7, 0.2, 0.9, 0.3, 0.5]
fake_data = [0.1, 0.8, 0.2, 0.7, 0.4]

real_score = discriminator.discriminate(real_data)
fake_score = discriminator.discriminate(fake_data)

print(f"Real data score: {real_score:.4f}")
print(f"Fake data score: {fake_score:.4f}")

The Adversarial Training Process

The training process in GANs involves a competition between the generator and discriminator. The generator tries to produce increasingly realistic data, while the discriminator becomes better at identifying fake samples. This adversarial process continues until an equilibrium is reached.

import random

class GAN:
    def __init__(self, latent_dim, data_dim):
        self.generator = Generator(latent_dim, data_dim)
        self.discriminator = Discriminator(data_dim)

    def train_step(self, real_data):
        # Train discriminator
        noise = [random.uniform(-1, 1) for _ in range(latent_dim)]
        fake_data = self.generator.generate(noise)
        
        real_score = self.discriminator.discriminate(real_data)
        fake_score = self.discriminator.discriminate(fake_data)
        
        d_loss = -0.5 * (log(real_score) + log(1 - fake_score))
        
        # Train generator
        g_loss = -log(self.discriminator.discriminate(fake_data))
        
        return d_loss, g_loss

# Example usage
latent_dim, data_dim = 10, 5
gan = GAN(latent_dim, data_dim)
real_data = [random.uniform(0, 1) for _ in range(data_dim)]

d_loss, g_loss = gan.train_step(real_data)
print(f"Discriminator loss: {d_loss:.4f}")
print(f"Generator loss: {g_loss:.4f}")

Challenges in GAN Training

Training GANs can be challenging due to the need for a delicate balance between the generator and discriminator. Common issues include mode collapse, where the generator produces limited varieties of outputs, and training instability, where one network overpowers the other.

import random

def train_gan(gan, epochs, batch_size):
    for epoch in range(epochs):
        d_losses, g_losses = [], []
        for _ in range(batch_size):
            real_data = [random.uniform(0, 1) for _ in range(gan.data_dim)]
            d_loss, g_loss = gan.train_step(real_data)
            d_losses.append(d_loss)
            g_losses.append(g_loss)
        
        avg_d_loss = sum(d_losses) / batch_size
        avg_g_loss = sum(g_losses) / batch_size
        
        print(f"Epoch {epoch+1}/{epochs}:")
        print(f"  Discriminator loss: {avg_d_loss:.4f}")
        print(f"  Generator loss: {avg_g_loss:.4f}")
        
        if avg_d_loss < 0.1 or avg_g_loss < 0.1:
            print("Potential mode collapse detected!")
            break

# Example usage
latent_dim, data_dim = 10, 5
gan = GAN(latent_dim, data_dim)
train_gan(gan, epochs=10, batch_size=32)

Evaluating GAN Performance

Evaluating the performance of GANs can be challenging due to the lack of a single, universally accepted metric. Common approaches include visual inspection, Inception Score, and Fréchet Inception Distance (FID). These methods help assess the quality and diversity of generated samples.

import random
import math

def inception_score(generated_samples, num_classes):
    # Simplified Inception Score calculation
    scores = []
    for sample in generated_samples:
        # Simulate class probabilities
        probs = [random.random() for _ in range(num_classes)]
        total = sum(probs)
        probs = [p / total for p in probs]
        
        # Calculate KL divergence
        kl_divergence = sum(p * math.log(p * num_classes) for p in probs if p > 0)
        scores.append(math.exp(kl_divergence))
    
    return sum(scores) / len(scores)

# Example usage
num_samples = 1000
num_classes = 10
generated_samples = [[random.random() for _ in range(5)] for _ in range(num_samples)]

is_score = inception_score(generated_samples, num_classes)
print(f"Inception Score: {is_score:.4f}")

Real-life Example: Image Generation

One common application of GANs is generating realistic images. For example, a GAN could be trained on a dataset of faces to generate new, synthetic face images. This has applications in various fields, including entertainment, privacy protection, and data augmentation for machine learning tasks.

import random

class ImageGenerator:
    def __init__(self, image_size):
        self.image_size = image_size

    def generate_face(self):
        # Simplified face generation
        face = [[random.uniform(0, 1) for _ in range(self.image_size)] for _ in range(self.image_size)]
        
        # Add basic face features
        for i in range(self.image_size):
            for j in range(self.image_size):
                # Eyes
                if (i-self.image_size//4)**2 + (j-self.image_size//3)**2 < (self.image_size//10)**2:
                    face[i][j] = 0
                if (i-self.image_size//4)**2 + (j-2*self.image_size//3)**2 < (self.image_size//10)**2:
                    face[i][j] = 0
                # Mouth
                if (i-3*self.image_size//4)**2 + (j-self.image_size//2)**2 < (self.image_size//8)**2:
                    face[i][j] = 0
        
        return face

    def display_face(self, face):
        for row in face:
            print(''.join('#' if pixel < 0.5 else ' ' for pixel in row))

# Example usage
generator = ImageGenerator(image_size=20)
face = generator.generate_face()
generator.display_face(face)

Real-life Example: Text Generation

Another application of GANs is in the domain of text generation. GANs can be used to generate realistic text passages, which has applications in content creation, chatbots, and language translation. Here's a simplified example of how a GAN might generate text:

import random

class TextGenerator:
    def __init__(self):
        self.vocabulary = ["The", "quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"]
        self.transition_probs = {
            "The": {"quick": 0.7, "brown": 0.3},
            "quick": {"brown": 0.8, "fox": 0.2},
            "brown": {"fox": 1.0},
            "fox": {"jumps": 1.0},
            "jumps": {"over": 1.0},
            "over": {"the": 1.0},
            "the": {"lazy": 1.0},
            "lazy": {"dog": 1.0},
            "dog": {"The": 0.3, "quick": 0.3, "brown": 0.4}
        }

    def generate_sentence(self, max_length=10):
        sentence = ["The"]
        while len(sentence) < max_length:
            last_word = sentence[-1]
            if last_word not in self.transition_probs:
                break
            next_word = random.choices(list(self.transition_probs[last_word].keys()),
                                       weights=list(self.transition_probs[last_word].values()))[0]
            sentence.append(next_word)
            if next_word == "dog":
                break
        return " ".join(sentence)

# Example usage
generator = TextGenerator()
for _ in range(5):
    print(generator.generate_sentence())

Mode Collapse in GANs

Mode collapse is a common issue in GAN training where the generator produces limited varieties of outputs, failing to capture the full diversity of the target distribution. This can result in the generator creating only a small subset of plausible outputs.

import random

class SimpleGAN:
    def __init__(self, modes):
        self.modes = modes
        self.generator_mode = random.choice(modes)

    def generate(self):
        return self.generator_mode

    def train_step(self):
        real_sample = random.choice(self.modes)
        generated_sample = self.generate()
        
        if generated_sample == real_sample:
            # Generator succeeded, no change
            return False
        else:
            # Generator failed, switch to the real sample
            self.generator_mode = real_sample
            return True

# Example usage
modes = ['A', 'B', 'C', 'D']
gan = SimpleGAN(modes)

for step in range(20):
    mode_changed = gan.train_step()
    print(f"Step {step + 1}: Generated {gan.generate()}, Mode changed: {mode_changed}")

print(f"Final generator mode: {gan.generator_mode}")

Strategies to Mitigate Mode Collapse

To address mode collapse, various techniques have been developed. These include minibatch discrimination, which encourages the discriminator to look at multiple examples simultaneously, and diversity-promoting loss functions that explicitly reward the generator for producing diverse outputs.

import random

class DiverseGAN:
    def __init__(self, modes, diversity_weight):
        self.modes = modes
        self.diversity_weight = diversity_weight
        self.generator_modes = random.sample(modes, k=2)  # Start with 2 modes

    def generate(self):
        return random.choice(self.generator_modes)

    def calculate_diversity(self):
        return len(set(self.generator_modes)) / len(self.modes)

    def train_step(self):
        real_sample = random.choice(self.modes)
        generated_sample = self.generate()
        
        if generated_sample == real_sample:
            # Generator succeeded
            diversity = self.calculate_diversity()
            if diversity < self.diversity_weight and len(self.generator_modes) < len(self.modes):
                # Add a new mode to increase diversity
                new_mode = random.choice([m for m in self.modes if m not in self.generator_modes])
                self.generator_modes.append(new_mode)
            return False
        else:
            # Generator failed, add the real sample to its modes
            if real_sample not in self.generator_modes:
                self.generator_modes.append(real_sample)
            return True

# Example usage
modes = ['A', 'B', 'C', 'D', 'E']
gan = DiverseGAN(modes, diversity_weight=0.6)

for step in range(20):
    mode_changed = gan.train_step()
    print(f"Step {step + 1}: Generated {gan.generate()}, "
          f"Modes: {gan.generator_modes}, "
          f"Diversity: {gan.calculate_diversity():.2f}")

print(f"Final generator modes: {gan.generator_modes}")

Conditional GANs (cGANs)

Conditional GANs extend the GAN framework by allowing the generator and discriminator to condition on additional information. This enables more controlled generation of data based on specific attributes or labels.

import random

class ConditionalGAN:
    def __init__(self, num_classes):
        self.num_classes = num_classes

    def generate(self, noise, condition):
        # Simplified conditional generation
        return [n + condition / self.num_classes for n in noise]

    def discriminate(self, data, condition):
        # Simplified conditional discrimination
        return sum(data) / len(data) > 0.5 + condition / (2 * self.num_classes)

    def train_step(self, condition):
        noise = [random.uniform(-1, 1) for _ in range(10)]
        generated_data = self.generate(noise, condition)
        is_real = self.discriminate(generated_data, condition)
        return generated_data, is_real

# Example usage
cgan = ConditionalGAN(num_classes=10)
condition = 3
generated_data, is_real = cgan.train_step(condition)
print(f"Generated data: {generated_data}")
print(f"Discriminator output: {is_real}")

Progressive Growing of GANs

Progressive Growing is a technique used to improve the stability and quality of GAN training, especially for high-resolution image generation. The process starts with low-resolution images and gradually increases the resolution during training.

class ProgressiveGAN:
    def __init__(self, initial_resolution, final_resolution):
        self.current_resolution = initial_resolution
        self.final_resolution = final_resolution

    def train_epoch(self):
        # Simulate training for current resolution
        print(f"Training at resolution: {self.current_resolution}x{self.current_resolution}")

    def increase_resolution(self):
        if self.current_resolution < self.final_resolution:
            self.current_resolution *= 2
            print(f"Increased resolution to: {self.current_resolution}x{self.current_resolution}")
        else:
            print("Reached final resolution")

# Example usage
gan = ProgressiveGAN(initial_resolution=4, final_resolution=32)

for epoch in range(5):
    gan.train_epoch()
    gan.increase_resolution()

Wasserstein GAN (WGAN)

Wasserstein GAN is a variant that uses the Wasserstein distance as its loss function, addressing some of the stability issues in traditional GANs. WGANs often provide more stable training and better quality results.

import random

class WGAN:
    def __init__(self):
        self.generator = lambda z: [x + random.uniform(-0.1, 0.1) for x in z]
        self.critic = lambda x: sum(x) / len(x)

    def wasserstein_loss(self, real_data, fake_data):
        return self.critic(real_data) - self.critic(fake_data)

    def train_step(self, real_data):
        z = [random.uniform(-1, 1) for _ in range(len(real_data))]
        fake_data = self.generator(z)
        loss = self.wasserstein_loss(real_data, fake_data)
        return loss

# Example usage
wgan = WGAN()
real_data = [random.uniform(0, 1) for _ in range(5)]
loss = wgan.train_step(real_data)
print(f"Wasserstein loss: {loss}")

CycleGAN for Image-to-Image Translation

CycleGAN is a type of GAN used for image-to-image translation without paired training data. It learns to translate an image from a source domain to a target domain and back, maintaining consistency through cycle consistency loss.

class CycleGAN:
    def __init__(self):
        self.G_AB = lambda x: [1 - val for val in x]  # Simplified generator A->B
        self.G_BA = lambda x: [1 - val for val in x]  # Simplified generator B->A
        self.D_A = lambda x: sum(x) / len(x) > 0.5  # Simplified discriminator A
        self.D_B = lambda x: sum(x) / len(x) > 0.5  # Simplified discriminator B

    def cycle_consistency_loss(self, real_A, real_B):
        fake_B = self.G_AB(real_A)
        reconstructed_A = self.G_BA(fake_B)
        fake_A = self.G_BA(real_B)
        reconstructed_B = self.G_AB(fake_A)
        return sum((a - ra)**2 for a, ra in zip(real_A, reconstructed_A)) + \
               sum((b - rb)**2 for b, rb in zip(real_B, reconstructed_B))

# Example usage
cycle_gan = CycleGAN()
real_A = [0.1, 0.3, 0.5, 0.7, 0.9]
real_B = [0.2, 0.4, 0.6, 0.8, 1.0]
loss = cycle_gan.cycle_consistency_loss(real_A, real_B)
print(f"Cycle consistency loss: {loss}")

Additional Resources

For more in-depth information on GANs, consider exploring these resources:

  1. "Generative Adversarial Networks" by Ian Goodfellow et al. (2014) ArXiv: https://arxiv.org/abs/1406.2661
  2. "Wasserstein GAN" by Martin Arjovsky et al. (2017) ArXiv: https://arxiv.org/abs/1701.07875
  3. "Progressive Growing of GANs for Improved Quality, Stability, and Variation" by Tero Karras et al. (2017) ArXiv: https://arxiv.org/abs/1710.10196
  4. "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks" by Jun-Yan Zhu et al. (2017) ArXiv: https://arxiv.org/abs/1703.10593

These papers provide foundational and advanced concepts in GAN architecture and training techniques.

More in CNN and computer vision

  1. Building Dynamic Topology Neural Networks with Multi-Objective Optimization in Python10 min

    Dynamic Topology Neural Networks (DTNNs) are an advanced form of artificial neural networks that can adapt their structure during training. This adaptation...

  2. Adversarial Attacks on Neural Networks12 min

    Adversarial attacks are techniques used to manipulate input data, often images, to deceive machine learning models, particularly neural networks. These...

  3. Diffusion Models for Imaging and Vision in Python10 min

    Diffusion models are a class of generative models that have gained significant attention in the field of computer vision and image processing. These models...

  4. Adversarial Attacks on Machine Learning Models in Python7 min

    Introduction to Adversarial Attacks on Machine Learning Models

  5. Exploring 1x1 Convolutions in Deep Learning9 min

    1x1 convolutions, also known as pointwise convolutions, are a special type of convolutional layer in neural networks. They operate on a single pixel across...

  6. Advanced Generative Adversarial Networks with Python11 min

    Generative Adversarial Networks (GANs) are a class of deep learning models consisting of two neural networks: a generator and a discriminator. These networks...