Understanding Backpropagation with Python

Feedforward Propagation Neural networks consist of layers of nodes, each computing a weighted sum of inputs and applying an activation function. Feedforward propagation computes the output of the network given inputs.

import numpy as np

# Sample input data
X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])

# Sample weights
W1 = np.array([[0.1, 0.2], [0.3, 0.4]])
W2 = np.array([[0.5, 0.6], [0.7, 0.8]])

# Activation function (sigmoid)
def sigmoid(x):
    return 1 / (1 + np.exp(-x))

# Feedforward propagation
def feedforward(X):
    layer1 = sigmoid(np.dot(X, W1.T))
    layer2 = sigmoid(np.dot(layer1, W2.T))
    return layer2

Cost Function To train the network, we need a cost function that measures the error between the predicted output and the true output.

# Sample true outputs
y_true = np.array([[0], [1], [1], [0]])

# Mean squared error cost function
def cost(y_true, y_pred):
    return np.mean((y_true - y_pred)**2)

Backpropagation - Overview Backpropagation is an algorithm to compute the gradients of the cost function with respect to the weights, which allows us to update the weights to minimize the cost.

Backpropagation - Step 1: Output Layer Error Compute the error at the output layer by taking the derivative of the cost function with respect to the output layer's weighted inputs.

def backprop(X, y_true):
    layer1 = sigmoid(np.dot(X, W1.T))
    layer2 = sigmoid(np.dot(layer1, W2.T))
    
    # Output layer error
    error_output = (layer2 - y_true) * layer2 * (1 - layer2)
    
    # ...

Backpropagation - Step 2: Hidden Layer Error Propagate the error back to the hidden layer by taking the dot product of the output layer error and the weights between the hidden and output layers.

    # Hidden layer error
    error_hidden = np.dot(error_output, W2) * layer1 * (1 - layer1)
    
    # ...

Backpropagation - Step 3: Weight Gradients Compute the gradients of the cost function with respect to the weights by taking the dot product of the errors and the inputs.

    # Output layer gradients
    output_gradients = np.dot(layer1.T, error_output)
    
    # Hidden layer gradients
    hidden_gradients = np.dot(X.T, error_hidden)
    
    return hidden_gradients, output_gradients

Backpropagation - Step 4: Weight Update Update the weights by subtracting the gradients multiplied by a learning rate from the current weights.

# Learning rate
alpha = 0.1

# Update weights
W1 -= alpha * hidden_gradients
W2 -= alpha * output_gradients

Training the Network Train the network by repeatedly performing feedforward propagation, backpropagation, and weight updates until the cost function is minimized.

# Training loop
for epoch in range(10000):
    y_pred = feedforward(X)
    cost_val = cost(y_true, y_pred)
    
    hidden_grads, output_grads = backprop(X, y_true)
    
    W1 -= alpha * hidden_grads
    W2 -= alpha * output_grads
    
    if epoch % 1000 == 0:
        print(f"Epoch {epoch}, Cost: {cost_val}")

Using an API (Finance Example) Let's fetch stock data using the Alpha Vantage API and preprocess it for training.

import requests

# API key (replace with your own)
api_key = 'YOUR_API_KEY'

# Stock symbol
symbol = 'AAPL'

# API endpoint
url = f'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol={symbol}&apikey={api_key}'

# Fetch data
response = requests.get(url)
data = response.json()

# Preprocess data
# ...

Feature Engineering Extract relevant features from the stock data, such as open, close, high, low prices, and volume.

import pandas as pd

# Convert data to DataFrame
df = pd.DataFrame(data['Time Series (Daily)']).T

# Extract features
df['open'] = df['1. open'].astype(float)
df['close'] = df['4. close'].astype(float)
df['high'] = df['2. high'].astype(float)
df['low'] = df['3. low'].astype(float)
df['volume'] = df['5. volume'].astype(float)

# Drop unnecessary columns
df = df[['open', 'close', 'high', 'low', 'volume']]

Data Preparation Split the data into training and testing sets, and normalize the features.

from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

# Split data
X_train, X_test, y_train, y_test = train_test_split(df.drop('close', axis=1), df['close'], test_size=0.2)

# Normalize features
scaler = MinMaxScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

Training the Neural Network Train the neural network on the stock data using the backpropagation algorithm.

import tensorflow as tf

# Build the model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(X_train.shape[1],)),
    tf.keras.layers.Dense(32, activation='relu'),
    tf.keras.layers.Dense(1)
])

# Compile the model
model.compile(optimizer='adam', loss='mse')

# Train the model
model.fit(X_train, y_train, epochs=100, batch_size=32, validation_data=(X_test, y_test))

Model Evaluation Evaluate the performance of the trained model on the test set.

# Make predictions
y_pred = model.predict(X_test)

# Compute evaluation metrics
mse = mean_squared_error(y_test, y_pred)
print(f"Mean Squared Error: {mse}")

Conclusion Backpropagation is a powerful algorithm for training neural networks by computing gradients of the cost function with respect to the weights and updating the weights accordingly. It enables neural networks to learn complex patterns and make accurate predictions on various tasks, including finance and stock prediction.

Meta

Here's a title, description, and hashtags for a TikTok video on "Understanding Backpropagation in Neural Networks: The Mathematics Behind It" with an institutional tone:

"Decoding Backpropagation: The Engine of Neural Networks"

Explore the mathematical foundations of backpropagation, the powerful algorithm that drives the training of neural networks. This video delves into the mechanics of feedforward propagation, cost functions, and the step-by-step process of backpropagation, including output layer error computation, hidden layer error propagation, weight gradient calculations, and weight updates. Gain insight into how this algorithm enables neural networks to learn and make accurate predictions across various domains, including finance and stock prediction. #NeuralNetworks #Backpropagation #DeepLearning #MachineLearning #AI #DataScience #Mathematics #InstitutionalLearning

Relevant Hashtags: #NeuralNetworks #Backpropagation #DeepLearning #MachineLearning #AI #DataScience #Mathematics #InstitutionalLearning #TechnicalEducation #AcademicContent #EducationalVideo #STEM #ComputerScience