xbe.at

Python / Python for ML and AI

Detecting Moving Objects in Videos with Python and OpenCV

12 min read, 14 sections

Contents
  1. Introduction to Object Detection in Videos using OpenCV and Python
  2. Background Subtraction
  3. Contour Detection and Bounding Boxes
  4. Tracking Moving Objects
  5. Motion Detection using Optical Flow
  6. Background Subtraction with KNN Algorithm
  7. Motion Detection with Frame Differencing
  8. Motion Detection with Gaussian Mixture Models
  9. Object Tracking with CAMSHIFT Algorithm
  10. Object Tracking with CSRT Algorithm
  11. Object Tracking with MOSSE Algorithm
  12. Object Tracking with BOOSTING Algorithm
  13. Deep Learning-based Object Detection
  14. Additional Resources

Introduction to Object Detection in Videos using OpenCV and Python

Object detection is the process of identifying and locating objects within an image or video. In this slideshow, we will explore how to detect moving objects in a video using OpenCV, a popular computer vision library, and Python programming language. This technique finds applications in various domains such as surveillance systems, traffic monitoring, and robotics.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Display the frame
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Background Subtraction

Background subtraction is a fundamental technique used for detecting moving objects in a video. It involves creating a background model and then subtracting it from the current frame to identify the foreground objects (moving objects).

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Create a background subtractor object
fgbg = cv2.createBackgroundSubtractorMOG2()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Apply background subtraction
    fgmask = fgbg.apply(frame)

    # Display the frame and the foreground mask
    cv2.imshow('Video', frame)
    cv2.imshow('Foreground Mask', fgmask)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Contour Detection and Bounding Boxes

After obtaining the foreground mask, we can apply contour detection to identify the boundaries of the moving objects. Then, we can draw bounding boxes around these objects to visualize their locations in the frame.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Create a background subtractor object
fgbg = cv2.createBackgroundSubtractorMOG2()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Apply background subtraction
    fgmask = fgbg.apply(frame)

    # Find contours in the foreground mask
    contours, _ = cv2.findContours(fgmask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Draw bounding boxes around the contours
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with bounding boxes
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Tracking Moving Objects

Once we have identified the moving objects, we can track their movements across subsequent frames using object tracking algorithms like the Lucas-Kanade method or the CAMSHIFT algorithm.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the tracker
tracker = cv2.MultiTracker_create()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Update the tracker
    success, boxes = tracker.update(frame)

    # Draw bounding boxes around the tracked objects
    for box in boxes:
        x, y, w, h = [int(v) for v in box]
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with tracked objects
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Motion Detection using Optical Flow

Optical flow is a technique for estimating the motion of objects between consecutive frames. It can be used to detect and visualize the motion of objects in a video.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Parameters for the optical flow algorithm
lk_params = dict(winSize=(15, 15),
                 maxLevel=2,
                 criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

# Create some random colors
color = np.random.randint(0, 255, (100, 3))

# Initialize the previous frame
ret, prev_frame = cap.read()
prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)

while True:
    # Read the next frame
    ret, frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Calculate the optical flow
    p1, st, err = cv2.calcOpticalFlowPyrLK(prev_gray, frame_gray, None, None, **lk_params)

    # Select good points
    good_new = p1[st == 1]
    good_old = p0[st == 1]

    # Draw the tracks
    for i, (new, old) in enumerate(zip(good_new, good_old)):
        a, b = new.ravel()
        c, d = old.ravel()
        frame = cv2.line(frame, (a, b), (c, d), color[i].tolist(), 2)
        frame = cv2.circle(frame, (a, b), 5, color[i].tolist(), -1)

    # Update the previous frame
    prev_gray = frame_gray.copy()

    # Display the frame
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Slide 6 Background Modeling with Mixture of Gaussians

The Mixture of Gaussians (MOG) background subtraction algorithm is a popular method for modeling the background in a video. It adaptively updates the background model based on the observed pixel values over time. This algorithm can handle dynamic backgrounds and lighting changes effectively.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Create a background subtractor object
fgbg = cv2.createBackgroundSubtractorMOG2()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Apply background subtraction
    fgmask = fgbg.apply(frame)

    # Apply some post-processing on the foreground mask
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)

    # Find contours in the foreground mask
    contours, _ = cv2.findContours(fgmask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Draw bounding boxes around the contours
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with bounding boxes
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Background Subtraction with KNN Algorithm

The K-Nearest Neighbors (KNN) background subtraction algorithm is another approach for background modeling. It uses a sample of recently observed values at each pixel to determine whether the current pixel belongs to the background or the foreground.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Create a background subtractor object
fgbg = cv2.createBackgroundSubtractorKNN()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Apply background subtraction
    fgmask = fgbg.apply(frame)

    # Apply some post-processing on the foreground mask
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)

    # Find contours in the foreground mask
    contours, _ = cv2.findContours(fgmask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Draw bounding boxes around the contours
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with bounding boxes
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Motion Detection with Frame Differencing

Frame differencing is a simple yet effective technique for detecting moving objects in a video. It involves subtracting the current frame from the previous frame to obtain the areas of motion.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the previous frame
ret, prev_frame = cap.read()
prev_gray = cv2.cvtColor(prev_frame, cv2.COLOR_BGR2GRAY)

while True:
    # Read the next frame
    ret, frame = cap.read()
    frame_gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Compute the frame difference
    frame_diff = cv2.absdiff(frame_gray, prev_gray)

    # Apply thresholding to obtain the motion mask
    _, motion_mask = cv2.threshold(frame_diff, 25, 255, cv2.THRESH_BINARY)

    # Apply some post-processing on the motion mask
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    motion_mask = cv2.morphologyEx(motion_mask, cv2.MORPH_OPEN, kernel)

    # Find contours in the motion mask
    contours, _ = cv2.findContours(motion_mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Draw bounding boxes around the contours
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with bounding boxes
    cv2.imshow('Video', frame)

    # Update the previous frame
    prev_gray = frame_gray

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Motion Detection with Gaussian Mixture Models

Gaussian Mixture Models (GMM) can be used for background subtraction and motion detection. This approach models each pixel's background as a mixture of Gaussian distributions, making it robust to gradual changes in illumination and dynamic backgrounds.

import cv2
import numpy as np

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Create a background subtractor object
fgbg = cv2.createBackgroundSubtractorGMG()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Apply background subtraction
    fgmask = fgbg.apply(frame)

    # Apply some post-processing on the foreground mask
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
    fgmask = cv2.morphologyEx(fgmask, cv2.MORPH_OPEN, kernel)

    # Find contours in the foreground mask
    contours, _ = cv2.findContours(fgmask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    # Draw bounding boxes around the contours
    for cnt in contours:
        x, y, w, h = cv2.boundingRect(cnt)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with bounding boxes
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Object Tracking with CAMSHIFT Algorithm

The Continuously Adaptive Mean-Shift (CAMSHIFT) algorithm is a powerful tool for object tracking. It iteratively adjusts the search window to track an object based on its color histogram. This technique can handle object movement, rotation, and scaling.

import cv2
import numpy as np

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the tracking window
ret, frame = cap.read()
x, y, w, h = 300, 200, 100, 50  # Initial window position and size
track_window = (x, y, w, h)

# Setup the ROI for tracking
roi = frame[y:y+h, x:x+w]
hsv_roi = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)
mask = cv2.inRange(hsv_roi, np.array((0., 60., 32.)), np.array((180., 255., 255.)))
roi_hist = cv2.calcHist([hsv_roi], [0], mask, [180], [0, 180])
cv2.normalize(roi_hist, roi_hist, 0, 255, cv2.NORM_MINMAX)

# Setup the termination criteria
term_crit = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 1)

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    if ret:
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
        dst = cv2.calcBackProject([hsv], [0], roi_hist, [0, 180], 1)

        # Apply CAMSHIFT to get the new location
        ret, track_window = cv2.CamShift(dst, track_window, term_crit)

        # Draw the tracking window
        pts = cv2.boxPoints(ret)
        pts = np.int0(pts)
        img2 = cv2.polylines(frame, [pts], True, (0, 255, 0), 2)

        # Display the frame
        cv2.imshow('Video', img2)

        # Break the loop if 'q' is pressed
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Object Tracking with CSRT Algorithm

The Discriminative Correlation Filter with Channel and Spatial Reliability (CSRT) is a robust and accurate object tracking algorithm. It combines the discriminative correlation filter with spatial and channel reliability, making it capable of handling various challenging scenarios.

import cv2

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the tracker
tracker = cv2.MultiTracker_create()

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    # Update the tracker
    success, boxes = tracker.update(frame)

    # Draw bounding boxes around the tracked objects
    for box in boxes:
        x, y, w, h = [int(v) for v in box]
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame with tracked objects
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Object Tracking with MOSSE Algorithm

The Minimum Output Sum of Squared Error (MOSSE) is a correlation filter-based object tracking algorithm. It adapts the filter coefficients over time, making it capable of handling appearance changes and deformations of the tracked object.

import cv2
import numpy as np

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the tracker
tracker = cv2.legacy.TrackerMOSSE_create()

# Read the first frame and select the bounding box
ret, frame = cap.read()
bbox = cv2.selectROI(frame, False)
ok = tracker.init(frame, bbox)

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    if not ret:
        break

    # Update the tracker
    success, bbox = tracker.update(frame)

    if success:
        # Draw the bounding box
        x, y, w, h = [int(v) for v in bbox]
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Object Tracking with BOOSTING Algorithm

The BOOSTING algorithm is a machine learning-based object tracking approach that uses an online boosting algorithm to update the discriminative model for tracking the target object. It can handle various challenges like occlusion, background clutter, and appearance changes.

import cv2
import numpy as np

# Load the video
cap = cv2.VideoCapture('video.mp4')

# Initialize the tracker
tracker = cv2.legacy.TrackerBoosting_create()

# Read the first frame and select the bounding box
ret, frame = cap.read()
bbox = cv2.selectROI(frame, False)
ok = tracker.init(frame, bbox)

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    if not ret:
        break

    # Update the tracker
    success, bbox = tracker.update(frame)

    if success:
        # Draw the bounding box
        x, y, w, h = [int(v) for v in bbox]
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Deep Learning-based Object Detection

While the previous techniques relied on traditional computer vision methods, deep learning-based approaches have revolutionized object detection in recent years. Convolutional Neural Networks (CNNs) like YOLO, Faster R-CNN, and SSD can accurately detect and localize multiple objects in real-time.

import cv2
import numpy as np

# Load the pre-trained object detection model
net = cv2.dnn.readNetFromCaffe('MobileNetSSD_deploy.prototxt', 'MobileNetSSD_deploy.caffemodel')

# Load the video
cap = cv2.VideoCapture('video.mp4')

while True:
    # Read a frame from the video
    ret, frame = cap.read()

    if not ret:
        break

    # Preprocess the frame
    blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 0.007843, (300, 300), 127.5)

    # Pass the blob through the network and obtain the detections
    net.setInput(blob)
    detections = net.forward()

    # Draw bounding boxes around the detected objects
    for i in range(detections.shape[2]):
        confidence = detections[0, 0, i, 2]
        if confidence > 0.5:
            idx = int(detections[0, 0, i, 1])
            box = detections[0, 0, i, 3:7] * np.array([frame.shape[1], frame.shape[0], frame.shape[1], frame.shape[0]])
            (x, y, w, h) = box.astype("int")
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    # Display the frame
    cv2.imshow('Video', frame)

    # Break the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the video capture object
cap.release()
# Close all windows
cv2.destroyAllWindows()

Additional Resources

For further exploration and learning, here are some additional resources on object detection and tracking in videos using OpenCV and Python:

These resources provide in-depth explanations, advanced techniques, and additional examples to enhance your understanding and implementation of object detection and tracking algorithms.

More in Python for ML and AI

  1. Diagrams as Code Turning Python into Architecture7 min

    Diagrams as Code is a powerful approach that allows developers to create cloud system architecture diagrams using Python code. This method eliminates the...

  2. Denoising Diffusion Probabilistic Models in Python6 min

    Denoising Diffusion Probabilistic Models (DDPMs) are a class of generative models that have gained significant attention in recent years. They work by...

  3. Difference-in-Differences (DiD) Analysis in Python13 min

    Difference-in-Differences (DiD) is a statistical method used to estimate the causal effect of a treatment or intervention by comparing the changes in...

  4. Data Preparation with Python's singledispatch5 min

    The singledispatch decorator in Python allows you to create generic functions that can handle different types of input. It's particularly useful for data...

  5. Discovered Universal Number with Python7 min

    The concept of a "discovered universal number" is not a well-established mathematical or scientific concept. There is no single number that has been...

  6. Data Modeling with Python7 min

    Data modeling is the process of creating a conceptual representation of data and its relationships. In Python, we use various libraries and techniques to...