xbe.at

Miscellany / Data engineering and tools

Converting Podcasts to Searchable Chat with Groq, Whisper, and PineCone

5 min read

Introduction to Podcast-to-Chat Conversion

This slideshow will guide you through the process of converting multiple audio podcast files into a searchable and queryable chat application. We'll leverage the power of Groq Whisper for speech recognition, PineCone for vector search, and Streamlit for building a user-friendly interface. The code examples are written in Python and are suitable for beginner to intermediate level programmers.

import os
import whisper
import pinecone
import streamlit as st

Setting up Whisper

Groq Whisper is an automatic speech recognition (ASR) system that can transcribe audio files into text. In this step, we'll install and load the Whisper model.

# Install Whisper
!pip install git+https://github.com/openai/whisper.git

# Load the Whisper model
model = whisper.load_model("base")

Transcribing Podcast Audio Files

With the Whisper model loaded, we can transcribe our podcast audio files into text. This step involves iterating over the audio files and applying the transcribe function.

import glob

# List of audio files
audio_files = glob.glob("path/to/audio/files/*.mp3")

# Transcribe audio files
transcripts = []
for audio_file in audio_files:
    result = model.transcribe(audio_file)
    transcript = result["text"]
    transcripts.append(transcript)

Indexing Transcripts with PineCone

PineCone is a vector database that allows us to store and search our transcripts efficiently. Here, we'll create a PineCone index and populate it with the transcripts.

import pinecone

# Initialize PineCone
pinecone.init(api_key="your_api_key", environment="your_environment")

# Create or get an index
index = pinecone.Index("podcast-index")

# Index the transcripts
index.upsert(texts=transcripts, ids=range(len(transcripts)))

Setting up Streamlit

Streamlit is a Python library that allows us to build interactive web applications. We'll use it to create a user-friendly chat interface for querying the podcast transcripts.

import streamlit as st

# Set page configuration
st.set_page_config(page_title="Podcast Chat")

# Create a sidebar for search
st.sidebar.header("Search Transcripts")
query = st.sidebar.text_input("Enter your query")

Querying PineCone with Streamlit

Now, we'll integrate the PineCone search functionality into our Streamlit application. Users can enter their queries, and relevant excerpts from the transcripts will be displayed.

if query:
    # Search for relevant transcripts
    results = index.query(query, top_k=5, include_metadata=True)

    # Display the results
    for result in results["matches"]:
        st.write(f"**Score: {result['score']}**")
        st.write(result["metadata"]["text"])
        st.write("---")

Handling Multiple Queries

To enhance the user experience, we'll add functionality to handle multiple queries and display their results in separate sections.

query_history = []
results_history = []

if query:
    # Search for relevant transcripts
    results = index.query(query, top_k=5, include_metadata=True)

    # Store the query and results
    query_history.append(query)
    results_history.append(results)

# Display the query history and results
for i, (q, r) in enumerate(zip(query_history, results_history)):
    st.header(f"Query {i+1}: {q}")
    for result in r["matches"]:
        st.write(f"**Score: {result['score']}**")
        st.write(result["metadata"]["text"])
        st.write("---")

Improving Search Relevance

To improve the relevance of search results, we can preprocess the transcripts by removing stop words, stemming, and vectorizing the text.

import nltk
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer

# Download required NLTK resources
nltk.download('stopwords')
nltk.download('punkt')

# Preprocess transcripts
stemmer = PorterStemmer()
stop_words = set(stopwords.words('english'))

processed_transcripts = []
for transcript in transcripts:
    # Remove stop words and stem
    words = [stemmer.stem(word.lower()) for word in nltk.word_tokenize(transcript)
             if word.lower() not in stop_words]
    processed_transcripts.append(" ".join(words))

# Vectorize the processed transcripts
vectors = vectorizer.fit_transform(processed_transcripts)

# Index the vectors with PineCone
index.upsert(vectors=vectors)

Handling Audio File Uploads

To allow users to upload their own audio files and search the transcripts, we'll add functionality for file uploads in Streamlit.

# File uploader
uploaded_file = st.file_uploader("Upload an audio file", type=["mp3", "wav"])

if uploaded_file:
    # Transcribe the uploaded file
    result = model.transcribe(uploaded_file)
    transcript = result["text"]

    # Preprocess and vectorize the transcript
    processed_transcript = preprocess_text(transcript)
    vector = vectorizer.transform([processed_transcript])

    # Search the index with the new vector
    results = index.query(vector, top_k=5, include_metadata=True)

    # Display the results
    for result in results["matches"]:
        st.write(f"**Score: {result['score']}**")
        st.write(result["metadata"]["text"])
        st.write("---")

Handling Long Transcripts

For very long transcripts, we may need to split them into smaller chunks to improve search performance and memory efficiency.

import textwrap

# Function to split text into chunks
def split_text(text, chunk_size=1000):
    chunks = textwrap.wrap(text, chunk_size, break_long_words=False)
    return chunks

# Split transcripts into chunks
chunked_transcripts = []
for transcript in transcripts:
    chunks = split_text(transcript)
    chunked_transcripts.extend(chunks)

# Index the chunked transcripts
index.upsert(texts=chunked_transcripts, ids=range(len(chunked_transcripts)))

Improving User Interface

To enhance the user experience further, we can add interactive elements to the Streamlit app, such as search suggestions, filtering, and sorting options.

# Search suggestions
suggestions = index.query(query, top_k=5, include_metadata=True)
suggested_queries = [result["metadata"]["text"][:50] + "..." for result in suggestions["matches"]]
st.write("Search suggestions:")
for suggestion in suggested_queries:
    st.write("- " + suggestion)

# Filtering and sorting options
filter_option = st.selectbox("Filter results by:", ["None", "Relevance", "Date", "Speaker"])
sort_option = st.radio("Sort results by:", ["Relevance", "Date", "Speaker"])

Deploying the Application

Once you've completed the development and testing of your Podcast-to-Chat application, you can deploy it using various hosting platforms like Streamlit Sharing, Heroku, or AWS.

# Run the Streamlit app
if __name__ == "__main__":
    st.title("Podcast Chat")
    main()

Additional Resources

For further learning and exploration, you can refer to the following resources:

# Example code for accessing additional resources
import requests

# Fetch the latest version of Groq Whisper from GitHub
whisper_release = requests.get("https://api.github.com/repos/openai/whisper/releases/latest").json()
latest_version = whisper_release["tag_name"]
print(f"The latest version of Groq Whisper is {latest_version}")

# Access the PineCone documentation
pinecone_docs_url = "https://docs.pinecone.io/"
print(f"You can find the PineCone documentation at {pinecone_docs_url}")

Additional Resources (continued)

Here are some additional resources from arXiv.org that can further enhance your understanding of the topics covered in this slideshow:

  • "Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context" by Zihang Dai et al. (arXiv:1901.02860)
  • "Speech Transformer: A No-Recurrence Sequence-to-Sequence Model for Speech Recognition" by Linhao Dong et al. (arXiv:1904.11660)
  • "Transformer-based Acoustic Modeling for Hybrid Speech Recognition" by Hang Lyu et al. (arXiv:2104.03823)
  • "Neural Vector Search: A Revisiting of LSTM-based Language Model with Vector Space Embedding" by Kexin Huang et al. (arXiv:2010.02239)
# Example code for accessing arXiv resources
import arxiv

# Search for papers on Transformer models
search_query = "Transformer models"
papers = arxiv.Search(query=search_query, max_results=5)

print(f"Top {len(papers)} papers on '{search_query}':")
for paper in papers:
    print(f"- {paper.title} ({paper.entry_id})")

More in Data engineering and tools

  1. Creating and Manipulating Tensors with PyTorch7 min

    Tensors are the fundamental data structure in PyTorch, representing multi-dimensional arrays. They are similar to NumPy arrays but can be used on GPUs for...

  2. Cloud Load Balancer Cheat Sheet9 min

    Cloud load balancing distributes incoming network traffic across multiple servers to ensure no single server becomes overwhelmed. This process optimizes...

  3. Data Engineering Workflow Stages15 min

    Modern data engineering requires robust connection handling to multiple data sources simultaneously. This implementation demonstrates connecting to various...

  4. Data Mesh vs. Data Fabric Comparing Modern Data Architectures17 min

    Data Mesh represents a paradigm shift in data architecture, emphasizing domain-driven ownership and data as a product. This implementation demonstrates how...

  5. Data Structures and Algorithms for AI Enthusiasts10 min

    Data Structures and Algorithms (DSA) form the backbone of efficient AI systems. They enable us to organize, store, and process data effectively, which is...

  6. Designing Data-Intensive Applications Consistency Tradeoffs16 min

    Synchronous replication ensures strong consistency by waiting for acknowledgment from all replicas before confirming writes. This approach guarantees that...