Building a RAG Pipeline from YouTube Transcripts

A RAG pipeline is only as good as its citations: if you can't point back to where an answer came from, you can't verify it. YouTube transcripts are a good fit because each segment already carries a timestamp — chunk on those boundaries and every retrieved passage comes with a jump-to-moment link, not just a video title.

This is a real gap most text corpora don't solve for you. A scraped article gives you a URL at best; a transcript segment gives you a URL and a specific second, which is the difference between "this document mentions it" and "here's exactly where, down to the sentence."

What you need per chunk

Two things beyond the text itself: a timestamp to cite, and enough video metadata that a chunk is useful outside the context of the batch it came from. Scriptube gives you both on the same read:

• include_segments=true on a batch poll returns timecoded {text, start, duration} segments per item.
• include_metadata=true adds title, channel, URL, and publish date — attach this to every chunk, not just the video record, so a retrieved chunk is self-describing once it's sitting alone in a vector store.

Chunking by time, not by token count

Segment boundaries from auto-captions are too short to embed individually (often a few words each) and too numerous to keep as separate vectors. Group them into fixed-length windows — 30–90 seconds works well for talk-heavy content — and keep the window's start time as the citation anchor:

import requests

HEADERS = {"X-API-Key": "YOUR_API_KEY"}
BASE = "https://scriptube.me"
batch_id = "00000000-0000-0000-0000-000000000000"

page = requests.get(
    f"{BASE}/api/v1/transcripts/{batch_id}",
    headers=HEADERS,
    params={"include_segments": "true", "include_metadata": "true"},
).json()

CHUNK_SECONDS = 60

def chunk_segments(segments, video):
    chunk, chunk_start = [], None
    for seg in segments:
        if chunk_start is None:
            chunk_start = seg["start"]
        chunk.append(seg["text"])
        if seg["start"] + seg["duration"] - chunk_start >= CHUNK_SECONDS:
            yield {
                "text": " ".join(chunk),
                "start_seconds": chunk_start,
                "video_id": video["video_id"],
                "title": video["title"],
                "url": f"{video['url']}&t={int(chunk_start)}s",
            }
            chunk, chunk_start = [], None
    if chunk:
        yield {
            "text": " ".join(chunk),
            "start_seconds": chunk_start,
            "video_id": video["video_id"],
            "title": video["title"],
            "url": f"{video['url']}&t={int(chunk_start)}s",
        }

records = []
for item in page["items"]:
    segments = item.get("transcript", {}).get("segments") or []
    for rec in chunk_segments(segments, item):
        records.append(rec)  # embed rec["text"], store the rest as citation metadata

print(f"{len(records)} citeable chunks ready to embed")

Embed rec["text"] with whatever model you're using, and store the rest of the dict as metadata alongside the vector. When retrieval surfaces a chunk, you already have a working YouTube link timestamped to the exact moment, plus the video title for display.

Picking a chunk size

There's a real trade-off here, not just a default to copy. Short chunks (15–30 seconds) retrieve more precisely — the embedded text is tightly about one thing — but a user reading the citation gets less surrounding context, and you need more of them to cover a video. Long chunks (90+ seconds) read more naturally as a citation but dilute the embedding with multiple topics, hurting retrieval precision for anything that isn't the chunk's dominant subject. 30–60 seconds is a reasonable starting point for narrated or lecture-style content; shorter for fast-moving interviews or panel discussions where the topic shifts quickly.

If you want smoother retrieval boundaries, overlap adjacent chunks by a few seconds (start each new window slightly before the previous one ends) so a statement that happens to fall right on a chunk boundary isn't split in half with neither chunk making full sense alone.

Scale: a channel or playlist at once

For a real knowledge base — a whole channel of lectures, a product's demo library — submit the channel or playlist URL as one batch instead of listing videos by hand; see channel batch costs, and building a larger JSONL dataset if you're exporting for offline processing rather than pulling live via the API.

Keeping citations honest

A few things worth being deliberate about:

• Language. If a video has multiple caption tracks, segments follow whatever language the batch (or your ?language= override) requested — mixing languages in one index without recording which one defeats the citation's purpose.
• Auto-generated vs. manual captions. is_generated on the transcript tells you which you got; ASR transcripts carry more transcription noise worth accounting for in retrieval quality, not just generation quality.
• Rights. Transcripts come from captions YouTube publishes. Respect the creator's rights and YouTube's terms for whatever you build on top — this isn't legal advice, just the baseline to check before shipping anything public-facing.
• Stale content. A video can be edited, taken down, or have its captions updated after you've indexed it. If your use case needs freshness guarantees, periodically re-check the source videos rather than treating an index as permanent.

Keeping the index current

A knowledge base built from a channel isn't static — new videos get published. Re-submitting the same channel URL (or playlist) periodically picks up new uploads without re-processing what's already stored; see keeping a playlist's transcripts current for the pattern. On your side, track which video_ids you've already chunked and embedded so a re-run only indexes what's new, rather than re-embedding a whole channel's worth of chunks on every refresh.

Next steps

• Every way to get timestamps — segments, SRT/VTT, and the dashboard UI.
• Building an LLM dataset from channels — the batch-export version of this pipeline.
• Using transcripts for AI training data — rights and cleaning considerations.

Full reference: API documentation.