How to Build an LLM Training Dataset from YouTube Channels

Building an LLM training dataset from YouTube channels means turning a list of channel URLs into one structured file of transcript text plus metadata, with duplicates removed and the source attribution intact. Scriptube does the extraction and export; you still own the decision about what you have the rights to train on.

Before you pull anything: rights come first

Transcripts come from captions YouTube publishes for a video, not from a license to redistribute or train on that content. Respect the channel owner's rights and YouTube's Terms of Service before you build a corpus, and don't treat "technically downloadable" as "legally clear for training." This article is not legal advice — see is-downloading-youtube-transcripts-legal for the general considerations, and read the longer rights discussion in using-youtube-transcripts-for-ai-training-data before you scale this past a handful of test videos.

Step 1: Submit the channel as a batch

Paste a channel URL or @handle into a batch request. Scriptube resolves it to a channel batch, which is available on every plan and billed per video like any other submission — there's no separate "channel" price. You can mix channel URLs with individual video and playlist URLs in the same request (up to 10 channels and 10 playlists per request), and duplicate videos across them are removed automatically.

Check the cost before you commit. For a channel with hundreds of videos, run the estimate endpoint first:

curl -X POST https://scriptube.me/api/v1/transcripts/estimate \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://www.youtube.com/@example-channel"],
    "translate_to_english": false
  }'

The response includes a per-video breakdown, a total_cost in credits, your current balance, and a precheck_token you can hand to the submit call so the price doesn't drift between the quote and the charge.

Step 2: Submit and wait for completion

curl -X POST https://scriptube.me/api/v1/transcripts \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://www.youtube.com/@example-channel"],
    "language": "en",
    "external_id": "dataset-build-2026-10"
  }'

This returns a batches[] array — one entry for the channel batch, since channel URLs each become their own background batch. For a dataset build, register a webhook for batch.completed instead of polling (see n8n-youtube-transcript-automation if you want this wired into a workflow tool rather than a script). If you'd rather poll, GET /api/v1/transcripts/{batch_id} reports item_count and has_more as it works through the channel.

Step 3: Export as JSONL

JSONL is the natural format for training data — one JSON object per line, one line per video, streamable into any tokenizer pipeline:

curl "https://scriptube.me/api/v1/transcripts/{batch_id}/export?format=jsonl&language=en" \
  -H "X-API-Key: YOUR_API_KEY" \
  -o channel_dataset.jsonl

Each row carries the transcript text, plus columns like channel_id, published_at, duration_seconds, is_generated, and status/error_code for anything that failed. If you need word-level or sentence-level timing instead of a flat text blob, pull the batch with ?include_segments=true and build your own chunking from the {text, start, duration} segments rather than the export — segments are what a RAG pipeline wants (see youtube-rag-pipeline-transcripts for that approach) while flat text is usually enough for fine-tuning.

Add &include_failed=false if you don't want skipped or permanently-failed items showing up as empty rows in your training file — by default they're kept, with a status and error_code, so you can audit what didn't make it in.

Step 4: Clean before you train

Auto-generated captions carry artifacts a training set shouldn't: [Music] and [Applause] markers, run-on sentences with no punctuation, and the occasional misheard proper noun. None of this is Scriptube-specific — it's a property of YouTube's ASR captions — so plan a cleaning pass: strip bracketed non-speech tags, re-punctuate if your downstream task needs it, and drop very short transcripts (a 20-second clip rarely adds signal to a training set). Keep the metadata columns even after cleaning; knowing which channel and which date a line came from is what lets you de-duplicate across multiple pulls later and attribute content correctly.

Step 5: Re-running without re-paying for everything

Channels add new videos. When you re-run the same channel URL later, youtube-channel-transcripts-limits-and-costs covers how re-submission is scoped and billed — the short version is that processing always charges per video, so budget for new uploads rather than re-running the whole channel on a schedule without checking the estimate first.

Keeping the pipeline unattended

For a dataset that needs to stay current, script the submit step with an API key scoped to exactly what this job needs — see api-keys-scopes-rotation-revocation for the difference between a read-only key (fine for exporting what's already processed) and a full-scope key (required to submit new batches), and rotate it on a schedule rather than leaving one long-lived secret in a cron job forever.

A note on scale

There are no plan caps on batch size, videos per day, or parallel batches, so credits are the only limit on a dataset build — check /ui/pricing and the dashboard estimate before you plan a multi-channel pull. A large build can be one request or several channel batches running in parallel. If credits run out mid-batch, finished transcripts are kept and the rest is cancelled and not charged.

JSON if you'd rather load the whole batch at once

JSON and JSONL carry the identical columns — one row per video, text plus metadata, no timing either way — the only difference is a single array (format=json) versus one object per line (format=jsonl). JSONL is the better default for a streaming training pipeline; JSON is better when you want to inspect or transform the whole dataset in memory first. Neither format includes caption tracks or segments — for per-segment timing, use include_segments=true as described above.

Start with one channel, verify the JSONL output looks right, then scale up: read the API docs or create a Scriptube account to get an API key.