Reference & Scaling: Costs, Engines, and Turning the Toolkit into a Product

📚 Part 3 of a 4-part series. Part 1 covered setup & architecture; Part 2 walked the full creative pipeline. This post is everything you need to run it economically and turn it into a product.

Everything in Parts 1–2 runs a script on a machine you control. This post is about the numbers behind that, the engine trade-offs, and the three deployment shapes — from a single pay-per-request endpoint to a billed, multi-tenant API.

27. Training specs & costs

Cost is GPU-hours, not per-file. Indicative rates (June 2026; verify current):

Dataset sizing: under 50 files risks overfit; 500–1,500 curated = sweet spot; 3k+ only if uniformly on-aesthetic (a messy 3k trains worse than a clean 800). A whole launch to a sellable model is typically under $20 of GPU. The expensive resource is your curation/QA time, not the GPU.

28. Sourcing lossless audio

You can't un-compress a lossy file — converting MP3→WAV adds nothing. Get true source from: ripping your own CDs (EAC/dBpoweramp), buying WAV/FLAC (Bandcamp, Beatport, Qobuz), recording vinyl, or cleared-sample services (Tracklib, Splice). Streaming "lossless" tiers are fine to listen to, not to rip for training. Verify with deep_listen.py (rolloff95_hz + the lossy-upsample flag) or Spek.

For a pristine commercial model, a smaller true-WAV core (300–500) beats 1,500 lossy — source quality > count, and it's the cleaner legal footing.

Format conversion (mp3_to_wav.py). Batch-decode MP3 (and m4a/ogg/opus/flac/aac) to WAV for DAW/toolkit compatibility — recursive, --mirror to keep folders, --resume, ffmpeg-backed with a librosa fallback. It's a lossy→PCM decode, so it does not recover quality the MP3 discarded.

python scripts/mp3_to_wav.py --input "F:/RAP_ARCHIVES/mp3" --output "F:/RAP_ARCHIVES/wav" --bit-depth 24 --mirror --resume

29. yt-dlp commands

Reference/listening only (lossy source; rights caveats apply — don't train a sellable model on these):

# playlist -> WAV, resumable; --download-archive skips finished
.\yt-dlp.exe --playlist-start 1 -x --audio-format wav --download-archive done.txt -o "%(playlist_index)s - %(title)s.%(ext)s" "PLAYLIST_URL"

-P "F:\Downloads" targets a drive; .\yt-dlp.exe -U updates.

30. Business & learning path

Packaging as a service: the SaaS backend now ships — an authenticated REST API, a Redis/RQ job queue with CPU/GPU worker lanes, credit metering, Stripe billing, a pricing page, rate limiting, and a tested, CI-gated, Docker-compose deployment (section 35, and server/; go-live steps in DEPLOY.md). What's left to launch is operational, not code: managed Postgres + Redis, TLS, your own auth/onboarding, and live Stripe products.

Differentiators vs Splice / Waves / Loudly: hip-hop depth, private models trained on a customer's own sounds, provenance certificates, and groove-level control.

Products: (1) a provenance-verified ecosystem pack line; (2) "your sound as a model" private fine-tunes (SA3 LoRA collapses the unit cost); (3) groove-DNA template packs.

Learning path (O'Reilly): Géron Hands-On ML → Programming PyTorch (audio ch.) → Foster Generative Deep Learning → HF Hands-On Generative AI → Think DSP. Free: HF Audio Course, "The Sound of AI." Study repos: stable-audio-tools, stable-audio-3, audiocraft, demucs, pedalboard, librosa, CLAP, AbletonOSC.

(Not financial/legal advice — verify license thresholds and trademarks before commercializing.)

31. Full script reference

Every runnable script and what it does:

requirements.txt lists core + per-feature optional deps. cloud/ has pod setup scripts; configs/ has dataset/VST-chain configs; prompts/ has pack plans + example lyrics.

32. Engine choice: Stable Audio 3 vs ACE-Step 1.5

Two interchangeable, first-class generation engines ship in the toolkit. A/B them and pick per project — same dataset, same captions, different sound and licensing.

Setup — ACE-Step (cloud pod default; runs locally too):

bash cloud/ace_step_setup.sh                 # clone ACE-Step-1.5 + uv sync (models auto-download)
cd ACE-Step-1.5 && ACESTEP_API_HOST=0.0.0.0 uv run acestep-api    # REST API on :8001

The toolkit talks to that server over HTTP, so start it first. Examples:

# batch instrumentals from a pack plan (same plan files SA3 uses)
python scripts/ace_step_workflow.py generate --plan prompts/pack_plan.example.json --out generated_ace
# a full SONG with vocals from your lyrics + style tags (this is what SA3 can't do)
python scripts/ace_step_workflow.py song --prompt "boom bap, dusty, male rap vocals, 90 BPM" \
  --lyrics-file verse.txt --bpm 90 --key "F minor" --duration 180 --out song
# COVER / restyle an existing track (lower strength = subtler)
python scripts/ace_step_workflow.py cover --src mybeat.wav --prompt "drum and bass, reese bass" --strength 0.5 --out remix

When to use which. Reach for ACE-Step when you want vocal songs, a clean MIT license for paid work, or low-VRAM/local generation. Stay on Stable Audio 3 when you've invested in an SA3 LoRA you like, or want its inpaint/extend editing. Output the same pack plan through both, listen, keep the winner.

33. Serverless API (RunPod) — host the toolkit as an endpoint

What it is. A serverless endpoint wraps one toolkit job (generate a beat, tag a file, flip a sample) behind an HTTPS URL that auto-scales to zero — you pay only for the seconds a request runs, with no idle GPU bill. This is how you turn the toolkit into a service (a website "Generate" button, a Discord bot) instead of something you SSH into. Keep using a normal pod (section 34) for training and interactive work.

The dev loop: write a handler → test locally → package as a Docker image → push to a registry → create the endpoint → send requests.

1 · The handler

A handler takes a job and returns JSON-serializable output. Drop this at serverless/handler.py — it routes one task field to the scripts you already have:

# serverless/handler.py  -  wraps Co-Produce AI scripts as a RunPod handler
import base64, subprocess, tempfile, os, runpod

def _wav_b64(path):
    with open(path, "rb") as f:
        return base64.b64encode(f.read()).decode()

def handler(event):
    """event['input'] = {"task": "beat"|"tag"|"flip", ...task-specific args}"""
    inp = event.get("input", {}) or {}
    task = inp.get("task", "beat")
    out = tempfile.mkdtemp()

    if task == "beat":                         # beats from your own samples
        wav = os.path.join(out, "beat.wav")
        subprocess.run(["python", "scripts/beat_builder.py",
                        "--style", inp.get("style", "boom_bap"),
                        "--bpm", str(inp.get("bpm", 90)),
                        "--out", wav], check=True)
        return {"wav_b64": _wav_b64(wav), "style": inp.get("style", "boom_bap")}

    if task == "tag":                          # heuristic tagger (no model dl)
        import sys; sys.path.insert(0, "scripts")
        import auto_tag
        tags, cap = auto_tag.caption_heuristic(inp["path"])
        return {"tags": tags, "caption": cap}

    if task == "flip":                         # audio-to-audio derive
        wav = os.path.join(out, "flip.wav")
        subprocess.run(["python", "scripts/audio2audio.py",
                        "--input", inp["path"], "--prompt", inp.get("prompt", ""),
                        "--strength", str(inp.get("strength", 0.6)),
                        "--out", wav], check=True)
        return {"wav_b64": _wav_b64(wav)}

    return {"error": f"unknown task '{task}'"}

runpod.serverless.start({"handler": handler})  # required entrypoint

Test it locally first — no Docker, no cloud:

pip install runpod
python serverless/handler.py --test_input '{"input":{"task":"beat","style":"boom_bap","bpm":90}}'

2 · Package as a Docker image

# serverless/Dockerfile
FROM runpod/base:0.6.2-cuda12.1.0
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt runpod
COPY scripts/ scripts/
COPY serverless/handler.py .
CMD ["python", "-u", "handler.py"]

3 · Build & push (must be linux/amd64)

docker build --platform linux/amd64 -t YOUR_DOCKERHUB_USER/co-produce-ai-sls:latest -f serverless/Dockerfile .
docker push YOUR_DOCKERHUB_USER/co-produce-ai-sls:latest

No Docker locally? Use RunPod's GitHub integration — point an endpoint at the repo and RunPod builds the image on each push.

4 · Create the endpoint

In the RunPod console → Serverless → New Endpoint: set the container image (or GitHub repo), pick a GPU tier, and set scaling — Max workers (burst ceiling), Active workers (kept warm; 0 = cheapest, ≥1 = no cold start), Idle timeout, and enable FlashBoot to shrink cold starts. Attach your network volume if the handler needs your dataset or model weights.

5 · Send requests

$H = @{ Authorization = "Bearer $env:RUNPOD_API_KEY"; "Content-Type" = "application/json" }
$body = '{"input":{"task":"beat","style":"drill","bpm":142}}'
Invoke-RestMethod -Method Post -Headers $H -Body $body `
  -Uri "https://api.runpod.ai/v2/$env:ENDPOINT_ID/runsync"

The repo also ships a typed Go client at clients/go/ that submits, polls, and decodes the returned wav_b64:

cd clients/go
go mod tidy
$env:RUNPOD_API_KEY="your_runpod_api_key"; $env:ENDPOINT_ID="your_endpoint_id"
go run . -task beat -style trap -bpm 140 -out trap.wav

34. Pod workflow — SSH, SCP & cloning the repo to a pod

A pod is a full GPU box you SSH into and run the toolkit on directly — the right place for training, dataset prep, and heavy tagging. You need an SSH key registered with RunPod and a pod with a public IP.

ssh-keygen -t ed25519 -C "patcampbell82@gmail.com"
Get-Content $env:USERPROFILE\.ssh\id_ed25519.pub | Set-Clipboard   # paste into console -> Settings -> SSH Public Keys

Connect with the "SSH over exposed TCP" command from the pod's Connect tab (the one that supports SCP):

ssh root@<POD_IP> -p <SSH_PORT> -i $env:USERPROFILE\.ssh\id_ed25519

Clone the repo onto the pod (ideally onto the network volume at /workspace so code + models persist):

cd /workspace
git clone https://github.com/pjcampbe11/Co-Produce-AI.git
pip install -r Co-Produce-AI/requirements.txt

Or the one-shot bootstrap — clone + install + cache routing in one paste:

curl -fsSL https://raw.githubusercontent.com/pjcampbe11/Co-Produce-AI/main/cloud/pod_bootstrap.sh | bash

Move files with SCP (run in a local terminal, not the SSH session):

# push your beats up
scp -P <SSH_PORT> -i $env:USERPROFILE\.ssh\id_ed25519 -r "F:\RAP_ARCHIVES\raw_beats" root@<POD_IP>:/workspace/
# pull a results folder back down
scp -P <SSH_PORT> -i $env:USERPROFILE\.ssh\id_ed25519 -r root@<POD_IP>:/workspace/Co-Produce-AI/out "F:\RAP_ARCHIVES\out"

Or upload your dataset to the S3 network volume once and mount it on any pod:

aws s3 cp "F:\RAP_ARCHIVES\raw_beats" s3://<VOLUME_ID>/raw_beats/ --recursive `
  --profile runpod --region eu-ro-1 --endpoint-url https://s3api-eu-ro-1.runpod.io --checksum-algorithm CRC32

35. SaaS server — job queue, REST API & Stripe billing

What it is. The pieces that turn Co-Produce AI from scripts into a product: an authenticated REST API, a Redis-backed job queue with scalable workers, per-job credit metering, and Stripe subscription billing. Lives in server/; one docker compose up runs the whole stack.

client ──HTTPS──> FastAPI (api) ──enqueue──> Redis ──> Worker(s) ──> scripts/*
                  │  API keys · credit metering          beat/tag/flip/remix/song
                  └── Stripe checkout + webhooks → grants monthly credits
   SQLModel DB: users · api keys · jobs · credit ledger   results on a shared volume

Spin it up, sign up, run a paid job, download the result:

cp server/.env.example server/.env      # add Stripe keys + price map
docker compose up --build               # api :8000, worker, redis

curl -s -X POST localhost:8000/v1/signup -H 'content-type: application/json' -d '{"email":"you@example.com"}'
# -> {"user_id":"a1b2","api_key":"bt_9f...","credits":10,...}

KEY=bt_9f...
curl -s -X POST localhost:8000/v1/jobs -H "authorization: Bearer $KEY" \
  -H 'content-type: application/json' -d '{"task":"beat","params":{"style":"trap","bpm":140}}'
curl -s localhost:8000/v1/jobs/3c4d/result -H "authorization: Bearer $KEY" -o trap.wav

What's in the box. The API (server/app.py) exposes signup + API-key auth (Authorization: Bearer bt_…), job submit/list/get, a binary result download, account/usage, a Stripe checkout endpoint, and a Stripe webhook. Jobs enqueue to Redis/RQ (server/queue.py) and run on workers (server/worker.py → server/tasks.py) that invoke the same scripts/ you run by hand. State persists with SQLModel — SQLite by default, point DATABASE_URL at Postgres for production.

Credits & metering. Submitting reserves credits; the worker refunds them if the job fails. Edit TASK_COSTS in server/tasks.py:

Stripe billing. Map each price id to a plan + monthly credit grant in STRIPE_PRICES (JSON), set STRIPE_SECRET_KEY, expose POST /v1/webhooks/stripe with its signing secret in STRIPE_WEBHOOK_SECRET. checkout.session.completed / invoice.paid grant credits; customer.subscription.deleted downgrades to free. Locally: stripe listen --forward-to localhost:8000/v1/webhooks/stripe.

GPU vs CPU workers. Jobs route to two lanes: CPU tasks (beat, tag) → beat-cpu; GPU tasks (flip, remix, song) → beat-gpu. Run CPU workers on a cheap box and GPU workers on GPU hosts/pods against the same Redis + results volume:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build
# fan out a pack run across 8 workers:
docker compose up --scale worker=8

Locking down signup. /v1/signup is open by default for self-hosting. Before exposing publicly, set ALLOW_SIGNUP=false and an ADMIN_TOKEN — then accounts are minted only with X-Admin-Token: <token> (verified: 403 without, 200 with). A static Bears-themed pricing page is served at /pricing.

36. Requirements & dependencies (with venv setup)

The toolkit runs on Python 3.11 in a virtual environment (recap from Part 1):

python -m venv .venv
.venv\Scripts\Activate.ps1        # macOS/Linux: source .venv/bin/activate
pip install -r requirements.txt

requirements.txt is organized into a core set plus per-feature optional deps, so you only install what a given workflow needs. Notable optional groups pulled in by their sections: audio-separator[gpu] (vocal removal), panns-inference laion-clap beat-this (Deep Listen), transformers accelerate (Qwen auto-tagging), requests (Genius + Ollama), pedalboard (VST3), gradio (dashboard), and runpod (serverless). The repo's requirements appendix annotates every line — what it is and which feature needs it — so you can trim the install for a minimal deployment (e.g., a CPU-only tagging worker doesn't need the GPU audio stack).

37. Spotify playlist metadata extractor

playlist_meta.py pulls structured metadata from a public Spotify playlist — playlist name/owner/description/follower count, and per track: title, artists, album, release date, duration, popularity, explicit flag, ISRC, Spotify URL, and who added it and when. Optional flags add audio features (--audio-features: BPM/key/energy/etc., though Spotify deprecated this endpoint for apps created after 2024‑11‑27) and sample/interpolation data from Genius (--samples) or WhoSampled via RapidAPI (--whosampled). No audio is taken from Spotify — it's a pure metadata read.

Auth (free, no user login for the basics): set SPOTIFY_CLIENT_ID and SPOTIFY_CLIENT_SECRET from developer.spotify.com. Spotify now requires a one-time browser authorization to read playlist items — run once with --login; the refresh token is cached.

# markdown report to stdout, newest-added first (default)
python scripts/playlist_meta.py -pl https://open.spotify.com/playlist/<id>
# full JSON with audio features + sample data, to a file
python scripts/playlist_meta.py --playlist <id> --audio-features --samples --format json --out playlist.json
# CSV for a spreadsheet
python scripts/playlist_meta.py -pl <url> -f csv -o playlist.csv

Flags: -pl/--playlist (URL, URI, or bare id — required), -f/--format (md/json/csv), -o/--out, --sort (added/popularity/release/name), --limit, --audio-features, --samples, --whosampled, --login, --redirect-uri. Companion scripts playlist_catalog.py (a song catalog with Genius links — metadata only, never lyrics) and sample_dna.py (turns sample lineage into prompts) round out this group.

38. Hip-hop beats "inspired by" (playlist)

This workflow chains the playlist metadata into generation: take a reference playlist's characteristics (era, BPM ranges, mood/producer descriptors), translate them into pack-plan prompts, and generate beats in that spirit from your own trained model — never copying audio, only borrowing the descriptive aesthetic. In practice it's playlist_meta.py → a derived pack plan → sa3_workflow.py plan (or ace_step_workflow.py generate) → postprocess.py.

39. Cheat sheets & genre playlist finder

genre_playlists.py finds the best playlists for a genre across Spotify, YouTube, Apple Music, and SoundCloud — live API search where keys are set, and ready-to-click search URLs where they aren't (it degrades gracefully with zero keys). Supported genres map to the ones the toolkit works in: hiphop, boom_bap, trap, drill, lofi, rock, metal, rockmetal, dubstep, dnb (or all). Pair the Spotify hits with playlist_meta.py to pull full metadata.

python scripts/genre_playlists.py -g hiphop
python scripts/genre_playlists.py --genre dnb --limit 8 --format md --out dnb_playlists.md
python scripts/genre_playlists.py -g all --format json --out playlists.json

Flags: -g/--genre (default hiphop, or all), --limit (results per platform, default 5), -f/--format (md/json), -o/--out. The repo also ships cheat sheets under cheatsheets/, including cheatsheets/api-keys.md with the full per-service steps for the tokens the toolkit uses (Genius, Spotify, YouTube, RunPod, Stripe). When a section says "needs a free token," the cheat sheet has the click-by-click.

40. Engines & unified generation router

Beyond SA3 and ACE-Step, the toolkit includes a unified generation router, generate_engine.py, that exposes a single --engine switch over multiple back-ends — alongside dedicated workflows for yue_workflow.py, diffrhythm_workflow.py, musicgen_workflow.py, and ace_step_workflow.py. engine_doctor.py checks readiness and can auto-install what an engine needs. Full details live in docs/engines.md.

# route a generation through a chosen engine via the unified switch
python scripts/generate_engine.py --engine ace-step --plan prompts/pack_plan.example.json --out generated
# check which engines are installed and ready (and auto-install gaps)
python scripts/engine_doctor.py

41. Sample chopper (MPC One+ / Ableton Push)

sample_chop.py chops a sample into 5 rearranged variations ready for MPC One+ and Ableton Push 2. It detects chop points (transients, or an even --grid per bar), then rebuilds the loop using classic sampling moves — dilla (off-grid swing + micro pitch drift), chipmunk (pitched-up soul chop), stutter, reverse, and halftime. Per variation you get a rendered master.wav, numbered one-shot slices/ (one per pad), a pattern.mid, a cue-marked master_sliced.wav for auto-slicing, a manifest.json, plus best-effort MPC program.xpm and Ableton Drum Rack files.

The headline feature: 10 producer presets via --producer — j_dilla, kanye_west, dj_premier, 9th_wonder, rza, madlib, pete_rock, just_blaze, the_alchemist, knxwledge — each defining its own 5-variation style set and feel (swing, density, pitch bias, vinyl dust, quantize).

# transient-chop a loop into the default 5 variations
python scripts/sample_chop.py --input soul_loop.wav --bpm 90 --out chops
# chop in a known producer's style, 16 pads, also write an Ableton Drum Rack
python scripts/sample_chop.py --input vocal.mp3 --producer j_dilla --pads 16 --adg --out chops
# list the producer presets and exit
python scripts/sample_chop.py --list-producers

Flags: --input (required), --out (default chops), --bpm (90), --bars (2), --pads (16), --grid (0 = transient chops), --styles (comma list of 5; ignored if --producer set), --producer, --list-producers, --seed (7), --target (mpc/ableton/both), --adg (write an experimental Drum Rack), --stems (AI: split source and chop the melodic stem), --reimagine (AI: also write an audio2audio flip of each master).

42. License & notice

Co-Produce AI is built on open-weight and open-source components, each with its own terms — notably Stable Audio (Stability Community License, free for commercial use under $1M/yr), Stable Audio Open 1.0, HeartMuLa (Apache-2.0), and ACE-Step 1.5 (MIT, no revenue cap). The toolkit's own code and the rights to your trained models and outputs follow the repo's LICENSE and NOTICE. As covered in Part 1: base-model licenses, your training-data rights, and provenance are three separate things — keep all three clean before you commercialize, and treat this series as documentation, not legal advice.