Benchmarks

Benchmark methodology

Reproducible tests instead of vibes. Here's exactly how we measure, and what's an estimate versus a real measurement today.

How we test

1. Provision the target plan on each provider in the same region.
2. Deploy an identical Docker Compose stack (pinned image versions).
3. For Ollama: run a fixed prompt set per model, record tokens/second and first-token latency.
4. For n8n / Supabase: run a fixed workflow / query load, record throughput and p95 latency.
5. Publish raw numbers (CSV) so anyone can reproduce or challenge them.

Baseline resource estimates

Conservative starting points used by the calculator until measured data replaces them:

estimate — replace with measured data per provider as benchmarks complete.