AI providers
Settings → AI Providers (/admin/settings/ai/providers) is where the LLM judge-vendor credentials live: the API keys — and the per-vendor concurrency bound — for the server-side eval judge. This page is credentials only; it doesn’t pick which vendor is active (that’s Evals:Agent:Provider on Evals), and it no longer holds embedding credentials — those moved to Embeddings.
Judge vendor credentials
Section titled “Judge vendor credentials”Configure one or more vendors to enable the server-side LLM-as-judge runner. When several are configured, the active one is chosen on Evals; the judge model is set there too (Evals:Agent:Model), not per-vendor here.
Anthropic
Section titled “Anthropic”| Key | Range | Default | Description |
|---|---|---|---|
Evals:ServerRunner:Anthropic:ApiKey | secret | — | The Anthropic API key. Stored encrypted at rest. |
Evals:ServerRunner:Anthropic:MaxParallelJudges | 1–16 | 4 | Judge fan-out bound for this vendor. |
OpenAI
Section titled “OpenAI”| Key | Range | Default | Description |
|---|---|---|---|
Evals:ServerRunner:OpenAI:ApiKey | secret | — | The OpenAI API key. Stored encrypted at rest. |
Evals:ServerRunner:OpenAI:MaxParallelJudges | 1–16 | 4 | Judge fan-out bound for this vendor. |
Kurrent AI
Section titled “Kurrent AI”Kurrent AI is Capacitor’s fully-managed judge and embedding service — Kurrent-hosted, with no API key to supply, available on the Team and Enterprise plans. Learn more about Kurrent AI and how it’s gated across plans.
Its API token, endpoint, and gateway are managed by Kurrent (set at deployment); the only editable setting here is the concurrency bound:
| Key | Range | Default | Description |
|---|---|---|---|
Evals:ServerRunner:KurrentAI:MaxParallelJudges | 1–16 | 4 | Judge fan-out bound for this vendor. |
Cohere
Section titled “Cohere”Hosted or on-prem:
- Hosted Cohere API — set just the API key.
- On-prem / airgapped deployment — set the base URL to your API base, e.g.
https://cohere.internal(omit/v2/chat; it’s appended). The key is then optional.
| Key | Range | Default | Description |
|---|---|---|---|
Evals:ServerRunner:Cohere:BaseUrl | string | — | Blank uses the hosted SaaS API. Set for an on-prem deployment. |
Evals:ServerRunner:Cohere:ApiKey | secret | — | Required for the hosted API; optional on-prem. |
Evals:ServerRunner:Cohere:MaxParallelJudges | 1–16 | 4 | Judge fan-out bound for this vendor. |
OpenRouter
Section titled “OpenRouter”Bring-your-own, OpenAI-compatible — point the runner at OpenRouter or any compatible gateway.
| Key | Range | Default | Description |
|---|---|---|---|
Evals:ServerRunner:OpenRouter:ApiKey | secret | — | The OpenRouter API key. Stored encrypted at rest. |
Evals:ServerRunner:OpenRouter:BaseUrl | string | https://openrouter.ai/api/v1 | Optional; override for a compatible gateway. |
Evals:ServerRunner:OpenRouter:MaxParallelJudges | 1–16 | 4 | Judge fan-out bound for this vendor. |
Embedding credentials
Section titled “Embedding credentials”Embedding provider and credentials moved to their own page — see Embeddings.
Model pricing
Section titled “Model pricing”Estimated cost rates sync automatically from models.dev — no configuration needed. The page shows the pricing source, the count of priced models, and the last sync time, with a warning if the last successful sync is more than 48 hours old.
Related
Section titled “Related”- Evals — selects the active judge vendor and model, and sets server-runner run limits.
- Embeddings — the embedding provider and credentials that used to live here.
- Assistants — reuses these same vendor credentials.