View as MarkdownPlain text for this page Open in ChatGPTQuestions about this page Open in ClaudeQuestions about this page

Models

Configure LLM models with provider factories, capabilities, and pricing tracking

What are Models?

Models in Standard Agents define the LLM configurations that your agents and prompts use for AI responses. Each model configuration specifies which provider to use, the specific model ID, capabilities, fallback models for resilience, and optional pricing for cost tracking.

Decision inference

Use state.decide({ provider, model, context, questions }) for explicit classification, scoring, and boolean probabilities. A decision provider is an independent capability; it does not serve a prompt's conversational model. TypeSafe AI implements this surface through Jev's native multi-question API. Calls automatically record request, response, duration, status, usage, and known cost in application logs. Context is supplied explicitly, and answers enter message history only when the caller explicitly adds them.

Quick Example

agents/models/heavy_thinking.ts

import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineModel({
  name: 'heavy-thinking',
  provider: openai,
  model: 'gpt-6.1-sol',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
});

Reference in a prompt:

agents/prompts/my_prompt.ts

import { definePrompt } from '@standardagents/spec';

export default definePrompt({
  name: 'my_prompt',
  toolDescription: 'Answer requests with the heavy-thinking model.',
  model: 'heavy-thinking',  // Type-safe reference
  prompt: 'You are a helpful assistant...',
});

Naming Models

This approach provides several benefits:

Swap models easily - Change the underlying model without updating prompts
Clear intent - Team members understand what each model is for
Environment flexibility - Use different models in dev vs production

Good model names:

heavy-thinking - Complex reasoning tasks
fast-response - Quick, simple responses
code-generation - Code-focused tasks
eval - Evaluation and scoring
budget - Cost-conscious general use
creative-writing - Creative content generation

Avoid:

gpt-6.1-sol - Ties the name to a specific model
claude-sonnet - Same issue, hard to swap later
model-1 - No indication of purpose

Model Providers

Standard Agents uses provider factory functions imported from provider packages. This enables typed providerOptions, runtime validation, and local provider definitions.

OpenAI

import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineModel({
  name: 'gpt-6.1-sol',
  provider: openai,
  model: 'gpt-6.1-sol',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
  providerOptions: {
    service_tier: 'default',
  },
});

API Key: Set OPENAI_API_KEY environment variable

Package: @standardagents/openai

User-Defined Providers

Use defineProvider when another API is compatible with a base provider but needs its own endpoint, API key, icon, model list, or pricing behavior.

agents/providers/darkbloom.ts

import { defineProvider, providerEnv } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineProvider({
  name: 'darkbloom',
  label: 'Darkbloom',
  baseProvider: openai,
  config: {
    apiKey: providerEnv('DARKBLOOM_API_KEY', { valueType: 'secret' }),
    baseUrl: providerEnv('DARKBLOOM_BASE_URL', {
      valueType: 'url',
      default: 'https://api.darkbloom.dev/v1',
    }),
  },
});

agents/models/darkbloom_chat.ts

import { defineModel } from '@standardagents/spec';
import darkbloom from '../providers/darkbloom';

export default defineModel({
  name: 'darkbloom_chat',
  provider: darkbloom,
  model: 'darkbloom-chat',
});

Sub-providers inherit the base provider's providerOptions type. They can override any provider method with overrides; call next() inside an override to compose with the base provider, or return your own value to replace it.

Anthropic

Anthropic's Claude models are available through a dedicated provider package built on the official @anthropic-ai/sdk:

import { defineModel } from '@standardagents/spec';
import { anthropic } from '@standardagents/anthropic';

export default defineModel({
  name: 'claude-opus',
  provider: anthropic,
  model: 'claude-opus-5-5',
  // Prices change. Check https://platform.claude.com/docs/en/about-claude/pricing
  inputPrice: 4,
  outputPrice: 20,
  providerOptions: {
    cacheSystemPrompt: true,
  },
});

API Key: Set ANTHROPIC_API_KEY environment variable

Package: @standardagents/anthropic

Cerebras

Cerebras exposes an OpenAI-compatible Chat Completions API with a dedicated provider package:

import { defineModel } from '@standardagents/spec';
import { cerebras } from '@standardagents/cerebras';

export default defineModel({
  name: 'fast-cerebras',
  provider: cerebras,
  model: 'qwen-3.8-27b',
  // Prices change. Check https://www.cerebras.ai/pricing
  inputPrice: 0.99,
  outputPrice: 1.49,
  providerOptions: {
    service_tier: 'default',
  },
});

API Key: Set CEREBRAS_API_KEY environment variable

Package: @standardagents/cerebras

Google Gemini / Imagen

Google's dedicated provider uses the official @google/genai SDK for Gemini text generation and Imagen image generation/editing:

import { defineModel } from '@standardagents/spec';
import { google } from '@standardagents/google';

export default defineModel({
  name: 'gemini-fast',
  provider: google,
  model: 'gemini-3.8-flash',
  providerOptions: {
    responseModalities: ['TEXT'],
  },
});

API Key: Set GOOGLE_API_KEY environment variable

Package: @standardagents/google

Groq

Groq exposes low-latency chat completions through the official groq-sdk:

import { defineModel } from '@standardagents/spec';
import { groq } from '@standardagents/groq';

export default defineModel({
  name: 'groq-fast',
  provider: groq,
  model: 'llama-3.1-8b-instant',
  providerOptions: {
    reasoning_format: 'parsed',
  },
});

API Key: Set GROQ_API_KEY environment variable

Package: @standardagents/groq

Novita AI

Novita AI exposes an OpenAI-compatible Chat Completions API with live model catalog pricing:

import { defineModel } from '@standardagents/spec';
import { novita } from '@standardagents/novita';

export default defineModel({
  name: 'novita-reasoning',
  provider: novita,
  model: 'deepseek/deepseek-v3.2',
  providerOptions: {
    seed: 42,
  },
});

API Key: Set NOVITA_API_KEY environment variable

Package: @standardagents/novita

DeepInfra

DeepInfra provides an OpenAI-compatible Chat Completions API backed by a live public model catalog:

import { defineModel } from '@standardagents/spec';
import { deepinfra } from '@standardagents/deepinfra';

export default defineModel({
  name: 'deepinfra-chat',
  provider: deepinfra,
  model: 'deepseek-ai/DeepSeek-V3',
  providerOptions: {
    reasoning_effort: 'high',
    service_tier: 'priority',
  },
});

API Key: Set DEEPINFRA_API_KEY

Catalog models carry live input, output, and cache-read pricing. DeepInfra's per-request usage.estimated_cost is preserved as Standard Agents usage.cost.

Package: @standardagents/deepinfra

Cloudflare Workers AI

Cloudflare Workers AI exposes an OpenAI-compatible Chat Completions endpoint scoped to your Cloudflare account:

import { defineModel } from '@standardagents/spec';
import { cloudflare } from '@standardagents/cloudflare';

export default defineModel({
  name: 'workers-fast',
  provider: cloudflare,
  model: '@cf/google/gemma-4-26b-a4b-it',
  // Prices change. Check https://developers.cloudflare.com/workers-ai/platform/pricing/
  inputPrice: 0.1,
  outputPrice: 0.3,
  providerOptions: {
    seed: 42,
  },
});

Environment: Set both CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID

Package: @standardagents/cloudflare

OpenRouter

OpenRouter provides access to models from multiple providers (OpenAI, Anthropic, Google, Meta, etc.) through a single API:

import { defineModel } from '@standardagents/spec';
import { openrouter } from '@standardagents/openrouter';

export default defineModel({
  name: 'claude-sonnet',
  provider: openrouter,
  model: 'anthropic/claude-sonnet-5.5',
  providerOptions: {
    provider: {
      zdr: true,  // Zero Data Retention
      only: ['anthropic'],
    },
  },
});

Model ID format: provider/model-name

Example models:

openai/gpt-6.1-sol
anthropic/claude-sonnet-5.5
google/gemini-3.8-flash
meta-llama/llama-3.3-70b-instruct

API Key: Set OPENROUTER_API_KEY environment variable

Package: @standardagents/openrouter

xAI

xAI uses the official @ai-sdk/xai provider for Grok chat and image models:

import { defineModel } from '@standardagents/spec';
import { xai } from '@standardagents/xai';

export default defineModel({
  name: 'grok-default',
  provider: xai,
  model: 'grok-4-0709',
  providerOptions: {
    reasoningEffort: 'high',
  },
});

API Key: Set XAI_API_KEY environment variable

Package: @standardagents/xai

Provider Options

Each provider has typed options that are validated at build time and runtime:

OpenAI Provider Options

providerOptions: {
  service_tier: 'auto', // 'auto', 'default', or 'flex'
  user: 'user-123',
  seed: 42,
  frequency_penalty: 0.5,
  presence_penalty: 0.5,
  logprobs: true,
  top_logprobs: 5,
  store: true,
  metadata: { key: 'value' },
}

OpenRouter Provider Options

providerOptions: {
  provider: {
    order: ['anthropic', 'openai'],
    allow_fallbacks: true,
    zdr: true,
    only: ['anthropic'],
    ignore: ['azure'],
    sort: 'price', // 'price', 'throughput', or 'latency'
    max_price: {
      prompt: 1,
      completion: 5,
    },
    quantizations: ['fp16', 'bf16'],
  },
}

Cerebras Provider Options

providerOptions: {
  service_tier: 'default', // 'priority', 'default', 'auto', or 'flex'
  reasoning_effort: 'high', // 'none', 'low', 'medium', or 'high'
  clear_thinking: false,
  user: 'user-123',
  seed: 42,
  logprobs: true,
  top_logprobs: 5,
}

Google Provider Options

providerOptions: {
  responseModalities: ['TEXT', 'IMAGE'],
  candidateCount: 1,
  cachedContent: 'cachedContents/...',
  thinkingConfig: {
    thinkingBudget: 1024,
  },
  imageConfig: {
    aspectRatio: '1:1',
  },
}

Groq Provider Options

providerOptions: {
  reasoning_format: 'parsed',
  reasoning_effort: 'high',
  citation_options: 'enabled',
  service_tier: 'auto',
  search_settings: {
    include_domains: ['example.com'],
  },
}

Cloudflare Provider Options

providerOptions: {
  baseUrl: 'https://api.cloudflare.com/client/v4/accounts/ACCOUNT_ID/ai/v1',
  seed: 42,
  user: 'user-123',
  logprobs: true,
  top_logprobs: 5,
  frequency_penalty: 0.5,
  presence_penalty: 0.5,
}

xAI Provider Options

providerOptions: {
  reasoningEffort: 'high',
  logprobs: true,
  topLogprobs: 5,
  aspect_ratio: '16:9',
  quality: 'high',
}

For xAI search, enable web_search or x_search in the model's providerTools. Select the enabled names in the prompt's tools.

See the Anthropic, Baseten, Cloudflare Workers AI, Cerebras, DeepInfra, Google, Groq, Novita AI, OpenAI, OpenRouter, and xAI documentation for complete options.

Model Capabilities

Set capabilities to help the framework understand what features the model supports:

export default defineModel({
  name: 'gpt-6.1-sol',
  provider: openai,
  model: 'gpt-6.1-sol',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
  capabilities: {
    supportsImages: true,
    supportsToolCalls: true,
    supportsJsonMode: true,
    supportsStreaming: true,
    maxContextTokens: 1050000,
    maxOutputTokens: 128000,
  },
});

Capability	Type	Description
`supportsImages`	`boolean`	Model can process image inputs
`supportsToolCalls`	`boolean`	Model supports function/tool calling
`supportsJsonMode`	`boolean`	Model supports structured JSON output
`supportsStreaming`	`boolean`	Model supports streaming responses
`maxContextTokens`	`number`	Maximum context window size
`maxOutputTokens`	`number`	Maximum output tokens
`reasoningLevels`	`Record<number, string \| null>`	Mapping of 0-100 scale to model-specific reasoning levels

Provider Tools

Some providers offer built-in tools. Enable the provider-defined tool names on the model with providerTools:

import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineModel({
  name: 'gpt-6.1-sol-with-tools',
  provider: openai,
  model: 'gpt-6.1-sol',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
  providerTools: ['web_search', 'code_interpreter'],
});

OpenAI Provider Tools:

web_search - Search the web with citations
file_search - Search uploaded files (requires the vectorStoreId variable)
code_interpreter - Execute Python code
image_generation - Generate images with an OpenAI image model

OpenRouter Provider Tools:

web_search - Search the web through OpenRouter server tools
web_fetch - Fetch and extract URL content through OpenRouter
datetime - Resolve the current date and time through OpenRouter
image_generation - Generate images through OpenRouter server tools

Prompts must also select the enabled provider-tool names in their tools arrays.

Provider packages translate these names into their native request format and execute them through the provider API. Standard Agents Builder discovers and selects the tools, but does not execute provider tools locally. Completed calls are recorded through the generic provider-tool log path.

Fallback Models

Fallback models provide resilience when the primary model is unavailable or fails. The system automatically retries with fallback models for:

Network errors
Rate limits (429)
Server errors (5xx)
Authentication errors (401)

Fallbacks reference other models you've defined with defineModel by their name:

// agents/models/fallback_cheap.ts
import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineModel({
  name: 'fallback-cheap',
  provider: openai,
  model: 'gpt-6-luna',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 0.1,
  outputPrice: 0.5,
});

// agents/models/primary.ts
import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

export default defineModel({
  name: 'primary-model',
  provider: openai,
  model: 'gpt-6.1-sol',
  fallbacks: ['fallback-cheap'],
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
});

Retry Sequence

Each model gets up to two attempts. Stream timeouts and missing provider credentials skip the same-model retry. A user stop ends the retry chain. Other failures use backoff. A provider's retryAfter hint can extend the delay.

For a failure that permits a same-model retry:

Primary model (attempt 1) → Failed? Retry...
Primary model (attempt 2) → Failed? Try fallback...
Fallback 1 (attempt 1) → Failed? Retry...
Fallback 1 (attempt 2) → Failed? Try next fallback...
Fallback 2 (attempt 1) → Failed? Retry...
Fallback 2 (attempt 2) → Failed? Throw error

Pricing Configuration

Configure pricing to track token costs across your application:

The examples use standard token rates for the stated context range. Prices change. Check the provider's pricing page before you set an override. Cache writes, tools, and other billing units can add costs.

defineModel({
  name: 'gpt-6.1-sol',
  provider: openai,
  model: 'gpt-6.1-sol',
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 2,
  outputPrice: 10,
  cachedPrice: 0.1,
});

Pricing units: All prices are in USD per 1 million tokens

Live pricing

Use the provider's pricing page for current rates: OpenAI, Anthropic, Google, Cerebras, or Cloudflare Workers AI.

OpenRouter rates come from its live model catalog. OpenRouter models fetch those rates at runtime.

For platform requests, check the recent requests in Settings → Billing. The request cost includes the router fee.

Type Safety

After you define models, Standard Agents populates StandardAgentSpec.ModelRegistry. StandardAgentSpec.Models resolves to its registered names:

// Auto-generated
declare global {
  namespace StandardAgentSpec {
    interface ModelRegistry {
      'gpt-6.1-sol': true;
      'claude-sonnet': true;
      'gemini-pro': true;
    }
  }
}
export {};

This enables type-safe model references in prompts:

definePrompt({
  name: 'my_prompt',
  model: 'gpt-6.1-sol',  // TypeScript validates this
  // model: 'typo',  // ❌ Error: not a valid model
  // ...
});

Common Patterns

Cost-Optimized Chain

Use cheaper models with expensive fallbacks:

import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';

defineModel({
  name: 'budget-primary',
  provider: openai,
  model: 'gpt-6-luna',
  fallbacks: ['claude-haiku'],
  // Prices change. Check https://developers.openai.com/api/docs/pricing
  // Standard rates apply to prompts up to 272K tokens.
  inputPrice: 0.1,
  outputPrice: 0.5,
});

Multi-Provider Resilience

Fallbacks across providers for maximum uptime:

import { defineModel } from '@standardagents/spec';
import { openai } from '@standardagents/openai';
import { openrouter } from '@standardagents/openrouter';

// Primary OpenAI model
defineModel({
  name: 'resilient-model',
  provider: openai,
  model: 'gpt-6.1-sol',
  fallbacks: ['openrouter-claude'],
});

// Fallback via OpenRouter
defineModel({
  name: 'openrouter-claude',
  provider: openrouter,
  model: 'anthropic/claude-sonnet-5.5',
});

Environment Setup

Configure provider API keys as environment variables. Use .dev.vars for local development and your project's environment settings on the Standard Agents platform for a deployed project. A project linked to the Standard Agents platform can use STANDARD_AGENTS_API_KEY in place of keys for Standard Agents provider packages. A request that uses the platform key costs what the Standard Agents router charged for it, router fee included. Request logs and thread cost totals show that charge in local development and in deployed projects, and include failed and retried attempts the router charged. An amount marked "~" is an estimate; the router reports the final charge within about an hour. Model prices such as inputPrice apply to requests that use your own provider key. A provider from defineProvider needs its own required variables.

A model whose provider has no key fails its run. The thread transcript names the variable to set and where to set it. For other failures the transcript shows the failure class, and the request log holds the full error. The Providers page marks the provider "Not configured".

.dev.vars

# Cloudflare Workers AI
CLOUDFLARE_API_TOKEN=...
CLOUDFLARE_ACCOUNT_ID=...

# Cerebras
CEREBRAS_API_KEY=...

# DeepInfra
DEEPINFRA_API_KEY=...

# Google
GOOGLE_API_KEY=...

# Groq
GROQ_API_KEY=...

# Novita AI
NOVITA_API_KEY=...

# Baseten Model APIs
BASETEN_API_KEY=...

# OpenAI (direct)
OPENAI_API_KEY=sk-...

# Anthropic
ANTHROPIC_API_KEY=sk-ant-...

# OpenRouter (access to multiple providers)
OPENROUTER_API_KEY=sk-or-...

# xAI
XAI_API_KEY=...

# User-defined OpenAI-compatible provider
DARKBLOOM_API_KEY=...
# DARKBLOOM_BASE_URL=https://api.darkbloom.dev/v1

For a deployed project, configure production provider keys in your project's environment settings on the Standard Agents platform.

File Organization

Models are auto-discovered from the agents/models/ directory:

agents/
└── models/
    ├── heavy_thinking.ts   # export default defineModel({...})
    ├── fast_response.ts    # export default defineModel({...})
    └── budget.ts           # export default defineModel({...})

Requirements:

Use snake_case for file names: heavy_thinking.ts, fast_response.ts
One model per file
Default export required

Next Steps

Model API Reference

Complete defineModel specification

Baseten Provider

Baseten Model APIs

Cloudflare Provider

Workers AI chat completions

Cerebras Provider

OpenAI-compatible chat completions

DeepInfra Provider

Live catalog of OpenAI-compatible chat models

Google Provider

Gemini and Imagen

Groq Provider

Groq chat completions

Novita Provider

OpenAI-compatible Novita models

OpenAI Provider

OpenAI-specific configuration

OpenRouter Provider

Multi-provider gateway

xAI Provider

Grok chat and image models

Prompts

Learn how to use models in prompts