Inference modes

Valar optimizes for throughput and cost on long-running agent work, not the latency of a single call. You pick an execution mode by how soon you need each result, then tune the cost-versus-latency trade-off with a completion window.

Completion windows at a glance

A completion window sets how much wall-clock time per turn you’re willing to trade for a lower rate. Valar runs four. Choose one with metadata.completion_window (or the X-Valar-Completion-Window header), or leave it off and get standard. Each is detailed in Completion windows below.

Window	Turnaround	Best for	Relative cost
Now (`asap`)	Immediate	Interactive and human-in-the-loop requests	Baseline
Priority (`priority`)	Under ~10 seconds	Latency-sensitive agent loops that need a firm deadline	~25% lower
Standard (`standard`)	Under a minute, usually much faster	Everyday agent loops where cost matters	~50% lower
Flex (`flex`)	Up to 5 minutes, background only	Bulk jobs, evals, and offline runs	~50% lower

The three modes

Realtime

A normal synchronous request that returns the result immediately. You send the call without background and read the output from the response. This is the lowest-latency path, finishing in seconds, and it runs on the Now completion window. Realtime works across the Responses API (/v1/responses), Chat Completions (/v1/chat/completions). Use it for interactive chat, prototyping, and human-in-the-loop steps.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.valarhq.ai/v1", # or read OPENAI_BASE_URL from the environment
    api_key="YOUR_VALAR_API_KEY",  # or read OPENAI_API_KEY from the environment
)

response = client.responses.create(
    model="moonshotai/Kimi-K2.7",
    input="Summarize the latest support ticket in one sentence.",
)

print(response.output_text)

Async

Set background=True on the Responses API. The create call returns a response id immediately, then you poll the retrieve endpoint or receive a webhook when the work finishes. On the Standard window a turn takes at most a minute and is usually much faster, so async clears high throughput at lower cost. Async jobs usually run on the Standard window (standard), the lower-cost default. Use async for agent loops, background jobs, and large fan-out. See Sending requests at scale.

started = client.responses.create(
    model="moonshotai/Kimi-K2.7",
    input="Classify this ticket and draft a reply.",
    background=True,  # returns a response id right away
    metadata={"completion_window": "standard"},
)

print("Queued:", started.id)

Batch (Private Preview)

Batch lets you submit many requests at once and retrieve the results when the set completes - the lowest cost and highest throughput, with the longest turnaround. It runs on the Standard window and suits large datasets, evals, and offline transforms. For the end-to-end workflow, see Sending requests at scale.

Compare the modes

Mode	How you call it	Typical latency	Cost	Best for
Realtime	Synchronous request, no `background`	Seconds	Highest	Interactive chat, prototyping, human-in-the-loop
Async	Responses API with `background=True`	Minutes	Lower	Agent loops, background jobs, large fan-out
Batch	Batches API, retrieve on completion	Up to hours	Lowest	Large datasets, evals, offline transforms

Completion windows

A completion window tells Valar how much wall-clock time per turn your workload can tolerate, and you pay less the more time you give it. There are four, from fastest to cheapest:

Window	Avg. turn time	Price	Best for
Now (`asap`)	Immediate	Highest	Realtime calls, interactive UIs, human-in-the-loop
Priority (`priority`)	Up to ~10 s	~25% below Now	Latency-sensitive agents that need a firm deadline
Standard (`standard`)	Up to 1 min, usually much faster	Lower	Cost-optimized agents, async loops
Flex (`flex`)	Up to 5 min	Lowest	Background bulk work that can wait

Realtime uses the Now window; async work uses Priority for a firm ~10 s deadline, Standard for lower cost, or Flex for the cheapest background runs. Each model-and-window price pairing is on the Pricing page.

Set the window

Pass metadata.completion_window on the request:

response = client.responses.create(
    model="zai-org/GLM-5.2-FP8",
    input="Explain the key ideas behind transformers.",
    background=True,
    metadata={"completion_window": "standard"},
)

Alternatively, set the X-Valar-Completion-Window header to the same value. This is useful when a client owns the request body on your behalf (for example, the Claude Agent SDK) and body metadata isn’t available. The body field takes precedence when both are set. Accepted values are "asap" (Now), "priority", "standard", and "flex".

How each tier behaves

Now runs immediately on the fastest available hardware in a latency-optimized setup, at the higher on-demand rate. Use it for realtime, interactive requests where a person or another system is waiting on the result. Priority is a durable async window that targets a tight ~10-second turn-time ceiling - a firm, low-latency completion target for agent loops - priced about 25% below the Now rate. Like the other async windows, request it explicitly with background=True. Standard is the default wherever a model supports it. It runs on Valar’s maximum-efficiency serving stack with a turn-time ceiling of one minute - and in practice most turns complete much faster than that. Most of Valar’s published prices reference this window, and it’s the right default for async agent loops and batch jobs. Flex is the lowest-cost tier, for background bulk work that can tolerate a little extra latency. It targets a turn-time ceiling of about five minutes and requires background=True - it isn’t available for realtime or synchronous calls. Flex is always explicit: Valar never selects it for you, so request it with metadata.completion_window: "flex" when you want it.

Default behavior

Leave completion_window off and the request defaults to standard when the model supports it; otherwise it falls back to the Now tier. Priority and Flex are never selected automatically - set either explicitly when you want it.

Explicitly choosing a window the model does not support fails with 400 invalid_request_error. The error message lists the model’s supported windows, and the Pricing page keeps a current support matrix.

Getting started

Build agents

Run in production

ValarCode

Platform

Inference modes

Completion windows at a glance

The three modes

Realtime

Async

Batch (Private Preview)

Compare the modes

Completion windows

Set the window

How each tier behaves

Default behavior

Next steps

Quickstart

Sending requests at scale

​Completion windows at a glance

​The three modes

​Realtime

​Async

​Batch (Private Preview)

​Compare the modes

​Completion windows

​Set the window

​How each tier behaves

​Default behavior

​Next steps

Quickstart

Sending requests at scale

Completion windows at a glance

The three modes

Realtime

Async

Batch (Private Preview)

Compare the modes

Completion windows

Set the window

How each tier behaves

Default behavior

Next steps