> ## Documentation Index
> Fetch the complete documentation index at: https://docs.valarhq.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Inference modes
> Pick how to run inference on Valar by how soon you need each result, and tune cost with completion windows
Valar optimizes for throughput and cost on long-running agent work, not the latency of a single call. You pick an execution mode by how soon you need each result, then tune the cost-versus-latency trade-off with a completion window.
## Completion windows at a glance
A completion window sets how much wall-clock time per turn you're willing to trade for a lower rate. Valar runs four. Choose one with `metadata.completion_window` (or the `X-Valar-Completion-Window` header), or leave it off and get `standard`. Each is detailed in [Completion windows](#completion-windows) below.
| Window | Turnaround | Best for | Relative cost |
| --------------------- | ----------------------------------- | ------------------------------------------------------- | ------------- |
| Now (`asap`) | Immediate | Interactive and human-in-the-loop requests | Baseline |
| Priority (`priority`) | Under \~10 seconds | Latency-sensitive agent loops that need a firm deadline | \~25% lower |
| Standard (`standard`) | Under a minute, usually much faster | Everyday agent loops where cost matters | \~50% lower |
| Flex (`flex`) | Up to 5 minutes, background only | Bulk jobs, evals, and offline runs | \~50% lower |
## The three modes
### Realtime
A normal synchronous request that returns the result immediately. You send the call without `background` and read the output from the response. This is the lowest-latency path, finishing in seconds, and it runs on the **Now** completion window.
Realtime works across the [Responses API](/api-reference/responses-api/create-a-response) (`/v1/responses`), Chat Completions (`/v1/chat/completions`). Use it for interactive chat, prototyping, and human-in-the-loop steps.
```python theme={"system"}
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.valarhq.ai/v1", # or read OPENAI_BASE_URL from the environment
api_key="YOUR_VALAR_API_KEY", # or read OPENAI_API_KEY from the environment
)
response = client.responses.create(
model="moonshotai/Kimi-K2.7",
input="Summarize the latest support ticket in one sentence.",
)
print(response.output_text)
```
### Async
Set `background=True` on the Responses API. The create call returns a response id immediately, then you poll the [retrieve endpoint](/api-reference/responses-api/retrieve-a-response) or receive a [webhook](/webhooks) when the work finishes. On the Standard window a turn takes at most a minute and is usually much faster, so async clears high throughput at lower cost.
Async jobs usually run on the **Standard** window (`standard`), the lower-cost default. Use async for agent loops, background jobs, and large fan-out. See [Sending requests at scale](/requests_at_scale).
```python theme={"system"}
started = client.responses.create(
model="moonshotai/Kimi-K2.7",
input="Classify this ticket and draft a reply.",
background=True, # returns a response id right away
metadata={"completion_window": "standard"},
)
print("Queued:", started.id)
```
### Batch (Private Preview)
Batch lets you submit many requests at once and retrieve the results when the set completes - the lowest cost and highest throughput, with the longest turnaround. It runs on the **Standard** window and suits large datasets, evals, and offline transforms. For the end-to-end workflow, see [Sending requests at scale](/requests_at_scale).
## Compare the modes
| Mode | How you call it | Typical latency | Cost | Best for |
| -------- | ------------------------------------ | --------------- | ------- | ------------------------------------------------ |
| Realtime | Synchronous request, no `background` | Seconds | Highest | Interactive chat, prototyping, human-in-the-loop |
| Async | Responses API with `background=True` | Minutes | Lower | Agent loops, background jobs, large fan-out |
| Batch | Batches API, retrieve on completion | Up to hours | Lowest | Large datasets, evals, offline transforms |
## Completion windows
A completion window tells Valar how much wall-clock time per turn your workload can tolerate, and you pay less the more time you give it. There are four, from fastest to cheapest:
| Window | Avg. turn time | Price | Best for |
| --------------------- | -------------------------------- | --------------- | -------------------------------------------------- |
| Now (`asap`) | Immediate | Highest | Realtime calls, interactive UIs, human-in-the-loop |
| Priority (`priority`) | Up to \~10 s | \~25% below Now | Latency-sensitive agents that need a firm deadline |
| Standard (`standard`) | Up to 1 min, usually much faster | Lower | Cost-optimized agents, async loops |
| Flex (`flex`) | Up to 5 min | Lowest | Background bulk work that can wait |
Realtime uses the **Now** window; async work uses **Priority** for a firm \~10 s deadline, **Standard** for lower cost, or **Flex** for the cheapest background runs. Each model-and-window price pairing is on the [Pricing](/pricing) page.
### Set the window
Pass `metadata.completion_window` on the request:
```python theme={"system"}
response = client.responses.create(
model="zai-org/GLM-5.2-FP8",
input="Explain the key ideas behind transformers.",
background=True,
metadata={"completion_window": "standard"},
)
```
Alternatively, set the `X-Valar-Completion-Window` header to the same value. This is useful when a client owns the request body on your behalf (for example, the Claude Agent SDK) and body metadata isn't available. The body field takes precedence when both are set.
Accepted values are `"asap"` (Now), `"priority"`, `"standard"`, and `"flex"`.
### How each tier behaves
**Now** runs immediately on the fastest available hardware in a latency-optimized setup, at the higher on-demand rate. Use it for realtime, interactive requests where a person or another system is waiting on the result.
**Priority** is a durable async window that targets a tight \~10-second turn-time ceiling - a firm, low-latency completion target for agent loops - priced about 25% below the Now rate. Like the other async windows, request it explicitly with `background=True`.
**Standard** is the default wherever a model supports it. It runs on Valar's maximum-efficiency serving stack with a turn-time ceiling of one minute - and in practice most turns complete much faster than that. Most of Valar's published prices reference this window, and it's the right default for async agent loops and batch jobs.
**Flex** is the lowest-cost tier, for background bulk work that can tolerate a little extra latency. It targets a turn-time ceiling of about five minutes and **requires `background=True`** - it isn't available for realtime or synchronous calls. Flex is always explicit: Valar never selects it for you, so request it with `metadata.completion_window: "flex"` when you want it.
### Default behavior
Leave `completion_window` off and the request defaults to `standard` when the model supports it; otherwise it falls back to the **Now** tier. **Priority** and **Flex** are never selected automatically - set either explicitly when you want it.
Explicitly choosing a window the model does not support fails with `400 invalid_request_error`. The error message lists the model's supported windows, and the [Pricing](/pricing) page keeps a current support matrix.
## Next steps
Send your first request with an OpenAI-compatible client.
Fan out async and batch work across many requests.