---
title: Models endpoints
description: Request and response reference for Messages, Chat Completions, Responses, Gemini and model listing on Half Off Dev, with the headers each accepts.
layout: reference
---

These are the vendors' own endpoints, answered at `https://halfoff.dev`. Bodies follow each vendor's schema and pass through untouched, so this reference covers the parameters you'll reach for most and links out to the vendor docs for everything else. Behavior shared by all of them, like failover and caching, is explained in [How the gateway works](/docs/models).

Any endpoint accepts your key as `Authorization: Bearer`, `x-api-key`, `x-goog-api-key` or a `?key=` query parameter. Errors use the format of the endpoint you called; see [Errors](/docs/api/errors).

:::params Response headers
- `x-halfoff-request-id` string
  Id of the call, such as `req_6f1c0b9e2d4a47a1b8c3d5e7`, also listed in [Logs](/app/logs). Sent on successful responses and on `429` capacity and `502` errors.
- `x-halfoff-group` string
  The [channel group](/docs/models/channel-groups) that answered.
- `x-halfoff-channel` string
  That group's channel label, URL-encoded.
- `x-halfoff-queued-ms` integer
  Milliseconds spent waiting for capacity. Absent when there was no wait.
- `retry-after` integer
  Seconds to wait. Sent with `429` responses from key limits or busy capacity.
:::

## List models {#list-models}

::endpoint GET /v1/models

Returns every model you can call right now, with its context window, prices and the channel groups that serve it. No key needed. The shape is compatible with `models.list()` in both the OpenAI and Anthropic SDKs. Prices are what Half Off Dev charges, half the official list price. Field details are in [Models & pricing](/docs/models/pricing).

:::params Response fields
- `data` array
  One entry per model.
  - `id` string
    The id to put in requests.
  - `display_name` string
    Readable name.
  - `owned_by` string
    Vendor: `Anthropic`, `OpenAI` or `Google`.
  - `context_length` integer
    Context window, in tokens.
  - `pricing` object
    `input`, `output`, `cache_read` and `cache_write` in `USD per 1M tokens`. A `null` cache price means those tokens are billed at the input price.
  - `groups` array
    Channel groups serving the model.
- `has_more` boolean
  Always `false`: you get the full list in one response.
:::

```bash title="cURL" aside
curl https://halfoff.dev/v1/models
```

```python title="Python" aside
from openai import OpenAI

client = OpenAI(base_url="https://halfoff.dev/v1", api_key="not-needed-here")
print([m.id for m in client.models.list()])
```

```js title="Node.js" aside
const res = await fetch('https://halfoff.dev/v1/models');
const { data } = await res.json();
for (const m of data) console.log(m.id, m.pricing.input, m.pricing.output);
```

:::aside
```json title="Response"
{
  "object": "list",
  "data": [
    {
      "id": "claude-sonnet-5.5",
      "object": "model",
      "type": "model",
      "display_name": "Claude Sonnet 5.5",
      "created": 1700000000,
      "created_at": "2025-01-01T00:00:00Z",
      "owned_by": "Anthropic",
      "context_length": 1000000,
      "pricing": {
        "input": 1,
        "output": 5,
        "cache_read": 0.05,
        "cache_write": 1.25,
        "unit": "USD per 1M tokens"
      },
      "groups": ["claude-value"]
    }
  ],
  "has_more": false,
  "first_id": "claude-fable-5.1",
  "last_id": "gpt-5.6-luna"
}
```
:::

## Create a message {#create-a-message}

::endpoint POST /v1/messages

The [Anthropic Messages API](https://platform.claude.com/docs/en/api/messages). Set `stream` for server-sent events. Extended thinking, tool use and prompt caching work exactly as Anthropic documents them. Guide: [Anthropic Messages](/docs/models/anthropic).

:::params Headers
- `x-api-key` string, required
  Your Half Off Dev key. `Authorization: Bearer` is accepted too.
- `anthropic-version` string, default `2023-06-01`
  Passed through to the model.
- `anthropic-beta` string
  Passed through unchanged.
- `x-halfoff-group` string
  Channel group for this call, for example `claude-value`. Ignored if the key is bound to a group.
- `x-session-id` string
  A stable conversation id, so every turn goes to the channel that holds its cache.
:::

:::params Body parameters
- `model` string, required
  A model id from `GET /v1/models`, for example `claude-sonnet-5.5`. Variants like `claude-sonnet-5-5` resolve to the same model.
- `messages` array, required
  The conversation so far, alternating `user` and `assistant` turns.
- `max_tokens` integer, required
  Most tokens the model may generate.
- `system` string or array
  System prompt. Text blocks accept `cache_control`.
- `stream` boolean, default `false`
  Stream the answer as server-sent events.
- `thinking` object
  Extended thinking settings.
- `output_config` object
  Output settings such as `effort` and `format`.
- `tools` array
  Tools the model may call.
- `tool_choice` object
  How the model picks a tool.
- `metadata` object
  Request metadata.
  - `user_id` string
    Doubles as the conversation id for routing when there's no `x-session-id` header.
:::

```bash title="cURL" aside
curl https://halfoff.dev/v1/messages \
  -H "x-api-key: $HALFOFF_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-sonnet-5.5",
    "max_tokens": 512,
    "messages": [{"role": "user", "content": "Suggest a name for a budget tracking app."}]
  }'
```

```python title="Python" aside
import os
import anthropic

client = anthropic.Anthropic(
    base_url="https://halfoff.dev",
    api_key=os.environ["HALFOFF_API_KEY"],
)
message = client.messages.create(
    model="claude-sonnet-5.5",
    max_tokens=512,
    messages=[{"role": "user", "content": "Suggest a name for a budget tracking app."}],
)
print(message.content[0].text)
```

```js title="Node.js" aside
import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic({
  baseURL: 'https://halfoff.dev',
  apiKey: process.env.HALFOFF_API_KEY,
});
const message = await client.messages.create({
  model: 'claude-sonnet-5.5',
  max_tokens: 512,
  messages: [{ role: 'user', content: 'Suggest a name for a budget tracking app.' }],
});
console.log(message.content[0].text);
```

:::aside
```json title="Response"
{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "model": "claude-sonnet-5-5",
  "content": [
    {"type": "text", "text": "How about \"Pennywise\"?"}
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 16,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 0,
    "output_tokens": 9
  }
}
```
:::

## Count message tokens {#count-message-tokens}

::endpoint POST /v1/messages/count_tokens

Estimates how many input tokens a Messages request would use, without running it. Free, and it works even with a zero balance. The number is approximate: good for context-window checks and budgeting, while the actual charge comes from a real call's `usage`. See [Anthropic's token counting guide](https://platform.claude.com/docs/en/build-with-claude/token-counting).

:::params Headers
- `x-api-key` string, required
  Your Half Off Dev key. `Authorization: Bearer` is accepted too.
- `x-halfoff-group` string
  Channel group to check the model against. Ignored if the key is bound to a group.
:::

:::params Body parameters
- `model` string, required
  Model id, same as for `POST /v1/messages`.
- `messages` array, required
  Messages to count.
- `system` string or array
  System prompt to include.
- `tools` array
  Tool definitions to include.
:::

```bash title="cURL" aside
curl https://halfoff.dev/v1/messages/count_tokens \
  -H "x-api-key: $HALFOFF_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-sonnet-5.5",
       "messages": [{"role": "user", "content": "How long is this prompt?"}]}'
```

```python title="Python" aside
count = client.messages.count_tokens(
    model="claude-sonnet-5.5",
    messages=[{"role": "user", "content": "How long is this prompt?"}],
)
print(count.input_tokens)
```

```js title="Node.js" aside
const count = await client.messages.countTokens({
  model: 'claude-sonnet-5.5',
  messages: [{ role: 'user', content: 'How long is this prompt?' }],
});
console.log(count.input_tokens);
```

:::aside
```json title="Response"
{
  "input_tokens": 15
}
```
:::

## Create a chat completion {#create-a-chat-completion}

::endpoint POST /v1/chat/completions

The [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create). Every model in `GET /v1/models` answers in this format, Claude and Gemini included. Streaming requests always get `stream_options.include_usage: true`, so the final chunk carries `usage`. Guide: [OpenAI Chat Completions](/docs/models/openai).

:::params Headers
- `Authorization` string, required
  `Bearer` and your Half Off Dev key.
- `openai-beta` string
  Passed through unchanged.
- `x-halfoff-group` string
  Channel group for this call, for example `gpt-value`. Ignored if the key is bound to a group.
- `x-session-id` string
  A stable conversation id, so every turn goes to the channel that holds its cache.
:::

:::params Body parameters
- `model` string, required
  A model id from `GET /v1/models`, for example `gpt-6.1-sol`.
- `messages` array, required
  The conversation so far: `system` or `developer`, `user`, `assistant` and `tool` messages.
- `stream` boolean, default `false`
  Stream server-sent events, ending with `data: [DONE]`.
- `max_completion_tokens` integer
  Cap on generated tokens, reasoning tokens included.
- `reasoning_effort` string
  How hard a reasoning model thinks. Accepted values depend on the model.
- `tools` array
  Functions the model may call.
- `tool_choice` string or object
  How the model picks a function.
- `response_format` object
  Ask for JSON output or a JSON Schema.
- `prompt_cache_key` string
  Raises cache hit rates; also used as the conversation id for routing.
- `user` string
  Your end user's id. Used as the conversation id when nothing more specific is sent.
:::

```bash title="cURL" aside
curl https://halfoff.dev/v1/chat/completions \
  -H "Authorization: Bearer $HALFOFF_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "messages": [{"role": "user", "content": "Translate \"good morning\" into French."}]
  }'
```

```python title="Python" aside
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://halfoff.dev/v1",
    api_key=os.environ["HALFOFF_API_KEY"],
)
completion = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "Translate \"good morning\" into French."}],
)
print(completion.choices[0].message.content)
```

```js title="Node.js" aside
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://halfoff.dev/v1',
  apiKey: process.env.HALFOFF_API_KEY,
});
const completion = await client.chat.completions.create({
  model: 'gpt-6.1-sol',
  messages: [{ role: 'user', content: 'Translate "good morning" into French.' }],
});
console.log(completion.choices[0].message.content);
```

:::aside
```json title="Response"
{
  "id": "chatcmpl-B9MHDbslfkBeAs8l4bebGdFOJ6PeG",
  "object": "chat.completion",
  "created": 1791700000,
  "model": "gpt-6.1-sol",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Bonjour."},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 3,
    "total_tokens": 18,
    "prompt_tokens_details": {"cached_tokens": 0},
    "completion_tokens_details": {"reasoning_tokens": 0}
  }
}
```
:::

## Create a response {#create-a-response}

::endpoint POST /v1/responses

The [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses/create), which Codex CLI uses. GPT models only; any other model returns `404`. Nothing is stored between calls, so send the full conversation in `input` each time. Guide: [OpenAI Chat Completions](/docs/models/openai).

:::params Headers
- `Authorization` string, required
  `Bearer` and your Half Off Dev key.
- `openai-beta` string
  Passed through unchanged.
- `x-halfoff-group` string
  Channel group for this call, for example `gpt-value`. Ignored if the key is bound to a group.
- `x-session-id` string
  A stable conversation id, so every turn goes to the channel that holds its cache.
:::

:::params Body parameters
- `model` string, required
  A GPT model id from `GET /v1/models`, for example `gpt-6.1-sol`.
- `input` string or array, required
  A prompt, or the conversation as a list of input items.
- `instructions` string
  System instructions. The response echoes them back, or `null`.
- `stream` boolean, default `false`
  Stream typed server-sent events, ending with `response.completed`, which carries `usage`.
- `max_output_tokens` integer
  Cap on generated tokens, reasoning tokens included.
- `reasoning` object
  Reasoning settings such as `effort`. Accepted values depend on the model.
- `tools` array
  Functions and other tools the model may call.
- `store` boolean
  Send `false`: responses aren't kept for later retrieval.
- `prompt_cache_key` string
  Raises cache hit rates; also used as the conversation id for routing.
:::

```bash title="cURL" aside
curl https://halfoff.dev/v1/responses \
  -H "Authorization: Bearer $HALFOFF_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "input": "Give me a one-line git tip.",
    "store": false
  }'
```

```python title="Python" aside
import os
from openai import OpenAI

client = OpenAI(base_url="https://halfoff.dev/v1", api_key=os.environ["HALFOFF_API_KEY"])
response = client.responses.create(model="gpt-6.1-sol", input="Give me a one-line git tip.", store=False)
print(response.output_text)
```

```js title="Node.js" aside
import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'https://halfoff.dev/v1', apiKey: process.env.HALFOFF_API_KEY });
const response = await client.responses.create({ model: 'gpt-6.1-sol', input: 'Give me a one-line git tip.', store: false });
console.log(response.output_text);
```

```json title="Response" aside
{
  "id": "resp_06c0301443…",
  "object": "response",
  "status": "completed",
  "model": "gpt-6.1-sol",
  "instructions": null,
  "output": [
    { "type": "message", "role": "assistant", "content": [{ "type": "output_text", "text": "Use git switch - to jump back to your previous branch." }] }
  ],
  "usage": {
    "input_tokens": 14,
    "input_tokens_details": { "cached_tokens": 0 },
    "output_tokens": 13,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 27
  }
}
```

## Generate content {#generate-content}

::endpoint POST /v1beta/models/{model}:generateContent

The [Gemini API](https://ai.google.dev/api/generate-content). Guide: [Gemini](/docs/models/gemini).

:::params Path parameters
- `model` string, required
  Model id, for example `gemini-3.1-pro`. Google's spellings such as `gemini-3.1-pro-preview` resolve to the same model.
:::

:::params Headers
- `x-goog-api-key` string, required
  Your Half Off Dev key. `Authorization: Bearer` or a `?key=` query parameter also work.
- `x-halfoff-group` string
  Channel group for this call, for example `gemini-value`. Ignored if the key is bound to a group.
- `x-session-id` string
  A stable conversation id, so every turn goes to the channel that holds its cache.
:::

:::params Body parameters
- `contents` array, required
  The conversation so far, as `user` and `model` turns built from `parts`.
- `systemInstruction` object
  System instruction, as a content object with text parts.
- `generationConfig` object
  Output settings: `maxOutputTokens`, `temperature`, `responseMimeType`, `thinkingConfig` and more.
- `tools` array
  Function declarations and other tools.
- `toolConfig` object
  How the model picks functions.
- `safetySettings` array
  Safety thresholds per category.
:::

```bash title="cURL" aside
curl "https://halfoff.dev/v1beta/models/gemini-3.1-pro:generateContent" \
  -H "x-goog-api-key: $HALFOFF_API_KEY" \
  -H "content-type: application/json" \
  -d '{"contents": [{"role": "user", "parts": [{"text": "What is a haiku?"}]}]}'
```

```python title="Python" aside
import os
from google import genai
from google.genai import types

client = genai.Client(
    api_key=os.environ["HALFOFF_API_KEY"],
    http_options=types.HttpOptions(base_url="https://halfoff.dev"),
)
response = client.models.generate_content(model="gemini-3.1-pro", contents="What is a haiku?")
print(response.text)
```

```js title="Node.js" aside
import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({
  apiKey: process.env.HALFOFF_API_KEY,
  httpOptions: { baseUrl: 'https://halfoff.dev' },
});
const response = await ai.models.generateContent({ model: 'gemini-3.1-pro', contents: 'What is a haiku?' });
console.log(response.text);
```

:::aside
```json title="Response"
{
  "candidates": [
    {
      "content": {"role": "model", "parts": [{"text": "A haiku is a three-line Japanese poem of 5, 7 and 5 syllables."}]},
      "finishReason": "STOP",
      "index": 0
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 5,
    "candidatesTokenCount": 18,
    "thoughtsTokenCount": 84,
    "totalTokenCount": 107
  },
  "responseId": "mFHoaNC1Lo3HqtsP"
}
```
:::

## Stream generated content {#stream-generated-content}

::endpoint POST /v1beta/models/{model}:streamGenerateContent

The same request as `generateContent`, delivered as a stream. Add `alt=sse` for server-sent events, one partial response per event; the SDKs add it for you. Without it you get a single JSON array streamed piece by piece. See [Streaming](/docs/models/streaming).

:::params Query parameters
- `alt` string
  `sse` for server-sent events. Recommended.
:::

Path parameter, headers and body are identical to [Generate content](#generate-content).

```bash title="cURL" aside
curl -N "https://halfoff.dev/v1beta/models/gemini-3-flash:streamGenerateContent?alt=sse" \
  -H "x-goog-api-key: $HALFOFF_API_KEY" \
  -H "content-type: application/json" \
  -d '{"contents": [{"role": "user", "parts": [{"text": "List three primary colors."}]}]}'
```

```python title="Python" aside
for chunk in client.models.generate_content_stream(
    model="gemini-3-flash", contents="List three primary colors."
):
    print(chunk.text or "", end="")
```

```js title="Node.js" aside
const stream = await ai.models.generateContentStream({
  model: 'gemini-3-flash',
  contents: 'List three primary colors.',
});
for await (const chunk of stream) process.stdout.write(chunk.text ?? '');
```

:::aside
```sse title="Response"
data: {"candidates":[{"content":{"role":"model","parts":[{"text":"Red, yellow"}]}}],"usageMetadata":{"promptTokenCount":6,"candidatesTokenCount":3,"totalTokenCount":9}}

data: {"candidates":[{"content":{"role":"model","parts":[{"text":" and blue."}]},"finishReason":"STOP"}],"usageMetadata":{"promptTokenCount":6,"candidatesTokenCount":6,"totalTokenCount":12}}
```
:::

## Count Gemini tokens {#count-gemini-tokens}

::endpoint POST /v1beta/models/{model}:countTokens

Estimates the tokens in `contents` without running the model. Free, and it works even with a zero balance. The number is approximate. [Gemini reference](https://ai.google.dev/api/tokens).

:::params Body parameters
- `contents` array, required
  Content to count, shaped as for `generateContent`.
:::

Path parameter and headers are identical to [Generate content](#generate-content).

```bash title="cURL" aside
curl "https://halfoff.dev/v1beta/models/gemini-3.1-pro:countTokens" \
  -H "x-goog-api-key: $HALFOFF_API_KEY" \
  -H "content-type: application/json" \
  -d '{"contents": [{"role": "user", "parts": [{"text": "How long is this prompt?"}]}]}'
```

```python title="Python" aside
result = client.models.count_tokens(model="gemini-3.1-pro", contents="How long is this prompt?")
print(result.total_tokens)
```

```js title="Node.js" aside
const result = await ai.models.countTokens({ model: 'gemini-3.1-pro', contents: 'How long is this prompt?' });
console.log(result.totalTokens);
```

:::aside
```json title="Response"
{
  "totalTokens": 7
}
```
:::

## List models in Gemini format {#list-gemini-models}

::endpoint GET /v1beta/models

Lists the models served through the Gemini endpoints, shaped the way the Gemini API shapes them. No key needed. The Google Gen AI SDKs call it from `models.list()`. Prices appear only in `GET /v1/models`.

```bash title="cURL" aside
curl https://halfoff.dev/v1beta/models
```

:::aside
```json title="Response"
{
  "models": [
    {
      "name": "models/gemini-3.1-pro",
      "displayName": "Gemini 3.1 Pro",
      "inputTokenLimit": 1048576,
      "supportedGenerationMethods": ["generateContent", "streamGenerateContent", "countTokens"]
    }
  ]
}
```
:::