Errors

Half-price tokens are only cheap if your code handles failures well. Half Off Dev keeps this simple: errors use plain HTTP status codes, and the body is written in the format of the API you called, so the Anthropic, OpenAI and Google SDKs raise the same exceptions they would against the vendor. As a rule of thumb, 4xx means change the request (or add credit), while 429 and 5xx are temporary and worth retrying.

Status codes at a glance#

Handle any other 5xx the way you handle 502. Requests refused before they reach a model are never charged: 400 for invalid JSON or an unknown group, 401, 402, 403, 404 for an unknown model or endpoint, and 429 from a key limit.

Error body formats#

The same 401 looks like this in each of the three formats:

{"type": "error", "error": {"type": "authentication_error", "message": "Invalid or revoked API key"}}

Type and status for each code#

A Gemini body's error.code is always the HTTP status, and both 402 and 429 map to RESOURCE_EXHAUSTED, so branch on the number, not the string. Even errors raised before the body is read, like an unknown path, use the format that matches the path you called.

Messages you may see#

Every error response carries an x-halfoff-request-id header, even when the request never reached a model (bad key, empty balance, unknown model). Include it when you write to support.

Errors inside a stream#

Once a stream is open, the 200 status has already been sent, so a later problem arrives as an event in the stream. Failures before the first token are retried on another channel behind the scenes, which means a stream usually either succeeds or fails with an ordinary HTTP error before it starts.

If the model goes silent for 3 minutes in the middle of a stream, Half Off Dev closes it with an error event in that stream's format:

event: error
data: {"type": "error", "error": {"type": "api_error", "message": "Upstream stopped responding mid-stream"}}

Errors that a channel reports partway through a stream are swapped for a generic event in the same format. Busy capacity becomes Anthropic overloaded_error, OpenAI rate_limit_exceeded or Gemini RESOURCE_EXHAUSTED, with the message Upstream capacity for this model is busy. Retry shortly. Everything else becomes api_error (Gemini UNAVAILABLE) with The model provider returned an error. Please retry. On the Responses API, the event is an error event such as {"type": "error", "code": "server_error", "message": "…"}. Start the request over; output streamed before the error is charged.

Retry strategy#

Half Off Dev already moves a failing request to other channels before returning an error. Add your own retries for what's left:

• Retry on 429, 500, 502, 503, 504 and network errors.
• Wait for retry-after when the header is there. Otherwise back off exponentially, 1, 2, 4, 8 seconds and so on up to 30, with up to a second of random jitter so your workers don't retry in lockstep.
• Give up after about five attempts and report the error.
• Don't repeat 400, 401, 403, 404, 413 or 422 unchanged. A 402 only makes sense to retry after adding credit.
• Remember timeouts. A request that timed out on your side may have completed and been charged. See Retries and idempotency.

The official Anthropic and OpenAI SDKs already retry 429 and 5xx twice with backoff; set max_retries (Python) or maxRetries (Node.js) on the client to change that. For raw HTTP calls, a helper like this does the job:

import os
import random
import time

import requests

RETRY_ON = {429, 500, 502, 503, 504}


def wait_time(attempt, retry_after=None):
    if retry_after:
        return float(retry_after)
    return min(30, 2 ** (attempt - 1)) + random.random()


def call_with_retries(url, payload, attempts=5):
    headers = {"Authorization": f"Bearer {os.environ['HALFOFF_API_KEY']}"}
    for attempt in range(1, attempts + 1):
        try:
            resp = requests.post(url, json=payload, headers=headers, timeout=300)
        except (requests.ConnectionError, requests.Timeout):
            if attempt == attempts:
                raise
            time.sleep(wait_time(attempt))
            continue
        if resp.status_code in RETRY_ON and attempt < attempts:
            time.sleep(wait_time(attempt, resp.headers.get("retry-after")))
            continue
        if not resp.ok:  # a 4xx we shouldn't repeat, or the last attempt failed
            raise RuntimeError(f"{resp.status_code}: {resp.text[:500]}")
        return resp.json()


answer = call_with_retries(
    "https://halfoff.dev/v1/chat/completions",
    {
        "model": "gpt-6.1-sol",
        "messages": [{"role": "user", "content": "Name three uses for a paperclip."}],
    },
)
print(answer["choices"][0]["message"]["content"])

Whatever the format, error.message is written for humans. Log it together with the status code and the x-halfoff-request-id header.