# OpenAI-Compatible API

> Use FlyMy.AI LLMs with the official OpenAI SDK: base_url setup, streaming, image inputs, token usage, supported and unsupported parameters.

Every FlyMy.AI LLM endpoint also speaks the OpenAI Chat Completions protocol.
Point the official OpenAI SDK at a per-model `base_url` and it works unchanged -
same `client.chat.completions.create`, same streaming iterator, same `usage`
object. This is the fastest way to try FlyMy.AI from an existing OpenAI
codebase: change two constructor arguments, keep the rest.

```text
base_url = https://api.flymy.ai/api/v1/{endpoint_id}/openai/v1
```

The model is selected by the `base_url`, not by the `model` request field.
`model` is required by the SDK and echoed back in responses, but it does not
switch models - to use a different model, change the `base_url`.

This route is for [LLM endpoints](https://docs.flymy.ai/guides/use-llms.md) only. Image, video, and audio
models have their own APIs - see [Image Generation](https://docs.flymy.ai/guides/generate-images.md) and
[Video Generation](https://docs.flymy.ai/guides/generate-videos.md). Calling this route on a non-LLM
endpoint is unsupported: if the model happens to accept a bare prompt, the
request runs and is billed, and you get a raw file URL back instead of a chat
reply.

## Quickstart

```python
from openai import OpenAI

MODEL = "flymyai/google-gemini-25-flash"

client = OpenAI(
    base_url=f"https://api.flymy.ai/api/v1/{MODEL}/openai/v1",
    api_key="fly-***",
)

response = client.chat.completions.create(
    model=MODEL,
    messages=[
        {"role": "system", "content": "Answer briefly."},
        {"role": "user", "content": "What is a vector database?"},
    ],
)

print(response.choices[0].message.content)
print(response.usage)
```

The API key is your regular FlyMy.AI key, passed the way the OpenAI SDK always
passes it: as a `Bearer` token in the `Authorization` header. There is no
`X-API-KEY` header on this route.

Unlike the native FlyMy.AI endpoint, the sync (non-streaming) call works here:
the gateway collects the model's stream internally and returns one complete
`chat.completion` response.

## Streaming

```python
stream = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "Explain SSE in one paragraph."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\nusage:", chunk.usage)
```

Streaming follows the OpenAI wire format: `chat.completion.chunk` events with
`choices[0].delta.content`, then a final chunk with empty `choices` and the
`usage` object, then `data: [DONE]`. Guard your handler against chunks with an
empty `choices` list - the usage chunk has one, and SDK examples that index
`chunk.choices[0]` unconditionally will crash on it.

## Image inputs

Vision-capable models accept images as standard Chat Completions content
parts - public HTTP(S) URLs or base64 `data:` URLs, several per message if
needed:

```python
response = client.chat.completions.create(
    model=MODEL,
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe both images."},
            {"type": "image_url",
             "image_url": {"url": "https://example.com/photo.jpg"}},
            {"type": "image_url",
             "image_url": {"url": "data:image/png;base64,iVBORw0KG..."}},
        ],
    }],
)
```

Images are passed to the provider as-is and are billed as input tokens - the
image tokens appear in `usage.prompt_tokens`. Base64 `data:` URLs are the most
reliable way to send an image; remote URLs are fetched by the provider and add
its fetch latency to the request. Keep in mind that base64 inflates the request
body by about a third.

## Token usage and billing

Responses carry the standard OpenAI `usage` object (`prompt_tokens`,
`completion_tokens`, `total_tokens`); in streaming it arrives in the final
chunk before `[DONE]`. These are the exact counts you are billed on, priced per
1M input and output tokens. Charges land asynchronously - see
[token usage and billing](https://docs.flymy.ai/guides/use-llms.md#token-usage-and-billing) for how the
wallet sync works.

## Supported and unsupported parameters

| Parameter | Behavior |
| --- | --- |
| `messages` | Supported: text content, image `content` parts, `system` role, full history |
| `stream`, `stream_options` | Supported |
| `model` | Required, echoed back; does not select the model |
| `tools`, `tool_choice` | Function tools supported on [Grok 4.7](https://docs.flymy.ai/guides/grok-4-7.md), [DeepSeek V4.1 Flash](https://docs.flymy.ai/guides/deepseek-v4-1-flash.md) and [Kimi K3.1 Pro](https://docs.flymy.ai/guides/kimi-k3-1-pro.md) (`tool_choice` `auto` or `required` only); rejected with `400 unsupported_parameter` on other endpoints |
| `response_format` | Rejected with `400 unsupported_parameter` |
| `reasoning_effort` | `none`, `minimal`, `low`, `medium` or `high` where the model supports it; [Grok 4.7](https://docs.flymy.ai/guides/grok-4-7.md) also takes `xhigh` |
| `temperature`, `max_tokens`, `top_p`, others | Silently ignored, except on [Grok 4.7](https://docs.flymy.ai/guides/grok-4-7.md) and [DeepSeek V4.1 Flash](https://docs.flymy.ai/guides/deepseek-v4-1-flash.md), which apply `max_tokens`, `temperature` and `top_p`. [Kimi K3.1 Pro](https://docs.flymy.ai/guides/kimi-k3-1-pro.md) applies `max_tokens` and sets its own `temperature` and `top_p`: leave them out, they make a call with `tools` fail |
| `service_tier` | `default` or `priority` (alias `nitro`) on [DeepSeek V4.1 Flash](https://docs.flymy.ai/guides/deepseek-v4-1-flash.md) and [Kimi K3.1 Pro](https://docs.flymy.ai/guides/kimi-k3-1-pro.md); ignored on other endpoints |

> **Caution: Sampling parameters are ignored, not rejected**
>
> `temperature`, `max_tokens` and other tuning fields are accepted for SDK
> compatibility but have no effect. Steer the model through the prompt instead.
> Grok 4.7 and DeepSeek V4.1 Flash are the exceptions and apply them - see
> [Grok 4.7](https://docs.flymy.ai/guides/grok-4-7.md) and [DeepSeek V4.1 Flash](https://docs.flymy.ai/guides/deepseek-v4-1-flash.md).
> Kimi K3.1 Pro applies `max_tokens` only; send it no `temperature` or `top_p` -
> see [Kimi K3.1 Pro](https://docs.flymy.ai/guides/kimi-k3-1-pro.md).

## Errors

Errors use the OpenAI envelope (`error.message`, `error.type`, `error.code`),
so SDK exception handling works as usual:

| Status | `error.code` | Meaning |
| --- | --- | --- |
| 401 | `missing_authorization` / `invalid_authorization` | No or malformed `Bearer` header |
| 403 | `invalid_api_key` | Bad key, or no access to the model |
| 400 | `unsupported_parameter` | You sent `response_format`, or tools on an endpoint that does not support them |
| 422 | - | Malformed request body (standard validation response) |

## Which API should I use?

| | OpenAI-compatible | Native FlyMy.AI |
| --- | --- | --- |
| Best for | Porting existing OpenAI code, LangChain and other OpenAI-protocol tooling | New integrations, the `flymyai` SDK, uniform access to all FlyMy.AI models |
| Auth | `Authorization: Bearer fly-***` | `X-API-KEY: fly-***` |
| Chat history | `messages` array | `messages` form field (JSON string) plus `prompt` |
| Non-streaming | Yes | No - streaming endpoint only |
| Usage in response | `usage` object | `stream_details` in the final SSE event |

Both routes hit the same models at the same prices. The native format is
described in [Using LLMs](https://docs.flymy.ai/guides/use-llms.md).