# Introduction

> FlyMy.AI Serverless CLI (fma): deploy custom Python models with auto-scaling GPU endpoints — pay only for compute you use.

FlyMyAI CLI (`fma`) is a command-line tool for deploying custom machine learning models onto FlyMyAI cloud infrastructure. It uses Pydantic with custom field types to define model inputs/outputs, and interacts with FlyMyAI services to upload and run your models on our hardware.

## Installation

```bash
pip install git+https://github.com/flymyai/fma-custom-models.git
```

Verify the installation:

```bash
fma --help
```

## Quickstart

### 1. Initialize a project

Create a new empty folder and run:

```bash
mkdir my-project && cd my-project
fma init my_model
```

This creates the following structure:

```
my-project/
├── my_model/
│   └── model.py
└── metadata.yaml
```

The generated `model.py` contains a template:

```python
from typing import List

from pydantic import BaseModel

from fma.toolkit import model as fma_model


class Input(BaseModel):
    pass


class Output(BaseModel):
    pass


class Model(fma_model.Model):
    requirements: List[str] = []

    def initialize(self):
        pass

    def predict(self, input: Input) -> Output:
        return {}
```

### 2. Develop your model

Fill in the `initialize` and `predict` methods with your inference logic:

- **`initialize()`** - runs once when the model starts. Load weights, set up pipelines, etc.
- **`predict(input)`** - runs on every inference request. Takes input and returns a result.

All imports should be placed **inside** the methods. You can store state on `self` and list Python dependencies in the `requirements` field.

> **Caution: Important**
>
> Even though I/O is described via Pydantic models, the `predict` method currently accepts and returns **plain Python dicts**, not Pydantic objects. Full Pydantic support is planned for a future release.

### 3. Log in and deploy

```bash
fma login --username <username> --password <password>
fma deploy
```

After a successful deploy, model metadata is saved into `metadata.yaml`.

## Field types

All field types can be used in both `Input` and `Output` classes.

| Type | Schema | Import | Description |
|------|--------|--------|-------------|
| `str` | `"string"` | built-in | Text data |
| `int` | `"int"` | built-in | Integer numbers |
| `float` | `"float"` | built-in | Floating-point numbers |
| `bool` | `"bool"` | built-in | Boolean values |
| `Image` | `"image"` | `fma.toolkit.fields.image` | Image data (`PIL.Image.Image`) |
| `Audio` | `"audio"` | `fma.toolkit.fields.audio` | Audio data |
| `File` | `"file"` | `fma.toolkit.fields.file` | Binary data (`io.BytesIO`) |

```python
from fma.toolkit.fields.image import Image
from fma.toolkit.fields.audio import Audio
from fma.toolkit.fields.file import File
```

> **Warning: Unsupported types**
>
> `list`, `dict`, `bytes`, `Literal`, `Enum`, and nested Pydantic models are **not supported**. Using them will raise a `ValueError` during deployment.

### Optional fields

Wrap a type in `Optional` to mark it as not required:

```python
from typing import Optional

class Input(BaseModel):
    prompt: str                    # required
    width: Optional[int]           # optional, no default
    negative_prompt: Optional[str] # optional, no default
```

> **Caution**
>
> Use `Optional[X]` from `typing`, **not** the `X | None` syntax (PEP 604). The pipe syntax is not recognized by the CLI and will cause a deployment error.

### Default values

You can set default values for fields using Pydantic's `Field` or plain Python defaults. Default values are serialized and sent to the backend, so the API will use them when a caller omits the field:

```python
from pydantic import BaseModel, Field

class Input(BaseModel):
    prompt: str
    guidance_scale: float = Field(default=7.5)
    num_steps: int = 50
```

Default values and optionality are independent - a field can have a default without being `Optional`, and vice versa.

> **Note**
>
> `None` as a default value is not serialized to the backend schema. If you need an optional field with no value by default, use `Optional[X]` without assigning a default.

## Project structure

Your project directory must follow this layout:

```
<your_model_name>/
    └── model.py
metadata.yaml
```

There should be **no other folders** in the same directory alongside your model folder - this will cause an error during deployment.

## Supported third-party packages

The infrastructure provides these pre-installed packages at fixed versions:

| Package          | Version   |
|------------------|-----------|
| `torch`          | 2.5.0     |
| `bitsandbytes`   | 0.44.1    |
| `transformers`   | 4.45.2    |
| `tokenizers`     | 0.20.1    |
| `sentencepiece`  | 0.2.0     |
| `accelerate`     | 1.0.1     |
| `numpy`          | ≤1.26.4   |

You can add any other packages via the `requirements` field in your `Model` class. The above packages are guaranteed to be available at the specified versions.