# Deploying BgFilter to a Linux server

This guide covers everything needed to run the BgFilter HTTP service
(`POST /remove-background`, `GET /healthz`) on a Linux server. It assumes a CPU
deployment (the default); GPU notes are called out where relevant.

The service entry point is `app.py` (`uvicorn app:app`). See [README.md](README.md)
for the API contract and algorithm details.

---

## 1. Prerequisites

- Linux x86-64 (tested on Ubuntu; any modern distro works).
- **Python 3.11–3.13** recommended. Newer (3.14) can work, but some ML wheels may
  not be published for it yet — stick to a version with prebuilt wheels for
  `torch`, `onnxruntime`, `opencv`, `pymatting`.
- `git`, and the venv module (`python3-venv` on Debian/Ubuntu).
- ~3 GB free disk for the Python env + ~1 GB for model weights.
- Outbound network **once** to fetch dependencies and model weights (the running
  service can then run fully offline).

```bash
sudo apt update
sudo apt install -y python3-venv python3-pip git
```

## 2. Get the code

```bash
sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
cd /opt/genarrative-image-host/bgfilter-service
git clone <repo-url> .            # or copy the repo here; use the server-edition branch
```

## 3. Python environment

```bash
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -U pip
```

## 4. Install dependencies (CPU)

Install **CPU-only PyTorch first** from the CPU wheel index — otherwise `pip` pulls
the default CUDA build and ~2–3 GB of `nvidia-*` packages you don't need on a CPU box:

```bash
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
```

On a headless server use **`opencv-python-headless`** instead of `opencv-python`
(the GUI build needs `libGL.so.1` etc. that servers usually lack):

```bash
sed -i 's/^opencv-python$/opencv-python-headless/' requirements.txt
```

Then install the rest. `torch`/`torchvision` are already satisfied, so this won't
re-pull CUDA:

```bash
pip install -r requirements.txt
```

Verify the stack imports and torch is the CPU build (`cuda= None`):

```bash
python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"
```

> Behind the Great Firewall, add `-i https://pypi.tuna.tsinghua.edu.cn/simple` to the
> second `pip install` for speed.

## 5. Provide the model weights

Three models are used: `hustvl/vitmatte-base-composition-1k` (matting),
`ZhengPeng7/BiRefNet` (default segmenter), `skytnt/anime-seg` (optional segmenter).
Pick **one** of the two layouts below.

### Option A — HuggingFace cache + offline mode (recommended)

Download once into a cache dir, then run offline:

```bash
export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
hf download hustvl/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet
hf download skytnt/anime-seg
```

At runtime set `HF_HOME` to the same path plus `HF_HUB_OFFLINE=1
TRANSFORMERS_OFFLINE=1` (see §6). You can also download on another machine and copy
the `hub/` folder over.

### Option B — plain folders inside the project

Drop each model into `models/<repo-basename>/` and the loader uses it directly (no
HF cache, no network). Absent folders fall back to the repo id, so this is opt-in:

```bash
hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
hf download ZhengPeng7/BiRefNet                 --local-dir models/BiRefNet
hf download skytnt/anime-seg                     --local-dir models/anime-seg
```

Override the base directory with `BGFILTER_WEIGHTS_DIR`. `models/` is gitignored.

### Downloading behind a firewall (CN networks)

If direct HuggingFace access is blocked, use the mirror **with the local proxy
bypassed and Xet disabled** — the reliable combination:

```bash
export HF_ENDPOINT=https://hf-mirror.com   # domestic mirror
export NO_PROXY='*'                        # bypass the proxy; the mirror is direct
export HF_HUB_DISABLE_XET=1                 # repos are Xet-backed; force classic HTTP
```

This avoids two failure modes: (1) mirror + an overseas proxy makes the mirror
redirect back to `huggingface.co`, which recent `huggingface_hub` rejects with
`FileMetadataError`; (2) with `hf-xet` installed the Xet path fails instantly.

## 6. Configuration (environment variables)

| Variable | Default | Purpose |
| --- | --- | --- |
| `BGFILTER_CONFIG` | `configs/default.yaml` | Pipeline config file |
| `BGFILTER_DEVICE` | `cpu` | Overrides **both** model and segmentation device (`cuda` for GPU) |
| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | Reject larger inputs with `413` (~4 MP) |
| `BGFILTER_PRELOAD` | `1` | Load default models at startup (first request isn't cold) |
| `BGFILTER_WEIGHTS_DIR` | `models` | Where local weight folders are looked up (Option B) |
| `HF_HOME` | — | HF cache location (Option A) |
| `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | Set to `1` in production once weights are cached |
| `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | Cap CPU threads (e.g. `4`) |

## 7. Run the service

```bash
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
```

- **Keep `--workers 1`.** Each worker loads its own full copy of the models; more
  workers multiply memory (an 8 GB box will OOM). The service already serializes
  inference with a global lock.
- `--host 127.0.0.1` exposes it to the local host only (intended for a co-located
  caller). See §10 for remote access.

## 8. Run as a systemd service

`/etc/systemd/system/genarrative-bgfilter.service`:

```ini
[Unit]
Description=Genarrative BgFilter background removal service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
Environment=HF_HUB_OFFLINE=1
Environment=TRANSFORMERS_OFFLINE=1
Environment=HF_HUB_DISABLE_XET=1
Environment=BGFILTER_CONFIG=configs/default.yaml
Environment=BGFILTER_DEVICE=cpu
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
Environment=BGFILTER_PRELOAD=1
Environment=OMP_NUM_THREADS=4
Environment=MKL_NUM_THREADS=4
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
Restart=on-failure
RestartSec=5
MemoryMax=8G
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=/opt/genarrative-image-host

[Install]
WantedBy=multi-user.target
```

If you use Option B (local `models/` folders) instead of the HF cache, drop the
`HF_HOME`/offline lines — those weights need no cache or network.

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now genarrative-bgfilter
sudo systemctl status genarrative-bgfilter
journalctl -u genarrative-bgfilter -f          # wait for "Application startup complete"
```

## 9. Verify

```bash
curl -s http://127.0.0.1:18083/healthz
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}

curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
  http://127.0.0.1:18083/remove-background -o out.png
```

## 10. Exposing the service

The unit binds `127.0.0.1` — intended to be called by a co-located client (e.g. the
Genarrative Rust BFF via `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`). The
service has **no authentication**. To reach it from other hosts:

- **Reverse proxy (recommended):** put nginx/caddy in front, terminate TLS, add
  auth/rate-limiting, and forward to `127.0.0.1:18083`.
- **Direct:** change `--host` to `0.0.0.0` and restrict source IPs with a firewall.
  Do not expose it unauthenticated to the public internet.

## 11. Capacity & performance

- CPU throughput is ~**45 s per 1.5 MP image** with the default (base ViTMatte +
  BiRefNet); requests are serialized by the global lock, so ~1–2 images/min per
  instance. Bottleneck is full-resolution ViTMatte + pymatting foreground estimation.
- For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) —
  roughly an order of magnitude faster — and/or run multiple instances behind a load
  balancer (each with `--workers 1`).
- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G`
  is a safe ceiling for a single CPU worker.

## 12. Troubleshooting

| Symptom | Cause / fix |
| --- | --- |
| `pip` pulls `nvidia-*`/`cuda-*` (huge) | You skipped the CPU index in §4. Install torch from `--index-url https://download.pytorch.org/whl/cpu`. |
| `ImportError: libGL.so.1` on `import cv2` | Using `opencv-python` on a headless box. Switch to `opencv-python-headless` (§4). |
| `ensurepip is not available` on venv create | Install `python3-venv` (§1). |
| `FileMetadataError: ... not on huggingface.co` while downloading | hf-mirror + a proxy conflict. Bypass the proxy (`NO_PROXY='*'`) and set `HF_HUB_DISABLE_XET=1` (§5). |
| `LocalEntryNotFoundError`, 0 bytes | Xet download path failing; set `HF_HUB_DISABLE_XET=1`. |
| `503 service not ready` | Models still loading at startup; wait for "Application startup complete". |
| OOM / killed under load | More than one worker, or `MemoryMax` too low. Keep `--workers 1`. |
| `400 could not auto-detect background colour` | Input has no clean flat border; pass `screen_color=#RRGGBB`. |