diff --git a/DEPLOY.md b/DEPLOY.md new file mode 100644 index 0000000..c063639 --- /dev/null +++ b/DEPLOY.md @@ -0,0 +1,241 @@ +# Deploying BgFilter to a Linux server + +This guide covers everything needed to run the BgFilter HTTP service +(`POST /remove-background`, `GET /healthz`) on a Linux server. It assumes a CPU +deployment (the default); GPU notes are called out where relevant. + +The service entry point is `app.py` (`uvicorn app:app`). See [README.md](README.md) +for the API contract and algorithm details. + +--- + +## 1. Prerequisites + +- Linux x86-64 (tested on Ubuntu; any modern distro works). +- **Python 3.11–3.13** recommended. Newer (3.14) can work, but some ML wheels may + not be published for it yet — stick to a version with prebuilt wheels for + `torch`, `onnxruntime`, `opencv`, `pymatting`. +- `git`, and the venv module (`python3-venv` on Debian/Ubuntu). +- ~3 GB free disk for the Python env + ~1 GB for model weights. +- Outbound network **once** to fetch dependencies and model weights (the running + service can then run fully offline). + +```bash +sudo apt update +sudo apt install -y python3-venv python3-pip git +``` + +## 2. Get the code + +```bash +sudo mkdir -p /opt/genarrative-image-host/bgfilter-service +sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service +cd /opt/genarrative-image-host/bgfilter-service +git clone . # or copy the repo here; use the server-edition branch +``` + +## 3. Python environment + +```bash +python3 -m venv .venv +. .venv/bin/activate +python -m pip install -U pip +``` + +## 4. Install dependencies (CPU) + +Install **CPU-only PyTorch first** from the CPU wheel index — otherwise `pip` pulls +the default CUDA build and ~2–3 GB of `nvidia-*` packages you don't need on a CPU box: + +```bash +pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu +``` + +On a headless server use **`opencv-python-headless`** instead of `opencv-python` +(the GUI build needs `libGL.so.1` etc. that servers usually lack): + +```bash +sed -i 's/^opencv-python$/opencv-python-headless/' requirements.txt +``` + +Then install the rest. `torch`/`torchvision` are already satisfied, so this won't +re-pull CUDA: + +```bash +pip install -r requirements.txt +``` + +Verify the stack imports and torch is the CPU build (`cuda= None`): + +```bash +python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \ +print('torch', torch.__version__, 'cuda=', torch.version.cuda)" +``` + +> Behind the Great Firewall, add `-i https://pypi.tuna.tsinghua.edu.cn/simple` to the +> second `pip install` for speed. + +## 5. Provide the model weights + +Three models are used: `hustvl/vitmatte-base-composition-1k` (matting), +`ZhengPeng7/BiRefNet` (default segmenter), `skytnt/anime-seg` (optional segmenter). +Pick **one** of the two layouts below. + +### Option A — HuggingFace cache + offline mode (recommended) + +Download once into a cache dir, then run offline: + +```bash +export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf +hf download hustvl/vitmatte-base-composition-1k +hf download ZhengPeng7/BiRefNet +hf download skytnt/anime-seg +``` + +At runtime set `HF_HOME` to the same path plus `HF_HUB_OFFLINE=1 +TRANSFORMERS_OFFLINE=1` (see §6). You can also download on another machine and copy +the `hub/` folder over. + +### Option B — plain folders inside the project + +Drop each model into `models//` and the loader uses it directly (no +HF cache, no network). Absent folders fall back to the repo id, so this is opt-in: + +```bash +hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k +hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet +hf download skytnt/anime-seg --local-dir models/anime-seg +``` + +Override the base directory with `BGFILTER_WEIGHTS_DIR`. `models/` is gitignored. + +### Downloading behind a firewall (CN networks) + +If direct HuggingFace access is blocked, use the mirror **with the local proxy +bypassed and Xet disabled** — the reliable combination: + +```bash +export HF_ENDPOINT=https://hf-mirror.com # domestic mirror +export NO_PROXY='*' # bypass the proxy; the mirror is direct +export HF_HUB_DISABLE_XET=1 # repos are Xet-backed; force classic HTTP +``` + +This avoids two failure modes: (1) mirror + an overseas proxy makes the mirror +redirect back to `huggingface.co`, which recent `huggingface_hub` rejects with +`FileMetadataError`; (2) with `hf-xet` installed the Xet path fails instantly. + +## 6. Configuration (environment variables) + +| Variable | Default | Purpose | +| --- | --- | --- | +| `BGFILTER_CONFIG` | `configs/default.yaml` | Pipeline config file | +| `BGFILTER_DEVICE` | `cpu` | Overrides **both** model and segmentation device (`cuda` for GPU) | +| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | Reject larger inputs with `413` (~4 MP) | +| `BGFILTER_PRELOAD` | `1` | Load default models at startup (first request isn't cold) | +| `BGFILTER_WEIGHTS_DIR` | `models` | Where local weight folders are looked up (Option B) | +| `HF_HOME` | — | HF cache location (Option A) | +| `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | Set to `1` in production once weights are cached | +| `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | Cap CPU threads (e.g. `4`) | + +## 7. Run the service + +```bash +python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 +``` + +- **Keep `--workers 1`.** Each worker loads its own full copy of the models; more + workers multiply memory (an 8 GB box will OOM). The service already serializes + inference with a global lock. +- `--host 127.0.0.1` exposes it to the local host only (intended for a co-located + caller). See §10 for remote access. + +## 8. Run as a systemd service + +`/etc/systemd/system/genarrative-bgfilter.service`: + +```ini +[Unit] +Description=Genarrative BgFilter background removal service +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +WorkingDirectory=/opt/genarrative-image-host/bgfilter-service +Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf +Environment=HF_HUB_OFFLINE=1 +Environment=TRANSFORMERS_OFFLINE=1 +Environment=HF_HUB_DISABLE_XET=1 +Environment=BGFILTER_CONFIG=configs/default.yaml +Environment=BGFILTER_DEVICE=cpu +Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304 +Environment=BGFILTER_PRELOAD=1 +Environment=OMP_NUM_THREADS=4 +Environment=MKL_NUM_THREADS=4 +ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1 +Restart=on-failure +RestartSec=5 +MemoryMax=8G +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=full +ReadWritePaths=/opt/genarrative-image-host + +[Install] +WantedBy=multi-user.target +``` + +If you use Option B (local `models/` folders) instead of the HF cache, drop the +`HF_HOME`/offline lines — those weights need no cache or network. + +```bash +sudo systemctl daemon-reload +sudo systemctl enable --now genarrative-bgfilter +sudo systemctl status genarrative-bgfilter +journalctl -u genarrative-bgfilter -f # wait for "Application startup complete" +``` + +## 9. Verify + +```bash +curl -s http://127.0.0.1:18083/healthz +# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"} + +curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ + http://127.0.0.1:18083/remove-background -o out.png +``` + +## 10. Exposing the service + +The unit binds `127.0.0.1` — intended to be called by a co-located client (e.g. the +Genarrative Rust BFF via `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`). The +service has **no authentication**. To reach it from other hosts: + +- **Reverse proxy (recommended):** put nginx/caddy in front, terminate TLS, add + auth/rate-limiting, and forward to `127.0.0.1:18083`. +- **Direct:** change `--host` to `0.0.0.0` and restrict source IPs with a firewall. + Do not expose it unauthenticated to the public internet. + +## 11. Capacity & performance + +- CPU throughput is ~**45 s per 1.5 MP image** with the default (base ViTMatte + + BiRefNet); requests are serialized by the global lock, so ~1–2 images/min per + instance. Bottleneck is full-resolution ViTMatte + pymatting foreground estimation. +- For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) — + roughly an order of magnitude faster — and/or run multiple instances behind a load + balancer (each with `--workers 1`). +- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G` + is a safe ceiling for a single CPU worker. + +## 12. Troubleshooting + +| Symptom | Cause / fix | +| --- | --- | +| `pip` pulls `nvidia-*`/`cuda-*` (huge) | You skipped the CPU index in §4. Install torch from `--index-url https://download.pytorch.org/whl/cpu`. | +| `ImportError: libGL.so.1` on `import cv2` | Using `opencv-python` on a headless box. Switch to `opencv-python-headless` (§4). | +| `ensurepip is not available` on venv create | Install `python3-venv` (§1). | +| `FileMetadataError: ... not on huggingface.co` while downloading | hf-mirror + a proxy conflict. Bypass the proxy (`NO_PROXY='*'`) and set `HF_HUB_DISABLE_XET=1` (§5). | +| `LocalEntryNotFoundError`, 0 bytes | Xet download path failing; set `HF_HUB_DISABLE_XET=1`. | +| `503 service not ready` | Models still loading at startup; wait for "Application startup complete". | +| OOM / killed under load | More than one worker, or `MemoryMax` too low. Keep `--workers 1`. | +| `400 could not auto-detect background colour` | Input has no clean flat border; pass `screen_color=#RRGGBB`. | diff --git a/README.md b/README.md index bc5ba34..c899a31 100644 --- a/README.md +++ b/README.md @@ -131,6 +131,8 @@ metadata.json An HTTP wrapper (`app.py` + `bgfilter/service.py`) exposes the pipeline as a long-running FastAPI service. Models load once and are reused across requests. +For deploying to a Linux server (dependencies, weights, systemd, offline mode), see +[DEPLOY.md](DEPLOY.md). ```powershell D:\MiniConda\envs\lightML\python.exe -m uvicorn app:app `