Add DEPLOY.md: Linux server deployment guide
Document deploying the FastAPI service to a Linux server end to end: venv + CPU-only torch (avoid the CUDA build), opencv-python-headless, the two weight layouts (HF cache offline vs local models/ folders) with the mirror/proxy/Xet download workaround, environment variables, uvicorn --workers 1, a systemd unit, verification, remote exposure, capacity notes, and a troubleshooting table. Link to it from README. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,241 @@
|
||||
# Deploying BgFilter to a Linux server
|
||||
|
||||
This guide covers everything needed to run the BgFilter HTTP service
|
||||
(`POST /remove-background`, `GET /healthz`) on a Linux server. It assumes a CPU
|
||||
deployment (the default); GPU notes are called out where relevant.
|
||||
|
||||
The service entry point is `app.py` (`uvicorn app:app`). See [README.md](README.md)
|
||||
for the API contract and algorithm details.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
- Linux x86-64 (tested on Ubuntu; any modern distro works).
|
||||
- **Python 3.11–3.13** recommended. Newer (3.14) can work, but some ML wheels may
|
||||
not be published for it yet — stick to a version with prebuilt wheels for
|
||||
`torch`, `onnxruntime`, `opencv`, `pymatting`.
|
||||
- `git`, and the venv module (`python3-venv` on Debian/Ubuntu).
|
||||
- ~3 GB free disk for the Python env + ~1 GB for model weights.
|
||||
- Outbound network **once** to fetch dependencies and model weights (the running
|
||||
service can then run fully offline).
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y python3-venv python3-pip git
|
||||
```
|
||||
|
||||
## 2. Get the code
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/genarrative-image-host/bgfilter-service
|
||||
sudo chown "$USER" /opt/genarrative-image-host/bgfilter-service
|
||||
cd /opt/genarrative-image-host/bgfilter-service
|
||||
git clone <repo-url> . # or copy the repo here; use the server-edition branch
|
||||
```
|
||||
|
||||
## 3. Python environment
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
. .venv/bin/activate
|
||||
python -m pip install -U pip
|
||||
```
|
||||
|
||||
## 4. Install dependencies (CPU)
|
||||
|
||||
Install **CPU-only PyTorch first** from the CPU wheel index — otherwise `pip` pulls
|
||||
the default CUDA build and ~2–3 GB of `nvidia-*` packages you don't need on a CPU box:
|
||||
|
||||
```bash
|
||||
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
|
||||
```
|
||||
|
||||
On a headless server use **`opencv-python-headless`** instead of `opencv-python`
|
||||
(the GUI build needs `libGL.so.1` etc. that servers usually lack):
|
||||
|
||||
```bash
|
||||
sed -i 's/^opencv-python$/opencv-python-headless/' requirements.txt
|
||||
```
|
||||
|
||||
Then install the rest. `torch`/`torchvision` are already satisfied, so this won't
|
||||
re-pull CUDA:
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Verify the stack imports and torch is the CPU build (`cuda= None`):
|
||||
|
||||
```bash
|
||||
python -c "import torch,cv2,fastapi,transformers,pymatting,onnxruntime,kornia,timm; \
|
||||
print('torch', torch.__version__, 'cuda=', torch.version.cuda)"
|
||||
```
|
||||
|
||||
> Behind the Great Firewall, add `-i https://pypi.tuna.tsinghua.edu.cn/simple` to the
|
||||
> second `pip install` for speed.
|
||||
|
||||
## 5. Provide the model weights
|
||||
|
||||
Three models are used: `hustvl/vitmatte-base-composition-1k` (matting),
|
||||
`ZhengPeng7/BiRefNet` (default segmenter), `skytnt/anime-seg` (optional segmenter).
|
||||
Pick **one** of the two layouts below.
|
||||
|
||||
### Option A — HuggingFace cache + offline mode (recommended)
|
||||
|
||||
Download once into a cache dir, then run offline:
|
||||
|
||||
```bash
|
||||
export HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
|
||||
hf download hustvl/vitmatte-base-composition-1k
|
||||
hf download ZhengPeng7/BiRefNet
|
||||
hf download skytnt/anime-seg
|
||||
```
|
||||
|
||||
At runtime set `HF_HOME` to the same path plus `HF_HUB_OFFLINE=1
|
||||
TRANSFORMERS_OFFLINE=1` (see §6). You can also download on another machine and copy
|
||||
the `hub/` folder over.
|
||||
|
||||
### Option B — plain folders inside the project
|
||||
|
||||
Drop each model into `models/<repo-basename>/` and the loader uses it directly (no
|
||||
HF cache, no network). Absent folders fall back to the repo id, so this is opt-in:
|
||||
|
||||
```bash
|
||||
hf download hustvl/vitmatte-base-composition-1k --local-dir models/vitmatte-base-composition-1k
|
||||
hf download ZhengPeng7/BiRefNet --local-dir models/BiRefNet
|
||||
hf download skytnt/anime-seg --local-dir models/anime-seg
|
||||
```
|
||||
|
||||
Override the base directory with `BGFILTER_WEIGHTS_DIR`. `models/` is gitignored.
|
||||
|
||||
### Downloading behind a firewall (CN networks)
|
||||
|
||||
If direct HuggingFace access is blocked, use the mirror **with the local proxy
|
||||
bypassed and Xet disabled** — the reliable combination:
|
||||
|
||||
```bash
|
||||
export HF_ENDPOINT=https://hf-mirror.com # domestic mirror
|
||||
export NO_PROXY='*' # bypass the proxy; the mirror is direct
|
||||
export HF_HUB_DISABLE_XET=1 # repos are Xet-backed; force classic HTTP
|
||||
```
|
||||
|
||||
This avoids two failure modes: (1) mirror + an overseas proxy makes the mirror
|
||||
redirect back to `huggingface.co`, which recent `huggingface_hub` rejects with
|
||||
`FileMetadataError`; (2) with `hf-xet` installed the Xet path fails instantly.
|
||||
|
||||
## 6. Configuration (environment variables)
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `BGFILTER_CONFIG` | `configs/default.yaml` | Pipeline config file |
|
||||
| `BGFILTER_DEVICE` | `cpu` | Overrides **both** model and segmentation device (`cuda` for GPU) |
|
||||
| `BGFILTER_MAX_IMAGE_PIXELS` | `4194304` | Reject larger inputs with `413` (~4 MP) |
|
||||
| `BGFILTER_PRELOAD` | `1` | Load default models at startup (first request isn't cold) |
|
||||
| `BGFILTER_WEIGHTS_DIR` | `models` | Where local weight folders are looked up (Option B) |
|
||||
| `HF_HOME` | — | HF cache location (Option A) |
|
||||
| `HF_HUB_OFFLINE` / `TRANSFORMERS_OFFLINE` | — | Set to `1` in production once weights are cached |
|
||||
| `OMP_NUM_THREADS` / `MKL_NUM_THREADS` | — | Cap CPU threads (e.g. `4`) |
|
||||
|
||||
## 7. Run the service
|
||||
|
||||
```bash
|
||||
python -m uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
|
||||
```
|
||||
|
||||
- **Keep `--workers 1`.** Each worker loads its own full copy of the models; more
|
||||
workers multiply memory (an 8 GB box will OOM). The service already serializes
|
||||
inference with a global lock.
|
||||
- `--host 127.0.0.1` exposes it to the local host only (intended for a co-located
|
||||
caller). See §10 for remote access.
|
||||
|
||||
## 8. Run as a systemd service
|
||||
|
||||
`/etc/systemd/system/genarrative-bgfilter.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Genarrative BgFilter background removal service
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/genarrative-image-host/bgfilter-service
|
||||
Environment=HF_HOME=/opt/genarrative-image-host/bgfilter-service/model-cache/hf
|
||||
Environment=HF_HUB_OFFLINE=1
|
||||
Environment=TRANSFORMERS_OFFLINE=1
|
||||
Environment=HF_HUB_DISABLE_XET=1
|
||||
Environment=BGFILTER_CONFIG=configs/default.yaml
|
||||
Environment=BGFILTER_DEVICE=cpu
|
||||
Environment=BGFILTER_MAX_IMAGE_PIXELS=4194304
|
||||
Environment=BGFILTER_PRELOAD=1
|
||||
Environment=OMP_NUM_THREADS=4
|
||||
Environment=MKL_NUM_THREADS=4
|
||||
ExecStart=/opt/genarrative-image-host/bgfilter-service/.venv/bin/uvicorn app:app --host 127.0.0.1 --port 18083 --workers 1
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
MemoryMax=8G
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ReadWritePaths=/opt/genarrative-image-host
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
If you use Option B (local `models/` folders) instead of the HF cache, drop the
|
||||
`HF_HOME`/offline lines — those weights need no cache or network.
|
||||
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now genarrative-bgfilter
|
||||
sudo systemctl status genarrative-bgfilter
|
||||
journalctl -u genarrative-bgfilter -f # wait for "Application startup complete"
|
||||
```
|
||||
|
||||
## 9. Verify
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:18083/healthz
|
||||
# {"ok":true,"service":"bgfilter","version":"0.1.0","defaultSegModel":"birefnet","device":"cpu"}
|
||||
|
||||
curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
|
||||
http://127.0.0.1:18083/remove-background -o out.png
|
||||
```
|
||||
|
||||
## 10. Exposing the service
|
||||
|
||||
The unit binds `127.0.0.1` — intended to be called by a co-located client (e.g. the
|
||||
Genarrative Rust BFF via `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_BASE_URL`). The
|
||||
service has **no authentication**. To reach it from other hosts:
|
||||
|
||||
- **Reverse proxy (recommended):** put nginx/caddy in front, terminate TLS, add
|
||||
auth/rate-limiting, and forward to `127.0.0.1:18083`.
|
||||
- **Direct:** change `--host` to `0.0.0.0` and restrict source IPs with a firewall.
|
||||
Do not expose it unauthenticated to the public internet.
|
||||
|
||||
## 11. Capacity & performance
|
||||
|
||||
- CPU throughput is ~**45 s per 1.5 MP image** with the default (base ViTMatte +
|
||||
BiRefNet); requests are serialized by the global lock, so ~1–2 images/min per
|
||||
instance. Bottleneck is full-resolution ViTMatte + pymatting foreground estimation.
|
||||
- For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) —
|
||||
roughly an order of magnitude faster — and/or run multiple instances behind a load
|
||||
balancer (each with `--workers 1`).
|
||||
- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G`
|
||||
is a safe ceiling for a single CPU worker.
|
||||
|
||||
## 12. Troubleshooting
|
||||
|
||||
| Symptom | Cause / fix |
|
||||
| --- | --- |
|
||||
| `pip` pulls `nvidia-*`/`cuda-*` (huge) | You skipped the CPU index in §4. Install torch from `--index-url https://download.pytorch.org/whl/cpu`. |
|
||||
| `ImportError: libGL.so.1` on `import cv2` | Using `opencv-python` on a headless box. Switch to `opencv-python-headless` (§4). |
|
||||
| `ensurepip is not available` on venv create | Install `python3-venv` (§1). |
|
||||
| `FileMetadataError: ... not on huggingface.co` while downloading | hf-mirror + a proxy conflict. Bypass the proxy (`NO_PROXY='*'`) and set `HF_HUB_DISABLE_XET=1` (§5). |
|
||||
| `LocalEntryNotFoundError`, 0 bytes | Xet download path failing; set `HF_HUB_DISABLE_XET=1`. |
|
||||
| `503 service not ready` | Models still loading at startup; wait for "Application startup complete". |
|
||||
| OOM / killed under load | More than one worker, or `MemoryMax` too low. Keep `--workers 1`. |
|
||||
| `400 could not auto-detect background colour` | Input has no clean flat border; pass `screen_color=#RRGGBB`. |
|
||||
@@ -131,6 +131,8 @@ metadata.json
|
||||
|
||||
An HTTP wrapper (`app.py` + `bgfilter/service.py`) exposes the pipeline as a
|
||||
long-running FastAPI service. Models load once and are reused across requests.
|
||||
For deploying to a Linux server (dependencies, weights, systemd, offline mode), see
|
||||
[DEPLOY.md](DEPLOY.md).
|
||||
|
||||
```powershell
|
||||
D:\MiniConda\envs\lightML\python.exe -m uvicorn app:app `
|
||||
|
||||
Reference in New Issue
Block a user