diff --git a/DEPLOY.md b/DEPLOY.md index 194cac1..108047a 100644 --- a/DEPLOY.md +++ b/DEPLOY.md @@ -289,6 +289,19 @@ service has **no authentication**. To reach it from other hosts: - For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) — roughly an order of magnitude faster — and/or run multiple instances behind a load balancer (each with `--workers 1`). +- **GPU deployment (measured, RTX 5070 Ti 16 GB, full default pipeline — + cross-check on, reused as seg, bf16):** ~1–2 s/image (2 s at 4 MP), peak VRAM + ~10.1 GB allocated. The number that must fit on the card is the allocator's + *reserved* footprint; set this in the service environment on Linux: + ```ini + Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True + ``` + Measured effect: reserved 15.3 GB → **12.0 GB** (fragmentation 5.2 → 1.9 GB) and + ~10 % *faster* — with it, the full pipeline fits a 16 GB card with ~3.5 GB + headroom even next to a desktop session. Without it (or on native Windows, where + the option is silently unsupported), reserved is ~15.3 GB: a headless 16 GB card + still fits, but a desktop-attached one is borderline. Precision `bf16` is what + makes 16 GB viable at all — fp32 reserved ~17 GB overflows. - Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G` is a safe ceiling for a single CPU worker. - **Keeping RSS flat (glibc).** After each image the service calls glibc diff --git a/DEPLOY_ZH.md b/DEPLOY_ZH.md index 7cae7d4..077fe0a 100644 --- a/DEPLOY_ZH.md +++ b/DEPLOY_ZH.md @@ -270,6 +270,17 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \ 串行,所以单实例约 1–2 张/分钟。瓶颈是全分辨率的 ViTMatte 和 pymatting 前景估计。 - 要提吞吐:上 GPU(`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级; 和/或多实例 + 负载均衡(每个都 `--workers 1`)。 +- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整默认管线——cross-check 开 + reuse-as-seg, + bf16):** 约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.1 GB。真正决定 + 「装不装得下」的是分配器的 *reserved* 占用;Linux 上在服务环境里加: + ```ini + Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True + ``` + 实测效果:reserved 15.3 GB → **12.0 GB**(碎片 5.2 → 1.9 GB),而且还**快约 10%**—— + 加上它之后,完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 3.5 GB 余量。不加 + (或在原生 Windows 上——该选项会被静默忽略)reserved 约 15.3 GB:无头 16 GB 卡勉强 + 能装,带桌面输出的卡就很悬。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved + 约 17 GB,直接溢出。 - 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用 `MemoryMax=8G` 作上限比较稳妥。 - **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`