Document the measured GPU deployment profile (bf16 + expandable_segments)

Measured on an RTX 5070 Ti 16 GB, full default pipeline (cross-check on, reused as seg, bf16): ~1-2 s/image, ~10.1 GB allocated. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (Linux/WSL2 only; silently unsupported on native Windows) cuts the allocator's reserved footprint 15.3 -> 12.0 GB and is ~10% faster, leaving ~3.5 GB headroom on a 16 GB card even beside a desktop session. bf16 is the precondition: fp32 reserved ~17 GB overflows. reuse-as-seg also saves ~0.9 GB VRAM / ~0.25 s per image on GPU (reuse off measured 13.3 GB reserved under expandable_segments).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-07 20:32:13 +08:00
parent a48d70d9f7
commit 6bf648cd64
2 changed files with 24 additions and 0 deletions
+13
View File
@@ -289,6 +289,19 @@ service has **no authentication**. To reach it from other hosts:
- For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) —
roughly an order of magnitude faster — and/or run multiple instances behind a load
balancer (each with `--workers 1`).
- **GPU deployment (measured, RTX 5070 Ti 16 GB, full default pipeline —
cross-check on, reused as seg, bf16):** ~1–2 s/image (2 s at 4 MP), peak VRAM
~10.1 GB allocated. The number that must fit on the card is the allocator's
*reserved* footprint; set this in the service environment on Linux:
```ini
Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
```
Measured effect: reserved 15.3 GB → **12.0 GB** (fragmentation 5.2 → 1.9 GB) and
~10 % *faster* — with it, the full pipeline fits a 16 GB card with ~3.5 GB
headroom even next to a desktop session. Without it (or on native Windows, where
the option is silently unsupported), reserved is ~15.3 GB: a headless 16 GB card
still fits, but a desktop-attached one is borderline. Precision `bf16` is what
makes 16 GB viable at all — fp32 reserved ~17 GB overflows.
- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G`
is a safe ceiling for a single CPU worker.
- **Keeping RSS flat (glibc).** After each image the service calls glibc
+11
View File
@@ -270,6 +270,17 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
串行,所以单实例约 1–2 张/分钟。瓶颈是全分辨率的 ViTMatte 和 pymatting 前景估计。
- 要提吞吐:上 GPU(`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级;
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整默认管线——cross-check 开 + reuse-as-seg,
bf16):** 约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.1 GB。真正决定
「装不装得下」的是分配器的 *reserved* 占用;Linux 上在服务环境里加:
```ini
Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
```
实测效果:reserved 15.3 GB → **12.0 GB**(碎片 5.2 → 1.9 GB),而且还**快约 10%**——
加上它之后,完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 3.5 GB 余量。不加
(或在原生 Windows 上——该选项会被静默忽略)reserved 约 15.3 GB:无头 16 GB 卡勉强
能装,带桌面输出的卡就很悬。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved
约 17 GB,直接溢出。
- 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用
`MemoryMax=8G` 作上限比较稳妥。
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`