Document the measured GPU deployment profile (bf16 + expandable_segments)
Measured on an RTX 5070 Ti 16 GB, full default pipeline (cross-check on, reused as seg, bf16): ~1-2 s/image, ~10.1 GB allocated. PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (Linux/WSL2 only; silently unsupported on native Windows) cuts the allocator's reserved footprint 15.3 -> 12.0 GB and is ~10% faster, leaving ~3.5 GB headroom on a 16 GB card even beside a desktop session. bf16 is the precondition: fp32 reserved ~17 GB overflows. reuse-as-seg also saves ~0.9 GB VRAM / ~0.25 s per image on GPU (reuse off measured 13.3 GB reserved under expandable_segments). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -289,6 +289,19 @@ service has **no authentication**. To reach it from other hosts:
|
||||
- For throughput: use a GPU (`BGFILTER_DEVICE=cuda`, install a CUDA torch build) —
|
||||
roughly an order of magnitude faster — and/or run multiple instances behind a load
|
||||
balancer (each with `--workers 1`).
|
||||
- **GPU deployment (measured, RTX 5070 Ti 16 GB, full default pipeline —
|
||||
cross-check on, reused as seg, bf16):** ~1–2 s/image (2 s at 4 MP), peak VRAM
|
||||
~10.1 GB allocated. The number that must fit on the card is the allocator's
|
||||
*reserved* footprint; set this in the service environment on Linux:
|
||||
```ini
|
||||
Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
||||
```
|
||||
Measured effect: reserved 15.3 GB → **12.0 GB** (fragmentation 5.2 → 1.9 GB) and
|
||||
~10 % *faster* — with it, the full pipeline fits a 16 GB card with ~3.5 GB
|
||||
headroom even next to a desktop session. Without it (or on native Windows, where
|
||||
the option is silently unsupported), reserved is ~15.3 GB: a headless 16 GB card
|
||||
still fits, but a desktop-attached one is borderline. Precision `bf16` is what
|
||||
makes 16 GB viable at all — fp32 reserved ~17 GB overflows.
|
||||
- Memory: one worker holds all three models (~1 GB weights + runtime). `MemoryMax=8G`
|
||||
is a safe ceiling for a single CPU worker.
|
||||
- **Keeping RSS flat (glibc).** After each image the service calls glibc
|
||||
|
||||
@@ -270,6 +270,17 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
|
||||
串行,所以单实例约 1–2 张/分钟。瓶颈是全分辨率的 ViTMatte 和 pymatting 前景估计。
|
||||
- 要提吞吐:上 GPU(`BGFILTER_DEVICE=cuda`,装 CUDA 版 torch),大约快一个数量级;
|
||||
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
|
||||
- **GPU 部署(实测,RTX 5070 Ti 16 GB,完整默认管线——cross-check 开 + reuse-as-seg,
|
||||
bf16):** 约 1–2 秒/张(4 MP 也只要 2 秒),峰值 VRAM 分配约 10.1 GB。真正决定
|
||||
「装不装得下」的是分配器的 *reserved* 占用;Linux 上在服务环境里加:
|
||||
```ini
|
||||
Environment=PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
||||
```
|
||||
实测效果:reserved 15.3 GB → **12.0 GB**(碎片 5.2 → 1.9 GB),而且还**快约 10%**——
|
||||
加上它之后,完整管线在 16 GB 卡上即使旁边挂着桌面会话也有约 3.5 GB 余量。不加
|
||||
(或在原生 Windows 上——该选项会被静默忽略)reserved 约 15.3 GB:无头 16 GB 卡勉强
|
||||
能装,带桌面输出的卡就很悬。另外 `bf16` 是 16 GB 可行的前提——fp32 的 reserved
|
||||
约 17 GB,直接溢出。
|
||||
- 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用
|
||||
`MemoryMax=8G` 作上限比较稳妥。
|
||||
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`
|
||||
|
||||
Reference in New Issue
Block a user