Return freed activations to the OS on Linux; recommend jemalloc (A+B)
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed activations in the arena, so RSS ratchets up across requests and the env var is a no-op there. A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at runtime (no-op on Windows/musl/other allocators). Wired into service.process (after each request) and the CLI batch loop (after each image), the two long-lived paths where RSS accumulates. Single-image CLI exits, so it is left alone. B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/ MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc malloc_trim simply no-ops. Also fixes the stale off-by-default cross-check heading in DEPLOY.md. Verified: modules import; release_freed_memory() returns False (no-op) on Windows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -272,6 +272,18 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
|
||||
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
|
||||
- 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用
|
||||
`MemoryMax=8G` 作上限比较稳妥。
|
||||
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`
|
||||
(`bgfilter/memtune.py`)把 BiRefNet@2048 前向释放的大块激活还给 OS——否则 ptmalloc
|
||||
会把它们留在 arena 里,RSS 会随请求逐步抬高。想进一步降内存**并**提升 CPU 吞吐,可以
|
||||
预载 **jemalloc**(PyTorch 对 CPU 推理的推荐),并让它激进归还脏页:
|
||||
```ini
|
||||
# 放进 systemd unit 的 [Service] 段
|
||||
Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
|
||||
Environment=MALLOC_CONF=background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0
|
||||
```
|
||||
(`apt install libjemalloc2`;tcmalloc 用 `libtcmalloc_minimal.so.4` 同理。)
|
||||
jemalloc/tcmalloc 自己管理归还,此时 `malloc_trim` 会自动空转。若无法预载分配器,
|
||||
在 unit 环境里设 `MALLOC_ARENA_MAX=2`、`MALLOC_TRIM_THRESHOLD_=0` 也能抑制 ptmalloc 滞留。
|
||||
|
||||
## 12. 故障排查
|
||||
|
||||
|
||||
Reference in New Issue
Block a user