Return freed activations to the OS on Linux; recommend jemalloc (A+B)

MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.

A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
   runtime (no-op on Windows/musl/other allocators). Wired into service.process
   (after each request) and the CLI batch loop (after each image), the two
   long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
   alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
   improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
   MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
   malloc_trim simply no-ops.

Also fixes the stale off-by-default cross-check heading in DEPLOY.md.

Verified: modules import; release_freed_memory() returns False (no-op) on Windows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 16:40:58 +08:00
parent ad445611ad
commit 3992f65ebf
6 changed files with 85 additions and 2 deletions
+12
View File
@@ -272,6 +272,18 @@ curl -sS -F "file=@input.png" -F "screen_color=#CFEFFF" \
和/或多实例 + 负载均衡(每个都 `--workers 1`)。
- 内存:单个 worker 持有全部三个模型(约 1 GB 权重 + 运行时)。单 CPU worker 用
`MemoryMax=8G` 作上限比较稳妥。
- **让 RSS 保持平稳(glibc)。** 每处理完一张图,服务会调用 glibc 的 `malloc_trim(0)`
(`bgfilter/memtune.py`)把 BiRefNet@2048 前向释放的大块激活还给 OS——否则 ptmalloc
会把它们留在 arena 里,RSS 会随请求逐步抬高。想进一步降内存**并**提升 CPU 吞吐,可以
预载 **jemalloc**(PyTorch 对 CPU 推理的推荐),并让它激进归还脏页:
```ini
# 放进 systemd unit 的 [Service] 段
Environment=LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
Environment=MALLOC_CONF=background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0
```
(`apt install libjemalloc2`;tcmalloc 用 `libtcmalloc_minimal.so.4` 同理。)
jemalloc/tcmalloc 自己管理归还,此时 `malloc_trim` 会自动空转。若无法预载分配器,
在 unit 环境里设 `MALLOC_ARENA_MAX=2`、`MALLOC_TRIM_THRESHOLD_=0` 也能抑制 ptmalloc 滞留。
## 12. 故障排查