Return freed activations to the OS on Linux; recommend jemalloc (A+B)

MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux
server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed
activations in the arena, so RSS ratchets up across requests and the env var is
a no-op there.

A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at
   runtime (no-op on Windows/musl/other allocators). Wired into service.process
   (after each request) and the CLI batch loop (after each image), the two
   long-lived paths where RSS accumulates. Single-image CLI exits, so it is left
   alone.
B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also
   improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/
   MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc
   malloc_trim simply no-ops.

Also fixes the stale off-by-default cross-check heading in DEPLOY.md.

Verified: modules import; release_freed_memory() returns False (no-op) on Windows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-06 16:40:58 +08:00
parent ad445611ad
commit 3992f65ebf
6 changed files with 85 additions and 2 deletions
+7 -1
View File
@@ -7,6 +7,7 @@ from dataclasses import replace
import numpy as np
from .config import _SEG_MODELS, override_settings
from .memtune import release_freed_memory
from .pipeline import MattingPipeline, MattingResult
from .segmentation import BiRefNetSegmenter, make_segmenter
from .settings import PipelineSettings, SegmentationSettings
@@ -151,4 +152,9 @@ class PipelineManager:
pipeline = MattingPipeline(
settings, runner=runner, segmenter=segmenter, cross_checker=cross_checker
)
return pipeline.run_rgb(rgb)
try:
return pipeline.run_rgb(rgb)
finally:
# Return this request's large freed activations to the OS so RSS does
# not ratchet up across requests in the long-lived service process.
release_freed_memory()