Return freed activations to the OS on Linux; recommend jemalloc (A+B)
MIMALLOC_PURGE_DELAY=0 only bites on Windows (bundled mimalloc). On a Linux server PyTorch uses glibc ptmalloc, which keeps a BiRefNet@2048 forward's freed activations in the arena, so RSS ratchets up across requests and the env var is a no-op there. A. bgfilter/memtune.py: release_freed_memory() calls glibc malloc_trim(0) at runtime (no-op on Windows/musl/other allocators). Wired into service.process (after each request) and the CLI batch loop (after each image), the two long-lived paths where RSS accumulates. Single-image CLI exits, so it is left alone. B. DEPLOY(_ZH): document preloading jemalloc via LD_PRELOAD + MALLOC_CONF (also improves CPU throughput) as the production alternative, with MALLOC_ARENA_MAX/ MALLOC_TRIM_THRESHOLD_ as an allocator-free fallback. Under jemalloc/tcmalloc malloc_trim simply no-ops. Also fixes the stale off-by-default cross-check heading in DEPLOY.md. Verified: modules import; release_freed_memory() returns False (no-op) on Windows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+7
-1
@@ -7,6 +7,7 @@ from dataclasses import replace
|
||||
import numpy as np
|
||||
|
||||
from .config import _SEG_MODELS, override_settings
|
||||
from .memtune import release_freed_memory
|
||||
from .pipeline import MattingPipeline, MattingResult
|
||||
from .segmentation import BiRefNetSegmenter, make_segmenter
|
||||
from .settings import PipelineSettings, SegmentationSettings
|
||||
@@ -151,4 +152,9 @@ class PipelineManager:
|
||||
pipeline = MattingPipeline(
|
||||
settings, runner=runner, segmenter=segmenter, cross_checker=cross_checker
|
||||
)
|
||||
return pipeline.run_rgb(rgb)
|
||||
try:
|
||||
return pipeline.run_rgb(rgb)
|
||||
finally:
|
||||
# Return this request's large freed activations to the OS so RSS does
|
||||
# not ratchet up across requests in the long-lived service process.
|
||||
release_freed_memory()
|
||||
|
||||
Reference in New Issue
Block a user