Cut pipeline cost: seg-reuse, unified bf16 knob, eager mimalloc purge
Three optimizations from profiling the cross-check-dominated pipeline (9700X CPU, all pilot-validated on TestImage3/FixImage1): - Reuse the cross-check HR-matting@2048 forward as the segmentation mask (cross_check.reuse_as_seg, default ON; --no-cross-check-as-seg to opt out). Skips the BiRefNet@1024 load+forward entirely: ~66s -> ~45s, one less 0.9GB model. Trimap 99.8% identical, no structural change. - --precision bf16 now fans out to all three models: ViTMatte keeps its weight cast; both BiRefNets run their forward under autocast with a dispatcher-level AutocastCPU fp32 shim for torchvision::deform_conv2d (no bf16 CPU kernel, no autocast wrapper upstream). Shared hardware gate in bgfilter/precision.py falls back to fp32 off native-bf16 hardware. TestImage3: 51.9s -> 33.1s; alpha diff max 0.15, none >0.25. - MIMALLOC_PURGE_DELAY=0 (bgfilter/__init__.py, before torch loads): Windows torch's bundled mimalloc lazily retains ~10GB of freed BiRefNet activations, stacking under ViTMatte's attention peak. 2048x2048 bf16: peak 25.2 -> 21.4GB and slightly faster (73 -> 63s). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -105,9 +105,16 @@ def override_settings(settings: PipelineSettings, **overrides: Any) -> PipelineS
|
||||
cross_check_updates: dict[str, Any] = {}
|
||||
if overrides.get("cross_check") is not None:
|
||||
cross_check_updates["enabled"] = overrides["cross_check"]
|
||||
if overrides.get("cross_check_as_seg") is not None:
|
||||
cross_check_updates["reuse_as_seg"] = overrides["cross_check_as_seg"]
|
||||
seg_updates: dict[str, Any] = {}
|
||||
if overrides.get("device") is not None:
|
||||
seg_updates["device"] = overrides["device"]
|
||||
# --precision is a single knob for all three models (ViTMatte handled via
|
||||
# model_updates above; per-model values remain settable in the config file).
|
||||
if overrides.get("precision") is not None:
|
||||
seg_updates["precision"] = overrides["precision"]
|
||||
cross_check_updates["precision"] = overrides["precision"]
|
||||
if overrides.get("seg_backend") is not None:
|
||||
backend = overrides["seg_backend"]
|
||||
if backend not in _SEG_MODELS:
|
||||
|
||||
Reference in New Issue
Block a user