BGfilter

Files

T

lhk229 2cc0ce8684 Chunk ViTMatte's global attention to cap its N^2 memory spike

ViTMatte's VitDet backbone runs 4 global attention blocks that materialize
the full [heads x N x N] map: ~19 GB transient at 2048x2048 (16384 tokens),
the pipeline's memory peak. transformers has no SDPA path for this
architecture (the decomposed rel-pos bias is added to raw scores), so
compute the same attention in query-row chunks instead: the bias
factorizes over query rows, making the chunked form mathematically exact
-- output verified BITWISE-identical (unit: fp32/bf16 x 3 sizes; full
pipeline: TestImage3 fp32 and 2048x2048 bf16, all byte-equal).

model.attn_query_chunk (default 2048, 0 = stock one-shot) engages only on
blocks seeing more tokens than the chunk size, so window blocks keep the
original path. Measured @2048x2048 bf16 (9700X): ViTMatte spike
19.8 -> 4.0 GB for ~15% more ViTMatte time; whole pipeline peak
21.4 -> 11.4 GB, warm 51 -> 53 s.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-06 15:26:59 +08:00

__init__.py

Cut pipeline cost: seg-reuse, unified bf16 knob, eager mimalloc purge

2026-07-06 15:15:12 +08:00

alpha_post.py

Add cross-model veto of hair-gap background residue (default on)

2026-07-04 19:52:24 +08:00

attn_chunk.py

Chunk ViTMatte's global attention to cap its N^2 memory spike

2026-07-06 15:26:59 +08:00

chroma.py

Add cross-model veto of hair-gap background residue (default on)