Benchmarking c7a2819 on CPU showed unconditional fp16 is pathological
there: ~1870 s/image vs ~14.4 s fp32 (~130x) -- x86 has no native fp16
conv/gemm kernels so PyTorch falls back to a slow reference path. fp16
only pays off on CUDA, so ViTMatteRunner and BiRefNetSegmenter now pick
float16 iff the resolved device is cuda, float32 otherwise; inputs cast
to the same dtype. The dtype is intentionally not a setting.
Also replaces the deprecated torch_dtype= kwarg with dtype=.
Verified on CPU: single-image CLI run back to normal speed (~31 s wall
including model load) and output byte-identical (SHA256) to the
fp32-era baseline, confirming inference_mode alone changes nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lhk229
291e9faac5