Sync docs to current architecture

README rewritten for the current design: two pipelines (chroma-only vs single
segmenter, default anime-seg), the screen_color prior (--screen-color / config,
green default), correct debug-file list, and HF/hf-mirror notes for the seg models.
No rescue/recolour layer is documented (it was removed).

green_screen_matting_workflow.md keeps the phase-1 green-screen spec but gains a
status note pointing to the README for the evolved architecture, and its debug-file
list is corrected (foreground_rgb / color_mask / seg_mask, not the old
foreground_background / foreground_correction / despill_mask).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-01 15:46:21 +08:00
parent 7555321cfc
commit 22ef1d6336
2 changed files with 72 additions and 37 deletions
+58 -33
View File
@@ -1,14 +1,44 @@
# BgFilter
Offline green screen character matting for AI-generated character images.
The first implementation follows the workflow in
[`docs/green_screen_matting_workflow.md`](docs/green_screen_matting_workflow.md):
Offline character matting for AI-generated images on a flat-colour background.
Green screen is the default; any flat colour (e.g. a pastel `#CFEFFF`) works via a
`screen_color` prior.
```text
RGB input -> chroma confidence -> trimap -> ViTMatte -> alpha cleanup -> despill -> RGBA PNG -> QA previews
RGB input
-> chroma bg-confidence (keyed to the screen colour)
-> [optional] semantic segmentation mask
-> trimap -> ViTMatte -> alpha cleanup
-> pymatting foreground -> despill -> RGBA PNG -> QA previews
```
## Pipelines
Two pipelines, selected by `segmentation.enabled` in the config:
- **Chroma-only** (`enabled: false`) — Chroma + ViTMatte, no segmentation model.
Lightest / fastest; leans entirely on the colour key for topology.
- **Single segmenter** (`enabled: true`, default) — one segmentation model drives the
trimap topology (holes, hair), ViTMatte then refines the soft edges. Backend is
switchable: `anime-seg` (default, ONNX) or `birefnet` (needs `trust_remote_code`).
Both share pymatting foreground estimation and a colour de-spill. There is **no**
green-contamination rescue / recolour layer — with clean source images it is
unnecessary, so it was removed.
## Background colour
Default is green auto-detection. For any other flat background, pass the colour as a
hex prior so both the chroma key and the de-spill work off it:
```powershell
... --screen-color "#CFEFFF"
```
or set `screen_color: "#CFEFFF"` in the config (`null` = green auto-detect). The
chroma key then scores pixels by perceptual (Lab/RGB) distance to that colour, and
de-spill removes chroma along that colour's direction instead of the green channel.
## Environment
Use the conda environment `lightML`.
@@ -32,23 +62,13 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--output Outputs\TestImage_rgba.png `
--debug-dir Outputs\TestImage_debug `
--config configs\default.yaml `
--screen-color "#CFEFFF" `
--device cuda
```
The CLI reads `configs/default.yaml` when `--config` is provided. Command-line
options override config values, so tuning can usually happen in YAML while
runtime choices such as `--device cpu` stay on the command line.
Use CPU for validation when CUDA is unavailable:
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--input Samples\TestImage.png `
--output Outputs\TestImage_rgba.png `
--debug-dir Outputs\TestImage_debug `
--config configs\default.yaml `
--device cpu
```
options override config values, so tuning usually happens in YAML while runtime
choices such as `--device cpu` or `--screen-color` stay on the command line.
## Batch
@@ -61,10 +81,11 @@ D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
--device cuda
```
## Chroma-Only Debug Mode
## Chroma-alpha debug mode
This mode skips ViTMatte and uses chroma confidence as an alpha seed. It is useful
for fast debugging of chroma confidence, trimap, despill, and QA outputs.
`--matting-method chroma` skips ViTMatte and uses chroma confidence directly as the
alpha seed. Useful for fast inspection of chroma confidence, trimap, and despill.
(Distinct from the chroma-only *pipeline* above, which still runs ViTMatte.)
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.cli `
@@ -83,11 +104,10 @@ For each processed image, the CLI writes an RGBA PNG and optional debug files:
```text
bg_confidence.png
trimap.png
seg_mask.png # segmentation pipeline only
alpha.png
foreground_rgb.png
foreground_background.png
foreground_correction.png
despill_mask.png
foreground_rgb.png # despilled foreground colour
color_mask.png # per-pixel despill weight
preview_black.png
preview_white.png
preview_gray.png
@@ -99,8 +119,8 @@ metadata.json
## Quality Check
The quality checker measures alpha validity and green spill on semi-transparent
edge pixels.
The quality checker measures alpha validity and edge spill on semi-transparent edge
pixels.
```powershell
D:\MiniConda\envs\lightML\python.exe -m bgfilter.quality_cli `
@@ -120,11 +140,16 @@ D:\MiniConda\envs\lightML\python.exe scripts\smoke_samples.py `
--max-edge-green-excess-p95 0.30
```
## Current Notes
## Notes
- `Samples/` and `Outputs/` are ignored by Git.
- `ViTMatte` model weights are loaded from Hugging Face on first use.
- Foreground color estimation uses pymatting's `estimate_foreground_ml` to
propagate clean foreground colour into semi-transparent edges before final
de-spill, writing the estimated foreground, background, and a correction map.
Set `foreground.method: unmix` to fall back to the legacy heuristic.
- ViTMatte and segmentation weights load from Hugging Face on first use. Behind a
firewall set `HF_ENDPOINT=https://hf-mirror.com` (and bypass a flaky local proxy).
`anime-seg` (`skytnt/anime-seg`) is a plain ONNX download; `birefnet`
(`ZhengPeng7/BiRefNet`) ships custom modelling code so it needs
`trust_remote_code=True` plus `timm` / `einops` / `kornia`.
- Foreground colour estimation uses pymatting's `estimate_foreground_ml` to propagate
clean foreground colour into semi-transparent edges before de-spill. Set
`foreground.method: unmix` to fall back to the legacy heuristic.
- `docs/green_screen_matting_workflow.md` is the original phase-1 green-screen spec;
this README reflects the current, generalised architecture.
+14 -4
View File
@@ -4,6 +4,16 @@
本方案不使用 YOLO / Faster R-CNN 作为主流程。原因是本任务不是目标检测,而是像素级 alpha matting。
## 0. 现状说明(架构已演进)
本文档是**第一阶段的绿幕方案原始规格**,仍准确描述 “Chroma + ViTMatte” 这条管线。实现此后已泛化,**当前架构以 [README](../README.md) 为准**,主要变化:
- **两条管线**(`segmentation.enabled` 切换):无分割模型的 Chroma + ViTMatte;或单个语义分割模型(默认 anime-seg,可切 birefnet)定 trimap 拓扑,ViTMatte 细化。
- **背景色泛化**:`screen_color` 先验(如 `#CFEFFF`),chroma / despill 不再写死绿色;`null` 时才走本文档的绿幕自动探测。
- **已移除**:染绿修复(recolor)、发丝补救(dropout-fill)、BiRefNet+anime-seg 结合方案——它们只是给“生图即染色/断裂”的坏素材擦屁股;源头改用干净背景生图后不再需要。
下文其余章节描述绿幕基线流程,仍然有效。
## 1. 目标
- 输入:RGB / PNG 图像,背景主色为绿色,但不要求严格等于 `#00FF00`。
@@ -371,17 +381,17 @@ edge = alpha 梯度区域 + 外扩 1 - 3 px
result_rgba.png
debug/bg_confidence.png
debug/trimap.png
debug/seg_mask.png # 分割管线才有
debug/alpha.png
debug/foreground_rgb.png
debug/foreground_background.png
debug/foreground_correction.png
debug/despill_mask.png
debug/foreground_rgb.png # 去溢后的前景色
debug/color_mask.png # 每像素去溢权重
debug/preview_black.png
debug/preview_white.png
debug/preview_gray.png
debug/preview_red.png
debug/preview_blue.png
debug/qa_grid.png
debug/metadata.json
```
## 16. QA 质检