接通第四步
This commit is contained in:
Ukawka
@@ -20,11 +20,11 @@
|
||||
|
||||
## 跨会话上下文与阅读顺序
|
||||
|
||||
本节于 2026-09-21 根据当前代码与已检查产物整理。它是后续对话的上下文入口；后续修改应同步更新对应状态，实际源码、配置和具体实验 manifest 优先于历史总结。不要把旧实验统计套到新输出上。
|
||||
本节于 2026-09-22 接入通用第四阶段后更新。它是后续对话的上下文入口；后续修改应同步更新对应状态，实际源码、配置和具体实验 manifest 优先于历史总结。不要把旧实验统计套到新输出上。
|
||||
|
||||
1. 先阅读本文件和 `git status --short`，保留用户已有修改。
|
||||
2. 阅读 [README](README.md) 了解项目、环境、配置与命令。
|
||||
3. 按任务阅读 [第一阶段](docs/1-understanding-stage.md)、[第二阶段](docs/2-segmentation-stage.md)、[第三阶段](docs/3-completion-stage.md)，再核对相关实现与实际产物。
|
||||
3. 按任务阅读 [第一阶段](docs/1-understanding-stage.md)、[第二阶段](docs/2-segmentation-stage.md)、[第三阶段](docs/3-completion-stage.md)、[第四阶段](docs/4-spine-stage.md)，再核对相关实现与实际产物。
|
||||
4. 用户说“继续”时，先核对工作区和最近任务进展，不能只凭历史总结重复请求付费模型或覆盖实验。
|
||||
|
||||
## 用户目标与固定偏好
|
||||
@@ -37,8 +37,8 @@
|
||||
|
||||
## 文档维护约定
|
||||
|
||||
- 各文档文件名保持不变。`docs/` 当前保留三份阶段文档；`docs/model-stages.md` 在本次文档整理前已处于用户删除状态，不自行恢复，相关说明归入 README 和阶段文档。
|
||||
- 三份阶段文档有且仅有以下八个二级章节，细节必须归入对应章节：
|
||||
- 原有文档文件名保持不变。`docs/` 保留前三份阶段文档，并新增 `4-spine-stage.md`；`docs/model-stages.md` 在此前文档整理前已处于用户删除状态，不自行恢复，相关说明归入 README 和阶段文档。
|
||||
- 四份阶段文档有且仅有以下八个二级章节，细节必须归入对应章节：
|
||||
1. 阶段简介
|
||||
2. 输入、输出与执行链路
|
||||
3. 阶段执行命令示例及参数说明
|
||||
@@ -65,7 +65,7 @@
|
||||
- 用户提供的密钥仅留在本地忽略的 `.env`；禁止在文档、命令日志、聊天或提交中暴露，不要读取/打印完整 .env 来诊断。
|
||||
- 三份配置的图片编辑能力均为 multiple_images=false、mask=false。接口已预留分支不代表中转已支持；开启前需验证。Astra 理解接口与图片编辑接口不同，当前没有 Astra Responses 图片生成工具链。
|
||||
|
||||
## 三阶段实现与不能丢失的边界
|
||||
## 四阶段实现与不能丢失的边界
|
||||
|
||||
### 第一阶段：理解与几何候选
|
||||
|
||||
@@ -97,9 +97,21 @@
|
||||
- 验收阈值：可见不透明像素 RGB MAE≤30、保留率≥95%；有效新增须在允许区且连接可见部件；原可见 RGBA 随后逐像素恢复。
|
||||
- unfulfilled_regions 当前只记录、不硬拒绝；语义错误、接缝、关节补够与否尚无可靠验收。错误 SAM 可见像素会被强制保留。
|
||||
- prepare-only 不联网；candidates-dir 可读平铺 ID.png 或匹配 manifest 的旧任务首张候选。--resume 不是失败断点续发，准备/生成/复检使用独立目录。
|
||||
- parts/ 只含通过检查的补全部件，不包含全部原拆件。完整角色组装、旋转验收、新管线 Spine 导出尚未接入。
|
||||
- parts/ 只含通过检查的补全部件，不包含全部原拆件。完整角色组装、Spine 导出与旋转检查动画已独立接入第四阶段；运动质量自动验收仍未实现。
|
||||
- 旧 `--stage completion --image ... --prompt-file ...` 是接口探针，不能和 input-dir 混用，不执行完整几何验收。
|
||||
|
||||
### 第四阶段：通用组装与 Spine 导出
|
||||
|
||||
- 用户明确要求可复用管线，不能写死具体角色、部件 ID、数量或人形比例。入口 `--stage spine --input-dir SAM目录 --completion-dir 补全目录 --out-dir 新目录`；completion-dir 可省略。
|
||||
- `stages/spine-stage.mjs` 与 `spine_assembly.py` 完全离线，校验 v2、输入快照及 masks 哈希、PNG 和裁剪坐标；不读取模型配置/密钥，不调用 SAM。
|
||||
- 原资产从 source + mask 重建，采用补全结果前检查 selected_file、状态、qa.accepted、image、crop_xyxy 以及原可见 RGBA 保持。不能混用其他实验。
|
||||
- 默认补全失败回退 SAM 并记录原因；fallback-policy=error 拒绝需补全但无通过结果的可见部件。缺可见 mask 直接报错；隐藏骨骼保留但不生成图片。
|
||||
- pivot 转换为底边中点原点的 Spine 坐标；骨骼 setup 旋转为零，局部位置取父子 pivot 差；附件中心相对绑定骨骼偏移。bone_part 使用 parent，其他角色使用 bind_to。
|
||||
- occluded_by 稳定拓扑排序；无约束重叠写入诊断，可用 draw-order-file 全量覆盖，冲突留档。骨骼/遮挡环路拒绝。
|
||||
- 导出完整 images、assembly.json、静态组装/对比/覆盖诊断、skeleton.json、atlas、report.html 和真实 preview.html。依赖外部只读图集和播放器脚本，其哈希纳入复用指纹。
|
||||
- 目标 Spine 4.2；setup 静态姿态及 check__ID 单骨骼检查动画。没有成品 idle/walk、mesh/weights/IK，semantic_verified=false、quality_status=needs_review；几何通过和可加载不代表资产合格。
|
||||
- 生成离线；播放器从 CDN 加载资源，打开骨架预览需要网络。静态报告可离线查看。
|
||||
|
||||
## 已检查实验与质量结论
|
||||
|
||||
以下为已有实验记录，不代表后续每次运行的结果；不必为了文档修改重跑 GPU 或付费模型。
|
||||
@@ -108,30 +120,34 @@
|
||||
| --- | --- |
|
||||
| output/sombrero-grounding-v2 | 旧计划离线几何检查，28 个程序候选通过前景检查；未证明语义正确 |
|
||||
| output/sombrero-sam31-v3 | 较早 SAM 实验，28 个 mask，约 67.18% 覆盖率；肢体/鞋子重复、漏分，不合格 |
|
||||
| output/20260918-183532/sam | 当前讨论基线：32 个计划项、10 个 hidden、22 个 mask；约 83.3% 覆盖，32719 像素未分配；前臂/手混入衣料、袖子与上衣缺损 |
|
||||
| output/20260918-183532/sam | 较早基线：32 个计划项、10 个 hidden、22 个 mask；约 83.3% 覆盖，32719 像素未分配；前臂/手混入衣料、袖子与上衣缺损 |
|
||||
| output/20260918-183532/complete2_gpt | 真实 Tiantoken gpt-image-2 请求 12 部件；旧请求未传 size，返回 1248/1254，预期 1024，全部 CANVAS_SIZE_MISMATCH |
|
||||
| output/20260918-183532/complete2_gpt-size-recheck-v2 | 离线 uniform 复检，未重生成：8 个几何通过；head、pants_thigh_l、pants_thigh_r、shoe_l 4 个拒绝；通过项仍有误分和未完成区域 |
|
||||
| output/20260918-183532/complete-recovered-head-check | 更早 Qwen 头部候选恢复检查，不能与 GPT 批次混为一谈 |
|
||||
| output/20260920-225137 | 较新实验：27 个语义项、2 个 hidden、25 个 SAM 部件；补全 16 项中 13 个几何通过、3 个对齐拒绝 |
|
||||
| output/spine-validation-latest-20260922 | 第四阶段离线验证：25 个 slot、16 个骨骼（含 root）；采用 13 个补全部件，5 个需补全部件回退 SAM（含两袖无允许区域）；上游缺口仍存在 |
|
||||
| output/spine-validation-baseline-20260922 | 另一批输入验证：22 个 slot、16 个骨骼（含 root）；采用 8 个补全部件、4 个回退；输入源为旧基线和 size-recheck-v2 |
|
||||
|
||||
- 当前 22→12 的原因：sombrero、sleeve_l、sleeve_r、eye_l、eye_r、eyebrow_l、eyebrow_r、nose、mouth、mustache 共 10 个被第一阶段标记 NO_COMPLETION_NEEDED；不能解释为它们的实际 SAM mask 已完整。
|
||||
- 2026-09-20 尺寸修复后曾通过 22 项 Node 测试（含 Python 补全回归）。这是历史验证；新改动按范围重新验证，不自动沿用“全部通过”。显式 size 构造已测试，修复后中转是否遵守尚无新实测。
|
||||
- 总体判断：上游部件归属、模型缺少原图参考与遮罩、区域规则粗糙、验收不足共同导致低质量。不能仅换模型、强制处理全部 mask、放宽阈值或解决尺寸错误就宣称修好。
|
||||
- 2026-09-22 第四阶段接入后，Conda ai-spine 下 npm test 共 26 项通过（含新增 6 项 Python 组装用例）。另使用官方 Spine 4.2 core IIFE 对上述两份导出分别验证 setup 顶点和 30 个单关节旋转采样，全部通过。最新导出的 preview.html 已在 headless Chrome 实际加载渲染，截图为 output/spine-runtime-validation-20260922/preview.png；这不是动作或资产质量验收。复检工具为 `tools/verify-spine-runtime.mjs`，需调用者提供本地官方 runtime 文件，不自动下载/安装依赖。
|
||||
|
||||
## 当前改进方向与后续顺序
|
||||
|
||||
本轮用户要求重整文档，以下功能是待实现需求，不因写入文档就成为已完成功能：
|
||||
当前用户优先推进可复用第四阶段；以下上游质量改进仍待实现，不因导出链路完成而自动解决：
|
||||
|
||||
1. 第二阶段输出仅色彩遮罩组成的完整角色图，保留当前原图叠色预览，并明确未分配/重叠区域。
|
||||
2. 第二阶段每个 SAM 部件可选经过一次大模型去毛边，保留原始拆件、原图参考、坐标、变更区域与前后对照。
|
||||
3. 第三阶段将角色原图或部件相关原图局部作为参考交给补全模型，验证中转多图/遮罩能力及实际发送内容。
|
||||
4. 第三阶段输出使用补全部件组装的完整角色，复用无需补全的原部件，明确失败/回退状态。
|
||||
5. 技术优先级：先修正 SAM 部件归属与输入门槛，再按实际 mask 重建补全计划；用头、前臂、上衣三类小样验证参考图和遮罩，然后增加回贴与关节旋转验收，最后批量与 Spine 导出。
|
||||
4. 完整角色组装及失败回退记录已在第四阶段实现；接缝与关节旋转质量自动验收仍待实现。
|
||||
5. 后续质量工作：修正 SAM 归属和补全计划、验证参考图与遮罩，然后加强接缝/动作验收。第四阶段遵循统一输入契约，不为某个实验硬编码修正。
|
||||
|
||||
详细改进说明保留在各阶段文档第 8 节；若后续用户改变优先级，应更新此处。
|
||||
|
||||
## 旧管线与范围识别
|
||||
|
||||
- `pipeline.mjs` 未指定 --mode annotated-parts 时走旧“生成拆件图→CV 定位→Spine”流程，不是新三阶段自动串联。
|
||||
- `pipeline.mjs` 未指定 --mode annotated-parts 时走旧“生成拆件图→CV 定位→Spine”流程，不是新四阶段自动串联。
|
||||
- `segment.py` 为旧拆件图分割；`compose_preview.py` 为旧布局组装；`run_case_matrix.py` 为旧定位对照工具。不要因为 IDE 当前打开这些文件就误改新阶段入口。
|
||||
- 相邻 `spine-animation-ai` 为外部只读依赖，保留其 PolyForm Noncommercial 许可证约束；新阶段尚未复用为完成的导出流程。
|
||||
- 相邻 `spine-animation-ai` 为外部只读依赖，保留其 PolyForm Noncommercial 许可证约束；第四阶段复用图集和播放器脚本，不复用旧特征定位或 root 平铺骨架逻辑。
|
||||
- 不要把文档改写任务扩大成付费图片生成、SAM 推理、依赖安装或功能实现；需要实现时遵从用户后续任务范围。
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
# AI Spine 角色拆件与动画管线
|
||||
|
||||
本项目探索将单张 2D 角色图转换为可用于 Spine 动画的部件与骨骼资源。当前主线分为三个阶段：视觉 LLM 理解结构、本地 SAM 分割可见部件、图片编辑模型补全隐藏区域和关节连接。
|
||||
本项目探索将单张 2D 角色图转换为可用于 Spine 动画的部件与骨骼资源。当前主线分为四个阶段：视觉 LLM 理解结构、本地 SAM 分割可见部件、图片编辑模型补全隐藏区域和关节连接、离线组装与 Spine 导出。
|
||||
|
||||
| 阶段 | 输入 → 输出 | 当前状态 |
|
||||
| --- | --- | --- |
|
||||
| 1. 结构理解 | PNG → v2 语义计划、粗框、pivot、程序候选点 | 可在线理解或离线复检；语义待复核 |
|
||||
| 2. SAM 分割 | 第一阶段目录 → mask、原像素透明部件、标注图与拆解图 | 本机 GPU 实测可运行；存在误分与漏分 |
|
||||
| 3. 部件补全 | SAM 目录 → 局部编辑候选、几何 QA、通过检查的补全部件 | 已真实调用图片模型；结果质量尚不合格 |
|
||||
| 4. Spine 导出 | SAM + 可选补全目录 → 完整资产、原位组装、骨骼、图集、预览 | 通用离线入口；含单骨骼旋转检查动画，资产与动作质量需复核 |
|
||||
|
||||
第三阶段尚未输出使用补全部件组装的完整角色，新管线 Spine 导出也未完成。执行成功、mask 覆盖率高、几何检查通过，均不等于动画资产可用。
|
||||
第三阶段输出通过几何检查的补全部件；第四阶段整合所有可绘制部件、组装并导出 Spine 4.2。执行成功、mask 覆盖率高、几何检查通过，均不等于动画资产可用。
|
||||
|
||||
仓库保留旧版“生成白底拆件图 → OpenCV 拆分 → Genielabs 定位 → Spine 导出”作为对照；它与当前三阶段主线并存，产物不能直接混用。
|
||||
仓库保留旧版“生成白底拆件图 → OpenCV 拆分 → Genielabs 定位 → Spine 导出”作为对照；它与当前四阶段主线并存，产物不能直接混用。
|
||||
|
||||
## 文档入口
|
||||
|
||||
@@ -18,6 +19,7 @@
|
||||
- [第一阶段](docs/1-understanding-stage.md)：理解、数据契约、Alpha 候选与粗标注。
|
||||
- [第二阶段](docs/2-segmentation-stage.md)：本地 SAM、mask 选择、拆件与质量边界。
|
||||
- [第三阶段](docs/3-completion-stage.md)：筛选、局部画布、图片请求、几何验收及补全改进。
|
||||
- [第四阶段](docs/4-spine-stage.md)：通用资产整合、坐标与骨骼、绘制顺序、Spine 导出与回退策略。
|
||||
|
||||
阶段文档统一采用八节：简介；输入输出与链路；命令参数；输入格式；重要代码原理；职责索引；产物；可改进点。配置与实验说明已归入上述文档。
|
||||
|
||||
@@ -112,7 +114,7 @@ node .\tools\model-check.mjs --config .\configs\tiantoken.json --dry-run
|
||||
|
||||
前者联网，后者不请求模型；列表存在某个 ID 不代表其所有能力已验证。
|
||||
|
||||
## 三阶段执行命令
|
||||
## 四阶段执行命令
|
||||
|
||||
以下命令按顺序在同一个 PowerShell 会话执行。反引号后不能带空格。每次实验使用新的 runId；分阶段重开终端时，将路径变量设置成已有的对应目录。
|
||||
|
||||
@@ -204,6 +206,23 @@ node .\tools\model-check.mjs --stage completion `
|
||||
|
||||
uniform 仅恢复同宽高比的整画布尺寸，不纠正物体位移、变形或语义。在线请求现已显式传 size；参数及中转实测边界见[第三阶段文档](docs/3-completion-stage.md)。
|
||||
|
||||
### 第四阶段：离线组装与 Spine 导出
|
||||
|
||||
接续本轮 SAM 与补全目录，无需模型配置、密钥或 GPU：
|
||||
|
||||
```powershell
|
||||
node .\tools\model-check.mjs --stage spine `
|
||||
--input-dir "$runRoot\sam" `
|
||||
--completion-dir "$runRoot\complete" `
|
||||
--out-dir "$runRoot\spine"
|
||||
```
|
||||
|
||||
省略 `--completion-dir` 可只用 SAM 资产；加 `--dry-run` 完整检查输入而不写输出。默认补全失败回退 SAM 并记录原因；`--fallback-policy error` 要求需补全的可见部件全部有通过结果。可见部件缺 mask、混用不同输入或非法补全坐标会直接报错。
|
||||
|
||||
检查 `report.html`、`comparison.png` 和 `assembly.json`；`preview.html` 显示真实骨架，setup 为静态姿态，`check__骨骼ID` 为单关节旋转检查动画。生成过程离线；播放器页面从 CDN 加载 Spine Player，需要网络。`skeleton.json` 配合 `images/` 供编辑器导入，`atlas/` 用于运行时。
|
||||
|
||||
部件数量、名称、父子关系均由 v2 计划驱动；隐藏骨骼保留，附件按 bind_to 绑定。第四阶段只复用外部只读 `spine-animation-ai` 的图集和播放器脚本，保留其 PolyForm Noncommercial 约束。完整参数、输入契约和绘制顺序覆盖见[第四阶段文档](docs/4-spine-stage.md)。
|
||||
|
||||
## 旧管线与验证
|
||||
|
||||
旧版依赖相邻 `spine-animation-ai` 外部项目，使用时遵守其 PolyForm Noncommercial 许可证；不要修改外部依赖。旧版命令会调用图片生成并尝试定位、导出：
|
||||
@@ -218,7 +237,7 @@ node .\pipeline.mjs `
|
||||
--animations idle,walk,wave
|
||||
```
|
||||
|
||||
`segment.py` 为旧版拆件分割，`compose_preview.py` 组装预览，`run_case_matrix.py` 对照定位方案；当前 SAM 入口为 sam_segment.py。旧产物需检查 assembled.png 和 preview-rigged.html，不能据此声称新三阶段已经完成 Spine 导出。
|
||||
`segment.py` 为旧版拆件分割，`compose_preview.py` 组装预览，`run_case_matrix.py` 对照定位方案；当前 SAM 入口为 sam_segment.py。旧产物检查 assembled.png 和 preview-rigged.html；新管线导出使用独立的第四阶段入口。
|
||||
|
||||
测试与命令帮助：
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
读取 SAM 可见 mask，为需要遮挡补全或关节延伸的部件准备局部画布，调用中转站图片编辑模型，再将允许区域内的新增像素与原可见像素合并。Python 几何处理固定使用 Conda `ai-spine`；图片生成在远端完成，本阶段不加载本地 SAM。
|
||||
|
||||
当前已实现准备、真实图片请求、离线候选导入、几何检查与透明补全部件输出。尚未输出使用补全部件组装的完整角色，尚未接入新管线 Spine 导出。几何通过仍为 `needs_review`，不能等同于部件可用。
|
||||
当前已实现准备、真实图片请求、离线候选导入、几何检查与透明补全部件输出。[第四阶段](4-spine-stage.md) 独立整合 SAM 与补全部件，输出完整角色组装、Spine 骨架和预览。几何通过仍为 `needs_review`，不能等同于部件可用。
|
||||
|
||||
## 2. 输入、输出与执行链路
|
||||
|
||||
@@ -179,13 +179,13 @@ node .\tools\model-check.mjs --stage completion `
|
||||
| `completion-result.json` | 每部件状态、候选 QA、原图 crop_xyxy 和路径 |
|
||||
| `report.html` / `manifest.json` | 人工检查页 / 模式、网络标记、状态、输入/代码/产物哈希 |
|
||||
|
||||
parts/ 只含本次选中且通过检查的补全部件，不是全部 SAM 部件集合。无需补全、失败和未选中的部件不会自动复制进来，也没有完整角色组装图。
|
||||
parts/ 只含本次选中且通过检查的补全部件，不是全部 SAM 部件集合。无需补全、失败和未选中的部件不会自动复制进来；完整角色组装由第四阶段读取 SAM 与本目录后生成。
|
||||
|
||||
当前实验：`output/20260918-183532/complete2_gpt` 已真实请求 Tiantoken gpt-image-2 共 12 个部件。旧请求未传 size，返回 1248/1254 方形图，预期为 1024，全部严格尺寸拒绝。`complete2_gpt-size-recheck-v2` 离线复用原候选后，8 个几何通过；head、pants_thigh_l、pants_thigh_r、shoe_l 共 4 个因对齐或无有效新增拒绝。通过项中仍有衣料误分、错误手部和未补全子区域。显式 size 的请求构造已验证，但尚无修复后中转遵守 size 的新实测结论。
|
||||
|
||||
## 8. 当前阶段可改进点说明
|
||||
|
||||
- **补全后输出使用补全部件的完整角色。** 按原图坐标和遮挡顺序组装，复用无需补全的原拆件，明确标记失败项及回退来源；输出补全前后对比、接缝/重叠诊断。不能只展示少数 parts 文件作为完整结果。
|
||||
- **完整角色组装已接入第四阶段。** 按原坐标与遮挡约束组装，复用原拆件并记录失败回退，输出补全前后对比、覆盖与重叠诊断及单骨骼旋转检查动画。可靠的接缝与运动质量自动验收仍待实现。
|
||||
- **把原图或部件相关原图区域交给补全模型参考。** 接入并实测全角色/局部上下文参考和编辑 mask，记录实际发送内容。不能仅把 capabilities 改为 true 就宣称中转支持；需验证图片顺序、mask 语义、返回尺寸和画面保持能力。
|
||||
- **根据 SAM 实际结果重建需求。** 区分分割缺口修复、遮挡补全、关节延伸、附件底图清理；对应建立保留区域与可编辑区域，避免误分像素被永久保护。
|
||||
- **改进画布与补全区域。** 固定 32 像素不适配所有部件；小部件在 1024 大白底中占比低。可验证按部件设计区域、显式局部放大并记录逆变换，保证回贴坐标可检查。
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
# 第四阶段：角色组装与 Spine 导出
|
||||
|
||||
## 1. 阶段简介
|
||||
|
||||
将第二阶段的可见部件与第三阶段通过几何检查的补全部件整合，按原图坐标组装，并依据 v2 语义计划生成 Spine 4.2 骨架、region 附件、图集及预览。所有部件名称、数量、骨骼父子关系和附件绑定来自输入，不使用角色名称或人形比例模板。
|
||||
|
||||
生成过程完全离线，不调用图片模型、不运行 SAM。骨架预览页面嵌入资产，但会从 CDN 加载 Spine Player 4.2，打开播放器需要网络。静态报告与 PNG 可离线查看。
|
||||
|
||||
导出完成仍标记 `quality_status=needs_review`。当前只有 setup 与单骨骼旋转检查动画；没有成品 idle/walk，也没有 mesh、权重或 IK。关节、语义、接缝及运动质量需人工复核。
|
||||
|
||||
## 2. 输入、输出与执行链路
|
||||
|
||||
```text
|
||||
SAM 目录 + 可选补全目录 + 可选绘制顺序覆盖
|
||||
→ 校验 v2 计划与骨骼关系
|
||||
→ 对比两阶段 source / 计划 / segmentation / masks 的 SHA-256
|
||||
→ 合并部件：已选补全 / 原 SAM 像素 / 隐藏无纹理节点
|
||||
→ 根据遮挡约束稳定拓扑排序，记录没有约束的重叠
|
||||
→ 按 crop_xyxy 组装，输出来源与覆盖诊断
|
||||
→ 根据 parent / bind_to / pivot_norm 创建骨架与附件
|
||||
→ 生成 skeleton.json、图集、播放器、报告与 manifest
|
||||
```
|
||||
|
||||
SAM 原部件直接由 source + mask 重建，保持 RGB/Alpha 和原坐标；无需依赖 SAM 的紧裁剪 parts/。补全部件使用 `completion-result.json` 中选中结果的 `image` 和 `crop_xyxy`，同时检查尺寸及可见 RGBA 是否保持不变。
|
||||
|
||||
## 3. 阶段执行命令示例及参数说明
|
||||
|
||||
在项目根目录运行；环境统一为 Conda `ai-spine`。以下路径仅是现有输入示例，换角色时替换输入目录，不修改代码。输出目录每次独立。
|
||||
|
||||
```powershell
|
||||
Set-Location F:\Study\taonier\AI_spine
|
||||
conda activate ai-spine
|
||||
$samDir = '.\output\20260920-225137\sam'
|
||||
$completionDir = '.\output\20260920-225137\complete'
|
||||
$spineOut = '.\output\spine-' + (Get-Date -Format 'yyyyMMdd-HHmmss')
|
||||
|
||||
node .\tools\model-check.mjs --stage spine `
|
||||
--input-dir $samDir `
|
||||
--completion-dir $completionDir `
|
||||
--out-dir $spineOut
|
||||
|
||||
Start-Process "$spineOut\report.html"
|
||||
```
|
||||
|
||||
| 参数 | 含义 |
|
||||
| --- | --- |
|
||||
| `--input-dir DIR` | 必需，SAM 输出目录 |
|
||||
| `--completion-dir DIR` | 可选，同批输入的补全输出；省略则使用 SAM 资产 |
|
||||
| `--out-dir DIR` | 独立输出目录；dry-run 时可省略 |
|
||||
| `--fallback-policy sam/error` | 默认 sam；需要补全但没有通过结果时回退可见像素并记录原因。error 则终止 |
|
||||
| `--draw-order-file FILE` | JSON 对象，`draw_order` 包含所有可绘制 ID，按从后到前排列，各出现一次 |
|
||||
| `--check-angle FLOAT` | 单骨骼检查动画正负角度，默认 10，范围 (0,45] |
|
||||
| `--genielabs-dir DIR` | 默认相邻 `spine-animation-ai`；复用其图集和播放器脚本，保留 PolyForm Noncommercial 约束 |
|
||||
| `--python FILE` | 仅允许 ai-spine 解释器 |
|
||||
| `--dry-run` | 完整检查输入、PNG、坐标及依赖脚本，不写输出，不联网 |
|
||||
| `--resume` | 仅复用输入、参数、代码与产物哈希全部匹配的完整运行 |
|
||||
|
||||
该阶段不读取模型配置与密钥，也不接受 `--config`、模型参数或补全请求参数。自动化可用 `conda run --no-capture-output -n ai-spine node ...`。
|
||||
|
||||
可选 runtime 复检：将已有的官方 Spine 4.2 core IIFE 文件路径赋给 `$runtimeFile`，执行以下命令。工具不下载或安装依赖；检查图集尺寸、静态附件顶点和正负旋转下的后代跟随，不验证浏览器 WebGL 渲染。
|
||||
|
||||
```powershell
|
||||
$runtimeFile = '.\output\spine-runtime-validation-20260922\spine-core.js'
|
||||
conda run -n ai-spine node .\tools\verify-spine-runtime.mjs `
|
||||
--runtime-file $runtimeFile --input-dir $spineOut
|
||||
```
|
||||
|
||||
## 4. 输入格式要求
|
||||
|
||||
SAM 目录需含 `source.png`、有效 v2 `semantic-parts.json`、`segmentation.json`、其中记录的全画布二值 masks。可见部件缺少 mask 直接报错；hidden 不得有可见 mask。
|
||||
|
||||
补全目录需含相同输入快照、对应 masks 和 `completion-result.json`。只有 `selected_file` 非空、`qa.accepted=true` 且 `status=geometry_passed_needs_review` 的结果会被采用；其 `image` 文件必须存在，裁剪坐标为原画布内的整数半开区间，尺寸与 PNG 一致，可见像素必须逐像素匹配 SAM 源图。输入哈希不一致时拒绝混用。图片路径必须位于相应输入目录内。
|
||||
|
||||
补全失败、未处理、未提供或 `NO_ALLOWED_EXTENSION` 的需补全部件遵守 fallback-policy。hidden 保留语义骨骼但不生成纹理，也不因缺纹理触发严格回退错误。
|
||||
|
||||
可选绘制顺序文件格式为 `{"draw_order":["部件ID一","部件ID二"]}`。必须列出本次全部可绘制部件，不包含 hidden。显式顺序允许覆盖模型遮挡关系，冲突会在报告中列出。
|
||||
|
||||
## 5. 执行链路中重要部分代码文件及原理说明（包括重点说明）
|
||||
|
||||
**资产整合。** 按 ID 合并，保留来源、跳过或拒绝原因、补全 QA 与上游告警。通过几何检查的补全仍是待复核资产，失败结果不会当作补全成功使用。缺少图片或非法坐标不会被静默回退掩盖。
|
||||
|
||||
**位置与骨骼。** 图像坐标原点为左上角，Spine 原点设为原画布底边中点：`X=x-W/2`，`Y=H-y`。pivot 使用连续坐标 `x=pivot_norm[0]*W`、`y=pivot_norm[1]*H`。所有骨骼 setup 旋转为零、缩放为一，因此子骨骼局部位置是世界 pivot 减父 pivot；附件位置是裁剪中心减绑定骨骼 pivot。骨骼按父先子后输出，hidden bone_part 也创建骨骼。attachment/facial_feature 绑定 bind_to，null 绑定 root，不创建额外独立骨骼。
|
||||
|
||||
**绘制顺序。** `A.occluded_by` 包含 B 时，A 排在 B 后方。排序保留隐藏节点的传递约束，最终只输出有纹理的 slot。没有约束时使用计划中的稳定顺序；若这些部件确有 Alpha 重叠，记录 unconstrained_overlaps，不能声称已确定遮挡。循环关系拒绝导出。
|
||||
|
||||
**检查动画。** 每个语义骨骼生成 `check__ID`：0→正角度→0→负角度→0，时长 2 秒，子骨骼与附件按层级跟随。setup 是静态姿态。它们供人工检查绑定和露底，不自动判定动作质量。
|
||||
|
||||
Spine 字段依据 [官方 JSON 格式说明](https://esotericsoftware.com/spine-json-format)，导出目标为 4.2；旋转时间线使用 4.2 的 `value` 字段。
|
||||
|
||||
## 6. 各个代码文件职责索引
|
||||
|
||||
| 文件 | 职责 |
|
||||
| --- | --- |
|
||||
| `tools/model-check.mjs` | 独立离线入口、参数与其他阶段隔离 |
|
||||
| `stages/spine-stage.mjs` | 验证计划、Conda 调用、指纹、导出调度与复用 |
|
||||
| `spine_assembly.py` | 资产核对、顺序、组装、诊断、骨架及报告 |
|
||||
| `lib/validate-parts.mjs` | v2 Schema、引用、骨骼和遮挡环路校验 |
|
||||
| `lib/run-artifacts.mjs` | 独立输出目录和产物哈希 |
|
||||
| 外部 `scripts/make_atlas.py` | 透明 PNG 图集打包，不修改外部依赖 |
|
||||
| 外部 `scripts/generate_spine_player.py` | 嵌入骨架和图集的 Spine Player HTML |
|
||||
| `tests/test_spine_assembly.py` / `tests/spine-stage.test.mjs` | 非人形合成数据、坐标、父子绑定、输入身份、回退、CLI 与完整导出回归 |
|
||||
| `tools/verify-spine-runtime.mjs` | 可选官方 runtime 加载、图集匹配与世界顶点/旋转复检，需提供本地 core IIFE |
|
||||
|
||||
## 7. 产物详细说明
|
||||
|
||||
| 文件 | 用途 |
|
||||
| --- | --- |
|
||||
| `source.png` / `semantic-parts.json` | 原图与语义计划快照 |
|
||||
| `images/<id>.png` | 完整可绘制部件集合，供 Spine 编辑器导入 |
|
||||
| `assembly.json` | 坐标、来源、回退、顺序、QA、输入哈希与覆盖诊断 |
|
||||
| `assembled-sam.png` / `assembled.png` | 同一绘制顺序的原 SAM / 最终资产透明组装图 |
|
||||
| `comparison.png` | 左原图、中 SAM 组装、右最终组装 |
|
||||
| `coverage-diagnostic.png` | 红色为原图前景未覆盖，黄色为多部件重叠；重叠不一定是错误 |
|
||||
| `skeleton.json` | Spine 4.2 骨骼、slot、region、setup 和关节检查动画；images 路径为 `./images/` |
|
||||
| `atlas/skeleton.atlas` / `atlas/skeleton.png` | 运行时图集 |
|
||||
| `preview.html` | 真正的骨架预览，可切换检查动画；需要联网加载播放器 |
|
||||
| `report.html` | 静态对比、诊断、来源列表与预览入口 |
|
||||
| `manifest.json` | 执行状态、质量状态、输入/代码/产物哈希、计数、依赖及限制 |
|
||||
|
||||
2026-09-22 验证记录：`conda run --no-capture-output -n ai-spine npm test` 的 26 项测试全部通过，包含 6 项新增 Python 组装用例。`output/spine-validation-latest-20260922` 和 `output/spine-validation-baseline-20260922` 分别导出 25/22 个 slot、各 16 个骨骼（含 root）；官方 Spine 4.2 runtime 对两者的 setup 顶点与各 30 个旋转采样验证通过。最新导出的播放器另经 headless Chrome 实际加载，显示 Loaded successfully 并渲染角色，截图位于 `output/spine-runtime-validation-20260922/preview.png`。这不代表资产或动作质量合格，现有输入的漏分仍显示在组装对比图中。
|
||||
|
||||
## 8. 当前阶段可改进点说明
|
||||
|
||||
- 待实现：针对未确定遮挡、pivot 和绑定的交互修正界面；目前可通过语义计划与绘制顺序文件调整。
|
||||
- 待实现：可靠的接缝、露底与旋转质量自动验收；目前输出检查动画与诊断，质量状态始终 needs_review。
|
||||
- 待实现：mesh、权重、IK，以及由动作语义驱动的成品动画；不通过固定部件名称硬套人形模板。
|
||||
- 待验证：不同角色类别与更复杂遮挡关系的资产质量。合成回归只验证数据契约及几何逻辑。
|
||||
- 待实现：本地播放器资源打包，使骨架预览也可完全离线打开。
|
||||
@@ -0,0 +1,326 @@
|
||||
"""Offline asset assembly and Spine 4.2 region export, driven by semantic v2 data."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def write_json(file, data):
|
||||
Path(file).write_text(json.dumps(data, ensure_ascii=False, indent=2, allow_nan=False) + '\n', encoding='utf-8')
|
||||
|
||||
|
||||
def safe_file(root, name):
|
||||
if not isinstance(name, str) or not name or Path(name).is_absolute():
|
||||
raise ValueError(f'Expected relative asset path: {name}')
|
||||
root = Path(root).resolve()
|
||||
file = (root / name).resolve()
|
||||
if not file.is_relative_to(root) or not file.is_file():
|
||||
raise ValueError(f'Missing asset or path outside input directory: {name}')
|
||||
return file
|
||||
|
||||
|
||||
def ordered(nodes, edges):
|
||||
"""Stable topological order; do not silently break contradictory constraints."""
|
||||
pending = list(nodes)
|
||||
result = []
|
||||
while pending:
|
||||
ready = next((n for n in pending if not any(b == n and a in pending for a, b in edges)), None)
|
||||
if ready is None:
|
||||
raise ValueError('Cyclic bone or draw-order constraints')
|
||||
pending.remove(ready)
|
||||
result.append(ready)
|
||||
return result
|
||||
|
||||
|
||||
def load_assets(input_dir, completion_dir=None, fallback_policy='sam', draw_order_file=None):
|
||||
root = Path(input_dir).resolve()
|
||||
hashes = {}
|
||||
|
||||
def tracked(folder, name, label):
|
||||
file = safe_file(folder, name)
|
||||
hashes[f'{label}/{name}'] = hashlib.sha256(file.read_bytes()).hexdigest()
|
||||
return file
|
||||
|
||||
def read(folder, name, label):
|
||||
return json.loads(tracked(folder, name, label).read_text(encoding='utf-8-sig'))
|
||||
|
||||
source = Image.open(tracked(root, 'source.png', 'sam')).convert('RGBA')
|
||||
rgba = np.array(source)
|
||||
w, h = source.size
|
||||
plan = read(root, 'semantic-parts.json', 'sam')
|
||||
seg = read(root, 'segmentation.json', 'sam')
|
||||
parts = plan['parts']
|
||||
by_id = {p['id']: p for p in parts}
|
||||
if len(by_id) != len(parts):
|
||||
raise ValueError('Duplicate semantic ID')
|
||||
|
||||
def records(items, label):
|
||||
result = {}
|
||||
for item in items:
|
||||
name = item['id']
|
||||
if name not in by_id or name in result:
|
||||
raise ValueError(f'{label}: unknown or duplicate ID: {name}')
|
||||
result[name] = item
|
||||
return result
|
||||
|
||||
segmented = records(seg['parts'], 'segmentation')
|
||||
completed, skipped = {}, {}
|
||||
completion = Path(completion_dir).resolve() if completion_dir else None
|
||||
# A completion run snapshots its SAM inputs, so compare content, not directory names.
|
||||
if completion:
|
||||
for name in ['source.png', 'semantic-parts.json', 'segmentation.json']:
|
||||
tracked(completion, name, 'completion')
|
||||
if hashes[f'sam/{name}'] != hashes[f'completion/{name}']:
|
||||
raise ValueError(f'Completion input mismatch: {name}')
|
||||
result = read(completion, 'completion-result.json', 'completion')
|
||||
completed = records(result['parts'], 'completion')
|
||||
skipped = records(result.get('skipped', []), 'completion skipped')
|
||||
if set(completed) & set(skipped):
|
||||
raise ValueError('Completion ID occurs in both parts and skipped')
|
||||
|
||||
images, original_images, asset_records = {}, {}, []
|
||||
masks = {}
|
||||
for name, record in segmented.items():
|
||||
if by_id[name]['visibility'] == 'hidden':
|
||||
raise ValueError(f'Hidden part unexpectedly has a SAM mask: {name}')
|
||||
mask_name = record.get('mask', f'masks/{name}.png')
|
||||
mask = np.array(Image.open(tracked(root, mask_name, 'sam')))
|
||||
if mask.shape != (h, w) or not np.isin(mask, [0, 255]).all():
|
||||
raise ValueError(f'Invalid full-canvas binary mask: {name}')
|
||||
mask = mask > 0
|
||||
if not mask.any() or np.any(mask & (rgba[:, :, 3] == 0)):
|
||||
raise ValueError(f'Empty mask or mask outside source alpha: {name}')
|
||||
if completion:
|
||||
tracked(completion, mask_name, 'completion')
|
||||
if hashes[f'sam/{mask_name}'] != hashes[f'completion/{mask_name}']:
|
||||
raise ValueError(f'Completion mask mismatch: {name}')
|
||||
masks[name] = mask
|
||||
|
||||
for part in parts:
|
||||
name = part['id']
|
||||
if part['visibility'] == 'hidden':
|
||||
asset_records.append({'id': name, 'source': 'hidden', 'image': None,
|
||||
'reason': 'HIDDEN_NO_TEXTURE', 'semantic_verified': False})
|
||||
continue
|
||||
if name not in masks:
|
||||
raise ValueError(f'Visible part has no SAM mask: {name}')
|
||||
mask = masks[name]
|
||||
ys, xs = np.nonzero(mask)
|
||||
original_box = [int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1]
|
||||
cut = rgba.copy()
|
||||
cut[~mask] = 0
|
||||
original = Image.fromarray(cut).crop(original_box)
|
||||
original_images[name] = (original, original_box)
|
||||
candidate = completed.get(name, {})
|
||||
accepted = bool(candidate.get('selected_file')) and candidate.get('qa', {}).get('accepted') is True
|
||||
warnings = list(segmented[name].get('warnings', []))
|
||||
if accepted:
|
||||
if candidate.get('status') != 'geometry_passed_needs_review':
|
||||
raise ValueError(f'Inconsistent selected completion status: {name}')
|
||||
image = Image.open(tracked(completion, candidate.get('image'), 'completion')).convert('RGBA')
|
||||
box = candidate.get('crop_xyxy')
|
||||
if (not isinstance(box, list) or len(box) != 4 or
|
||||
any(type(x) is not int for x in box) or
|
||||
not (0 <= box[0] < box[2] <= w and 0 <= box[1] < box[3] <= h) or
|
||||
image.size != (box[2] - box[0], box[3] - box[1])):
|
||||
raise ValueError(f'Invalid completion crop/size: {name}')
|
||||
full = Image.new('RGBA', source.size)
|
||||
full.paste(image, (box[0], box[1]))
|
||||
if not np.array_equal(np.array(full)[mask], rgba[mask]):
|
||||
raise ValueError(f'Completion does not preserve original visible RGBA: {name}')
|
||||
origin, reason = 'completion', 'GEOMETRY_PASSED_NEEDS_REVIEW'
|
||||
if candidate.get('qa', {}).get('unfulfilled_regions'):
|
||||
warnings.append('UNFULFILLED_COMPLETION_REGIONS')
|
||||
else:
|
||||
image, box = original, original_box
|
||||
origin = 'sam'
|
||||
if not part['needs_completion']:
|
||||
reason = 'NO_COMPLETION_NEEDED'
|
||||
else:
|
||||
reason = skipped.get(name, {}).get('reason') or (
|
||||
'COMPLETION_' + str(candidate.get('status', 'NOT_PROVIDED')).upper())
|
||||
warnings.append('USING_UNCOMPLETED_SAM')
|
||||
if fallback_policy == 'error':
|
||||
raise ValueError(f'Completion required for {name}: {reason}')
|
||||
images[name] = (image, box)
|
||||
asset_records.append({'id': name, 'source': origin, 'reason': reason, 'image': f'images/{name}.png',
|
||||
'crop_xyxy': box, 'original_crop_xyxy': original_box,
|
||||
'semantic_verified': False, 'warnings': warnings,
|
||||
'completion_status': candidate.get('status'),
|
||||
'completion_qa': candidate.get('qa')})
|
||||
|
||||
if not images:
|
||||
raise ValueError('No drawable assets')
|
||||
edges = [(p['id'], front) for p in parts for front in p['occluded_by']]
|
||||
# Keep hidden nodes while sorting so transitive constraints survive their removal.
|
||||
draw_order = [n for n in ordered(by_id, edges) if n in images]
|
||||
overridden = False
|
||||
if draw_order_file:
|
||||
file = Path(draw_order_file).resolve()
|
||||
hashes['draw_order_override'] = hashlib.sha256(file.read_bytes()).hexdigest()
|
||||
override = json.loads(file.read_text(encoding='utf-8-sig'))
|
||||
draw_order = override['draw_order']
|
||||
if not isinstance(draw_order, list) or len(draw_order) != len(images) or set(draw_order) != set(images):
|
||||
raise ValueError('draw_order must contain each drawable ID exactly once')
|
||||
overridden = True
|
||||
reach = {n: set() for n in by_id}
|
||||
for n in reversed(ordered(by_id, edges)):
|
||||
for a, b in edges:
|
||||
if a == n:
|
||||
reach[n].add(b)
|
||||
reach[n].update(reach[b])
|
||||
unresolved = []
|
||||
for i, a in enumerate(draw_order):
|
||||
for b in draw_order[i + 1:]:
|
||||
if b in reach[a] or a in reach[b]:
|
||||
continue
|
||||
ai, ab = images[a]
|
||||
bi, bb = images[b]
|
||||
x0, y0, x1, y1 = max(ab[0], bb[0]), max(ab[1], bb[1]), min(ab[2], bb[2]), min(ab[3], bb[3])
|
||||
if x0 < x1 and y0 < y1:
|
||||
am = np.array(ai.getchannel('A'))[y0-ab[1]:y1-ab[1], x0-ab[0]:x1-ab[0]] > 0
|
||||
bm = np.array(bi.getchannel('A'))[y0-bb[1]:y1-bb[1], x0-bb[0]:x1-bb[0]] > 0
|
||||
overlap = int((am & bm).sum())
|
||||
if overlap:
|
||||
unresolved.append({'parts': [a, b], 'overlap_pixels': overlap})
|
||||
violated = [[a, b] for a in draw_order for b in reach[a]
|
||||
if b in images and draw_order.index(a) > draw_order.index(b)]
|
||||
assembly = {'schema_version': 1, 'quality_status': 'needs_review', 'semantic_verified': False,
|
||||
'width': w, 'height': h, 'coordinate_space': 'pixel_xy_top_left',
|
||||
'parts': asset_records, 'draw_order': draw_order,
|
||||
'draw_order_source': 'override' if overridden else 'occlusion_stable_topological',
|
||||
'unconstrained_overlaps': unresolved, 'overridden_occlusion_constraints': violated,
|
||||
'input_hashes': hashes}
|
||||
return source, plan, assembly, images, original_images
|
||||
|
||||
|
||||
def build_skeleton(plan, assembly, check_angle=10):
|
||||
w, h = assembly['width'], assembly['height']
|
||||
bone_parts = {p['id']: p for p in plan['parts'] if p['rig_role'] == 'bone_part'}
|
||||
edges = [(p['parent'], name) for name, p in bone_parts.items() if p['parent']]
|
||||
bone_order = ordered(bone_parts, edges)
|
||||
world = {'root': (0, 0)}
|
||||
bones = [{'name': 'root'}]
|
||||
for name in bone_order:
|
||||
part = bone_parts[name]
|
||||
world[name] = (part['pivot_norm'][0] * w - w / 2, h - part['pivot_norm'][1] * h)
|
||||
parent = part['parent'] or 'root'
|
||||
x, y = world[name]
|
||||
px, py = world[parent]
|
||||
# Setup axes remain aligned with the canvas. Parent rotation still propagates
|
||||
# to children; attachment offsets below preserve the exact original pose.
|
||||
bones.append({'name': name, 'parent': parent, 'x': x - px, 'y': y - py})
|
||||
semantic = {p['id']: p for p in plan['parts']}
|
||||
assets = {p['id']: p for p in assembly['parts']}
|
||||
slots, attachments = [], {}
|
||||
for name in assembly['draw_order']:
|
||||
part, asset = semantic[name], assets[name]
|
||||
bone = name if part['rig_role'] == 'bone_part' else part['bind_to'] or 'root'
|
||||
bx, by = world[bone]
|
||||
x0, y0, x1, y1 = asset['crop_xyxy']
|
||||
slots.append({'name': name, 'bone': bone, 'attachment': name})
|
||||
attachments[name] = {name: {'type': 'region', 'path': name,
|
||||
'x': (x0 + x1) / 2 - w / 2 - bx,
|
||||
'y': h - (y0 + y1) / 2 - by,
|
||||
'width': x1 - x0, 'height': y1 - y0}}
|
||||
animations = {'setup': {}}
|
||||
for name in bone_order:
|
||||
animations[f'check__{name}'] = {'bones': {name: {'rotate': [
|
||||
{'time': 0, 'value': 0}, {'time': .5, 'value': check_angle},
|
||||
{'time': 1, 'value': 0}, {'time': 1.5, 'value': -check_angle}, {'time': 2, 'value': 0}]}}}
|
||||
return {'skeleton': {'spine': '4.2.0', 'x': -w / 2, 'y': 0, 'width': w, 'height': h,
|
||||
'images': './images/'},
|
||||
'bones': bones, 'slots': slots, 'skins': [{'name': 'default', 'attachments': attachments}],
|
||||
'animations': animations}
|
||||
|
||||
|
||||
def compose(size, images, order):
|
||||
canvas = Image.new('RGBA', size)
|
||||
coverage = np.zeros((size[1], size[0]), np.uint16)
|
||||
for name in order:
|
||||
image, (x0, y0, x1, y1) = images[name]
|
||||
canvas.alpha_composite(image, (x0, y0))
|
||||
coverage[y0:y1, x0:x1] += np.array(image.getchannel('A')) > 0
|
||||
return canvas, coverage
|
||||
|
||||
|
||||
def export_assets(data, output, check_angle=10):
|
||||
source, plan, assembly, images, original_images = data
|
||||
out = Path(output)
|
||||
(out / 'images').mkdir(parents=True, exist_ok=True)
|
||||
source.save(out / 'source.png')
|
||||
write_json(out / 'semantic-parts.json', plan)
|
||||
for name, (im, _) in images.items():
|
||||
im.save(out / 'images' / f'{name}.png')
|
||||
assembled, coverage = compose(source.size, images, assembly['draw_order'])
|
||||
before, _ = compose(source.size, original_images, assembly['draw_order'])
|
||||
assembled.save(out / 'assembled.png')
|
||||
before.save(out / 'assembled-sam.png')
|
||||
foreground = np.array(source.getchannel('A')) > 0
|
||||
missing = foreground & (coverage == 0)
|
||||
diagnostic = np.zeros((source.height, source.width, 4), np.uint8)
|
||||
diagnostic[coverage > 1] = [255, 180, 0, 220]
|
||||
diagnostic[missing] = [255, 0, 80, 255]
|
||||
Image.fromarray(diagnostic).save(out / 'coverage-diagnostic.png')
|
||||
comparison = Image.new('RGBA', (source.width * 3, source.height), 'white')
|
||||
for i, im in enumerate([source, before, assembled]):
|
||||
comparison.alpha_composite(im, (i * source.width, 0))
|
||||
comparison.convert('RGB').save(out / 'comparison.png')
|
||||
assembly['diagnostics'] = {'uncovered_source_pixels': int(missing.sum()),
|
||||
'overlap_pixels': int((coverage > 1).sum()),
|
||||
'rotation_quality_verified': False}
|
||||
write_json(out / 'assembly.json', assembly)
|
||||
skeleton = build_skeleton(plan, assembly, check_angle)
|
||||
write_json(out / 'skeleton.json', skeleton)
|
||||
rows = []
|
||||
for part in assembly['parts']:
|
||||
picture = f'<img src="{part["image"]}">' if part['image'] else ''
|
||||
rows.append('<tr><td>' + html.escape(part['id']) + '</td><td>' + html.escape(part['source']) +
|
||||
'</td><td>' + html.escape(part['reason']) + '</td><td>' + picture + '</td></tr>')
|
||||
notes = {key: assembly[key] for key in ['diagnostics', 'draw_order_source', 'unconstrained_overlaps',
|
||||
'overridden_occlusion_constraints']}
|
||||
(out / 'report.html').write_text('''<!doctype html><html lang="zh-CN"><meta charset="utf-8">
|
||||
<title>第四阶段：Spine 组装与导出</title>
|
||||
<style>body{font:16px system-ui;margin:24px}img{max-width:100%;background:#ddd}td img{max-height:150px}td,th{border:1px solid #ccc;padding:8px}pre{white-space:pre-wrap}</style>
|
||||
<h1>第四阶段：Spine 组装与导出</h1>
|
||||
<p>状态：needs_review。几何通过与成功导出不代表语义、接缝或动作质量合格。</p>
|
||||
<p><a href="preview.html">Spine 预览</a>（播放器从 CDN 加载，需要网络）；选择 setup 查看静态姿态，选择 check__骨骼ID 检查单关节旋转及子骨骼跟随。检查动画不代表成品动作。</p>
|
||||
<p><a href="skeleton.json">骨架 JSON</a> · <a href="assembly.json">来源与诊断</a></p>
|
||||
<h2>原图 / SAM 组装 / 最终组装</h2><img src="comparison.png">
|
||||
<h2>覆盖诊断：红色为原图漏分，黄色为部件重叠</h2><img src="coverage-diagnostic.png">
|
||||
<p>重叠可能是正常遮挡；覆盖率不能判断语义。没有遮挡约束的重叠使用语义计划中的稳定顺序，需复核。</p>
|
||||
<pre>''' + html.escape(json.dumps(notes, ensure_ascii=False, indent=2)) + '''</pre>
|
||||
<h2>资产来源</h2><table><tr><th>ID</th><th>来源</th><th>原因</th><th>部件</th></tr>''' +
|
||||
''.join(rows) + '</table></html>', encoding='utf-8')
|
||||
return {'parts': len(images), 'bones': len(skeleton['bones']),
|
||||
'completion_parts': sum(p['source'] == 'completion' for p in assembly['parts']),
|
||||
'fallback_parts': sum('USING_UNCOMPLETED_SAM' in p.get('warnings', []) for p in assembly['parts']),
|
||||
'quality_status': 'needs_review', 'input_hashes': assembly['input_hashes']}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--input-dir', required=True)
|
||||
parser.add_argument('--completion-dir')
|
||||
parser.add_argument('--output')
|
||||
parser.add_argument('--draw-order-file')
|
||||
parser.add_argument('--fallback-policy', choices=['sam', 'error'], default='sam')
|
||||
parser.add_argument('--check-angle', type=float, default=10)
|
||||
args = parser.parse_args()
|
||||
if not 0 < args.check_angle <= 45:
|
||||
parser.error('check-angle must be in (0, 45]')
|
||||
data = load_assets(args.input_dir, args.completion_dir, args.fallback_policy, args.draw_order_file)
|
||||
if args.output:
|
||||
result = export_assets(data, args.output, args.check_angle)
|
||||
else:
|
||||
skeleton = build_skeleton(data[1], data[2], args.check_angle)
|
||||
result = {'input_hashes': data[2]['input_hashes'], 'parts': len(data[3]), 'bones': len(skeleton['bones'])}
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,79 @@
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {readFileSync, readdirSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import {projectRoot} from '../lib/config.mjs';
|
||||
import {selectPython} from '../lib/python-runtime.mjs';
|
||||
import {validateParts} from '../lib/validate-parts.mjs';
|
||||
import {sha256} from '../lib/images.mjs';
|
||||
import {prepareRun, jsonFile, finishRun} from '../lib/run-artifacts.mjs';
|
||||
|
||||
const readJson = (file) => JSON.parse(readFileSync(file, 'utf8').replace(/^\uFEFF/u, ''));
|
||||
const filesIn = (dir, prefix = '') => readdirSync(dir, {withFileTypes: true}).flatMap((entry) => entry.isDirectory()
|
||||
? filesIn(path.join(dir, entry.name), `${prefix}${entry.name}/`) : [`${prefix}${entry.name}`]);
|
||||
|
||||
export async function runSpine(options) {
|
||||
if (!options.inputDir || (!options.outDir && !options.dryRun)) throw new Error('Spine 阶段需要 --input-dir SAM目录 和独立的 --out-dir');
|
||||
const input = path.resolve(options.inputDir);
|
||||
const completion = options.completionDir ? path.resolve(options.completionDir) : undefined;
|
||||
const plan = readJson(path.join(input, 'semantic-parts.json'));
|
||||
const validation = validateParts(plan);
|
||||
if (!validation.valid || plan.schema_version !== 2) throw new Error(`Spine 阶段需要有效 v2 计划：${validation.errors.join(', ')}`);
|
||||
const fallback = options.fallbackPolicy || 'sam';
|
||||
const angle = Number(options.checkAngle ?? 10);
|
||||
if (!['sam', 'error'].includes(fallback)) throw new Error('--fallback-policy 仅支持 sam / error');
|
||||
if (!(angle > 0 && angle <= 45)) throw new Error('--check-angle 必须在 (0, 45] 度之间');
|
||||
const python = selectPython(options.python);
|
||||
const external = path.resolve(options.genielabsDir || path.join(projectRoot, '..', 'spine-animation-ai'));
|
||||
const scripts = ['make_atlas.py', 'generate_spine_player.py'].map((name) => path.join(external, 'scripts', name));
|
||||
const codeFiles = ['spine_assembly.py', 'stages/spine-stage.mjs', 'tools/model-check.mjs',
|
||||
'lib/python-runtime.mjs', 'lib/run-artifacts.mjs', 'lib/images.mjs', 'lib/validate-parts.mjs',
|
||||
'schemas/semantic-parts.schema.json'].map((name) => path.join(projectRoot, name));
|
||||
const codeHash = sha256([...codeFiles, ...scripts].map((file) => readFileSync(file)).map((bytes) => sha256(bytes)).join('\n'));
|
||||
function call(script, args) {
|
||||
try {
|
||||
return execFileSync(python, [script, ...args], {cwd: projectRoot, windowsHide: true,
|
||||
encoding: 'utf8', timeout: 120000, maxBuffer: 8 * 1024 * 1024,
|
||||
env: {...process.env, PYTHONUTF8: '1'}, stdio: ['ignore', 'pipe', 'pipe']});
|
||||
} catch (error) { throw new Error(`Spine 离线处理失败：${error.stderr || error.message}`); }
|
||||
}
|
||||
const worker = path.join(projectRoot, 'spine_assembly.py');
|
||||
const args = ['--input-dir', input, '--fallback-policy', fallback, '--check-angle', String(angle)];
|
||||
if (completion) args.push('--completion-dir', completion);
|
||||
if (options.drawOrderFile) args.push('--draw-order-file', path.resolve(options.drawOrderFile));
|
||||
// Inspect all masks, selected PNGs, crop coordinates and input identities before creating output.
|
||||
const inspected = JSON.parse(call(worker, args));
|
||||
if (options.dryRun) {
|
||||
const result = {dryRun: true, stage: 'spine', ...inspected, network_requested: false};
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
return result;
|
||||
}
|
||||
const identity = {schema_version: 1, stage: 'spine', spine_version: '4.2.0',
|
||||
inputs: inspected.input_hashes, code_sha256: codeHash, fallback_policy: fallback, check_angle: angle};
|
||||
const fingerprint = sha256(JSON.stringify(identity));
|
||||
const run = prepareRun(options.outDir, fingerprint, options.resume);
|
||||
if (run.reused) return {out: run.out, reused: true};
|
||||
const manifest = {...identity, fingerprint, status: 'running', quality_status: 'needs_review',
|
||||
network_requested: false, started_at: new Date().toISOString(),
|
||||
external_dependency: {name: 'spine-animation-ai', license: 'PolyForm-Noncommercial-1.0.0'},
|
||||
limitations: ['region attachments only; no mesh, weights or IK',
|
||||
'setup and per-bone check animations only; semantic and motion quality need review',
|
||||
'preview embeds assets but loads Spine Player 4.2 from CDN']};
|
||||
jsonFile(run.manifestPath, manifest);
|
||||
try {
|
||||
const result = JSON.parse(call(worker, [...args, '--output', run.out]));
|
||||
if (JSON.stringify(result.input_hashes) !== JSON.stringify(inspected.input_hashes)) throw new Error('处理期间输入发生变化，请使用新输出目录重跑');
|
||||
call(scripts[0], ['--parts', path.join(run.out, 'images'), '--output', path.join(run.out, 'atlas'), '--name', 'skeleton']);
|
||||
call(scripts[1], ['--skeleton', path.join(run.out, 'skeleton.json'),
|
||||
'--atlas', path.join(run.out, 'atlas', 'skeleton.atlas'), '--atlas-image', path.join(run.out, 'atlas', 'skeleton.png'),
|
||||
'--output', path.join(run.out, 'preview.html'), '--animation', 'setup', '--title', 'Spine Assembly Review']);
|
||||
Object.assign(manifest, {part_count: result.parts, bone_count: result.bones,
|
||||
completion_part_count: result.completion_parts, fallback_part_count: result.fallback_parts,
|
||||
completed_at: new Date().toISOString()});
|
||||
finishRun(run, manifest, filesIn(run.out).filter((name) => name !== 'manifest.json'));
|
||||
return {out: run.out, reused: false, ...result};
|
||||
} catch (error) {
|
||||
manifest.status = 'failed'; manifest.error = error.message;
|
||||
jsonFile(run.manifestPath, manifest);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {mkdtempSync, readFileSync, writeFileSync, realpathSync, rmSync, existsSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {projectRoot} from '../lib/config.mjs';
|
||||
import {selectPython} from '../lib/python-runtime.mjs';
|
||||
import {runSpine} from '../stages/spine-stage.mjs';
|
||||
|
||||
const python = selectPython();
|
||||
const execution = {cwd: projectRoot, windowsHide: true, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe']};
|
||||
function fixture(t) {
|
||||
const base = realpathSync(tmpdir());
|
||||
const root = mkdtempSync(path.join(base, 'spine-export-'));
|
||||
t.after(() => {
|
||||
const resolved = realpathSync(root);
|
||||
assert.equal(path.dirname(resolved), base);
|
||||
assert.ok(path.basename(resolved).startsWith('spine-export-'));
|
||||
rmSync(resolved, {recursive: true});
|
||||
});
|
||||
execFileSync(python, ['-c', 'import sys; sys.path.insert(0, "tests"); from test_spine_assembly import make_fixture; make_fixture(sys.argv[1])', root], execution);
|
||||
return {root, inputDir: path.join(root, 'sam'), completionDir: path.join(root, 'complete'), outDir: path.join(root, 'out')};
|
||||
}
|
||||
|
||||
test('Spine assembly preserves coordinates, hidden hierarchy, bindings and validates asset identity', () => {
|
||||
assert.equal(execFileSync(python, ['-m', 'unittest', 'discover', '-s', 'tests', '-p', 'test_spine_assembly.py'], execution), '');
|
||||
});
|
||||
|
||||
test('Spine exports complete assets, atlas, player, diagnostics and verifies resume hashes', async (t) => {
|
||||
const opts = fixture(t);
|
||||
const result = await runSpine(opts);
|
||||
assert.equal(result.parts, 4);
|
||||
assert.equal(result.bones, 4);
|
||||
for (const file of ['assembly.json', 'skeleton.json', 'atlas/skeleton.atlas', 'atlas/skeleton.png',
|
||||
'images/shell.png', 'assembled.png', 'comparison.png', 'coverage-diagnostic.png', 'report.html', 'preview.html']) {
|
||||
assert.ok(existsSync(path.join(opts.outDir, file)), file);
|
||||
}
|
||||
const manifest = JSON.parse(readFileSync(path.join(opts.outDir, 'manifest.json')));
|
||||
assert.equal(manifest.status, 'completed');
|
||||
assert.equal(manifest.quality_status, 'needs_review');
|
||||
assert.equal(manifest.network_requested, false);
|
||||
assert.equal(manifest.fallback_part_count, 1);
|
||||
const skeleton = JSON.parse(readFileSync(path.join(opts.outDir, 'skeleton.json')));
|
||||
assert.equal(skeleton.skeleton.images, './images/');
|
||||
const atlas = readFileSync(path.join(opts.outDir, 'atlas/skeleton.atlas'), 'utf8').replaceAll('\r\n', '\n');
|
||||
for (const slot of skeleton.slots) assert.ok(atlas.includes(`\n${slot.attachment}\n`));
|
||||
assert.match(readFileSync(path.join(opts.outDir, 'preview.html'), 'utf8'), /animation: "setup"/u);
|
||||
assert.equal((await runSpine({...opts, resume: true})).reused, true);
|
||||
await assert.rejects(runSpine(opts), /非空/u);
|
||||
await assert.rejects(runSpine({...opts, resume: true, checkAngle: 5}), /变化/u);
|
||||
writeFileSync(path.join(opts.outDir, 'images/shell.png'), 'tampered');
|
||||
await assert.rejects(runSpine({...opts, resume: true}), /产物缺失或被修改/u);
|
||||
});
|
||||
|
||||
test('Spine CLI is independent of model config and dry-run writes nothing', (t) => {
|
||||
const opts = fixture(t);
|
||||
const args = ['tools/model-check.mjs', '--stage', 'spine', '--input-dir', opts.inputDir,
|
||||
'--completion-dir', opts.completionDir, '--out-dir', opts.outDir];
|
||||
const env = {...process.env, VECTOR_ENGINE_API_KEY: '', TIANTOKEN_API_KEY: ''};
|
||||
assert.match(execFileSync(process.execPath, [...args, '--dry-run'], {...execution, env}), /"network_requested": false/u);
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
assert.throws(() => execFileSync(process.execPath, [...args, '--config', 'missing-config.json'], execution),
|
||||
(error) => /独立离线阶段/u.test(error.stderr));
|
||||
assert.throws(() => execFileSync(process.execPath, [...args, '--fallback-policy', 'error'], execution),
|
||||
(error) => /Completion required/u.test(error.stderr));
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
assert.match(execFileSync(process.execPath, args, {...execution, env}), /导出完成/u);
|
||||
});
|
||||
|
||||
test('Spine rejects malformed semantic hierarchy before export', async (t) => {
|
||||
const opts = fixture(t);
|
||||
const file = path.join(opts.inputDir, 'semantic-parts.json');
|
||||
const plan = JSON.parse(readFileSync(file));
|
||||
plan.parts.find((p) => p.id === 'hub').parent = 'tip_q';
|
||||
writeFileSync(file, JSON.stringify(plan));
|
||||
await assert.rejects(runSpine(opts), /存在环/u);
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
});
|
||||
@@ -0,0 +1,180 @@
|
||||
import copy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from spine_assembly import load_assets, build_skeleton, export_assets, write_json
|
||||
|
||||
|
||||
def make_fixture(directory):
|
||||
"""An asymmetric mechanical fixture; no humanoid names or assumed part count."""
|
||||
root = Path(directory)
|
||||
sam, completion = root / 'sam', root / 'complete'
|
||||
(sam / 'masks').mkdir(parents=True)
|
||||
source = np.zeros((48, 64, 4), np.uint8)
|
||||
specs = [('shell', [16, 12, 30, 24], [40, 70, 130, 255]),
|
||||
('link_z', [30, 22, 40, 32], [160, 70, 10, 255]),
|
||||
('tip_q', [40, 28, 45, 36], [80, 170, 30, 255]),
|
||||
('decal', [19, 15, 22, 18], [200, 50, 130, 255])]
|
||||
for _, (x0, y0, x1, y1), color in specs:
|
||||
source[y0:y1, x0:x1] = color
|
||||
source[12, 16, 3] = 90
|
||||
source[25, 33] = [230, 140, 40, 255]
|
||||
Image.fromarray(source).save(sam / 'source.png')
|
||||
|
||||
def part(name, parent, role, pivot, box, hidden=False, needs=False, occluded=None):
|
||||
return {'id': name, 'semantic': name, 'parent': parent, 'rig_role': role,
|
||||
'bind_to': parent if role != 'bone_part' else None,
|
||||
'pivot_norm': [pivot[0] / 64, pivot[1] / 48],
|
||||
'visibility': 'hidden' if hidden else 'visible',
|
||||
'bbox_norm_xyxy': None if hidden else [box[0]/64, box[1]/48, box[2]/64, box[3]/48],
|
||||
'positive_points_norm': [], 'negative_points_norm': [],
|
||||
'occluded_by': occluded or [], 'confidence': .9,
|
||||
'needs_completion': hidden or needs,
|
||||
'completion': {'occlusion_fill': hidden, 'joint_extension_parent': needs,
|
||||
'joint_extension_child': False}}
|
||||
|
||||
parts = [part('tip_q', 'link_z', 'bone_part', [40, 29], specs[2][1], needs=True),
|
||||
part('decal', 'hub', 'attachment', [19, 15], specs[3][1]),
|
||||
part('shell', 'hub', 'attachment', [20, 16], specs[0][1], occluded=['decal']),
|
||||
part('link_z', 'hub', 'bone_part', [31, 23], specs[1][1], needs=True),
|
||||
part('hub', None, 'bone_part', [21, 17], None, hidden=True, occluded=['shell'])]
|
||||
write_json(sam / 'semantic-parts.json', {'schema_version': 2, 'coordinate_space': 'normalized_xy_top_left',
|
||||
'parts': parts, 'notes': []})
|
||||
records = []
|
||||
for name, (x0, y0, x1, y1), _ in specs:
|
||||
mask = np.zeros(source.shape[:2], np.uint8)
|
||||
mask[y0:y1, x0:x1] = 255
|
||||
Image.fromarray(mask).save(sam / 'masks' / f'{name}.png')
|
||||
records.append({'id': name, 'mask': f'masks/{name}.png', 'crop_xyxy': [x0, y0, x1, y1]})
|
||||
write_json(sam / 'segmentation.json', {'schema_version': 1, 'parts': records})
|
||||
shutil.copytree(sam, completion)
|
||||
(completion / 'parts').mkdir()
|
||||
# Add pixels to the left of link_z; the original visible pixels are exact.
|
||||
candidate = np.zeros((10, 14, 4), np.uint8)
|
||||
candidate[:, :4] = [160, 70, 10, 255]
|
||||
candidate[:, 4:] = source[22:32, 30:40]
|
||||
Image.fromarray(candidate).save(completion / 'parts' / 'link_z.png')
|
||||
write_json(completion / 'completion-result.json', {'parts': [
|
||||
{'id': 'link_z', 'status': 'geometry_passed_needs_review', 'selected_file': 'tasks/link_z/merged-1.png',
|
||||
'qa': {'accepted': True}, 'image': 'parts/link_z.png', 'crop_xyxy': [26, 22, 40, 32]},
|
||||
{'id': 'tip_q', 'status': 'rejected', 'qa': {'accepted': False, 'reasons': ['VISIBLE_ALIGNMENT_MISMATCH']}}
|
||||
], 'skipped': [{'id': 'hub', 'reason': 'NO_VISIBLE_SAM_MASK'},
|
||||
{'id': 'shell', 'reason': 'NO_COMPLETION_NEEDED'},
|
||||
{'id': 'decal', 'reason': 'NO_COMPLETION_NEEDED'}]})
|
||||
|
||||
|
||||
class AssemblyTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
make_fixture(self.root)
|
||||
self.sam, self.complete = self.root / 'sam', self.root / 'complete'
|
||||
|
||||
def load(self, **kwargs):
|
||||
return load_assets(self.sam, self.complete, **kwargs)
|
||||
|
||||
def result(self):
|
||||
return json.loads((self.complete / 'completion-result.json').read_text())
|
||||
|
||||
def test_pose_coordinates_hidden_hierarchy_and_attachment_binding(self):
|
||||
data = self.load()
|
||||
result = export_assets(data, self.root / 'out')
|
||||
self.assertEqual((result['parts'], result['bones'], result['completion_parts'], result['fallback_parts']), (4, 4, 1, 1))
|
||||
skeleton = build_skeleton(data[1], data[2])
|
||||
self.assertEqual([b['name'] for b in skeleton['bones']], ['root', 'hub', 'link_z', 'tip_q'])
|
||||
bones = {b['name']: b for b in skeleton['bones']}
|
||||
self.assertAlmostEqual(bones['hub']['x'], -11)
|
||||
self.assertAlmostEqual(bones['hub']['y'], 31)
|
||||
self.assertAlmostEqual(bones['link_z']['x'], 10)
|
||||
self.assertAlmostEqual(bones['link_z']['y'], -6)
|
||||
self.assertAlmostEqual(bones['tip_q']['x'], 9)
|
||||
self.assertAlmostEqual(bones['tip_q']['y'], -6)
|
||||
slots = {s['name']: s for s in skeleton['slots']}
|
||||
self.assertEqual(slots['shell']['bone'], 'hub')
|
||||
self.assertNotIn('hub', slots)
|
||||
self.assertLess(data[2]['draw_order'].index('shell'), data[2]['draw_order'].index('decal'))
|
||||
# Independent forward kinematics reconstructs each crop centre exactly.
|
||||
world = {'root': (0, 0)}
|
||||
for bone in skeleton['bones'][1:]:
|
||||
px, py = world[bone['parent']]
|
||||
world[bone['name']] = px + bone['x'], py + bone['y']
|
||||
for name, (_, box) in data[3].items():
|
||||
att = skeleton['skins'][0]['attachments'][name][name]
|
||||
bx, by = world[slots[name]['bone']]
|
||||
self.assertAlmostEqual(att['x'] + bx + 32, (box[0] + box[2]) / 2)
|
||||
self.assertAlmostEqual(48 - att['y'] - by, (box[1] + box[3]) / 2)
|
||||
image = np.array(Image.open(self.root / 'out' / 'assembled.png'))
|
||||
np.testing.assert_array_equal(image[12, 16], [40, 70, 130, 90])
|
||||
self.assertEqual(skeleton['animations']['check__link_z']['bones']['link_z']['rotate'][1]['value'], 10)
|
||||
|
||||
def test_rejection_fallback_and_strict_policy(self):
|
||||
data = self.load()
|
||||
assets = {p['id']: p for p in data[2]['parts']}
|
||||
self.assertEqual(assets['tip_q']['source'], 'sam')
|
||||
self.assertEqual(assets['tip_q']['reason'], 'COMPLETION_REJECTED')
|
||||
with self.assertRaisesRegex(ValueError, 'Completion required for tip_q'):
|
||||
self.load(fallback_policy='error')
|
||||
only_sam = load_assets(self.sam)
|
||||
self.assertTrue(all(p['source'] in ['sam', 'hidden'] for p in only_sam[2]['parts']))
|
||||
|
||||
def test_mixed_runs_and_changed_masks_rejected(self):
|
||||
mask = self.complete / 'masks' / 'tip_q.png'
|
||||
Image.new('L', (64, 48), 255).save(mask)
|
||||
with self.assertRaisesRegex(ValueError, 'Completion mask mismatch'):
|
||||
self.load()
|
||||
shutil.copy2(self.sam / 'masks' / 'tip_q.png', mask)
|
||||
(self.complete / 'semantic-parts.json').write_text('{}')
|
||||
with self.assertRaisesRegex(ValueError, 'Completion input mismatch'):
|
||||
self.load()
|
||||
|
||||
def test_bad_crop_alignment_path_and_missing_visible_parts_rejected(self):
|
||||
original = self.result()
|
||||
for box in [[26, 22, 39, 32], [-1, 22, 13, 32], [27, 22, 41, 32]]:
|
||||
result = copy.deepcopy(original)
|
||||
result['parts'][0]['crop_xyxy'] = box
|
||||
write_json(self.complete / 'completion-result.json', result)
|
||||
with self.assertRaises(ValueError):
|
||||
self.load()
|
||||
result = copy.deepcopy(original)
|
||||
result['parts'][0]['image'] = '../sam/source.png'
|
||||
write_json(self.complete / 'completion-result.json', result)
|
||||
with self.assertRaisesRegex(ValueError, 'outside input'):
|
||||
self.load()
|
||||
seg = json.loads((self.sam / 'segmentation.json').read_text())
|
||||
seg['parts'] = seg['parts'][1:]
|
||||
write_json(self.sam / 'segmentation.json', seg)
|
||||
with self.assertRaisesRegex(ValueError, 'Visible part has no SAM mask'):
|
||||
load_assets(self.sam)
|
||||
|
||||
def test_draw_order_override_and_unconstrained_overlap(self):
|
||||
data = self.load()
|
||||
self.assertTrue(data[2]['unconstrained_overlaps'])
|
||||
override = self.root / 'order.json'
|
||||
order = ['decal', 'shell', 'link_z', 'tip_q']
|
||||
write_json(override, {'draw_order': order})
|
||||
modified = self.load(draw_order_file=override)[2]
|
||||
self.assertEqual(modified['draw_order'], order)
|
||||
self.assertIn(['shell', 'decal'], modified['overridden_occlusion_constraints'])
|
||||
write_json(override, {'draw_order': ['shell'] * 4})
|
||||
with self.assertRaisesRegex(ValueError, 'exactly once'):
|
||||
self.load(draw_order_file=override)
|
||||
|
||||
def test_attachment_only_and_root_binding(self):
|
||||
plan = {'parts': [{'id': 'panel', 'rig_role': 'attachment', 'bind_to': None}]}
|
||||
assembly = {'width': 64, 'height': 48, 'draw_order': ['panel'],
|
||||
'parts': [{'id': 'panel', 'crop_xyxy': [1, 2, 10, 12]}]}
|
||||
skeleton = build_skeleton(plan, assembly)
|
||||
self.assertEqual(skeleton['bones'], [{'name': 'root'}])
|
||||
self.assertEqual(skeleton['slots'][0]['bone'], 'root')
|
||||
self.assertEqual(skeleton['animations'], {'setup': {}})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
+23
-2
@@ -7,12 +7,14 @@ import {runUnderstanding, runCompletionProbe} from '../stages/model-stage.mjs';
|
||||
import {readImage} from '../lib/images.mjs';
|
||||
import {runSegmentation} from '../stages/segmentation-stage.mjs';
|
||||
import {runCompletion} from '../stages/completion-stage.mjs';
|
||||
import {runSpine} from '../stages/spine-stage.mjs';
|
||||
|
||||
const help = `模型接入与结构理解（新增模式，不调用旧拆件流程）
|
||||
node pipeline.mjs --mode annotated-parts --reference image.png --out-dir output/run --stop-after understand
|
||||
node tools/model-check.mjs --stage completion --image local.png --prompt-file task.txt --out-dir output/edit
|
||||
node tools/model-check.mjs --stage segmentation --input-dir output/understand --out-dir output/segment --sam-dir F:/Study/SAM3
|
||||
node tools/model-check.mjs --stage completion --input-dir output/segment --out-dir output/complete --part head --prepare-only
|
||||
node tools/model-check.mjs --stage spine --input-dir output/segment --completion-dir output/complete --out-dir output/spine
|
||||
node tools/model-check.mjs --list-models
|
||||
node tools/model-check.mjs --dry-run
|
||||
|
||||
@@ -28,6 +30,11 @@ const help = `模型接入与结构理解（新增模式，不调用旧拆件流
|
||||
--parts-file FILE 离线读取已有 v1/v2 计划，做 Alpha 校验与种子生成，不请求模型
|
||||
--python FILE ai-spine Conda 环境解释器（默认使用已激活环境）
|
||||
--input-dir DIR SAM 阶段读理解目录；补全阶段读 SAM 目录
|
||||
--completion-dir DIR Spine 阶段可选补全目录；input-dir 指向同批 SAM 目录
|
||||
--fallback-policy sam|error Spine 未完成补全时回退 SAM（默认）或报错
|
||||
--draw-order-file FILE Spine 绘制顺序覆盖：{"draw_order":[全部可绘制ID，从后到前]}
|
||||
--check-angle FLOAT Spine 单骨骼检查动画角度，默认 10，范围 (0,45]
|
||||
--genielabs-dir DIR 外部 spine-animation-ai 路径，仅用于 Spine 图集和播放器
|
||||
--part ID 补全部件 ID，可重复；默认处理所有需补全部件
|
||||
--prepare-only 仅准备补全画布、掩码和提示词，不请求模型
|
||||
--candidates-dir DIR 离线导入 ID.png，或旧补全目录的首张候选，不请求模型
|
||||
@@ -44,7 +51,7 @@ const help = `模型接入与结构理解（新增模式，不调用旧拆件流
|
||||
--dry-run 检查配置与本地输入，不请求模型、不生成产物
|
||||
|
||||
--stop-after 支持 understand / segment。SAM 在临时 GPU 子进程运行，完成后退出。
|
||||
部件补全使用 --stage completion --input-dir；可见像素恢复与几何检查已接入，Spine 导出尚未接入。`;
|
||||
部件补全使用 --stage completion --input-dir；--stage spine 离线组装并导出 Spine 4.2，资产和动作质量仍需复核。`;
|
||||
|
||||
export function parseOptions(argv) {
|
||||
const options = {stage: 'understanding', imagePaths: [], partIds: []};
|
||||
@@ -58,6 +65,8 @@ export function parseOptions(argv) {
|
||||
['--sam-checkpoint', 'samCheckpoint'], ['--sam-confidence', 'samConfidence'], ['--sam-precision', 'samPrecision'],
|
||||
['--candidates-dir', 'candidatesDir'], ['--completion-size', 'completionSize'], ['--extension-radius', 'extensionRadius'],
|
||||
['--candidate-size-policy', 'candidateSizePolicy'],
|
||||
['--completion-dir', 'completionDir'], ['--fallback-policy', 'fallbackPolicy'],
|
||||
['--draw-order-file', 'drawOrderFile'], ['--check-angle', 'checkAngle'], ['--genielabs-dir', 'genielabsDir'],
|
||||
]);
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
const flag = argv[i];
|
||||
@@ -80,7 +89,19 @@ export function parseOptions(argv) {
|
||||
export async function main(argv = process.argv.slice(2)) {
|
||||
const options = parseOptions(argv);
|
||||
if (options.help) { console.log(help); return; }
|
||||
if (!['understanding', 'completion', 'segmentation'].includes(options.stage)) throw new Error('--stage 仅支持 understanding / completion / segmentation');
|
||||
if (!['understanding', 'completion', 'segmentation', 'spine'].includes(options.stage)) throw new Error('--stage 仅支持 understanding / completion / segmentation / spine');
|
||||
if (options.stage === 'spine') {
|
||||
const allowed = new Set(['stage', 'inputDir', 'outDir', 'completionDir', 'fallbackPolicy', 'drawOrderFile',
|
||||
'checkAngle', 'genielabsDir', 'python', 'resume', 'dryRun', 'mode', 'imagePaths', 'partIds']);
|
||||
if (Object.keys(options).some((key) => !allowed.has(key)) || options.imagePaths.length || options.partIds.length ||
|
||||
(options.mode && options.mode !== 'annotated-parts')) throw new Error('Spine 为独立离线阶段，不接受模型配置、请求或补全参数');
|
||||
const result = await runSpine(options);
|
||||
if (!result.dryRun) console.log(`Spine ${result.reused ? '复用完成' : '导出完成，需复核'}：${path.join(result.out, 'report.html')}`);
|
||||
return result;
|
||||
}
|
||||
if (['completionDir', 'fallbackPolicy', 'drawOrderFile', 'checkAngle', 'genielabsDir'].some((key) => options[key] !== undefined)) {
|
||||
throw new Error('Spine 导出参数仅用于 --stage spine');
|
||||
}
|
||||
if (options.partsFile && (options.stage !== 'understanding' || options.listModels)) throw new Error('--parts-file 仅用于离线结构理解');
|
||||
if (options.streamRaw !== undefined && !['true', 'false'].includes(options.streamRaw)) throw new Error('--stream 仅支持 true 或 false');
|
||||
options.stream = options.streamRaw === undefined ? undefined : options.streamRaw === 'true';
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
// Optional, offline compatibility check using a caller-provided official Spine 4.2 core IIFE.
|
||||
// node tools/verify-spine-runtime.mjs --runtime-file /path/spine-core.js --input-dir output/spine
|
||||
import assert from 'node:assert/strict';
|
||||
import {readFileSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import vm from 'node:vm';
|
||||
import {readImage} from '../lib/images.mjs';
|
||||
|
||||
const options = {};
|
||||
for (let i = 2; i < process.argv.length; i += 2) {
|
||||
const flag = process.argv[i];
|
||||
if (!['--runtime-file', '--input-dir'].includes(flag) || !process.argv[i + 1]) {
|
||||
throw new Error('Usage: node tools/verify-spine-runtime.mjs --runtime-file spine-core.js --input-dir output/spine');
|
||||
}
|
||||
options[flag] = process.argv[i + 1];
|
||||
}
|
||||
if (!options['--runtime-file'] || !options['--input-dir']) throw new Error('Both --runtime-file and --input-dir are required');
|
||||
const context = vm.createContext({console});
|
||||
vm.runInContext(readFileSync(options['--runtime-file'], 'utf8'), context, {timeout: 10000});
|
||||
const spine = context.spine;
|
||||
const input = path.resolve(options['--input-dir']);
|
||||
const assembly = JSON.parse(readFileSync(path.join(input, 'assembly.json')));
|
||||
const json = JSON.parse(readFileSync(path.join(input, 'skeleton.json')));
|
||||
assert.match(json.skeleton.spine, /^4\.2\./u);
|
||||
const atlas = new spine.TextureAtlas(readFileSync(path.join(input, 'atlas/skeleton.atlas'), 'utf8'));
|
||||
for (const page of atlas.pages) {
|
||||
const png = readImage(path.join(input, 'atlas', page.name), {pngOnly: true});
|
||||
assert.equal(page.width, png.width);
|
||||
assert.equal(page.height, png.height);
|
||||
// CPU geometry needs region metadata, not a WebGL texture upload.
|
||||
page.setTexture({setFilters() {}, setWraps() {}, getImage() { return {width: png.width, height: png.height}; }});
|
||||
}
|
||||
const data = new spine.SkeletonJson(new spine.AtlasAttachmentLoader(atlas)).readSkeletonData(json);
|
||||
const skeleton = new spine.Skeleton(data);
|
||||
function vertices(slot) {
|
||||
const values = new Float32Array(8);
|
||||
slot.getAttachment().computeWorldVertices(slot, values, 0, 2);
|
||||
assert.ok([...values].every(Number.isFinite));
|
||||
return values;
|
||||
}
|
||||
function close(actual, expected, label) {
|
||||
assert.ok(Math.abs(actual - expected) < .002, `${label}: ${actual} != ${expected}`);
|
||||
}
|
||||
skeleton.setToSetupPose();
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
const originals = new Map();
|
||||
const assets = new Map(assembly.parts.map((p) => [p.id, p]));
|
||||
for (const slot of skeleton.slots) {
|
||||
const values = vertices(slot);
|
||||
originals.set(slot.data.name, values);
|
||||
const [x0, y0, x1, y1] = assets.get(slot.data.name).crop_xyxy;
|
||||
const xs = [values[0], values[2], values[4], values[6]];
|
||||
const ys = [values[1], values[3], values[5], values[7]];
|
||||
close(Math.min(...xs), x0 - assembly.width / 2, 'setup x0');
|
||||
close(Math.max(...xs), x1 - assembly.width / 2, 'setup x1');
|
||||
close(Math.min(...ys), assembly.height - y1, 'setup y0');
|
||||
close(Math.max(...ys), assembly.height - y0, 'setup y1');
|
||||
const region = atlas.findRegion(slot.data.name);
|
||||
assert.equal(region.originalWidth, x1 - x0);
|
||||
assert.equal(region.originalHeight, y1 - y0);
|
||||
}
|
||||
let samples = 0;
|
||||
for (const animation of data.animations) {
|
||||
if (animation.name === 'setup') continue;
|
||||
const boneName = animation.name.replace(/^check__/u, '');
|
||||
const frames = json.animations[animation.name].bones[boneName].rotate;
|
||||
for (const time of [.5, 1.5]) {
|
||||
skeleton.setToSetupPose();
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
const bone = skeleton.findBone(boneName);
|
||||
const px = bone.worldX, py = bone.worldY;
|
||||
const angle = frames.find((f) => f.time === time).value * Math.PI / 180;
|
||||
animation.apply(skeleton, 0, time, false, [], 1, spine.MixBlend.replace, spine.MixDirection.mixIn);
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
for (const slot of skeleton.slots) {
|
||||
let parent = slot.bone;
|
||||
while (parent && parent !== bone) parent = parent.parent;
|
||||
const before = originals.get(slot.data.name), after = vertices(slot);
|
||||
for (let i = 0; i < 8; i += 2) {
|
||||
const x = before[i] - px, y = before[i + 1] - py;
|
||||
close(after[i], parent ? px + x * Math.cos(angle) - y * Math.sin(angle) : before[i], 'rotated x');
|
||||
close(after[i + 1], parent ? py + x * Math.sin(angle) + y * Math.cos(angle) : before[i + 1], 'rotated y');
|
||||
}
|
||||
}
|
||||
samples++;
|
||||
}
|
||||
}
|
||||
console.log(JSON.stringify({input, spine_version: data.version, bones: data.bones.length,
|
||||
slots: data.slots.length, animations: data.animations.length, rotation_samples: samples,
|
||||
setup_geometry_verified: true, descendant_rotation_verified: true, browser_render_verified: false}, null, 2));
|
||||
Reference in New Issue
Block a user