Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| b3b51a48cf | |||
| 00f476022d | |||
| 8080d52a99 |
@@ -16,3 +16,6 @@ references/external/kenney-platformer/
|
||||
|
||||
# OS/editor noise
|
||||
.DS_Store
|
||||
|
||||
#data
|
||||
data/
|
||||
@@ -20,11 +20,11 @@
|
||||
|
||||
## 跨会话上下文与阅读顺序
|
||||
|
||||
本节于 2026-09-21 根据当前代码与已检查产物整理。它是后续对话的上下文入口;后续修改应同步更新对应状态,实际源码、配置和具体实验 manifest 优先于历史总结。不要把旧实验统计套到新输出上。
|
||||
本节于 2026-09-22 接入通用第四阶段及 adaptive 补全后更新。它是后续对话的上下文入口;后续修改应同步更新对应状态,实际源码、配置和具体实验 manifest 优先于历史总结。不要把旧实验统计套到新输出上。
|
||||
|
||||
1. 先阅读本文件和 `git status --short`,保留用户已有修改。
|
||||
2. 阅读 [README](README.md) 了解项目、环境、配置与命令。
|
||||
3. 按任务阅读 [第一阶段](docs/1-understanding-stage.md)、[第二阶段](docs/2-segmentation-stage.md)、[第三阶段](docs/3-completion-stage.md),再核对相关实现与实际产物。
|
||||
3. 按任务阅读 [第一阶段](docs/1-understanding-stage.md)、[第二阶段](docs/2-segmentation-stage.md)、[第三阶段](docs/3-completion-stage.md)、[第四阶段](docs/4-spine-stage.md),再核对相关实现与实际产物。
|
||||
4. 用户说“继续”时,先核对工作区和最近任务进展,不能只凭历史总结重复请求付费模型或覆盖实验。
|
||||
|
||||
## 用户目标与固定偏好
|
||||
@@ -37,8 +37,8 @@
|
||||
|
||||
## 文档维护约定
|
||||
|
||||
- 各文档文件名保持不变。`docs/` 当前保留三份阶段文档;`docs/model-stages.md` 在本次文档整理前已处于用户删除状态,不自行恢复,相关说明归入 README 和阶段文档。
|
||||
- 三份阶段文档有且仅有以下八个二级章节,细节必须归入对应章节:
|
||||
- 原有文档文件名保持不变。`docs/` 保留前三份阶段文档,并新增 `4-spine-stage.md`;`docs/model-stages.md` 在此前文档整理前已处于用户删除状态,不自行恢复,相关说明归入 README 和阶段文档。
|
||||
- 四份阶段文档有且仅有以下八个二级章节,细节必须归入对应章节:
|
||||
1. 阶段简介
|
||||
2. 输入、输出与执行链路
|
||||
3. 阶段执行命令示例及参数说明
|
||||
@@ -63,9 +63,9 @@
|
||||
- `configs/tiantoken.json`:独立 TIANTOKEN_BASE_URL / TIANTOKEN_API_KEY,地址 https://api.tiantoken.com;理解 gpt-6-astra,补全 gpt-image-2;模型变量为 TIANTOKEN_UNDERSTANDING_MODEL / TIANTOKEN_COMPLETION_MODEL。
|
||||
- CLI 模型 > stage.model > 对应模型环境变量 > default_model;shell 同名变量优先于 .env。模型 ID 原样发送,不加隐式别名。
|
||||
- 用户提供的密钥仅留在本地忽略的 `.env`;禁止在文档、命令日志、聊天或提交中暴露,不要读取/打印完整 .env 来诊断。
|
||||
- 三份配置的图片编辑能力均为 multiple_images=false、mask=false。接口已预留分支不代表中转已支持;开启前需验证。Astra 理解接口与图片编辑接口不同,当前没有 Astra Responses 图片生成工具链。
|
||||
- 三份配置的图片编辑能力仍为 multiple_images=false、mask=false。adaptive auto 在此配置下发送单图参考板,包含局部和完整角色参考,mask 仅本地约束。独立多图/mask 和单部件探针已接入,但本轮未调用付费模型,不能宣称中转已支持。Astra 理解接口与图片编辑接口不同,当前没有 Astra Responses 图片生成工具链。
|
||||
|
||||
## 三阶段实现与不能丢失的边界
|
||||
## 四阶段实现与不能丢失的边界
|
||||
|
||||
### 第一阶段:理解与几何候选
|
||||
|
||||
@@ -87,19 +87,34 @@
|
||||
|
||||
### 第三阶段:受限部件补全
|
||||
|
||||
- 入口 `--stage completion --input-dir SAM目录`;实现为 stages/completion-stage.mjs、completion_geometry.py、providers/image-edits.mjs。
|
||||
- 入口 `--stage completion --input-dir SAM目录`;实现为 stages/completion-stage.mjs、completion_geometry.py、completion_planning.py、providers/image-edits.mjs。默认 completion-profile=adaptive,legacy 保留旧行为和候选兼容。
|
||||
- 输入须含 source.png、v2 计划、segmentation.json 及所有记录的全画布 masks;单独 parts/ 不够。
|
||||
- 仅处理 needs_completion=true、非隐藏、有可见 mask 且允许区域非空的部件。--part 不绕过筛选;目前不重新评估 SAM 缺损。
|
||||
- 默认 1024 方形局部画布,原像素不缩放。允许区来自轮廓外 32 像素范围与遮挡 mask / pivot 圆盘交集,排除可见 mask;不是完整隐藏形状生成。
|
||||
- 当前真实请求只发白底 input.png;context.png、edit-mask.png 默认不发送。开启 multiple_images 也只是发送同坐标局部 context,并非独立全角色参考。
|
||||
- adaptive 从实际 mask 重建遮挡/关节、闭合漏分孔洞、附件底图清理需求,不单凭 needs_completion 筛选。自动孔洞只取原图前景且无其他 mask 归属的区域;开放缺口和歧义归属仍需复核。无明确遮挡的近重复 mask(IoU≥0.8)要求人工确认。
|
||||
- adaptive preserve=SAM−editable−removed;仅保留区锁定原 RGBA,清理和替换区可重绘,人工 remove 可删除。completion-review JSON(schema_version=1、parts 映射、confirmed=true)支持 repair/replace/remove 全画布二值 masks 与部件半径,原文件及哈希留档,不改 SAM。
|
||||
- 默认请求边长 1024;adaptive 半径按 mask 短边×0.15 向上取整并限于 4~96 原像素,CLI 或部件 review 可覆盖。native 裁剪边长约目标包围框最长边×1.15;小部件放大、大部件缩小,保存正反仿射和工作区。逆缩放后恢复保留原像素,坐标可供第四阶段使用。
|
||||
- reference auto:multiple_images=true 时 input→context→reference,默认 false 时发送 board(左目标,右局部和全角色参考),参考面板不进入最终资产。支持 none/local/full/both/board;mask auto/required/off 独立选择。request-plan.json 记录计划,request.json 仅实际发送时生成,含有序图片角色、哈希、尺寸、字段、mask 哈希与透明语义。
|
||||
- legacy 才沿用 needs_completion、原像素不缩放、默认 32 半径和原可见像素全保护。旧候选复检必须显式 --completion-profile legacy。新旧画布/参考布局/复核文件不同不能混用候选。
|
||||
- 图片解码已修复“空 b64_json 但有有效 URL”情况,回退下载 URL;该问题与画布尺寸不符不同。
|
||||
- 在线请求现显式传 size,配置冲突失败。strict 默认拒绝尺寸不同;uniform 仅恢复同宽高比整画布尺寸,保留原候选,继续原有验收,不单独移动/缩放物体。
|
||||
- 验收阈值:可见不透明像素 RGB MAE≤30、保留率≥95%;有效新增须在允许区且连接可见部件;原可见 RGBA 随后逐像素恢复。
|
||||
- unfulfilled_regions 当前只记录、不硬拒绝;语义错误、接缝、关节补够与否尚无可靠验收。错误 SAM 可见像素会被强制保留。
|
||||
- 对齐阈值保持 RGB MAE≤30、保留率≥95%;adaptive 在请求工作区的保留不透明像素上检查,再逆采样,避免把插值 Alpha 误作位移。编辑像素必须位于允许区且连接保留区;可见缺口使用原图纹理,移除区清零。
|
||||
- adaptive 清理/人工替换要求≥95% 有效填充,其余 unfulfilled_regions 仍仅诊断。语义、接缝和关节质量不能由填充率证明。未明确编辑的误分仍被保护,需复核文件修正。
|
||||
- --probe-edit-capabilities 要求 adaptive 在线且恰好一个 --part,发送主图/局部/全角色+mask,n=1、maxRetries=0。记录 capability-evidence 和合并前 raw_outside_edit 指标;不会把 reference_use_verified/mask_semantics_verified 自动置 true 或修改配置。prepare-only/mock 不是中转实测。
|
||||
- prepare-only 不联网;candidates-dir 可读平铺 ID.png 或匹配 manifest 的旧任务首张候选。--resume 不是失败断点续发,准备/生成/复检使用独立目录。
|
||||
- parts/ 只含通过检查的补全部件,不包含全部原拆件。完整角色组装、旋转验收、新管线 Spine 导出尚未接入。
|
||||
- parts/ 只含通过检查的补全部件,不包含全部原拆件。完整角色组装、Spine 导出与旋转检查动画已独立接入第四阶段;运动质量自动验收仍未实现。
|
||||
- 旧 `--stage completion --image ... --prompt-file ...` 是接口探针,不能和 input-dir 混用,不执行完整几何验收。
|
||||
|
||||
### 第四阶段:通用组装与 Spine 导出
|
||||
|
||||
- 用户明确要求可复用管线,不能写死具体角色、部件 ID、数量或人形比例。入口 `--stage spine --input-dir SAM目录 --completion-dir 补全目录 --out-dir 新目录`;completion-dir 可省略。
|
||||
- `stages/spine-stage.mjs` 与 `spine_assembly.py` 完全离线,校验 v2、输入快照及 masks 哈希、PNG 和裁剪坐标;不读取模型配置/密钥,不调用 SAM。
|
||||
- 原资产从 source + mask 重建,采用补全结果前检查 selected_file、状态、qa.accepted、image、crop_xyxy。legacy 全可见 RGBA 保持;adaptive 验证 preservation 分区、保留 RGBA 精确匹配、编辑/移除互斥及输出不越界,契约 mask 哈希留档。不能混用其他实验。
|
||||
- 默认补全失败回退 SAM 并记录原因;fallback-policy=error 拒绝需补全但无通过结果的可见部件。缺可见 mask 直接报错;隐藏骨骼保留但不生成图片。
|
||||
- pivot 转换为底边中点原点的 Spine 坐标;骨骼 setup 旋转为零,局部位置取父子 pivot 差;附件中心相对绑定骨骼偏移。bone_part 使用 parent,其他角色使用 bind_to。
|
||||
- occluded_by 稳定拓扑排序;无约束重叠写入诊断,可用 draw-order-file 全量覆盖,冲突留档。骨骼/遮挡环路拒绝。
|
||||
- 导出完整 images、assembly.json、静态组装/对比/覆盖诊断、skeleton.json、atlas、report.html 和真实 preview.html。依赖外部只读图集和播放器脚本,其哈希纳入复用指纹。
|
||||
- 目标 Spine 4.2;setup 静态姿态及 check__ID 单骨骼检查动画。没有成品 idle/walk、mesh/weights/IK,semantic_verified=false、quality_status=needs_review;几何通过和可加载不代表资产合格。
|
||||
- 生成离线;播放器从 CDN 加载资源,打开骨架预览需要网络。静态报告可离线查看。
|
||||
|
||||
## 已检查实验与质量结论
|
||||
|
||||
以下为已有实验记录,不代表后续每次运行的结果;不必为了文档修改重跑 GPU 或付费模型。
|
||||
@@ -108,30 +123,37 @@
|
||||
| --- | --- |
|
||||
| output/sombrero-grounding-v2 | 旧计划离线几何检查,28 个程序候选通过前景检查;未证明语义正确 |
|
||||
| output/sombrero-sam31-v3 | 较早 SAM 实验,28 个 mask,约 67.18% 覆盖率;肢体/鞋子重复、漏分,不合格 |
|
||||
| output/20260918-183532/sam | 当前讨论基线:32 个计划项、10 个 hidden、22 个 mask;约 83.3% 覆盖,32719 像素未分配;前臂/手混入衣料、袖子与上衣缺损 |
|
||||
| output/20260918-183532/sam | 较早基线:32 个计划项、10 个 hidden、22 个 mask;约 83.3% 覆盖,32719 像素未分配;前臂/手混入衣料、袖子与上衣缺损 |
|
||||
| output/20260918-183532/complete2_gpt | 真实 Tiantoken gpt-image-2 请求 12 部件;旧请求未传 size,返回 1248/1254,预期 1024,全部 CANVAS_SIZE_MISMATCH |
|
||||
| output/20260918-183532/complete2_gpt-size-recheck-v2 | 离线 uniform 复检,未重生成:8 个几何通过;head、pants_thigh_l、pants_thigh_r、shoe_l 4 个拒绝;通过项仍有误分和未完成区域 |
|
||||
| output/20260918-183532/complete-recovered-head-check | 更早 Qwen 头部候选恢复检查,不能与 GPT 批次混为一谈 |
|
||||
| output/20260920-225137 | 较新实验:27 个语义项、2 个 hidden、25 个 SAM 部件;补全 16 项中 13 个几何通过、3 个对齐拒绝 |
|
||||
| output/spine-validation-latest-20260922 | 第四阶段离线验证:25 个 slot、16 个骨骼(含 root);采用 13 个补全部件,5 个需补全部件回退 SAM(含两袖无允许区域);上游缺口仍存在 |
|
||||
| output/spine-validation-baseline-20260922 | 另一批输入验证:22 个 slot、16 个骨骼(含 root);采用 8 个补全部件、4 个回退;输入源为旧基线和 size-recheck-v2 |
|
||||
| output/completion-adaptive-prepare-20260922 | adaptive 离线准备:16 个任务、11 个跳过;上衣规划 3708 个封闭漏分像素,头部规划 10205 个附件清理像素;这些是待复核区域,不是补全成功结果,未请求模型 |
|
||||
|
||||
- 当前 22→12 的原因:sombrero、sleeve_l、sleeve_r、eye_l、eye_r、eyebrow_l、eyebrow_r、nose、mouth、mustache 共 10 个被第一阶段标记 NO_COMPLETION_NEEDED;不能解释为它们的实际 SAM mask 已完整。
|
||||
- 2026-09-20 尺寸修复后曾通过 22 项 Node 测试(含 Python 补全回归)。这是历史验证;新改动按范围重新验证,不自动沿用“全部通过”。显式 size 构造已测试,修复后中转是否遵守尚无新实测。
|
||||
- 总体判断:上游部件归属、模型缺少原图参考与遮罩、区域规则粗糙、验收不足共同导致低质量。不能仅换模型、强制处理全部 mask、放宽阈值或解决尺寸错误就宣称修好。
|
||||
- 2026-09-22 第四阶段接入后,Conda ai-spine 下 npm test 共 26 项通过(含新增 6 项 Python 组装用例)。另使用官方 Spine 4.2 core IIFE 对上述两份导出分别验证 setup 顶点和 30 个单关节旋转采样,全部通过。最新导出的 preview.html 已在 headless Chrome 实际加载渲染,截图为 output/spine-runtime-validation-20260922/preview.png;这不是动作或资产质量验收。复检工具为 `tools/verify-spine-runtime.mjs`,需调用者提供本地官方 runtime 文件,不自动下载/安装依赖。
|
||||
|
||||
- 随后接入 adaptive 补全,Conda ai-spine 下 npm test 共 29 项全部通过(含 8 项新增 Python adaptive 用例):覆盖参考顺序/mask mock、放大/缩小、位移拒绝、实际缺口、清理与区域契约、候选身份及第四阶段 strict 回退。未进行付费中转试验,真实参考使用与 mask 语义仍待验证。
|
||||
|
||||
## 当前改进方向与后续顺序
|
||||
|
||||
本轮用户要求重整文档,以下功能是待实现需求,不因写入文档就成为已完成功能:
|
||||
当前用户已要求落实第三阶段的参考图/区域重建/自适应画布,必须保持可复用且与第四阶段贯通。当前状态:
|
||||
|
||||
1. 第二阶段输出仅色彩遮罩组成的完整角色图,保留当前原图叠色预览,并明确未分配/重叠区域。
|
||||
2. 第二阶段每个 SAM 部件可选经过一次大模型去毛边,保留原始拆件、原图参考、坐标、变更区域与前后对照。
|
||||
3. 第三阶段将角色原图或部件相关原图局部作为参考交给补全模型,验证中转多图/遮罩能力及实际发送内容。
|
||||
4. 第三阶段输出使用补全部件组装的完整角色,复用无需补全的原部件,明确失败/回退状态。
|
||||
5. 技术优先级:先修正 SAM 部件归属与输入门槛,再按实际 mask 重建补全计划;用头、前臂、上衣三类小样验证参考图和遮罩,然后增加回贴与关节旋转验收,最后批量与 Spine 导出。
|
||||
3. 第三阶段参考图、实际 mask 规划、自适应区域/可逆缩放、发送审计和单部件能力探针已实现并离线验证;真实多图/mask 语义和新布局生成质量仍待付费小样实测,默认 capabilities 不变。
|
||||
4. 完整角色组装及失败回退记录已在第四阶段实现;接缝与关节旋转质量自动验收仍待实现。
|
||||
5. 后续质量工作:复核自动规划和 SAM 归属、实测参考图与遮罩、加强接缝/动作验收。第三/四阶段遵循统一输入契约,不为某个实验硬编码修正。
|
||||
|
||||
详细改进说明保留在各阶段文档第 8 节;若后续用户改变优先级,应更新此处。
|
||||
|
||||
## 旧管线与范围识别
|
||||
|
||||
- `pipeline.mjs` 未指定 --mode annotated-parts 时走旧“生成拆件图→CV 定位→Spine”流程,不是新三阶段自动串联。
|
||||
- `pipeline.mjs` 未指定 --mode annotated-parts 时走旧“生成拆件图→CV 定位→Spine”流程,不是新四阶段自动串联。
|
||||
- `segment.py` 为旧拆件图分割;`compose_preview.py` 为旧布局组装;`run_case_matrix.py` 为旧定位对照工具。不要因为 IDE 当前打开这些文件就误改新阶段入口。
|
||||
- 相邻 `spine-animation-ai` 为外部只读依赖,保留其 PolyForm Noncommercial 许可证约束;新阶段尚未复用为完成的导出流程。
|
||||
- 相邻 `spine-animation-ai` 为外部只读依赖,保留其 PolyForm Noncommercial 许可证约束;第四阶段复用图集和播放器脚本,不复用旧特征定位或 root 平铺骨架逻辑。
|
||||
- 不要把文档改写任务扩大成付费图片生成、SAM 推理、依赖安装或功能实现;需要实现时遵从用户后续任务范围。
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
# AI Spine 角色拆件与动画管线
|
||||
|
||||
本项目探索将单张 2D 角色图转换为可用于 Spine 动画的部件与骨骼资源。当前主线分为三个阶段:视觉 LLM 理解结构、本地 SAM 分割可见部件、图片编辑模型补全隐藏区域和关节连接。
|
||||
本项目探索将单张 2D 角色图转换为可用于 Spine 动画的部件与骨骼资源。当前主线分为四个阶段:视觉 LLM 理解结构、本地 SAM 分割可见部件、图片编辑模型补全隐藏区域和关节连接、离线组装与 Spine 导出。
|
||||
|
||||
| 阶段 | 输入 → 输出 | 当前状态 |
|
||||
| --- | --- | --- |
|
||||
| 1. 结构理解 | PNG → v2 语义计划、粗框、pivot、程序候选点 | 可在线理解或离线复检;语义待复核 |
|
||||
| 2. SAM 分割 | 第一阶段目录 → mask、原像素透明部件、标注图与拆解图 | 本机 GPU 实测可运行;存在误分与漏分 |
|
||||
| 3. 部件补全 | SAM 目录 → 局部编辑候选、几何 QA、通过检查的补全部件 | 已真实调用图片模型;结果质量尚不合格 |
|
||||
| 4. Spine 导出 | SAM + 可选补全目录 → 完整资产、原位组装、骨骼、图集、预览 | 通用离线入口;含单骨骼旋转检查动画,资产与动作质量需复核 |
|
||||
|
||||
第三阶段尚未输出使用补全部件组装的完整角色,新管线 Spine 导出也未完成。执行成功、mask 覆盖率高、几何检查通过,均不等于动画资产可用。
|
||||
第三阶段输出通过几何检查的补全部件;第四阶段整合所有可绘制部件、组装并导出 Spine 4.2。执行成功、mask 覆盖率高、几何检查通过,均不等于动画资产可用。
|
||||
|
||||
仓库保留旧版“生成白底拆件图 → OpenCV 拆分 → Genielabs 定位 → Spine 导出”作为对照;它与当前三阶段主线并存,产物不能直接混用。
|
||||
仓库保留旧版“生成白底拆件图 → OpenCV 拆分 → Genielabs 定位 → Spine 导出”作为对照;它与当前四阶段主线并存,产物不能直接混用。
|
||||
|
||||
## 文档入口
|
||||
|
||||
@@ -18,6 +19,7 @@
|
||||
- [第一阶段](docs/1-understanding-stage.md):理解、数据契约、Alpha 候选与粗标注。
|
||||
- [第二阶段](docs/2-segmentation-stage.md):本地 SAM、mask 选择、拆件与质量边界。
|
||||
- [第三阶段](docs/3-completion-stage.md):筛选、局部画布、图片请求、几何验收及补全改进。
|
||||
- [第四阶段](docs/4-spine-stage.md):通用资产整合、坐标与骨骼、绘制顺序、Spine 导出与回退策略。
|
||||
|
||||
阶段文档统一采用八节:简介;输入输出与链路;命令参数;输入格式;重要代码原理;职责索引;产物;可改进点。配置与实验说明已归入上述文档。
|
||||
|
||||
@@ -101,7 +103,7 @@ SAM_CODE_DIR=F:/Study/SAM3/sam3
|
||||
|
||||
CLI > 配置 stage.model > 对应模型环境变量 > default_model;shell 环境变量优先于 `.env`。不指定配置时入口默认 astra-qwen.json,建议实验始终显式指定。模型 ID 原样发送,中转能力以实际请求为准。Tiantoken 图片编辑已真实执行;第三阶段尚未适配 Astra Responses 图片生成工具。
|
||||
|
||||
三个配置当前均关闭多图参考与编辑 mask 能力。第三阶段默认只发送白底单部件图;本地 context.png/edit-mask.png 已生成但不默认发送。改进计划见第三阶段文档。
|
||||
三个配置当前仍关闭原生多图与编辑 mask 能力。第三阶段默认 adaptive:实际 mask 重建需求、自适应半径和可逆缩放;当前配置下 auto 会发送含目标、局部参考、完整角色参考的单张参考板。只有支持原生多图时才独立发送参考图;mask 是否发送由配置和 completion-mask 决定,未发送时明确标记 local_only。不能把本地裁剪与恢复当成模型遵循 mask 的证据。
|
||||
|
||||
查询模型列表和检查配置:
|
||||
|
||||
@@ -112,7 +114,7 @@ node .\tools\model-check.mjs --config .\configs\tiantoken.json --dry-run
|
||||
|
||||
前者联网,后者不请求模型;列表存在某个 ID 不代表其所有能力已验证。
|
||||
|
||||
## 三阶段执行命令
|
||||
## 四阶段执行命令
|
||||
|
||||
以下命令按顺序在同一个 PowerShell 会话执行。反引号后不能带空格。每次实验使用新的 runId;分阶段重开终端时,将路径变量设置成已有的对应目录。
|
||||
|
||||
@@ -160,6 +162,8 @@ node .\tools\model-check.mjs --stage segmentation `
|
||||
|
||||
### 第三阶段:准备与补全
|
||||
|
||||
默认 `--completion-profile adaptive` 会重新检查实际 SAM 缺口、骨骼连接和附件重叠,即使旧计划 needs_completion=false 也可能生成修复任务。先检查 preparation 报告的区域与参考图,再请求模型;缺口归属和附件底图仍需语义复核。
|
||||
|
||||
先离线准备任务,不请求模型:
|
||||
|
||||
```powershell
|
||||
@@ -189,7 +193,9 @@ node .\tools\model-check.mjs --stage completion `
|
||||
--out-dir "$runRoot\complete"
|
||||
```
|
||||
|
||||
省略 part 不意味着强制补全全部 mask。检查 `completion-plan.json` 的跳过原因和 `completion-result.json` 的 QA;parts/ 仅包含通过检查的补全部件。所有结果仍需语义复核。
|
||||
省略 part 不意味着强制补全全部 mask。检查 `completion-plan.json` 的 assessments/跳过原因、每个任务的 input.png、request-plan.json 和编辑区域;parts/ 仅包含通过检查的补全部件。所有结果仍需语义复核。`--completion-review FILE` 可按 ID 提供修复/替换/移除区域及部件半径,格式见[第三阶段文档](docs/3-completion-stage.md)。
|
||||
|
||||
`--completion-reference auto` 默认在原生多图可用时发送主图→局部参考→全角色参考,否则使用 board;可显式选择 board/local/full/both/none。`--completion-mask auto|required|off` 控制 mask 请求,auto 遵循配置。`--probe-edit-capabilities --part ID` 是一次真实多图+mask 小样,固定 n=1、无 HTTP 重试,保存原候选及能力证据,不自动启用 capabilities。该实测会产生费用;当前实现只完成离线验证。
|
||||
|
||||
旧候选尺寸不符时,可离线导入复检,避免重新生成:
|
||||
|
||||
@@ -199,10 +205,28 @@ node .\tools\model-check.mjs --stage completion `
|
||||
--input-dir .\output\20260918-183532\sam `
|
||||
--candidates-dir .\output\20260918-183532\complete2_gpt `
|
||||
--out-dir "$runRoot\complete-size-recheck" `
|
||||
--completion-profile legacy `
|
||||
--candidate-size-policy uniform
|
||||
```
|
||||
|
||||
uniform 仅恢复同宽高比的整画布尺寸,不纠正物体位移、变形或语义。在线请求现已显式传 size;参数及中转实测边界见[第三阶段文档](docs/3-completion-stage.md)。
|
||||
旧画布候选必须指定 legacy。adaptive 候选需匹配源图、masks、profile、参考布局、半径和复核文件哈希;不能把旧候选直接映射到新画布。uniform 仅恢复同宽高比整画布尺寸,不纠正物体位移、变形或语义。在线请求显式传 size;参数及中转实测边界见[第三阶段文档](docs/3-completion-stage.md)。
|
||||
|
||||
### 第四阶段:离线组装与 Spine 导出
|
||||
|
||||
接续本轮 SAM 与补全目录,无需模型配置、密钥或 GPU:
|
||||
|
||||
```powershell
|
||||
node .\tools\model-check.mjs --stage spine `
|
||||
--input-dir "$runRoot\sam" `
|
||||
--completion-dir "$runRoot\complete" `
|
||||
--out-dir "$runRoot\spine"
|
||||
```
|
||||
|
||||
省略 `--completion-dir` 可只用 SAM 资产;加 `--dry-run` 完整检查输入而不写输出。默认补全失败回退 SAM 并记录原因;`--fallback-policy error` 要求需补全的可见部件全部有通过结果。可见部件缺 mask、混用不同输入或非法补全坐标会直接报错。
|
||||
|
||||
检查 `report.html`、`comparison.png` 和 `assembly.json`;`preview.html` 显示真实骨架,setup 为静态姿态,`check__骨骼ID` 为单关节旋转检查动画。生成过程离线;播放器页面从 CDN 加载 Spine Player,需要网络。`skeleton.json` 配合 `images/` 供编辑器导入,`atlas/` 用于运行时。
|
||||
|
||||
部件数量、名称、父子关系均由 v2 计划驱动;隐藏骨骼保留,附件按 bind_to 绑定。第四阶段只复用外部只读 `spine-animation-ai` 的图集和播放器脚本,保留其 PolyForm Noncommercial 约束。完整参数、输入契约和绘制顺序覆盖见[第四阶段文档](docs/4-spine-stage.md)。
|
||||
|
||||
## 旧管线与验证
|
||||
|
||||
@@ -218,7 +242,7 @@ node .\pipeline.mjs `
|
||||
--animations idle,walk,wave
|
||||
```
|
||||
|
||||
`segment.py` 为旧版拆件分割,`compose_preview.py` 组装预览,`run_case_matrix.py` 对照定位方案;当前 SAM 入口为 sam_segment.py。旧产物需检查 assembled.png 和 preview-rigged.html,不能据此声称新三阶段已经完成 Spine 导出。
|
||||
`segment.py` 为旧版拆件分割,`compose_preview.py` 组装预览,`run_case_matrix.py` 对照定位方案;当前 SAM 入口为 sam_segment.py。旧产物检查 assembled.png 和 preview-rigged.html;新管线导出使用独立的第四阶段入口。
|
||||
|
||||
测试与命令帮助:
|
||||
|
||||
|
||||
+21
-5
@@ -145,6 +145,9 @@ def merge_candidate(folder, candidate_file, output_file, size_policy='strict'):
|
||||
raise ValueError('Unknown candidate size policy')
|
||||
folder = Path(folder)
|
||||
task = read_json(folder/'task.json')
|
||||
if task.get('profile') == 'adaptive':
|
||||
from completion_planning import merge_adaptive
|
||||
return merge_adaptive(folder, candidate_file, output_file, size_policy)
|
||||
original = np.array(Image.open(folder/'visible.png').convert('RGBA'))
|
||||
allowed = np.array(Image.open(folder/'allowed-mask.png').convert('L')) > 0
|
||||
candidate = Image.open(candidate_file).convert('RGBA')
|
||||
@@ -228,6 +231,9 @@ def finalize(directory):
|
||||
meta = read_json(out/'tasks'/name/'task.json')
|
||||
ox, oy = meta['origin_xy']
|
||||
task.update(image=f'parts/{name}.png', crop_xyxy=[bbox[0]+ox, bbox[1]+oy, bbox[2]+ox, bbox[3]+oy])
|
||||
if meta.get('preservation'):
|
||||
task['preservation'] = meta['preservation']
|
||||
task['transform'] = meta['transform']
|
||||
view = f'<img src="{task["image"]}">' if task.get('image') else ''
|
||||
for index, item in enumerate(task.get('candidates', []), 1):
|
||||
view += f'<p><a href="{html.escape(item["file"], quote=True)}">原候选 {index}</a> · <a href="tasks/{name}/qa-{index}.json">QA</a>'
|
||||
@@ -235,11 +241,14 @@ def finalize(directory):
|
||||
view += f' · <a href="tasks/{name}/merged-{index}-normalized.png">缩放后候选</a>'
|
||||
view += '</p>'
|
||||
warnings = html.escape(', '.join(task.get('upstream_warnings', [])))
|
||||
rows.append(f'<tr><td>{html.escape(name)}<p>{warnings}</p><a href="tasks/{name}/task.json">任务参数</a> · <a href="tasks/{name}/prompt.txt">提示词</a></td><td>{html.escape(task["status"])}</td><td><a href="tasks/{name}/visible.png"><img src="tasks/{name}/visible.png"></a></td><td><a href="tasks/{name}/allowed-mask.png"><img src="tasks/{name}/allowed-mask.png"></a><p><a href="tasks/{name}/context.png">原图局部参考</a></p></td><td>{view}</td><td><pre>{html.escape(json.dumps(qa, ensure_ascii=False, indent=2))}</pre></td></tr>')
|
||||
reference = f'<p><a href="tasks/{name}/input.png">实际请求主图</a> · <a href="tasks/{name}/request-plan.json">发送计划</a></p>'
|
||||
if (out/'tasks'/name/'reference.png').exists():
|
||||
reference += f'<a href="tasks/{name}/reference.png">完整角色参考</a>'
|
||||
rows.append(f'<tr><td>{html.escape(name)}<p>{warnings}</p><a href="tasks/{name}/task.json">任务参数</a> · <a href="tasks/{name}/prompt.txt">提示词</a>{reference}</td><td>{html.escape(task["status"])}</td><td><a href="tasks/{name}/visible.png"><img src="tasks/{name}/visible.png"></a></td><td><a href="tasks/{name}/allowed-mask.png"><img src="tasks/{name}/allowed-mask.png"></a><p><a href="tasks/{name}/context.png">原图局部参考</a></p></td><td>{view}</td><td><pre>{html.escape(json.dumps(qa, ensure_ascii=False, indent=2))}</pre></td></tr>')
|
||||
write_json(out/'completion-result.json', result)
|
||||
(out/'report.html').write_text('''<!doctype html><html lang="zh-CN"><meta charset="utf-8"><title>部件补全</title>
|
||||
<style>body{font:16px system-ui;margin:24px}td,th{border:1px solid #ccc;padding:8px}img{max-width:260px;max-height:300px;background:#eee}pre{white-space:pre-wrap}table{border-collapse:collapse}</style>
|
||||
<h1>第三阶段:部件补全</h1><p>新增区域受允许掩码限制,原图可见像素逐像素恢复。几何通过仅表示候选可供复核,不代表遮挡和关节纹理已正确补全。上游 SAM 错分仍需修正。</p>
|
||||
<h1>第三阶段:部件补全</h1><p>输出受允许掩码限制,保留区域逐像素恢复;adaptive 允许明确规划的附件清理、替换和移除,原始 SAM 始终保留。几何通过仅表示候选可供复核,不代表遮挡和关节纹理已正确补全。</p>
|
||||
<table><tr><th>部件 / 上游告警</th><th>状态</th><th>原可见区域</th><th>允许新增区域(白色)</th><th>补全候选</th><th>几何检查</th></tr>'''+''.join(rows)+'</table><h2>跳过</h2><pre>'+html.escape(json.dumps(result['skipped'], ensure_ascii=False, indent=2))+'</pre></html>', encoding='utf-8')
|
||||
|
||||
|
||||
@@ -249,16 +258,23 @@ def main():
|
||||
parser.add_argument('--out', required=True)
|
||||
parser.add_argument('--part', action='append', default=[])
|
||||
parser.add_argument('--size', type=int, default=1024)
|
||||
parser.add_argument('--radius', type=int, default=32)
|
||||
parser.add_argument('--radius', type=int)
|
||||
parser.add_argument('--profile', choices=['legacy', 'adaptive'], default='legacy')
|
||||
parser.add_argument('--reference-mode', choices=['none', 'local', 'full', 'both', 'board'], default='board')
|
||||
parser.add_argument('--review-file')
|
||||
parser.add_argument('--candidate')
|
||||
parser.add_argument('--merged')
|
||||
parser.add_argument('--qa')
|
||||
parser.add_argument('--candidate-size-policy', choices=['strict', 'uniform'], default='strict')
|
||||
args = parser.parse_args()
|
||||
if args.action == 'prepare':
|
||||
if not 128 <= args.size <= 4096 or not 1 <= args.radius <= 256:
|
||||
if not 128 <= args.size <= 4096 or (args.radius is not None and not 1 <= args.radius <= 256):
|
||||
parser.error('size must be 128..4096 and radius 1..256')
|
||||
prepare(args.out, args.part, args.size, args.radius)
|
||||
if args.profile == 'adaptive':
|
||||
from completion_planning import prepare_adaptive
|
||||
prepare_adaptive(args.out, args.part, args.size, args.radius, args.reference_mode, args.review_file)
|
||||
else:
|
||||
prepare(args.out, args.part, args.size, args.radius or 32)
|
||||
elif args.action == 'merge':
|
||||
write_json(args.qa, merge_candidate(args.out, args.candidate, args.merged, args.candidate_size_policy))
|
||||
else:
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
"""Adaptive completion planning, reference canvases and reversible native-pixel merging."""
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
from pathlib import Path
|
||||
|
||||
import cv2
|
||||
import numpy as np
|
||||
from PIL import Image, ImageDraw
|
||||
|
||||
from normalize_reference import prepare_rgba
|
||||
|
||||
|
||||
def read(file):
|
||||
return json.loads(Path(file).read_text(encoding='utf-8-sig'))
|
||||
|
||||
|
||||
def write(file, value):
|
||||
Path(file).write_text(json.dumps(value, ensure_ascii=False, indent=2, allow_nan=False), encoding='utf-8')
|
||||
|
||||
|
||||
def binary(file, shape):
|
||||
a = np.array(Image.open(file))
|
||||
if a.shape != shape or not np.isin(a, [0, 255]).all():
|
||||
raise ValueError(f'Expected full-canvas binary PNG: {file}')
|
||||
return a > 0
|
||||
|
||||
|
||||
def review_inputs(file, ids, shape):
|
||||
"""Review masks are explicitly authored data; never guess ownership from scores."""
|
||||
if not file:
|
||||
return {}, {}
|
||||
file = Path(file).resolve()
|
||||
data = read(file)
|
||||
if data.get('schema_version') != 1 or not isinstance(data.get('parts'), dict):
|
||||
raise ValueError('Completion review requires schema_version=1 and parts object')
|
||||
hashes = {'review.json': hashlib.sha256(file.read_bytes()).hexdigest()}
|
||||
result = {}
|
||||
for name, entry in data['parts'].items():
|
||||
if name not in ids or not isinstance(entry, dict) or entry.get('confirmed') is not True:
|
||||
raise ValueError(f'Review entry requires known ID and confirmed=true: {name}')
|
||||
if set(entry) - {'confirmed', 'extension_radius', 'repair_mask', 'replace_mask', 'remove_mask'}:
|
||||
raise ValueError(f'Unknown review fields: {name}')
|
||||
item = {'confirmed': True}
|
||||
if 'extension_radius' in entry:
|
||||
radius = entry['extension_radius']
|
||||
if type(radius) is not int or not 1 <= radius <= 256:
|
||||
raise ValueError(f'Invalid reviewed radius: {name}')
|
||||
item['extension_radius'] = radius
|
||||
for key in ['repair_mask', 'replace_mask', 'remove_mask']:
|
||||
if key not in entry:
|
||||
continue
|
||||
ref = entry[key]
|
||||
if not isinstance(ref, str) or Path(ref).is_absolute():
|
||||
raise ValueError('Review mask must be relative to review file')
|
||||
mask_file = (file.parent / ref).resolve()
|
||||
if not mask_file.is_relative_to(file.parent):
|
||||
raise ValueError('Review mask path escapes review directory')
|
||||
item[key] = binary(mask_file, shape)
|
||||
hashes[f'{name}/{key}'] = hashlib.sha256(mask_file.read_bytes()).hexdigest()
|
||||
result[name] = item
|
||||
return result, hashes
|
||||
|
||||
|
||||
def bounding(mask):
|
||||
ys, xs = np.nonzero(mask)
|
||||
return [int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1] if len(xs) else None
|
||||
|
||||
|
||||
def enclosed(mask):
|
||||
padded = np.pad(mask.astype(np.uint8), 1)
|
||||
fill = padded.copy()
|
||||
cv2.floodFill(fill, None, (0, 0), 1)
|
||||
return ~fill[1:-1, 1:-1].astype(bool)
|
||||
|
||||
|
||||
def white_image(array):
|
||||
im = Image.fromarray(array).convert('RGBA')
|
||||
return Image.alpha_composite(Image.new('RGBA', im.size, 'white'), im).convert('RGB')
|
||||
|
||||
|
||||
def prepare_adaptive(directory, selected_ids, size=1024, radius=None, reference_mode='board', review_file=None):
|
||||
# Import shared native coordinate helpers lazily to avoid a module cycle.
|
||||
from completion_geometry import localize, allowed_regions
|
||||
out = Path(directory)
|
||||
plan, seg = read(out / 'semantic-parts.json'), read(out / 'segmentation.json')
|
||||
rgba = np.array(Image.open(out / 'source.png').convert('RGBA'))
|
||||
h, w = rgba.shape[:2]
|
||||
ids = {p['id']: p for p in plan['parts']}
|
||||
if set(selected_ids) - set(ids):
|
||||
raise ValueError('Unknown requested part ID')
|
||||
records = {p['id']: p for p in seg['parts']}
|
||||
if len(records) != len(seg['parts']) or set(records) - set(ids):
|
||||
raise ValueError('Invalid segmentation IDs')
|
||||
masks = {name: binary(out / 'masks' / f'{name}.png', (h, w)) for name in records}
|
||||
for name, mask in masks.items():
|
||||
if not mask.any() or np.any(mask & (rgba[:, :, 3] == 0)):
|
||||
raise ValueError(f'Empty mask or mask outside original alpha: {name}')
|
||||
reviews, review_hashes = review_inputs(review_file, ids, (h, w))
|
||||
coverage = np.zeros((h, w), np.uint16)
|
||||
for mask in masks.values():
|
||||
coverage += mask
|
||||
tasks, skipped, assessments = [], [], []
|
||||
(out / 'preservation').mkdir(exist_ok=True)
|
||||
for part in plan['parts']:
|
||||
name = part['id']
|
||||
if selected_ids and name not in selected_ids:
|
||||
continue
|
||||
if name not in masks or part['visibility'] == 'hidden':
|
||||
skipped.append({'id': name, 'reason': 'NO_VISIBLE_SAM_MASK'})
|
||||
continue
|
||||
visible = masks[name]
|
||||
box = bounding(visible)
|
||||
review = reviews.get(name, {})
|
||||
resolved_radius = review.get('extension_radius', radius if radius is not None else
|
||||
int(np.clip(math.ceil(min(box[2]-box[0], box[3]-box[1]) * .15), 4, 96)))
|
||||
inferred = dict(part)
|
||||
inferred['completion'] = {
|
||||
'occlusion_fill': bool(part['occluded_by']),
|
||||
'joint_extension_parent': part['rig_role'] == 'bone_part' and part['parent'] is not None,
|
||||
'joint_extension_child': part['rig_role'] == 'bone_part' and any(
|
||||
p['parent'] == name and p['rig_role'] == 'bone_part' for p in plan['parts'])}
|
||||
_, regions, warnings = allowed_regions(inferred, plan, masks, visible, resolved_radius)
|
||||
# Only enclosed source foreground without another mask owner is an automatic
|
||||
# repair candidate. Open gaps and ambiguous ownership stay diagnostic.
|
||||
gap = enclosed(visible) & (rgba[:, :, 3] > 0) & (coverage == 0)
|
||||
cleanup = np.zeros_like(visible)
|
||||
for front in part['occluded_by']:
|
||||
p = ids[front]
|
||||
if front in masks and p['rig_role'] != 'bone_part' and p['bind_to'] == name:
|
||||
cleanup |= visible & masks[front]
|
||||
replace = review.get('replace_mask', np.zeros_like(visible))
|
||||
removed = review.get('remove_mask', np.zeros_like(visible))
|
||||
repair = review.get('repair_mask', np.zeros_like(visible))
|
||||
if np.any((replace | removed) & ~visible) or np.any(repair & visible) or np.any(replace & removed):
|
||||
raise ValueError(f'{name}: replacement/removal must be disjoint subsets of SAM; repair must be outside SAM')
|
||||
regions['segmentation_gap'] = gap | repair
|
||||
regions['attachment_cleanup'] = cleanup & ~removed
|
||||
regions['reviewed_replacement'] = replace
|
||||
allowed = np.zeros_like(visible)
|
||||
for region in regions.values():
|
||||
allowed |= region
|
||||
allowed &= ~removed
|
||||
protected = visible & ~allowed & ~removed
|
||||
unexpected = []
|
||||
duplicates = []
|
||||
for other, mask in masks.items():
|
||||
if other == name:
|
||||
continue
|
||||
overlap = int((mask & visible).sum())
|
||||
related = other in part['occluded_by'] or name in ids[other]['occluded_by']
|
||||
if overlap and not related:
|
||||
unexpected.append({'id': other, 'pixels': overlap})
|
||||
if overlap / max(1, int((mask | visible).sum())) >= .8:
|
||||
duplicates.append(other)
|
||||
assessment = {'id': name, 'original_needs_completion': part['needs_completion'],
|
||||
'inferred_completion': inferred['completion'], 'radius': resolved_radius,
|
||||
'region_pixels': {k: int(v.sum()) for k, v in regions.items()},
|
||||
'protected_pixels': int(protected.sum()), 'removed_pixels': int(removed.sum()),
|
||||
'unexpected_overlaps': unexpected, 'review_confirmed': bool(review)}
|
||||
assessments.append(assessment)
|
||||
if duplicates and not review:
|
||||
skipped.append({'id': name, 'reason': 'AMBIGUOUS_MASK_REQUIRES_REVIEW', 'overlaps': duplicates})
|
||||
continue
|
||||
if not protected.any() or not (protected & (rgba[:, :, 3] >= 240)).any():
|
||||
skipped.append({'id': name, 'reason': 'NO_PRESERVED_ALIGNMENT_ANCHOR'})
|
||||
continue
|
||||
if not allowed.any() and not removed.any():
|
||||
skipped.append({'id': name, 'reason': 'NO_OBSERVED_EDIT_REGION'})
|
||||
continue
|
||||
region_box = bounding(visible | allowed)
|
||||
bw, bh = region_box[2] - region_box[0], region_box[3] - region_box[1]
|
||||
native_size = max(8, math.ceil(max(bw, bh) * 1.15))
|
||||
origin = [region_box[0] - (native_size-bw)//2, region_box[1] - (native_size-bh)//2]
|
||||
work_size = int(size * .68) if reference_mode == 'board' else size
|
||||
offset = [int(size * .02), (size-work_size)//2] if reference_mode == 'board' else [0, 0]
|
||||
scale = work_size / native_size
|
||||
folder = out / 'tasks' / name
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
native_source = localize(rgba, origin, native_size)
|
||||
native_visible = localize(visible, origin, native_size)
|
||||
native_keep = localize(protected, origin, native_size)
|
||||
native_edit = localize(allowed, origin, native_size)
|
||||
native_remove = localize(removed, origin, native_size)
|
||||
cut = native_source.copy()
|
||||
cut[~native_keep] = 0
|
||||
Image.fromarray(cut).save(folder / 'native-visible.png')
|
||||
Image.fromarray(native_source).save(folder / 'native-source.png')
|
||||
for label, mask in [('keep', native_keep), ('allowed', native_edit), ('remove', native_remove), ('sam', native_visible)]:
|
||||
Image.fromarray(mask.astype(np.uint8)*255).save(folder / f'native-{label}.png')
|
||||
|
||||
def project(array, is_mask=False):
|
||||
small = np.array(Image.fromarray(array.astype(np.uint8)*255 if is_mask else array).resize(
|
||||
(work_size, work_size), Image.Resampling.NEAREST if is_mask else Image.Resampling.LANCZOS))
|
||||
canvas = np.zeros((size, size) + small.shape[2:], np.uint8)
|
||||
canvas[offset[1]:offset[1]+work_size, offset[0]:offset[0]+work_size] = small
|
||||
return canvas
|
||||
|
||||
model_visible = project(cut)
|
||||
Image.fromarray(model_visible).save(folder / 'visible.png')
|
||||
model_allowed = project(native_edit, True) > 0
|
||||
model_remove = project(native_remove, True) > 0
|
||||
Image.fromarray(model_allowed.astype(np.uint8)*255).save(folder / 'allowed-mask.png')
|
||||
mask_rgba = np.full((size, size, 4), 255, np.uint8)
|
||||
mask_rgba[model_allowed | model_remove, 3] = 0
|
||||
Image.fromarray(mask_rgba).save(folder / 'edit-mask.png')
|
||||
context = white_image(project(native_source))
|
||||
context.save(folder / 'context.png')
|
||||
full_reference = white_image(rgba)
|
||||
full_reference.thumbnail((size, size), Image.Resampling.LANCZOS)
|
||||
full_reference.save(folder / 'reference.png')
|
||||
model_input = white_image(model_visible)
|
||||
panel_boxes = []
|
||||
if reference_mode == 'board':
|
||||
draw = ImageDraw.Draw(model_input)
|
||||
x = int(size * .74)
|
||||
draw.line([(x-4, 0), (x-4, size)], fill='#777777', width=1)
|
||||
for label, image, top in [('LOCAL REFERENCE', white_image(native_source), 0),
|
||||
('FULL REFERENCE', full_reference.copy(), size//2)]:
|
||||
draw.text((x, top+2), label, fill='black')
|
||||
image.thumbnail((size-x-4, size//2-24), Image.Resampling.LANCZOS)
|
||||
model_input.paste(image, (x, top+22))
|
||||
panel_boxes.append([x, top+22, x+image.width, top+22+image.height])
|
||||
model_input.save(folder / 'input.png')
|
||||
bounds, native_counts = {}, {}
|
||||
for kind, region in regions.items():
|
||||
native = localize(region & allowed, origin, native_size)
|
||||
model = project(native, True)
|
||||
Image.fromarray(native.astype(np.uint8)*255).save(folder / f'native-{kind}.png')
|
||||
Image.fromarray(model).save(folder / f'{kind}.png')
|
||||
bounds[kind] = bounding(model > 0)
|
||||
native_counts[kind] = int(native.sum())
|
||||
preservation = {}
|
||||
for key, mask in [('preserve_mask', protected), ('editable_mask', allowed), ('removed_mask', removed)]:
|
||||
relative = f'preservation/{name}-{key}.png'
|
||||
Image.fromarray(mask.astype(np.uint8)*255).save(out / relative)
|
||||
preservation[key] = relative
|
||||
if unexpected:
|
||||
warnings.append('UNEXPECTED_OVERLAP_REVIEW_REQUIRED')
|
||||
if cleanup.any():
|
||||
warnings.append('ATTACHMENT_BASE_CLEANUP_NEEDS_REVIEW')
|
||||
task = {'id': name, 'semantic': part['semantic'], 'profile': 'adaptive',
|
||||
'completion': inferred['completion'], 'original_completion': part['completion'],
|
||||
'origin_xy': origin, 'native_size': native_size, 'canvas_size': [size, size], 'source_size': [w, h],
|
||||
'extension_radius': resolved_radius, 'allowed_pixels': int(native_edit.sum()),
|
||||
'removed_pixels': int(native_remove.sum()), 'regions': native_counts,
|
||||
'region_bounds_xyxy': bounds, 'occluded_by': part['occluded_by'],
|
||||
'warnings': records[name].get('warnings', []) + warnings, 'preservation': preservation,
|
||||
'reference_mode': reference_mode, 'reference_panel_boxes': panel_boxes,
|
||||
'transform': {'scale': scale, 'working_xyxy': [*offset, offset[0]+work_size, offset[1]+work_size],
|
||||
'source_to_canvas': [scale, 0, offset[0]-scale*origin[0]+(scale-1)/2,
|
||||
0, scale, offset[1]-scale*origin[1]+(scale-1)/2],
|
||||
'canvas_to_source': [1/scale, 0, origin[0]-offset[0]/scale+(1/scale-1)/2,
|
||||
0, 1/scale, origin[1]-offset[1]/scale+(1/scale-1)/2]},
|
||||
'semantic_verified': False, 'status': 'prepared'}
|
||||
write(folder / 'task.json', task)
|
||||
tasks.append(task)
|
||||
result = {'schema_version': 2, 'profile': 'adaptive', 'tasks': tasks, 'skipped': skipped,
|
||||
'assessments': assessments, 'review_hashes': review_hashes, 'reference_mode': reference_mode,
|
||||
'canvas_size': size, 'quality_status': 'needs_review'}
|
||||
write(out / 'completion-plan.json', result)
|
||||
return result
|
||||
|
||||
|
||||
def merge_adaptive(folder, candidate_file, output_file, size_policy):
|
||||
folder = Path(folder)
|
||||
task = read(folder / 'task.json')
|
||||
candidate = Image.open(candidate_file).convert('RGBA')
|
||||
qa = {'accepted': False, 'semantic_verified': False, 'reasons': [], 'profile': 'adaptive',
|
||||
'expected_size': task['canvas_size'], 'returned_size': list(candidate.size),
|
||||
'size_policy': size_policy, 'canvas_resampled': False, 'transform': task['transform']}
|
||||
tw, th = task['canvas_size']
|
||||
if candidate.size != (tw, th):
|
||||
if size_policy == 'strict' or candidate.width * th != candidate.height * tw:
|
||||
qa['reasons'].append('CANVAS_SIZE_MISMATCH' if size_policy == 'strict' else 'CANVAS_ASPECT_RATIO_MISMATCH')
|
||||
return qa
|
||||
qa.update(canvas_resampled=True, canvas_scale=tw/candidate.width, resampling='LANCZOS')
|
||||
candidate = candidate.resize((tw, th), Image.Resampling.LANCZOS)
|
||||
candidate.save(Path(output_file).with_name(Path(output_file).stem + '-normalized.png'))
|
||||
# Record raw behavior before masking/restoration; an accepted HTTP response is
|
||||
# not evidence that the relay/model respected the supplied edit mask.
|
||||
expected_input = np.array(Image.open(folder / 'input.png').convert('RGB')).astype(float)
|
||||
raw = np.array(white_image(np.array(candidate))).astype(float)
|
||||
editable = np.array(Image.open(folder / 'edit-mask.png'))[:, :, 3] == 0
|
||||
outside_delta = np.abs(raw - expected_input).mean(axis=2)
|
||||
qa['raw_outside_edit_mae'] = float(outside_delta[~editable].mean()) if (~editable).any() else 0
|
||||
qa['raw_outside_edit_changed_pixels'] = int(((outside_delta > 30) & ~editable).sum())
|
||||
rect = task['transform']['working_xyxy']
|
||||
try:
|
||||
cropped, preprocessing = prepare_rgba(candidate.crop(rect))
|
||||
except ValueError as error:
|
||||
qa.update(reasons=['CANNOT_EXTRACT_FOREGROUND'], detail=str(error))
|
||||
return qa
|
||||
side = task['native_size']
|
||||
generated = np.array(cropped.resize((side, side), Image.Resampling.LANCZOS))
|
||||
original = np.array(Image.open(folder / 'native-visible.png').convert('RGBA'))
|
||||
source = np.array(Image.open(folder / 'native-source.png').convert('RGBA'))
|
||||
keep = binary(folder / 'native-keep.png', (side, side))
|
||||
allowed = binary(folder / 'native-allowed.png', (side, side))
|
||||
removed = binary(folder / 'native-remove.png', (side, side))
|
||||
# Judge alignment before the inverse resampling. Alpha at native boundary
|
||||
# pixels changes during interpolation even for an exactly aligned candidate.
|
||||
model_original = np.array(Image.open(folder / 'visible.png').convert('RGBA').crop(rect))
|
||||
model_generated = np.array(cropped)
|
||||
opaque = model_original[:, :, 3] >= 240
|
||||
if not opaque.any():
|
||||
qa['reasons'].append('NO_OPAQUE_ALIGNMENT_ANCHOR')
|
||||
return qa
|
||||
rgb = np.array(white_image(model_generated)).astype(float)
|
||||
mae = float(np.abs(rgb[opaque] - model_original[:, :, :3][opaque]).mean())
|
||||
retained = float((model_generated[:, :, 3][opaque] >= 240).mean())
|
||||
proposed = (generated[:, :, 3] > 0) & ~keep
|
||||
additions = proposed & allowed
|
||||
_, labels = cv2.connectedComponents((keep | additions).astype(np.uint8), connectivity=8)
|
||||
touching = np.unique(labels[keep])
|
||||
connected = additions & np.isin(labels, touching[touching != 0])
|
||||
counts = {kind: int((connected & binary(folder / f'native-{kind}.png', (side, side))).sum())
|
||||
for kind in task['regions']}
|
||||
qa.update(alignment_mae=mae, visible_retention=retained, alignment_space='request_work_panel', new_pixels=int(connected.sum()),
|
||||
removed_pixels=int(removed.sum()), region_additions=counts,
|
||||
unfulfilled_regions=[k for k, n in task['regions'].items() if n and not counts[k]],
|
||||
discarded_outside_allowed=int((proposed & ~allowed).sum()), preprocessing=preprocessing)
|
||||
if mae > 30 or retained < .95:
|
||||
qa['reasons'].append('VISIBLE_ALIGNMENT_MISMATCH')
|
||||
if not connected.any() and not removed.any():
|
||||
qa['reasons'].append('NO_CONNECTED_NEW_PIXELS')
|
||||
for kind in ['attachment_cleanup', 'reviewed_replacement']:
|
||||
if task['regions'][kind] and counts[kind] / task['regions'][kind] < .95:
|
||||
qa['reasons'].append(f'INCOMPLETE_{kind.upper()}')
|
||||
if qa['reasons']:
|
||||
return qa
|
||||
merged = np.zeros_like(original)
|
||||
merged[connected] = generated[connected]
|
||||
# Visible segmentation gaps already have ground-truth texture in source.
|
||||
# Restore it where the proposed fill connects, rather than trusting a repaint.
|
||||
repair = binary(folder / 'native-segmentation_gap.png', (side, side)) & connected & (source[:, :, 3] > 0)
|
||||
merged[repair] = source[repair]
|
||||
merged[keep] = original[keep]
|
||||
merged[removed] = 0
|
||||
qa.update(accepted=True, quality_status='needs_review', visible_pixels_exact=True,
|
||||
preservation_scope='preserve_mask', repaired_source_pixels=int(repair.sum()))
|
||||
Image.fromarray(merged).save(output_file)
|
||||
return qa
|
||||
+79
-27
@@ -2,9 +2,11 @@
|
||||
|
||||
## 1. 阶段简介
|
||||
|
||||
读取 SAM 可见 mask,为需要遮挡补全或关节延伸的部件准备局部画布,调用中转站图片编辑模型,再将允许区域内的新增像素与原可见像素合并。Python 几何处理固定使用 Conda `ai-spine`;图片生成在远端完成,本阶段不加载本地 SAM。
|
||||
读取 SAM 可见 mask,规划分割缺口修复、遮挡补全、关节延伸和附件底图清理,向图片模型提供部件与原图参考,再把结果映射回原图坐标。Python 几何处理固定使用 Conda `ai-spine`;图片生成在远端完成,本阶段不加载本地 SAM。
|
||||
|
||||
当前已实现准备、真实图片请求、离线候选导入、几何检查与透明补全部件输出。尚未输出使用补全部件组装的完整角色,尚未接入新管线 Spine 导出。几何通过仍为 `needs_review`,不能等同于部件可用。
|
||||
默认 `--completion-profile adaptive` 使用实际 mask 重建需求、自适应半径和可逆缩放;`legacy` 保留旧画布与筛选规则,供旧候选复检。原 SAM 输入始终留档,只有保留区域锁定原像素;明确规划的编辑区域可以替换,人工确认的移除区域可删除。
|
||||
|
||||
当前已实现准备、真实图片请求、离线候选导入、几何检查与透明补全部件输出。[第四阶段](4-spine-stage.md) 独立整合 SAM 与补全部件,输出完整角色组装、Spine 骨架和预览。几何通过仍为 `needs_review`,不能等同于部件可用。
|
||||
|
||||
## 2. 输入、输出与执行链路
|
||||
|
||||
@@ -14,18 +16,19 @@
|
||||
tools/model-check.mjs --stage completion --input-dir SAM目录
|
||||
→ stages/completion-stage.mjs 校验计划、mask 文件、参数及 Conda
|
||||
→ completion_geometry.py prepare
|
||||
筛选需求 → 计算允许区域 → 建立局部画布与编辑 mask
|
||||
adaptive:completion_planning.py 重建需求 → 区分保留/编辑/移除 → 缩放工作区并加入参考
|
||||
legacy:沿用 needs_completion → 固定半径 → 原像素平移
|
||||
→ prepare-only:保存任务和报告后结束
|
||||
→ 否则:串行请求 images/edits,或读取离线候选
|
||||
→ completion_geometry.py merge
|
||||
检查画布/前景/对齐 → 限制新增区域与连通性 → 恢复原可见像素
|
||||
检查画布/前景/对齐 → 逆变换回原像素尺寸 → 限制编辑和连通性 → 恢复保留像素
|
||||
→ 每部件选第一个通过几何检查的候选
|
||||
→ finalize 输出透明部件、原图坐标和报告
|
||||
```
|
||||
|
||||
任务必须同时满足:计划 `needs_completion=true`、非 hidden、有可见 SAM mask、允许新增区域非空。指定 `--part` 只缩小范围,不强制绕过筛选。分别记录 `NO_COMPLETION_NEEDED`、`NO_VISIBLE_SAM_MASK`、`NO_ALLOWED_EXTENSION`。
|
||||
adaptive 检查所有选中部件的实际 mask,不再以第一阶段 needs_completion 为唯一门槛。必须有可见 mask、保留对齐锚点,以及非空编辑或移除区域。隐藏/缺 mask、无观察到的编辑区域、无保留锚点分别跳过;无明确遮挡关系的近重复 mask(IoU≥0.8)要求人工复核。一般有歧义的重叠只记录诊断,不自动猜测归属。指定 `--part` 只缩小处理范围。
|
||||
|
||||
以 `output/20260918-183532/sam` 为例:32 个计划项中 10 个 hidden 没有 mask;22 个有 mask,其中 12 个进入补全。帽子、两袖、两眼、两眉、鼻、嘴、胡子这 10 个因第一阶段标记不需补全而跳过,没有重新由模型判断实际 mask 完整性。
|
||||
legacy 才要求 `needs_completion=true` 且允许新增非空。旧实验 `output/20260918-183532/sam` 的 22 个可见部件中 12 个进入补全,是旧筛选结果,不能套用于 adaptive。
|
||||
|
||||
## 3. 阶段执行命令示例及参数说明
|
||||
|
||||
@@ -68,6 +71,7 @@ node .\tools\model-check.mjs --stage completion `
|
||||
--input-dir $samInput `
|
||||
--candidates-dir .\output\20260918-183532\complete2_gpt `
|
||||
--out-dir ".\output\$runId\complete-recheck" `
|
||||
--completion-profile legacy `
|
||||
--candidate-size-policy uniform
|
||||
```
|
||||
|
||||
@@ -83,6 +87,18 @@ node .\tools\model-check.mjs --stage completion `
|
||||
|
||||
探针只保存候选,不执行部件准备与几何验收,不能与 `--input-dir` 混用。
|
||||
|
||||
新的单部件多图/mask 能力探针会产生一次真实请求,并执行新几何链路;不会修改默认 capabilities。仅在决定进行付费小样时运行,替换 part 为实际存在的可执行任务:
|
||||
|
||||
```powershell
|
||||
node .\tools\model-check.mjs --stage completion `
|
||||
--config .\configs\tiantoken.json `
|
||||
--input-dir $samInput `
|
||||
--out-dir ".\output\$runId\capability-probe" `
|
||||
--part head --probe-edit-capabilities
|
||||
```
|
||||
|
||||
该探针发送主图、局部参考、全角色参考和透明可编辑 mask,强制 n=1、禁用 HTTP 重试。查看 request.json、原候选、QA 和 capability-evidence.json;HTTP 接受不等于参考图被使用或 mask 语义正确,因此不会自动启用能力。prepare-only 可配 `--completion-reference both --completion-mask required` 检查计划发送内容而不联网。
|
||||
|
||||
| 参数 | 说明 |
|
||||
| --- | --- |
|
||||
| `--input-dir DIR` | 完整 SAM 输出目录,部件补全必需 |
|
||||
@@ -90,8 +106,13 @@ node .\tools\model-check.mjs --stage completion `
|
||||
| `--part ID` | 可重复,ID 必须存在且不重复;省略时处理所有符合筛选条件的部件 |
|
||||
| `--prepare-only` | 只生成画布、区域、提示词和报告,不请求模型 |
|
||||
| `--candidates-dir DIR` | 离线导入候选,与 prepare-only 互斥 |
|
||||
| `--completion-size N` | 方形画布边长,默认 1024,整数 128~4096;原像素不缩放 |
|
||||
| `--extension-radius N` | 轮廓延展距离与关节圆盘半径,默认 32,整数 1~256 |
|
||||
| `--completion-profile adaptive/legacy` | 默认 adaptive;旧候选需显式 legacy |
|
||||
| `--completion-reference auto/board/local/full/both/none` | auto:支持多图时 local+full,否则单图参考板;legacy auto 为旧单图/局部参考行为 |
|
||||
| `--completion-mask auto/required/off` | auto 遵循配置;required 要求已配置支持,或用单部件能力探针试验;off 不发送但仍本地限制编辑 |
|
||||
| `--completion-review FILE` | 人工确认的补全区域文件,格式见第 4 节;只用于 adaptive |
|
||||
| `--probe-edit-capabilities` | adaptive 在线单部件试验多图+mask,不持久修改配置,不自动认定能力支持 |
|
||||
| `--completion-size N` | 请求方形画布边长,默认 1024,整数 128~4096;adaptive 记录显式缩放和逆变换 |
|
||||
| `--extension-radius N` | 原图像素半径覆盖,整数 1~256;adaptive 默认 min(mask宽,高)×0.15 向上取整并限于 4~96;legacy 默认 32 |
|
||||
| `--candidate-size-policy NAME` | strict 默认拒绝尺寸不符;uniform 允许同宽高比整画布缩放后复检 |
|
||||
| `--completion-model ID` | 图片模型 ID,原样发送 |
|
||||
| `--config FILE` / `--env-file FILE` | 默认 astra-qwen.json / 项目 .env;shell 同名变量优先 |
|
||||
@@ -122,19 +143,42 @@ node .\tools\model-check.mjs --stage completion `
|
||||
|
||||
不同原图、mask 或参数的候选不能混用。尺寸之外,部件位置、朝向、比例也必须保持一致。
|
||||
|
||||
adaptive 的旧运行候选还要匹配 profile、参考布局、复核文件/掩码哈希和规划版本;board 与独立多图的画布不同,不可混用。平铺候选仍需人工保证它对应当前请求画布。
|
||||
|
||||
人工复核文件示例(JSON 中的 ID 取自当前语义计划,字段均可省略但 confirmed 必须为 true):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"parts": {
|
||||
"part_id": {
|
||||
"confirmed": true,
|
||||
"extension_radius": 24,
|
||||
"repair_mask": "review-masks/repair.png",
|
||||
"replace_mask": "review-masks/replace.png",
|
||||
"remove_mask": "review-masks/remove.png"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
mask 是与 source 同尺寸的单通道 0/255 PNG,白色表示区域;路径相对于复核 JSON 且不得越出其目录。repair 必须在当前 SAM mask 外;replace/remove 必须位于 SAM 内且互不重叠。replace 用于误分重绘或额外底图清理,remove 用于明确不属于该部件的像素。半径优先级为部件复核值 > CLI 全局值 > 自适应值。文件及 masks 会复制进输出,哈希纳入复用身份。
|
||||
|
||||
## 5. 执行链路中重要部分代码文件及原理说明(包括重点说明)
|
||||
|
||||
**允许区域。** `completion_geometry.py::allowed_regions()` 先计算可见 mask 外、距轮廓不超过 extension-radius 的区域,再分别与以下区域取交集:遮挡补全用 occluded_by 的 mask 并集;父关节延展用本部件 pivot 周围圆盘;子关节延展用直接子骨骼部件 pivot 圆盘。始终排除原可见 mask,新增范围限制在原图画布内。它是保守几何范围,不是完整隐藏形状推断。
|
||||
**按实际 mask 重建需求。** 遮挡延伸使用轮廓外半径带与 occluded_by 的实际 mask 交集;父/子关节需求来自骨骼层级与 pivot。自动分割修复仅选择 mask 内部闭合孔洞、原图有前景且无其他 mask 归属的像素;开放缺口和重叠归属不自动猜测,可用复核文件指定。自动底图清理仅针对 occluded_by 中直接 bind_to 当前骨骼的附件/五官与当前 mask 的交集。这些仍是几何候选,语义不自动获证。
|
||||
|
||||
**局部坐标。** `prepare()` 将可见区和允许区居中放入方形画布,至少留 16 像素边距,只平移和补边。`origin_xy` 是局部左上角在原图的位置,可为负数;原图位置=局部位置+origin。放不下时失败,不能偷偷缩放原像素。
|
||||
**保留与编辑。** editable 为遮挡/关节/修复/清理/人工替换区域的并集,扣除 removed;preserve = SAM − editable − removed。保留原 mask 和源图,并另存三种全画布区域供第四阶段核对。只有原始 SAM 或单一图片目录不足以描述这个契约。
|
||||
|
||||
**重点:当前实际发送什么。** `stages/completion-stage.mjs` 默认只发送白底 `input.png`。三个现有配置均为 `multiple_images=false`、`mask=false`,因此本地生成 context.png 和 edit-mask.png 不意味着发给模型。只有开启并验证对应能力后,分支才发送第二张同坐标局部原图参考和编辑遮罩;当前没有独立全角色参考图输入。文字边界说明及返回后的区域裁剪不能替代模型侧遮罩约束。
|
||||
**可逆画布。** native 方形裁剪边长为可见+编辑区域最长边的 1.15 倍向上取整,记录 origin。工作区缩放到请求画布(board 工作区边长为请求边长约 68%,其余为参考板;独立参考时工作区占整幅)。RGBA 用 LANCZOS,区域 mask 用最近邻。task.json 保存 native_size、scale、working_xyxy 和含像素中心约定的正反仿射变换。返回后只取工作区,逆缩放到 native,再裁剪回原图坐标;保留像素从原图恢复,所以缩放不改变最终保留像素。legacy 仍仅平移补边。
|
||||
|
||||
**实际发送内容。** 当前三个提供方默认仍关闭 multiple_images/mask。adaptive auto 因此发送一张 input.png:左侧是待编辑工作区,右侧为局部及完整角色参考,参考区域不参与最终资产。支持多图时 auto 顺序为 input.png → context.png → reference.png;也可选 local/full/both。request-plan.json 保存计划;只有实际请求才写 request.json,包含有序文件角色、SHA-256、尺寸、image 字段、mask 哈希、透明可编辑语义与是否发送。mask 不支持时明确标 local_only,不能宣称模型侧已受 mask 约束。
|
||||
|
||||
**图片解码与尺寸。** `providers/image-edits.mjs` 使用 multipart images/edits,支持非空 b64_json 或返回 URL 下载;空 b64_json 会回退有效 URL。当前部件请求显式发送 size=画布宽x高,并写入 request.json;配置 params.size 冲突会报错。strict 模式拒绝 CANVAS_SIZE_MISMATCH;uniform 仅将同宽高比整画布 LANCZOS 缩放到目标尺寸,保留原候选,不平移、不裁剪、不单独缩放物体,也不改变原图/mask。
|
||||
|
||||
**几何验收与合并。** `merge_candidate()` 去除边界连通白底,在原部件 Alpha≥240 的像素处检查 RGB 平均绝对误差≤30、可见保留率≥95%;无不透明锚点拒绝。只取允许区域内且与原部件连通的新增像素,没有有效新增即拒绝。通过后逐像素恢复原可见 RGBA,模型只提供新增部分。
|
||||
**几何验收与合并。** adaptive 在请求工作区的保留不透明像素处检查 RGB MAE≤30、保留率≥95%,先验对齐再逆采样,避免把逆采样边界 Alpha 变化误判成位移。返回前景只保留 editable 内且连接 preserve 的像素;清理/人工替换区域至少 95% 有效填充,否则拒绝。没有有效修改且没有人工移除则拒绝。最后恢复 preserve 的原 RGBA,已连接的可见缺口使用原图纹理,removed 清零。legacy 沿用原可见像素全保护规则。
|
||||
|
||||
**重点:验收仍有缺口。** `unfulfilled_regions` 记录未补到的子区域,但当前不是硬性拒绝条件;某一区域新增少量像素也不代表关节补够。语义、接缝、材质、运动露底没有自动验收。错误 SAM 像素会被原样恢复;若头部 mask 已含眼睛/胡子,当前规则也不能清除这些内部附件生成干净脸底。
|
||||
**能力证据与质量边界。** adaptive QA 记录合并前的 raw_outside_edit_mae / changed_pixels,避免本地恢复掩盖模型越界修改。capability-evidence.json 记录探针的输入顺序、尺寸、候选和 QA,但 reference_use_verified、mask_semantics_verified 不自动置 true,仍需视觉验证。区域填满不能证明生成的是正确底图;未明确划入 editable/remove 的误分仍受保护。接缝、材质、关节补足和运动露底尚无可靠自动验收。
|
||||
|
||||
## 6. 各个代码文件职责索引
|
||||
|
||||
@@ -143,6 +187,7 @@ node .\tools\model-check.mjs --stage completion `
|
||||
| [tools/model-check.mjs](../tools/model-check.mjs) | 部件补全与独立探针分发、参数互斥检查 |
|
||||
| [stages/completion-stage.mjs](../stages/completion-stage.mjs) | 输入指纹、准备、串行请求/离线读取、候选选择、状态留档 |
|
||||
| [completion_geometry.py](../completion_geometry.py) | allowed_regions、prepare、merge_candidate、finalize 与报告 |
|
||||
| [completion_planning.py](../completion_planning.py) | adaptive 需求重建、复核输入、参考板、正反变换、编辑分区与合并 |
|
||||
| [normalize_reference.py](../normalize_reference.py) | 候选 Alpha 保留、边界连通白底提取 |
|
||||
| [providers/image-edits.mjs](../providers/image-edits.mjs) | multipart 编辑请求、能力检查、Base64/URL 图片解码 |
|
||||
| [providers/http.mjs](../providers/http.mjs) | HTTP、超时、重试、诊断和脱敏 |
|
||||
@@ -153,6 +198,7 @@ node .\tools\model-check.mjs --stage completion `
|
||||
| [prompts/complete-v1.txt](../prompts/complete-v1.txt) | 保持姿势、尺寸、可见纹理与隐藏区域补全约束 |
|
||||
| [Tiantoken](../configs/tiantoken.json)、[DeepSeek/Qwen](../configs/deepseek-qwen.json)、[Astra/Qwen](../configs/astra-qwen.json) 配置 | 图片模型、认证变量、请求参数与能力开关 |
|
||||
| [tests/completion-stage.test.mjs](../tests/completion-stage.test.mjs) / [tests/test_completion.py](../tests/test_completion.py) | mock 编辑、离线复用、尺寸/对齐拒绝、像素保真与区域回归 |
|
||||
| [tests/test_completion_adaptive.py](../tests/test_completion_adaptive.py) | 放大/缩小回贴、漏分重建、附件清理、区域复核、拒绝位移和第四阶段契约 |
|
||||
| [tests/model-stages.test.mjs](../tests/model-stages.test.mjs) | 提供方与探针相关回归 |
|
||||
|
||||
## 7. 产物详细说明
|
||||
@@ -160,34 +206,40 @@ node .\tools\model-check.mjs --stage completion `
|
||||
| 路径 | 内容与用途 |
|
||||
| --- | --- |
|
||||
| `source.png`、`semantic-parts.json`、`segmentation.json`、`masks/` | 输入快照,用于复现与离线候选身份核对 |
|
||||
| `completion-plan.json` | 准备任务、允许区域、参数和跳过原因 |
|
||||
| `tasks/<id>/input.png` | 白底局部可见部件,默认唯一模型输入图 |
|
||||
| `tasks/<id>/visible.png` | 原图可见 RGBA,合并时恢复像素 |
|
||||
| `tasks/<id>/context.png` | 同坐标局部原图参考,当前默认不发送 |
|
||||
| `tasks/<id>/edit-mask.png` | RGBA 编辑遮罩,透明处可编辑,当前默认不发送 |
|
||||
| `completion-plan.json` | 准备任务、区域、跳过原因;adaptive 含 assessments、重建需求、半径及歧义重叠 |
|
||||
| `tasks/<id>/input.png` | 实际请求主图;board 含目标、局部参考与全角色参考 |
|
||||
| `tasks/<id>/visible.png` | 请求坐标的保留 RGBA;legacy 为原可见 RGBA |
|
||||
| `tasks/<id>/context.png` / `reference.png` | 局部 / 全角色原图参考,由 reference mode 决定是否独立发送 |
|
||||
| `tasks/<id>/edit-mask.png` | RGBA 编辑遮罩,透明处为 editable 或 removed;是否发送以 request.json 为准 |
|
||||
| `tasks/<id>/allowed-mask.png` | 白色表示允许新增区域 |
|
||||
| `tasks/<id>/<补全字段>.png` | 启用的补全子区域 mask |
|
||||
| `tasks/<id>/task.json` / `prompt.txt` | origin、区域、需求和告警 / 实际提示词 |
|
||||
| `tasks/<id>/request.json` | 在线请求模型、图片哈希、是否发送 mask;修复后含 size |
|
||||
| `tasks/<id>/task.json` / `prompt.txt` | origin、区域、变换、参考布局、重建需求和告警 / 实际提示词 |
|
||||
| `tasks/<id>/native-*.png` | 原像素尺寸的源图、保留像素与各区域,用于逆映射和精确恢复 |
|
||||
| `preservation/<id>-*.png` | 原图全画布 preserve/editable/removed 契约,第四阶段核对用 |
|
||||
| `review-source.json`、`review.json`、`review/` | 可选原始复核文件、路径规范化快照及对应 masks |
|
||||
| `tasks/<id>/request-plan.json` / `request.json` | 计划 / 实际请求;记录有序图片角色、哈希、尺寸、mask 语义和发送状态 |
|
||||
| `tasks/<id>/capability-evidence.json` | 仅能力探针:请求接受情况及原始越界指标,能力需人工确认 |
|
||||
| `tasks/<id>/response.raw.json` / `request-metadata.json` | 在线响应留档 / 请求 ID、耗时等元数据 |
|
||||
| `tasks/<id>/response.error.json` | 可获取时保存的脱敏请求失败诊断 |
|
||||
| `tasks/<id>/candidate-N.*` | 模型或离线导入的原始候选,拒绝也保留 |
|
||||
| `tasks/<id>/merged-N-normalized.png` | uniform 实际重采样时保存的候选;不代表通过 |
|
||||
| `tasks/<id>/qa-N.json` | 尺寸、对齐、新增/丢弃像素、各区域完成情况、拒绝原因 |
|
||||
| `tasks/<id>/merged-N.png` | 通过几何检查且恢复原可见像素后的局部 RGBA |
|
||||
| `tasks/<id>/merged-N.png` | 原像素尺寸的局部 RGBA;adaptive 已逆采样并恢复 preserve |
|
||||
| `parts/<id>.png` | 每部件选中结果的紧裁剪透明 PNG |
|
||||
| `completion-result.json` | 每部件状态、候选 QA、原图 crop_xyxy 和路径 |
|
||||
| `completion-result.json` | 每部件状态、候选 QA、原图 crop_xyxy、preservation 路径和变换 |
|
||||
| `report.html` / `manifest.json` | 人工检查页 / 模式、网络标记、状态、输入/代码/产物哈希 |
|
||||
|
||||
parts/ 只含本次选中且通过检查的补全部件,不是全部 SAM 部件集合。无需补全、失败和未选中的部件不会自动复制进来,也没有完整角色组装图。
|
||||
parts/ 只含本次选中且通过检查的补全部件,不是全部 SAM 部件集合。无需补全、失败和未选中的部件不会自动复制进来;完整角色组装由第四阶段读取 SAM 与本目录后生成。
|
||||
|
||||
当前实验:`output/20260918-183532/complete2_gpt` 已真实请求 Tiantoken gpt-image-2 共 12 个部件。旧请求未传 size,返回 1248/1254 方形图,预期为 1024,全部严格尺寸拒绝。`complete2_gpt-size-recheck-v2` 离线复用原候选后,8 个几何通过;head、pants_thigh_l、pants_thigh_r、shoe_l 共 4 个因对齐或无有效新增拒绝。通过项中仍有衣料误分、错误手部和未补全子区域。显式 size 的请求构造已验证,但尚无修复后中转遵守 size 的新实测结论。
|
||||
|
||||
2026-09-22 adaptive 验证:Conda ai-spine 下 npm test 共 29 项通过(包括 8 项新 Python adaptive 用例),覆盖 board/独立参考顺序、mask mock、放大与缩小回贴、位移拒绝、实际孔洞、附件清理、人工编辑契约及第四阶段接续。`output/completion-adaptive-prepare-20260922` 使用既有 SAM 数据离线生成 16 个任务、11 个跳过,上衣规划 3708 个孔洞像素、头部规划 10205 个底图清理像素。它们是准备阶段的候选区域,不代表补全质量;本轮未请求付费模型,也未验证中转真实能力。
|
||||
|
||||
## 8. 当前阶段可改进点说明
|
||||
|
||||
- **补全后输出使用补全部件的完整角色。** 按原图坐标和遮挡顺序组装,复用无需补全的原拆件,明确标记失败项及回退来源;输出补全前后对比、接缝/重叠诊断。不能只展示少数 parts 文件作为完整结果。
|
||||
- **把原图或部件相关原图区域交给补全模型参考。** 接入并实测全角色/局部上下文参考和编辑 mask,记录实际发送内容。不能仅把 capabilities 改为 true 就宣称中转支持;需验证图片顺序、mask 语义、返回尺寸和画面保持能力。
|
||||
- **根据 SAM 实际结果重建需求。** 区分分割缺口修复、遮挡补全、关节延伸、附件底图清理;对应建立保留区域与可编辑区域,避免误分像素被永久保护。
|
||||
- **改进画布与补全区域。** 固定 32 像素不适配所有部件;小部件在 1024 大白底中占比低。可验证按部件设计区域、显式局部放大并记录逆变换,保证回贴坐标可检查。
|
||||
- **完整角色组装已接入第四阶段。** 按原坐标与遮挡约束组装,复用原拆件并记录失败回退,输出补全前后对比、覆盖与重叠诊断及单骨骼旋转检查动画。可靠的接缝与运动质量自动验收仍待实现。
|
||||
- **参考输入已实现,真实能力待验证。** 支持 board 与独立 local/full/both、mask 发送策略、有序请求记录及单部件能力探针。离线 mock 已覆盖图片顺序、mask、返回尺寸和越界记录;本轮未请求付费模型,未宣称中转支持或生成质量提升。
|
||||
- **实际 mask 需求重建已实现。** 分别规划缺口、遮挡、关节和附件清理,保存 preserve/editable/removed,并支持人工复核区域。保守孔洞检测不会自动解决所有开放缺口、误分与语义歧义;这些仍需复核。
|
||||
- **自适应画布与区域已实现。** 半径随部件尺寸变化并可覆盖;小部件放大、大部件缩小,请求与原图间变换留档,最终恢复原像素尺寸和保护区。离线测试覆盖放大/缩小、Alpha 保真、位移拒绝及第四阶段接续;真实模型效果待小样验证。
|
||||
- **加强验收。** 对必需区域定义有效覆盖要求,增加部件语义、接缝、材质、回贴和关节旋转露底检查;不能放宽阈值来掩盖重绘、位移或错误输入。
|
||||
- **按小样验证后批量。** 先修正 SAM 输入,再用头、前臂、上衣验证参考图与遮罩,比较原输入、原候选、合并图和回贴图,确认改善后再跑全量。
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
# 第四阶段:角色组装与 Spine 导出
|
||||
|
||||
## 1. 阶段简介
|
||||
|
||||
将第二阶段的可见部件与第三阶段通过几何检查的补全部件整合,按原图坐标组装,并依据 v2 语义计划生成 Spine 4.2 骨架、region 附件、图集及预览。所有部件名称、数量、骨骼父子关系和附件绑定来自输入,不使用角色名称或人形比例模板。
|
||||
|
||||
生成过程完全离线,不调用图片模型、不运行 SAM。骨架预览页面嵌入资产,但会从 CDN 加载 Spine Player 4.2,打开播放器需要网络。静态报告与 PNG 可离线查看。
|
||||
|
||||
导出完成仍标记 `quality_status=needs_review`。当前只有 setup 与单骨骼旋转检查动画;没有成品 idle/walk,也没有 mesh、权重或 IK。关节、语义、接缝及运动质量需人工复核。
|
||||
|
||||
## 2. 输入、输出与执行链路
|
||||
|
||||
```text
|
||||
SAM 目录 + 可选补全目录 + 可选绘制顺序覆盖
|
||||
→ 校验 v2 计划与骨骼关系
|
||||
→ 对比两阶段 source / 计划 / segmentation / masks 的 SHA-256
|
||||
→ 合并部件:已选补全 / 原 SAM 像素 / 隐藏无纹理节点
|
||||
→ 根据遮挡约束稳定拓扑排序,记录没有约束的重叠
|
||||
→ 按 crop_xyxy 组装,输出来源与覆盖诊断
|
||||
→ 根据 parent / bind_to / pivot_norm 创建骨架与附件
|
||||
→ 生成 skeleton.json、图集、播放器、报告与 manifest
|
||||
```
|
||||
|
||||
SAM 原部件直接由 source + mask 重建,保持 RGB/Alpha 和原坐标;无需依赖 SAM 的紧裁剪 parts/。补全部件使用 `completion-result.json` 中选中结果的 `image` 和 `crop_xyxy`,同时检查尺寸及可见 RGBA 是否保持不变。
|
||||
|
||||
## 3. 阶段执行命令示例及参数说明
|
||||
|
||||
在项目根目录运行;环境统一为 Conda `ai-spine`。以下路径仅是现有输入示例,换角色时替换输入目录,不修改代码。输出目录每次独立。
|
||||
|
||||
```powershell
|
||||
Set-Location F:\Study\taonier\AI_spine
|
||||
conda activate ai-spine
|
||||
$samDir = '.\output\20260920-225137\sam'
|
||||
$completionDir = '.\output\20260920-225137\complete'
|
||||
$spineOut = '.\output\spine-' + (Get-Date -Format 'yyyyMMdd-HHmmss')
|
||||
|
||||
node .\tools\model-check.mjs --stage spine `
|
||||
--input-dir $samDir `
|
||||
--completion-dir $completionDir `
|
||||
--out-dir $spineOut
|
||||
|
||||
Start-Process "$spineOut\report.html"
|
||||
```
|
||||
|
||||
| 参数 | 含义 |
|
||||
| --- | --- |
|
||||
| `--input-dir DIR` | 必需,SAM 输出目录 |
|
||||
| `--completion-dir DIR` | 可选,同批输入的补全输出;省略则使用 SAM 资产 |
|
||||
| `--out-dir DIR` | 独立输出目录;dry-run 时可省略 |
|
||||
| `--fallback-policy sam/error` | 默认 sam;需要补全但没有通过结果时回退可见像素并记录原因。error 则终止 |
|
||||
| `--draw-order-file FILE` | JSON 对象,`draw_order` 包含所有可绘制 ID,按从后到前排列,各出现一次 |
|
||||
| `--check-angle FLOAT` | 单骨骼检查动画正负角度,默认 10,范围 (0,45] |
|
||||
| `--genielabs-dir DIR` | 默认相邻 `spine-animation-ai`;复用其图集和播放器脚本,保留 PolyForm Noncommercial 约束 |
|
||||
| `--python FILE` | 仅允许 ai-spine 解释器 |
|
||||
| `--dry-run` | 完整检查输入、PNG、坐标及依赖脚本,不写输出,不联网 |
|
||||
| `--resume` | 仅复用输入、参数、代码与产物哈希全部匹配的完整运行 |
|
||||
|
||||
该阶段不读取模型配置与密钥,也不接受 `--config`、模型参数或补全请求参数。自动化可用 `conda run --no-capture-output -n ai-spine node ...`。
|
||||
|
||||
可选 runtime 复检:将已有的官方 Spine 4.2 core IIFE 文件路径赋给 `$runtimeFile`,执行以下命令。工具不下载或安装依赖;检查图集尺寸、静态附件顶点和正负旋转下的后代跟随,不验证浏览器 WebGL 渲染。
|
||||
|
||||
```powershell
|
||||
$runtimeFile = '.\output\spine-runtime-validation-20260922\spine-core.js'
|
||||
conda run -n ai-spine node .\tools\verify-spine-runtime.mjs `
|
||||
--runtime-file $runtimeFile --input-dir $spineOut
|
||||
```
|
||||
|
||||
## 4. 输入格式要求
|
||||
|
||||
SAM 目录需含 `source.png`、有效 v2 `semantic-parts.json`、`segmentation.json`、其中记录的全画布二值 masks。可见部件缺少 mask 直接报错;hidden 不得有可见 mask。
|
||||
|
||||
补全目录需含相同输入快照、对应 masks 和 `completion-result.json`。只有 `selected_file` 非空、`qa.accepted=true` 且 `status=geometry_passed_needs_review` 的结果会被采用;其 `image` 文件必须存在,裁剪坐标为原画布内的整数半开区间,尺寸与 PNG 一致。legacy 可见像素必须逐像素匹配 SAM;adaptive 使用 preservation 的 preserve/editable/removed 全画布二值区域,要求 preserve=SAM−editable−removed、编辑与移除不重叠、保留像素精确匹配、输出没有超出保留/编辑区域。输入哈希不一致时拒绝混用。图片和契约路径必须位于补全目录内。
|
||||
|
||||
补全失败、未处理、未提供或没有允许区域的需补全部件遵守 fallback-policy;adaptive 已重建并尝试修复的部件也视为有需求,不因原计划 needs_completion=false 就忽略失败。hidden 保留语义骨骼但不生成纹理,也不因缺纹理触发严格回退错误。
|
||||
|
||||
可选绘制顺序文件格式为 `{"draw_order":["部件ID一","部件ID二"]}`。必须列出本次全部可绘制部件,不包含 hidden。显式顺序允许覆盖模型遮挡关系,冲突会在报告中列出。
|
||||
|
||||
## 5. 执行链路中重要部分代码文件及原理说明(包括重点说明)
|
||||
|
||||
**资产整合。** 按 ID 合并,保留来源、跳过或拒绝原因、补全 QA 与上游告警。通过几何检查的补全仍是待复核资产,失败结果不会当作补全成功使用。缺少图片或非法坐标不会被静默回退掩盖。
|
||||
|
||||
**位置与骨骼。** 图像坐标原点为左上角,Spine 原点设为原画布底边中点:`X=x-W/2`,`Y=H-y`。pivot 使用连续坐标 `x=pivot_norm[0]*W`、`y=pivot_norm[1]*H`。所有骨骼 setup 旋转为零、缩放为一,因此子骨骼局部位置是世界 pivot 减父 pivot;附件位置是裁剪中心减绑定骨骼 pivot。骨骼按父先子后输出,hidden bone_part 也创建骨骼。attachment/facial_feature 绑定 bind_to,null 绑定 root,不创建额外独立骨骼。
|
||||
|
||||
**绘制顺序。** `A.occluded_by` 包含 B 时,A 排在 B 后方。排序保留隐藏节点的传递约束,最终只输出有纹理的 slot。没有约束时使用计划中的稳定顺序;若这些部件确有 Alpha 重叠,记录 unconstrained_overlaps,不能声称已确定遮挡。循环关系拒绝导出。
|
||||
|
||||
**检查动画。** 每个语义骨骼生成 `check__ID`:0→正角度→0→负角度→0,时长 2 秒,子骨骼与附件按层级跟随。setup 是静态姿态。它们供人工检查绑定和露底,不自动判定动作质量。
|
||||
|
||||
Spine 字段依据 [官方 JSON 格式说明](https://esotericsoftware.com/spine-json-format),导出目标为 4.2;旋转时间线使用 4.2 的 `value` 字段。
|
||||
|
||||
## 6. 各个代码文件职责索引
|
||||
|
||||
| 文件 | 职责 |
|
||||
| --- | --- |
|
||||
| `tools/model-check.mjs` | 独立离线入口、参数与其他阶段隔离 |
|
||||
| `stages/spine-stage.mjs` | 验证计划、Conda 调用、指纹、导出调度与复用 |
|
||||
| `spine_assembly.py` | 资产核对、顺序、组装、诊断、骨架及报告 |
|
||||
| `lib/validate-parts.mjs` | v2 Schema、引用、骨骼和遮挡环路校验 |
|
||||
| `lib/run-artifacts.mjs` | 独立输出目录和产物哈希 |
|
||||
| 外部 `scripts/make_atlas.py` | 透明 PNG 图集打包,不修改外部依赖 |
|
||||
| 外部 `scripts/generate_spine_player.py` | 嵌入骨架和图集的 Spine Player HTML |
|
||||
| `tests/test_spine_assembly.py` / `tests/spine-stage.test.mjs` | 非人形合成数据、坐标、父子绑定、输入身份、回退、CLI 与完整导出回归 |
|
||||
| `tools/verify-spine-runtime.mjs` | 可选官方 runtime 加载、图集匹配与世界顶点/旋转复检,需提供本地 core IIFE |
|
||||
|
||||
## 7. 产物详细说明
|
||||
|
||||
| 文件 | 用途 |
|
||||
| --- | --- |
|
||||
| `source.png` / `semantic-parts.json` | 原图与语义计划快照 |
|
||||
| `images/<id>.png` | 完整可绘制部件集合,供 Spine 编辑器导入 |
|
||||
| `assembly.json` | 坐标、来源、回退、顺序、QA、输入哈希与覆盖诊断 |
|
||||
| `assembled-sam.png` / `assembled.png` | 同一绘制顺序的原 SAM / 最终资产透明组装图 |
|
||||
| `comparison.png` | 左原图、中 SAM 组装、右最终组装 |
|
||||
| `coverage-diagnostic.png` | 红色为原图前景未覆盖,黄色为多部件重叠;重叠不一定是错误 |
|
||||
| `skeleton.json` | Spine 4.2 骨骼、slot、region、setup 和关节检查动画;images 路径为 `./images/` |
|
||||
| `atlas/skeleton.atlas` / `atlas/skeleton.png` | 运行时图集 |
|
||||
| `preview.html` | 真正的骨架预览,可切换检查动画;需要联网加载播放器 |
|
||||
| `report.html` | 静态对比、诊断、来源列表与预览入口 |
|
||||
| `manifest.json` | 执行状态、质量状态、输入/代码/产物哈希、计数、依赖及限制 |
|
||||
|
||||
2026-09-22 验证记录:`conda run --no-capture-output -n ai-spine npm test` 的 26 项测试全部通过,包含 6 项新增 Python 组装用例。`output/spine-validation-latest-20260922` 和 `output/spine-validation-baseline-20260922` 分别导出 25/22 个 slot、各 16 个骨骼(含 root);官方 Spine 4.2 runtime 对两者的 setup 顶点与各 30 个旋转采样验证通过。最新导出的播放器另经 headless Chrome 实际加载,显示 Loaded successfully 并渲染角色,截图位于 `output/spine-runtime-validation-20260922/preview.png`。这不代表资产或动作质量合格,现有输入的漏分仍显示在组装对比图中。
|
||||
|
||||
## 8. 当前阶段可改进点说明
|
||||
|
||||
- 待实现:针对未确定遮挡、pivot 和绑定的交互修正界面;目前可通过语义计划与绘制顺序文件调整。
|
||||
- 待实现:可靠的接缝、露底与旋转质量自动验收;目前输出检查动画与诊断,质量状态始终 needs_review。
|
||||
- 待实现:mesh、权重、IK,以及由动作语义驱动的成品动画;不通过固定部件名称硬套人形模板。
|
||||
- 待验证:不同角色类别与更复杂遮挡关系的资产质量。合成回归只验证数据契约及几何逻辑。
|
||||
- 待实现:本地播放器资源打包,使骨架预览也可完全离线打开。
|
||||
@@ -1,5 +1,8 @@
|
||||
Complete only the hidden portion of the specified character part on the first
|
||||
input image (the local canvas). Any subsequent images are context references.
|
||||
Repair and complete the specified character part on the first input image.
|
||||
The task declares editable regions for segmentation gaps, occlusion, joint
|
||||
extensions, and removal of overlaid attachments from their underlying base.
|
||||
Only regions explicitly marked editable may be changed. Preserve all other
|
||||
visible pixels. Any subsequent images or labeled reference panels are context.
|
||||
Preserve the exact original pose, scale, screen direction, perspective, colors,
|
||||
line thickness and shading. Do not mirror, rotate, resize, recenter or redesign.
|
||||
Keep the canvas dimensions and the position of every already-visible feature.
|
||||
@@ -8,5 +11,6 @@ body parts, props, labels, annotations or extra versions of the same part.
|
||||
If an edit mask is supplied, transparent mask pixels are editable. Otherwise,
|
||||
use the task's bounded extension instructions and keep the remaining background
|
||||
pure white or transparent. Return the same canvas size without recentering.
|
||||
Downstream checks will reject misalignment, restrict additions to allowed regions,
|
||||
and restore the source visible pixels. Do not rely on restoration to hide changes.
|
||||
Downstream checks reject misalignment and constrain modifications to edit regions,
|
||||
restore protected source pixels, and reverse the documented canvas transform.
|
||||
Do not rely on restoration to hide movement or redesign.
|
||||
|
||||
@@ -0,0 +1,345 @@
|
||||
"""Offline asset assembly and Spine 4.2 region export, driven by semantic v2 data."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import html
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
def write_json(file, data):
|
||||
Path(file).write_text(json.dumps(data, ensure_ascii=False, indent=2, allow_nan=False) + '\n', encoding='utf-8')
|
||||
|
||||
|
||||
def safe_file(root, name):
|
||||
if not isinstance(name, str) or not name or Path(name).is_absolute():
|
||||
raise ValueError(f'Expected relative asset path: {name}')
|
||||
root = Path(root).resolve()
|
||||
file = (root / name).resolve()
|
||||
if not file.is_relative_to(root) or not file.is_file():
|
||||
raise ValueError(f'Missing asset or path outside input directory: {name}')
|
||||
return file
|
||||
|
||||
|
||||
def ordered(nodes, edges):
|
||||
"""Stable topological order; do not silently break contradictory constraints."""
|
||||
pending = list(nodes)
|
||||
result = []
|
||||
while pending:
|
||||
ready = next((n for n in pending if not any(b == n and a in pending for a, b in edges)), None)
|
||||
if ready is None:
|
||||
raise ValueError('Cyclic bone or draw-order constraints')
|
||||
pending.remove(ready)
|
||||
result.append(ready)
|
||||
return result
|
||||
|
||||
|
||||
def load_assets(input_dir, completion_dir=None, fallback_policy='sam', draw_order_file=None):
|
||||
root = Path(input_dir).resolve()
|
||||
hashes = {}
|
||||
|
||||
def tracked(folder, name, label):
|
||||
file = safe_file(folder, name)
|
||||
hashes[f'{label}/{name}'] = hashlib.sha256(file.read_bytes()).hexdigest()
|
||||
return file
|
||||
|
||||
def read(folder, name, label):
|
||||
return json.loads(tracked(folder, name, label).read_text(encoding='utf-8-sig'))
|
||||
|
||||
source = Image.open(tracked(root, 'source.png', 'sam')).convert('RGBA')
|
||||
rgba = np.array(source)
|
||||
w, h = source.size
|
||||
plan = read(root, 'semantic-parts.json', 'sam')
|
||||
seg = read(root, 'segmentation.json', 'sam')
|
||||
parts = plan['parts']
|
||||
by_id = {p['id']: p for p in parts}
|
||||
if len(by_id) != len(parts):
|
||||
raise ValueError('Duplicate semantic ID')
|
||||
|
||||
def records(items, label):
|
||||
result = {}
|
||||
for item in items:
|
||||
name = item['id']
|
||||
if name not in by_id or name in result:
|
||||
raise ValueError(f'{label}: unknown or duplicate ID: {name}')
|
||||
result[name] = item
|
||||
return result
|
||||
|
||||
segmented = records(seg['parts'], 'segmentation')
|
||||
completed, skipped = {}, {}
|
||||
completion = Path(completion_dir).resolve() if completion_dir else None
|
||||
# A completion run snapshots its SAM inputs, so compare content, not directory names.
|
||||
if completion:
|
||||
for name in ['source.png', 'semantic-parts.json', 'segmentation.json']:
|
||||
tracked(completion, name, 'completion')
|
||||
if hashes[f'sam/{name}'] != hashes[f'completion/{name}']:
|
||||
raise ValueError(f'Completion input mismatch: {name}')
|
||||
result = read(completion, 'completion-result.json', 'completion')
|
||||
completed = records(result['parts'], 'completion')
|
||||
skipped = records(result.get('skipped', []), 'completion skipped')
|
||||
if set(completed) & set(skipped):
|
||||
raise ValueError('Completion ID occurs in both parts and skipped')
|
||||
|
||||
images, original_images, asset_records = {}, {}, []
|
||||
masks = {}
|
||||
for name, record in segmented.items():
|
||||
if by_id[name]['visibility'] == 'hidden':
|
||||
raise ValueError(f'Hidden part unexpectedly has a SAM mask: {name}')
|
||||
mask_name = record.get('mask', f'masks/{name}.png')
|
||||
mask = np.array(Image.open(tracked(root, mask_name, 'sam')))
|
||||
if mask.shape != (h, w) or not np.isin(mask, [0, 255]).all():
|
||||
raise ValueError(f'Invalid full-canvas binary mask: {name}')
|
||||
mask = mask > 0
|
||||
if not mask.any() or np.any(mask & (rgba[:, :, 3] == 0)):
|
||||
raise ValueError(f'Empty mask or mask outside source alpha: {name}')
|
||||
if completion:
|
||||
tracked(completion, mask_name, 'completion')
|
||||
if hashes[f'sam/{mask_name}'] != hashes[f'completion/{mask_name}']:
|
||||
raise ValueError(f'Completion mask mismatch: {name}')
|
||||
masks[name] = mask
|
||||
|
||||
for part in parts:
|
||||
name = part['id']
|
||||
if part['visibility'] == 'hidden':
|
||||
asset_records.append({'id': name, 'source': 'hidden', 'image': None,
|
||||
'reason': 'HIDDEN_NO_TEXTURE', 'semantic_verified': False})
|
||||
continue
|
||||
if name not in masks:
|
||||
raise ValueError(f'Visible part has no SAM mask: {name}')
|
||||
mask = masks[name]
|
||||
ys, xs = np.nonzero(mask)
|
||||
original_box = [int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1]
|
||||
cut = rgba.copy()
|
||||
cut[~mask] = 0
|
||||
original = Image.fromarray(cut).crop(original_box)
|
||||
original_images[name] = (original, original_box)
|
||||
candidate = completed.get(name, {})
|
||||
accepted = bool(candidate.get('selected_file')) and candidate.get('qa', {}).get('accepted') is True
|
||||
warnings = list(segmented[name].get('warnings', []))
|
||||
if accepted:
|
||||
if candidate.get('status') != 'geometry_passed_needs_review':
|
||||
raise ValueError(f'Inconsistent selected completion status: {name}')
|
||||
image = Image.open(tracked(completion, candidate.get('image'), 'completion')).convert('RGBA')
|
||||
box = candidate.get('crop_xyxy')
|
||||
if (not isinstance(box, list) or len(box) != 4 or
|
||||
any(type(x) is not int for x in box) or
|
||||
not (0 <= box[0] < box[2] <= w and 0 <= box[1] < box[3] <= h) or
|
||||
image.size != (box[2] - box[0], box[3] - box[1])):
|
||||
raise ValueError(f'Invalid completion crop/size: {name}')
|
||||
full = Image.new('RGBA', source.size)
|
||||
full.paste(image, (box[0], box[1]))
|
||||
preserve = mask
|
||||
preservation = candidate.get('preservation')
|
||||
if preservation:
|
||||
areas = {}
|
||||
for key in ['preserve_mask', 'editable_mask', 'removed_mask']:
|
||||
area = np.array(Image.open(tracked(completion, preservation.get(key), 'completion')))
|
||||
if area.shape != (h, w) or not np.isin(area, [0, 255]).all():
|
||||
raise ValueError(f'Invalid completion preservation mask: {name}/{key}')
|
||||
areas[key] = area > 0
|
||||
preserve, editable, removed = (areas[k] for k in ['preserve_mask', 'editable_mask', 'removed_mask'])
|
||||
if (np.any(editable & removed) or np.any(removed & ~mask) or
|
||||
not np.array_equal(preserve, mask & ~editable & ~removed) or not preserve.any()):
|
||||
raise ValueError(f'Inconsistent completion preservation partition: {name}')
|
||||
alpha = np.array(full)[:, :, 3] > 0
|
||||
if np.any(alpha & ~(preserve | editable)):
|
||||
raise ValueError(f'Completion pixels outside preservation/edit contract: {name}')
|
||||
warnings.append('VISIBLE_EDITS_DECLARED_NEEDS_REVIEW')
|
||||
if not np.array_equal(np.array(full)[preserve], rgba[preserve]):
|
||||
raise ValueError(f'Completion does not preserve original visible RGBA: {name}')
|
||||
origin, reason = 'completion', 'GEOMETRY_PASSED_NEEDS_REVIEW'
|
||||
if candidate.get('qa', {}).get('unfulfilled_regions'):
|
||||
warnings.append('UNFULFILLED_COMPLETION_REGIONS')
|
||||
else:
|
||||
image, box = original, original_box
|
||||
origin = 'sam'
|
||||
inferred_need = bool(candidate) or skipped.get(name, {}).get('reason') == 'AMBIGUOUS_MASK_REQUIRES_REVIEW'
|
||||
if not part['needs_completion'] and not inferred_need:
|
||||
reason = 'NO_COMPLETION_NEEDED'
|
||||
else:
|
||||
reason = skipped.get(name, {}).get('reason') or (
|
||||
'COMPLETION_' + str(candidate.get('status', 'NOT_PROVIDED')).upper())
|
||||
warnings.append('USING_UNCOMPLETED_SAM')
|
||||
if fallback_policy == 'error':
|
||||
raise ValueError(f'Completion required for {name}: {reason}')
|
||||
images[name] = (image, box)
|
||||
asset_records.append({'id': name, 'source': origin, 'reason': reason, 'image': f'images/{name}.png',
|
||||
'crop_xyxy': box, 'original_crop_xyxy': original_box,
|
||||
'semantic_verified': False, 'warnings': warnings,
|
||||
'completion_status': candidate.get('status'),
|
||||
'preservation': candidate.get('preservation'),
|
||||
'completion_qa': candidate.get('qa')})
|
||||
|
||||
if not images:
|
||||
raise ValueError('No drawable assets')
|
||||
edges = [(p['id'], front) for p in parts for front in p['occluded_by']]
|
||||
# Keep hidden nodes while sorting so transitive constraints survive their removal.
|
||||
draw_order = [n for n in ordered(by_id, edges) if n in images]
|
||||
overridden = False
|
||||
if draw_order_file:
|
||||
file = Path(draw_order_file).resolve()
|
||||
hashes['draw_order_override'] = hashlib.sha256(file.read_bytes()).hexdigest()
|
||||
override = json.loads(file.read_text(encoding='utf-8-sig'))
|
||||
draw_order = override['draw_order']
|
||||
if not isinstance(draw_order, list) or len(draw_order) != len(images) or set(draw_order) != set(images):
|
||||
raise ValueError('draw_order must contain each drawable ID exactly once')
|
||||
overridden = True
|
||||
reach = {n: set() for n in by_id}
|
||||
for n in reversed(ordered(by_id, edges)):
|
||||
for a, b in edges:
|
||||
if a == n:
|
||||
reach[n].add(b)
|
||||
reach[n].update(reach[b])
|
||||
unresolved = []
|
||||
for i, a in enumerate(draw_order):
|
||||
for b in draw_order[i + 1:]:
|
||||
if b in reach[a] or a in reach[b]:
|
||||
continue
|
||||
ai, ab = images[a]
|
||||
bi, bb = images[b]
|
||||
x0, y0, x1, y1 = max(ab[0], bb[0]), max(ab[1], bb[1]), min(ab[2], bb[2]), min(ab[3], bb[3])
|
||||
if x0 < x1 and y0 < y1:
|
||||
am = np.array(ai.getchannel('A'))[y0-ab[1]:y1-ab[1], x0-ab[0]:x1-ab[0]] > 0
|
||||
bm = np.array(bi.getchannel('A'))[y0-bb[1]:y1-bb[1], x0-bb[0]:x1-bb[0]] > 0
|
||||
overlap = int((am & bm).sum())
|
||||
if overlap:
|
||||
unresolved.append({'parts': [a, b], 'overlap_pixels': overlap})
|
||||
violated = [[a, b] for a in draw_order for b in reach[a]
|
||||
if b in images and draw_order.index(a) > draw_order.index(b)]
|
||||
assembly = {'schema_version': 1, 'quality_status': 'needs_review', 'semantic_verified': False,
|
||||
'width': w, 'height': h, 'coordinate_space': 'pixel_xy_top_left',
|
||||
'parts': asset_records, 'draw_order': draw_order,
|
||||
'draw_order_source': 'override' if overridden else 'occlusion_stable_topological',
|
||||
'unconstrained_overlaps': unresolved, 'overridden_occlusion_constraints': violated,
|
||||
'input_hashes': hashes}
|
||||
return source, plan, assembly, images, original_images
|
||||
|
||||
|
||||
def build_skeleton(plan, assembly, check_angle=10):
|
||||
w, h = assembly['width'], assembly['height']
|
||||
bone_parts = {p['id']: p for p in plan['parts'] if p['rig_role'] == 'bone_part'}
|
||||
edges = [(p['parent'], name) for name, p in bone_parts.items() if p['parent']]
|
||||
bone_order = ordered(bone_parts, edges)
|
||||
world = {'root': (0, 0)}
|
||||
bones = [{'name': 'root'}]
|
||||
for name in bone_order:
|
||||
part = bone_parts[name]
|
||||
world[name] = (part['pivot_norm'][0] * w - w / 2, h - part['pivot_norm'][1] * h)
|
||||
parent = part['parent'] or 'root'
|
||||
x, y = world[name]
|
||||
px, py = world[parent]
|
||||
# Setup axes remain aligned with the canvas. Parent rotation still propagates
|
||||
# to children; attachment offsets below preserve the exact original pose.
|
||||
bones.append({'name': name, 'parent': parent, 'x': x - px, 'y': y - py})
|
||||
semantic = {p['id']: p for p in plan['parts']}
|
||||
assets = {p['id']: p for p in assembly['parts']}
|
||||
slots, attachments = [], {}
|
||||
for name in assembly['draw_order']:
|
||||
part, asset = semantic[name], assets[name]
|
||||
bone = name if part['rig_role'] == 'bone_part' else part['bind_to'] or 'root'
|
||||
bx, by = world[bone]
|
||||
x0, y0, x1, y1 = asset['crop_xyxy']
|
||||
slots.append({'name': name, 'bone': bone, 'attachment': name})
|
||||
attachments[name] = {name: {'type': 'region', 'path': name,
|
||||
'x': (x0 + x1) / 2 - w / 2 - bx,
|
||||
'y': h - (y0 + y1) / 2 - by,
|
||||
'width': x1 - x0, 'height': y1 - y0}}
|
||||
animations = {'setup': {}}
|
||||
for name in bone_order:
|
||||
animations[f'check__{name}'] = {'bones': {name: {'rotate': [
|
||||
{'time': 0, 'value': 0}, {'time': .5, 'value': check_angle},
|
||||
{'time': 1, 'value': 0}, {'time': 1.5, 'value': -check_angle}, {'time': 2, 'value': 0}]}}}
|
||||
return {'skeleton': {'spine': '4.2.0', 'x': -w / 2, 'y': 0, 'width': w, 'height': h,
|
||||
'images': './images/'},
|
||||
'bones': bones, 'slots': slots, 'skins': [{'name': 'default', 'attachments': attachments}],
|
||||
'animations': animations}
|
||||
|
||||
|
||||
def compose(size, images, order):
|
||||
canvas = Image.new('RGBA', size)
|
||||
coverage = np.zeros((size[1], size[0]), np.uint16)
|
||||
for name in order:
|
||||
image, (x0, y0, x1, y1) = images[name]
|
||||
canvas.alpha_composite(image, (x0, y0))
|
||||
coverage[y0:y1, x0:x1] += np.array(image.getchannel('A')) > 0
|
||||
return canvas, coverage
|
||||
|
||||
|
||||
def export_assets(data, output, check_angle=10):
|
||||
source, plan, assembly, images, original_images = data
|
||||
out = Path(output)
|
||||
(out / 'images').mkdir(parents=True, exist_ok=True)
|
||||
source.save(out / 'source.png')
|
||||
write_json(out / 'semantic-parts.json', plan)
|
||||
for name, (im, _) in images.items():
|
||||
im.save(out / 'images' / f'{name}.png')
|
||||
assembled, coverage = compose(source.size, images, assembly['draw_order'])
|
||||
before, _ = compose(source.size, original_images, assembly['draw_order'])
|
||||
assembled.save(out / 'assembled.png')
|
||||
before.save(out / 'assembled-sam.png')
|
||||
foreground = np.array(source.getchannel('A')) > 0
|
||||
missing = foreground & (coverage == 0)
|
||||
diagnostic = np.zeros((source.height, source.width, 4), np.uint8)
|
||||
diagnostic[coverage > 1] = [255, 180, 0, 220]
|
||||
diagnostic[missing] = [255, 0, 80, 255]
|
||||
Image.fromarray(diagnostic).save(out / 'coverage-diagnostic.png')
|
||||
comparison = Image.new('RGBA', (source.width * 3, source.height), 'white')
|
||||
for i, im in enumerate([source, before, assembled]):
|
||||
comparison.alpha_composite(im, (i * source.width, 0))
|
||||
comparison.convert('RGB').save(out / 'comparison.png')
|
||||
assembly['diagnostics'] = {'uncovered_source_pixels': int(missing.sum()),
|
||||
'overlap_pixels': int((coverage > 1).sum()),
|
||||
'rotation_quality_verified': False}
|
||||
write_json(out / 'assembly.json', assembly)
|
||||
skeleton = build_skeleton(plan, assembly, check_angle)
|
||||
write_json(out / 'skeleton.json', skeleton)
|
||||
rows = []
|
||||
for part in assembly['parts']:
|
||||
picture = f'<img src="{part["image"]}">' if part['image'] else ''
|
||||
rows.append('<tr><td>' + html.escape(part['id']) + '</td><td>' + html.escape(part['source']) +
|
||||
'</td><td>' + html.escape(part['reason']) + '</td><td>' + picture + '</td></tr>')
|
||||
notes = {key: assembly[key] for key in ['diagnostics', 'draw_order_source', 'unconstrained_overlaps',
|
||||
'overridden_occlusion_constraints']}
|
||||
(out / 'report.html').write_text('''<!doctype html><html lang="zh-CN"><meta charset="utf-8">
|
||||
<title>第四阶段:Spine 组装与导出</title>
|
||||
<style>body{font:16px system-ui;margin:24px}img{max-width:100%;background:#ddd}td img{max-height:150px}td,th{border:1px solid #ccc;padding:8px}pre{white-space:pre-wrap}</style>
|
||||
<h1>第四阶段:Spine 组装与导出</h1>
|
||||
<p>状态:needs_review。几何通过与成功导出不代表语义、接缝或动作质量合格。</p>
|
||||
<p><a href="preview.html">Spine 预览</a>(播放器从 CDN 加载,需要网络);选择 setup 查看静态姿态,选择 check__骨骼ID 检查单关节旋转及子骨骼跟随。检查动画不代表成品动作。</p>
|
||||
<p><a href="skeleton.json">骨架 JSON</a> · <a href="assembly.json">来源与诊断</a></p>
|
||||
<h2>原图 / SAM 组装 / 最终组装</h2><img src="comparison.png">
|
||||
<h2>覆盖诊断:红色为原图漏分,黄色为部件重叠</h2><img src="coverage-diagnostic.png">
|
||||
<p>重叠可能是正常遮挡;覆盖率不能判断语义。没有遮挡约束的重叠使用语义计划中的稳定顺序,需复核。</p>
|
||||
<pre>''' + html.escape(json.dumps(notes, ensure_ascii=False, indent=2)) + '''</pre>
|
||||
<h2>资产来源</h2><table><tr><th>ID</th><th>来源</th><th>原因</th><th>部件</th></tr>''' +
|
||||
''.join(rows) + '</table></html>', encoding='utf-8')
|
||||
return {'parts': len(images), 'bones': len(skeleton['bones']),
|
||||
'completion_parts': sum(p['source'] == 'completion' for p in assembly['parts']),
|
||||
'fallback_parts': sum('USING_UNCOMPLETED_SAM' in p.get('warnings', []) for p in assembly['parts']),
|
||||
'quality_status': 'needs_review', 'input_hashes': assembly['input_hashes']}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--input-dir', required=True)
|
||||
parser.add_argument('--completion-dir')
|
||||
parser.add_argument('--output')
|
||||
parser.add_argument('--draw-order-file')
|
||||
parser.add_argument('--fallback-policy', choices=['sam', 'error'], default='sam')
|
||||
parser.add_argument('--check-angle', type=float, default=10)
|
||||
args = parser.parse_args()
|
||||
if not 0 < args.check_angle <= 45:
|
||||
parser.error('check-angle must be in (0, 45]')
|
||||
data = load_assets(args.input_dir, args.completion_dir, args.fallback_policy, args.draw_order_file)
|
||||
if args.output:
|
||||
result = export_assets(data, args.output, args.check_angle)
|
||||
else:
|
||||
skeleton = build_skeleton(data[1], data[2], args.check_angle)
|
||||
result = {'input_hashes': data[2]['input_hashes'], 'parts': len(data[3]), 'bones': len(skeleton['bones'])}
|
||||
print(json.dumps(result, ensure_ascii=False))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+96
-15
@@ -23,14 +23,34 @@ function filesIn(dir, prefix = '') {
|
||||
}
|
||||
|
||||
export async function runCompletion({stage, inputDir, outDir, partIds = [], prepareOnly = false,
|
||||
candidatesDir, completionSize = 1024, extensionRadius = 32, python: requestedPython,
|
||||
resume = false, dryRun = false, candidateSizePolicy = 'strict'}, transport) {
|
||||
candidatesDir, completionSize = 1024, extensionRadius, python: requestedPython,
|
||||
resume = false, dryRun = false, candidateSizePolicy = 'strict', completionProfile = 'adaptive',
|
||||
completionReference = 'auto', completionMask = 'auto', completionReview, probeEditCapabilities = false}, transport) {
|
||||
if (!inputDir) throw new Error('部件补全需要 --input-dir 指向 SAM 输出目录');
|
||||
if (!outDir && !dryRun) throw new Error('部件补全需要独立的 --out-dir');
|
||||
if (prepareOnly && candidatesDir) throw new Error('--prepare-only 与 --candidates-dir 不能同时使用');
|
||||
const size = Number(completionSize), radius = Number(extensionRadius);
|
||||
const size = Number(completionSize), radius = extensionRadius === undefined
|
||||
? completionProfile === 'legacy' ? 32 : null : Number(extensionRadius);
|
||||
if (!['legacy', 'adaptive'].includes(completionProfile)) throw new Error('--completion-profile 仅支持 adaptive / legacy');
|
||||
if (!['auto', 'none', 'local', 'full', 'both', 'board'].includes(completionReference)) throw new Error('不支持的 --completion-reference');
|
||||
if (!['auto', 'required', 'off'].includes(completionMask)) throw new Error('--completion-mask 仅支持 auto / required / off');
|
||||
if (probeEditCapabilities && (partIds.length !== 1 || prepareOnly || candidatesDir || completionProfile !== 'adaptive')) {
|
||||
throw new Error('--probe-edit-capabilities 需要 adaptive、恰好一个 --part 和在线请求');
|
||||
}
|
||||
const referenceMode = probeEditCapabilities ? 'both' : completionReference === 'auto'
|
||||
? completionProfile === 'legacy' ? stage.capabilities.multiple_images ? 'local' : 'none'
|
||||
: stage.capabilities.multiple_images ? 'both' : 'board' : completionReference;
|
||||
if (completionProfile === 'legacy' && (completionReview || !['local', 'none'].includes(referenceMode))) {
|
||||
throw new Error('legacy 不支持 review 或 full/both/board 参考,请使用 adaptive');
|
||||
}
|
||||
const nativeReferences = ['local', 'full', 'both'].includes(referenceMode);
|
||||
const maskSent = probeEditCapabilities || completionMask === 'required' || (completionMask === 'auto' && stage.capabilities.mask);
|
||||
const capabilitiesAvailable = (!nativeReferences || stage.capabilities.multiple_images) && (!maskSent || stage.capabilities.mask);
|
||||
if (!capabilitiesAvailable && !probeEditCapabilities && !prepareOnly && !candidatesDir && !dryRun) {
|
||||
throw new Error('请求需要未确认的多图/mask 能力;可先 prepare-only,或用单部件 --probe-edit-capabilities 实测,不要直接更改默认 capabilities');
|
||||
}
|
||||
if (!['strict', 'uniform'].includes(candidateSizePolicy)) throw new Error('--candidate-size-policy 仅支持 strict / uniform');
|
||||
if (!Number.isInteger(size) || size < 128 || size > 4096 || !Number.isInteger(radius) || radius < 1 || radius > 256) {
|
||||
if (!Number.isInteger(size) || size < 128 || size > 4096 || (radius !== null && (!Number.isInteger(radius) || radius < 1 || radius > 256))) {
|
||||
throw new Error('--completion-size 必须为 128~4096 的整数;--extension-radius 必须为 1~256 的整数');
|
||||
}
|
||||
const input = path.resolve(inputDir);
|
||||
@@ -46,15 +66,43 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
const source = readImage(path.join(input, 'source.png'), {pngOnly: true});
|
||||
const inputFiles = ['source.png', 'semantic-parts.json', 'segmentation.json', ...segmentation.parts.map((p) => `masks/${p.id}.png`)];
|
||||
const inputs = Object.fromEntries(inputFiles.map((file) => [file, sha256(readFileSync(path.join(input, file)))]));
|
||||
const reviewFiles = [], reviewHashes = {};
|
||||
let reviewData;
|
||||
if (completionReview) {
|
||||
const file = path.resolve(completionReview);
|
||||
reviewData = readJson(file);
|
||||
if (reviewData.schema_version !== 1 || !reviewData.parts || Array.isArray(reviewData.parts)) throw new Error('review 需要 schema_version=1 和 parts 对象');
|
||||
reviewHashes['review.json'] = sha256(readFileSync(file));
|
||||
reviewFiles.push({file, target: 'review-source.json'});
|
||||
for (const [id, item] of Object.entries(reviewData.parts)) {
|
||||
if (!names.has(id) || !item || item.confirmed !== true) throw new Error('review 部件必须有效且 confirmed=true');
|
||||
for (const field of ['repair_mask', 'replace_mask', 'remove_mask']) {
|
||||
if (!Object.hasOwn(item, field)) continue;
|
||||
const ref = item[field];
|
||||
if (typeof ref !== 'string' || path.isAbsolute(ref)) throw new Error('review mask 必须是相对路径');
|
||||
const maskFile = path.resolve(path.dirname(file), ref), relative = path.relative(path.dirname(file), maskFile);
|
||||
if (relative === '..' || relative.startsWith(`..${path.sep}`) || path.isAbsolute(relative)) throw new Error('review mask 不能越出 review 目录');
|
||||
const target = `review/${id}-${field}.png`;
|
||||
reviewHashes[`${id}/${field}`] = sha256(readFileSync(maskFile));
|
||||
reviewFiles.push({file: maskFile, target});
|
||||
item[field] = target;
|
||||
}
|
||||
}
|
||||
}
|
||||
const planning = {profile: completionProfile, reference_mode: referenceMode, review_hashes: reviewHashes,
|
||||
extension_radius: radius, planner_version: completionProfile === 'adaptive' ? 2 : 1};
|
||||
const imported = {};
|
||||
if (candidatesDir) {
|
||||
const priorManifest = path.resolve(candidatesDir, 'manifest.json');
|
||||
const previousRun = existsSync(priorManifest) ? readJson(priorManifest) : null;
|
||||
const previousPlanning = previousRun?.planning || {profile: 'legacy', reference_mode: previousRun?.resolved_stage?.capabilities?.multiple_images ? 'local' : 'none',
|
||||
review_hashes: {}, extension_radius: previousRun?.extension_radius, planner_version: 1};
|
||||
if (previousRun && (previousRun.output_kind !== 'bounded_part_completion' || previousRun.completion_size !== size || previousRun.extension_radius !== radius ||
|
||||
JSON.stringify(previousPlanning) !== JSON.stringify(planning) ||
|
||||
Object.entries(inputs).some(([file, hash]) => previousRun.inputs?.[file] !== hash))) {
|
||||
throw new Error('旧补全目录的输入或画布/延展参数不一致,不能复用其候选');
|
||||
}
|
||||
for (const p of plan.parts.filter((p) => p.needs_completion && (!partIds.length || partIds.includes(p.id)))) {
|
||||
for (const p of plan.parts.filter((p) => (completionProfile === 'adaptive' || p.needs_completion) && (!partIds.length || partIds.includes(p.id)))) {
|
||||
const file = previousRun
|
||||
? ['png', 'jpg', 'webp'].map((ext) => path.resolve(candidatesDir, 'tasks', p.id, `candidate-1.${ext}`)).find(existsSync)
|
||||
: path.resolve(candidatesDir, `${p.id}.png`);
|
||||
@@ -66,16 +114,18 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
if (dryRun) {
|
||||
console.log(JSON.stringify({stage: 'completion', input, width: source.width, height: source.height,
|
||||
selected_parts: partIds, completion_size: size, extension_radius: radius, candidate_size_policy: candidateSizePolicy,
|
||||
planning, mask_requested: maskSent, capabilities_available: capabilitiesAvailable,
|
||||
mode: prepareOnly ? 'prepare' : candidatesDir ? 'offline_candidates' : 'image_edit', network_requested: false}, null, 2));
|
||||
return {dryRun: true};
|
||||
}
|
||||
const codeFiles = ['stages/completion-stage.mjs', 'completion_geometry.py', 'normalize_reference.py',
|
||||
const codeFiles = ['stages/completion-stage.mjs', 'completion_geometry.py', 'completion_planning.py', 'normalize_reference.py',
|
||||
'lib/config.mjs', 'lib/python-runtime.mjs', 'lib/run-artifacts.mjs', 'lib/images.mjs',
|
||||
'lib/validate-parts.mjs', 'schemas/semantic-parts.schema.json', 'providers/image-edits.mjs', 'providers/http.mjs'];
|
||||
const identity = {schema_version: 1, stage: 'completion', output_kind: 'bounded_part_completion',
|
||||
mode: prepareOnly ? 'prepare' : candidatesDir ? 'offline_candidates' : 'image_edit',
|
||||
resolved_stage: publicStage(stage), endpoint_sha256: sha256(stage.baseUrl), prompt_sha256: sha256(stage.prompt),
|
||||
inputs, part_ids: partIds, completion_size: size, extension_radius: radius, candidate_size_policy: candidateSizePolicy,
|
||||
planning, mask_requested: maskSent, capability_probe: probeEditCapabilities,
|
||||
imported_candidates: Object.fromEntries(Object.entries(imported).map(([id, value]) => [id, value.sha256])),
|
||||
code_sha256: sha256(codeFiles.map((file) => `${file}\n${readFileSync(path.join(projectRoot, file), 'utf8')}`).join('\n'))};
|
||||
const fingerprint = sha256(JSON.stringify(identity));
|
||||
@@ -83,24 +133,51 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
if (run.reused) return {out: run.out, reused: true};
|
||||
const manifest = {...identity, fingerprint, status: 'running', quality_status: 'pending', started_at: new Date().toISOString(), network_requested: false};
|
||||
jsonFile(run.manifestPath, manifest, stage.apiKey);
|
||||
const result = {schema_version: 1, quality_status: 'needs_review', parts: [], skipped: []};
|
||||
const result = {schema_version: completionProfile === 'adaptive' ? 2 : 1, profile: completionProfile,
|
||||
quality_status: 'needs_review', parts: [], skipped: []};
|
||||
try {
|
||||
const requestSize = `${size}x${size}`;
|
||||
if (!prepareOnly && !candidatesDir && stage.params.size !== undefined && stage.params.size !== requestSize) {
|
||||
throw new Error(`completion.params.size 必须与局部画布一致:${requestSize}`);
|
||||
}
|
||||
const editStage = {...stage, params: {...stage.params, size: requestSize}};
|
||||
const editStage = {...stage, params: {...stage.params, size: requestSize},
|
||||
...(probeEditCapabilities ? {capabilities: {multiple_images: true, mask: true}, maxRetries: 0} : {})};
|
||||
if (probeEditCapabilities) editStage.params.n = 1;
|
||||
for (const file of inputFiles) {
|
||||
mkdirSync(path.dirname(path.join(run.out, file)), {recursive: true});
|
||||
copyFileSync(path.join(input, file), path.join(run.out, file));
|
||||
}
|
||||
pythonCall(python, ['prepare', '--out', run.out, '--size', String(size), '--radius', String(radius), ...partIds.flatMap((id) => ['--part', id])]);
|
||||
for (const {file, target} of reviewFiles) {
|
||||
mkdirSync(path.dirname(path.join(run.out, target)), {recursive: true});
|
||||
copyFileSync(file, path.join(run.out, target));
|
||||
}
|
||||
if (reviewData) jsonFile(path.join(run.out, 'review.json'), reviewData);
|
||||
pythonCall(python, ['prepare', '--out', run.out, '--size', String(size), '--profile', completionProfile,
|
||||
'--reference-mode', referenceMode, ...(radius === null ? [] : ['--radius', String(radius)]),
|
||||
...(reviewData ? ['--review-file', path.join(run.out, 'review.json')] : []), ...partIds.flatMap((id) => ['--part', id])]);
|
||||
const prepared = readJson(path.join(run.out, 'completion-plan.json'));
|
||||
result.skipped = prepared.skipped;
|
||||
result.parts = prepared.tasks.map((task) => ({id: task.id, status: 'prepared', upstream_warnings: task.warnings, semantic_verified: false}));
|
||||
if (probeEditCapabilities && prepared.tasks.length !== 1) throw new Error('所选部件没有可执行任务;请检查 completion-plan.json,不会发送能力探针');
|
||||
for (const [index, task] of prepared.tasks.entries()) {
|
||||
const folder = path.join(run.out, 'tasks', task.id);
|
||||
const prompt = `${stage.prompt}\n\nPart: ${task.id}: ${task.semantic}\nCompletion requirements: ${JSON.stringify(task.completion)}\nCanvas: ${size} x ${size} pixels. Keep the part at its current position and native scale.\nExtend only the hidden contour and requested joint connections, by at most ${radius} pixels from the visible silhouette. Keep all other background pure white or transparent. Do not add the occluder or neighboring body parts.\n${stage.capabilities.multiple_images ? 'The second image is local context only; do not copy its other parts into the output.' : 'Only the isolated visible part is provided.'}\nReturn exactly the same canvas dimensions.\n`;
|
||||
const imageRoles = [{file: 'input.png', role: referenceMode === 'board' ? 'target_with_reference_panels' : 'target'}];
|
||||
if (['local', 'both'].includes(referenceMode)) imageRoles.push({file: 'context.png', role: 'local_reference'});
|
||||
if (['full', 'both'].includes(referenceMode)) imageRoles.push({file: 'reference.png', role: 'full_character_reference'});
|
||||
const images = imageRoles.map(({file}) => readImage(path.join(folder, file), {pngOnly: true}));
|
||||
const mask = maskSent ? readImage(path.join(folder, 'edit-mask.png'), {pngOnly: true}) : null;
|
||||
const requestPlan = {model: stage.model, size: requestSize, reference_mode: referenceMode,
|
||||
images: images.map((im) => im.sha256), image_roles: images.map((im, i) => ({...imageRoles[i], index: i, sha256: im.sha256, width: im.width, height: im.height})),
|
||||
image_field: stage.imageField, mask_editable: mask ? stage.maskEditable : null,
|
||||
edit_mask_sent: Boolean(mask), mask_sha256: mask?.sha256 ?? null,
|
||||
mask_enforcement: mask ? 'provider_requested_and_local' : 'local_only',
|
||||
capability_status: probeEditCapabilities ? 'unverified_probe' : 'configured_not_proven_by_this_run',
|
||||
allowed_mask_sha256: sha256(readFileSync(path.join(folder, 'allowed-mask.png')))};
|
||||
jsonFile(path.join(folder, 'request-plan.json'), {...requestPlan, request_sent: false}, stage.apiKey);
|
||||
const referenceInstruction = referenceMode === 'board'
|
||||
? `The LEFT work panel ${JSON.stringify(task.transform.working_xyxy)} is the only output target. RIGHT panels are LOCAL and FULL character references. Keep the entire ${size}x${size} board layout unchanged; never place reference characters in the target panel. Preserve reference panels.`
|
||||
: `Image order: ${imageRoles.map((im, i) => `${i + 1}=${im.role}`).join(', ')}. Only edit the first image. Subsequent images are references, never extra output parts.`;
|
||||
const prompt = `${stage.prompt}\n\nPart: ${task.id}: ${task.semantic}\nCompletion requirements: ${JSON.stringify(task.completion)}\nCanvas: ${size} x ${size} pixels. Preserve the target's exact current position and displayed scale.\nTask regions: ${JSON.stringify(task.regions)}. segmentation_gap: restore source texture missing from segmentation; attachment_cleanup: paint the underlying material WITHOUT the overlaid attachment; reviewed_replacement: repair incorrect pixels; joint/occlusion regions: extend this part only.\n${referenceInstruction}\n${mask ? 'Transparent mask pixels are editable; opaque mask pixels must remain unchanged.' : 'No provider mask is sent. Follow the bounded edit instructions; output is constrained locally afterwards.'}\nReturn exactly the same canvas dimensions.\n`;
|
||||
const fullPrompt = `${prompt}\nOccluded by: ${task.occluded_by.join(', ') || 'not specified'}.\nLocal pixel bounds for proposed extensions (xyxy, exclusive upper bounds): ${JSON.stringify(task.region_bounds_xyxy)}. Null means no supported region for that requirement. These boxes bound the allowed regions; do not fill whole rectangles.\n`;
|
||||
writeFileSync(path.join(folder, 'prompt.txt'), fullPrompt);
|
||||
if (prepareOnly) continue;
|
||||
@@ -111,11 +188,7 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
if (!imported[task.id]) { record.status = 'missing_candidate'; continue; }
|
||||
outputs = [{bytes: readFileSync(imported[task.id].file), extension: path.extname(imported[task.id].file).slice(1)}];
|
||||
} else {
|
||||
const images = [readImage(path.join(folder, 'input.png'), {pngOnly: true})];
|
||||
if (stage.capabilities.multiple_images) images.push(readImage(path.join(folder, 'context.png'), {pngOnly: true}));
|
||||
const mask = stage.capabilities.mask ? readImage(path.join(folder, 'edit-mask.png'), {pngOnly: true}) : null;
|
||||
jsonFile(path.join(folder, 'request.json'), {model: stage.model, size: requestSize, images: images.map((im) => im.sha256),
|
||||
edit_mask_sent: Boolean(mask), allowed_mask_sha256: sha256(readFileSync(path.join(folder, 'allowed-mask.png')))}, stage.apiKey);
|
||||
jsonFile(path.join(folder, 'request.json'), {...requestPlan, request_sent: true}, stage.apiKey);
|
||||
manifest.network_requested = true;
|
||||
jsonFile(run.manifestPath, manifest, stage.apiKey);
|
||||
try {
|
||||
@@ -126,6 +199,10 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
} catch (error) {
|
||||
record.status = 'request_failed';
|
||||
if (error.diagnostics) jsonFile(path.join(folder, 'response.error.json'), error.diagnostics, stage.apiKey);
|
||||
if (probeEditCapabilities) jsonFile(path.join(folder, 'capability-evidence.json'), {
|
||||
request_accepted: false, request: requestPlan, error: error.message,
|
||||
reference_use_verified: false, mask_semantics_verified: false,
|
||||
verdict: 'request_failed_no_capability_promotion'}, stage.apiKey);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -145,6 +222,10 @@ export async function runCompletion({stage, inputDir, outDir, partIds = [], prep
|
||||
record.status = record.selected_file ? 'geometry_passed_needs_review' : 'rejected';
|
||||
if (!record.qa) record.qa = {...record.candidates[0]?.qa, accepted: false,
|
||||
reasons: [...new Set(record.candidates.flatMap((c) => c.qa.reasons))]};
|
||||
if (probeEditCapabilities) jsonFile(path.join(folder, 'capability-evidence.json'), {
|
||||
request_accepted: true, request: requestPlan, candidates: record.candidates.map((c) => c.qa),
|
||||
reference_use_verified: false, mask_semantics_verified: false,
|
||||
verdict: 'needs_visual_review_no_automatic_capability_promotion'}, stage.apiKey);
|
||||
jsonFile(path.join(run.out, 'completion-result.json'), result, stage.apiKey);
|
||||
}
|
||||
manifest.quality_status = prepareOnly ? 'prepared_needs_review' : 'needs_review';
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {readFileSync, readdirSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import {projectRoot} from '../lib/config.mjs';
|
||||
import {selectPython} from '../lib/python-runtime.mjs';
|
||||
import {validateParts} from '../lib/validate-parts.mjs';
|
||||
import {sha256} from '../lib/images.mjs';
|
||||
import {prepareRun, jsonFile, finishRun} from '../lib/run-artifacts.mjs';
|
||||
|
||||
const readJson = (file) => JSON.parse(readFileSync(file, 'utf8').replace(/^\uFEFF/u, ''));
|
||||
const filesIn = (dir, prefix = '') => readdirSync(dir, {withFileTypes: true}).flatMap((entry) => entry.isDirectory()
|
||||
? filesIn(path.join(dir, entry.name), `${prefix}${entry.name}/`) : [`${prefix}${entry.name}`]);
|
||||
|
||||
export async function runSpine(options) {
|
||||
if (!options.inputDir || (!options.outDir && !options.dryRun)) throw new Error('Spine 阶段需要 --input-dir SAM目录 和独立的 --out-dir');
|
||||
const input = path.resolve(options.inputDir);
|
||||
const completion = options.completionDir ? path.resolve(options.completionDir) : undefined;
|
||||
const plan = readJson(path.join(input, 'semantic-parts.json'));
|
||||
const validation = validateParts(plan);
|
||||
if (!validation.valid || plan.schema_version !== 2) throw new Error(`Spine 阶段需要有效 v2 计划:${validation.errors.join(', ')}`);
|
||||
const fallback = options.fallbackPolicy || 'sam';
|
||||
const angle = Number(options.checkAngle ?? 10);
|
||||
if (!['sam', 'error'].includes(fallback)) throw new Error('--fallback-policy 仅支持 sam / error');
|
||||
if (!(angle > 0 && angle <= 45)) throw new Error('--check-angle 必须在 (0, 45] 度之间');
|
||||
const python = selectPython(options.python);
|
||||
const external = path.resolve(options.genielabsDir || path.join(projectRoot, '..', 'spine-animation-ai'));
|
||||
const scripts = ['make_atlas.py', 'generate_spine_player.py'].map((name) => path.join(external, 'scripts', name));
|
||||
const codeFiles = ['spine_assembly.py', 'stages/spine-stage.mjs', 'tools/model-check.mjs',
|
||||
'lib/python-runtime.mjs', 'lib/run-artifacts.mjs', 'lib/images.mjs', 'lib/validate-parts.mjs',
|
||||
'schemas/semantic-parts.schema.json'].map((name) => path.join(projectRoot, name));
|
||||
const codeHash = sha256([...codeFiles, ...scripts].map((file) => readFileSync(file)).map((bytes) => sha256(bytes)).join('\n'));
|
||||
function call(script, args) {
|
||||
try {
|
||||
return execFileSync(python, [script, ...args], {cwd: projectRoot, windowsHide: true,
|
||||
encoding: 'utf8', timeout: 120000, maxBuffer: 8 * 1024 * 1024,
|
||||
env: {...process.env, PYTHONUTF8: '1'}, stdio: ['ignore', 'pipe', 'pipe']});
|
||||
} catch (error) { throw new Error(`Spine 离线处理失败:${error.stderr || error.message}`); }
|
||||
}
|
||||
const worker = path.join(projectRoot, 'spine_assembly.py');
|
||||
const args = ['--input-dir', input, '--fallback-policy', fallback, '--check-angle', String(angle)];
|
||||
if (completion) args.push('--completion-dir', completion);
|
||||
if (options.drawOrderFile) args.push('--draw-order-file', path.resolve(options.drawOrderFile));
|
||||
// Inspect all masks, selected PNGs, crop coordinates and input identities before creating output.
|
||||
const inspected = JSON.parse(call(worker, args));
|
||||
if (options.dryRun) {
|
||||
const result = {dryRun: true, stage: 'spine', ...inspected, network_requested: false};
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
return result;
|
||||
}
|
||||
const identity = {schema_version: 1, stage: 'spine', spine_version: '4.2.0',
|
||||
inputs: inspected.input_hashes, code_sha256: codeHash, fallback_policy: fallback, check_angle: angle};
|
||||
const fingerprint = sha256(JSON.stringify(identity));
|
||||
const run = prepareRun(options.outDir, fingerprint, options.resume);
|
||||
if (run.reused) return {out: run.out, reused: true};
|
||||
const manifest = {...identity, fingerprint, status: 'running', quality_status: 'needs_review',
|
||||
network_requested: false, started_at: new Date().toISOString(),
|
||||
external_dependency: {name: 'spine-animation-ai', license: 'PolyForm-Noncommercial-1.0.0'},
|
||||
limitations: ['region attachments only; no mesh, weights or IK',
|
||||
'setup and per-bone check animations only; semantic and motion quality need review',
|
||||
'preview embeds assets but loads Spine Player 4.2 from CDN']};
|
||||
jsonFile(run.manifestPath, manifest);
|
||||
try {
|
||||
const result = JSON.parse(call(worker, [...args, '--output', run.out]));
|
||||
if (JSON.stringify(result.input_hashes) !== JSON.stringify(inspected.input_hashes)) throw new Error('处理期间输入发生变化,请使用新输出目录重跑');
|
||||
call(scripts[0], ['--parts', path.join(run.out, 'images'), '--output', path.join(run.out, 'atlas'), '--name', 'skeleton']);
|
||||
call(scripts[1], ['--skeleton', path.join(run.out, 'skeleton.json'),
|
||||
'--atlas', path.join(run.out, 'atlas', 'skeleton.atlas'), '--atlas-image', path.join(run.out, 'atlas', 'skeleton.png'),
|
||||
'--output', path.join(run.out, 'preview.html'), '--animation', 'setup', '--title', 'Spine Assembly Review']);
|
||||
Object.assign(manifest, {part_count: result.parts, bone_count: result.bones,
|
||||
completion_part_count: result.completion_parts, fallback_part_count: result.fallback_parts,
|
||||
completed_at: new Date().toISOString()});
|
||||
finishRun(run, manifest, filesIn(run.out).filter((name) => name !== 'manifest.json'));
|
||||
return {out: run.out, reused: false, ...result};
|
||||
} catch (error) {
|
||||
manifest.status = 'failed'; manifest.error = error.message;
|
||||
jsonFile(run.manifestPath, manifest);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -31,7 +31,7 @@ test('completion geometry rejects moved/empty outputs and preserves original alp
|
||||
|
||||
test('part completion prepares without HTTP, sends declared capabilities, merges, resumes, and detects changed masks', async (t) => {
|
||||
const {root, input, stage} = fixture(t);
|
||||
const opts = {stage, inputDir: input, outDir: path.join(root, 'run'), completionSize: 128, extensionRadius: 16, partIds: ['torso']};
|
||||
const opts = {stage, inputDir: input, outDir: path.join(root, 'run'), completionProfile: 'legacy', completionSize: 128, extensionRadius: 16, partIds: ['torso']};
|
||||
let requests = 0;
|
||||
const transport = {fetchImpl: async (_url, request) => {
|
||||
requests++;
|
||||
@@ -66,7 +66,7 @@ test('offline candidate import needs no HTTP; mismatched candidate is retained a
|
||||
const candidates = path.join(root, 'candidates'); mkdirSync(candidates);
|
||||
writeFileSync(path.join(candidates, 'torso.png'), readFileSync(path.join(input, 'source.png')));
|
||||
const outDir = path.join(root, 'run');
|
||||
const result = await runCompletion({stage, inputDir: input, outDir, candidatesDir: candidates, completionSize: 128}, {fetchImpl: () => { throw new Error('Must remain offline'); }});
|
||||
const result = await runCompletion({stage, inputDir: input, outDir, candidatesDir: candidates, completionProfile: 'legacy', completionSize: 128}, {fetchImpl: () => { throw new Error('Must remain offline'); }});
|
||||
assert.equal(result.rejected, 1);
|
||||
assert.equal(result.accepted, 0);
|
||||
const data = JSON.parse(readFileSync(path.join(outDir, 'completion-result.json')));
|
||||
@@ -81,7 +81,7 @@ test('provider mask/context capabilities are explicit and failure preserves diag
|
||||
stage.capabilities = {mask: true, multiple_images: true};
|
||||
stage.maxRetries = 0;
|
||||
const outDir = path.join(root, 'run');
|
||||
await assert.rejects(runCompletion({stage, inputDir: input, outDir, completionSize: 128}, {fetchImpl: async (_url, request) => {
|
||||
await assert.rejects(runCompletion({stage, inputDir: input, outDir, completionProfile: 'legacy', completionSize: 128}, {fetchImpl: async (_url, request) => {
|
||||
assert.equal(request.body.getAll('image').length, 2);
|
||||
assert.ok(request.body.get('mask'));
|
||||
return new Response(JSON.stringify({error: {message: 'test-secret failed'}}), {status: 400});
|
||||
@@ -98,7 +98,7 @@ test('completion CLI prepares without credentials and rejects mixed probe argume
|
||||
const envFile = path.join(root, 'empty.env'); writeFileSync(envFile, '');
|
||||
const outDir = path.join(root, 'cli');
|
||||
const args = ['tools/model-check.mjs', '--stage', 'completion', '--input-dir', input,
|
||||
'--out-dir', outDir, '--prepare-only', '--part', 'torso', '--completion-size', '128', '--env-file', envFile];
|
||||
'--out-dir', outDir, '--prepare-only', '--part', 'torso', '--completion-profile', 'legacy', '--completion-size', '128', '--env-file', envFile];
|
||||
const options = {cwd: projectRoot, windowsHide: true, encoding: 'utf8',
|
||||
env: {...process.env, VECTOR_ENGINE_BASE_URL: '', VECTOR_ENGINE_API_KEY: ''}, stdio: ['ignore', 'pipe', 'pipe']};
|
||||
assert.match(execFileSync(process.execPath, args, options), /未请求模型/u);
|
||||
@@ -107,3 +107,62 @@ test('completion CLI prepares without credentials and rejects mixed probe argume
|
||||
assert.equal(manifest.prepared_count, 1);
|
||||
assert.throws(() => execFileSync(process.execPath, [...args, '--image', path.join(input, 'source.png')], options), (error) => /不接受探针输入/u.test(error.stderr));
|
||||
});
|
||||
|
||||
test('adaptive geometry: reversible scaling, real-mask replanning, cleanup, review masks and Spine contract', () => {
|
||||
assert.equal(execFileSync(python, ['-m', 'unittest', 'discover', '-s', 'tests', '-p', 'test_completion_adaptive.py'],
|
||||
{cwd: projectRoot, windowsHide: true, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe']}), '');
|
||||
});
|
||||
|
||||
function adaptiveResponse(folder) {
|
||||
execFileSync(python, ['-c', 'import sys; sys.path.insert(0, "tests"); from test_completion_adaptive import mock_candidate; mock_candidate(sys.argv[1])', folder],
|
||||
{cwd: projectRoot, windowsHide: true, stdio: ['ignore', 'pipe', 'pipe']});
|
||||
return new Response(JSON.stringify({data: [{b64_json: readFileSync(path.join(folder, 'test-candidate.png')).toString('base64')}]}),
|
||||
{headers: {'Content-Type': 'application/json'}});
|
||||
}
|
||||
|
||||
test('adaptive default sends a single reference board and keeps native coordinates; planning changes invalidate candidates', async (t) => {
|
||||
const {root, input, stage} = fixture(t);
|
||||
const outDir = path.join(root, 'adaptive');
|
||||
const opts = {stage, inputDir: input, outDir, partIds: ['torso'], completionSize: 256, extensionRadius: 16};
|
||||
const result = await runCompletion(opts, {fetchImpl: async (_url, request) => {
|
||||
assert.equal(request.body.getAll('image').length, 1);
|
||||
assert.equal(request.body.has('mask'), false);
|
||||
assert.match(request.body.get('prompt'), /RIGHT panels/u);
|
||||
return adaptiveResponse(path.join(outDir, 'tasks/torso'));
|
||||
}});
|
||||
assert.equal(result.accepted, 1);
|
||||
const record = JSON.parse(readFileSync(path.join(outDir, 'completion-result.json'))).parts[0];
|
||||
assert.deepEqual(record.crop_xyxy, [20, 20, 50, 40]);
|
||||
assert.ok(record.preservation.preserve_mask);
|
||||
const request = JSON.parse(readFileSync(path.join(outDir, 'tasks/torso/request.json')));
|
||||
assert.equal(request.reference_mode, 'board');
|
||||
assert.equal(request.image_roles[0].role, 'target_with_reference_panels');
|
||||
assert.equal(request.mask_enforcement, 'local_only');
|
||||
assert.equal((await runCompletion({...opts, resume: true})).reused, true);
|
||||
const recovered = await runCompletion({...opts, outDir: path.join(root, 'adaptive-recheck'), candidatesDir: outDir});
|
||||
assert.equal(recovered.accepted, 1);
|
||||
await assert.rejects(runCompletion({...opts, outDir: path.join(root, 'bad-recheck'), candidatesDir: outDir, completionReference: 'none'}), /画布/u);
|
||||
});
|
||||
|
||||
test('native full/local references and mask preserve order and hashes; probe never promotes capabilities', async (t) => {
|
||||
const {root, input, stage} = fixture(t);
|
||||
const opts = {stage, inputDir: input, outDir: path.join(root, 'probe'), partIds: ['torso'], completionSize: 256};
|
||||
await assert.rejects(runCompletion({...opts, completionReference: 'both', completionMask: 'required'}), /未确认/u);
|
||||
await assert.rejects(runCompletion({...opts, partIds: [], probeEditCapabilities: true}), /恰好一个/u);
|
||||
await runCompletion({...opts, probeEditCapabilities: true}, {fetchImpl: async (_url, request) => {
|
||||
const images = request.body.getAll('image');
|
||||
assert.deepEqual(images.map((im) => im.name), ['input.png', 'context.png', 'reference.png']);
|
||||
assert.equal(request.body.get('mask').name, 'edit-mask.png');
|
||||
assert.equal(request.body.get('n'), '1');
|
||||
return adaptiveResponse(path.join(opts.outDir, 'tasks/torso'));
|
||||
}});
|
||||
assert.deepEqual(stage.capabilities, {multiple_images: false, mask: false});
|
||||
const evidence = JSON.parse(readFileSync(path.join(opts.outDir, 'tasks/torso/capability-evidence.json')));
|
||||
assert.equal(evidence.request_accepted, true);
|
||||
assert.equal(evidence.mask_semantics_verified, false);
|
||||
assert.equal(evidence.reference_use_verified, false);
|
||||
assert.equal(evidence.request.image_roles.length, 3);
|
||||
assert.ok(evidence.request.mask_sha256);
|
||||
const qa = JSON.parse(readFileSync(path.join(opts.outDir, 'tasks/torso/qa-1.json')));
|
||||
assert.equal(typeof qa.raw_outside_edit_changed_pixels, 'number');
|
||||
});
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {execFileSync} from 'node:child_process';
|
||||
import {mkdtempSync, readFileSync, writeFileSync, realpathSync, rmSync, existsSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import {tmpdir} from 'node:os';
|
||||
import {projectRoot} from '../lib/config.mjs';
|
||||
import {selectPython} from '../lib/python-runtime.mjs';
|
||||
import {runSpine} from '../stages/spine-stage.mjs';
|
||||
|
||||
const python = selectPython();
|
||||
const execution = {cwd: projectRoot, windowsHide: true, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe']};
|
||||
function fixture(t) {
|
||||
const base = realpathSync(tmpdir());
|
||||
const root = mkdtempSync(path.join(base, 'spine-export-'));
|
||||
t.after(() => {
|
||||
const resolved = realpathSync(root);
|
||||
assert.equal(path.dirname(resolved), base);
|
||||
assert.ok(path.basename(resolved).startsWith('spine-export-'));
|
||||
rmSync(resolved, {recursive: true});
|
||||
});
|
||||
execFileSync(python, ['-c', 'import sys; sys.path.insert(0, "tests"); from test_spine_assembly import make_fixture; make_fixture(sys.argv[1])', root], execution);
|
||||
return {root, inputDir: path.join(root, 'sam'), completionDir: path.join(root, 'complete'), outDir: path.join(root, 'out')};
|
||||
}
|
||||
|
||||
test('Spine assembly preserves coordinates, hidden hierarchy, bindings and validates asset identity', () => {
|
||||
assert.equal(execFileSync(python, ['-m', 'unittest', 'discover', '-s', 'tests', '-p', 'test_spine_assembly.py'], execution), '');
|
||||
});
|
||||
|
||||
test('Spine exports complete assets, atlas, player, diagnostics and verifies resume hashes', async (t) => {
|
||||
const opts = fixture(t);
|
||||
const result = await runSpine(opts);
|
||||
assert.equal(result.parts, 4);
|
||||
assert.equal(result.bones, 4);
|
||||
for (const file of ['assembly.json', 'skeleton.json', 'atlas/skeleton.atlas', 'atlas/skeleton.png',
|
||||
'images/shell.png', 'assembled.png', 'comparison.png', 'coverage-diagnostic.png', 'report.html', 'preview.html']) {
|
||||
assert.ok(existsSync(path.join(opts.outDir, file)), file);
|
||||
}
|
||||
const manifest = JSON.parse(readFileSync(path.join(opts.outDir, 'manifest.json')));
|
||||
assert.equal(manifest.status, 'completed');
|
||||
assert.equal(manifest.quality_status, 'needs_review');
|
||||
assert.equal(manifest.network_requested, false);
|
||||
assert.equal(manifest.fallback_part_count, 1);
|
||||
const skeleton = JSON.parse(readFileSync(path.join(opts.outDir, 'skeleton.json')));
|
||||
assert.equal(skeleton.skeleton.images, './images/');
|
||||
const atlas = readFileSync(path.join(opts.outDir, 'atlas/skeleton.atlas'), 'utf8').replaceAll('\r\n', '\n');
|
||||
for (const slot of skeleton.slots) assert.ok(atlas.includes(`\n${slot.attachment}\n`));
|
||||
assert.match(readFileSync(path.join(opts.outDir, 'preview.html'), 'utf8'), /animation: "setup"/u);
|
||||
assert.equal((await runSpine({...opts, resume: true})).reused, true);
|
||||
await assert.rejects(runSpine(opts), /非空/u);
|
||||
await assert.rejects(runSpine({...opts, resume: true, checkAngle: 5}), /变化/u);
|
||||
writeFileSync(path.join(opts.outDir, 'images/shell.png'), 'tampered');
|
||||
await assert.rejects(runSpine({...opts, resume: true}), /产物缺失或被修改/u);
|
||||
});
|
||||
|
||||
test('Spine CLI is independent of model config and dry-run writes nothing', (t) => {
|
||||
const opts = fixture(t);
|
||||
const args = ['tools/model-check.mjs', '--stage', 'spine', '--input-dir', opts.inputDir,
|
||||
'--completion-dir', opts.completionDir, '--out-dir', opts.outDir];
|
||||
const env = {...process.env, VECTOR_ENGINE_API_KEY: '', TIANTOKEN_API_KEY: ''};
|
||||
assert.match(execFileSync(process.execPath, [...args, '--dry-run'], {...execution, env}), /"network_requested": false/u);
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
assert.throws(() => execFileSync(process.execPath, [...args, '--config', 'missing-config.json'], execution),
|
||||
(error) => /独立离线阶段/u.test(error.stderr));
|
||||
assert.throws(() => execFileSync(process.execPath, [...args, '--fallback-policy', 'error'], execution),
|
||||
(error) => /Completion required/u.test(error.stderr));
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
assert.match(execFileSync(process.execPath, args, {...execution, env}), /导出完成/u);
|
||||
});
|
||||
|
||||
test('Spine rejects malformed semantic hierarchy before export', async (t) => {
|
||||
const opts = fixture(t);
|
||||
const file = path.join(opts.inputDir, 'semantic-parts.json');
|
||||
const plan = JSON.parse(readFileSync(file));
|
||||
plan.parts.find((p) => p.id === 'hub').parent = 'tip_q';
|
||||
writeFileSync(file, JSON.stringify(plan));
|
||||
await assert.rejects(runSpine(opts), /存在环/u);
|
||||
assert.equal(existsSync(opts.outDir), false);
|
||||
});
|
||||
@@ -0,0 +1,205 @@
|
||||
import copy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from completion_geometry import localize, merge_candidate, finalize
|
||||
from completion_planning import prepare_adaptive, read, write, review_inputs
|
||||
from spine_assembly import load_assets
|
||||
from test_completion import make_fixture
|
||||
|
||||
|
||||
def candidate_for(folder, native):
|
||||
"""Independent fixture: insert known native ground truth into documented work rectangle."""
|
||||
folder = Path(folder)
|
||||
task = read(folder / 'task.json')
|
||||
rect = task['transform']['working_xyxy']
|
||||
canvas = Image.open(folder / 'input.png').convert('RGBA')
|
||||
content = Image.fromarray(native).resize((rect[2]-rect[0], rect[3]-rect[1]), Image.Resampling.NEAREST)
|
||||
canvas.paste(content, (rect[0], rect[1]))
|
||||
target = folder / 'test-candidate.png'
|
||||
canvas.save(target)
|
||||
return target
|
||||
|
||||
|
||||
def mock_candidate(folder):
|
||||
folder = Path(folder)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
edit = np.array(Image.open(folder / 'native-allowed.png')) > 0
|
||||
native[edit] = [180, 50, 20, 255]
|
||||
return candidate_for(folder, native)
|
||||
|
||||
|
||||
class AdaptiveTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
self.source = make_fixture(self.root)
|
||||
|
||||
def task(self, mode='board', size=256, review=None):
|
||||
plan = prepare_adaptive(self.root, ['torso'], size, 16, mode, review)
|
||||
return plan['tasks'][0], self.root / 'tasks' / 'torso'
|
||||
|
||||
def test_small_part_scales_up_roundtrip_and_reference_panels_are_excluded(self):
|
||||
task, folder = self.task()
|
||||
self.assertGreater(task['transform']['scale'], 1)
|
||||
self.assertEqual(len(task['reference_panel_boxes']), 2)
|
||||
forward = task['transform']['source_to_canvas']
|
||||
inverse = task['transform']['canvas_to_source']
|
||||
for x, y in [(20, 20), (49, 39), (0, 79)]:
|
||||
mx, my = forward[0]*x+forward[2], forward[4]*y+forward[5]
|
||||
self.assertAlmostEqual(inverse[0]*mx+inverse[2], x)
|
||||
self.assertAlmostEqual(inverse[4]*my+inverse[5], y)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
edit = np.array(Image.open(folder / 'native-allowed.png')) > 0
|
||||
native[edit] = [180, 50, 20, 255]
|
||||
output = folder / 'merged.png'
|
||||
qa = merge_candidate(folder, candidate_for(folder, native), output)
|
||||
self.assertTrue(qa['accepted'], qa)
|
||||
keep = np.array(Image.open(folder / 'native-keep.png')) > 0
|
||||
np.testing.assert_array_equal(np.array(Image.open(output))[keep], native[keep])
|
||||
write(self.root / 'completion-result.json', {'parts': [{'id': 'torso', 'status': 'geometry_passed_needs_review',
|
||||
'selected_file': 'tasks/torso/merged.png', 'qa': qa}], 'skipped': []})
|
||||
finalize(self.root)
|
||||
result = read(self.root / 'completion-result.json')['parts'][0]
|
||||
self.assertEqual(result['crop_xyxy'], [20, 20, 50, 40])
|
||||
self.assertIn('preservation', result)
|
||||
self.assertTrue((folder / 'reference.png').exists())
|
||||
mask = np.array(Image.open(folder / 'edit-mask.png'))
|
||||
for x0, y0, x1, y1 in task['reference_panel_boxes']:
|
||||
self.assertTrue(np.all(mask[y0:y1, x0:x1, 3] == 255))
|
||||
|
||||
def test_actual_hole_creates_task_despite_false_semantic_flag(self):
|
||||
plan = read(self.root / 'semantic-parts.json')
|
||||
plan['parts'][0].update(needs_completion=False, visibility='visible', occluded_by=[],
|
||||
completion={k: False for k in plan['parts'][0]['completion']})
|
||||
write(self.root / 'semantic-parts.json', plan)
|
||||
mask = np.array(Image.open(self.root / 'masks/torso.png'))
|
||||
mask[26:30, 26:30] = 0
|
||||
Image.fromarray(mask).save(self.root / 'masks/torso.png')
|
||||
task, folder = self.task()
|
||||
self.assertEqual(task['regions']['segmentation_gap'], 16)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
edit = np.array(Image.open(folder / 'native-allowed.png')) > 0
|
||||
native[edit] = [200, 40, 10, 255]
|
||||
output = folder / 'merged.png'
|
||||
qa = merge_candidate(folder, candidate_for(folder, native), output)
|
||||
self.assertTrue(qa['accepted'], qa)
|
||||
# Repair restores source texture, even though the model proposed a different colour.
|
||||
gap = np.array(Image.open(folder / 'native-segmentation_gap.png')) > 0
|
||||
source = np.array(Image.open(folder / 'native-source.png'))
|
||||
np.testing.assert_array_equal(np.array(Image.open(output))[gap], source[gap])
|
||||
|
||||
def test_attachment_cleanup_releases_only_declared_overlay_and_spine_accepts_contract(self):
|
||||
# Move collar inside torso to represent an overlaid detail needing a clean base.
|
||||
mask = np.zeros((80, 80), np.uint8)
|
||||
mask[26:30, 26:30] = 255
|
||||
Image.fromarray(mask).save(self.root / 'masks/collar.png')
|
||||
task, folder = self.task('both')
|
||||
self.assertEqual(task['regions']['attachment_cleanup'], 16)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
edit = np.array(Image.open(folder / 'native-allowed.png')) > 0
|
||||
native[edit] = [20, 100, 200, 255]
|
||||
output = folder / 'merged.png'
|
||||
qa = merge_candidate(folder, candidate_for(folder, native), output)
|
||||
self.assertTrue(qa['accepted'], qa)
|
||||
write(self.root / 'completion-result.json', {'parts': [{'id': 'torso', 'status': 'geometry_passed_needs_review',
|
||||
'selected_file': 'tasks/torso/merged.png', 'qa': qa}], 'skipped': []})
|
||||
finalize(self.root)
|
||||
data = load_assets(self.root, self.root)
|
||||
self.assertEqual(data[2]['parts'][0]['source'], 'completion')
|
||||
# Contract tampering cannot turn a protected pixel into an untracked edit.
|
||||
result = read(self.root / 'completion-result.json')['parts'][0]
|
||||
image = np.array(Image.open(self.root / result['image']))
|
||||
image[0, 0] = [255, 0, 0, 255]
|
||||
Image.fromarray(image).save(self.root / result['image'])
|
||||
with self.assertRaisesRegex(ValueError, 'preserve original visible'):
|
||||
load_assets(self.root, self.root)
|
||||
|
||||
def test_review_replacement_removal_and_invalid_partition(self):
|
||||
replace = np.zeros((80, 80), np.uint8)
|
||||
replace[26:29, 26:29] = 255
|
||||
remove = np.zeros_like(replace)
|
||||
remove[35:40, 35:40] = 255
|
||||
Image.fromarray(replace).save(self.root / 'replace.png')
|
||||
Image.fromarray(remove).save(self.root / 'remove.png')
|
||||
review = self.root / 'review-input.json'
|
||||
write(review, {'schema_version': 1, 'parts': {'torso': {'confirmed': True,
|
||||
'replace_mask': 'replace.png', 'remove_mask': 'remove.png', 'extension_radius': 8}}})
|
||||
task, folder = self.task(review=review)
|
||||
self.assertEqual(task['extension_radius'], 8)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
edit = np.array(Image.open(folder / 'native-allowed.png')) > 0
|
||||
native[edit] = [180, 50, 20, 255]
|
||||
qa = merge_candidate(folder, candidate_for(folder, native), folder / 'merged.png')
|
||||
self.assertTrue(qa['accepted'], qa)
|
||||
deleted = np.array(Image.open(folder / 'native-remove.png')) > 0
|
||||
self.assertTrue(np.all(np.array(Image.open(folder / 'merged.png'))[deleted] == 0))
|
||||
Image.fromarray(remove).save(self.root / 'replace.png')
|
||||
with self.assertRaisesRegex(ValueError, 'disjoint'):
|
||||
self.task(review=review)
|
||||
|
||||
def test_large_part_downscales_and_invalid_review_cannot_escape(self):
|
||||
# Upsize the entire source and masks to exceed request dimensions.
|
||||
for file in [self.root / 'source.png', *list((self.root / 'masks').glob('*.png'))]:
|
||||
im = Image.open(file)
|
||||
im.resize((800, 800), Image.Resampling.NEAREST).save(file)
|
||||
task, folder = self.task(size=128)
|
||||
self.assertLess(task['transform']['scale'], 1)
|
||||
native = np.array(Image.open(folder / 'native-visible.png'))
|
||||
native[np.array(Image.open(folder / 'native-allowed.png')) > 0] = [180, 50, 20, 255]
|
||||
output = folder / 'merged.png'
|
||||
qa = merge_candidate(folder, candidate_for(folder, native), output)
|
||||
self.assertTrue(qa['accepted'], qa)
|
||||
keep = np.array(Image.open(folder / 'native-keep.png')) > 0
|
||||
np.testing.assert_array_equal(np.array(Image.open(output))[keep], native[keep])
|
||||
review = self.root / 'bad-review.json'
|
||||
write(review, {'schema_version': 1, 'parts': {'torso': {'confirmed': True, 'replace_mask': '../escape.png'}}})
|
||||
with self.assertRaisesRegex(ValueError, 'escapes'):
|
||||
review_inputs(review, ['torso'], (800, 800))
|
||||
|
||||
def test_adaptive_rejects_movement_and_missing_cleanup(self):
|
||||
task, folder = self.task()
|
||||
candidate = mock_candidate(folder)
|
||||
image = np.array(Image.open(candidate))
|
||||
Image.fromarray(np.roll(image, 30, axis=1)).save(candidate)
|
||||
qa = merge_candidate(folder, candidate, folder / 'bad.png')
|
||||
self.assertFalse(qa['accepted'])
|
||||
self.assertIn('VISIBLE_ALIGNMENT_MISMATCH', qa['reasons'])
|
||||
mask = np.zeros((80, 80), np.uint8)
|
||||
mask[26:30, 26:30] = 255
|
||||
Image.fromarray(mask).save(self.root / 'masks/collar.png')
|
||||
_, folder = self.task('both')
|
||||
no_fill = np.array(Image.open(folder / 'native-visible.png'))
|
||||
qa = merge_candidate(folder, candidate_for(folder, no_fill), folder / 'bad-cleanup.png')
|
||||
self.assertFalse(qa['accepted'])
|
||||
self.assertIn('INCOMPLETE_ATTACHMENT_CLEANUP', qa['reasons'])
|
||||
|
||||
def test_duplicate_masks_blocked_until_reviewed(self):
|
||||
plan = read(self.root / 'semantic-parts.json')
|
||||
plan['parts'][0]['occluded_by'] = []
|
||||
write(self.root / 'semantic-parts.json', plan)
|
||||
Image.open(self.root / 'masks/torso.png').save(self.root / 'masks/collar.png')
|
||||
result = prepare_adaptive(self.root, ['torso'])
|
||||
self.assertEqual(result['skipped'][0]['reason'], 'AMBIGUOUS_MASK_REQUIRES_REVIEW')
|
||||
|
||||
def test_spine_strict_policy_respects_rebuilt_need_even_if_original_flag_was_false(self):
|
||||
plan = read(self.root / 'semantic-parts.json')
|
||||
plan['parts'][0].update(needs_completion=False, visibility='visible', occluded_by=[],
|
||||
completion={k: False for k in plan['parts'][0]['completion']})
|
||||
write(self.root / 'semantic-parts.json', plan)
|
||||
write(self.root / 'completion-result.json', {'schema_version': 2, 'profile': 'adaptive', 'parts': [
|
||||
{'id': 'torso', 'status': 'rejected', 'qa': {'accepted': False}}], 'skipped': []})
|
||||
with self.assertRaisesRegex(ValueError, 'Completion required for torso'):
|
||||
load_assets(self.root, self.root, fallback_policy='error')
|
||||
data = load_assets(self.root, self.root)
|
||||
self.assertEqual(data[2]['parts'][0]['reason'], 'COMPLETION_REJECTED')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -0,0 +1,180 @@
|
||||
import copy
|
||||
import json
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
from spine_assembly import load_assets, build_skeleton, export_assets, write_json
|
||||
|
||||
|
||||
def make_fixture(directory):
|
||||
"""An asymmetric mechanical fixture; no humanoid names or assumed part count."""
|
||||
root = Path(directory)
|
||||
sam, completion = root / 'sam', root / 'complete'
|
||||
(sam / 'masks').mkdir(parents=True)
|
||||
source = np.zeros((48, 64, 4), np.uint8)
|
||||
specs = [('shell', [16, 12, 30, 24], [40, 70, 130, 255]),
|
||||
('link_z', [30, 22, 40, 32], [160, 70, 10, 255]),
|
||||
('tip_q', [40, 28, 45, 36], [80, 170, 30, 255]),
|
||||
('decal', [19, 15, 22, 18], [200, 50, 130, 255])]
|
||||
for _, (x0, y0, x1, y1), color in specs:
|
||||
source[y0:y1, x0:x1] = color
|
||||
source[12, 16, 3] = 90
|
||||
source[25, 33] = [230, 140, 40, 255]
|
||||
Image.fromarray(source).save(sam / 'source.png')
|
||||
|
||||
def part(name, parent, role, pivot, box, hidden=False, needs=False, occluded=None):
|
||||
return {'id': name, 'semantic': name, 'parent': parent, 'rig_role': role,
|
||||
'bind_to': parent if role != 'bone_part' else None,
|
||||
'pivot_norm': [pivot[0] / 64, pivot[1] / 48],
|
||||
'visibility': 'hidden' if hidden else 'visible',
|
||||
'bbox_norm_xyxy': None if hidden else [box[0]/64, box[1]/48, box[2]/64, box[3]/48],
|
||||
'positive_points_norm': [], 'negative_points_norm': [],
|
||||
'occluded_by': occluded or [], 'confidence': .9,
|
||||
'needs_completion': hidden or needs,
|
||||
'completion': {'occlusion_fill': hidden, 'joint_extension_parent': needs,
|
||||
'joint_extension_child': False}}
|
||||
|
||||
parts = [part('tip_q', 'link_z', 'bone_part', [40, 29], specs[2][1], needs=True),
|
||||
part('decal', 'hub', 'attachment', [19, 15], specs[3][1]),
|
||||
part('shell', 'hub', 'attachment', [20, 16], specs[0][1], occluded=['decal']),
|
||||
part('link_z', 'hub', 'bone_part', [31, 23], specs[1][1], needs=True),
|
||||
part('hub', None, 'bone_part', [21, 17], None, hidden=True, occluded=['shell'])]
|
||||
write_json(sam / 'semantic-parts.json', {'schema_version': 2, 'coordinate_space': 'normalized_xy_top_left',
|
||||
'parts': parts, 'notes': []})
|
||||
records = []
|
||||
for name, (x0, y0, x1, y1), _ in specs:
|
||||
mask = np.zeros(source.shape[:2], np.uint8)
|
||||
mask[y0:y1, x0:x1] = 255
|
||||
Image.fromarray(mask).save(sam / 'masks' / f'{name}.png')
|
||||
records.append({'id': name, 'mask': f'masks/{name}.png', 'crop_xyxy': [x0, y0, x1, y1]})
|
||||
write_json(sam / 'segmentation.json', {'schema_version': 1, 'parts': records})
|
||||
shutil.copytree(sam, completion)
|
||||
(completion / 'parts').mkdir()
|
||||
# Add pixels to the left of link_z; the original visible pixels are exact.
|
||||
candidate = np.zeros((10, 14, 4), np.uint8)
|
||||
candidate[:, :4] = [160, 70, 10, 255]
|
||||
candidate[:, 4:] = source[22:32, 30:40]
|
||||
Image.fromarray(candidate).save(completion / 'parts' / 'link_z.png')
|
||||
write_json(completion / 'completion-result.json', {'parts': [
|
||||
{'id': 'link_z', 'status': 'geometry_passed_needs_review', 'selected_file': 'tasks/link_z/merged-1.png',
|
||||
'qa': {'accepted': True}, 'image': 'parts/link_z.png', 'crop_xyxy': [26, 22, 40, 32]},
|
||||
{'id': 'tip_q', 'status': 'rejected', 'qa': {'accepted': False, 'reasons': ['VISIBLE_ALIGNMENT_MISMATCH']}}
|
||||
], 'skipped': [{'id': 'hub', 'reason': 'NO_VISIBLE_SAM_MASK'},
|
||||
{'id': 'shell', 'reason': 'NO_COMPLETION_NEEDED'},
|
||||
{'id': 'decal', 'reason': 'NO_COMPLETION_NEEDED'}]})
|
||||
|
||||
|
||||
class AssemblyTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.temp.cleanup)
|
||||
self.root = Path(self.temp.name)
|
||||
make_fixture(self.root)
|
||||
self.sam, self.complete = self.root / 'sam', self.root / 'complete'
|
||||
|
||||
def load(self, **kwargs):
|
||||
return load_assets(self.sam, self.complete, **kwargs)
|
||||
|
||||
def result(self):
|
||||
return json.loads((self.complete / 'completion-result.json').read_text())
|
||||
|
||||
def test_pose_coordinates_hidden_hierarchy_and_attachment_binding(self):
|
||||
data = self.load()
|
||||
result = export_assets(data, self.root / 'out')
|
||||
self.assertEqual((result['parts'], result['bones'], result['completion_parts'], result['fallback_parts']), (4, 4, 1, 1))
|
||||
skeleton = build_skeleton(data[1], data[2])
|
||||
self.assertEqual([b['name'] for b in skeleton['bones']], ['root', 'hub', 'link_z', 'tip_q'])
|
||||
bones = {b['name']: b for b in skeleton['bones']}
|
||||
self.assertAlmostEqual(bones['hub']['x'], -11)
|
||||
self.assertAlmostEqual(bones['hub']['y'], 31)
|
||||
self.assertAlmostEqual(bones['link_z']['x'], 10)
|
||||
self.assertAlmostEqual(bones['link_z']['y'], -6)
|
||||
self.assertAlmostEqual(bones['tip_q']['x'], 9)
|
||||
self.assertAlmostEqual(bones['tip_q']['y'], -6)
|
||||
slots = {s['name']: s for s in skeleton['slots']}
|
||||
self.assertEqual(slots['shell']['bone'], 'hub')
|
||||
self.assertNotIn('hub', slots)
|
||||
self.assertLess(data[2]['draw_order'].index('shell'), data[2]['draw_order'].index('decal'))
|
||||
# Independent forward kinematics reconstructs each crop centre exactly.
|
||||
world = {'root': (0, 0)}
|
||||
for bone in skeleton['bones'][1:]:
|
||||
px, py = world[bone['parent']]
|
||||
world[bone['name']] = px + bone['x'], py + bone['y']
|
||||
for name, (_, box) in data[3].items():
|
||||
att = skeleton['skins'][0]['attachments'][name][name]
|
||||
bx, by = world[slots[name]['bone']]
|
||||
self.assertAlmostEqual(att['x'] + bx + 32, (box[0] + box[2]) / 2)
|
||||
self.assertAlmostEqual(48 - att['y'] - by, (box[1] + box[3]) / 2)
|
||||
image = np.array(Image.open(self.root / 'out' / 'assembled.png'))
|
||||
np.testing.assert_array_equal(image[12, 16], [40, 70, 130, 90])
|
||||
self.assertEqual(skeleton['animations']['check__link_z']['bones']['link_z']['rotate'][1]['value'], 10)
|
||||
|
||||
def test_rejection_fallback_and_strict_policy(self):
|
||||
data = self.load()
|
||||
assets = {p['id']: p for p in data[2]['parts']}
|
||||
self.assertEqual(assets['tip_q']['source'], 'sam')
|
||||
self.assertEqual(assets['tip_q']['reason'], 'COMPLETION_REJECTED')
|
||||
with self.assertRaisesRegex(ValueError, 'Completion required for tip_q'):
|
||||
self.load(fallback_policy='error')
|
||||
only_sam = load_assets(self.sam)
|
||||
self.assertTrue(all(p['source'] in ['sam', 'hidden'] for p in only_sam[2]['parts']))
|
||||
|
||||
def test_mixed_runs_and_changed_masks_rejected(self):
|
||||
mask = self.complete / 'masks' / 'tip_q.png'
|
||||
Image.new('L', (64, 48), 255).save(mask)
|
||||
with self.assertRaisesRegex(ValueError, 'Completion mask mismatch'):
|
||||
self.load()
|
||||
shutil.copy2(self.sam / 'masks' / 'tip_q.png', mask)
|
||||
(self.complete / 'semantic-parts.json').write_text('{}')
|
||||
with self.assertRaisesRegex(ValueError, 'Completion input mismatch'):
|
||||
self.load()
|
||||
|
||||
def test_bad_crop_alignment_path_and_missing_visible_parts_rejected(self):
|
||||
original = self.result()
|
||||
for box in [[26, 22, 39, 32], [-1, 22, 13, 32], [27, 22, 41, 32]]:
|
||||
result = copy.deepcopy(original)
|
||||
result['parts'][0]['crop_xyxy'] = box
|
||||
write_json(self.complete / 'completion-result.json', result)
|
||||
with self.assertRaises(ValueError):
|
||||
self.load()
|
||||
result = copy.deepcopy(original)
|
||||
result['parts'][0]['image'] = '../sam/source.png'
|
||||
write_json(self.complete / 'completion-result.json', result)
|
||||
with self.assertRaisesRegex(ValueError, 'outside input'):
|
||||
self.load()
|
||||
seg = json.loads((self.sam / 'segmentation.json').read_text())
|
||||
seg['parts'] = seg['parts'][1:]
|
||||
write_json(self.sam / 'segmentation.json', seg)
|
||||
with self.assertRaisesRegex(ValueError, 'Visible part has no SAM mask'):
|
||||
load_assets(self.sam)
|
||||
|
||||
def test_draw_order_override_and_unconstrained_overlap(self):
|
||||
data = self.load()
|
||||
self.assertTrue(data[2]['unconstrained_overlaps'])
|
||||
override = self.root / 'order.json'
|
||||
order = ['decal', 'shell', 'link_z', 'tip_q']
|
||||
write_json(override, {'draw_order': order})
|
||||
modified = self.load(draw_order_file=override)[2]
|
||||
self.assertEqual(modified['draw_order'], order)
|
||||
self.assertIn(['shell', 'decal'], modified['overridden_occlusion_constraints'])
|
||||
write_json(override, {'draw_order': ['shell'] * 4})
|
||||
with self.assertRaisesRegex(ValueError, 'exactly once'):
|
||||
self.load(draw_order_file=override)
|
||||
|
||||
def test_attachment_only_and_root_binding(self):
|
||||
plan = {'parts': [{'id': 'panel', 'rig_role': 'attachment', 'bind_to': None}]}
|
||||
assembly = {'width': 64, 'height': 48, 'draw_order': ['panel'],
|
||||
'parts': [{'id': 'panel', 'crop_xyxy': [1, 2, 10, 12]}]}
|
||||
skeleton = build_skeleton(plan, assembly)
|
||||
self.assertEqual(skeleton['bones'], [{'name': 'root'}])
|
||||
self.assertEqual(skeleton['slots'][0]['bone'], 'root')
|
||||
self.assertEqual(skeleton['animations'], {'setup': {}})
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
+35
-5
@@ -7,12 +7,14 @@ import {runUnderstanding, runCompletionProbe} from '../stages/model-stage.mjs';
|
||||
import {readImage} from '../lib/images.mjs';
|
||||
import {runSegmentation} from '../stages/segmentation-stage.mjs';
|
||||
import {runCompletion} from '../stages/completion-stage.mjs';
|
||||
import {runSpine} from '../stages/spine-stage.mjs';
|
||||
|
||||
const help = `模型接入与结构理解(新增模式,不调用旧拆件流程)
|
||||
node pipeline.mjs --mode annotated-parts --reference image.png --out-dir output/run --stop-after understand
|
||||
node tools/model-check.mjs --stage completion --image local.png --prompt-file task.txt --out-dir output/edit
|
||||
node tools/model-check.mjs --stage segmentation --input-dir output/understand --out-dir output/segment --sam-dir F:/Study/SAM3
|
||||
node tools/model-check.mjs --stage completion --input-dir output/segment --out-dir output/complete --part head --prepare-only
|
||||
node tools/model-check.mjs --stage spine --input-dir output/segment --completion-dir output/complete --out-dir output/spine
|
||||
node tools/model-check.mjs --list-models
|
||||
node tools/model-check.mjs --dry-run
|
||||
|
||||
@@ -28,12 +30,22 @@ const help = `模型接入与结构理解(新增模式,不调用旧拆件流
|
||||
--parts-file FILE 离线读取已有 v1/v2 计划,做 Alpha 校验与种子生成,不请求模型
|
||||
--python FILE ai-spine Conda 环境解释器(默认使用已激活环境)
|
||||
--input-dir DIR SAM 阶段读理解目录;补全阶段读 SAM 目录
|
||||
--completion-dir DIR Spine 阶段可选补全目录;input-dir 指向同批 SAM 目录
|
||||
--fallback-policy sam|error Spine 未完成补全时回退 SAM(默认)或报错
|
||||
--draw-order-file FILE Spine 绘制顺序覆盖:{"draw_order":[全部可绘制ID,从后到前]}
|
||||
--check-angle FLOAT Spine 单骨骼检查动画角度,默认 10,范围 (0,45]
|
||||
--genielabs-dir DIR 外部 spine-animation-ai 路径,仅用于 Spine 图集和播放器
|
||||
--part ID 补全部件 ID,可重复;默认处理所有需补全部件
|
||||
--prepare-only 仅准备补全画布、掩码和提示词,不请求模型
|
||||
--candidates-dir DIR 离线导入 ID.png,或旧补全目录的首张候选,不请求模型
|
||||
--candidate-size-policy NAME strict(默认)/ uniform(同宽高比缩放后重新检查对齐)
|
||||
--completion-size INT 补全方形画布边长,默认 1024,不缩放原像素
|
||||
--extension-radius INT 补全区域距可见轮廓的最大像素距离,默认 32
|
||||
--completion-profile NAME adaptive(默认)重建需求与可逆缩放;legacy 复用旧候选
|
||||
--completion-reference NAME auto(默认)/ board / local / full / both / none
|
||||
--completion-mask NAME auto(默认)/ required / off;auto 遵循提供方能力
|
||||
--completion-review FILE 复核 JSON,按 ID 指定修复/替换/移除 mask 与半径
|
||||
--probe-edit-capabilities 单部件真实多图+mask 探针,不自动修改提供方能力配置
|
||||
--completion-size INT 请求方形画布边长,默认 1024;adaptive 显式记录缩放逆变换
|
||||
--extension-radius INT 全局原图像素半径覆盖;adaptive 默认按部件尺寸,legacy 默认 32
|
||||
--sam-dir DIR 本地 SAM 根目录,包含 sam3 与 sam3.1checkpoint
|
||||
--sam-code-dir DIR SAM 源码仓库目录覆盖
|
||||
--sam-checkpoint FILE 本地权重路径覆盖
|
||||
@@ -44,7 +56,7 @@ const help = `模型接入与结构理解(新增模式,不调用旧拆件流
|
||||
--dry-run 检查配置与本地输入,不请求模型、不生成产物
|
||||
|
||||
--stop-after 支持 understand / segment。SAM 在临时 GPU 子进程运行,完成后退出。
|
||||
部件补全使用 --stage completion --input-dir;可见像素恢复与几何检查已接入,Spine 导出尚未接入。`;
|
||||
部件补全使用 --stage completion --input-dir;--stage spine 离线组装并导出 Spine 4.2,资产和动作质量仍需复核。`;
|
||||
|
||||
export function parseOptions(argv) {
|
||||
const options = {stage: 'understanding', imagePaths: [], partIds: []};
|
||||
@@ -58,6 +70,10 @@ export function parseOptions(argv) {
|
||||
['--sam-checkpoint', 'samCheckpoint'], ['--sam-confidence', 'samConfidence'], ['--sam-precision', 'samPrecision'],
|
||||
['--candidates-dir', 'candidatesDir'], ['--completion-size', 'completionSize'], ['--extension-radius', 'extensionRadius'],
|
||||
['--candidate-size-policy', 'candidateSizePolicy'],
|
||||
['--completion-dir', 'completionDir'], ['--fallback-policy', 'fallbackPolicy'],
|
||||
['--draw-order-file', 'drawOrderFile'], ['--check-angle', 'checkAngle'], ['--genielabs-dir', 'genielabsDir'],
|
||||
['--completion-profile', 'completionProfile'], ['--completion-reference', 'completionReference'],
|
||||
['--completion-mask', 'completionMask'], ['--completion-review', 'completionReview'],
|
||||
]);
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
const flag = argv[i];
|
||||
@@ -65,6 +81,7 @@ export function parseOptions(argv) {
|
||||
else if (flag === '--resume') options.resume = true;
|
||||
else if (flag === '--dry-run') options.dryRun = true;
|
||||
else if (flag === '--prepare-only') options.prepareOnly = true;
|
||||
else if (flag === '--probe-edit-capabilities') options.probeEditCapabilities = true;
|
||||
else if (flag === '--list-models') options.listModels = true;
|
||||
else if (valueFlags.has(flag) || flag === '--image' || flag === '--part') {
|
||||
const value = argv[++i];
|
||||
@@ -80,7 +97,19 @@ export function parseOptions(argv) {
|
||||
export async function main(argv = process.argv.slice(2)) {
|
||||
const options = parseOptions(argv);
|
||||
if (options.help) { console.log(help); return; }
|
||||
if (!['understanding', 'completion', 'segmentation'].includes(options.stage)) throw new Error('--stage 仅支持 understanding / completion / segmentation');
|
||||
if (!['understanding', 'completion', 'segmentation', 'spine'].includes(options.stage)) throw new Error('--stage 仅支持 understanding / completion / segmentation / spine');
|
||||
if (options.stage === 'spine') {
|
||||
const allowed = new Set(['stage', 'inputDir', 'outDir', 'completionDir', 'fallbackPolicy', 'drawOrderFile',
|
||||
'checkAngle', 'genielabsDir', 'python', 'resume', 'dryRun', 'mode', 'imagePaths', 'partIds']);
|
||||
if (Object.keys(options).some((key) => !allowed.has(key)) || options.imagePaths.length || options.partIds.length ||
|
||||
(options.mode && options.mode !== 'annotated-parts')) throw new Error('Spine 为独立离线阶段,不接受模型配置、请求或补全参数');
|
||||
const result = await runSpine(options);
|
||||
if (!result.dryRun) console.log(`Spine ${result.reused ? '复用完成' : '导出完成,需复核'}:${path.join(result.out, 'report.html')}`);
|
||||
return result;
|
||||
}
|
||||
if (['completionDir', 'fallbackPolicy', 'drawOrderFile', 'checkAngle', 'genielabsDir'].some((key) => options[key] !== undefined)) {
|
||||
throw new Error('Spine 导出参数仅用于 --stage spine');
|
||||
}
|
||||
if (options.partsFile && (options.stage !== 'understanding' || options.listModels)) throw new Error('--parts-file 仅用于离线结构理解');
|
||||
if (options.streamRaw !== undefined && !['true', 'false'].includes(options.streamRaw)) throw new Error('--stream 仅支持 true 或 false');
|
||||
options.stream = options.streamRaw === undefined ? undefined : options.streamRaw === 'true';
|
||||
@@ -90,7 +119,8 @@ export async function main(argv = process.argv.slice(2)) {
|
||||
}
|
||||
const loaded = loadConfig(options);
|
||||
const partCompletion = options.stage === 'completion' && options.inputDir;
|
||||
if (!partCompletion && (options.prepareOnly || options.candidatesDir || options.partIds.length || options.completionSize || options.extensionRadius || options.candidateSizePolicy)) {
|
||||
if (!partCompletion && (options.prepareOnly || options.candidatesDir || options.partIds.length || options.completionSize || options.extensionRadius || options.candidateSizePolicy ||
|
||||
options.completionProfile || options.completionReference || options.completionMask || options.completionReview || options.probeEditCapabilities)) {
|
||||
throw new Error('部件补全参数需要 --stage completion --input-dir');
|
||||
}
|
||||
if (partCompletion) {
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
// Optional, offline compatibility check using a caller-provided official Spine 4.2 core IIFE.
|
||||
// node tools/verify-spine-runtime.mjs --runtime-file /path/spine-core.js --input-dir output/spine
|
||||
import assert from 'node:assert/strict';
|
||||
import {readFileSync} from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import vm from 'node:vm';
|
||||
import {readImage} from '../lib/images.mjs';
|
||||
|
||||
const options = {};
|
||||
for (let i = 2; i < process.argv.length; i += 2) {
|
||||
const flag = process.argv[i];
|
||||
if (!['--runtime-file', '--input-dir'].includes(flag) || !process.argv[i + 1]) {
|
||||
throw new Error('Usage: node tools/verify-spine-runtime.mjs --runtime-file spine-core.js --input-dir output/spine');
|
||||
}
|
||||
options[flag] = process.argv[i + 1];
|
||||
}
|
||||
if (!options['--runtime-file'] || !options['--input-dir']) throw new Error('Both --runtime-file and --input-dir are required');
|
||||
const context = vm.createContext({console});
|
||||
vm.runInContext(readFileSync(options['--runtime-file'], 'utf8'), context, {timeout: 10000});
|
||||
const spine = context.spine;
|
||||
const input = path.resolve(options['--input-dir']);
|
||||
const assembly = JSON.parse(readFileSync(path.join(input, 'assembly.json')));
|
||||
const json = JSON.parse(readFileSync(path.join(input, 'skeleton.json')));
|
||||
assert.match(json.skeleton.spine, /^4\.2\./u);
|
||||
const atlas = new spine.TextureAtlas(readFileSync(path.join(input, 'atlas/skeleton.atlas'), 'utf8'));
|
||||
for (const page of atlas.pages) {
|
||||
const png = readImage(path.join(input, 'atlas', page.name), {pngOnly: true});
|
||||
assert.equal(page.width, png.width);
|
||||
assert.equal(page.height, png.height);
|
||||
// CPU geometry needs region metadata, not a WebGL texture upload.
|
||||
page.setTexture({setFilters() {}, setWraps() {}, getImage() { return {width: png.width, height: png.height}; }});
|
||||
}
|
||||
const data = new spine.SkeletonJson(new spine.AtlasAttachmentLoader(atlas)).readSkeletonData(json);
|
||||
const skeleton = new spine.Skeleton(data);
|
||||
function vertices(slot) {
|
||||
const values = new Float32Array(8);
|
||||
slot.getAttachment().computeWorldVertices(slot, values, 0, 2);
|
||||
assert.ok([...values].every(Number.isFinite));
|
||||
return values;
|
||||
}
|
||||
function close(actual, expected, label) {
|
||||
assert.ok(Math.abs(actual - expected) < .002, `${label}: ${actual} != ${expected}`);
|
||||
}
|
||||
skeleton.setToSetupPose();
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
const originals = new Map();
|
||||
const assets = new Map(assembly.parts.map((p) => [p.id, p]));
|
||||
for (const slot of skeleton.slots) {
|
||||
const values = vertices(slot);
|
||||
originals.set(slot.data.name, values);
|
||||
const [x0, y0, x1, y1] = assets.get(slot.data.name).crop_xyxy;
|
||||
const xs = [values[0], values[2], values[4], values[6]];
|
||||
const ys = [values[1], values[3], values[5], values[7]];
|
||||
close(Math.min(...xs), x0 - assembly.width / 2, 'setup x0');
|
||||
close(Math.max(...xs), x1 - assembly.width / 2, 'setup x1');
|
||||
close(Math.min(...ys), assembly.height - y1, 'setup y0');
|
||||
close(Math.max(...ys), assembly.height - y0, 'setup y1');
|
||||
const region = atlas.findRegion(slot.data.name);
|
||||
assert.equal(region.originalWidth, x1 - x0);
|
||||
assert.equal(region.originalHeight, y1 - y0);
|
||||
}
|
||||
let samples = 0;
|
||||
for (const animation of data.animations) {
|
||||
if (animation.name === 'setup') continue;
|
||||
const boneName = animation.name.replace(/^check__/u, '');
|
||||
const frames = json.animations[animation.name].bones[boneName].rotate;
|
||||
for (const time of [.5, 1.5]) {
|
||||
skeleton.setToSetupPose();
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
const bone = skeleton.findBone(boneName);
|
||||
const px = bone.worldX, py = bone.worldY;
|
||||
const angle = frames.find((f) => f.time === time).value * Math.PI / 180;
|
||||
animation.apply(skeleton, 0, time, false, [], 1, spine.MixBlend.replace, spine.MixDirection.mixIn);
|
||||
skeleton.updateWorldTransform(spine.Physics.none);
|
||||
for (const slot of skeleton.slots) {
|
||||
let parent = slot.bone;
|
||||
while (parent && parent !== bone) parent = parent.parent;
|
||||
const before = originals.get(slot.data.name), after = vertices(slot);
|
||||
for (let i = 0; i < 8; i += 2) {
|
||||
const x = before[i] - px, y = before[i + 1] - py;
|
||||
close(after[i], parent ? px + x * Math.cos(angle) - y * Math.sin(angle) : before[i], 'rotated x');
|
||||
close(after[i + 1], parent ? py + x * Math.sin(angle) + y * Math.cos(angle) : before[i + 1], 'rotated y');
|
||||
}
|
||||
}
|
||||
samples++;
|
||||
}
|
||||
}
|
||||
console.log(JSON.stringify({input, spine_version: data.version, bones: data.bones.length,
|
||||
slots: data.slots.length, animations: data.animations.length, rotation_samples: samples,
|
||||
setup_geometry_verified: true, descendant_rotation_verified: true, browser_render_verified: false}, null, 2));
|
||||
Reference in New Issue
Block a user