diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 11bc64ec6..1b8337812 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -4583,3 +4583,14 @@ - 失败与限制：两条图标拆分路径共同限制单边 `4096`、总像素 `2048×2048`、最多 `64` 个切片，并在持久化前完成校验。自动拆分仍是 best-effort，失败后保留整张透明图集并返回 `sliceWarning`；手动拆分失败返回接口错误。UI 设计图素材提取继续使用全连通域识别，不受提示词数量影响。 - 验证方式：调整既有前端提交、Prompt、连通域切片、上限和响应契约测试，不新增仅用于证明旧解析函数已删除的测试；运行前后端定向测试、类型与 Rust 检查、编码检查和 `git diff --check`。 - 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/openapi/genarrative-external-v1.openapi.json`。 + +## 2026-07-30 已有静态图片增加免费一键完美像素化 + +- UI 决策：图片选中浮动工具栏的栅格处理顺序固定为 `裁扩 → 去除背景 → 完美像素`。完美像素只对当前活动的静态栅格图层一键执行，不打开参数面板；音频、视频、图片序列和 `character-animation` 不显示。请求期间按 layer id 禁用并显示 busy，首个 await 前用同步 ref 防双击重复提交；结果保留源图并在右侧新增同尺寸 PNG。 +- API 与执行边界：新增登录态 `POST /api/editor/images/pixel-art-snaps`，复用 `platform-image` 纯内存 snapper、进程级并发 2、30 秒排队加处理总预算以及既有输入尺寸上限。该入口免费 inline，不调用外部 provider，不创建 `external_generation_job`，不打开或刷新任务侧栏，也不进入泥点扣费 / 退款；它与生成请求 `style="pixelArt"` 的 best-effort 后处理是两个契约。 +- 媒体与归属：前端先创建关闭 composer 的右侧占位，再解析或上传源图以取得稳定引用，随后 flush 包含该占位的当前项目布局；正式请求使用 `sourceImageSrc` 承载源图 `objectKey / resourceId / assetId` 候选稳定引用，`projectId / canvasCompletion` 必填且 `canvasCompletion.dialogId` 必须非空，并可携带 `sourceResourceId / assetKind / generationInputs / assetFolderId / assetLabel`。请求禁止 `data:` / `blob:`、signed URL 和普通外链。BFF 下载前必须将候选解析为当前 owner 已登记的私有 OSS object key，并校验 project / resource / asset 归属。 +- 失败与持久化：已有图片入口使用 strict 语义，只接受静态 PNG / JPEG / WebP，拒绝 GIF、APNG、动画 WebP 和非静态素材；必须检测到双轴一致且达到置信门槛的既有逻辑像素网格，普通照片或未识别网格不能使用生成风格的统一网格兜底。读取、解码、输入校验、并发排队、像素规整或 PNG 编码失败 / 超时 / 不适用时，不保存原图副本冒充成功，不执行最终 OSS PUT，也不创建 asset object、project resource、账号素材或结果 layer。成功时只对最终 PNG 做一次 PUT，至多各创建一个 `editor_project_resource` 和一个 `editor_asset`；源图已有正式 project resource 时，结果以 `source_resource_id` 关联该资源，再由 `canvasCompletion` 写入至多一个右侧派生 layer；不保存逻辑低分辨率图、诊断图或前后对比图。 +- 非事务边界：strict 零写入只覆盖首个最终 PNG PUT 前的引用 / owner / 项目 / 类型 / 静态编码 / 元数据 / 网格适用性 / CPU 处理门禁。进入持久化后沿用现有 `OSS + asset object → project resource → editor asset → canvas completion` 非事务顺序，后段失败可能保留此前已确认对象或记录；不做删除补偿或 unsafe POST 自动重放，按 `task_id / object_key / resource_id` 读取权威快照排障，跨系统单事务留待独立 procedure 方案。 +- 占位删除与重试：completion 必须读取当前权威 dialog；若删除已先持久化，只跳过画布 layer / dialog 写回，不得使用请求中的旧 placeholder 复活图层，已经成功持久化的 project resource / 账号素材允许保留。若回包时本地占位已删除，前端不得应用完成快照或写历史；现有布局 CAS 没有 deletion tombstone，因此 completion 先提交、删除保存后冲突的极端竞态仍按权威快照收口，绝对“删除意图胜出”留待 targeted delete / tombstone 方案。该路由是 unsafe POST，客户端不得配置 `EDITOR_REQUEST_RETRY_OPTIONS`；请求字节可能已发出后不因 transport 异常或 `408 / 425 / 429 / 502 / 503 / 504` 自动重放，Bearer 中间件在 handler 前拒绝请求后的既有认证恢复继续保留。结果未知时先 GET 权威项目 / 素材快照，由用户显式决定是否再次执行。 +- 历史边界：成功加入画布时写一条 `perfect-pixel` 历史，中文标签为“完美像素”，并纳入新增结果保护；撤销不得让派生 PNG 消失。像素处理失败或 completion 因占位删除未落画布时不写该历史。 +- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【图片画布】撤销范围与操作提示方案-2026-07-17.md`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`、`docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md`。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 2c169cfe6..e3325f508 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -42,6 +42,12 @@ - 逻辑低分辨率图只存在于内存；snapper 在规整内部使用 nearest 恢复到当前 RGBA 输入尺寸，并直接替换原本即将持久化的最终图片字节。普通图片和角色的该输入已经过前置 Lanczos 交付尺寸归一，或在无法安全归一时保留 provider 实际尺寸；图标输入以已持久化平底原图的实际尺寸为准。nearest 不替代前置尺寸归一，规整完成后不再执行第二次 Lanczos 或其它尺寸恢复。角色和图标应复用 Alpha 回贴阶段已经读取的平底原图；确需重新读取时，最多增加一次对已有 provider 对象的 OSS GET，不得新增 OSS PUT。 - 像素模式的持久化增量必须为零：普通图片仍只上传原有一张最终主图；角色仍只保留原有 provider 原图与透明主图；图标仍只保留原有 provider 原图、透明图集和实际成功的切片。禁止保存逻辑低分辨率图、像素化前后双份主图、预览图、网格诊断图或报告，禁止新增 asset / resource 类型、项目资源、画布 item、队列 job kind 或数据库字段。 - 像素后处理属于 best-effort：失败时保留进入该步骤前的图片，继续原有最终上传与画布完成，并通过既有通用 `warning` 返回非阻断原因，不把任务改为失败或退款。BgFilter 自身失败时仍按原 source-only fallback 收口，像素处理不运行；图标后处理成功后再执行原有自动拆分，拆分告警继续使用现有 `sliceWarning` 语义。 +- 选中已有静态栅格图层后的 `完美像素` 是独立的一键派生操作，不等同于生成请求上的 `style="pixelArt"`。它不打开参数面板，只处理当前活动图层，保留源图，并在源图右侧创建同尺寸 PNG 派生结果；音频、视频、图片序列和 `character-animation` 不显示该按钮。 +- 已有图片像素规整固定调用登录态同源 `POST /api/editor/images/pixel-art-snaps`，复用同一纯内存 Rust snapper、并发许可、输入尺寸上限和 30 秒 CPU 总预算。该入口免费、只走当前 HTTP 请求内的 inline 处理，不创建 `external_generation_job`，不刷新或自动打开任务侧栏，也不进入泥点扣费 / 退款链路。 +- 前端提交前先创建关闭 composer 的右侧生成占位，再解析或上传源图以取得稳定引用，随后 flush 包含该占位的当前项目布局，最后把承载稳定候选引用的 `sourceImageSrc`、必填 `projectId / canvasCompletion`（其中 `canvasCompletion.dialogId` 必须非空）以及可选 `sourceResourceId / assetFolderId / assetLabel` 一次提交。`sourceImageSrc` 优先由当前图层已有的 `objectKey / resourceId / sourceAssetId` 解析；尚未登记的浏览器本地图片必须先上传，正式请求不得包含 `data:` / `blob:`、signed URL 或普通外链。后端在读取源图前必须把该字段解析为当前 owner 已登记的私有 OSS object key，并核对 project / resource / asset 归属。 +- 该已有图片入口使用 strict 语义：只接受静态 PNG / JPEG / WebP，GIF、APNG、动画 WebP、图片序列及其它非静态媒体必须在处理前拒绝；必须检测到双轴一致且达到置信门槛的既有逻辑像素网格，普通照片或未识别网格不能套用生成风格中的 `min(width,height)/64` 兜底像素化。源图读取、解码、尺寸校验、排队、像素规整或 PNG 编码任一步失败 / 超时 / 不适用时，请求失败，不保留原图副本冒充成功，不执行最终 OSS PUT，也不创建 project resource、账号素材或结果图层。成功时只对最终 PNG 执行一次 OSS PUT，并至多各创建一个 `editor_project_resource` 和一个 `editor_asset`，再按 `canvasCompletion` 写回一个派生图层；不得保存逻辑低分辨率图、诊断图或前后对比图。 +- strict 的零写入边界截至首个最终 PNG PUT：所有可预判的引用、归属、类型、静态编码、元数据、网格适用性和 CPU 处理错误必须在此前失败。进入 OSS / SpacetimeDB 持久化后沿用现有非事务顺序，后段 resource / asset / completion 失败可能保留此前已确认对象或记录；客户端不做自动重放或删除补偿，而是按 `taskId / objectKey / resourceId` 重新读取项目和素材快照。 +- `POST /api/editor/images/pixel-art-snaps` 是有副作用的 unsafe POST。客户端不得为它配置 `EDITOR_REQUEST_RETRY_OPTIONS`，请求字节可能已发出后不因 transport 异常或 `408 / 425 / 429 / 502 / 503 / 504` 自动重放；Bearer 中间件在 handler 前以 `401` 拒绝、刷新 token 后的既有认证恢复不属于业务副作用重放，保持通用行为。结果未知时先重新读取项目 / 素材快照，由用户显式决定是否再次执行。处理成功后若对应 generation dialog 的删除已先持久化，后端沿用现有 completion 语义跳过画布插入，不得复活占位或结果图层；回包时本地占位已删除则前端不应用完成快照，已经成功持久化的 project resource / 账号素材可以保留。现有布局 CAS 没有 deletion tombstone，completion 先提交、删除保存后冲突的极端竞态仍按权威快照收口。 ### 角色动作帧抠图像素边界 @@ -64,6 +70,7 @@ ## 数据与持久化 +- 完美像素派生结果沿用 `editor_project_resource / editor_asset / canvasCompletion` 三段式持久化，不新增表、资源类型或队列类型。源图已有正式 project resource 时，成功结果的 `sourceResourceId` 指向该资源；最终 PNG 只 PUT 一次，同一请求至多创建一个 project resource 和一个账号素材。像素处理本身失败时三类业务写入都不发生；像素处理成功但 completion 读取的权威占位已删除时，只保留已创建的资源 / 素材并跳过画布 layer，不能用提交时旧 placeholder 把图层重新插回。 - 新增 `editor_project` 表保存图片画布工程：`projectId`、`ownerUserId`、标题、创建时间和更新时间；历史 layout 字段暂保留为兼容列，不再作为权威画布数据。 - 新增 `editor_canvas` 表保存工程下的画布：`canvasId`、`projectId`、`ownerUserId`、标题、viewport、图层布局 JSON、创建时间和更新时间。当前编辑器使用项目默认画布，后续可扩展为一个 project 下多个 canvas。 - 新增 `editor_asset_folder` 表保存账号级素材文件夹：`folderId`、`ownerUserId`、名称、排序、折叠状态、系统默认标记、创建时间和更新时间。素材文件夹不归属于 project，同一个账号进入任一项目都能看到。 @@ -111,6 +118,7 @@ - `POST /api/editor/images/background-removals`：接收当前图片的 `objectKey`、`resourceId` 或 `assetId` 候选引用，登录态和稳定引用入口校验通过后创建外部生成任务，响应只返回 `queueState`。父 `external-generation-worker` 负责把候选引用解析为已登记、已校验当前账号归属的私有 OSS object key，只向唯一 `bgfilter-worker` 发起一次内部 HTTP RPC，传递 object key、`maxQueueWaitMs`、公式化 `callBudgetMs` 以及固定的 `background_mode=complex + seg_model=birefnet + cross_check=off`；父侧不下载原图、不签发 URL，也不发送 `file` 或 `screen_color`。子 worker 在每次真实 provider attempt 前签发 600 秒 OSS URL，以默认 `Q=2048` admission 保险丝和 provider 并发 `N=16` 限流，取得 provider permit 后才启动 `callBudgetMs`，并对同一次逻辑调用最多执行两次顺序 provider attempt；成功图片以内部 HTTP 二进制 body 返回父流程，父侧不重试已被 worker 接收的内部 RPC（连接从未建立时按调度方案 §5.1 有界重连）。complex 任意最终失败都直接使父任务失败，不进入阿里云或本地键色 fallback。请求可携带 `projectId`、`targetLayerId`、`assetFolderId`、`assetLabel`、`sourceResourceId` 和 `canvasCompletion`；成功后仍由父流程完成最终 OSS / project resource 持久化，有 `canvasCompletion` 时按生成占位写入结果图层，否则沿用旧的目标图层替换路径。provider 令牌只在子 worker 服务端通过 `GENARRATIVE_EDITOR_BGFILTER_TOKEN` 注入，未配置时兼容回退旧 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN`；父子内部调用另使用独立内部 Token。 - `POST /api/editor/icon-spritesheets/generations`：按图标规范图和完整用户需求生成 spritesheet；为兼容现有契约，画布前端把完整文本作为 `iconDescriptions` 的唯一数组元素提交，不按分隔符或语义枚举解析数量。api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet，并与手动 `POST /api/editor/icon-spritesheets/slices` 复用同一套全连通域识别：识别多少个有效素材就拆多少个，按视觉阅读顺序命名为 `素材 N`，不读取 `iconDescriptions` 数量决定切片数。两条拆分路径共同限制单边 `4096`、总像素 `2048×2048`、最多 `64` 个切片。请求支持 `model`、`screenColor`、`segModel`、`aspectRatio`、`imageSize`、`priceMudPoints`、`projectId`、`assetFolderId` 和 `generationInputs`；`priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位（如 `nanobanana2` 的 `0.5K / 1K / 2K` 或 `gpt-image-2` 的 `1K / 2K`），后端用 `editor_generation_config` 校验后才调用上游；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但自动拆分失败时保留整张透明图并返回非阻断 `sliceWarning`，手动拆分失败时返回接口错误。响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`。 - `POST /api/editor/images/generations` 与 `POST /api/editor/icon-spritesheets/generations` 还可携带可选 `style`；公开合法字符串为 `none / pixelArt`，兼容归一化、支持的 `kind`、非阻断告警和零新增持久化规则以“静态图片风格与像素规整边界”为准。`POST /api/editor/ui-designs/assets/extractions` 不接受该字段。 +- `POST /api/editor/images/pixel-art-snaps`：对已登记的静态图片执行免费的同步完美像素化。请求使用 `sourceImageSrc` 承载当前 owner 可读取的 `objectKey / resourceId / assetId` 候选稳定引用，`projectId / canvasCompletion` 必填且 `canvasCompletion.dialogId` 必须非空，`sourceResourceId / assetKind / generationInputs / assetFolderId / assetLabel` 可选；拒绝内联媒体、未登记对象和非静态栅格素材。像素处理使用 strict 失败语义且不进入外部生成队列；成功时只持久化一张最终 PNG，并返回对应 project / resource / asset 快照。权威 `canvasCompletion.dialogId` 仍存在时在源图右侧完成占位；其删除已先持久化时只跳过画布回填，不删除已经成功写入的资源或账号素材。 - `POST /api/editor/ui-designs/assets/extractions`：前端把红色框选轮廓绘入本地临时图后，先将该图上传 OSS 并确认 asset object，再以返回的 `objectKey` 作为参考图入队；Data URL / Blob URL 只允许停留在上传前的浏览器临时态。接口固定 `gpt-image-2` 和自动决策纯色背景素材提取提示词生成素材 spritesheet；api-server 先保存带纯色背景 spritesheet 源图，透明处理成功后再保存透明 spritesheet 并按连通域尝试拆分为 `素材 1..N`，返回结构复用图标 spritesheet 响应。请求必须携带 `screenColor`、`segModel`、`aspectRatio: "1:1"`、`imageSize: "1K" | "2K"` 和 `priceMudPoints`；框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交，超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格，只允许 `1:1 / 1K / 2K`。透明处理最终失败时只保存并返回原图主结果，不生成透明图或切片；透明图成功但拆分失败时保留整张透明图并返回 `sliceWarning`。请求可携带 `projectId`、`assetFolderId`、`generationInputs` 和 `spritesheetLabel`，响应只返回实际产物对应的 project / resource / asset 快照及可选通用 `warning`；前端按后端快照落画布，不补造缺失产物。 - `POST /api/editor/images/edits`：按提示词和当前图片的已登记 `objectKey` / `resourceId` 修改图片，返回新的生成图片元数据；图片快速编辑当前只提交 `sourceImageSrc`，不提交隐藏的 `referenceImageSrcs`，并随用户当前选择提交 `model / aspectRatio / imageSize / size`。api-server 必须先归一模型再选择 VectorEngine 协议：`nanobanana2` 调用 `/v1beta/models/{model}:generateContent` 并把原图作为 `inline_data`、比例和清晰度写入 `generationConfig.imageConfig`；`gpt-image-2` 调用 `/v1/images/edits` multipart。gpt-image-2 路径在 provider 边界把目标尺寸和所有 multipart 参考图临时补齐到 16 的倍数；nanobanana2 路径保留 provider 的比例 / 清晰度请求，但两条路径回图后都以统一业务目标尺寸尝试归一。只允许缩小和轻微裁切；回图任意一边小于目标或比例偏差过大时保留 provider 实际回图及尺寸，并返回通用 `warning`，不得放大伪造所选档位。无论是否发生尺寸恢复都只创建一个 project resource / 账号素材，不显示重复“原始输出”。provider 对齐尺寸或原生 K 档像素不得泄漏到正常完成的最终响应、资源或图层 Resolution；变换失败降级时以实际 provider 原图尺寸为准。本地红框标记图必须先上传再提交 objectKey；请求携带 project / asset 上下文时由后端创建新 resource / asset，前端只消费响应快照。 - `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音、默认联网搜索标记和泥点价格生成视频。前端可选模型为 `seedance2.0-fast`、`seedance2.0`、`kling3.0`、`kling3.0-omni`，默认 `seedance2.0-fast`；后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`，将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`，两者不得混用。后端允许 6 类比例、4 到 15 秒整数、`480p / 720p / 1080p`，并拒绝 `seedance2.0-fast + 1080p`；`sound=on/off` 映射 Ark `generate_audio=true/false`。后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS；请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `asset` 快照，基础响应返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。 @@ -155,7 +163,7 @@ - 发送消息后，面板先展示本地用户消息和请求等待态，再应用普通 JSON 响应中的 `deltaMessages`；客户端取消等待只终止本次 transport 等待，不把已经确认入队的外部生成任务改成停止态。 - Agent 工具任务完成并懒回填后，消息内缩略图只作纯预览，不显示名称也不点击聚焦图层；前端同时重新读取工程快照和素材库。对话入口触发生成时不创建“即将生成”画布占位，生成完成后由后端 `canvasCompletion` 落新图层。规划或工具失败时消息内必须保留可回读的失败状态和错误气泡，不能只弹一次性 toast 或返回瞬时 `errorMessage`。 - 画布 Agent 会话刷新后能从后端恢复会话标题、消息、附件和生成记录；前端不得根据本地临时状态伪造会话持久化结果。 -- 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`；父流程解析并校验私有 OSS object key 后只调用一次唯一内部 `bgfilter-worker` 的 complex 链路，子 worker 负责签发 600 秒 URL、`N / Q` 限流和最多两次顺序 provider attempt，complex 失败不接入 fallback，成功二进制返回后仍由父流程完成最终持久化。有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K` 和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和自动决策纯色背景素材提取提示词。透明处理及拆分正常完成时，透明 spritesheet 和拆分素材都按后端快照保留为画布图层；透明处理失败时仅原图作为主结果，既不要求透明图也不要求切片；透明图成功但拆分失败时保留整张透明图并展示拆分告警。三种完成结果都以后端项目快照为准。 +- 图片选中后的浮动工具栏按钮顺序固定为：快速编辑、分割线、裁扩按钮、去除背景按钮、完美像素按钮、UI设计图专属提取素材、角色图专属生成动画、分割线、重绘、下载按钮。完美像素只对当前静态栅格图层一键执行，按钮在请求期间按 layer id 进入 disabled / busy，首个 await 前用同步 ref 抢占，连续点击不得重复提交；完成后保留源图并在右侧显示派生 PNG，失败占位保留明确错误且释放 busy。裁扩通过画布边界拖拉完成，不再展示四边数值输入；默认自由比例，选择固定比例后拖拉边界保持对应比例，完成后在原素材旁边新增裁扩结果图层，扩展区域透明填充。去除背景调用同源 BFF `POST /api/editor/images/background-removals`；父流程解析并校验私有 OSS object key 后只调用一次唯一内部 `bgfilter-worker` 的 complex 链路，子 worker 负责签发 600 秒 URL、`N / Q` 限流和最多两次顺序 provider attempt，complex 失败不接入 fallback，成功二进制返回后仍由父流程完成最终持久化。有项目上下文时先在画布创建关闭面板的去背景生成占位，完成后由后端通过 `canvasCompletion` 把新 project resource 写入该占位并返回快照，无占位上下文时才用新的 project resource 引用替换当前图层。画布任务侧栏按“排队/生成中”和“已完成”分页，生成中排在排队前，生成中耗时从任务开始时间戳实时计算，排队中不计时；进行中任务只显示阶段文本和已用时，不显示百分比；完成态生成任务副标题显示用户提示词并单行截断；点击任务只聚焦对应画布内容，不激活生成面板或改变任务顺序，聚焦时必须预留图片上方工具栏、底部工具栏和可见生成对话框空间。UI设计图的提取素材必须先进入红框素材框选状态，默认启用矩形框选，右侧框选工具与快速编辑统一且可再次点击取消启用态，当前启用工具按钮必须保持高亮。素材提取面板必须在素材下方，使用与生成新素材一致的面板宽度和底部模型 / 按钮样式，提示语显示 `使用框选工具框选你希望从画面中提取的素材`，并展示按原图坐标准确裁剪的框选区域截图预览、固定模型 `gpt-image-2`、左下角计划规格 `1:1·1K/2K` 和 `提取 · N泥点` 按钮，不显示额外取消按钮；点击素材和面板以外的画布区域即退出 UI 素材提取。至少框选一个区域后才可提交，前端把红色轮廓绘入原图后固定走 `gpt-image-2` 和自动决策纯色背景素材提取提示词。透明处理及拆分正常完成时，透明 spritesheet 和拆分素材都按后端快照保留为画布图层；透明处理失败时仅原图作为主结果，既不要求透明图也不要求切片；透明图成功但拆分失败时保留整张透明图并展示拆分告警。三种完成结果都以后端项目快照为准。 - 重绘生成资源后，右侧出现新生成结果图层，并自动 fit 原图 + 新图，且重绘面板保持打开。 - 快速编辑 / 重绘站内 public 示例图、历史 generated 图或 OSS generated 图时，优先复用当前图层已有 `objectKey` / `resourceId` / `sourceAssetId`；尚未登记且没有稳定引用的浏览器本地图片或普通 public 图片路径都必须先上传并取得 objectKey。前端不得再把正式对象下载成 `data:image/*;base64,...` 后提交，也不得把 Data URL / Blob URL 写入外部生成持久任务 JSON；后端收到引用后统一做 owner 归属校验并签名读取。 - 快速编辑不保留额外参考图入口；点击修改时只把原图或红框序号标注图作为 `/api/editor/images/edits` 的 `sourceImageSrc` 提交给后端。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index 8ed03321b..8bec4f3ba 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -244,9 +244,12 @@ npm run check:server-rs-ddd 7. 队列任务按 `job_id + claim_attempt` 使用独立 consume/refund ledger。新 attempt 结算旧 attempt 时必须先写 `asset_operation_wallet_settlement`：旧 consume 已存在则原子退款，尚不存在则写取消 intent；迟到 consume 在同一 SpacetimeDB 事务内看到 intent 后必须失败关闭。重复 consume/refund 只有用户、金额、来源和配对 ledger 全部一致时才可视为幂等成功。lease 过期时只有 `attempt < max_attempts` 才能递增并重领；最终 attempt 已耗尽时，claim transaction 必须直接把 job 收口为 `failed`、清理 lease、写失败事件并结算当前 attempt，不能再把任务返回 worker 或调用 provider。 8. 音频生成的编辑器链路虽然任务提交和结果发布分离，仍必须把提交时后端计算出的模型价格写入 `AudioAssetBindingTarget.billing_points_cost`，最终发布落资产时按该价格扣费；创作音频目标未提供该字段时才使用旧的创作音频固定成本。 9. 编辑器进入外部生成持久队列的图片生成、图片修改、去背景、图标 spritesheet、UI 设计图提取、角色动作和视频参考图，调用方必须提交 `objectKey` / `resourceId` / `assetId` 候选引用；BFF 只做内联媒体与 payload 门禁，登记状态和归属由 worker 统一解析。任务 `request_payload_json` / `result_payload_json` 任意层级都禁止 `data:` / `blob:`，并受统一字节上限保护。无效普通字符串可以入队，但必须在签名和 provider 调用前失败；本次不增加 API 侧数据库查询或同步 owner 校验。若以后要求无效引用同步返回 400，应作为独立改造。objectKey 最终必须归属于当前账号的 `editor_project_resource`、`editor_asset` 或 `asset_object`，由 worker 在解析后、签名读取 OSS 前完成归属校验。本地红框序号标注图必须先上传并确认对象，再把 objectKey 入队；不得把既有 objectKey 下载成 Data URL 后写入任务。图标素材和 UI 素材提取的额外参考图必须真正传入 provider，不得只写入 `generationInputs` 展示快照；图片快速编辑当前不开放额外参考图。UI 素材提取额外参考图上限为 5 张，普通图片生成上限 5 张，图标素材上限 8 张额外参考图。同步且不持久化的历史兼容入口即使仍能解析 Data URL，也不能把该值转存到工程、素材、元数据、审计或任务表。 +10. 已有静态图片的 `POST /api/editor/images/pixel-art-snaps` 是免费 inline 派生操作，不调用外部 provider、不创建 `external_generation_job`、不读写泥点 ledger，也不进入任务侧栏。免费不放宽 owner、稳定引用、输入上限、持久化或处理阶段零持久化门禁。 ## 外部服务与资产 +- 已有图片完美像素化：登录态 `POST /api/editor/images/pixel-art-snaps` 使用 `sourceImageSrc` 承载 `objectKey / resourceId / assetId` 候选稳定引用，要求 `projectId / canvasCompletion` 且 `canvasCompletion.dialogId` 必须非空，并可携带 `sourceResourceId / assetKind / generationInputs / assetFolderId / assetLabel`；BFF 必须在下载前将候选解析为当前 owner 已登记的私有 OSS object key，并校验 project / resource / asset 归属，拒绝 `data:` / `blob:`、signed URL、普通外链和音频、视频、图片序列等非静态栅格输入。编码门禁只接受静态 PNG / JPEG / WebP，明确拒绝 GIF、带 `acTL` 的 APNG 及带动画标志 / `ANIM` / `ANMF` chunk 的 WebP。处理复用 `platform-image` 纯内存 snapper、进程级并发 `2`、30 秒排队加处理总预算、单边 `10000` 与总像素 `8294400` 上限，但该入口采用 strict 而非生成风格的 best-effort 语义：必须检测到双轴一致且达到置信门槛的既有逻辑像素网格，不能对普通照片或未识别网格使用生成风格中的 `min(width,height)/64` 兜底；读取、解码、校验、排队、规整、PNG 编码任一步失败 / 超时 / 不适用时，在最终持久化前返回错误，OSS PUT、asset object、project resource、账号素材和画布 layer 增量都必须为零。成功结果保留源图，只对最终 PNG 做一次 OSS PUT，并至多各创建一个 `editor_project_resource` 和一个 `editor_asset`；源图已有正式 project resource 时，结果资源以 `source_resource_id` 关联该资源，再按 `canvasCompletion` 尝试写入一个右侧派生 layer。completion 读取的权威 dialog 已删除时沿用现有语义跳过画布写入，不得用请求中的旧 placeholder 复活图层；已经成功落库的 resource / asset 可以保留。客户端回包时若本地 dialog 已删除，不应用完成快照；现有布局 CAS 没有 deletion tombstone，completion 先提交、删除保存后冲突的极端竞态仍按权威快照收口。客户端不得为该 unsafe POST 配置 `EDITOR_REQUEST_RETRY_OPTIONS`，请求字节可能已发送后不因 transport 异常或 `408 / 425 / 429 / 502 / 503 / 504` 自动重放；Bearer 中间件在 handler 前拒绝请求后的既有认证恢复继续保留。结果未知时先 GET 权威项目 / 素材快照。 +- 完美像素持久化边界：所有可判定的稳定引用、owner、项目、来源资源、素材类型、静态编码、元数据、网格适用性、排队、CPU、解码、规整和编码校验都必须在首个最终 PNG PUT 前完成。进入持久化后沿用既有跨 OSS 与 SpacetimeDB 的非事务边界，依次确认 PNG / asset object、project resource、账号素材和 completion；后段失败可能保留此前已经确认的对象或记录，不做破坏性删除补偿，也不自动重放 unsafe POST。排障按响应或日志中的 `task_id / object_key / resource_id` 重新读取权威项目与素材快照；跨系统单事务 completion 留待独立 procedure 方案收口。 - LLM：通用 LLM 门面继续使用 `GENARRATIVE_LLM_*`；创意 Agent `gpt-5.4-mini` Chat Completions 文本链路已于 2026-06 从 APIMart 迁移到 VectorEngine，使用 `VECTOR_ENGINE_BASE_URL` / `VECTOR_ENGINE_API_KEY` 构造 OpenAI-compatible client，`api-server` 会把未带 `/v1` 的 VectorEngine base URL 规范化到 `/v1` 后请求 `/chat/completions`。通用 `/api/llm/chat/completions` 代理使用 `GENARRATIVE_LLM_PROVIDER=openai-compatible`、`GENARRATIVE_LLM_BASE_URL=https://api.vectorengine.cn/v1`、`GENARRATIVE_LLM_MODEL=gpt-5.4-mini`；未单独配置 `GENARRATIVE_LLM_API_KEY` 时可复用 `VECTOR_ENGINE_API_KEY`。`APIMART_BASE_URL` / `APIMART_API_KEY` 只作为历史残留，不再作为创意 Agent gpt-5.4-mini 客户端来源；后续排障时优先确认 VectorEngine `/v1/models`、`/v1/chat/completions` 和 `/v1/responses` 可用性。 - 图片生成：VectorEngine 图片 provider 归属 `platform-image`，密钥只在后端环境变量中；逻辑 SKU 与 provider 首选模型均固定为 `gpt-image-2`，只在明确模型不可用、408 / 非拒绝类 429 / 5xx、响应解析失败或非拒绝类缺图时切换兜底模型 `gpt-image-2-c`。401 / 403、普通参数或安全拒绝、本地配置 / 参考图错误、无法确认上游是否已受理的发送错误、request budget 耗尽和生成成功后的图片下载失败不得切模型。一次业务请求总发送上限仍为 5 次；切换兜底模型会消耗后续 attempt，不允许两个模型各重试 5 次。`api-server` 内的 `openai_image_generation.rs` 只是兼容调用面和外部失败审计桥接，不再承载 provider 协议实现。实际外部生成运行记录统一落 `tracking_event`，`event_key = external_generation_run`，metadata 记录开始 / 结束时间、耗时、状态、成功标记、失败原因、provider task id、结果摘要和 recovered failure 数量；首选模型失败但兜底模型成功时，首选失败仍落 `external_api_call_failure`。DashScope 只按仍在使用的历史能力单独处理，不作为 GPT-image-2 兜底。VectorEngine `/v1/images/generations` 和 `/v1/images/edits` 上游 POST 使用 `libcurl` 发送；`reqwest` 只保留给参考图 URL 下载和响应中图片 URL 下载。`/v1/images/edits` 的 multipart 参考图必须作为 libcurl 文件上传 part 发送，字段名为 `image`，实现上使用 `Form::buffer(file_name, bytes)` 并设置 `Content-Type`；不能只用 `contents(...).filename(...)`，否则上游会把请求转码为缺少图片并返回 `image is required`。`request_send` 阶段的 curl timeout / connect error 按可重试传输错误处理，最多尝试 5 次，并使用指数退避加短抖动；排障时优先看 `attempt`、`max_attempts`、`retry_delay_ms`、`fallback_from_model`、`fallback_to_model`、`reference_image_bytes_total` 和 `request_params`，不要把 `SendRequest` 当成上游业务错误。 - 抠图输入以私有 OSS 作为内存生命周期边界：生成原图和角色动作抽取帧上传时消费图片字节所有权，上传完成后不保留原图缓冲；手动去背景直接解析并校验已有 OSS object key，不下载原图。BgFilter 必须为 object key 签发 600 秒 GET URL 并通过 multipart `image_url` 提交，不用 `file` 重传；flat 链路进入阿里云 fallback 时由 `platform-matting` URL 接口单独下载并上传 `AuthorizeFileUpload` 临时对象，在推理前释放下载缓冲，继续 fallback 到本地键色时再单独下载一次原图，本地产出后释放本次原图下载缓冲。签名 URL 不得写入日志、审计或持久化。 @@ -562,6 +565,7 @@ npm run check:server-rs-ddd - Rust 结构体：`EditorCanvasGenerationDialog` - 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs` - 说明：图片画布生成对话框表，保存 canvas / project / owner、生成模式、状态、可选 source / generated layer、占位几何和有界扩展字段；typed 列为快照真相，`dialog_json` 只保留未结构化参数。当前 worker completion 以读取时 revision 调用 CAS 保存，冲突时拒绝覆盖；`job_id + worker_id + lease_token` 栅栏下的资源 / layer / dialog / job 单事务完成仍是后续收口。 +- 完美像素 completion：同步处理成功后按当前 revision 重新读取权威 dialog；目标 dialog 存在时只写入一个派生 layer 并关联结果，目标 dialog 的删除已先持久化时跳过 layer / dialog 写回，不得按请求快照重建占位。该分支不属于 external job completion，允许此前已成功创建的 resource / asset 保留。 - 索引：按 canvas 和 project 读取结构化行；当前未建立 external job 二级索引。 ### `editor_canvas_layout_migration` @@ -576,6 +580,7 @@ npm run check:server-rs-ddd - Rust 结构体：`EditorProjectResource` - 源码：`server-rs/crates/spacetime-module/src/editor_project_storage.rs` - 说明：图片画布工程资源元数据表，保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind`、`generation_inputs_json` 和历史 `public_showcase_enabled`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别；`generation_inputs_json` 保存用户可见生成输入快照，供图片信息页刷新后恢复。`public_showcase_enabled` 只保留旧接口兼容，不再作为 `/creation` 的 `陶泥儿精选` 事实源；精选公开改由账号级生成素材提交 `editor_showcase_asset` 审核决定。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端；前端只保存稳定 `resource_id` 布局引用，不能把同一生成结果再次作为正式业务真相写入。项目封面快照也落在该表，使用 `asset_kind = project-cover-snapshot`、`source_type = uploaded` 和私有 OSS / asset object 引用，代表画布当前视口栅格化后的静态封面；项目列表和创作主页最近项目只读取最新封面快照资源，不在列表页根据 layout 临时拼画布。从账号级素材库把同一生成素材拖回同一项目画布时，后端优先复用同项目内同源同媒体资源，避免每个图层实例都插入新的资源行。账号级素材删除不级联删除该表，避免历史画布丢图。结构化 canvas 的几何、层级、分组和资源引用以 `editor_canvas_layer` 为权威，生成器对象以 `editor_canvas_generation_dialog` 为权威；legacy canvas 才在 2 MiB 上限内从 `editor_canvas.layers_json` 兼容读取。新写入不再把素材生成输入快照作为图层布局真相保存。历史普通图层缺资源只能由 migration operator 调用 `repair_editor_canvas_resources_and_return` 定向修复：procedure 每次只处理一个尚无迁移记录的 legacy canvas，校验 owner/project、revision、canvas/project 两份 raw layout SHA-256、精确 layer/resource/sourceResourceId、同工程替换资源与 private asset_object 谱系；图片只替换引用，音频只恢复经核验的 `420x120` 项目资源行。运维入口 `npm run spacetime:editor-canvas-resources:repair` 默认 dry-run，apply 必须绑定 plan SHA-256 并在成功后自动复核 already-repaired，禁止手工 SQL 绕过事务 guard。 +- 完美像素资源：处理成功时只允许一个最终 PNG 对象对应一个新 project resource；源图已有正式 project resource 时，`source_resource_id` 指向该资源；结果尺寸与最终 PNG 一致，不写逻辑低分辨率图、诊断图或前后对比图。成功时同时创建一个同源 `editor_asset`，请求省略素材文件夹时落入默认素材文件夹；completion 因权威 dialog 的删除已先持久化而跳过画布写回时，这两类已确认资源无需回滚。 - 索引：`by_editor_project_resource_project_id`、`by_editor_project_resource_owner_user_id`。 ### `editor_asset_folder` diff --git a/docs/【图片画布】撤销范围与操作提示方案-2026-07-17.md b/docs/【图片画布】撤销范围与操作提示方案-2026-07-17.md index 1385fc039..57c2aef3c 100644 --- a/docs/【图片画布】撤销范围与操作提示方案-2026-07-17.md +++ b/docs/【图片画布】撤销范围与操作提示方案-2026-07-17.md @@ -15,7 +15,7 @@ 允许撤销的典型操作包括移动图片、移动生成结果、调整层级、组合与取消组合、删除或剪切图片、隐藏图片、锁定与解锁、翻转、修改素材类型和调整画布视图。删除、剪切和隐藏允许撤销，是因为目标只会让内容重新出现。 -添加素材、上传到画布、粘贴、创建副本、生成图片、扩图新增结果、显示隐藏图片、替换图片以及其它会让当前结果消失的撤销必须被安全检查阻止。`Ctrl+C`、选择变化、滚轮或抓手视口移动、导出下载、项目重命名、素材库后端删除和生成任务副作用不进入画布历史；素材库后端删除发生后，同时剪除撤销栈和恢复栈中所有包含关联图层的目标快照，不能让更早的画布历史复活已删除素材。 +添加素材、上传到画布、粘贴、创建副本、生成图片、扩图新增结果、完美像素新增结果、显示隐藏图片、替换图片以及其它会让当前结果消失的撤销必须被安全检查阻止。`Ctrl+C`、选择变化、滚轮或抓手视口移动、导出下载、项目重命名、素材库后端删除和生成任务副作用不进入画布历史；素材库后端删除发生后，同时剪除撤销栈和恢复栈中所有包含关联图层的目标快照，不能让更早的画布历史复活已删除素材。 恢复同样执行动态安全检查。移动、层级、分组、锁定、翻转和视图等不会减少内容的操作可以恢复；重新执行删除、剪切、隐藏、删除生成结果或替换当前素材时必须被阻止。 @@ -24,6 +24,7 @@ - 撤销栈和恢复栈均保存操作类型、目标 `CanvasHistorySnapshot` 和创建时间，分别最多保留 60 条。 - 新画布操作把操作前快照写入撤销栈并清空恢复栈；成功撤销把当前快照写入恢复栈，成功恢复把当前快照写回撤销栈。 - 操作类型用于生成用户提示，并对添加、上传、生成和替换等明确会移除当前结果的撤销做保护；其它操作能否应用由当前快照与目标快照的差异检查决定。 +- 已有图片完美像素化成功落入画布前记录独立 `perfect-pixel` 历史类型，中文提示使用“完美像素”。该类型与 `generate-image`、`expand-image`、`remove-background` 一样属于新增结果保护操作：撤销不得删除派生 PNG，源图继续保留也不改变这一保护语义。像素处理失败、超时或不适用时不写历史。 - 安全检查以稳定的 `layer.id` 判断当前图层是否仍存在；内容身份优先比较对象存储 key、对象标识和媒体地址，序列帧结果比较完整帧列表。`resourceId`、`sourceResourceId`、`sourceAssetId` 等内部关联 ID 的延迟回填不视为图片替换。 - 恢复历史快照时，相同 ID 的图层以当前对象为权威，只从目标快照覆盖 `x`、`y`、`zIndex`、`groupId`、`assetKind`、`hidden`、`locked`、`flipX`、`flipY`。当前图层的资源关联、内容、媒体、生成元数据、`width` / `height` / `originalWidth` / `originalHeight` 和标题必须保留，不能被异步回填前的旧快照覆盖。 - 当前生成对话框和非活动生成结果按稳定 ID 纳入内容存在性检查，避免恢复操作删除当前生成结果。相同 ID 的生成对话框只从目标快照恢复占位框 `x` / `y` 以及 active / inactive 槽位对应的 `composerOpen`，当前占位框的 `width` / `height` / `originalWidth` / `originalHeight`、当前比例与清晰度等参数、`generating` / `failed` / 完成态、提示词、参考图和任务结果继续以当前状态为准，不能被旧历史快照降级。 @@ -34,6 +35,7 @@ - 修改素材类型的撤销与恢复仍以当前图层内容为权威，但每次成功恢复类型后都要创建与恢复类型一致的正式项目 resource，再回填新 `resourceId`。同一图层的 resource 创建请求使用单调版本号，迟到的旧类型响应不得覆盖更晚的 undo / redo 结果。 - 鼠标拖动在按下时暂存操作前快照；屏幕位移达到点击阈值后才开始改变画布坐标并只提交一条历史，阈值内的指针抖动和单击都不产生位移或历史记录。 - 本地即时结果与后端项目快照结果都必须在生成图层加入画布前写入一条生成历史；生成完成后的自动适合视图不再额外压入视口历史，保证用户第一次撤销就命中生成保护。 +- 完美像素的关闭 composer 占位沿用 generation dialog 的内容存在性保护。占位删除已先持久化时，后端 completion 不得用旧 placeholder 复活占位或派生图层；回包时本地占位已删除则前端不应用完成快照或写 `perfect-pixel` 历史，已经持久化的 project resource / 账号素材仍可由资源或素材入口读取。现有布局 CAS 没有 deletion tombstone，completion 先提交、删除保存后冲突的极端竞态仍按权威快照收口。 - 顶部消息复用 `PlatformRuntimeStatusToast`，成功使用中性色，被阻止使用警告色；连续触发会替换消息并重新开始 3 秒计时。 ## 验收重点 @@ -55,3 +57,5 @@ 15. 图层移动后从素材库删除关联素材，随后撤销或恢复都不得把已删除图层重新加入画布；普通画布删除未伴随素材库删除时仍可撤销。 16. 打开“修改图片”并输入未提交提示词后，从按钮等非输入控件触发撤销不得关闭弹窗或回退当前草稿。 17. 修改素材类型后立即撤销或快速撤销再恢复，最终只允许最新类型的 resource 响应回填；刷新项目后类型与最后一次成功历史操作一致。 +18. 完美像素结果加入画布后只写一条 `perfect-pixel` 历史；第一次撤销命中保护提示，源图和派生 PNG 都不消失。 +19. 完美像素处理中删除占位后，本次完成回包不得在本地重新应用结果或写 `perfect-pixel` 历史；删除已先持久化时，后端不得复活占位或结果图层；已成功持久化的资源或账号素材允许保留。 diff --git a/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md b/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md index bb6795e99..9a4bafa93 100644 --- a/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md +++ b/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md @@ -61,6 +61,21 @@ worker 完成生成任务时，本次先用读取时 revision 调用 CAS 保存 重复 completion 必须返回同一资源、layer 和 dialog 终态，不得重复插入，也不能因 dialog 暂时缺失而返回 `changed=false` 后仍把任务标记完成。任一步失败时整笔业务写回回滚，任务保留可诊断的失败或可重试状态。 +### 3.6 免费同步栅格派生完成 + +`POST /api/editor/images/pixel-art-snaps` 的完美像素化不是 external job completion：它免费、在当前 HTTP 请求内 inline 执行，不创建任务行，也没有 `job_id / worker_id / lease_token`。前端仍须先创建关闭 composer 的右侧 generation dialog，再解析或上传源图以取得稳定引用，随后 flush 包含该占位的当前布局，最后把稳定源媒体引用和带非空 `dialogId` 的 `canvasCompletion` 一次提交；结构化 / legacy canvas 的完成分流继续由后端决定，前端不能直接写表或本地补造正式 layer。 + +像素读取、静态 PNG / JPEG / WebP 编码门禁、解码、输入限制、双轴网格置信检测、并发排队、规整和 PNG 编码全部发生在持久化前，并采用 strict 语义。GIF、APNG、动画 WebP、普通照片或未识别到既有像素网格的输入不得进入生成风格的统一网格兜底。任一步失败、超时或不适用时不执行最终 OSS PUT，不创建 asset object、`editor_project_resource`、`editor_asset` 或结果 layer；不得保存原图副本、逻辑低分辨率图、诊断图或前后对比图冒充结果。处理成功时只 PUT 一张最终 PNG，并至多各创建一个 project resource 和一个账号素材；源图已有正式 project resource 时，结果资源的 `source_resource_id` 指向该资源。 + +该零写入保证只覆盖首个最终 PNG PUT 前的可预判与处理阶段。进入持久化后，PNG / asset object、project resource、账号素材与 canvas completion 仍跨 OSS 和多个 SpacetimeDB procedure，沿用既有非事务顺序；后段失败可以保留此前已经确认的对象或记录，不做自动删除补偿，也不由客户端重放请求。调用方应按 `task_id / object_key / resource_id` 重新读取权威项目和素材快照后显式收口。 + +同步 completion 写画布前必须读取当前 revision 和 dialog，而不能信任请求中提交时的旧 placeholder： + +1. dialog 仍存在时，在同一当前布局上完成一个结果 layer，保留源图，并关联结果 resource； +2. dialog 的删除已先持久化时，跳过 layer / dialog 写回，不重建占位、不复活派生图层；已成功持久化的 project resource / 账号素材允许保留； +3. CAS 冲突时不得拿新 revision 原样重放旧整包；按当前项目保存冲突规则重新读取权威快照并显式收口； +4. 客户端回包时若本地 dialog 已删除，不应用完成快照或写历史；现有布局 CAS 没有 deletion tombstone，completion 先提交、删除保存后冲突的极端竞态仍按权威快照收口。客户端不得为该 unsafe POST 配置 `EDITOR_REQUEST_RETRY_OPTIONS`。请求字节可能已发出后的 transport 异常或 `408 / 425 / 429 / 502 / 503 / 504` 不自动重放，先通过 GET 核对项目 / 素材快照，再由用户显式决定是否再次执行；Bearer 中间件在 handler 前拒绝请求后的既有认证恢复继续保留。 + ## 4. 存量迁移 迁移按 canvas 执行 `backfill → hash 核对 → activate`，并保持幂等： @@ -102,5 +117,6 @@ SpacetimeDB 必须先于依赖新 procedure / bindings 的 API 发布；前端 - structured 模式下 typed 列而非扩展 JSON 决定几何、层级、分组、显示 / 锁定、资源引用和 dialog 状态；两个客户端基于同一 revision 写入时只允许一个成功，冲突方重载后端最新快照，不换上新 revision 原样重放旧整包。细粒度 batch mutation 是取消 2 MiB 兼容入口的后续项，不冒充为本次已完成。 - worker completion 当前以读取时 revision 做 CAS，冲突时拒绝覆盖；V2 保存和保存后快照在同一 procedure 结果内返回，避免“已提交但后续 GET 失败”的不确定结果。lease-fenced 资源 / layer / dialog / job 单事务 completion 仍是后续收口项。 - structured 快照刷新后，上传参考图、生成结果、占位与 dialog 状态均可恢复；资源存在但布局写入失败时不会伪装为保存成功。 +- 完美像素处理失败 / 超时时 OSS、resource、asset 和 layer 均无新增；成功时只有一个最终 PNG、至多一个 project resource 和一个账号素材。处理中占位删除已先持久化时，完成请求不复活 dialog 或结果 layer；回包时本地占位已删除则不应用完成快照，已成功创建的资源 / 素材仍可读取；传输结果未知时客户端不自动重放 unsafe POST。 - 回滚重组结果经 schema 校验、canonical hash / 资源引用核对且不超过 2 MiB；超限或不一致时明确拒绝且 structured 快照仍可读取。 - 完成 `npm run spacetime:generate`、`npm run check:spacetime-runtime-access`、`npm run check:spacetime-schema`、相关 Rust / API / 前端定向测试、`npm run check:encoding` 和 `git diff --check`。 diff --git a/scripts/check-module-runtime-artifact.mjs b/scripts/check-module-runtime-artifact.mjs index 00992cbab..b4def9203 100644 --- a/scripts/check-module-runtime-artifact.mjs +++ b/scripts/check-module-runtime-artifact.mjs @@ -152,8 +152,15 @@ function parseArchiveObjectMembers(artifact) { longNameTable = artifact.subarray(contentStart, contentEnd); } else if (/^\/\d+$/u.test(rawName) && longNameTable) { const nameOffset = Number.parseInt(rawName.slice(1), 10); - const nameEnd = longNameTable.indexOf(0x0a, nameOffset); - const resolvedEnd = nameEnd >= 0 ? nameEnd : longNameTable.length; + // GNU archives use newline-terminated names; MSVC COFF archives use NUL. + const candidateNameEnds = [ + longNameTable.indexOf(0x00, nameOffset), + longNameTable.indexOf(0x0a, nameOffset), + ].filter((nameEnd) => nameEnd >= 0); + const resolvedEnd = + candidateNameEnds.length > 0 + ? Math.min(...candidateNameEnds) + : longNameTable.length; memberName = longNameTable .subarray(nameOffset, resolvedEnd) .toString('utf8') diff --git a/server-rs/crates/api-server/src/app.rs b/server-rs/crates/api-server/src/app.rs index bb788cedc..a2d70713f 100644 --- a/server-rs/crates/api-server/src/app.rs +++ b/server-rs/crates/api-server/src/app.rs @@ -1963,6 +1963,147 @@ mod tests { } } + #[tokio::test] + async fn editor_pixel_art_snap_requires_bearer_auth() { + let app = build_router(AppState::new(AppConfig::default()).expect("state should build")); + let request_body = serde_json::json!({ + "sourceImageSrc": "editor-resource-source", + "projectId": "proj-source", + "canvasCompletion": { + "dialogId": "dialog-pixel-art", + "title": "完美像素", + "placeholder": { + "x": 0, + "y": 0, + "width": 128, + "height": 128, + "originalWidth": 128, + "originalHeight": 128 + } + } + }); + + let response = app + .oneshot( + Request::builder() + .method("POST") + .uri("/api/editor/images/pixel-art-snaps") + .header("content-type", "application/json") + .body(Body::from(request_body.to_string())) + .expect("request should build"), + ) + .await + .expect("request should succeed"); + + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); + } + + #[tokio::test] + async fn editor_pixel_art_snap_rejects_inline_source_before_processing() { + let state = AppState::new(AppConfig { + external_generation_mode: ExternalGenerationMode::Queue, + ..AppConfig::default() + }) + .expect("state should build"); + let seed_user = seed_phone_user_with_password(&state, "13800138230", TEST_PASSWORD).await; + let token = sign_test_user_token(&state, &seed_user, "sess_editor_pixel_snap_body"); + let app = build_router(state); + let request_body = serde_json::json!({ + "sourceImageSrc": "data:image/png;base64,AAAA", + "projectId": "proj-source", + "sourceResourceId": "editor-resource-source", + "assetKind": "character", + "generationInputs": { "fields": [], "references": [] }, + "canvasCompletion": { + "dialogId": "dialog-pixel-art", + "title": "完美像素", + "placeholder": { + "x": 0, + "y": 0, + "width": 128, + "height": 128, + "originalWidth": 128, + "originalHeight": 128 + } + } + }); + + let response = app + .oneshot( + Request::builder() + .method("POST") + .uri("/api/editor/images/pixel-art-snaps") + .header("authorization", format!("Bearer {token}")) + .header("content-type", "application/json") + .body(Body::from(request_body.to_string())) + .expect("request should build"), + ) + .await + .expect("request should succeed"); + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let body = response + .into_body() + .collect() + .await + .expect("response body should collect") + .to_bytes(); + let body_text = String::from_utf8_lossy(&body); + assert!( + body_text.contains("先上传 OSS"), + "handler should reject inline pixel-art sources: {body_text}" + ); + } + + #[tokio::test] + async fn editor_pixel_art_snap_requires_dialog_id_before_project_or_media_work() { + let state = AppState::new(AppConfig::default()).expect("state should build"); + let seed_user = seed_phone_user_with_password(&state, "13800138231", TEST_PASSWORD).await; + let token = sign_test_user_token(&state, &seed_user, "sess_editor_pixel_snap_dialog"); + let app = build_router(state); + let request_body = serde_json::json!({ + "sourceImageSrc": "editor-resource-source", + "projectId": "proj-does-not-exist", + "canvasCompletion": { + "title": "完美像素", + "placeholder": { + "x": 0, + "y": 0, + "width": 128, + "height": 128, + "originalWidth": 128, + "originalHeight": 128 + } + } + }); + + let response = app + .oneshot( + Request::builder() + .method("POST") + .uri("/api/editor/images/pixel-art-snaps") + .header("authorization", format!("Bearer {token}")) + .header("content-type", "application/json") + .body(Body::from(request_body.to_string())) + .expect("request should build"), + ) + .await + .expect("request should succeed"); + + assert_eq!(response.status(), StatusCode::BAD_REQUEST); + let body = response + .into_body() + .collect() + .await + .expect("response body should collect") + .to_bytes(); + let body_text = String::from_utf8_lossy(&body); + assert!( + body_text.contains("canvasCompletion.dialogId"), + "handler should reject missing dialog ids before project lookup: {body_text}" + ); + } + #[tokio::test] async fn editor_generation_json_validation_preserves_unsupported_media_type() { let state = AppState::new(AppConfig { diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index bf2d1194b..3ef06ab7f 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -134,6 +134,9 @@ pub(crate) const EDITOR_GENERATION_MULTIPLE_WARNINGS_CODE: &str = "multiple-gene const EDITOR_GENERATION_MAX_ASPECT_RATIO_DRIFT: f64 = 0.05; const EDITOR_PIXEL_ART_CPU_MAX_CONCURRENCY: usize = 2; const EDITOR_PIXEL_ART_MAX_PROCESSING_DURATION: Duration = Duration::from_secs(30); +const EDITOR_PIXEL_ART_SNAP_ASSET_KIND: &str = "editor_pixel_art_snap"; +const EDITOR_PIXEL_ART_SNAP_MODEL: &str = "Perfect Pixel"; +const EDITOR_PIXEL_ART_SNAP_PROVIDER: &str = "Genarrative"; static EDITOR_PIXEL_ART_CPU_LIMITER: LazyLock> = LazyLock::new(|| { Arc::new(tokio::sync::Semaphore::new( EDITOR_PIXEL_ART_CPU_MAX_CONCURRENCY, @@ -312,6 +315,19 @@ pub struct EditorBackgroundRemovalRequest { pub(crate) canvas_completion: Option, } +#[derive(Clone, Debug, Deserialize, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct EditorPixelArtSnapRequest { + pub(crate) source_image_src: String, + pub(crate) project_id: String, + pub(crate) source_resource_id: Option, + pub(crate) asset_kind: Option, + pub(crate) generation_inputs: Option, + pub(crate) asset_folder_id: Option, + pub(crate) asset_label: Option, + pub(crate) canvas_completion: EditorCanvasGenerationCompletionRequest, +} + #[derive(Clone, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorIconSpritesheetGenerationRequest { @@ -794,6 +810,23 @@ pub struct EditorBackgroundRemovalResponse { project: Option, } +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct EditorPixelArtSnapResponse { + image_src: String, + object_key: String, + asset_object_id: String, + width: u32, + height: u32, + source_type: &'static str, + task_id: String, + elapsed_ms: u64, + provider: &'static str, + resource: EditorProjectResourcePayload, + asset: EditorAssetPayload, + project: Option, +} + #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorIconSpritesheetIconResponse { @@ -3200,51 +3233,306 @@ async fn acquire_editor_pixel_art_cpu_permit( } } -async fn snap_editor_pixel_art_or_original( - rgba_source: DownloadedOpenAiImage, +fn editor_pixel_art_snap_failure(status: StatusCode, message: impl Into) -> AppError { + AppError::from_status(status).with_details(json!({ + "provider": "pixel-art-snapper", + "message": message.into(), + })) +} + +fn map_editor_pixel_art_snapper_error(error: platform_image::PixelArtSnapError) -> AppError { + let status = match &error { + platform_image::PixelArtSnapError::InvalidInput(_) + | platform_image::PixelArtSnapError::Decode { .. } => StatusCode::BAD_REQUEST, + platform_image::PixelArtSnapError::GridNotDetected => StatusCode::UNPROCESSABLE_ENTITY, + platform_image::PixelArtSnapError::DeadlineExceeded { .. } => StatusCode::GATEWAY_TIMEOUT, + platform_image::PixelArtSnapError::Encode(_) + | platform_image::PixelArtSnapError::Processing(_) => StatusCode::INTERNAL_SERVER_ERROR, + }; + editor_pixel_art_snap_failure(status, error.to_string()) +} + +fn validate_editor_pixel_art_static_asset_kind(asset_kind: Option<&str>) -> Result<(), AppError> { + let Some(asset_kind) = asset_kind.map(str::trim).filter(|value| !value.is_empty()) else { + return Ok(()); + }; + let normalized = asset_kind.to_ascii_lowercase(); + let is_non_static = normalized.contains("video") + || normalized.contains("audio") + || normalized.contains("animation") + || normalized.contains("image-sequence") + || matches!(normalized.as_str(), "sound-effect" | "background-music"); + if !is_non_static { + return Ok(()); + } + Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "assetKind", + "assetKind": asset_kind, + "message": "完美像素只支持静态图片素材。", + })), + ) +} + +fn resolve_editor_pixel_art_snap_asset_kind( + requested_asset_kind: Option<&str>, + source_asset_kind: Option>, + discovered_source_asset_kinds: &[String], + storage_asset_kinds: &[String], +) -> Result, AppError> { + let requested_asset_kind = requested_asset_kind + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned); + let preferred_source_asset_kind = source_asset_kind + .flatten() + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned); + let discovered_source_asset_kinds = discovered_source_asset_kinds + .iter() + .map(String::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned) + .collect::>(); + let storage_asset_kinds = storage_asset_kinds + .iter() + .map(String::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .collect::>(); + for authoritative_asset_kind in preferred_source_asset_kind + .iter() + .chain(discovered_source_asset_kinds.iter()) + { + // 中文注释：同一个 objectKey 可能同时留在项目资源与素材库。请求字段即使 + // 省略，也不能掩盖任一语义权威记录里的动画/音视频类型。 + validate_editor_pixel_art_static_asset_kind(Some(authoritative_asset_kind.as_str()))?; + } + for storage_asset_kind in storage_asset_kinds { + // 中文注释：asset_object.asset_kind 使用存储 taxonomy（例如 + // editor_generation_reference_image），只承担非静态媒体门禁，不能覆盖项目 + // 资源/素材库使用的语义 taxonomy，也不能拿来和请求 assetKind 做精确比较。 + validate_editor_pixel_art_static_asset_kind(Some(storage_asset_kind))?; + } + let authoritative_asset_kind = + preferred_source_asset_kind.or_else(|| discovered_source_asset_kinds.into_iter().next()); + if authoritative_asset_kind.is_some() + && requested_asset_kind.is_some() + && requested_asset_kind != authoritative_asset_kind + { + return Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "assetKind", + "requestedAssetKind": requested_asset_kind, + "sourceAssetKind": authoritative_asset_kind, + "message": "assetKind 与来源素材权威类型不一致。", + })), + ); + } + let asset_kind = authoritative_asset_kind.or(requested_asset_kind); + validate_editor_pixel_art_static_asset_kind(asset_kind.as_deref())?; + Ok(asset_kind) +} + +fn validate_editor_pixel_art_static_raster(image: &DownloadedOpenAiImage) -> Result<(), AppError> { + validate_editor_pixel_art_static_raster_bytes(image.bytes.as_slice()).map_err(|message| { + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "sourceImageSrc", + "message": message, + })) + }) +} + +fn validate_editor_pixel_art_static_raster_bytes(bytes: &[u8]) -> Result<(), &'static str> { + if bytes.starts_with(b"\x89PNG\r\n\x1a\n") { + return validate_editor_pixel_art_static_png(bytes); + } + if bytes.starts_with(&[0xff, 0xd8, 0xff]) { + return Ok(()); + } + if bytes.len() >= 12 && bytes.starts_with(b"RIFF") && &bytes[8..12] == b"WEBP" { + return validate_editor_pixel_art_static_webp(bytes); + } + if bytes.starts_with(b"GIF87a") || bytes.starts_with(b"GIF89a") { + return Err("完美像素不支持 GIF；请先转换为静态 PNG、JPEG 或 WebP。"); + } + Err("完美像素只支持静态 PNG、JPEG 或 WebP 图片。") +} + +fn validate_editor_pixel_art_static_png(bytes: &[u8]) -> Result<(), &'static str> { + let mut offset = 8usize; + let mut saw_iend = false; + while offset < bytes.len() { + let header_end = offset + .checked_add(8) + .filter(|end| *end <= bytes.len()) + .ok_or("PNG 文件结构不完整。")?; + let length = u32::from_be_bytes( + bytes[offset..offset + 4] + .try_into() + .map_err(|_| "PNG 文件结构不完整。")?, + ) as usize; + let chunk_type = &bytes[offset + 4..header_end]; + let chunk_end = header_end + .checked_add(length) + .and_then(|end| end.checked_add(4)) + .filter(|end| *end <= bytes.len()) + .ok_or("PNG 文件结构不完整。")?; + if chunk_type == b"acTL" { + return Err("完美像素不支持 APNG 动图；请先转换为静态 PNG。"); + } + offset = chunk_end; + if chunk_type == b"IEND" { + saw_iend = true; + break; + } + } + if !saw_iend { + return Err("PNG 文件结构不完整。"); + } + Ok(()) +} + +fn validate_editor_pixel_art_static_webp(bytes: &[u8]) -> Result<(), &'static str> { + let riff_size = u32::from_le_bytes( + bytes[4..8] + .try_into() + .map_err(|_| "WebP 文件结构不完整。")?, + ) as usize; + let riff_end = riff_size + .checked_add(8) + .filter(|end| *end <= bytes.len()) + .ok_or("WebP 文件结构不完整。")?; + let mut offset = 12usize; + let mut saw_image_chunk = false; + while offset < riff_end { + let header_end = offset + .checked_add(8) + .filter(|end| *end <= riff_end) + .ok_or("WebP 文件结构不完整。")?; + let chunk_type = &bytes[offset..offset + 4]; + let length = u32::from_le_bytes( + bytes[offset + 4..header_end] + .try_into() + .map_err(|_| "WebP 文件结构不完整。")?, + ) as usize; + let data_end = header_end + .checked_add(length) + .filter(|end| *end <= riff_end) + .ok_or("WebP 文件结构不完整。")?; + if chunk_type == b"ANIM" || chunk_type == b"ANMF" { + return Err("完美像素不支持动画 WebP；请先转换为静态 WebP。"); + } + if chunk_type == b"VP8X" + && bytes + .get(header_end) + .is_some_and(|feature_flags| feature_flags & 0x02 != 0) + { + return Err("完美像素不支持动画 WebP；请先转换为静态 WebP。"); + } + if chunk_type == b"VP8 " || chunk_type == b"VP8L" { + saw_image_chunk = true; + } + offset = data_end + .checked_add(length % 2) + .filter(|end| *end <= riff_end) + .ok_or("WebP 文件结构不完整。")?; + } + if !saw_image_chunk { + return Err("WebP 文件结构不完整。"); + } + Ok(()) +} + +async fn snap_editor_pixel_art_strict( + rgba_source: Arc, request_deadline: Option, -) -> (DownloadedOpenAiImage, Option) { +) -> Result { + snap_editor_pixel_art_with_grid_policy(rgba_source, request_deadline, true).await +} + +async fn snap_editor_pixel_art_with_grid_policy( + rgba_source: Arc, + request_deadline: Option, + require_detected_grid: bool, +) -> Result { let processing_deadline = resolve_editor_pixel_art_processing_deadline(Instant::now(), request_deadline); if Instant::now() >= processing_deadline { - return ( - rgba_source, - Some("像素规整处理预算已耗尽，已保留原始生成结果。".to_string()), - ); + return Err(editor_pixel_art_snap_failure( + StatusCode::GATEWAY_TIMEOUT, + "像素规整处理预算已耗尽。", + )); } - let permit = match acquire_editor_pixel_art_cpu_permit(processing_deadline).await { - Ok(permit) => permit, - Err(error) => return (rgba_source, Some(error)), - }; + let permit = acquire_editor_pixel_art_cpu_permit(processing_deadline) + .await + .map_err(|message| { + let status = if message.contains("预算已耗尽") { + StatusCode::GATEWAY_TIMEOUT + } else { + StatusCode::SERVICE_UNAVAILABLE + }; + editor_pixel_art_snap_failure(status, message.replace("，已保留原始生成结果。", "。")) + })?; - let rgba_source = Arc::new(rgba_source); let worker_rgba_source = Arc::clone(&rgba_source); let worker = tokio::task::spawn_blocking(move || { // 中文注释：permit 必须由 blocking 闭包持有，而不是只保护 spawn； // 即使上层 future 因 worker deadline 被取消，仍会限制尚未结束的 CPU 任务数量。 let _permit = permit; - platform_image::snap_pixel_art_with_deadline( - worker_rgba_source.as_ref(), - worker_rgba_source.as_ref(), - Some(processing_deadline), - ) + if require_detected_grid { + platform_image::snap_pixel_art_strict_with_deadline( + worker_rgba_source.as_ref(), + worker_rgba_source.as_ref(), + Some(processing_deadline), + ) + } else { + platform_image::snap_pixel_art_with_deadline( + worker_rgba_source.as_ref(), + worker_rgba_source.as_ref(), + Some(processing_deadline), + ) + } }); let result = tokio::time::timeout_at(tokio::time::Instant::from_std(processing_deadline), worker).await; match result { - Ok(Ok(Ok(image))) => (image, None), - Ok(Ok(Err(error))) => ( - take_arc_downloaded_image(rgba_source), - Some(error.to_string()), - ), - Ok(Err(error)) => ( - take_arc_downloaded_image(rgba_source), - Some(format!("像素规整工作线程异常：{error}")), - ), - Err(_) => ( - take_arc_downloaded_image(rgba_source), - Some("像素规整处理超时，已保留原始生成结果。".to_string()), - ), + Ok(Ok(Ok(image))) => Ok(image), + Ok(Ok(Err(error))) => Err(map_editor_pixel_art_snapper_error(error)), + Ok(Err(error)) => Err(editor_pixel_art_snap_failure( + StatusCode::INTERNAL_SERVER_ERROR, + format!("像素规整工作线程异常：{error}"), + )), + Err(_) => Err(editor_pixel_art_snap_failure( + StatusCode::GATEWAY_TIMEOUT, + "像素规整处理超时。", + )), + } +} + +async fn snap_editor_pixel_art_or_original( + rgba_source: DownloadedOpenAiImage, + request_deadline: Option, +) -> (DownloadedOpenAiImage, Option) { + let rgba_source = Arc::new(rgba_source); + match snap_editor_pixel_art_with_grid_policy(Arc::clone(&rgba_source), request_deadline, false) + .await + { + Ok(image) => (image, None), + Err(error) => { + let reason = error.body_text(); + let reason = + reason.trim_end_matches(|character| matches!(character, '。' | '！' | '!')); + ( + take_arc_downloaded_image(rgba_source), + Some(format!("{reason}，已保留原始生成结果。")), + ) + } } } @@ -4014,6 +4302,389 @@ struct EditorBackgroundRemovalImage { height: u32, } +fn validate_editor_pixel_art_snap_canvas_completion( + completion: &EditorCanvasGenerationCompletionRequest, +) -> Result<(), AppError> { + if normalized_canvas_completion_dialog_id(completion).is_some() { + return Ok(()); + } + Err( + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "canvasCompletion.dialogId", + "message": "完美像素必须关联有效的画布生成占位。", + })), + ) +} + +fn validate_editor_pixel_art_snap_placeholder_exists( + layers: &Value, + completion: &EditorCanvasGenerationCompletionRequest, +) -> Result<(), AppError> { + let dialog_id = normalized_canvas_completion_dialog_id(completion).ok_or_else(|| { + editor_pixel_art_snap_failure( + StatusCode::BAD_REQUEST, + "完美像素必须关联有效的画布生成占位。", + ) + })?; + if layers.as_array().is_some_and(|items| { + items + .iter() + .any(|item| generation_dialog_item_matches(item, dialog_id.as_str())) + }) { + return Ok(()); + } + Err( + AppError::from_status(StatusCode::CONFLICT).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "canvasCompletion.dialogId", + "dialogId": dialog_id, + "message": "完美像素画布占位不存在或尚未保存，请重试。", + })), + ) +} + +#[derive(Debug)] +struct EditorPixelArtSourceResolution { + object_key: String, + asset_kind: Option, +} + +fn push_editor_pixel_art_source_asset_kind( + asset_kinds: &mut Vec, + asset_kind: Option<&str>, +) { + let Some(asset_kind) = asset_kind + .map(str::trim) + .filter(|asset_kind| !asset_kind.is_empty()) + else { + return; + }; + if !asset_kinds.iter().any(|candidate| candidate == asset_kind) { + asset_kinds.push(asset_kind.to_string()); + } +} + +async fn resolve_editor_pixel_art_source_for_owner( + state: &AppState, + owner_user_id: &str, + source: &str, + project: &EditorProjectPayload, + source_resource: Option<&EditorProjectResourcePayload>, + requested_asset_kind: Option<&str>, +) -> Result { + let object_key = + resolve_editor_reference_object_key_for_owner(state, owner_user_id, source).await?; + if let Some(source_resource) = source_resource { + let source_resource_object_key = normalize_editor_record_object_key( + source_resource.object_key.as_deref(), + source_resource.image_src.as_str(), + ) + .ok_or_else(|| { + editor_pixel_art_snap_failure( + StatusCode::BAD_REQUEST, + "来源项目资源没有可用的稳定图片引用。", + ) + })?; + if source_resource_object_key != object_key { + return Err(editor_pixel_art_snap_failure( + StatusCode::BAD_REQUEST, + "sourceImageSrc 与 sourceResourceId 指向不同图片。", + )); + } + } + + let mut discovered_asset_kinds = Vec::new(); + let mut storage_asset_kinds = Vec::new(); + for resource in project.resources.iter().filter(|resource| { + editor_record_object_key_matches( + resource.object_key.as_deref(), + resource.image_src.as_str(), + object_key.as_str(), + ) + }) { + push_editor_pixel_art_source_asset_kind( + &mut discovered_asset_kinds, + resource.asset_kind.as_deref(), + ); + } + let projects = state + .spacetime_client() + .list_editor_projects(owner_user_id.to_string()) + .await + .map_err(map_editor_project_error)?; + for resource in projects + .iter() + .flat_map(|project| project.resources.iter()) + .filter(|resource| { + editor_record_object_key_matches( + resource.object_key.as_deref(), + resource.image_src.as_str(), + object_key.as_str(), + ) + }) + { + push_editor_pixel_art_source_asset_kind( + &mut discovered_asset_kinds, + resource.asset_kind.as_deref(), + ); + } + let library = state + .spacetime_client() + .get_editor_asset_library(owner_user_id.to_string(), current_utc_micros()) + .await + .map_err(map_editor_project_error)?; + for asset in library.assets.iter().filter(|asset| { + editor_record_object_key_matches( + asset.object_key.as_deref(), + asset.image_src.as_str(), + object_key.as_str(), + ) + }) { + push_editor_pixel_art_source_asset_kind( + &mut discovered_asset_kinds, + asset.asset_kind.as_deref(), + ); + } + if let Some(oss_client) = state.oss_client() + && let Some(asset_object) = state + .spacetime_client() + .get_asset_object_by_location(module_assets::AssetObjectLocationInput { + bucket: oss_client.config_bucket().to_string(), + object_key: object_key.clone(), + }) + .await + .map_err(map_editor_project_error)? + { + validate_editor_reference_asset_object( + &asset_object, + owner_user_id, + state, + object_key.as_str(), + )?; + push_editor_pixel_art_source_asset_kind( + &mut storage_asset_kinds, + Some(asset_object.asset_kind.as_str()), + ); + } + let asset_kind = resolve_editor_pixel_art_snap_asset_kind( + requested_asset_kind, + source_resource.map(|resource| resource.asset_kind.as_deref()), + discovered_asset_kinds.as_slice(), + storage_asset_kinds.as_slice(), + )?; + Ok(EditorPixelArtSourceResolution { + object_key, + asset_kind, + }) +} + +fn resolve_editor_pixel_art_asset_folder_id(asset_folder_id: Option) -> Option { + normalize_optional_string(asset_folder_id) + .or_else(|| Some(EDITOR_ASSET_DEFAULT_FOLDER_ID.to_string())) +} + +pub async fn snap_editor_image_to_pixel_art( + State(state): State, + Extension(request_context): Extension, + Extension(authenticated): Extension, + payload: Result, JsonRejection>, +) -> Result, AppError> { + let Json(payload) = parse_editor_generation_json_payload(payload)?; + let started_at = Instant::now(); + ensure_editor_reference_image_source_is_stable( + payload.source_image_src.as_str(), + "pixel-art-snapper", + "sourceImageSrc", + "待像素规整图片", + )?; + let project_id = + normalize_optional_string(Some(payload.project_id.clone())).ok_or_else(|| { + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "pixel-art-snapper", + "field": "projectId", + "message": "完美像素必须关联有效画布项目。", + })) + })?; + validate_editor_pixel_art_snap_canvas_completion(&payload.canvas_completion)?; + // 中文注释：在 CPU 处理和 OSS PUT 前完成所有无副作用校验；主动操作的像素规整 + // 失败必须直接返回错误，不能先创建与原图相同的派生资源。 + serialize_editor_asset_metadata(payload.generation_inputs.clone())?; + let owner_user_id = current_owner_user_id(&authenticated); + let project = state + .spacetime_client() + .get_editor_project(EditorProjectGetRecordInput { + project_id: project_id.clone(), + owner_user_id: owner_user_id.clone(), + }) + .await + .map_err(map_editor_project_error)?; + let project = editor_project_payload_from_record(project); + validate_editor_pixel_art_snap_placeholder_exists(&project.layers, &payload.canvas_completion)?; + let source_resource_id = normalize_optional_string(payload.source_resource_id.clone()); + let source_resource = if let Some(source_resource_id) = source_resource_id.as_deref() { + Some( + project + .resources + .iter() + .find(|resource| resource.resource_id.trim() == source_resource_id) + .ok_or_else(|| { + editor_pixel_art_snap_failure( + StatusCode::NOT_FOUND, + "来源项目资源不存在或不属于当前画布项目。", + ) + })?, + ) + } else { + None + }; + let source = resolve_editor_pixel_art_source_for_owner( + &state, + owner_user_id.as_str(), + payload.source_image_src.as_str(), + &project, + source_resource, + payload.asset_kind.as_deref(), + ) + .await?; + let source_object_key = source.object_key; + let asset_kind = source.asset_kind; + let source_image = + download_editor_persisted_image_object(&state, source_object_key.as_str()).await?; + validate_editor_pixel_art_static_raster(&source_image)?; + let snapped_image = snap_editor_pixel_art_strict( + Arc::new(source_image), + request_context.external_call_deadline(), + ) + .await?; + let (width, height) = image::load_from_memory(snapped_image.bytes.as_slice()) + .map(|image| (image.width(), image.height())) + .map_err(|error| { + editor_pixel_art_snap_failure( + StatusCode::INTERNAL_SERVER_ERROR, + format!("完美像素输出不是有效 PNG：{error}"), + ) + })?; + + let task_id = build_prefixed_uuid_id("pixel-art-snap-"); + let persisted = persist_editor_generated_image( + &state, + owner_user_id.as_str(), + task_id.as_str(), + &snapped_image, + "完美像素", + None, + EDITOR_PIXEL_ART_SNAP_ASSET_KIND, + "pixel-art-snaps", + "perfect-pixel", + "result", + "genarrative", + ) + .await?; + let image_src = editor_media_src_from_object_key(persisted.object_key.as_str()); + let generated_asset = persist_editor_generated_asset( + &state, + PersistEditorGeneratedAssetInput { + project_id: Some(project_id.clone()), + owner_user_id: owner_user_id.clone(), + folder_id: resolve_editor_pixel_art_asset_folder_id(payload.asset_folder_id), + label: resolve_editor_generated_asset_label(payload.asset_label, "完美像素"), + image_src: image_src.clone(), + object_key: Some(persisted.object_key.clone()), + asset_object_id: Some(persisted.asset_object_id.clone()), + width, + height, + prompt: "完美像素".to_string(), + actual_prompt: None, + model: EDITOR_PIXEL_ART_SNAP_MODEL.to_string(), + provider: EDITOR_PIXEL_ART_SNAP_PROVIDER.to_string(), + task_id: task_id.clone(), + group_task_id: None, + group_task_expected_asset_count: None, + source_resource_id, + asset_kind, + generation_inputs: payload.generation_inputs, + thumbnail_src: None, + generation_cost_mud_points: 0, + }, + ) + .await + .map_err(|error| { + tracing::warn!( + provider = EDITOR_PIXEL_ART_SNAP_PROVIDER, + task_id = %task_id, + object_key = %persisted.object_key, + error = %error, + "editor_pixel_art_snap_asset_persistence_failed_after_object_put" + ); + error + })?; + let resource = generated_asset.resource.ok_or_else(|| { + tracing::warn!( + provider = EDITOR_PIXEL_ART_SNAP_PROVIDER, + task_id = %task_id, + object_key = %persisted.object_key, + "editor_pixel_art_snap_project_resource_missing_after_object_put" + ); + editor_pixel_art_snap_failure( + StatusCode::INTERNAL_SERVER_ERROR, + "完美像素结果未创建项目资源。", + ) + })?; + let asset = generated_asset.asset.ok_or_else(|| { + tracing::warn!( + provider = EDITOR_PIXEL_ART_SNAP_PROVIDER, + task_id = %task_id, + object_key = %persisted.object_key, + resource_id = %resource.resource_id, + "editor_pixel_art_snap_account_asset_missing_after_object_put" + ); + editor_pixel_art_snap_failure( + StatusCode::INTERNAL_SERVER_ERROR, + "完美像素结果未创建账号素材。", + ) + })?; + // 中文注释：占位在处理期间被用户删除时，沿用现有 completion 规则，只保留已经 + // 登记的项目资源和账号素材，不复活图层，也不把该并发行为误报为像素处理失败。 + let completed_project = complete_editor_canvas_generation( + &state, + owner_user_id.as_str(), + Some(project_id.as_str()), + Some(&payload.canvas_completion), + Some(&resource), + ) + .await + .map_err(|error| { + tracing::warn!( + provider = EDITOR_PIXEL_ART_SNAP_PROVIDER, + task_id = %task_id, + object_key = %persisted.object_key, + resource_id = %resource.resource_id, + error = %error, + "editor_pixel_art_snap_canvas_completion_failed_after_asset_persistence" + ); + error + })?; + + Ok(json_success_body( + Some(&request_context), + EditorPixelArtSnapResponse { + image_src, + object_key: persisted.object_key, + asset_object_id: persisted.asset_object_id, + width, + height, + source_type: "generated", + task_id, + elapsed_ms: u64::try_from(started_at.elapsed().as_millis()).unwrap_or(u64::MAX), + provider: EDITOR_PIXEL_ART_SNAP_PROVIDER, + resource, + asset, + project: completed_project, + }, + )) +} + async fn validate_editor_background_removal_source( state: &AppState, source_object_key: &str, @@ -9607,10 +10278,69 @@ mod tests { assert!( error .expect("expired budget should warn") - .contains("预算已耗尽") + .contains("预算已耗尽，已保留原始生成结果") ); } + #[tokio::test] + async fn pixel_art_strict_expired_budget_returns_error_without_fallback_wording() { + let original = Arc::new(DownloadedOpenAiImage { + bytes: vec![1, 2, 3, 4], + mime_type: "image/png".to_string(), + extension: "png".to_string(), + }); + let expired = Instant::now() + .checked_sub(Duration::from_millis(1)) + .expect("expired test deadline should be representable"); + + let error = snap_editor_pixel_art_strict(original, Some(expired)) + .await + .expect_err("strict pixel snap should surface exhausted budgets"); + + assert_eq!(error.status_code(), StatusCode::GATEWAY_TIMEOUT); + assert!(error.body_text().contains("预算已耗尽")); + assert!(!error.body_text().contains("已保留原始生成结果")); + } + + #[tokio::test] + async fn pixel_art_strict_rejects_undetected_grids_but_best_effort_keeps_legacy_fallback() { + let image = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 128, + 128, + image::Rgba([20, 30, 40, 255]), + )); + let mut bytes = Cursor::new(Vec::new()); + image + .write_to(&mut bytes, image::ImageFormat::Png) + .expect("test PNG should encode"); + let source = DownloadedOpenAiImage { + bytes: bytes.into_inner(), + mime_type: "image/png".to_string(), + extension: "png".to_string(), + }; + + let strict_error = snap_editor_pixel_art_strict(Arc::new(source.clone()), None) + .await + .expect_err("explicit action should reject an undetected pixel grid"); + let (legacy_output, legacy_error) = + snap_editor_pixel_art_or_original(source.clone(), None).await; + + assert_eq!(strict_error.status_code(), StatusCode::UNPROCESSABLE_ENTITY); + assert!(strict_error.body_text().contains("未识别到")); + assert!(legacy_error.is_none()); + let legacy_output = image::load_from_memory(legacy_output.bytes.as_slice()) + .expect("legacy output should remain a valid image"); + assert_eq!((legacy_output.width(), legacy_output.height()), (128, 128)); + } + + #[test] + fn pixel_art_best_effort_warning_contract_stays_stable() { + let warning = editor_pixel_art_fallback_warning(); + + assert_eq!(warning.code, EDITOR_GENERATION_POSTPROCESS_WARNING_CODE); + assert_eq!(warning.reason, "像素规整未完成，已保留原始生成结果。"); + } + #[test] fn pixel_art_processing_deadline_uses_earlier_local_or_request_budget() { assert_eq!(EDITOR_PIXEL_ART_CPU_MAX_CONCURRENCY, 2); @@ -9633,6 +10363,307 @@ mod tests { ); } + #[test] + fn explicit_pixel_art_snap_contract_preserves_source_metadata() { + let request: EditorPixelArtSnapRequest = serde_json::from_value(json!({ + "sourceImageSrc": "editor-resource-source", + "projectId": "proj-source", + "sourceResourceId": "editor-resource-source", + "assetKind": "character", + "generationInputs": { + "fields": [{ "title": "角色", "value": "陶罐精灵" }], + "references": [], + }, + "assetFolderId": "project", + "assetLabel": "像素角色", + "canvasCompletion": { + "dialogId": "dialog-pixel-art", + "title": "完美像素", + "placeholder": { + "x": 10.0, + "y": 20.0, + "width": 128.0, + "height": 128.0, + "originalWidth": 128.0, + "originalHeight": 128.0 + } + } + })) + .expect("pixel-art snap request should deserialize"); + + assert_eq!(request.project_id, "proj-source"); + assert_eq!( + request.source_resource_id.as_deref(), + Some("editor-resource-source") + ); + assert_eq!(request.asset_kind.as_deref(), Some("character")); + assert_eq!( + request + .generation_inputs + .as_ref() + .and_then(|value| value["fields"][0]["value"].as_str()), + Some("陶罐精灵") + ); + assert_eq!( + request.canvas_completion.dialog_id.as_deref(), + Some("dialog-pixel-art") + ); + } + + #[test] + fn explicit_pixel_art_snap_rejects_non_static_or_forged_asset_kinds() { + assert_eq!( + resolve_editor_pixel_art_snap_asset_kind( + Some("character"), + Some(Some("character")), + &[], + &[], + ) + .expect("matching source kind should be preserved") + .as_deref(), + Some("character") + ); + assert_eq!( + resolve_editor_pixel_art_snap_asset_kind(Some("character"), Some(None), &[], &[]) + .expect("legacy source without a kind may adopt a validated static kind") + .as_deref(), + Some("character") + ); + let mismatch = resolve_editor_pixel_art_snap_asset_kind( + Some("image"), + Some(Some("character")), + &[], + &[], + ) + .expect_err("request kind must not override authoritative source metadata"); + assert_eq!(mismatch.status_code(), StatusCode::BAD_REQUEST); + + for asset_kind in [ + "video", + "editor_uploaded_video", + "audio", + "sound-effect", + "background-music", + "image-sequence", + "character-animation", + ] { + let error = resolve_editor_pixel_art_snap_asset_kind(Some(asset_kind), None, &[], &[]) + .expect_err("non-static source kinds must be rejected"); + assert_eq!(error.status_code(), StatusCode::BAD_REQUEST); + } + let discovered_animation = vec!["character-animation".to_string()]; + let omitted_kind_error = + resolve_editor_pixel_art_snap_asset_kind(None, None, &discovered_animation, &[]) + .expect_err("omitting both request metadata fields must not hide source metadata"); + assert_eq!(omitted_kind_error.status_code(), StatusCode::BAD_REQUEST); + + let local_reference_storage_kind = vec!["editor_generation_reference_image".to_string()]; + assert_eq!( + resolve_editor_pixel_art_snap_asset_kind( + Some("character"), + None, + &[], + &local_reference_storage_kind, + ) + .expect("storage taxonomy must not conflict with requested semantic kind") + .as_deref(), + Some("character") + ); + let animated_storage_kind = vec!["character-animation".to_string()]; + let omitted_storage_kind_error = + resolve_editor_pixel_art_snap_asset_kind(None, None, &[], &animated_storage_kind) + .expect_err("omitted request metadata must not bypass asset_object media guard"); + assert_eq!( + omitted_storage_kind_error.status_code(), + StatusCode::BAD_REQUEST + ); + + let image_kind = resolve_editor_pixel_art_snap_asset_kind(Some("image"), None, &[], &[]) + .expect("static image kind should be accepted"); + assert_eq!(generated_canvas_media_type(image_kind.as_deref()), "image"); + } + + #[test] + fn explicit_pixel_art_snap_blank_asset_folder_falls_back_to_project() { + for asset_folder_id in [None, Some(" ".to_string())] { + assert_eq!( + resolve_editor_pixel_art_asset_folder_id(asset_folder_id).as_deref(), + Some(EDITOR_ASSET_DEFAULT_FOLDER_ID) + ); + } + assert_eq!( + resolve_editor_pixel_art_asset_folder_id(Some( + " user-1:asset-folder:custom ".to_string() + )) + .as_deref(), + Some("user-1:asset-folder:custom") + ); + } + + #[test] + fn explicit_pixel_art_snap_accepts_only_static_png_jpeg_or_webp_bytes() { + let image = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 4, + 4, + image::Rgba([20, 30, 40, 255]), + )); + let mut png = Cursor::new(Vec::new()); + image + .write_to(&mut png, image::ImageFormat::Png) + .expect("test PNG should encode"); + assert!(validate_editor_pixel_art_static_raster_bytes(png.get_ref()).is_ok()); + assert!(validate_editor_pixel_art_static_raster_bytes(&[0xff, 0xd8, 0xff, 0xd9]).is_ok()); + let mut webp = Cursor::new(Vec::new()); + image + .write_to(&mut webp, image::ImageFormat::WebP) + .expect("test WebP should encode"); + assert!(validate_editor_pixel_art_static_raster_bytes(webp.get_ref()).is_ok()); + + let mut apng = b"\x89PNG\r\n\x1a\n".to_vec(); + apng.extend_from_slice(&8u32.to_be_bytes()); + apng.extend_from_slice(b"acTL"); + apng.extend_from_slice(&[0; 8]); + apng.extend_from_slice(&[0; 4]); + assert!( + validate_editor_pixel_art_static_raster_bytes(apng.as_slice()) + .expect_err("APNG should be rejected") + .contains("APNG") + ); + + let mut animated_webp = b"RIFF".to_vec(); + animated_webp.extend_from_slice(&12u32.to_le_bytes()); + animated_webp.extend_from_slice(b"WEBP"); + animated_webp.extend_from_slice(b"ANIM"); + animated_webp.extend_from_slice(&0u32.to_le_bytes()); + assert!( + validate_editor_pixel_art_static_raster_bytes(animated_webp.as_slice()) + .expect_err("animated WebP should be rejected") + .contains("动画 WebP") + ); + let mut animated_vp8x_webp = b"RIFF".to_vec(); + animated_vp8x_webp.extend_from_slice(&30u32.to_le_bytes()); + animated_vp8x_webp.extend_from_slice(b"WEBP"); + animated_vp8x_webp.extend_from_slice(b"VP8X"); + animated_vp8x_webp.extend_from_slice(&10u32.to_le_bytes()); + animated_vp8x_webp.extend_from_slice(&[0x02, 0, 0, 0, 0, 0, 0, 0, 0, 0]); + animated_vp8x_webp.extend_from_slice(b"VP8L"); + animated_vp8x_webp.extend_from_slice(&0u32.to_le_bytes()); + assert!( + validate_editor_pixel_art_static_raster_bytes(animated_vp8x_webp.as_slice()) + .expect_err("VP8X animation flag should be rejected") + .contains("动画 WebP") + ); + assert!( + validate_editor_pixel_art_static_raster_bytes(b"GIF89a") + .expect_err("GIF should be rejected") + .contains("GIF") + ); + assert!( + validate_editor_pixel_art_static_raster_bytes(b"not-an-image") + .expect_err("unknown image bytes should be rejected") + .contains("PNG、JPEG 或 WebP") + ); + } + + #[test] + fn explicit_pixel_art_snap_requires_non_empty_canvas_dialog_id() { + for dialog_id in [None, Some(" ")] { + let completion: EditorCanvasGenerationCompletionRequest = + serde_json::from_value(json!({ + "dialogId": dialog_id, + "title": "完美像素", + "placeholder": { + "x": 10.0, + "y": 20.0, + "width": 128.0, + "height": 128.0, + "originalWidth": 128.0, + "originalHeight": 128.0 + } + })) + .expect("completion request should deserialize"); + + let error = validate_editor_pixel_art_snap_canvas_completion(&completion) + .expect_err("pixel-art snap must require a stable dialog id"); + assert_eq!(error.status_code(), StatusCode::BAD_REQUEST); + assert_eq!( + error.details().and_then(|details| details.get("field")), + Some(&json!("canvasCompletion.dialogId")) + ); + } + } + + #[test] + fn explicit_pixel_art_snap_requires_a_persisted_placeholder_before_processing() { + let completion: EditorCanvasGenerationCompletionRequest = serde_json::from_value(json!({ + "dialogId": "dialog-pixel-art", + "title": "完美像素", + "placeholder": { + "x": 10.0, + "y": 20.0, + "width": 128.0, + "height": 128.0, + "originalWidth": 128.0, + "originalHeight": 128.0 + } + })) + .expect("completion request should deserialize"); + let missing = validate_editor_pixel_art_snap_placeholder_exists(&json!([]), &completion) + .expect_err("missing server placeholder should fail before processing"); + assert_eq!(missing.status_code(), StatusCode::CONFLICT); + + let layers = json!([{ + "itemType": "generation-dialog", + "dialog": { + "id": "dialog-pixel-art", + "placeholder": completion.placeholder.clone() + } + }]); + assert!(validate_editor_pixel_art_snap_placeholder_exists(&layers, &completion).is_ok()); + } + + #[test] + fn explicit_pixel_art_snap_is_inline_strict_and_persists_only_after_processing() { + let source = include_str!("editor_project.rs"); + assert_function_contains_in_order( + source, + "pub async fn snap_editor_image_to_pixel_art(", + "async fn validate_editor_background_removal_source", + &[ + "ensure_editor_reference_image_source_is_stable", + "validate_editor_pixel_art_snap_canvas_completion", + "serialize_editor_asset_metadata", + ".get_editor_project", + "validate_editor_pixel_art_snap_placeholder_exists", + "resolve_editor_pixel_art_source_for_owner", + "download_editor_persisted_image_object", + "validate_editor_pixel_art_static_raster", + "snap_editor_pixel_art_strict", + "persist_editor_generated_image", + "persist_editor_generated_asset", + "complete_editor_canvas_generation", + ], + ); + assert_function_occurrence_count( + source, + "pub async fn snap_editor_image_to_pixel_art(", + "async fn validate_editor_background_removal_source", + "persist_editor_generated_image(", + 1, + ); + assert_function_not_contains( + source, + "pub async fn snap_editor_image_to_pixel_art(", + "async fn validate_editor_background_removal_source", + &[ + "enqueue_editor_generation_job", + "external_generation_job", + "queue_state", + "snap_editor_pixel_art_or_original", + ], + ); + } + #[test] fn editor_image_generation_returns_never_drop_accumulated_warnings() { let source = include_str!("editor_project.rs"); diff --git a/server-rs/crates/api-server/src/modules/editor_project.rs b/server-rs/crates/api-server/src/modules/editor_project.rs index 90747e184..43bb64cd0 100644 --- a/server-rs/crates/api-server/src/modules/editor_project.rs +++ b/server-rs/crates/api-server/src/modules/editor_project.rs @@ -22,9 +22,9 @@ use crate::{ get_editor_asset_library, get_editor_generation_pricing, get_editor_project, list_editor_projects, list_public_editor_project_resources, load_recent_editor_project, remove_editor_image_background, rename_editor_project, save_editor_project_layout, - split_editor_icon_spritesheet, submit_editor_asset_showcase, - toggle_editor_showcase_asset_like, update_editor_asset, update_editor_asset_folder, - update_editor_project_resource_showcase, + snap_editor_image_to_pixel_art, split_editor_icon_spritesheet, + submit_editor_asset_showcase, toggle_editor_showcase_asset_like, update_editor_asset, + update_editor_asset_folder, update_editor_project_resource_showcase, }, state::AppState, }; @@ -205,6 +205,13 @@ pub fn router(state: AppState) -> Router { require_bearer_auth, )), ) + .route( + "/api/editor/images/pixel-art-snaps", + post(snap_editor_image_to_pixel_art).route_layer(middleware::from_fn_with_state( + state.clone(), + require_bearer_auth, + )), + ) .route( "/api/editor/icon-spritesheets/generations", post(generate_editor_icon_spritesheet).route_layer(middleware::from_fn_with_state( diff --git a/server-rs/crates/platform-image/src/lib.rs b/server-rs/crates/platform-image/src/lib.rs index 41289d985..b2bf21a2e 100644 --- a/server-rs/crates/platform-image/src/lib.rs +++ b/server-rs/crates/platform-image/src/lib.rs @@ -5,7 +5,8 @@ pub mod vector_engine; pub use pixel_art_snapper::{ PIXEL_ART_ALPHA_COVERAGE_THRESHOLD, PIXEL_ART_ANALYSIS_COLORS, PIXEL_ART_KMEANS_SAMPLE_LIMIT, - PIXEL_ART_MAX_IMAGE_PIXELS, PixelArtSnapError, snap_pixel_art, snap_pixel_art_with_deadline, + PIXEL_ART_MAX_IMAGE_PIXELS, PixelArtSnapError, snap_pixel_art, snap_pixel_art_strict, + snap_pixel_art_strict_with_deadline, snap_pixel_art_with_deadline, }; pub use vector_engine::{ DownloadedImage, GPT_IMAGE_2_C_MODEL, GPT_IMAGE_2_MODEL, GeneratedImages, NANOBANANA_2_MODEL, diff --git a/server-rs/crates/platform-image/src/pixel_art_snapper.rs b/server-rs/crates/platform-image/src/pixel_art_snapper.rs index 8ca7635b8..b7d388465 100644 --- a/server-rs/crates/platform-image/src/pixel_art_snapper.rs +++ b/server-rs/crates/platform-image/src/pixel_art_snapper.rs @@ -38,6 +38,14 @@ const WALKER_STRENGTH_THRESHOLD: f64 = 0.5; const MIN_CUTS_PER_AXIS: usize = 4; const FALLBACK_TARGET_SEGMENTS: usize = 64; const MAX_STEP_RATIO: f64 = 1.8; +const STRICT_GRID_MIN_PEAKS_PER_AXIS: usize = 4; +const STRICT_GRID_MAX_AXIS_STEP_RATIO: f64 = 1.25; +const STRICT_GRID_MAX_INTERVAL_RESIDUAL: f64 = 0.2; +const STRICT_GRID_MIN_INTERVAL_HIT_RATIO: f64 = 0.7; +const STRICT_GRID_MIN_PHASE_COHERENCE: f64 = 0.65; +const STRICT_GRID_MIN_STEP: f64 = 2.0; +const STRICT_GRID_MAX_STEP_SEARCH: f64 = 512.0; +const STRICT_GRID_STEP_SEARCH_INCREMENT: f64 = 0.25; // 0.375 expressed as an exact rational avoids a floating-point boundary // ambiguity when coverage lands exactly on the configured threshold. @@ -48,6 +56,7 @@ const DEADLINE_CHECK_INTERVAL: usize = 4_096; #[derive(Debug)] pub enum PixelArtSnapError { InvalidInput(String), + GridNotDetected, Decode { input: &'static str, message: String, @@ -63,6 +72,7 @@ impl fmt::Display for PixelArtSnapError { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { match self { Self::InvalidInput(message) => write!(formatter, "像素规整输入无效：{message}"), + Self::GridNotDetected => write!(formatter, "像素规整未识别到可规整的像素网格"), Self::Decode { input, message } => { write!(formatter, "像素规整无法解码 {input}：{message}") } @@ -161,6 +171,19 @@ pub fn snap_pixel_art( snap_pixel_art_with_deadline(grid_source, rgba_source, None) } +/// Snap an image only when the analysis detects an existing logical-pixel grid. +/// +/// Unlike [`snap_pixel_art`], this entry does not synthesize a uniform +/// min-dimension/64 grid when neither axis contains a detectable step. It is +/// intended for explicit "perfect pixel" actions where ordinary raster images +/// must fail instead of being turned into a pixelation filter. +pub fn snap_pixel_art_strict( + grid_source: &DownloadedImage, + rgba_source: &DownloadedImage, +) -> Result { + snap_pixel_art_strict_with_deadline(grid_source, rgba_source, None) +} + /// Deadline-aware variant of [`snap_pixel_art`]. /// /// The deadline is checked before and after non-cooperative codec/resize @@ -171,6 +194,24 @@ pub fn snap_pixel_art_with_deadline( grid_source: &DownloadedImage, rgba_source: &DownloadedImage, deadline: Option, +) -> Result { + snap_pixel_art_with_grid_policy(grid_source, rgba_source, deadline, true) +} + +/// Deadline-aware variant of [`snap_pixel_art_strict`]. +pub fn snap_pixel_art_strict_with_deadline( + grid_source: &DownloadedImage, + rgba_source: &DownloadedImage, + deadline: Option, +) -> Result { + snap_pixel_art_with_grid_policy(grid_source, rgba_source, deadline, false) +} + +fn snap_pixel_art_with_grid_policy( + grid_source: &DownloadedImage, + rgba_source: &DownloadedImage, + deadline: Option, + allow_undetected_grid_fallback: bool, ) -> Result { let deadline = DeadlineGuard::new(deadline); deadline.check("输入解码")?; @@ -194,18 +235,39 @@ pub fn snap_pixel_art_with_deadline( let config = SnapConfig::PRODUCTION; let quantized_grid = quantize_for_analysis(grid_image, config, deadline)?; - let (profile_x, profile_y) = compute_profiles(&quantized_grid, deadline)?; - let estimated_x = estimate_step_size(&profile_x, config); - let estimated_y = estimate_step_size(&profile_y, config); + let (profile_x, profile_y) = if allow_undetected_grid_fallback { + compute_profiles(&quantized_grid, deadline)? + } else { + compute_strict_boundary_profiles(&quantized_grid, deadline)? + }; + let plateau_aware = !allow_undetected_grid_fallback; + let estimated_x = estimate_step_size(&profile_x, config, plateau_aware); + let estimated_y = estimate_step_size(&profile_y, config, plateau_aware); + if !allow_undetected_grid_fallback { + validate_strict_pixel_grid(estimated_x.as_ref(), estimated_y.as_ref())?; + } let (step_x, step_y) = resolve_step_sizes( - estimated_x, - estimated_y, + estimated_x.as_ref().map(|estimate| estimate.step), + estimated_y.as_ref().map(|estimate| estimate.step), rgba_image.width(), rgba_image.height(), config, ); - let raw_columns = walk(&profile_x, step_x, rgba_image.width() as usize, config)?; - let raw_rows = walk(&profile_y, step_y, rgba_image.height() as usize, config)?; + let prefer_target_on_ties = !allow_undetected_grid_fallback; + let raw_columns = walk( + &profile_x, + step_x, + rgba_image.width() as usize, + config, + prefer_target_on_ties, + )?; + let raw_rows = walk( + &profile_y, + step_y, + rgba_image.height() as usize, + config, + prefer_target_on_ties, + )?; let (columns, rows) = stabilize_both_axes( &profile_x, &profile_y, @@ -556,6 +618,42 @@ fn compute_profiles( Ok((profile_x, profile_y)) } +fn compute_strict_boundary_profiles( + source: &RgbaImage, + deadline: DeadlineGuard, +) -> Result<(Vec, Vec), PixelArtSnapError> { + deadline.check("严格网格边缘分析")?; + let (width, height) = source.dimensions(); + if width < 3 || height < 3 { + return Err(PixelArtSnapError::Processing( + "网格分析要求图片至少为 3×3".to_string(), + )); + } + + let width = width as usize; + let height = height as usize; + let pixels = source.as_raw(); + let mut profile_x = vec![0.0f64; width]; + let mut profile_y = vec![0.0f64; height]; + for y in 0..height { + deadline.check("严格网格横向边缘分析")?; + for (x, value) in profile_x.iter_mut().enumerate().take(width).skip(1) { + let left = rgba_luminance(pixels, y * width + x - 1); + let current = rgba_luminance(pixels, y * width + x); + *value += (current - left).abs(); + } + } + for (y, value) in profile_y.iter_mut().enumerate().take(height).skip(1) { + deadline.check("严格网格纵向边缘分析")?; + for x in 0..width { + let previous = rgba_luminance(pixels, (y - 1) * width + x); + let current = rgba_luminance(pixels, y * width + x); + *value += (current - previous).abs(); + } + } + Ok((profile_x, profile_y)) +} + fn rgba_luminance(pixels: &[u8], pixel_index: usize) -> f64 { let offset = pixel_index * 4; if pixels[offset + 3] == 0 { @@ -567,26 +665,45 @@ fn rgba_luminance(pixels: &[u8], pixel_index: usize) -> f64 { } } -fn estimate_step_size(profile: &[f64], config: SnapConfig) -> Option { +#[derive(Debug)] +struct StepEstimate { + step: f64, + clean_peaks: Vec, +} + +fn estimate_step_size( + profile: &[f64], + config: SnapConfig, + plateau_aware: bool, +) -> Option { let maximum = profile.iter().copied().fold(0.0f64, f64::max); if maximum <= 0.0 { return None; } let threshold = maximum * config.peak_threshold_multiplier; - let peaks = (1..profile.len().saturating_sub(1)) - .filter(|&index| { - profile[index] > threshold - && profile[index] > profile[index - 1] - && profile[index] > profile[index + 1] - }) - .collect::>(); + let peaks = if plateau_aware { + find_profile_peaks_with_plateaus(profile, threshold) + } else { + (1..profile.len().saturating_sub(1)) + .filter(|&index| { + profile[index] > threshold + && profile[index] > profile[index - 1] + && profile[index] > profile[index + 1] + }) + .collect::>() + }; if peaks.len() < 2 { return None; } + let peak_distance_filter = if plateau_aware { + 1 + } else { + config.peak_distance_filter + }; let mut clean_peaks = vec![peaks[0]]; for peak in peaks.into_iter().skip(1) { - if peak - clean_peaks[clean_peaks.len() - 1] > config.peak_distance_filter - 1 { + if peak - clean_peaks[clean_peaks.len() - 1] > peak_distance_filter - 1 { clean_peaks.push(peak); } } @@ -604,10 +721,118 @@ fn estimate_step_size(profile: &[f64], config: SnapConfig) -> Option { let lower_index = percentile_position.floor() as usize; let upper_index = percentile_position.ceil() as usize; let interpolation = percentile_position - lower_index as f64; - Some( + let step = if plateau_aware { + resolve_strict_grid_step(clean_peaks.as_slice())? + } else { differences[lower_index] - + (differences[upper_index] - differences[lower_index]) * interpolation, - ) + + (differences[upper_index] - differences[lower_index]) * interpolation + }; + Some(StepEstimate { step, clean_peaks }) +} + +fn find_profile_peaks_with_plateaus(profile: &[f64], threshold: f64) -> Vec { + let mut peaks = Vec::new(); + let mut index = 1usize; + while index < profile.len().saturating_sub(1) { + let value = profile[index]; + if value <= threshold { + index += 1; + continue; + } + let start = index; + let mut end = index; + while end + 1 < profile.len().saturating_sub(1) + && (profile[end + 1] - value).abs() <= f64::EPSILON * value.abs().max(1.0) + { + end += 1; + } + if value > profile[start - 1] && value > profile[end + 1] { + peaks.push(start + (end - start).div_ceil(2)); + } + index = end + 1; + } + peaks +} + +fn validate_strict_pixel_grid( + estimated_x: Option<&StepEstimate>, + estimated_y: Option<&StepEstimate>, +) -> Result<(), PixelArtSnapError> { + let (Some(estimated_x), Some(estimated_y)) = (estimated_x, estimated_y) else { + return Err(PixelArtSnapError::GridNotDetected); + }; + if !strict_grid_axis_is_confident(estimated_x) || !strict_grid_axis_is_confident(estimated_y) { + return Err(PixelArtSnapError::GridNotDetected); + } + let axis_step_ratio = + estimated_x.step.max(estimated_y.step) / estimated_x.step.min(estimated_y.step); + if !axis_step_ratio.is_finite() || axis_step_ratio > STRICT_GRID_MAX_AXIS_STEP_RATIO { + return Err(PixelArtSnapError::GridNotDetected); + } + Ok(()) +} + +fn resolve_strict_grid_step(clean_peaks: &[usize]) -> Option { + if clean_peaks.len() < STRICT_GRID_MIN_PEAKS_PER_AXIS { + return None; + } + let max_interval = clean_peaks + .windows(2) + .map(|pair| (pair[1] - pair[0]) as f64) + .fold(0.0f64, f64::max) + .min(STRICT_GRID_MAX_STEP_SEARCH); + let mut candidate = (max_interval / STRICT_GRID_STEP_SEARCH_INCREMENT).ceil() + * STRICT_GRID_STEP_SEARCH_INCREMENT; + while candidate >= STRICT_GRID_MIN_STEP { + let (interval_hit_ratio, phase_coherence) = + strict_grid_axis_confidence(clean_peaks, candidate); + if interval_hit_ratio >= STRICT_GRID_MIN_INTERVAL_HIT_RATIO + && phase_coherence >= STRICT_GRID_MIN_PHASE_COHERENCE + { + return Some(candidate); + } + candidate -= STRICT_GRID_STEP_SEARCH_INCREMENT; + } + None +} + +fn strict_grid_axis_is_confident(estimate: &StepEstimate) -> bool { + if !estimate.step.is_finite() + || estimate.step < 1.0 + || estimate.clean_peaks.len() < STRICT_GRID_MIN_PEAKS_PER_AXIS + { + return false; + } + + let (interval_hit_ratio, phase_coherence) = + strict_grid_axis_confidence(estimate.clean_peaks.as_slice(), estimate.step); + interval_hit_ratio >= STRICT_GRID_MIN_INTERVAL_HIT_RATIO + && phase_coherence >= STRICT_GRID_MIN_PHASE_COHERENCE +} + +fn strict_grid_axis_confidence(clean_peaks: &[usize], step: f64) -> (f64, f64) { + let interval_count = clean_peaks.len().saturating_sub(1); + if interval_count == 0 || !step.is_finite() || step < STRICT_GRID_MIN_STEP { + return (0.0, 0.0); + } + let interval_hits = clean_peaks + .windows(2) + .filter(|pair| { + let normalized = (pair[1] - pair[0]) as f64 / step; + (normalized - normalized.round()).abs() <= STRICT_GRID_MAX_INTERVAL_RESIDUAL + }) + .count(); + let interval_hit_ratio = interval_hits as f64 / interval_count as f64; + + let (phase_cosine, phase_sine) = + clean_peaks + .iter() + .fold((0.0f64, 0.0f64), |(cosine, sine), peak| { + let phase = std::f64::consts::TAU * *peak as f64 / step; + (cosine + phase.cos(), sine + phase.sin()) + }); + let phase_coherence = phase_cosine.hypot(phase_sine) / clean_peaks.len() as f64; + (interval_hit_ratio, phase_coherence) } fn resolve_step_sizes( @@ -642,6 +867,7 @@ fn walk( step_size: f64, limit: usize, config: SnapConfig, + prefer_target_on_ties: bool, ) -> Result, PixelArtSnapError> { if profile.is_empty() { return Err(PixelArtSnapError::Processing( @@ -679,7 +905,16 @@ fn walk( let mut best_index = start; let mut best_value = -1.0f64; for (index, value) in profile.iter().enumerate().take(end).skip(start) { - if *value > best_value { + let tied = (*value - best_value).abs() + <= f64::EPSILON * value.abs().max(best_value.abs()).max(1.0); + let closer_to_target = + (index as f64 - target).abs() < (best_index as f64 - target).abs(); + let equally_close_and_later = (index as f64 - target).abs() + == (best_index as f64 - target).abs() + && index > best_index; + if *value > best_value + || (prefer_target_on_ties && tied && (closer_to_target || equally_close_and_later)) + { best_value = *value; best_index = index; } @@ -1102,6 +1337,162 @@ mod tests { )); } + #[test] + fn strict_mode_accepts_a_nearest_upscaled_sprite_grid() { + let palette = [ + Rgba([20, 30, 40, 255]), + Rgba([230, 90, 40, 255]), + Rgba([60, 190, 100, 255]), + Rgba([245, 220, 90, 255]), + ]; + let mut logical = RgbaImage::new(16, 16); + for y in 0..16 { + for x in 0..16 { + let palette_index = ((x * 3 + y * 5 + (x ^ y)) % palette.len() as u32) as usize; + logical.put_pixel(x, y, palette[palette_index]); + } + } + let sprite = imageops::resize(&logical, 128, 128, FilterType::Nearest); + let source = downloaded_png(sprite.clone()); + + let output = snap_pixel_art_strict(&source, &source) + .expect("nearest-upscaled sprite should expose a confident grid"); + + assert_eq!(decode_output(&output), sprite); + } + + #[test] + fn strict_mode_preserves_dense_nearest_sprites_at_small_scales() { + let palette = [ + Rgba([15, 25, 35, 255]), + Rgba([220, 70, 45, 255]), + Rgba([55, 180, 100, 255]), + Rgba([245, 215, 85, 255]), + ]; + let mut logical = RgbaImage::new(16, 16); + for y in 0..16 { + for x in 0..16 { + logical.put_pixel(x, y, palette[((x + y * 2) % 4) as usize]); + } + } + + for scale in [2, 3, 8] { + let sprite = imageops::resize( + &logical, + logical.width() * scale, + logical.height() * scale, + FilterType::Nearest, + ); + let source = downloaded_png(sprite.clone()); + let output = snap_pixel_art_strict(&source, &source) + .expect("dense nearest sprite should retain its detected logical grid"); + + assert_eq!( + decode_output(&output), + sprite, + "{scale}× nearest sprite should remain pixel-exact" + ); + } + } + + #[test] + fn strict_mode_accepts_a_slightly_jittered_sprite_grid() { + let palette = [ + Rgba([30, 40, 55, 255]), + Rgba([220, 75, 45, 255]), + Rgba([65, 175, 110, 255]), + Rgba([240, 215, 80, 255]), + ]; + let mut boundaries = (0..=16) + .map(|index| { + if index == 0 { + 0 + } else if index == 16 { + 128 + } else { + let jitter = match index % 3 { + 0 => -1, + 1 => 0, + _ => 1, + }; + (index * 8) as i32 + jitter + } + }) + .map(|value| value as u32) + .collect::>(); + boundaries.sort_unstable(); + let mut sprite = RgbaImage::new(128, 128); + for cell_y in 0..16 { + for cell_x in 0..16 { + let palette_index = + ((cell_x * 5 + cell_y * 3 + (cell_x ^ cell_y)) % palette.len()) as usize; + for y in boundaries[cell_y]..boundaries[cell_y + 1] { + for x in boundaries[cell_x]..boundaries[cell_x + 1] { + sprite.put_pixel(x, y, palette[palette_index]); + } + } + } + } + let source = downloaded_png(sprite); + + let output = snap_pixel_art_strict(&source, &source) + .expect("small grid-boundary jitter should remain a confident pixel grid"); + + assert_eq!(decode_output(&output).dimensions(), (128, 128)); + } + + #[test] + fn strict_mode_rejects_smooth_and_photo_like_inputs() { + let mut gradient = RgbaImage::new(128, 128); + for y in 0..128 { + for x in 0..128 { + gradient.put_pixel( + x, + y, + Rgba([ + (x * 255 / 127) as u8, + (y * 255 / 127) as u8, + ((x + y) * 255 / 254) as u8, + 255, + ]), + ); + } + } + + let mut seed = 0x6d2b_79f5u32; + let mut noise = RgbaImage::new(128, 128); + for pixel in noise.pixels_mut() { + seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223); + let red = (seed >> 24) as u8; + seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223); + let green = (seed >> 24) as u8; + seed = seed.wrapping_mul(1_664_525).wrapping_add(1_013_904_223); + let blue = (seed >> 24) as u8; + *pixel = Rgba([red, green, blue, 255]); + } + + for source in [downloaded_png(gradient), downloaded_png(noise)] { + snap_pixel_art(&source, &source) + .expect("legacy generation style should keep its existing fallback behavior"); + let error = snap_pixel_art_strict(&source, &source) + .expect_err("non-grid raster should not become a pixelation filter"); + assert!(matches!(error, PixelArtSnapError::GridNotDetected)); + } + } + + #[test] + fn legacy_mode_keeps_uniform_grid_fallback() { + let source = downloaded_png(RgbaImage::from_pixel(128, 128, Rgba([10, 20, 30, 255]))); + + let legacy = snap_pixel_art(&source, &source) + .expect("legacy generation style should retain its uniform fallback"); + let strict = snap_pixel_art_strict(&source, &source) + .expect_err("explicit strict action should reject an undetected grid"); + + assert_eq!(decode_output(&legacy).dimensions(), (128, 128)); + assert!(matches!(strict, PixelArtSnapError::GridNotDetected)); + } + #[test] fn output_keeps_physical_size_and_uses_nearest_blocks() { let grid_image = RgbaImage::from_pixel(128, 128, Rgba([0, 0, 0, 255])); diff --git a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx index 23f387937..c6eb4a796 100644 --- a/src/components/image-editor/ImageCanvasEditorShellView.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorShellView.test.tsx @@ -195,6 +195,7 @@ function createStageProps(): ImageCanvasStageViewProps { onOpenRedrawPanel: vi.fn(), onOpenCropExpandPanel: vi.fn(), onRemoveBackground: vi.fn(), + onPerfectPixel: vi.fn(), onSplitIconSpritesheet: vi.fn(), onExtractUiDesignAssets: vi.fn(), onUiAssetExtractionToolChange: vi.fn(), diff --git a/src/components/image-editor/ImageCanvasEditorTypes.ts b/src/components/image-editor/ImageCanvasEditorTypes.ts index 64b479ef9..44b54d4a7 100644 --- a/src/components/image-editor/ImageCanvasEditorTypes.ts +++ b/src/components/image-editor/ImageCanvasEditorTypes.ts @@ -289,6 +289,7 @@ export type CanvasHistoryActionType = | 'generate-image' | 'expand-image' | 'remove-background' + | 'perfect-pixel' | 'split-atlas' | 'replace-image' | 'show-image' diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index 85243ec91..f935d1d5b 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -1464,6 +1464,7 @@ export function ImageCanvasEditorView({ assetFolderId: activeUploadFolderId, upsertGeneratedAsset, applyProjectSnapshot: applyGeneratedProjectSnapshot, + flushProjectPersistence, onWalletBalanceMayHaveChanged: refreshEditorWalletState, }); const handleEditorAgentConfirmSent = useCallback(() => { @@ -1581,6 +1582,8 @@ export function ImageCanvasEditorView({ openRedrawPanel, openCropExpandPanel, removeSelectedLayerBackground, + snapSelectedLayerToPerfectPixels, + perfectPixelLayerIds, splitSelectedIconSpritesheet, splittingIconSpritesheetLayerIds, extractUiDesignAssets, @@ -2357,6 +2360,7 @@ export function ImageCanvasEditorView({ quickEditSelectionSourceLayer, generationComposerStyle, selectedToolbarStyle, + perfectPixelLayerIds, splittingIconSpritesheetLayerIds, persistingAssetKindLayerIds, uploadDropTarget, @@ -2409,6 +2413,7 @@ export function ImageCanvasEditorView({ onOpenRedrawPanel: openRedrawPanel, onOpenCropExpandPanel: openCropExpandPanel, onRemoveBackground: removeSelectedLayerBackground, + onPerfectPixel: snapSelectedLayerToPerfectPixels, onSplitIconSpritesheet: (layer: CanvasLayer) => { if (persistingAssetKindLayerIdsRef.current.has(layer.id)) { return; diff --git a/src/components/image-editor/ImageCanvasHistoryModel.test.ts b/src/components/image-editor/ImageCanvasHistoryModel.test.ts index 25401a6f4..abdde1cac 100644 --- a/src/components/image-editor/ImageCanvasHistoryModel.test.ts +++ b/src/components/image-editor/ImageCanvasHistoryModel.test.ts @@ -625,6 +625,12 @@ describe('ImageCanvasHistoryModel', () => { expect(isProtectedCanvasHistoryAction({ type: 'replace-image' })).toBe( true, ); + expect(formatCanvasHistoryAction({ type: 'perfect-pixel' })).toBe( + '完美像素', + ); + expect(isProtectedCanvasHistoryAction({ type: 'perfect-pixel' })).toBe( + true, + ); expect(isProtectedCanvasHistoryAction({ type: 'move-image' })).toBe(false); }); }); diff --git a/src/components/image-editor/ImageCanvasHistoryModel.ts b/src/components/image-editor/ImageCanvasHistoryModel.ts index 8ace4aed9..2d05224ba 100644 --- a/src/components/image-editor/ImageCanvasHistoryModel.ts +++ b/src/components/image-editor/ImageCanvasHistoryModel.ts @@ -22,6 +22,7 @@ const CANVAS_HISTORY_ACTION_LABELS: Record< 'generate-image': '生成图片', 'expand-image': '扩展图片', 'remove-background': '移除背景', + 'perfect-pixel': '完美像素', 'split-atlas': '拆分图集', 'replace-image': '替换图片', 'show-image': '显示图片', @@ -46,6 +47,7 @@ const PROTECTED_CANVAS_HISTORY_ACTION_TYPES = new Set< 'generate-image', 'expand-image', 'remove-background', + 'perfect-pixel', 'split-atlas', 'replace-image', ]); diff --git a/src/components/image-editor/ImageCanvasRasterEditModel.ts b/src/components/image-editor/ImageCanvasRasterEditModel.ts index 6facd9461..c46c29343 100644 --- a/src/components/image-editor/ImageCanvasRasterEditModel.ts +++ b/src/components/image-editor/ImageCanvasRasterEditModel.ts @@ -1,7 +1,10 @@ import { type EditorBackgroundRemovalInput, type EditorBackgroundRemovalResult, + type EditorPixelArtSnapInput, + type EditorPixelArtSnapResult, removeEditorImageBackground, + snapEditorImageToPixelArt, } from '../../services/image-editor/editorProjectClient'; export type CropExpandInsets = { @@ -222,3 +225,9 @@ export async function removeImageBackground( typeof input === 'string' ? { sourceImageSrc: input } : input, ); } + +export async function snapImageToPerfectPixels( + input: EditorPixelArtSnapInput, +): Promise { + return snapEditorImageToPixelArt(input); +} diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx index b06f0013e..8d16de5b3 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx @@ -42,6 +42,8 @@ function renderSelectedToolbar( onOpenRedrawPanel: vi.fn(), onOpenCropExpandPanel: vi.fn(), onRemoveBackground: vi.fn(), + onPerfectPixel: vi.fn(), + isPerfectPixelProcessing: false, isSplittingIconSpritesheet: false, isPersistingAssetKind: false, onSplitIconSpritesheet: vi.fn(), @@ -66,6 +68,7 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { '快速编辑', '裁扩按钮', '去除背景按钮', + '完美像素', '改造', '下载按钮', ]); @@ -75,6 +78,7 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { fireEvent.click( within(toolbar).getByRole('button', { name: '去除背景按钮' }), ); + fireEvent.click(within(toolbar).getByRole('button', { name: '完美像素' })); fireEvent.click(within(toolbar).getByRole('button', { name: '改造' })); fireEvent.click(within(toolbar).getByRole('button', { name: '下载按钮' })); @@ -85,6 +89,7 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { props.selectedLayer, ); expect(props.onRemoveBackground).toHaveBeenCalledWith(props.selectedLayer); + expect(props.onPerfectPixel).toHaveBeenCalledWith(props.selectedLayer); expect(props.onOpenRedrawPanel).toHaveBeenCalledWith(props.selectedLayer); expect(props.onDownloadLayer).toHaveBeenCalledWith(props.selectedLayer); expect( @@ -104,6 +109,19 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { .getByRole('button', { name: '去除背景按钮' }) .querySelector('.lucide-image-off'), ).toBeTruthy(); + expect( + within(toolbar) + .getByRole('button', { name: '完美像素' }) + .querySelector('.lucide-grid-2x2'), + ).toBeTruthy(); + expect( + within(toolbar).getByRole('button', { name: '完美像素' }).textContent, + ).toContain('完美像素'); + expect( + within(toolbar) + .getByRole('button', { name: '完美像素' }) + .getAttribute('title'), + ).toBe('自动识别并规整像素网格'); }); it('renders UI design asset extraction after remove background', () => { @@ -119,6 +137,7 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { '快速编辑', '裁扩按钮', '去除背景按钮', + '完美像素', '提取素材', '改造', '下载按钮', @@ -142,6 +161,7 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { '快速编辑', '裁扩按钮', '去除背景按钮', + '完美像素', '拆分图集', '改造', '下载按钮', @@ -184,6 +204,28 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect(props.onSplitIconSpritesheet).not.toHaveBeenCalled(); }); + it('renders a disabled loading state while perfect pixel is processing', () => { + const props = renderSelectedToolbar({ + isPerfectPixelProcessing: true, + }); + const button = screen.getByRole('button', { + name: '完美像素处理中', + }); + + expect(button.getAttribute('aria-busy')).toBe('true'); + expect((button as HTMLButtonElement).disabled).toBe(true); + expect( + button.querySelector('.lucide-loader-circle.animate-spin'), + ).toBeTruthy(); + expect(button.querySelector('.lucide-grid-2x2')).toBeNull(); + expect(button.textContent).toContain('处理中'); + + fireEvent.click(button); + fireEvent.click(button); + + expect(props.onPerfectPixel).not.toHaveBeenCalled(); + }); + it('does not invoke atlas splitting while the selected asset kind is persisting', () => { const layer = createLayer({ assetKind: 'icon-spritesheet' }); const props = renderSelectedToolbar({ @@ -227,6 +269,9 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect( within(toolbar).queryByRole('button', { name: '去除背景按钮' }), ).toBeNull(); + expect( + within(toolbar).queryByRole('button', { name: '完美像素' }), + ).toBeNull(); expect( within(toolbar).queryByRole('button', { name: '生成动画' }), ).toBeNull(); @@ -255,6 +300,9 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect( within(videoToolbar).queryByRole('button', { name: '去除背景按钮' }), ).toBeNull(); + expect( + within(videoToolbar).queryByRole('button', { name: '完美像素' }), + ).toBeNull(); expect( within(videoToolbar) .getAllByRole('button') @@ -278,6 +326,9 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect( within(actionToolbar).queryByRole('button', { name: '去除背景按钮' }), ).toBeNull(); + expect( + within(actionToolbar).queryByRole('button', { name: '完美像素' }), + ).toBeNull(); expect( within(actionToolbar) .getAllByRole('button') @@ -341,6 +392,8 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { onOpenRedrawPanel={vi.fn()} onOpenCropExpandPanel={vi.fn()} onRemoveBackground={vi.fn()} + onPerfectPixel={vi.fn()} + isPerfectPixelProcessing={false} isSplittingIconSpritesheet={false} isPersistingAssetKind={false} onSplitIconSpritesheet={vi.fn()} @@ -360,6 +413,8 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { onOpenRedrawPanel={vi.fn()} onOpenCropExpandPanel={vi.fn()} onRemoveBackground={vi.fn()} + onPerfectPixel={vi.fn()} + isPerfectPixelProcessing={false} isSplittingIconSpritesheet={false} isPersistingAssetKind={false} onSplitIconSpritesheet={vi.fn()} diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx index b1a423324..f76b3fc32 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx @@ -1,6 +1,7 @@ import { Crop, Download, + Grid2X2, ImageOff, Loader2, PersonStanding, @@ -22,6 +23,8 @@ type ImageCanvasSelectedLayerToolbarViewProps = { onOpenRedrawPanel: (layer: CanvasLayer) => void; onOpenCropExpandPanel: (layer: CanvasLayer) => void; onRemoveBackground: (layer: CanvasLayer) => void; + onPerfectPixel: (layer: CanvasLayer) => void; + isPerfectPixelProcessing: boolean; isSplittingIconSpritesheet: boolean; isPersistingAssetKind: boolean; onSplitIconSpritesheet: (layer: CanvasLayer) => void; @@ -37,6 +40,8 @@ export function ImageCanvasSelectedLayerToolbarView({ onOpenRedrawPanel, onOpenCropExpandPanel, onRemoveBackground, + onPerfectPixel, + isPerfectPixelProcessing, isSplittingIconSpritesheet, isPersistingAssetKind, onSplitIconSpritesheet, @@ -120,6 +125,27 @@ export function ImageCanvasSelectedLayerToolbarView({ icon={ImageOff} onClick={() => onRemoveBackground(selectedLayer)} /> + + ) : ( + + ) + } + disabled={isPerfectPixelProcessing} + aria-busy={isPerfectPixelProcessing} + onClick={() => onPerfectPixel(selectedLayer)} + > + {isPerfectPixelProcessing ? '处理中' : '完美像素'} + ) : null} {selectedLayer.assetKind === 'icon-spritesheet' ? ( diff --git a/src/components/image-editor/ImageCanvasStageView.tsx b/src/components/image-editor/ImageCanvasStageView.tsx index fee6cf878..74b2014a8 100644 --- a/src/components/image-editor/ImageCanvasStageView.tsx +++ b/src/components/image-editor/ImageCanvasStageView.tsx @@ -78,6 +78,7 @@ export type ImageCanvasStageViewProps = { generationComposerStyle: CSSProperties | null; selectedToolbarStyle: CSSProperties | null; splittingIconSpritesheetLayerIds?: ReadonlySet; + perfectPixelLayerIds?: ReadonlySet; persistingAssetKindLayerIds?: ReadonlySet; uploadDropTarget: 'canvas' | 'assets' | null; contextMenu: CanvasContextMenuState | null; @@ -146,6 +147,7 @@ export type ImageCanvasStageViewProps = { onOpenRedrawPanel: (layer: CanvasLayer) => void; onOpenCropExpandPanel: (layer: CanvasLayer) => void; onRemoveBackground: (layer: CanvasLayer) => void; + onPerfectPixel: (layer: CanvasLayer) => void; onSplitIconSpritesheet: (layer: CanvasLayer) => void; onExtractUiDesignAssets: (layer: CanvasLayer) => void; onUiAssetExtractionToolChange: (tool: UiAssetExtractionTool | null) => void; @@ -233,6 +235,7 @@ export function ImageCanvasStageView({ generationComposerStyle, selectedToolbarStyle, splittingIconSpritesheetLayerIds = EMPTY_LAYER_ID_SET, + perfectPixelLayerIds = EMPTY_LAYER_ID_SET, persistingAssetKindLayerIds = EMPTY_LAYER_ID_SET, uploadDropTarget, contextMenu, @@ -280,6 +283,7 @@ export function ImageCanvasStageView({ onOpenRedrawPanel, onOpenCropExpandPanel, onRemoveBackground, + onPerfectPixel, onSplitIconSpritesheet, onExtractUiDesignAssets, onUiAssetExtractionToolChange, @@ -404,10 +408,14 @@ export function ImageCanvasStageView({ isPersistingAssetKind={Boolean( selectedLayer && persistingAssetKindLayerIds.has(selectedLayer.id), )} + isPerfectPixelProcessing={Boolean( + selectedLayer && perfectPixelLayerIds.has(selectedLayer.id), + )} onOpenQuickEditPanel={onOpenQuickEditPanel} onOpenRedrawPanel={onOpenRedrawPanel} onOpenCropExpandPanel={onOpenCropExpandPanel} onRemoveBackground={onRemoveBackground} + onPerfectPixel={onPerfectPixel} onSplitIconSpritesheet={onSplitIconSpritesheet} onExtractUiDesignAssets={onExtractUiDesignAssets} onOpenCharacterAnimationPanel={onOpenCharacterAnimationPanel} diff --git a/src/components/image-editor/useImageCanvasGenerationSurface.tsx b/src/components/image-editor/useImageCanvasGenerationSurface.tsx index a95b296c7..0324aee26 100644 --- a/src/components/image-editor/useImageCanvasGenerationSurface.tsx +++ b/src/components/image-editor/useImageCanvasGenerationSurface.tsx @@ -106,6 +106,7 @@ type ImageCanvasGenerationSurfaceOptions = { project: EditorProjectSnapshot, action?: CanvasHistoryAction, ) => void; + flushProjectPersistence?: () => Promise; onWalletBalanceMayHaveChanged?: () => void; }; @@ -182,6 +183,7 @@ export function useImageCanvasGenerationSurface({ assetFolderId, upsertGeneratedAsset, applyProjectSnapshot, + flushProjectPersistence, onWalletBalanceMayHaveChanged, }: ImageCanvasGenerationSurfaceOptions) { const toolbarOptionCloseTimerRef = useRef vi.fn()); const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn()); const renderCropExpandImageMock = vi.hoisted(() => vi.fn()); const removeImageBackgroundMock = vi.hoisted(() => vi.fn()); +const snapImageToPerfectPixelsMock = vi.hoisted(() => vi.fn()); const resolveEditorImageReferenceDataUrlMock = vi.hoisted(() => vi.fn()); vi.mock('../../services/image-editor/editorImageReference', async () => { @@ -77,6 +78,7 @@ vi.mock('./ImageCanvasRasterEditModel', async () => { ...actual, removeImageBackground: removeImageBackgroundMock, renderCropExpandImage: renderCropExpandImageMock, + snapImageToPerfectPixels: snapImageToPerfectPixelsMock, }; }); @@ -119,6 +121,8 @@ function GenerationWorkflowHarness({ projectId, currentUserId, applyProjectSnapshot, + flushProjectPersistence, + upsertGeneratedAsset, }: { initialLayers?: CanvasLayer[]; initialViewport?: { x: number; y: number; scale: number }; @@ -127,6 +131,12 @@ function GenerationWorkflowHarness({ applyProjectSnapshot?: Parameters< typeof useImageCanvasGenerationWorkflow >[0]['applyProjectSnapshot']; + flushProjectPersistence?: Parameters< + typeof useImageCanvasGenerationWorkflow + >[0]['flushProjectPersistence']; + upsertGeneratedAsset?: Parameters< + typeof useImageCanvasGenerationWorkflow + >[0]['upsertGeneratedAsset']; }) { const [layers, setLayers] = useState(initialLayers); const [viewport, setViewport] = useState(initialViewport); @@ -175,6 +185,8 @@ function GenerationWorkflowHarness({ projectId, currentUserId, applyProjectSnapshot, + flushProjectPersistence, + upsertGeneratedAsset, }); const activeDialog = dialogs.generateDialog; @@ -292,6 +304,9 @@ function GenerationWorkflowHarness({ {fitLayersMockRef.current.mock.calls.length} {workflow.taskListRefreshKey} + + {workflow.perfectPixelLayerIds.has(layers[0]!.id) ? '处理中' : '空闲'} + @@ -529,6 +544,24 @@ function GenerationWorkflowHarness({ > 去除背景 + +