diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 0975b8153..14a50e7fe 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -29,10 +29,10 @@ ## 2026-07-31 修正抠图内部元数据的普通用户读取边界 - 背景:2026-07-30 的记录误把素材 owner 与后台审计并列为原始抠图执行信息的读取方。owner 是普通用户,前端不展示字段不能阻止其从项目资源、素材库、精选提交 / 点赞回包、画布布局或任务完成响应的网络 payload 读取 BgFilter、阿里云、本地键色、具体分割模型或背景色。 -- 决策:本条取代 2026-07-30 决策中“素材 owner、精选提交响应仍可读取原始值”的表述。普通用户(包括素材 owner)和匿名公开读取必须共同过滤素材顶层 `provider`、内部处理 `model`,以及 `generationInputs` 顶层 `screenColorHex`、`mattingProvider`、`mattingModel`;正常用户可见生成 `model` 和其他合法功能性顶层字段(例如 `characterAnimation`)保持不变。User/Owner mapper 先完成该清理,public mapper 在其基础上叠加公开字段规则;后台管理与服务端审计继续使用 raw mapper 和持久化原值。历史数据不迁移,统一在读取边界清理。 +- 决策:本条取代 2026-07-30 决策中“素材 owner、精选提交响应仍可读取原始值”的表述。普通用户(包括素材 owner)读取时必须过滤素材顶层 `provider`、内部处理 `model`,以及 `generationInputs` 顶层 `screenColorHex`、`mattingProvider`、`mattingModel`;正常用户可见生成 `model` 和其他合法功能性顶层字段(例如 `characterAnimation`)保持不变。匿名公开素材 payload 不包含整个 `generationInputs`。User/Owner mapper 完成普通用户清理,public mapper 在其基础上移除该 owner-only 配方字段;后台管理与服务端审计继续使用 raw mapper 和持久化原值。历史数据不迁移,统一在读取边界清理。 - 入站与持久化:客户端提交的 `generationInputs` 不得伪造上述内部键,服务端在实际处理完成后才写入可信值。手动去背景的正式素材 `model` 必须继承经服务端验证的正常源生图模型;若来源或祖先链不存在正常模型则为 `null`,不得写入 `BgFilter complex` 等内部处理模型。内部抠图 provider / model 可继续持久化供后台审计,普通用户完成响应和用户可见错误文本均不得暴露它们;手动去背景与角色动作透明化失败在 Owner HTTP / 任务状态边界统一替换为稳定业务文案,原始错误只留在任务记录、tracing 和后台审计。 - 影响范围:项目资源、素材库、精选提交 / 点赞、图片 / 图标 / 视频 / 音频 / 角色动画生成完成、Agent 紧凑结果和识别出的画布资源 / 图层快照的 User/Public mapper;手动去背景持久化和完成响应;External Editor API 创建素材 / 资源时的保留键入站清理;相应响应 / OpenAPI 契约、前端搜索 / 详情 / ZIP 过滤测试与后台 raw 审计测试。同源画布 BFF 的角色、图标和 UI 请求继续由前端自动提交默认 `segModel=birefnet`,后端继续校验并在缺失时回落默认值;该请求控制字段不进入 `generationInputs`、普通用户响应、搜索、详情、导出或错误详情。角色动作的 `seg_model` 继续由后端固定。不修改 SpacetimeDB schema、迁移或 bindings。 -- 验证方式:Owner 和匿名响应覆盖无 `provider`、无内部 `model`、无三个内部 `generationInputs` 键,且正常 `model` 与 `characterAnimation` 仍保留;Admin raw payload 保持完整。覆盖历史 `BgFilter complex`、三类入站伪造键、手动去背景源模型回溯和用户错误文本过滤;运行 api-server 定向测试、前端定向测试、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run typecheck`、`npm run check:encoding` 与 `git diff --check`。 +- 验证方式:Owner 响应覆盖无 `provider`、无内部 `model`、无三个内部 `generationInputs` 键,同时保留正常 `model` 与 `characterAnimation`;匿名公开素材响应完全不含 `generationInputs`;Admin raw payload 保持完整。覆盖历史 `BgFilter complex`、三类入站伪造键、手动去背景源模型回溯和用户错误文本过滤;运行 api-server 定向测试、前端定向测试、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run typecheck`、`npm run check:encoding` 与 `git diff --check`。 - 关联文档:`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 --- @@ -1024,7 +1024,7 @@ ## 2026-06-21 图片画布参考图元数据只保存项目内行引用 - 背景:参考图如果把 Data URL、signed URL 或 `objectKey` 写入 `generationInputs` 或生成器布局快照,会撑大资源 / 素材 / 画布 JSON,也无法稳定索引到项目内用户可见行数据。 -- 决策:`generationInputs.references` 只保存 `{ title, label, refType, refId }`,其中 `refType="project-resource"` 指向 `editor_project_resource.resourceId`,`refType="asset"` 指向 `editor_asset.assetId`。生成器 `itemType="generation-dialog"` 布局快照中的参考图也只保存 `resourceId/sourceAssetId` 和展示 label,不保存图片 Data URL、signed URL 或 `objectKey`;提交生成请求前的内存态可以临时持有 `src/objectKey`,刷新恢复时从 `editor_project_resource` / `editor_asset` 行补回请求所需图片源。不兼容旧 `src` 型参考图元数据。 +- 决策:`generationInputs.references` 只保存稳定行引用,不保存媒体本身。2026-08-03 起 V2 新写入结构为 `{ id, title, label?, refType, refId }`;其中 `refType="project-resource"` 指向 `editor_project_resource.resourceId`,`refType="asset"` 指向 `editor_asset.assetId`,媒体类型从对应行的查询结果获得。生成器 `itemType="generation-dialog"` 布局快照中的参考图也只保存 `resourceId/sourceAssetId` 和展示 label,不保存图片 Data URL、signed URL 或 `objectKey`;提交生成请求前的内存态可以临时持有 `src/objectKey`,刷新恢复时从 `editor_project_resource` / `editor_asset` 行补回请求所需图片源。不兼容旧 `src` 型参考图元数据。 - 影响范围:图片画布生成输入快照、生成器布局保存 / 恢复、参考图上传工作流、元数据弹窗和图片画布技术文档。 - 验证方式:运行图片画布生成模型、生成提交、上传工作流、项目持久化、元数据弹窗相关前端测试,执行 `npm run typecheck`、`npm run check:encoding` 和 `git diff --check`。 - 关联文档:`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 @@ -5908,3 +5908,12 @@ - 显式协作合同:autonomous 的旧 `code-prototype + quality-review` 首批合同退出。显式 project collaboration policy 或持久 batch 恢复若进入首批 `agent.delegate` 路径,只允许且要求三个 Director 各一次;策划与程序 Director 是只读规划且 `expectedArtifacts=[]`,美术 Director 是非只读规范图任务且必须交付 `assets/art-spec.png`。任何非 repair 底层委派与 isolated child 都在首批失败关闭;默认 manifest DAG 仍是唯一自动首轮执行链,不额外复制三个 Director 委派。 - 输出决策:保留未提交 `streaming / ready` 的当前 revision 门;已提交的专业 Agent final reply 继续使用既有 durable response-stream 身份,后续项目 revision 变化不再隐藏早期阶段回复。 - 关联:`apps/ai-game-creator-shell/scripts/start-tauri-dev.mjs`、`start-dev-stack.mjs`、`src-tauri/src/agent/runtime_protocol/autonomous_completion.rs`、`response_stream.rs`、`docs/technical/【技术方案】AI游戏创作智能体App实施计划-2026-06-24.md`。 + +## 2026-08-03 图片画布生成产物统一“改造”契约 + +- 背景:画布生成结果统一显示“改造”,但部分动画 / 音频结果无法恢复面板;另一些非生成型派生结果继承最近生成输入,产生错误可执行动作。中文标题和素材类别被同时当成显示文案、参数键和路由键,改名后容易漂移。 +- 决策:用户动作 `改造` 的语义是恢复原生成输入、编辑并生成新产物。沿用 `generation_inputs_json`,V2 以稳定 `action`、`fields[].id`、`references[].id/refType/refId` 作为唯一执行契约,`title` / `label` 只用于展示,字段值保留基础类型。引用的媒体类型从其资源行查回,不重复写入快照。提交前参数只归一一次,请求与持久快照共用同一归一值。 +- 兼容:恢复优先级为有效 V2 → 完整历史生成对话框 → legacy adapter。legacy 允许使用 `assetKind/mediaType`、历史标题别名、资源模型 / 尺寸 / 时长 / `sourceResourceId` 和当前默认值,但必须显示恢复告警;不回填存量数据,不做 SpacetimeDB schema 迁移。 +- 边界:独立裁扩、手动去背景和手动图集拆分结果不继承生成输入,不显示 `改造`;原生成任务内自动后处理产物可保留原输入。Owner 读取保留 V2 执行字段;匿名公开素材 payload 暂不返回 `generationInputs`,不沿用 owner 可执行配方 DTO。 +- 影响范围:图片、规范、角色、图标、UI、宣发、视频、音效、背景音乐、角色动作、生成型图片编辑和 UI 素材提取;不影响作品详情“作品改造”、`AI重绘` 或常规 `快速编辑`。 +- 关联文档:`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`、`docs/【编辑器】画板音乐生成入口设计-2026-06-18.md`。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index ef0e0f8ff..65799de4c 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -19,7 +19,7 @@ - 鼠标中键拖拽始终平移画布;长按 Space 临时进入抓手模式,松开后恢复原工具。 - 图片拖拽时显示水平 / 垂直吸附参考线,吸附到其它图层、生成占位框或画板的边缘与中心线;当移动元素接近两个同轴元素形成的等距位置时,支持横向或纵向等距吸附。 - 生成资源右上角显示元数据按钮,点击打开独立元数据窗口。图片信息页不展示后端组装后的生图 Prompt,也不提供复制 Prompt;只展示该图片生成时用户在面板里提交的输入快照,包括普通生成提示词、规范表单字段、角色设定、图标素材描述、快速编辑提示词、重绘提示词,以及角色规范 / 常规参考图 / 图标规范 / 编辑参考图等参考图卡片,并提供“复制信息”复制当前可见字段。参考图输入快照只保存 `refType/refId` 行引用,其中 `refType="project-resource"` 指向 `editor_project_resource.resourceId`,`refType="asset"` 指向 `editor_asset.assetId`;不得把图片 Data URL、普通 URL 或 `objectKey` 写入 `generationInputs.references`。旧数据或上传图片没有输入快照时显示 `-`,禁止回退展示内部 Prompt。 -- 对生成资源执行重绘时,在右侧创建新的生成结果图层,并自动调整视图显示原图和新图;重绘面板不因提交成功自动关闭,便于连续改提示词。重绘 / 改造输入框只允许从 `generationInputs.fields` 中恢复用户可见输入快照,例如普通生成提示词、视频描述、音效 `prompt`、背景音乐 `gpt_description_prompt`、角色设定、UI 用户输入、图标素材描述、规范表单和宣发素材字段;禁止回退展示资源 `prompt` / `actualPrompt` 中的后端拼接 Prompt、固定生成模板或模型默认提示词。没有用户输入快照的旧图层打开改造时保持空输入,等待用户重新填写。 +- 生成类产物的工具栏动作 `改造` 不在原产物上原地修改,而是把当次生成输入恢复到对应面板,允许编辑后生成新产物。V2 恢复严格按顶层 `action`、`fields[].id` 和 `references[].id/refType/refId` 定位,`title` / `label` 只是展示快照,不作为参数键。引用媒体类型通过 `refType/refId` 查回已 hydrate 的资源 / 图层获得,不重复写入 `generationInputs.references`。恢复顺序为:有效 V2;完整的历史生成对话框;按 `assetKind/mediaType`、历史中英文标题别名、资源模型 / 尺寸 / 时长 / `sourceResourceId` 的 legacy adapter,最后缺失值使用当前默认值并显示旧版恢复告警。禁止从中文标题、产物尺寸或 prompt 反推 V2 参数;legacy fallback 除外,但不得回退展示 `actualPrompt` 中的后端拼接 Prompt。 - 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和重绘保留既有 `gpt-image-2` 路径;图片快速编辑统一打开框选区域 + 单提示词 + 模型选择面板,默认沿用原图模型,不展示参考图或比例 / 尺寸控件;其中生成规范类图片固定 `16:9`、`2K`、`gpt-image-2`,面板底部用与可编辑面板一致的比例 / 尺寸 / 模型胶囊按钮展示固定参数,但按钮为禁用态,不允许在该面板改比例、尺寸或模型。`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`(`gemini-3.1-flash-image-preview`)和 `gpt-image-2`,默认 `nanobanana2`,并在两类面板之间沿用用户上次选择的模型;两类面板不展示抠图背景色或抠图模型选择;前端用户路径固定提交 `screenColor=auto` 和 `segModel=birefnet`,由后端自动决策具体抠图背景色,`anime-seg` 作为内部保留能力不在用户界面暴露。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`,请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`;`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`,请求体按 VectorEngine 文档映射 `size`。宣发素材三个工作流(游戏首图、详情五图、运营海报)固定使用 `gpt-image-2`,面板模型胶囊为禁用态,不提供 `nanobanana2` 入口;前端按 workflow 同时提交 `outputSize`、`aspectRatio` 和 `imageSize`,其中游戏首图为 `720x540 / 4:3`、详情单图为 `720x1280 / 9:16`、运营海报为 `1280x720 / 16:9`;后端收到 `kind: "publication-material"` 时也强制归一为 `gpt-image-2` 生成和计费,生成回填图层优先使用生成占位的 `originalWidth/originalHeight`,即使上游回包尺寸漂移也不得把宣发素材卡片变成随机 `1:1` 或 `4:3`。纯文本生成走 `/api/editor/images/generations`,重绘在前端优先复用当前图层 objectKey;尚未登记的本地图片先上传 OSS,再把 objectKey 交给同一图片生成 BFF,并在原图右侧生成一张新图;普通图层重绘作为 `quick-edit` 参考图提交,角色图层重绘必须按 `kind: "character"` 提交,继续套用角色生成器提示词限定、透明 PNG 后处理和角色资产持久化。`生成视频` 走 `/api/editor/videos/generations`,前端模型入口仅展示 Seedance 2.0 Fast / Seedance 2.0 / Kling 3.0 / Kling 3.0 Omni,不展示 Veo 入口,默认 Seedance 2.0 Fast;视频参数按当前正式面板支持的比例、时长、清晰度和声音开关提交,且 Seedance Fast 与 Seedance 标准版必须按各自真实模型 ID 独立映射,不得混用。生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框:点击生成图片后以当前视口世界中心为目标,经统一 placement 避让后创建选中的灰色占位框,输入框跟随占位框显示;普通图片、角色、图标图集、UI 设计图及其重绘 / 改造入口必须在比例或清晰度恢复、切换时同步把占位框 `width/height/originalWidth/originalHeight` 更新为目标像素尺寸,生成中不得继续显示默认 1K 框;UI 素材提取的 1K / 2K 图集占位和旧图片修改入口也分别使用本次目标尺寸与源图真实尺寸。待生成、生成中和失败后保留的占位图都必须继续支持拖动,生成完成时真实生成图或视频落在最新占位框位置,输入框继续跟随新生成图层;占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸,重新聚焦时再显示,且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定;点击所有图片 / 视频生成入口并确认请求开始后,必须隐藏对应设置面板,只保留画布内占位图或原图预览,并在预览上显示 Lovart 式生成中遮罩,避免“面板仍占屏”或“预览一起消失”。图片快速编辑和重绘在调用图片 BFF 前必须把当前图层图片解析为已上传的 objectKey 或资源 ID;浏览器临时图片需先上传 OSS;视频素材快速编辑走视频生成 BFF,不允许走图片模型;角色动作的 `生成动画` 仍固定使用 `seedance2.0-fast` 动作 / 视频模型,角色动作素材的 `快速编辑` 按当前帧图片走图片编辑。前端不持有 provider 密钥;上游失败或配置缺失时恢复当前生成设置面板展示失败,不创建 mock 成功图。 - 图片画布抠图统一通过唯一、只监听 loopback 的 `bgfilter-worker` 调用 BgFilter provider。手动去除背景面向用户任意图片,仍走登录态同源 BFF `POST /api/editor/images/background-removals` 和外部生成队列;API 在入队前拒绝 `data:` / `blob:` 内联媒体,父流程将稳定引用解析为当前账号已登记且归属已校验的私有 OSS object key,并在同一轮账号项目 / 素材快照读取中同时恢复用户可见源模型,禁止为 object key、所有权和源模型分别重复拉取全量快照;随后只通过一次内部 HTTP RPC 传递 object key、排队预算 `maxQueueWaitMs`、调用预算 `callBudgetMs` 和固定的 `background_mode=complex`、`seg_model=birefnet`、`cross_check=off`,不传源图字节、签名 URL、`file` 或 `screen_color`。子 worker 在每次真实 provider attempt 前签发 600 秒 URL,承担默认 `Q=2048` admission 保险丝、provider 并发 `N=16`、严格最多两次顺序 attempt、响应字节与图片尺寸校验,并把成功图片作为内部 HTTP 二进制 body 直接返回;父流程同步等待该响应且不重试已被 worker 接收的内部 RPC(连接从未建立时按调度方案 §5.1 有界重连)。排队只消耗 `maxQueueWaitMs`,取得 provider permit 后才启动 `callBudgetMs`;attempt 按 `N × est × 2`、调用预算按 `2 × attempt + 1s` 派生,冻结 `est=5000ms` 时分别为 `160s / 321s`。complex 的真实 provider 失败会累计并打开自身熔断,但与 flat 状态隔离;complex 任意失败或熔断仍直接返回父流程失败,不接入阿里云 / 本地键色降级。provider 配置继续统一使用 `GENARRATIVE_EDITOR_BGFILTER_BASE_URL` 和 `GENARRATIVE_EDITOR_BGFILTER_TOKEN`,父子共同使用 `GENARRATIVE_BGFILTER_WORKER_CONCURRENCY` 与 `GENARRATIVE_EDITOR_BGFILTER_SINGLE_IMAGE_ESTIMATE_MS` 派生预算;旧 `GENARRATIVE_EDITOR_BACKGROUND_REMOVAL_TOKEN` 只作为 provider token 兼容别名;内部调用另使用 `GENARRATIVE_BGFILTER_WORKER_BASE_URL` 和独立内部 Token。所有令牌只在服务端注入,前端不持有令牌。成功字节返回父流程后,仍由父流程完成最终处理、OSS / asset object 持久化、结果图层与最新项目快照写回;接口只向前端返回 `queueState`,有项目上下文时前端同时创建去背景生成占位并把 `canvasCompletion` 交给后端。 - 编辑器自己生成的标准纯色背景抠图资产在保存源图后统一以 `background_mode=flat` 调用内部 `bgfilter-worker`。角色形象生成、图标 spritesheet 生成、UI 设计图素材提取和角色动作的用户路径都固定使用 `screenColor=auto`,但用户可见 `generationInputs.fields` 不记录 `抠图背景色` 或 `抠图模型`;api-server 在组装 prompt 前调用背景决策模块,从 12 个候选色中选择具体 hex,最多重试 3 次,失败后兜底 `#CFEFFF`。后端仍保留手动 hex 解析能力供内部兼容。最终生图 prompt、动作视频实色背景和子 worker 发往 provider 的 `screen_color` multipart 字段只接收解析后的具体 hex,不透传 `auto`。角色、图标 spritesheet 和 UI 设计图素材提取的同源画布请求由前端自动提交默认 `segModel=birefnet`,api-server 负责 allowlist 校验并在缺失时回落默认值;角色动作逐帧去背的 `seg_model` 由后端固定。四条 flat 路径再由 api-server 向 worker 显式传递 `background_mode=flat` 与 `cross_check`:角色形象生成、图标 spritesheet 和角色动作逐帧去背传 `on`,UI 设计图素材提取传 `off`,不依赖 BgFilter 服务端默认值。前端不展示抠图模型选择,`segModel` 不进入 `generationInputs`、响应、搜索、详情或导出;`background_mode`、`cross_check` 只存在于 api-server 到 worker 的内部 RPC。子 worker 为 flat / complex 分别维护独立进程级熔断,并对一次逻辑调用严格最多执行两次顺序 provider attempt;两种模式共享 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_FAILURE_THRESHOLD=3` 和 `GENARRATIVE_EDITOR_BGFILTER_CIRCUIT_COOLDOWN_SECONDS=120` 默认值,但失败和成功只更新当前模式;父侧至多让 worker 接收一次内部 RPC(连接从未建立时按调度方案 §5.1 有界重连)。flat 两次失败、熔断、overload、内部 deadline 或断连后,只要父业务预算仍有效,父流程才按同一 object key 进入“阿里云通用抠图 → 本地 `editor_green_screen` 键色”降级;阿里云 fallback 不属于 `bgfilter-worker`。角色动作生成的序列帧背景色已与生图统一:后端把源角色图合成到视觉决策出的具体 hex 后再图生视频;抽帧后逐帧进入同一条 `内部 bgfilter-worker(background_mode=flat,cross_check=on)→ 父侧阿里云 → 父侧本地键色` 链路。 @@ -36,7 +36,7 @@ ### 静态图片风格与像素规整边界 -- 普通 `生成图片`、`生成角色形象` 和 `生成图标素材` 三个面板增加紧凑的 `像素艺术` 勾选项;移动端可独占一行,但不增加功能说明文案。当前生成对象以 `style: "none" | "pixelArt"` 保存选择并随现有请求 / 队列 payload 传递;该字段不写入用户可见 `generationInputs`,也不新增素材元数据字段。其它生成、编辑、UI 素材提取、角色动画及画布 Agent 入口不展示或设置该选项。 +- 普通 `生成图片`、`生成角色形象` 和 `生成图标素材` 三个面板增加紧凑的 `像素艺术` 勾选项;移动端可独占一行,但不增加功能说明文案。当前生成对象以 `style: "none" | "pixelArt"` 归一后同时写入请求 / 队列 payload 和 V2 `generationInputs.fields[id="style"]`,确保后续改造恢复与当次实际提交一致;不新增素材元数据字段。其它生成、编辑、UI 素材提取、角色动画及画布 Agent 入口不展示或设置该选项。 - `style` 是可选字符串兼容边界。省略、`null`、空字符串和 `"none"` 统一归一为内部 `None`,不返回告警;`"pixelArt"` 仅允许普通图片(`kind` 省略)与 `kind="character"`,图标图集请求单独允许该值。未知字符串或在 `spec / quick-edit / ui-design / publication-material` 等不支持的图片 `kind` 上请求 `"pixelArt"` 时,按 `None` 完成原管线并通过既有通用 `warning` 返回 `unsupported-image-style`;非字符串 JSON 仍是畸形请求并返回 `400`。旧 payload 缺少字段时等价于 `None`。 - `None` 必须保持现有生成、尺寸处理、BgFilter、上传、资源和画布链路不变。`PixelArt` 只增加父流程内的纯内存 Rust 后处理,不启动 Python 或独立服务,也不改变 BgFilter 的 `flat` 参数、Alpha 回贴、`cross_check`、fallback 或默认关闭 despill 的现有行为。 - 普通图片与角色在 provider 回图后先按统一业务像素矩阵尝试交付尺寸归一:允许无放大恢复时使用 Lanczos 重采样并居中裁切,无法安全恢复时保留 provider 实际尺寸并返回非阻断告警。普通图片随后以这张实际交付尺寸图同时作为网格分析源和 RGBA 采样源;角色先持久化同尺寸平底原图并交给 BgFilter,正常成功后把 Alpha 蒙版回贴到该平底原图,再以平底原图分析网格、以透明 RGBA 图采样。图标仍以已持久化的平底 provider 图尺寸为基准,BgFilter 成功并回贴 Alpha 后执行同样的双输入规整。固定首版参数为:分析色数 `16`、Alpha 覆盖阈值 `0.375`、像素格尺寸自动检测、相邻边缘峰间距使用线性插值 `P30` 估算步长、固定色板关闭、K-means 最大采样 `262144`。 @@ -77,7 +77,7 @@ - 封面图层读取与栅格化集中在 `ImageCanvasProjectCoverSnapshotRenderer`:先排除不与封面输出区相交的图层,再以最多 6 路并发加载实际可绘制图层,最终仍按 z-index 顺序绘制。封面换签必须复用画布预览的 `taskId ?? resourceId` cache version,序列帧和视频 poster 继续使用同源后缀,以命中已有 pending / 内存 / session signed-URL 缓存并尽量复用同一 OSS URL 的浏览器缓存。封面上传只需 `objectKey / assetObjectId / legacyPublicPath`,使用 object-only 上传路径,不得为从未使用的立即预览 `src` 强制 `bypassCache` 换签。 - 图片、音频、视频和角色动画帧文件本体继续走 OSS / asset object;浏览器读取私有 generated 对象统一经 `/api/assets/read-url` 换签,签名 URL 可在 session 内复用,但不得作为持久化真相。`/api/assets/read-url` 属于页面展示层高频后台请求,前端统一在 `assetReadUrlService` 内做同 key pending 去重、session 缓存和跨组件节流;UI 设计切片、角色动画帧或大量素材恢复时不得绕过该服务并发换签,否则单页可在同一秒内打满发布入口 `genarrative_api_rps` burst。 - 登录态上传和生成结果必须先落 OSS / asset object,再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/"`、`objectKey` 和 `assetObjectId`;未登录演示态可以在内存里使用 Data URL 预览,但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*`、`data:video/*`、`data:audio/*` 或 `blob:`。旧数据读取时如果已有 `objectKey`,`imageSrc` 归一成 `/`;没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。裁扩在项目上下文中虽然由前端 canvas 本地渲染 PNG,也必须先上传 OSS / asset object 并创建 `editor_project_resource`,再把带正式 `resourceId/objectKey/assetObjectId` 的裁扩图层加入画布;不能先把 `local-resource-*` + Data URL 图层交给项目保存或后续去背景。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行;没有当前工程 ID 时才创建账号级 `editor_asset` 行,随后把对应 `resourceId` 或 `assetId` 写入参考图临时状态;生成请求提交前必须把临时状态解析成 `objectKey`、项目资源 ID 或素材 ID,未登记的本地图片和普通图片路径先上传 OSS,不能直接提交 Data URL、Blob URL 或临时图片源。 -- 资源表保存资源和素材级元数据;图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组:普通图层按 `layerId/resourceId` 保存,生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存,不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON;刷新时优先从 `editor_project_resource` 恢复,旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`;角色、图标等纯色抠图生成器的前端用户路径不保存或恢复 `screenColor` / `segModel`,同源重绘也不再从 `generationInputs.fields` 恢复 `抠图背景色` 或 `抠图模型`;宣发素材生成器还必须保存并恢复 `publicationWorkflowId`、`publicationGameInfo` 和 `publicationReferences`,避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label,不保存图片 Data URL、signed URL 或 `objectKey`;刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照,只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引,只允许保存 `{ title, label, refType, refId }`;生成接口只接收提交前临时状态解析出的 `objectKey` 或资源 ID;Data URL、Blob URL 和 signed URL 不进入请求体,不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存,刷新与新建图层均按 `Resolution`(`originalWidth/originalHeight`)原分辨率显示。图层组第一版是画布内布局语义,不单独建表。 +- 资源表保存资源和素材级元数据;图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组:普通图层按 `layerId/resourceId` 保存,生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存,不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON;刷新时优先从 `editor_project_resource` 恢复,旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`;角色、图标等纯色抠图生成器的前端用户路径不保存或恢复 `screenColor` / `segModel`,同源重绘也不再从 `generationInputs.fields` 恢复 `抠图背景色` 或 `抠图模型`;宣发素材生成器还必须保存并恢复 `publicationWorkflowId`、`publicationGameInfo` 和 `publicationReferences`,避免刷新后生成卡片字段或参考图丢失。生成成功后仍保存对话框快照,只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。V2 `generationInputs` 沿用现有 JSON 列,不新增表或字段:顶层为 `{ version: 2, action, fields, references }`,`fields` 只保存 `{ id, title, value }`,其中 `value` 为 `string | number | boolean`;`references` 只保存 `{ id, title, label?, refType, refId }`,`refType="project-resource"` 指向工程资源,`refType="asset"` 指向账号素材,媒体类型从对应行查询结果获得。提交前对面板值只归一一次,同一份归一值同时进入生成请求与 `fields/references`;Data URL、Blob URL、signed URL 和 `objectKey` 均不进入元数据。图层展示尺寸不再作为独立 `Size` 真相保存,刷新与新建图层均按 `Resolution`(`originalWidth/originalHeight`)原分辨率显示。图层组第一版是画布内布局语义,不单独建表。 - 图片类、生成视频和音频结果除作为 `editor_project_resource` 和画布图层保存外,还要写入账号级 `editor_asset` 素材库;该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。角色、图标图集、UI 提取和角色动作等多产物任务把实际产生的 provider 原始输出及后处理结果分别入库:所有条目沿用 `character`、`icon-spritesheet`、`character-animation` 等真实类型,provider 原始输出承载任务模型成本,后处理派生产物阶段成本为 0。后台素材查询以最终产物为父行、每个中间产物为可展开的独立子行,分页只计算父任务;手动重拆图集保留独立 `taskId` 用于存储隔离和日志排障,通过私有 provenance 从服务端生成账号素材的 source resource、asset object 或 Object Key 取得可信来源任务,并把它写入 `groupTaskId`,不信任客户端可提交的 resource `taskId/assetKind`;跨项目复用后仍可通过稳定媒体引用找回来源。没有可信来源的新拆分显式归到自身任务,不走历史资源链回溯。每个手动切片同时写入 `groupTaskExpectedAssetCount`,全部切片落库后写独立 cohort 完成事实;后台 read model 只让同一根任务的一个已完成拆分批次并入原图集父项,用户后来删除单片不会让批次脱组,部分失败批次和后续重复拆分批次按各自真实任务分页,避免残缺批次抢占根任务、单组无限增长或素材丢失。历史行在项目资源仍存在时兼容回溯,删除项目资源前只固化直接受影响行的真实来源字段,有界展示 ID 不反写数据库。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`,前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照;`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底,不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后,右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额,不做本地乐观扣减。生成视频会单独抽取首帧封面写入 `thumbnailSrc`,素材栏和拖回画布时沿用该封面作为 poster。 - 生成面板不展示资源名称输入,默认使用原有自动编号;提示词输入保持统一可见边框。内部命名契约仍使用可选 `assetLabel`,最大 80 字符并在提交时 trim;历史状态或内部调用携带非空名称时,同一个名称必须贯穿 `assetLabel`、`canvasCompletion.title`、项目资源、账号素材和本地兜底图层,刷新后不得退回模板名。图标图集与角色动作请求同样兼容该字段,中间原图使用主名称加固定后缀;图标拆分素材按全连通域视觉顺序自动命名为 `素材 N`。 - 画布 Agent 会话按“SpacetimeDB 元数据 + OSS 消息正文”存储:`editor_agent_conversation` 只保存 `conversationId/projectId/ownerUserId/title/messagesObjectKey/deleted/createdAt/updatedAt` 等会话元数据;消息正文整体保存为私有 OSS JSON 文档 `editor-agent/{conversationId}.json`。消息文档单对象上限为 2 MiB,同一会话的消息追加和工具结果回填由 api-server 按 `conversationId` 串行化,避免“读 OSS → 改消息 → 写 OSS”并发覆盖。前端只通过 api-server BFF 读取和发送会话,不直接读写 SpacetimeDB,也不直接读写 OSS。 @@ -139,7 +139,7 @@ - 默认选择模式;底部工具栏能切换工具;中键拖拽和 Space 临时抓手都能平移画布。 - 拖拽图片或生成占位框接近其它图片 / 生成占位框边缘、中心或等距分布位置时显示吸附线,并保存吸附后的最终布局。 - 生成图片点击后显示画布内 `Image Generator` 占位框和跟随占位框的生成输入框,生成失败保留占位和输入状态,生成成功后在占位位置创建真实图层,并让输入框继续跟随该生成图。 -- 选择 `1K / 2K` 或切换比例后,占位框在待生成和生成中阶段都必须立即显示对应目标像素尺寸;从普通图片、角色、图标图集或 UI 设计图再次改造时同样适用,完成落图前后不得从默认 1K 框跳变为 2K 成品。 +- 选择 `1K / 2K` 或切换比例后,占位框在待生成和生成中阶段都必须立即显示对应目标像素尺寸;从普通图片、角色、图标图集或 UI 设计图进入改造时同样适用,完成落图前后不得从默认 1K 框跳变为 2K 成品。 - 普通图片、角色和图标面板显示 `像素艺术` 勾选项并正确提交 / 恢复 `style: "none" | "pixelArt"`;其它生成或编辑面板不显示该选项。旧 payload、未知字符串、不支持 `kind` 和非字符串输入分别按本方案约定的兼容或错误语义处理。 - `pixelArt` 输出 Alpha 只包含 `0 / 255`;普通图片和角色先完成 Lanczos 交付尺寸归一,再由 snapper 使用 nearest 把逻辑网格恢复到同一输入尺寸,规整后不得再次执行尺寸插值。成功和后处理失败两条路径都不得比 `none` 增加 OSS PUT、项目资源、账号素材或画布 item,逻辑低分辨率图不得出现在 OSS 或响应资源快照中。 - 生成中的占位图聚焦后支持键盘 `Delete` / `Backspace` 删除,不新增可见删除按钮;删除后对应异步回写必须按生成器 ID 判空并丢弃,不能把已删除素材重新落回画布。音乐 / 音频生成占位和已生成音频图层同样必须支持键盘删除。 diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index 61ed7e2aa..cd32e9bfe 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -620,6 +620,7 @@ Responses 的终态载荷既是工具调用的恢复源,也是正文的恢复 - 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs` - 说明:图片画布工程资源元数据表,保存已经放入某个 project 画布的上传 / 生成图片资源快照、OSS 引用、尺寸、来源类型、prompt、provider、task、源资源关系、`asset_kind`、`generation_inputs_json` 和历史 `public_showcase_enabled`。`asset_kind` 标记角色、图标、UI 设计图、视频、音频等素材类别;`generation_inputs_json` 保存用户可见生成输入快照,供图片信息页刷新后恢复。`public_showcase_enabled` 只保留旧接口兼容,不再作为 `/creation` 的 `陶泥儿精选` 事实源;精选公开改由账号级生成素材提交 `editor_showcase_asset` 审核决定。图片 / 图标 / UI 提取等生成 BFF 在请求携带 `project_id` 时负责创建该表记录并把 resource 快照返回前端;前端只保存稳定 `resource_id` 布局引用,不能把同一生成结果再次作为正式业务真相写入。项目封面快照也落在该表,使用 `asset_kind = project-cover-snapshot`、`source_type = uploaded` 和私有 OSS / asset object 引用,代表画布当前视口栅格化后的静态封面;项目列表和创作主页最近项目只读取最新封面快照资源,不在列表页根据 layout 临时拼画布。从账号级素材库把同一生成素材拖回同一项目画布时,后端优先复用同项目内同源同媒体资源,避免每个图层实例都插入新的资源行。账号级素材删除不级联删除该表,避免历史画布丢图。结构化 canvas 的几何、层级、分组和资源引用以 `editor_canvas_layer` 为权威,生成器对象以 `editor_canvas_generation_dialog` 为权威;legacy canvas 才在 2 MiB 上限内从 `editor_canvas.layers_json` 兼容读取。新写入不再把素材生成输入快照作为图层布局真相保存。历史普通图层缺资源只能由 migration operator 调用 `repair_editor_canvas_resources_and_return` 定向修复:procedure 每次只处理一个尚无迁移记录的 legacy canvas,校验 owner/project、revision、canvas/project 两份 raw layout SHA-256、精确 layer/resource/sourceResourceId、同工程替换资源与 private asset_object 谱系;图片只替换引用,音频只恢复经核验的 `420x120` 项目资源行。运维入口 `npm run spacetime:editor-canvas-resources:repair` 默认 dry-run,apply 必须绑定 plan SHA-256 并在成功后自动复核 already-repaired,禁止手工 SQL 绕过事务 guard。 - `generation_inputs_json` 包络契约:`fields` / `references` 是图片信息读取的用户可见生成输入快照;顶层允许保存后端内部结果扩展。现有 `screenColorHex` 保存实际背景色,角色、图标图集和 UI 图集抠图派生资产使用 `mattingProvider` / `mattingModel` 保存实际成功的处理后端与模型。BgFilter 保存本次 `seg_model`,阿里云通用抠图保存 `Aliyun Matting / segment-common-image`,本地键色保存 `Genarrative Local / screen-color-keying`。同源画布 BFF 的角色、图标和 UI 请求由前端自动提交 `screenColor=auto` 与默认 `segModel=birefnet`,其中 `segModel` 是不可由用户选择的请求控制字段,不进入 `generationInputs`;`background_mode` 和 `cross_check` 只属于 api-server 到 worker 的内部 RPC。External OpenAPI 不开放 `segModel`。上述内部结果字段不写入 `fields`,普通用户(包括素材 owner)与匿名公开读取均不得取得;普通用户响应还必须省略素材顶层 `provider` 和内部处理 `model`,但保留正常用户可见 `model` 与其他合法的顶层功能字段。后台管理和服务端审计可读取原始值。过滤只作用于普通用户 / 公开响应边界,不修改素材或精选快照,因此历史数据无需迁移。 +- `generation_inputs_json` V2 可执行改造契约沿用现有 JSON 列,无 SpacetimeDB schema 迁移或存量回填:顶层 `version=2` 和稳定 `action` 确定生成器,`fields[].id` 确定参数,`references[].id/refType/refId` 确定引用参数与资源;`fields[].value` 保持 `string | number | boolean` 类型,`title` / `label` 只作展示。引用媒体类型从 `refType/refId` 对应的工程资源或账号素材查询结果获得,不在快照内重复保存。Owner resource / asset payload 在普通用户元数据清理后保留这些执行字段;匿名公开素材 payload 暂不返回 `generationInputs`,避免公开接口沿用 owner 可执行配方 DTO。独立裁扩、手动去背景和手动图集拆分是确定性派生操作,新结果 `generation_inputs_json = null`;原生成任务内的自动透明化 / 拆分后处理可保留同任务的原生成输入。 - 普通用户生成结果契约:图片、图标图集、视频、音频和角色动画的完成响应与新建画布图层均不返回或写入生成 provider;项目资源、素材、精选和 Agent 紧凑结果使用同一读取边界。真实 provider 只保留在持久化、tracking / tracing 和后台管理原始审计中。该规则针对生成供应商元数据,不改变直传票据等必须由客户端执行的存储协议字段。 - 普通用户错误契约:手动去背景和角色动作透明化的原始服务端错误可能包含 BgFilter、分割模型或 provider 细节;Owner HTTP 响应与外部任务状态必须按 job kind 返回稳定业务文案,原始错误只保留在任务记录、tracing 与后台审计。 - 索引:`by_editor_project_resource_project_id`、`by_editor_project_resource_owner_user_id`。 diff --git a/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md b/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md index 7daef9a08..fc1b766cb 100644 --- a/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md +++ b/docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md @@ -112,8 +112,8 @@ - 视频 / 角色 / 角色动作 / 音效 / 背景音乐待生成占位的角标同样按 viewport 反向缩放,不随画布缩放变小。 - 已生成角色图、角色动作图或其它生成结果图被点击时只选中图层并收起已有生成输入框;重绘、快速编辑和生成动画面板必须由对应工具栏按钮或右键菜单显式打开。 - 角色图层打开“生成动作”后再点击“改造”,必须重新打开角色形象生成器;动作生成对话框只把角色图层作为输入来源,不得被识别为该角色图层自身的来源生成器。 -- 角色动作结果图层点击“改造”时,必须通过 `sourceResourceId` 找回原角色图层并重新打开角色动作生成器;关联原角色已不存在时应显示明确提示,不得无响应或降级成图片生成器。 -- 恢复已生成图层的来源生成器时,角色、动作、规范、图标、UI、宣发、视频、音效和背景音乐必须按 `assetKind / mediaType` 恢复对应面板;去背景或快速编辑产生的派生 `quick-edit` 对话框不得遮住图层原有来源生成器。 +- 角色动作结果图层点击“改造”时,V2 必须通过 `references[id="source"]` 找回原角色图层,legacy 数据才允许以 `sourceResourceId` 回退;关联原角色已不存在时应显示明确提示,不得无响应或降级成图片生成器。 +- `改造` 覆盖图片、规范、角色、图标、UI、宣发、视频、音效、背景音乐、角色动作和生成型图片编辑。有效 V2 只按 `action + fields[].id + references[].id` 恢复;历史对话框和 legacy 数据保留 `assetKind/mediaType`、标题别名、资源尺寸 / 模型 / 时长 / `sourceResourceId` 回退,并对默认值恢复显示告警。独立非生成型裁扩、手动去背景和手动图集拆分结果不继承 `generationInputs`,不显示该动作;原生成任务内自动后处理的同源产物可保留原生成输入。 - 任何会移除画布图层的入口,包括删除、右键剪切和素材库删除关联素材,都必须同步清理该图层关联的生成面板和派生状态,不得在保存或刷新后恢复成孤立占位。 ## 画布保存 diff --git a/docs/【编辑器】画板音乐生成入口设计-2026-06-18.md b/docs/【编辑器】画板音乐生成入口设计-2026-06-18.md index 0572907cc..7108785f2 100644 --- a/docs/【编辑器】画板音乐生成入口设计-2026-06-18.md +++ b/docs/【编辑器】画板音乐生成入口设计-2026-06-18.md @@ -52,7 +52,7 @@ - 音频元数据弹窗使用 `时长`,不使用图片 / 视频的分辨率语义;音频生成占位不显示分辨率或时长角标。 - 音频图层右上角标签显示在信息按钮左侧,和其他素材卡右上角信息区保持一致。 - 元数据弹窗按音频显示 `音频信息` / `音频类型` / `时长`,生成输入快照只展示用户面板字段。 -- 音频图层上方浮动工具栏只保留 `改造` 和 `下载按钮`。点击 `改造` 后打开对应的音效或背景音乐生成面板,不展示参考图组件;面板底部模型与参数位置和原生成入口一致,并允许继续修改后再次生成,新结果落在原音频旁边。 +- 音频图层上方浮动工具栏只保留 `改造` 和 `下载按钮`。只有当前 owner 的可执行生成输入存在时才显示 `改造`;点击后按 `action` 恢复对应的音效或背景音乐生成面板,音效再按 `fields[id="model"]` 和 `fields[id="durationSeconds"]` 恢复模型与时长;面板不展示参考图组件,允许修改后生成新音频,新结果落在原音频旁边。 ## 前端提交契约 @@ -131,6 +131,6 @@ POST /api/editor/audios/background-music/generations - 成功后的音频卡展示提示词;卡片中央按未悬停图标、悬停播放、播放中暂停切换,不在底部重复显示播放 / 暂停按钮,播放器辅助控件直接融入卡片底部。 - 鼠标未悬停音频卡时底部播放器辅助控件不可见,悬停时从底部滑动显现;提示词位于卡片左上角。 - 音频卡片、悬停角标和待生成占位不显示时长;信息弹窗显示时长。 -- 音频素材浮动工具栏只显示 `改造` 与 `下载按钮`,`改造` 复用对应生成面板且没有参考图组件。 +- 音频素材浮动工具栏在存在可执行生成输入时显示 `改造` 与 `下载按钮`,否则只显示下载;`改造` 恢复对应生成面板且没有参考图组件。 - 私有 generated 音频能先换签再预览播放,不出现播放条一直为 `0:00` 的裸路径失败状态。 - 刷新后 layout 能恢复音频生成器和音频图层。 diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 9457a2004..efff1c7cb 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -919,6 +919,8 @@ pub struct EditorProjectResourcePayload { asset_kind: Option, #[serde(skip_serializing_if = "Option::is_none")] showcase_category: Option, + // TODO: generation_inputs V2 的 action 暂以字符串保存在 JSON 中;待 shared-contracts 提供 Rust DTO 后收口为共享 enum。 + #[serde(skip_serializing_if = "Option::is_none")] generation_inputs: Option, public_showcase_enabled: bool, #[serde(skip_serializing_if = "Option::is_none")] @@ -5502,7 +5504,8 @@ pub async fn split_editor_icon_spritesheet( .provider .clone() .unwrap_or_else(|| "Genarrative".to_string()); - let generation_inputs = source_resource.generation_inputs.clone(); + // 手动拆图是确定性派生操作,切片不得继承源图集的生成输入,避免错误展示“改造”。 + let generation_inputs = None; let icon_image_srcs = persist_editor_spritesheet_slices( &state, slices, @@ -6764,6 +6767,7 @@ fn editor_project_resource_public_payload_from_record( resolve_editor_project_resource_public_author(state, payload.owner_user_id.as_str()); payload.author_display_name = display_name; payload.author_public_user_code = public_user_code; + payload.generation_inputs = None; payload } @@ -9952,14 +9956,16 @@ mod tests { } #[test] - fn editor_project_resource_public_payload_removes_internal_generation_metadata() { + fn editor_project_resource_public_payload_omits_generation_inputs() { let state = AppState::new(AppConfig::default()).expect("state should build"); let record = EditorShowcaseAssetRecord { model: Some("BgFilter complex".to_string()), provider: Some("BgFilter".to_string()), generation_inputs: Some(json!({ - "fields": [{"title": "处理阶段", "value": "角色抠图"}], - "references": [{"refType": "asset", "refId": "asset-source"}], + "version": 2, + "action": "image.generate", + "fields": [{"id": "prompt", "title": "生成提示词", "value": "角色立绘"}], + "references": [{"id": "reference", "label": "角色参考", "refType": "asset", "refId": "asset-source"}], "characterAnimation": {"action": "idle"}, "screenColorHex": "#CFEFFF", "mattingProvider": "BgFilter", @@ -9980,6 +9986,13 @@ mod tests { assert!(!owner_generation_inputs.contains_key("screenColorHex")); assert!(!owner_generation_inputs.contains_key("mattingProvider")); assert!(!owner_generation_inputs.contains_key("mattingModel")); + assert_eq!(owner_generation_inputs["version"], json!(2)); + assert_eq!(owner_generation_inputs["action"], json!("image.generate")); + assert_eq!(owner_generation_inputs["fields"][0]["id"], json!("prompt")); + assert_eq!( + owner_generation_inputs["references"][0]["id"], + json!("reference") + ); assert_eq!( owner_generation_inputs["characterAnimation"], json!({"action": "idle"}) @@ -9991,24 +10004,7 @@ mod tests { .expect("public payload should serialize"); assert!(public_payload.get("provider").is_none()); assert!(public_payload.get("model").is_none()); - let generation_inputs = public_payload["generationInputs"] - .as_object() - .expect("generation inputs should stay object"); - assert!(!generation_inputs.contains_key("screenColorHex")); - assert!(!generation_inputs.contains_key("mattingProvider")); - assert!(!generation_inputs.contains_key("mattingModel")); - assert_eq!( - generation_inputs["fields"], - json!([{"title": "处理阶段", "value": "角色抠图"}]) - ); - assert_eq!( - generation_inputs["references"], - json!([{"refType": "asset", "refId": "asset-source"}]) - ); - assert_eq!( - generation_inputs["characterAnimation"], - json!({"action": "idle"}) - ); + assert!(public_payload.get("generationInputs").is_none()); } #[test] diff --git a/src/components/creation-home/creationShowcaseModel.ts b/src/components/creation-home/creationShowcaseModel.ts index 259350faf..1573caaff 100644 --- a/src/components/creation-home/creationShowcaseModel.ts +++ b/src/components/creation-home/creationShowcaseModel.ts @@ -646,7 +646,7 @@ function getGroupsForTab(assets: EditorAssetSnapshot[], tab: ShowcaseTabId) { function resolveUserGenerationPrompt(asset: EditorAssetSnapshot) { const fields = asset.generationInputs?.fields ?? []; const promptField = fields.find((field) => { - const value = field.value.trim(); + const value = String(field.value).trim(); return ( value && USER_PROMPT_INPUT_TITLES.has( @@ -655,7 +655,7 @@ function resolveUserGenerationPrompt(asset: EditorAssetSnapshot) { !isBuiltInPromptText(value) ); }); - const promptValue = promptField?.value.trim(); + const promptValue = promptField ? String(promptField.value).trim() : ''; if (promptValue) { return promptValue; } @@ -668,7 +668,7 @@ function resolveUserGenerationPrompt(asset: EditorAssetSnapshot) { ) { return []; } - const value = field.value.trim(); + const value = String(field.value).trim(); return value && !isBuiltInPromptText(value) ? [`${field.title.trim()}:${value}`] : []; @@ -824,7 +824,7 @@ function inferVideoResolution( const fields = asset.generationInputs?.fields ?? []; const fieldValue = fields .flatMap((field) => [field.title, field.value]) - .map((value) => value.trim()) + .map((value) => String(value).trim()) .find((value) => /\b(?:480p|720p|1080p)\b/iu.test(value)); const matchedValue = fieldValue?.match(/\b(480p|720p|1080p)\b/iu)?.[1]; if (matchedValue && allowedValues.includes(matchedValue)) { diff --git a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx index c307a36ee..9cc473c9a 100644 --- a/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx @@ -874,8 +874,14 @@ describe('ImageCanvasEditorView generation integration', () => { }); const submittedGenerationInputs = generateEditorImageMock.mock.calls[0]?.[0]?.generationInputs; - expect(JSON.stringify(submittedGenerationInputs)).not.toContain( - 'pixelArt', + expect(submittedGenerationInputs).toEqual( + expect.objectContaining({ + version: 2, + action: 'image.generate', + fields: expect.arrayContaining([ + expect.objectContaining({ id: 'style', value: 'pixelArt' }), + ]), + }), ); await waitFor(() => { @@ -2485,18 +2491,23 @@ describe('ImageCanvasEditorView generation integration', () => { const characterGenerationInput = generateEditorImageMock.mock.calls[0]?.[0]; expect(characterGenerationInput?.generationInputs).toEqual( expect.objectContaining({ + version: 2, + action: 'character.generate', fields: expect.arrayContaining([ expect.objectContaining({ + id: 'prompt', title: '角色设定', value: '银发游侠,蓝色披风,弓箭手,适合像素风战棋。', }), ]), references: expect.arrayContaining([ expect.objectContaining({ + id: 'specReference', title: '角色规范', label: '角色规范', }), expect.objectContaining({ + id: 'reference', title: '常规参考图 1', label: '大鱼素材', }), @@ -2664,8 +2675,11 @@ describe('ImageCanvasEditorView generation integration', () => { projectId: 'editor-project-icons', assetFolderId: 'project', generationInputs: expect.objectContaining({ + version: 2, + action: 'icon.generate', fields: expect.arrayContaining([ expect.objectContaining({ + id: 'prompt', title: '素材描述', value: '返回按钮\n设置按钮', }), @@ -3255,7 +3269,6 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByText('角色')).toBeTruthy(); expect(screen.getByRole('button', { name: '生成动画' })).toBeTruthy(); expect(screen.queryByRole('dialog', { name: '重绘图片' })).toBeNull(); - expect(screen.queryByRole('dialog', { name: '改造图片' })).toBeNull(); expect( screen.queryByRole('dialog', { name: '角色动画生成面板' }), ).toBeNull(); @@ -3266,7 +3279,6 @@ describe('ImageCanvasEditorView generation integration', () => { expect(screen.getByRole('menuitem', { name: '生成动画' })).toBeTruthy(); expect(screen.queryByRole('dialog', { name: '重绘图片' })).toBeNull(); - expect(screen.queryByRole('dialog', { name: '改造图片' })).toBeNull(); fireEvent.click(screen.getByRole('button', { name: '生成动画' })); expect( @@ -3536,8 +3548,11 @@ describe('ImageCanvasEditorView generation integration', () => { assetFolderId: 'project', spritesheetLabel: '战斗UI设计图 素材图集', generationInputs: expect.objectContaining({ + version: 2, + action: 'ui-design.extract-assets', fields: expect.arrayContaining([ expect.objectContaining({ + id: 'prompt', title: '提取提示词', value: '仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用所选单一纯色抠图背景。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具,方便后续扣除背景;素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。', @@ -3545,6 +3560,7 @@ describe('ImageCanvasEditorView generation integration', () => { ]), references: expect.arrayContaining([ expect.objectContaining({ + id: 'source', title: 'UI设计图', label: '战斗UI设计图', }), diff --git a/src/components/image-editor/ImageCanvasEditorTypes.ts b/src/components/image-editor/ImageCanvasEditorTypes.ts index 64b479ef9..dad90a50a 100644 --- a/src/components/image-editor/ImageCanvasEditorTypes.ts +++ b/src/components/image-editor/ImageCanvasEditorTypes.ts @@ -64,19 +64,39 @@ export type EditorAsset = { uploadMessage?: string; }; +export type CanvasGenerationAction = + | 'image.generate' + | 'spec.generate' + | 'character.generate' + | 'icon.generate' + | 'ui-design.generate' + | 'publication.generate' + | 'video.generate' + | 'audio.sound-effect.generate' + | 'audio.background-music.generate' + | 'character-animation.generate' + | 'image.edit' + | 'ui-design.extract-assets'; + +export type CanvasGenerationInputValue = string | number | boolean; + export type CanvasGenerationInputField = { + id?: string; title: string; - value: string; + value: CanvasGenerationInputValue; }; export type CanvasGenerationInputReference = { + id?: string; title: string; - label: string; + label?: string; refType: 'project-resource' | 'asset'; refId: string; }; export type CanvasGenerationInputs = { + version?: 2; + action?: CanvasGenerationAction; fields: CanvasGenerationInputField[]; references: CanvasGenerationInputReference[]; }; @@ -201,6 +221,7 @@ export type GenerateDialogState = { assetLabel?: string; status: 'idle' | 'generating' | 'failed'; composerOpen?: boolean; + isRedrawAsNewArtifact?: boolean; sourceLayerId?: string; generatedLayerId?: string; specType?: SpecGenerationType; diff --git a/src/components/image-editor/ImageCanvasEditorView.test.tsx b/src/components/image-editor/ImageCanvasEditorView.test.tsx index 0182d2329..ec363fb02 100644 --- a/src/components/image-editor/ImageCanvasEditorView.test.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.test.tsx @@ -1278,7 +1278,9 @@ describe('ImageCanvasEditorView', () => { expect( within(infoPanel).queryByRole('button', { name: '复制Prompt' }), ).toBeNull(); - expect(screen.getByRole('button', { name: '改造' })).toBeTruthy(); + expect( + screen.queryByRole('button', { name: '改造' }), + ).toBeNull(); }); it('hydrates canvas images from Resolution instead of saved Size', async () => { @@ -1794,7 +1796,9 @@ describe('ImageCanvasEditorView', () => { expect( screen.getByRole('button', { name: '查看大鱼素材图片信息' }), ).toBeTruthy(); - expect(screen.getByRole('button', { name: '改造' })).toBeTruthy(); + expect( + screen.queryByRole('button', { name: '改造' }), + ).toBeNull(); fireEvent.click(screen.getByRole('button', { name: '打开素材' })); diff --git a/src/components/image-editor/ImageCanvasEditorView.tsx b/src/components/image-editor/ImageCanvasEditorView.tsx index e6f8a7236..1ddb6355e 100644 --- a/src/components/image-editor/ImageCanvasEditorView.tsx +++ b/src/components/image-editor/ImageCanvasEditorView.tsx @@ -1737,6 +1737,7 @@ export function ImageCanvasEditorView({ layersRef.current, layer, ), + availableLayers: layersRef.current, }); if (!restoredDraft) { return false; diff --git a/src/components/image-editor/ImageCanvasExportModel.ts b/src/components/image-editor/ImageCanvasExportModel.ts index 521ca5467..db5eb0cd4 100644 --- a/src/components/image-editor/ImageCanvasExportModel.ts +++ b/src/components/image-editor/ImageCanvasExportModel.ts @@ -8,6 +8,7 @@ import type { import { DEFAULT_ICON_DESCRIPTIONS, formatLayerImageType, + formatGenerationInputValue, getEditorLayerModelDisplayName, isEditorUserVisibleGenerationInputField, UI_DESIGN_ASSET_EXTRACTION_PROMPT, @@ -540,7 +541,7 @@ const BUILT_IN_GENERATION_INPUT_VALUES_BY_TITLE = new Map( function isBuiltInGenerationInputField(field: CanvasGenerationInputField) { const normalizedTitle = field.title.trim(); - const normalizedValue = field.value.trim(); + const normalizedValue = formatGenerationInputValue(field.value).trim(); const builtInValues = BUILT_IN_GENERATION_INPUT_VALUES_BY_TITLE.get(normalizedTitle); return ( @@ -557,12 +558,12 @@ function buildVisibleGenerationInputs(layer: CanvasLayer) { .filter((field) => !isBuiltInGenerationInputField(field)) .map((field) => ({ title: field.title, - value: field.value, + value: formatGenerationInputValue(field.value), })) ?? []; const references = layer.generationInputs?.references.map((reference) => ({ title: reference.title, - label: reference.label, + label: reference.label ?? '参考素材', refType: reference.refType, refId: reference.refId, })) ?? []; diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts index c568371b7..5523813fd 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts @@ -825,6 +825,191 @@ describe('ImageCanvasGenerationDialogModel', () => { }); }); + it('restores V2 video and sound parameters by stable ids instead of display labels', () => { + const imageReference = createLayer({ + id: 'layer-image-reference', + resourceId: 'resource-image-reference', + title: '视觉参考', + }); + const audioReference = createLayer({ + id: 'layer-audio-reference', + resourceId: 'resource-audio-reference', + title: '音轨参考', + mediaType: 'audio', + }); + const context = { + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + availableLayers: [imageReference, audioReference], + }; + + expect( + createLayerGenerationDialogDraft({ + ...context, + sourceLayer: createLayer({ + sourceType: 'generated', + mediaType: 'video', + generationInputs: { + version: 2, + action: 'video.generate', + fields: [ + { id: 'prompt', title: '可随时改名', value: '角色追逐镜头' }, + { id: 'model', title: '不用作键', value: 'seedance2.0-fast' }, + { id: 'aspectRatio', title: '显示快照', value: '9:16' }, + { id: 'resolution', title: '显示快照', value: '1080p' }, + { id: 'durationSeconds', title: '显示快照', value: 8 }, + { id: 'sound', title: '显示快照', value: 'off' }, + ], + references: [ + { + id: 'imageReference', + title: '任意标题', + label: '旧显示名', + refType: 'project-resource', + refId: 'resource-image-reference', + }, + { + id: 'audioReference', + title: '任意标题', + label: '旧显示名', + refType: 'project-resource', + refId: 'resource-audio-reference', + }, + ], + }, + }), + }), + ).toMatchObject({ + mode: 'video', + prompt: '角色追逐镜头', + videoModel: 'seedance2.0-fast', + videoAspectRatio: '9:16', + videoResolution: '1080p', + videoDurationSeconds: 8, + videoSound: 'off', + generationReferences: [ + expect.objectContaining({ + id: 'canvas-layer-image-reference', + label: '视觉参考', + }), + expect.objectContaining({ + id: 'canvas-layer-audio-reference', + label: '音轨参考', + mediaType: 'audio', + }), + ], + }); + + expect( + createLayerGenerationDialogDraft({ + ...context, + sourceLayer: createLayer({ + sourceType: 'generated', + mediaType: 'audio', + generationInputs: { + version: 2, + action: 'audio.sound-effect.generate', + fields: [ + { id: 'prompt', title: '非 prompt 标题', value: '短促点击音' }, + { id: 'model', title: '非 model 标题', value: 'audio1.0' }, + { id: 'durationSeconds', title: '非 duration 标题', value: 7 }, + ], + references: [], + }, + }), + }), + ).toMatchObject({ + mode: 'audio-sound-effect', + prompt: '短促点击音', + soundModel: 'audio1.0', + soundDurationSeconds: 7, + }); + }); + + it('keeps valid V2 fields ahead of conflicting legacy dialog and resource fallbacks', () => { + expect( + createLayerGenerationDialogDraft({ + sourceLayer: createLayer({ + sourceType: 'generated', + prompt: '资源旧提示词', + generationInputs: { + version: 2, + action: 'image.generate', + fields: [ + { id: 'model', title: '任意标题', value: IMAGE_MODEL_GPT_IMAGE_2 }, + { id: 'aspectRatio', title: '任意标题', value: '2:3' }, + { id: 'imageSize', title: '任意标题', value: '2K' }, + ], + references: [], + }, + }), + sourceDialog: { + id: 'legacy-dialog', + mode: 'generate', + prompt: '历史对话框提示词', + status: 'idle', + imageModel: IMAGE_MODEL_NANOBANANA2, + aspectRatio: '1:1', + imageSize: '1K', + }, + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + }), + ).toMatchObject({ + mode: 'generate', + prompt: '', + imageModel: IMAGE_MODEL_GPT_IMAGE_2, + aspectRatio: '2:3', + imageSize: '2K', + }); + }); + + it('uses the V2 source reference ahead of a conflicting legacy animation source', () => { + const v2Source = createLayer({ + id: 'layer-v2-character', + resourceId: 'resource-v2-character', + assetKind: 'character', + }); + const legacySource = createLayer({ + id: 'layer-legacy-character', + resourceId: 'resource-legacy-character', + assetKind: 'character', + }); + + expect( + createSameSourceGenerationDialogDraft({ + sourceLayer: createLayer({ + id: 'layer-animation-result', + sourceType: 'generated', + assetKind: 'character-animation', + sourceResourceId: 'resource-legacy-character', + generationInputs: { + version: 2, + action: 'character-animation.generate', + fields: [{ id: 'prompt', title: '任意标题', value: '循环奔跑' }], + references: [ + { + id: 'source', + title: '任意标题', + refType: 'project-resource', + refId: 'resource-v2-character', + }, + ], + }, + }), + canvasSize: { width: 960, height: 720 }, + viewport: { x: 0, y: 0, scale: 1 }, + mode: 'redraw', + sourceAnimationLayer: legacySource, + availableLayers: [legacySource, v2Source], + }), + ).toMatchObject({ + mode: 'character-animation', + prompt: '循环奔跑', + sourceLayerId: 'layer-v2-character', + }); + }); + it('inherits and normalizes quick edit model parameters from the source', () => { expect( createQuickEditPanelDraft( diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts index 6c8019b3b..983932734 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts @@ -30,12 +30,15 @@ import { DEFAULT_VIDEO_WEB_SEARCH_ENABLED, EDITOR_IMAGE_DIMENSION_OPTIONS, EDITOR_IMAGE_MODEL_OPTIONS, + formatGenerationInputValue, IMAGE_MODEL_GPT_IMAGE_2, inferEditorImageAspectRatio, inferEditorImageSizeLabel, + isNormalizedCanvasGenerationInputs, normalizeEditorImageModel, PUBLICATION_FRAME_ORIGINAL_SIZE, resizeGenerationPlaceholderToImageSelection, + resizeGenerationPlaceholderToVideoSelection, resolveEditorImageGenerationPixelSize, resolveEditorVideoGenerationPixelSize, SPEC_FRAME_ORIGINAL_SIZE, @@ -50,6 +53,7 @@ type SourceGenerationDialogDraftContext = { sourceDialog?: CanvasGenerationDialogState | null; mode: 'quick-edit' | 'redraw'; sourceAnimationLayer?: CanvasLayer | null; + availableLayers?: CanvasLayer[]; }; const VIDEO_REFERENCE_LIMITS = { image: 9, @@ -112,7 +116,8 @@ export function isIconSpecReferenceLayer(layer: CanvasLayer) { layer.title.includes('UI素材规范') || layer.generationInputs?.fields.some( (field) => - field.value.includes('游戏UI规范') || field.value.includes('图标规范'), + formatGenerationInputValue(field.value).includes('游戏UI规范') || + formatGenerationInputValue(field.value).includes('图标规范'), ) === true ); } @@ -156,7 +161,7 @@ function getGenerationInputFieldValues(layer: CanvasLayer) { const values = new Map(); for (const field of layer.generationInputs?.fields ?? []) { const title = field.title.trim().toLowerCase(); - const value = field.value.trim(); + const value = formatGenerationInputValue(field.value).trim(); if (title && value && !values.has(title)) { values.set(title, value); } @@ -440,11 +445,10 @@ function findGenerationInputFieldValue( titles: readonly string[], ) { const normalizedTitles = new Set(titles.map((title) => title.toLowerCase())); - return ( - layer.generationInputs?.fields.find((field) => - normalizedTitles.has(field.title.trim().toLowerCase()), - )?.value ?? null + const field = layer.generationInputs?.fields.find((candidate) => + normalizedTitles.has(candidate.title.trim().toLowerCase()), ); + return field ? formatGenerationInputValue(field.value) : null; } const USER_PROMPT_INPUT_TITLES = new Set( @@ -484,7 +488,9 @@ function resolveUserGenerationPromptSnapshot(sourceLayer: CanvasLayer) { const promptField = fields.find((field) => USER_PROMPT_INPUT_TITLES.has(field.title.trim().toLowerCase()), ); - const promptValue = promptField?.value.trim(); + const promptValue = promptField + ? formatGenerationInputValue(promptField.value).trim() + : undefined; if (promptValue) { return promptValue; } @@ -493,7 +499,7 @@ function resolveUserGenerationPromptSnapshot(sourceLayer: CanvasLayer) { if (!STRUCTURED_USER_INPUT_TITLES.has(field.title.trim().toLowerCase())) { return []; } - const value = field.value.trim(); + const value = formatGenerationInputValue(field.value).trim(); return value ? [`${field.title.trim()}:${value}`] : []; }); return structuredLines.join('\n'); @@ -682,6 +688,7 @@ export function createLayerGenerationDialogDraft({ viewport, sourceDialog, sourceAnimationLayer, + availableLayers, }: Omit): Omit< CanvasGenerationDialogState, 'id' @@ -692,15 +699,23 @@ export function createLayerGenerationDialogDraft({ viewport, sourceDialog, sourceAnimationLayer, + availableLayers, mode: 'redraw', }); if (!draft) { return null; } + const usesNormalizedInputs = isNormalizedCanvasGenerationInputs( + sourceLayer.generationInputs, + ); return { ...draft, - prompt: draft.prompt || sourceLayer.prompt?.trim() || '', - imageModel: sourceDialog?.imageModel ?? draft.imageModel, + prompt: usesNormalizedInputs + ? draft.prompt + : draft.prompt || sourceLayer.prompt?.trim() || '', + imageModel: usesNormalizedInputs + ? draft.imageModel + : sourceDialog?.imageModel ?? draft.imageModel, status: 'idle', composerOpen: true, generatedLayerId: sourceLayer.id, @@ -743,6 +758,390 @@ function placeDraftBesideSourceLayer< }; } +function getNormalizedFieldValue(sourceLayer: CanvasLayer, id: string) { + return sourceLayer.generationInputs?.fields.find((field) => field.id === id) + ?.value; +} + +function getNormalizedString( + sourceLayer: CanvasLayer, + id: string, + fallback = '', +) { + const value = getNormalizedFieldValue(sourceLayer, id); + return typeof value === 'string' ? value : fallback; +} + +function getNormalizedNumber( + sourceLayer: CanvasLayer, + id: string, + fallback: number, +) { + const value = getNormalizedFieldValue(sourceLayer, id); + return typeof value === 'number' && Number.isFinite(value) ? value : fallback; +} + +function resolveNormalizedReferences( + sourceLayer: CanvasLayer, + id: string, + availableLayers: CanvasLayer[] | undefined, +) { + const layers = availableLayers ?? []; + return (sourceLayer.generationInputs?.references ?? []) + .filter((reference) => reference.id === id) + .flatMap((reference) => { + const layer = layers.find((candidate) => + reference.refType === 'project-resource' + ? candidate.resourceId === reference.refId + : candidate.sourceAssetId === reference.refId, + ); + return layer ? [createCanvasLayerReference(layer)] : []; + }); +} + +function createNormalizedGenerationDialogDraft({ + sourceLayer, + canvasSize, + viewport, + availableLayers, +}: SourceGenerationDialogDraftContext): Omit< + CanvasGenerationDialogState, + 'id' +> | null { + const inputs = sourceLayer.generationInputs; + if (!isNormalizedCanvasGenerationInputs(inputs)) { + return null; + } + const prompt = getNormalizedString(sourceLayer, 'prompt'); + const references = resolveNormalizedReferences( + sourceLayer, + 'reference', + availableLayers, + ); + + if (inputs.action === 'image.generate') { + return placeDraftBesideSourceLayer( + resizeGenerationPlaceholderToImageSelection({ + ...createGenerateDialogDraft({ canvasSize, viewport }), + prompt, + sourceLayerId: sourceLayer.id, + imageModel: getNormalizedString( + sourceLayer, + 'model', + DEFAULT_IMAGE_MODEL, + ), + style: + getNormalizedString(sourceLayer, 'style') === 'pixelArt' + ? 'pixelArt' + : 'none', + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + generationReferences: references, + }), + sourceLayer, + ); + } + + if (inputs.action === 'spec.generate') { + const specTypeValue = getNormalizedString(sourceLayer, 'specType', 'custom'); + const specType: SpecGenerationType = + specTypeValue === 'character' || + specTypeValue === 'ui' || + specTypeValue === 'icon' + ? specTypeValue + : 'custom'; + return placeDraftBesideSourceLayer( + { + ...createSpecDialogDraft({ canvasSize, viewport, specType }), + sourceLayerId: sourceLayer.id, + specValues: { + playSetting: getNormalizedString(sourceLayer, 'playSetting'), + artStyle: getNormalizedString(sourceLayer, 'artStyle'), + bodyRatio: getNormalizedString(sourceLayer, 'bodyRatio'), + characterView: getNormalizedString(sourceLayer, 'characterView'), + customPrompt: getNormalizedString(sourceLayer, 'customPrompt'), + }, + specReference: + resolveNormalizedReferences(sourceLayer, 'reference', availableLayers)[0] ?? + null, + }, + sourceLayer, + ); + } + + if (inputs.action === 'character.generate') { + return placeDraftBesideSourceLayer( + resizeGenerationPlaceholderToImageSelection({ + ...createCharacterGenerationDialogDraft({ + canvasSize, + viewport, + imageModel: getNormalizedString(sourceLayer, 'model', DEFAULT_IMAGE_MODEL), + }), + prompt, + sourceLayerId: sourceLayer.id, + style: + getNormalizedString(sourceLayer, 'style') === 'pixelArt' + ? 'pixelArt' + : 'none', + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + characterSpecReference: + resolveNormalizedReferences(sourceLayer, 'specReference', availableLayers)[0] ?? + null, + characterReferences: references, + }), + sourceLayer, + ); + } + + if (inputs.action === 'icon.generate') { + return placeDraftBesideSourceLayer( + resizeGenerationPlaceholderToImageSelection({ + ...createIconGenerationDialogDraft({ + canvasSize, + viewport, + imageModel: getNormalizedString(sourceLayer, 'model', DEFAULT_IMAGE_MODEL), + }), + prompt, + sourceLayerId: sourceLayer.id, + style: + getNormalizedString(sourceLayer, 'style') === 'pixelArt' + ? 'pixelArt' + : 'none', + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + iconDescriptions: prompt ? [prompt] : [], + iconSpecReference: + resolveNormalizedReferences(sourceLayer, 'specReference', availableLayers)[0] ?? + null, + generationReferences: references, + }), + sourceLayer, + ); + } + + if (inputs.action === 'ui-design.generate') { + return placeDraftBesideSourceLayer( + resizeGenerationPlaceholderToImageSelection({ + ...createUiDesignGenerationDialogDraft({ + canvasSize, + viewport, + imageModel: getNormalizedString( + sourceLayer, + 'model', + IMAGE_MODEL_GPT_IMAGE_2, + ), + }), + prompt, + sourceLayerId: sourceLayer.id, + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '16:9'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + uiDesignSpecReference: + resolveNormalizedReferences(sourceLayer, 'specReference', availableLayers)[0] ?? + null, + generationReferences: references, + }), + sourceLayer, + ); + } + + if (inputs.action === 'publication.generate') { + const workflowValue = getNormalizedString( + sourceLayer, + 'workflowId', + 'publication-cover-image', + ); + const workflowId: PublicationMaterialsWorkflowId = + workflowValue === 'publication-detail-gallery' || + workflowValue === 'publication-promo-poster' + ? workflowValue + : 'publication-cover-image'; + return placeDraftBesideSourceLayer( + { + ...createPublicationGenerationDialogDraft({ + canvasSize, + viewport, + workflowId, + }), + sourceLayerId: sourceLayer.id, + publicationGameInfo: { + gameName: getNormalizedString(sourceLayer, 'gameName'), + gameCategories: getNormalizedString(sourceLayer, 'gameCategories'), + gameDescription: getNormalizedString(sourceLayer, 'gameDescription'), + }, + publicationReferences: references, + }, + sourceLayer, + ); + } + + if (inputs.action === 'video.generate') { + const videoDraft = resizeGenerationPlaceholderToVideoSelection({ + ...createVideoGenerationDialogDraft({ canvasSize, viewport }), + prompt, + sourceLayerId: sourceLayer.id, + videoModel: getNormalizedString( + sourceLayer, + 'model', + DEFAULT_VIDEO_MODEL, + ) as GenerateDialogState['videoModel'], + videoAspectRatio: getNormalizedString( + sourceLayer, + 'aspectRatio', + DEFAULT_VIDEO_ASPECT_RATIO, + ) as GenerateDialogState['videoAspectRatio'], + videoResolution: getNormalizedString( + sourceLayer, + 'resolution', + '480p', + ) as GenerateDialogState['videoResolution'], + videoDurationSeconds: getNormalizedNumber( + sourceLayer, + 'durationSeconds', + DEFAULT_VIDEO_DURATION_SECONDS, + ), + videoSound: getNormalizedString( + sourceLayer, + 'sound', + DEFAULT_VIDEO_SOUND, + ) as GenerateDialogState['videoSound'], + generationReferences: [ + ...resolveNormalizedReferences( + sourceLayer, + 'imageReference', + availableLayers, + ), + ...resolveNormalizedReferences( + sourceLayer, + 'videoReference', + availableLayers, + ), + ...resolveNormalizedReferences( + sourceLayer, + 'audioReference', + availableLayers, + ), + ], + }) as Omit; + return placeDraftBesideSourceLayer( + videoDraft, + sourceLayer, + ); + } + + if ( + inputs.action === 'audio.sound-effect.generate' || + inputs.action === 'audio.background-music.generate' + ) { + return placeDraftBesideSourceLayer( + { + ...(inputs.action === 'audio.sound-effect.generate' + ? createSoundEffectGenerationDialogDraft({ canvasSize, viewport }) + : createBackgroundMusicGenerationDialogDraft({ canvasSize, viewport })), + prompt, + sourceLayerId: sourceLayer.id, + soundModel: + inputs.action === 'audio.sound-effect.generate' + ? (getNormalizedString(sourceLayer, 'model', DEFAULT_SOUND_EFFECT_MODEL) as GenerateDialogState['soundModel']) + : undefined, + soundDurationSeconds: + inputs.action === 'audio.sound-effect.generate' + ? getNormalizedNumber( + sourceLayer, + 'durationSeconds', + DEFAULT_SOUND_EFFECT_DURATION_SECONDS, + ) + : undefined, + }, + sourceLayer, + ); + } + + if (inputs.action === 'character-animation.generate') { + const animationSource = resolveNormalizedReferences( + sourceLayer, + 'source', + availableLayers, + ) + .map((reference) => + (availableLayers ?? []).find( + (layer) => createCanvasLayerReference(layer).id === reference.id, + ), + ) + .find((layer): layer is CanvasLayer => Boolean(layer)); + if (!animationSource) { + return null; + } + const draft = createCharacterAnimationGenerationDialogDraft({ + canvasSize, + viewport, + layer: animationSource, + }); + return draft + ? placeDraftBesideSourceLayer( + { + ...draft, + prompt, + sourceLayerId: animationSource.id, + characterAnimationResolution: getNormalizedString( + sourceLayer, + 'resolution', + '480p', + ) as GenerateDialogState['characterAnimationResolution'], + characterAnimationRatio: getNormalizedString( + sourceLayer, + 'ratio', + 'same', + ) as GenerateDialogState['characterAnimationRatio'], + characterAnimationFrameCount: getNormalizedNumber( + sourceLayer, + 'frameCount', + 32, + ) as GenerateDialogState['characterAnimationFrameCount'], + characterAnimationDurationSeconds: getNormalizedNumber( + sourceLayer, + 'durationSeconds', + 4, + ) as GenerateDialogState['characterAnimationDurationSeconds'], + }, + sourceLayer, + ) + : null; + } + + if (inputs.action === 'image.edit') { + const source = + resolveNormalizedReferences(sourceLayer, 'source', availableLayers)[0]; + const sourceLayerForEdit = source + ? (availableLayers ?? []).find( + (layer) => createCanvasLayerReference(layer).id === source.id, + ) + : undefined; + if (!sourceLayerForEdit) { + return null; + } + const size = resolveEditorImageGenerationPixelSize({ + model: getNormalizedString(sourceLayer, 'model', DEFAULT_IMAGE_MODEL), + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + }); + return { + ...createQuickEditGenerationDialogDraft({ + sourceLayer: sourceLayerForEdit, + prompt, + model: getNormalizedString(sourceLayer, 'model', DEFAULT_IMAGE_MODEL), + aspectRatio: getNormalizedString(sourceLayer, 'aspectRatio', '1:1'), + imageSize: getNormalizedString(sourceLayer, 'imageSize', '1K'), + frame: size, + }), + composerOpen: true, + isRedrawAsNewArtifact: true, + }; + } + + return null; +} + function restoreSharedImageOptions( draft: Omit, sourceLayer: CanvasLayer, @@ -961,10 +1360,23 @@ export function createSameSourceGenerationDialogDraft({ sourceDialog, mode, sourceAnimationLayer, + availableLayers, }: SourceGenerationDialogDraftContext): Omit< CanvasGenerationDialogState, 'id' > | null { + const normalizedDraft = createNormalizedGenerationDialogDraft({ + sourceLayer, + canvasSize, + viewport, + sourceDialog, + mode, + sourceAnimationLayer, + availableLayers, + }); + if (normalizedDraft) { + return normalizedDraft; + } const sourceMode = resolveGeneratedSourceDialogMode({ sourceLayer, sourceDialog, diff --git a/src/components/image-editor/ImageCanvasGenerationModel.test.ts b/src/components/image-editor/ImageCanvasGenerationModel.test.ts index c2e21ecbf..f12954d3b 100644 --- a/src/components/image-editor/ImageCanvasGenerationModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationModel.test.ts @@ -18,6 +18,7 @@ import { buildSpecGenerationInputs, buildSpecPrompt, buildUiDesignAssetExtractionGenerationInputs, + canOpenRedrawPanel, calculateCharacterAnimationPrice, calculateEditorBackgroundMusicPrice, calculateEditorIconSpritesheetPrice, @@ -236,7 +237,19 @@ describe('ImageCanvasGenerationModel', () => { it('builds user-facing generation input snapshots instead of backend prompts', () => { expect(buildImageGenerationInputs(' 一张明亮主视觉 ')).toEqual({ - fields: [{ title: '生成提示词', value: '一张明亮主视觉' }], + version: 2, + action: 'image.generate', + fields: [ + { id: 'prompt', title: '生成提示词', value: '一张明亮主视觉' }, + { + id: 'model', + title: '模型', + value: 'gemini-3.1-flash-image-preview', + }, + { id: 'style', title: '风格', value: 'none' }, + { id: 'aspectRatio', title: '比例', value: '1:1' }, + { id: 'imageSize', title: '尺寸', value: '1K' }, + ], references: [], }); @@ -248,15 +261,18 @@ describe('ImageCanvasGenerationModel', () => { characterView: '右向三分之二侧身', customPrompt: '', }), - ).toEqual({ + ).toEqual(expect.objectContaining({ + version: 2, + action: 'spec.generate', fields: [ - { title: '玩法设定', value: '平台跳跃' }, - { title: '美术风格', value: '像素风' }, - { title: '头身比', value: '3' }, - { title: '角色视角', value: '右向三分之二侧身' }, + { id: 'specType', title: '规范类型', value: 'character' }, + { id: 'playSetting', title: '玩法设定', value: '平台跳跃' }, + { id: 'artStyle', title: '美术风格', value: '像素风' }, + { id: 'bodyRatio', title: '头身比', value: '3' }, + { id: 'characterView', title: '角色视角', value: '右向三分之二侧身' }, ], references: [], - }); + })); }); it('builds character, icon and edit reference snapshots', () => { @@ -280,23 +296,29 @@ describe('ImageCanvasGenerationModel', () => { }, ], ), - ).toEqual({ - fields: [{ title: '角色设定', value: '主角骑士' }], + ).toEqual(expect.objectContaining({ + version: 2, + action: 'character.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '角色设定', value: '主角骑士' }, + ]), references: [ { + id: 'specReference', title: '角色规范', label: '角色规范', refType: 'project-resource', refId: 'resource-spec', }, { + id: 'reference', title: '常规参考图 1', label: '盔甲参考', refType: 'project-resource', refId: 'resource-armor', }, ], - }); + })); expect( buildIconGenerationInputs(['返回按钮', '设置按钮'], { @@ -305,31 +327,41 @@ describe('ImageCanvasGenerationModel', () => { src: '/icon-spec.png', resourceId: 'resource-icon-spec', }), - ).toEqual({ - fields: [{ title: '素材描述', value: '返回按钮\n设置按钮' }], + ).toEqual(expect.objectContaining({ + version: 2, + action: 'icon.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '素材描述', value: '返回按钮\n设置按钮' }, + ]), references: [ { + id: 'specReference', title: '图标规范', label: '图标规范', refType: 'project-resource', refId: 'resource-icon-spec', }, ], - }); + })); expect( buildEditGenerationInputs('修改要求', '换成夜晚', sourceLayer), - ).toEqual({ - fields: [{ title: '修改要求', value: '换成夜晚' }], + ).toEqual(expect.objectContaining({ + version: 2, + action: 'image.edit', + fields: expect.arrayContaining([ + { id: 'prompt', title: '修改要求', value: '换成夜晚' }, + ]), references: [ { + id: 'source', title: '参考图', label: '原图', refType: 'project-resource', refId: 'resource-source', }, ], - }); + })); expect( buildUiDesignAssetExtractionGenerationInputs(sourceLayer, [ { @@ -339,29 +371,32 @@ describe('ImageCanvasGenerationModel', () => { resourceId: 'resource-ui-style', }, ]), - ).toEqual({ - fields: [ - { + ).toEqual(expect.objectContaining({ + version: 2, + action: 'ui-design.extract-assets', + fields: expect.arrayContaining([ + expect.objectContaining({ + id: 'prompt', title: '提取提示词', - value: - '仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用所选单一纯色抠图背景。纯色背景必须平整无纹理、无渐变、无阴影、无地面、无环境、无道具,方便后续扣除背景;素材自身不要出现与背景色相同或相近的描边、底板、投影或反光。', - }, - ], + }), + ]), references: [ { + id: 'source', title: 'UI设计图', label: '原图', refType: 'project-resource', refId: 'resource-source', }, { + id: 'reference', title: '参考图 1', label: 'UI风格参考', refType: 'project-resource', refId: 'resource-ui-style', }, ], - }); + })); expect( buildQuickEditGenerationInputs( '快速编辑提示词', @@ -376,23 +411,86 @@ describe('ImageCanvasGenerationModel', () => { }, ], ), - ).toEqual({ - fields: [{ title: '快速编辑提示词', value: '让图2下雨' }], + ).toEqual(expect.objectContaining({ + version: 2, + action: 'image.edit', + fields: expect.arrayContaining([ + { id: 'prompt', title: '快速编辑提示词', value: '让图2下雨' }, + ]), references: [ { + id: 'source', title: '原图', label: '原图', refType: 'project-resource', refId: 'resource-source', }, { + id: 'reference', title: '参考图 1', label: '天空', refType: 'project-resource', refId: 'resource-sky', }, ], - }); + })); + }); + + it('only exposes redraw for executable generation metadata', () => { + const generatedLayer = { + ...buildSourceLayer(), + sourceType: 'generated' as const, + }; + + expect( + canOpenRedrawPanel({ + ...generatedLayer, + generationInputs: buildImageGenerationInputs('生成城堡'), + }), + ).toBe(true); + expect( + canOpenRedrawPanel({ + ...generatedLayer, + generationInputs: { + fields: [{ title: '生成提示词', value: '旧版城堡' }], + references: [], + }, + }), + ).toBe(true); + expect( + canOpenRedrawPanel({ + ...generatedLayer, + generationInputs: null, + }), + ).toBe(false); + expect( + canOpenRedrawPanel({ + ...generatedLayer, + generationInputs: { + version: 2, + fields: [{ title: '生成提示词', value: '公开展示文本' }], + references: [], + }, + }), + ).toBe(false); + expect( + canOpenRedrawPanel({ + ...generatedLayer, + generationInputs: { + version: 2, + action: 'unknown.generate' as never, + fields: [{ id: 'prompt', title: '提示词', value: '未知动作' }], + references: [], + }, + }), + ).toBe(false); + expect( + canOpenRedrawPanel({ + ...generatedLayer, + sourceType: 'uploaded', + generationInputs: buildImageGenerationInputs('上传文件'), + }), + ).toBe(false); }); it('keeps generated prompts and quick edit options stable', () => { @@ -472,9 +570,19 @@ describe('ImageCanvasGenerationModel', () => { }, ), ).toEqual({ - fields: [{ title: '自定义规范提示词', value: '生成一张怪兽规范图' }], + version: 2, + action: 'spec.generate', + fields: [ + { id: 'specType', title: '规范类型', value: 'custom' }, + { + id: 'customPrompt', + title: '自定义规范提示词', + value: '生成一张怪兽规范图', + }, + ], references: [ { + id: 'reference', title: '参考图', label: '参考.png', refType: 'project-resource', @@ -530,10 +638,12 @@ describe('ImageCanvasGenerationModel', () => { expect( buildSoundEffectGenerationInputs('金币掉落叮当声', 'audio1.0', 7), ).toEqual({ + version: 2, + action: 'audio.sound-effect.generate', fields: [ - { title: 'prompt', value: '金币掉落叮当声' }, - { title: 'model', value: 'audio1.0' }, - { title: 'duration', value: '7秒' }, + { id: 'prompt', title: 'prompt', value: '金币掉落叮当声' }, + { id: 'model', title: 'model', value: 'audio1.0' }, + { id: 'durationSeconds', title: 'duration', value: 7 }, ], references: [], }); diff --git a/src/components/image-editor/ImageCanvasGenerationModel.ts b/src/components/image-editor/ImageCanvasGenerationModel.ts index 396d10029..86e1b3051 100644 --- a/src/components/image-editor/ImageCanvasGenerationModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationModel.ts @@ -10,8 +10,10 @@ import { } from './ImageCanvasEditorModel'; import type { CanvasGenerationDialogState, + CanvasGenerationAction, CanvasGenerationInputField, CanvasGenerationInputReference, + CanvasGenerationInputValue, CanvasGenerationInputs, CanvasLayer, CharacterReferenceImage, @@ -695,10 +697,40 @@ export function getLayerKindLabel(layer: CanvasLayer) { export function buildCharacterAnimationGenerationInputs( promptText: string, sourceLayer: CanvasLayer, + options: { + resolution?: string; + ratio?: string; + frameCount?: number; + durationSeconds?: number; + } = {}, ): CanvasGenerationInputs { return { - fields: createGenerationInputField('动作描述', promptText), - references: createLayerGenerationInputReference('角色图片', sourceLayer), + version: 2, + action: 'character-animation.generate', + fields: [ + ...createGenerationInputField('动作描述', promptText, 'prompt'), + ...createGenerationInputField( + '清晰度', + options.resolution ?? '480p', + 'resolution', + ), + ...createGenerationInputField('比例', options.ratio ?? 'same', 'ratio'), + ...createGenerationInputField( + '帧数', + options.frameCount ?? 32, + 'frameCount', + ), + ...createGenerationInputField( + '时长', + options.durationSeconds ?? 4, + 'durationSeconds', + ), + ], + references: createLayerGenerationInputReference( + '角色图片', + sourceLayer, + { id: 'source' }, + ), }; } @@ -951,6 +983,9 @@ export function createGenerationInputReference( CharacterReferenceImage, 'label' | 'resourceId' | 'sourceAssetId' >, + options: { + id?: string; + } = {}, ): CanvasGenerationInputReference[] { const resourceId = reference.resourceId?.trim(); if ( @@ -960,6 +995,7 @@ export function createGenerationInputReference( ) { return [ { + ...(options.id ? { id: options.id } : {}), title, label: reference.label, refType: 'project-resource', @@ -971,6 +1007,7 @@ export function createGenerationInputReference( return sourceAssetId && !sourceAssetId.startsWith('upload-') ? [ { + ...(options.id ? { id: options.id } : {}), title, label: reference.label, refType: 'asset', @@ -983,12 +1020,13 @@ export function createGenerationInputReference( export function createLayerGenerationInputReference( title: string, layer: CanvasLayer, + options: Parameters[2] = {}, ): CanvasGenerationInputReference[] { return createGenerationInputReference(title, { label: layer.title, resourceId: layer.resourceId, sourceAssetId: layer.sourceAssetId, - }); + }, options); } export function appendLimitedQuickEditReferences( @@ -1006,36 +1044,169 @@ export function appendLimitedQuickEditReferences( export function createGenerationInputField( title: string, - value: string | null | undefined, + value: CanvasGenerationInputValue | null | undefined, + id?: string, ): CanvasGenerationInputField[] { - const normalizedValue = value?.trim(); - return normalizedValue ? [{ title, value: normalizedValue }] : []; + const normalizedValue = + typeof value === 'string' ? value.trim() : value; + return normalizedValue === undefined || normalizedValue === null || normalizedValue === '' + ? [] + : [{ ...(id ? { id } : {}), title, value: normalizedValue }]; +} + +export function formatGenerationInputValue(value: CanvasGenerationInputValue) { + return typeof value === 'string' ? value : String(value); +} + +const CANVAS_GENERATION_ACTIONS = new Set([ + 'image.generate', + 'spec.generate', + 'character.generate', + 'icon.generate', + 'ui-design.generate', + 'publication.generate', + 'video.generate', + 'audio.sound-effect.generate', + 'audio.background-music.generate', + 'character-animation.generate', + 'image.edit', + 'ui-design.extract-assets', +]); + +export function isNormalizedCanvasGenerationInputs( + value: CanvasGenerationInputs | null | undefined, +): value is CanvasGenerationInputs & { + version: 2; + action: CanvasGenerationAction; +} { + return Boolean( + value?.version === 2 && + value.action && + CANVAS_GENERATION_ACTIONS.has(value.action) && + Array.isArray(value.fields) && + Array.isArray(value.references) && + value.fields.every((field) => Boolean(field && field.id?.trim())) && + value.references.every((reference) => + Boolean(reference && reference.id?.trim()), + ), + ); +} + +const LEGACY_REUSABLE_GENERATION_INPUT_TITLES = new Set( + [ + '生成提示词', + '视频描述', + 'prompt', + 'gpt_description_prompt', + '音效提示词', + '背景音乐提示词', + '角色设定', + '用户输入', + '素材描述', + '自定义规范提示词', + '玩法设定', + '美术风格', + '游戏名', + '修改要求', + '快速编辑提示词', + '重绘提示词', + '动作描述', + ].map((title) => title.toLowerCase()), +); + +export function canOpenRedrawPanel(layer: CanvasLayer) { + if (layer.sourceType === 'uploaded') { + return false; + } + if (isNormalizedCanvasGenerationInputs(layer.generationInputs)) { + return true; + } + if (layer.generationInputs?.version === 2) { + return false; + } + return Boolean( + layer.generationInputs?.fields.some((field) => + LEGACY_REUSABLE_GENERATION_INPUT_TITLES.has( + field.title.trim().toLowerCase(), + ), + ), + ); +} + +function buildNormalizedGenerationInputs( + action: CanvasGenerationAction, + fields: CanvasGenerationInputField[], + references: CanvasGenerationInputReference[], +): CanvasGenerationInputs { + return { version: 2, action, fields, references }; } export function buildImageGenerationInputs( prompt: string, references?: CharacterReferenceImage[], + options: { + model?: string; + style?: string; + aspectRatio?: string; + imageSize?: string; + } = {}, ): CanvasGenerationInputs { - return { - fields: createGenerationInputField('生成提示词', prompt), - references: (references ?? []).flatMap((reference, index) => - createGenerationInputReference(`参考图 ${index + 1}`, reference), + return buildNormalizedGenerationInputs( + 'image.generate', + [ + ...createGenerationInputField('生成提示词', prompt, 'prompt'), + ...createGenerationInputField( + '模型', + options.model ?? DEFAULT_IMAGE_MODEL, + 'model', + ), + ...createGenerationInputField('风格', options.style ?? 'none', 'style'), + ...createGenerationInputField( + '比例', + options.aspectRatio ?? '1:1', + 'aspectRatio', + ), + ...createGenerationInputField( + '尺寸', + options.imageSize ?? '1K', + 'imageSize', + ), + ], + (references ?? []).flatMap((reference, index) => + createGenerationInputReference(`参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), - }; + ); } export function buildVideoGenerationInputs( prompt: string, references?: CharacterReferenceImage[], + options: { + model?: string; + aspectRatio?: string; + resolution?: string; + durationSeconds?: number; + sound?: string; + } = {}, ): CanvasGenerationInputs { const mediaIndexes = { image: 0, video: 0, audio: 0, }; - return { - fields: createGenerationInputField('视频描述', prompt), - references: (references ?? []).flatMap((reference) => { + return buildNormalizedGenerationInputs( + 'video.generate', + [ + ...createGenerationInputField('视频描述', prompt, 'prompt'), + ...createGenerationInputField('模型', options.model ?? DEFAULT_VIDEO_MODEL, 'model'), + ...createGenerationInputField('比例', options.aspectRatio ?? DEFAULT_VIDEO_ASPECT_RATIO, 'aspectRatio'), + ...createGenerationInputField('清晰度', options.resolution ?? '480p', 'resolution'), + ...createGenerationInputField('时长', options.durationSeconds ?? DEFAULT_VIDEO_DURATION_SECONDS, 'durationSeconds'), + ...createGenerationInputField('声音', options.sound ?? DEFAULT_VIDEO_SOUND, 'sound'), + ], + (references ?? []).flatMap((reference) => { const mediaType = reference.mediaType === 'video' || reference.mediaType === 'audio' ? reference.mediaType @@ -1047,9 +1218,11 @@ export function buildVideoGenerationInputs( : mediaType === 'audio' ? `参考音频 ${mediaIndexes.audio}` : `参考图 ${mediaIndexes.image}`; - return createGenerationInputReference(title, reference); + return createGenerationInputReference(title, reference, { + id: `${mediaType}Reference`, + }); }), - }; + ); } export function buildSoundEffectGenerationInputs( @@ -1058,10 +1231,12 @@ export function buildSoundEffectGenerationInputs( duration: number, ): CanvasGenerationInputs { return { + version: 2, + action: 'audio.sound-effect.generate', fields: [ - ...createGenerationInputField('prompt', prompt), - ...createGenerationInputField('model', model), - ...createGenerationInputField('duration', `${duration}秒`), + ...createGenerationInputField('prompt', prompt, 'prompt'), + ...createGenerationInputField('model', model, 'model'), + ...createGenerationInputField('duration', duration, 'durationSeconds'), ], references: [], }; @@ -1071,9 +1246,12 @@ export function buildBackgroundMusicGenerationInputs( gptDescriptionPrompt: string, ): CanvasGenerationInputs { return { + version: 2, + action: 'audio.background-music.generate', fields: createGenerationInputField( 'gpt_description_prompt', gptDescriptionPrompt, + 'prompt', ), references: [], }; @@ -1085,29 +1263,40 @@ export function buildSpecGenerationInputs( reference?: CharacterReferenceImage | null, ): CanvasGenerationInputs { const references = reference - ? createGenerationInputReference('参考图', reference) + ? createGenerationInputReference('参考图', reference, { + id: 'reference', + }) : []; if (specType === 'custom') { return { - fields: createGenerationInputField( - '自定义规范提示词', - values.customPrompt, - ), + version: 2, + action: 'spec.generate', + fields: [ + ...createGenerationInputField('规范类型', specType, 'specType'), + ...createGenerationInputField( + '自定义规范提示词', + values.customPrompt, + 'customPrompt', + ), + ], references, }; } const baseFields = [ - ...createGenerationInputField('玩法设定', values.playSetting), - ...createGenerationInputField('美术风格', values.artStyle), + ...createGenerationInputField('规范类型', specType, 'specType'), + ...createGenerationInputField('玩法设定', values.playSetting, 'playSetting'), + ...createGenerationInputField('美术风格', values.artStyle, 'artStyle'), ]; if (specType === 'character') { baseFields.push( - ...createGenerationInputField('头身比', values.bodyRatio), - ...createGenerationInputField('角色视角', values.characterView), + ...createGenerationInputField('头身比', values.bodyRatio, 'bodyRatio'), + ...createGenerationInputField('角色视角', values.characterView, 'characterView'), ); } return { + version: 2, + action: 'spec.generate', fields: baseFields, references, }; @@ -1117,15 +1306,33 @@ export function buildCharacterGenerationInputs( prompt: string, specReference: CharacterReferenceImage | null | undefined, references: CharacterReferenceImage[] | undefined, + options: { + model?: string; + style?: string; + aspectRatio?: string; + imageSize?: string; + } = {}, ): CanvasGenerationInputs { return { - fields: createGenerationInputField('角色设定', prompt), + version: 2, + action: 'character.generate', + fields: [ + ...createGenerationInputField('角色设定', prompt, 'prompt'), + ...createGenerationInputField('模型', options.model ?? DEFAULT_IMAGE_MODEL, 'model'), + ...createGenerationInputField('风格', options.style ?? 'none', 'style'), + ...createGenerationInputField('比例', options.aspectRatio ?? '1:1', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), + ], references: [ ...(specReference - ? createGenerationInputReference('角色规范', specReference) + ? createGenerationInputReference('角色规范', specReference, { + id: 'specReference', + }) : []), ...(references ?? []).flatMap((reference, index) => - createGenerationInputReference(`常规参考图 ${index + 1}`, reference), + createGenerationInputReference(`常规参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), ], }; @@ -1135,15 +1342,27 @@ export function buildUiDesignGenerationInputs( prompt: string, specReference: CharacterReferenceImage | null | undefined, references?: CharacterReferenceImage[], + options: { model?: string; aspectRatio?: string; imageSize?: string } = {}, ): CanvasGenerationInputs { return { - fields: createGenerationInputField('用户输入', prompt), + version: 2, + action: 'ui-design.generate', + fields: [ + ...createGenerationInputField('用户输入', prompt, 'prompt'), + ...createGenerationInputField('模型', options.model ?? IMAGE_MODEL_GPT_IMAGE_2, 'model'), + ...createGenerationInputField('比例', options.aspectRatio ?? '16:9', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), + ], references: [ ...(specReference - ? createGenerationInputReference('图标规范', specReference) + ? createGenerationInputReference('图标规范', specReference, { + id: 'specReference', + }) : []), ...(references ?? []).flatMap((reference, index) => - createGenerationInputReference(`参考图 ${index + 1}`, reference), + createGenerationInputReference(`参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), ], }; @@ -1200,22 +1419,30 @@ export function buildPublicationMaterialsGenerationPrompt({ export function buildPublicationMaterialsGenerationInputs( gameInfo: PublicationMaterialsGameInfo | null | undefined, references: CharacterReferenceImage[] | undefined, + workflowId: PublicationMaterialsWorkflowId = 'publication-cover-image', ): CanvasGenerationInputs { const normalizedGameInfo = normalizePublicationGameInfo(gameInfo); return { + version: 2, + action: 'publication.generate', fields: [ - ...createGenerationInputField('游戏名', normalizedGameInfo.gameName), + ...createGenerationInputField('宣发类型', workflowId, 'workflowId'), + ...createGenerationInputField('游戏名', normalizedGameInfo.gameName, 'gameName'), ...createGenerationInputField( '游戏分类', normalizedGameInfo.gameCategories, + 'gameCategories', ), ...createGenerationInputField( '一句话描述游戏', normalizedGameInfo.gameDescription, + 'gameDescription', ), ], references: (references ?? []).flatMap((reference, index) => - createGenerationInputReference(`宣发参考图 ${index + 1}`, reference), + createGenerationInputReference(`宣发参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), }; } @@ -1224,18 +1451,35 @@ export function buildIconGenerationInputs( iconDescriptions: string[], specReference: CharacterReferenceImage, references?: CharacterReferenceImage[], + options: { + model?: string; + style?: string; + aspectRatio?: string; + imageSize?: string; + } = {}, ): CanvasGenerationInputs { return { + version: 2, + action: 'icon.generate', fields: [ { + id: 'prompt', title: '素材描述', value: iconDescriptions.join('\n'), }, + ...createGenerationInputField('模型', options.model ?? DEFAULT_IMAGE_MODEL, 'model'), + ...createGenerationInputField('风格', options.style ?? 'none', 'style'), + ...createGenerationInputField('比例', options.aspectRatio ?? '1:1', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), ], references: [ - ...createGenerationInputReference('图标规范', specReference), + ...createGenerationInputReference('图标规范', specReference, { + id: 'specReference', + }), ...(references ?? []).flatMap((reference, index) => - createGenerationInputReference(`参考图 ${index + 1}`, reference), + createGenerationInputReference(`参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), ], }; @@ -1244,18 +1488,29 @@ export function buildIconGenerationInputs( export function buildUiDesignAssetExtractionGenerationInputs( sourceLayer: CanvasLayer, references?: CharacterReferenceImage[], + options: { model?: string; imageSize?: string } = {}, ): CanvasGenerationInputs { return { + version: 2, + action: 'ui-design.extract-assets', fields: [ ...createGenerationInputField( '提取提示词', UI_DESIGN_ASSET_EXTRACTION_PROMPT, + 'prompt', ), + ...createGenerationInputField('模型', options.model ?? DEFAULT_IMAGE_MODEL, 'model'), + ...createGenerationInputField('比例', '1:1', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), ], references: [ - ...createLayerGenerationInputReference('UI设计图', sourceLayer), + ...createLayerGenerationInputReference('UI设计图', sourceLayer, { + id: 'source', + }), ...(references ?? []).flatMap((reference, index) => - createGenerationInputReference(`参考图 ${index + 1}`, reference), + createGenerationInputReference(`参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), ], }; @@ -1265,10 +1520,20 @@ export function buildEditGenerationInputs( title: '修改要求' | '快速编辑提示词' | '重绘提示词', prompt: string, sourceLayer: CanvasLayer, + options: { model?: string; aspectRatio?: string; imageSize?: string } = {}, ): CanvasGenerationInputs { return { - fields: createGenerationInputField(title, prompt), - references: createLayerGenerationInputReference('参考图', sourceLayer), + version: 2, + action: 'image.edit', + fields: [ + ...createGenerationInputField(title, prompt, 'prompt'), + ...createGenerationInputField('模型', options.model ?? DEFAULT_IMAGE_MODEL, 'model'), + ...createGenerationInputField('比例', options.aspectRatio ?? '1:1', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), + ], + references: createLayerGenerationInputReference('参考图', sourceLayer, { + id: 'source', + }), }; } @@ -1277,17 +1542,29 @@ export function buildQuickEditGenerationInputs( prompt: string, sourceLayer: CanvasLayer, references: CharacterReferenceImage[] | undefined, + options: { model?: string; aspectRatio?: string; imageSize?: string } = {}, ): CanvasGenerationInputs { const limitedReferences = (references ?? []).slice( 0, QUICK_EDIT_REFERENCE_LIMIT, ); return { - fields: createGenerationInputField(title, prompt), + version: 2, + action: 'image.edit', + fields: [ + ...createGenerationInputField(title, prompt, 'prompt'), + ...createGenerationInputField('模型', options.model ?? DEFAULT_IMAGE_MODEL, 'model'), + ...createGenerationInputField('比例', options.aspectRatio ?? '1:1', 'aspectRatio'), + ...createGenerationInputField('尺寸', options.imageSize ?? '1K', 'imageSize'), + ], references: [ - ...createLayerGenerationInputReference('原图', sourceLayer), + ...createLayerGenerationInputReference('原图', sourceLayer, { + id: 'source', + }), ...limitedReferences.flatMap((reference, index) => - createGenerationInputReference(`参考图 ${index + 1}`, reference), + createGenerationInputReference(`参考图 ${index + 1}`, reference, { + id: 'reference', + }), ), ], }; diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts index 1784b7056..43286036c 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts @@ -53,10 +53,14 @@ describe('ImageCanvasGenerationSubmissionModel', () => { }, result: { title: '生成图片 3', - generationInputs: { - fields: [{ title: '生成提示词', value: '一张发光主视觉' }], + generationInputs: expect.objectContaining({ + version: 2, + action: 'image.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '生成提示词', value: '一张发光主视觉' }, + ]), references: [], - }, + }), }, rememberImageModel: 'gemini-3.1-flash-image-preview', }); @@ -124,17 +128,22 @@ describe('ImageCanvasGenerationSubmissionModel', () => { kind: 'edit', normalizedPrompt: '修改当前图片', sourceLayer, - generationInputs: { - fields: [{ title: '修改要求', value: '修改当前图片' }], + generationInputs: expect.objectContaining({ + version: 2, + action: 'image.edit', + fields: expect.arrayContaining([ + { id: 'prompt', title: '修改要求', value: '修改当前图片' }, + ]), references: [ { + id: 'source', title: '参考图', label: '源图', refType: 'project-resource', refId: 'resource-source', }, ], - }, + }), }); }); @@ -178,17 +187,26 @@ describe('ImageCanvasGenerationSubmissionModel', () => { result: { title: '源图 快速编辑', assetKind: 'character', - generationInputs: { - fields: [{ title: '快速编辑提示词', value: '把当前图改成雨天' }], + generationInputs: expect.objectContaining({ + version: 2, + action: 'image.edit', + fields: expect.arrayContaining([ + { + id: 'prompt', + title: '快速编辑提示词', + value: '把当前图改成雨天', + }, + ]), references: [ { + id: 'source', title: '原图', label: '源图', refType: 'project-resource', refId: 'resource-source', }, ], - }, + }), }, rememberImageModel: IMAGE_MODEL_GPT_IMAGE_2, }); @@ -255,20 +273,23 @@ describe('ImageCanvasGenerationSubmissionModel', () => { }, }); expect(plan.kind === 'image' ? plan.result.generationInputs : null).toEqual( - { - fields: [ - { title: '玩法设定', value: '休闲消除' }, - { title: '美术风格', value: '清爽卡通' }, - ], + expect.objectContaining({ + version: 2, + action: 'spec.generate', + fields: expect.arrayContaining([ + { id: 'playSetting', title: '玩法设定', value: '休闲消除' }, + { id: 'artStyle', title: '美术风格', value: '清爽卡通' }, + ]), references: [ { + id: 'reference', title: '参考图', label: '参考.png', refType: 'project-resource', refId: 'resource-spec-ref', }, ], - }, + }), ); }); @@ -359,25 +380,29 @@ describe('ImageCanvasGenerationSubmissionModel', () => { rememberImageModel: 'gpt-image-2', }); expect(plan.kind === 'image' ? plan.result.generationInputs : null).toEqual( - { - fields: [ - { title: '角色设定', value: '白发骑士' }, - ], + expect.objectContaining({ + version: 2, + action: 'character.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '角色设定', value: '白发骑士' }, + ]), references: [ { + id: 'specReference', title: '角色规范', label: '角色规范', refType: 'project-resource', refId: 'resource-character-spec', }, { + id: 'reference', title: '常规参考图 1', label: '盔甲参考', refType: 'project-resource', refId: 'resource-armor', }, ], - }, + }), ); }); @@ -425,23 +450,29 @@ describe('ImageCanvasGenerationSubmissionModel', () => { result: { assetKind: 'ui-design', title: 'UI设计图 6', - generationInputs: { - fields: [{ title: '用户输入', value: '设计移动端商城首页' }], + generationInputs: expect.objectContaining({ + version: 2, + action: 'ui-design.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '用户输入', value: '设计移动端商城首页' }, + ]), references: [ { + id: 'specReference', title: '图标规范', label: 'UI图标规范', refType: 'project-resource', refId: 'resource-ui-spec', }, { + id: 'reference', title: '参考图 1', label: '界面风格参考', refType: 'project-resource', refId: 'resource-ui-ref', }, ], - }, + }), }, rememberImageModel: IMAGE_MODEL_GPT_IMAGE_2, }); @@ -491,24 +522,32 @@ describe('ImageCanvasGenerationSubmissionModel', () => { result: { assetKind: 'publication-material', title: '10 宣发素材', - generationInputs: { - fields: [ - { title: '游戏名', value: '重庆洪崖洞火锅' }, - { title: '游戏分类', value: '抓大鹅、休闲、治愈、手绘风' }, + generationInputs: expect.objectContaining({ + version: 2, + action: 'publication.generate', + fields: expect.arrayContaining([ + { id: 'gameName', title: '游戏名', value: '重庆洪崖洞火锅' }, { + id: 'gameCategories', + title: '游戏分类', + value: '抓大鹅、休闲、治愈、手绘风', + }, + { + id: 'gameDescription', title: '一句话描述游戏', value: '在一堆物品里点击,找到三个一样的物品将其消除', }, - ], + ]), references: [ { + id: 'reference', title: '宣发参考图 1', label: '参考图', refType: 'project-resource', refId: 'resource-publication-ref', }, ], - }, + }), }, }); const prompt = plan.kind === 'image' ? plan.input.prompt : ''; @@ -722,23 +761,29 @@ describe('ImageCanvasGenerationSubmissionModel', () => { aspectRatio: '3:2', imageSize: '2K', }, - generationInputs: { - fields: [{ title: '素材描述', value: '返回按钮\n\n设置按钮' }], + generationInputs: expect.objectContaining({ + version: 2, + action: 'icon.generate', + fields: expect.arrayContaining([ + { id: 'prompt', title: '素材描述', value: '返回按钮\n\n设置按钮' }, + ]), references: [ { + id: 'specReference', title: '图标规范', label: '图标规范', refType: 'project-resource', refId: 'resource-icon-spec', }, { + id: 'reference', title: '参考图 1', label: '按钮风格参考', refType: 'project-resource', refId: 'resource-icon-ref', }, ], - }, + }), rememberImageModel: 'gpt-image-2', resultTitle: '冒险游戏图标', }); @@ -850,28 +895,33 @@ describe('ImageCanvasGenerationSubmissionModel', () => { ], }, result: { - generationInputs: { + generationInputs: expect.objectContaining({ + version: 2, + action: 'video.generate', references: [ { + id: 'imageReference', title: '参考图 1', label: '参考图.png', refType: 'project-resource', refId: 'resource-video-image-ref', }, { + id: 'videoReference', title: '参考视频 1', label: '参考视频.mp4', refType: 'project-resource', refId: 'resource-video-ref', }, { + id: 'audioReference', title: '参考音频 1', label: '参考音频.mp3', refType: 'project-resource', refId: 'resource-audio-ref', }, ], - }, + }), }, }); }); @@ -1028,10 +1078,12 @@ describe('ImageCanvasGenerationSubmissionModel', () => { result: { title: '游戏音效 4', generationInputs: { + version: 2, + action: 'audio.sound-effect.generate', fields: [ - { title: 'prompt', value: '金币掉落叮当声' }, - { title: 'model', value: 'audio1.0' }, - { title: 'duration', value: '7秒' }, + { id: 'prompt', title: 'prompt', value: '金币掉落叮当声' }, + { id: 'model', title: 'model', value: 'audio1.0' }, + { id: 'durationSeconds', title: 'duration', value: 7 }, ], references: [], }, @@ -1060,11 +1112,14 @@ describe('ImageCanvasGenerationSubmissionModel', () => { }, result: { generationInputs: { + version: 2, + action: 'audio.sound-effect.generate', fields: [ - { title: 'prompt', value: '按钮确认短促音' }, - { title: 'model', value: 'audio1.0' }, - { title: 'duration', value: '5秒' }, + { id: 'prompt', title: 'prompt', value: '按钮确认短促音' }, + { id: 'model', title: 'model', value: 'audio1.0' }, + { id: 'durationSeconds', title: 'duration', value: 5 }, ], + references: [], }, }, }); @@ -1092,8 +1147,11 @@ describe('ImageCanvasGenerationSubmissionModel', () => { result: { title: '游戏背景音乐 5', generationInputs: { + version: 2, + action: 'audio.background-music.generate', fields: [ { + id: 'prompt', title: 'gpt_description_prompt', value: '温暖轻快的森林冒险背景音乐', }, diff --git a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts index f33d4f55f..bbd9c7d5d 100644 --- a/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts @@ -251,6 +251,11 @@ export function buildImageGenerationSubmissionPlan({ '修改要求', normalizedPrompt, sourceLayer, + { + model: dialog.imageModel, + aspectRatio: dialog.aspectRatio, + imageSize: dialog.imageSize, + }, ), }; } @@ -260,7 +265,7 @@ export function buildImageGenerationSubmissionPlan({ (layer) => layer.id === dialog.sourceLayerId, ); if (!sourceLayer) { - throw new Error('未找到要改造的图片'); + throw new Error('未找到要改造的原图'); } const basePrompt = dialog.prompt.trim() || '快速编辑图片'; const normalizedQuickEditPrompt = basePrompt; @@ -284,6 +289,11 @@ export function buildImageGenerationSubmissionPlan({ normalizedQuickEditPrompt, sourceLayer, [], + { + model: imageModel, + aspectRatio: dialog.aspectRatio, + imageSize: dialog.imageSize, + }, ), }, rememberImageModel: imageModel, @@ -368,6 +378,12 @@ export function buildImageGenerationSubmissionPlan({ normalizedPrompt, dialog.characterSpecReference, dialog.characterReferences, + { + model: imageModel, + style: dialog.style === 'pixelArt' ? 'pixelArt' : 'none', + aspectRatio: dialog.aspectRatio ?? '1:1', + imageSize: dialog.imageSize ?? '1K', + }, ), }, rememberImageModel: imageModel, @@ -405,6 +421,11 @@ export function buildImageGenerationSubmissionPlan({ normalizedPrompt, dialog.uiDesignSpecReference, dialog.generationReferences, + { + model: imageModel, + aspectRatio: dialog.aspectRatio ?? '16:9', + imageSize: normalizeGptImageSize(dialog.imageSize), + }, ), }, rememberImageModel: imageModel, @@ -450,6 +471,7 @@ export function buildImageGenerationSubmissionPlan({ generationInputs: buildPublicationMaterialsGenerationInputs( dialog.publicationGameInfo, dialog.publicationReferences, + workflow.id, ), }, }; @@ -503,6 +525,13 @@ export function buildImageGenerationSubmissionPlan({ generationInputs: buildVideoGenerationInputs( normalizedPrompt, dialog.generationReferences, + { + model, + aspectRatio, + resolution, + durationSeconds, + sound, + }, ), }, }; @@ -585,6 +614,12 @@ export function buildImageGenerationSubmissionPlan({ generationInputs: buildImageGenerationInputs( normalizedPrompt, dialog.generationReferences, + { + model: imageModel, + style: dialog.style === 'pixelArt' ? 'pixelArt' : 'none', + aspectRatio: dialog.aspectRatio ?? '1:1', + imageSize: dialog.imageSize ?? '1K', + }, ), }, rememberImageModel: imageModel, @@ -646,6 +681,12 @@ export function buildIconSpritesheetGenerationSubmissionPlan( iconDescriptions, dialog.iconSpecReference, dialog.generationReferences, + { + model: rememberImageModel, + style: dialog.style === 'pixelArt' ? 'pixelArt' : 'none', + aspectRatio: dialog.aspectRatio ?? '1:1', + imageSize: dialog.imageSize ?? '1K', + }, ), rememberImageModel, resultTitle: resolveGenerationAssetLabel( diff --git a/src/components/image-editor/ImageCanvasMetadataModalView.tsx b/src/components/image-editor/ImageCanvasMetadataModalView.tsx index 3b5a1768c..f597f6b3a 100644 --- a/src/components/image-editor/ImageCanvasMetadataModalView.tsx +++ b/src/components/image-editor/ImageCanvasMetadataModalView.tsx @@ -76,14 +76,16 @@ export function ImageCanvasMetadataModalView({
{generationInputReferences.map((reference) => (
{reference.title} - {reference.label} + + {reference.label ?? '参考素材'} + {formatReferenceType(reference.refType)} ·{' '} {reference.refId} diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx index b06f0013e..671905984 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.test.tsx @@ -25,7 +25,13 @@ function createLayer(overrides: Partial = {}): CanvasLayer { originalWidth: 320, originalHeight: 240, zIndex: 1, - sourceType: 'uploaded', + sourceType: 'generated', + generationInputs: { + version: 2, + action: 'image.generate', + fields: [{ id: 'prompt', title: '生成提示词', value: '测试图片' }], + references: [], + }, ...overrides, }; } @@ -85,7 +91,9 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { props.selectedLayer, ); expect(props.onRemoveBackground).toHaveBeenCalledWith(props.selectedLayer); - expect(props.onOpenRedrawPanel).toHaveBeenCalledWith(props.selectedLayer); + expect(props.onOpenRedrawPanel).toHaveBeenCalledWith( + props.selectedLayer, + ); expect(props.onDownloadLayer).toHaveBeenCalledWith(props.selectedLayer); expect( within(toolbar).queryByRole('button', { name: '删除图片' }), @@ -209,6 +217,12 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { src: 'data:audio/mpeg;base64,audio', mediaType: 'audio', assetKind: 'sound-effect', + generationInputs: { + version: 2, + action: 'audio.sound-effect.generate', + fields: [{ id: 'prompt', title: 'prompt', value: '金币声' }], + references: [], + }, }); const props = renderSelectedToolbar({ selectedLayer: layer }); const toolbar = screen.getByRole('toolbar', { name: '素材工具栏' }); @@ -238,6 +252,32 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { expect(props.onDownloadLayer).toHaveBeenCalledWith(layer); }); + it('hides redraw for non-generative derived and public display-only layers', () => { + renderSelectedToolbar({ + selectedLayer: createLayer({ + title: '手动裁扩结果', + generationInputs: null, + }), + }); + expect(screen.queryByRole('button', { name: '改造' })).toBeNull(); + + cleanup(); + renderSelectedToolbar({ + selectedLayer: createLayer({ + title: '公开音效', + mediaType: 'audio', + assetKind: 'sound-effect', + generationInputs: { + version: 2, + fields: [{ title: 'prompt', value: '公开展示文本' }], + references: [], + }, + }), + }); + expect(screen.queryByRole('button', { name: '改造' })).toBeNull(); + expect(screen.getByRole('button', { name: '下载按钮' })).toBeTruthy(); + }); + it('removes raster edit actions for video and character animation layers', () => { renderSelectedToolbar({ selectedLayer: createLayer({ @@ -245,6 +285,12 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { src: '/generated/video.mp4', mediaType: 'video', assetKind: 'video', + generationInputs: { + version: 2, + action: 'video.generate', + fields: [{ id: 'prompt', title: '视频描述', value: '测试视频' }], + references: [], + }, }), }); const videoToolbar = screen.getByRole('toolbar', { name: '图片工具栏' }); @@ -268,6 +314,12 @@ describe('ImageCanvasSelectedLayerToolbarView', () => { src: '/generated/action/frame-01.png', mediaType: 'image-sequence', assetKind: 'character-animation', + generationInputs: { + version: 2, + action: 'character-animation.generate', + fields: [{ id: 'prompt', title: '动作描述', value: '挥手' }], + references: [], + }, }), }); const actionToolbar = screen.getByRole('toolbar', { name: '图片工具栏' }); diff --git a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx index b1a423324..b61b0dc6e 100644 --- a/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx +++ b/src/components/image-editor/ImageCanvasSelectedLayerToolbarView.tsx @@ -13,7 +13,10 @@ import type { CSSProperties } from 'react'; import { PlatformIconButton } from '../common/PlatformIconButton'; import { EditorIconButton } from './ImageCanvasEditorPrimitives'; import type { CanvasLayer } from './ImageCanvasEditorTypes'; -import { isQuickEditUnsupportedAssetKind } from './ImageCanvasGenerationModel'; +import { + canOpenRedrawPanel, + isQuickEditUnsupportedAssetKind, +} from './ImageCanvasGenerationModel'; type ImageCanvasSelectedLayerToolbarViewProps = { selectedLayer: CanvasLayer | null; @@ -47,6 +50,7 @@ export function ImageCanvasSelectedLayerToolbarView({ if (!selectedLayer || !selectedToolbarStyle) { return null; } + const canRedraw = canOpenRedrawPanel(selectedLayer); if (selectedLayer.mediaType === 'audio') { return ( @@ -57,15 +61,17 @@ export function ImageCanvasSelectedLayerToolbarView({ aria-label="素材工具栏" onPointerDown={(event) => event.stopPropagation()} > - } - onClick={() => onOpenRedrawPanel(selectedLayer)} - > - 改造 - + {canRedraw ? ( + } + onClick={() => onOpenRedrawPanel(selectedLayer)} + > + 改造 + + ) : null} 生成动画 ) : null} -