diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index 5cb81863d..238fdbdae 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -24,6 +24,14 @@ - 验证方式：运行宣发素材提交模型 / 面板测试、`api-server` 宣发素材模型锁定测试、前端类型检查、编码检查和 `git diff --check`。 - 关联文档：`docs/【编辑器】宣发素材工具演示入口设计-2026-06-17.md`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 +## 2026-06-22 创作主页精选只展示项目内画布生成资源 + +- 背景：`/creation` 的 `陶泥儿精选` 曾从账号级素材库读取，并在素材为空时用公开作品图片补充，导致新创作页出现不属于任何当前图片画布项目的素材。 +- 决策：`陶泥儿精选` 只从用户图片画布项目中的 `editor_project_resource` 构建素材包和素材，且只保留 `sourceType="generated"` 的画布生成资源；账号级 `editor_asset`、上传素材、公开作品图片和 `mock_generated` 都不作为精选来源。 +- 影响范围：`/creation` 创作主页、`creationShowcaseModel`、最近项目列表资源读取、创作主页改版计划和精选素材相关测试。 +- 验证方式：运行 `src/components/creation-home/creationShowcaseModel.test.ts` 与 `CreationLandingView.test.tsx`，确认项目内生成资源展示、上传素材过滤、公开作品不再 fallback。 +- 关联文档：`docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 + ## 2026-06-22 编辑器生成模型默认定价调整 - 背景：图片画布生成按钮、后端模型定价扣费和后台定价页需要统一使用新的模型默认泥点。 @@ -2632,3 +2640,11 @@ - 2026-06-22 补充：所有明确设置绿幕用于后续抠图的 prompt 都必须固定写明 `#00FF00 / RGB(0,255,0)`，不能只写“纯绿色绿幕”或“接近 #00FF00”；编辑器角色图通用抠图额外开启暗绿 / 灰绿绿幕背景识别，只作为生成模型偏离标准亮绿时的兜底。该宽松识别只参与从画布边缘连通扩散出的背景清理，不参与全图断开绿色区域删除，避免误伤角色衣物或纹理。 - 影响范围：`server-rs/crates/api-server/src/editor_project.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。 - 验证方式：`cargo test -p api-server --manifest-path server-rs/Cargo.toml editor_character_image_general_cutout`、`cargo test -p platform-image --manifest-path server-rs/Cargo.toml generated_asset_sheet_muted_green_alpha_requires_explicit_option`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding`、`git diff --check`。 + +## 2026-06-22 图片画布外部生成统一改走 worker 队列 + +- 背景：图片画布的图片、改图、图标素材、UI 素材提取、角色动作、视频和音频生成都可能长时间等待外部 provider；如果继续由 HTTP handler 同步执行，生产只能扩 API 进程，不能独立扩生成吞吐。 +- 决策：`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画板所有外部 provider 生成入口统一入 `external_generation_job`，job kind 使用 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。worker 成功后由后端写 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`；前端只轮询 BFF job 状态并重新读取项目快照，不从队列 payload 或本地临时状态重建完成图层。 +- 补充：带 `dialogId` 的 `canvasCompletion` 必须读取后端当前 layout 中的最新 generation dialog placeholder；等待期间用户移动占位时，结果层要跟随最新占位。无 dialog 的重绘 / UI 素材提取等入口使用明确的右侧完成占位；生成器已删除时不把结果重新塞回画布。 +- 影响范围：`server-rs/crates/api-server/src/editor_generation_queue.rs`、`server-rs/crates/api-server/src/external_generation_worker.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`。 +- 验证方式：`cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_canvas_generation --manifest-path server-rs/Cargo.toml`、`cargo test -p shared-contracts --manifest-path server-rs/Cargo.toml`、`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。 diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index c8fbefec2..be82d798d 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -87,6 +87,14 @@ - 验证：对应测试应断言生成按钮点击后 `dialog` 消失但 `image-canvas-editor__generation-frame--generating` 仍然存在。 - 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`。 +## 图片画布素材多时拖拽卡顿先查等距吸附候选规模 + +- 现象：画布素材数量增加后，拖拽单个图层或生成占位框时 pointermove 明显卡顿，关闭或绕开吸附后体感恢复。 +- 原因：边缘 / 中心线吸附是线性扫描，但等距吸附如果对所有可吸附素材做两两配对，会在素材数量上来后进入 O(n²) 热路径。 +- 处理：保留边缘 / 中心线全量线性扫描；等距吸附先过滤跨轴相交素材，再只检查轴向邻近候选，不要为远处或不相交素材生成配对候选。 +- 验证：`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/ImageCanvasInteractionModel.test.ts`，并在多素材画布拖拽时确认参考线仍能命中邻近图层且 pointermove 不再明显掉帧。 +- 关联：`src/components/image-editor/ImageCanvasEditorModel.ts`、`src/components/image-editor/ImageCanvasInteractionModel.ts`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 + ## 图片编辑器宣发素材生成器刷新后不要丢快照 - 现象：图片画布刷新后，宣发素材生成卡片消失，或卡片仍在但游戏名、分类、描述和参考图丢失。 @@ -135,6 +143,14 @@ - 验证：`cargo test -p api-server editor_canvas_generation_completion --manifest-path server-rs/Cargo.toml` 覆盖后端完成态写 layout；`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/services/image-editor/editorProjectClient.test.ts src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx -- --runInBand` 覆盖前端提交 `canvasCompletion`、应用后端快照、项目加载不推断完成态和 `objectKey` 资源创建不提交大 URL。 - 关联：`server-rs/crates/api-server/src/editor_project.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/components/image-editor/useImageCanvasProjectPersistence.ts`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 +## 图片画布发布入口 429 先查自动保存 PATCH 并发 + +- 现象：发布域名访问画板时出现短时间密集 `429`，Nginx access log 中 `PATCH /api/editor/projects/`、生成接口和资料接口混杂，429 行常见 `request_time=0.000`、`upstream_status=-`，error log 写 `limiting connections by zone "genarrative_api_conn"`。 +- 原因：这类 429 是入口 Nginx `limit_conn` 在转发前拒绝，不是 api-server、SpacetimeDB、worker 或 VectorEngine 的业务 429。画布自动保存如果只有防抖、没有 in-flight 串行保护，慢 `PATCH /api/editor/projects/{projectId}` 未完成时，拖拽生成器、资源回填和后续状态变化会继续发起新的保存请求，同一客户端连接数被长请求撑满后触发入口连接限流。 +- 处理：不要先放大 Nginx 限流或把错误归给生成 provider；先看 access log 的 `upstream_status` / `request_time` 和 error log 的 `limit_conn` zone，再查前端保存路径。`useImageCanvasProjectPersistence` 中自动保存和资源创建后的布局保存必须共用串行队列：同一时刻只允许一个 `saveEditorProjectLayout` in-flight，期间新快照覆盖旧待保存快照，当前保存结束后只发送最新一次。 +- 验证：`npm run test -- src/components/image-editor/useImageCanvasProjectPersistence.test.tsx -t "serializes project layout saves" --reporter verbose` 应覆盖慢保存期间不启动第二个 PATCH，首个保存完成后只发送最新待保存快照；排查发布现场时 429 行应从 `upstream_status=-` / Nginx `limit_conn` 收敛。 +- 关联：`src/components/image-editor/useImageCanvasProjectPersistence.ts`、`src/components/image-editor/useImageCanvasProjectPersistence.test.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 + ## 图片编辑器 Seedance 2.0 参考媒体不要提交视频 Data URL - 现象：画板生成视频选择 Seedance 2.0 并上传参考视频后，请求体暴涨、可能返回 `413` 或上游拒绝 `video_url.url`；文档示例或测试如果写 `data:video/mp4;base64,...`，后续实现很容易照抄。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 77528b436..c4384e66b 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -33,7 +33,9 @@ - 右上角提供快捷键入口，点击后打开独立快捷键弹窗；画布不常驻展示说明文案。Windows 快捷键覆盖 `Ctrl+Z` / `Ctrl+Shift+Z` 撤销重做、`Ctrl+A/C/V/X/D` 全选 / 复制 / 粘贴 / 剪切 / 复制一份、`Ctrl+0/1/+/-` 视图控制、`V/H/U/G/Shift+V/M` 工具切换、`Alt+1/Alt+2/Alt+M` 面板切换、`Ctrl+]` / `Ctrl+[` / `Ctrl+Shift+]` / `Ctrl+Shift+[` 层级调整、方向键微移 / `Shift+方向键` 大步移动、`Ctrl+Shift+S` 下载画布素材和 `F2` 重命名项目；快捷键只触发对应画布交互，不绕过既有保存 / 生成 / 上传工作流。 - 背景色控件只修改编辑器工作区底色，不恢复网格线或棋盘格底纹，也不影响图片本体。 - 吸附阈值以屏幕像素为准，换算到世界坐标后参与拖拽计算；边缘 / 中心线和等距吸附共用同一阈值。拖拽结束后只保存最终图层或生成占位布局，不保存临时参考线。 -- 画布自动保存使用防抖策略：图层拖拽、缩放、资源新增和修改结果创建后延迟保存工程快照。 +- 项目页封面和画布图片图层必须先渲染项目卡、图层外框、标题、尺寸和操作 chrome；图片换签或解码未完成时，只在图片区域显示轻量加载态，不阻塞外框和文字等低成本信息先出现。 +- 素材量增大时，拖拽吸附热路径不得对所有素材做全量两两配对。边缘 / 中心线吸附保持线性扫描；等距吸附只在跨轴相交且轴向邻近的候选图层之间计算，避免大量远处素材拖慢 pointermove。 +- 画布自动保存使用防抖 + 串行队列：图层拖拽、缩放、资源新增和修改结果创建后延迟保存工程快照；如果上一次 `PATCH /api/editor/projects/{projectId}` 尚未完成，只保留最新待保存快照，待当前请求结束后再发送下一次保存，避免慢保存请求并发堆积触发发布入口连接限流。 - 移动端保留同一套状态模型，底部工具栏可横向滚动，侧边栏默认可收起。 - 项目页卡片默认点击打开工程；hover 项目卡片右下角显示 `...` 菜单，菜单承载重命名和删除。选择模式下项目卡片只切换选中态，不进入画布；底部批量工具栏提供全选 / 取消全选、已选数量、批量删除和退出选择模式。 @@ -43,11 +45,11 @@ - 新增 `editor_canvas` 表保存工程下的画布：`canvasId`、`projectId`、`ownerUserId`、标题、viewport、图层布局 JSON、创建时间和更新时间。当前编辑器使用项目默认画布，后续可扩展为一个 project 下多个 canvas。 - 新增 `editor_asset_folder` 表保存账号级素材文件夹：`folderId`、`ownerUserId`、名称、排序、折叠状态、系统默认标记、创建时间和更新时间。素材文件夹不归属于 project，同一个账号进入任一项目都能看到。 - 新增 `editor_asset` 表保存账号级素材：`assetId`、`ownerUserId`、`folderId`、名称、图片读取地址、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、`assetKind`、`generationInputs`、创建时间和更新时间。素材只跟账号走，不跟 project 走；角色、图标、UI 设计图、视频和音频等生成结果的用户可见输入快照随素材保存。 -- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId；图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource，并把 `resourceId` 随生成响应返回给前端。图片生成请求如果同时携带 `canvasCompletion`（生成器 `dialogId`、标题和占位框），BFF 在生成成功后必须直接读取当前项目布局，只有当前布局仍存在对应 `generation-dialog` 时才插入轻量结果图层、把生成器标记为 `idle` 并写入 `generatedLayerId`，沿用后端当前 viewport 保存布局，再返回最新项目快照；前端只应用该快照刷新显示，不把生成完成态作为本地业务真相，也不在项目加载时根据资源行推断完成态。有项目上下文但后端没有返回项目快照时，前端不得本地补结果图层，只保留当前生成器交互状态等待下一次项目刷新。 +- `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId；图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource，并把 `resourceId` 随生成响应返回给前端。图片生成请求如果同时携带 `canvasCompletion`（生成器 `dialogId`、标题和占位框，或无 dialog 的右侧完成占位），BFF / worker 在生成成功后必须直接读取当前项目布局，优先使用最新 `generation-dialog` 占位框位置；只有当前布局仍存在对应 `generation-dialog` 时才插入轻量结果图层、把生成器标记为 `idle` 并写入 `generatedLayerId`，沿用后端当前 viewport 保存布局，再返回或刷新最新项目快照；前端只应用该快照刷新显示，不把生成完成态作为本地业务真相，也不在项目加载时根据资源行推断完成态。有项目上下文但后端没有返回项目快照时，前端不得本地补结果图层，只保留当前生成器交互状态等待下一次项目刷新。 - 图片文件本体继续走 OSS，浏览器读取私有 generated 对象仍经 `/api/assets/read-url` 换签。 - 当前 MVP 的本地上传先以 data URL 持久化在素材记录中，保证刷新和跨项目可见；后续接入正式 OSS 上传时，只替换 `imageSrc/objectKey/assetObjectId` 的写入方式，账号级素材表和画布资源表不变。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行；没有当前工程 ID 时才创建账号级 `editor_asset` 行，随后把对应 `resourceId` 或 `assetId` 写入参考图临时状态，生成请求仍使用临时状态中的图片源或 `objectKey`。 - 资源表保存资源和素材级元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON；刷新时优先从 `editor_project_resource` 恢复，旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；宣发素材生成器还必须保存并恢复 `publicationWorkflowId`、`publicationGameInfo` 和 `publicationReferences`，避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label，不保存图片 Data URL、signed URL 或 `objectKey`；刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引，只允许保存 `{ title, label, refType, refId }`；生成接口所需的图片 Data URL、signed URL 或 `objectKey` 只存在于提交前的临时参考图状态和请求体字段，不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。 -- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。带 `canvasCompletion` 的图片生成以 BFF 返回的 `project` 快照刷新画布；无项目上下文或没有返回快照时，前端才使用响应中的 resource / asset 快照做本地落画布兜底，不再把同一生成结果二次调用素材创建接口。视频结果当前只保存为画布视频资源，不进入图片素材库。 +- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`，前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照；`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底，不再把同一生成结果二次调用素材创建接口。视频结果当前只保存为画布视频资源，不进入图片素材库。 - 前端不直接订阅 SpacetimeDB，统一通过 api-server 的 `/api/editor/projects*` BFF 读写。 - 未登录用户可以使用本地演示态，但不触发工程自动保存；真实图片生成 / 修改需要登录。编辑器 API 请求允许使用 refresh cookie 静默补 access token，但 401 / 403 只在编辑器局部提示登录，不清空整站登录态，也不把后端 requestId 直接作为生图弹窗主文案。 @@ -112,7 +114,7 @@ - 素材面板支持选择模式框选，一次选中多个素材，并可批量移动或删除上传素材。 - 图层面板支持选择多个图层后创建图层组，组名和 groupId 随画布布局保存。 - 小地图支持拖拽视口框，拖动时画布 viewport 跟随移动。 -- 鼠标滚轮默认垂直滚动画布视口；按住 Ctrl / Cmd 滚轮才缩放画布，并阻止浏览器页面缩放。 +- 鼠标滚轮默认垂直滚动画布视口；按住 Ctrl / Cmd 滚轮才缩放画布，并阻止浏览器页面缩放。缩放比例显示保持现有换算口径，最低可缩小到 `5%`。 - 工程刷新后能从后端恢复资源、图层布局和 viewport。 - “我的”页项目入口能进入 `/project`；项目页能列出工程、重命名 / 删除单个工程、批量选择和批量删除；点击工程后进入 `/editor/canvas?projectid=` 并按 query 加载该工程。 diff --git a/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md b/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md index d5d6b2b23..709d3c21e 100644 --- a/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md +++ b/docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md @@ -1,6 +1,6 @@ # 外部生成 Worker 化方案 -更新时间：`2026-06-12` +更新时间：`2026-06-22` ## 背景 @@ -13,7 +13,7 @@ - 多个 worker 进程通过 SpacetimeDB 任务表抢占任务，依赖 lease 超时恢复，支持按进程数和单进程并发动态缩扩容。 - 本地或小流量同步排查可显式启用 `inline` 模式，由 HTTP handler 复用同一 worker executor 同步执行并返回 `completed`；该模式不创建队列任务，也不具备 worker 横向扩容能力。 - SpacetimeDB reducer / procedure 只做任务状态流转，不做网络、文件系统或外部 provider I/O。 -- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`；本轮扩展到跳一跳、拼消消和敲木鱼的外部图片生成动作。后续玩法继续复用同一队列 Module，不再为每个玩法发明独立队列。 +- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`，跳一跳、拼消消和敲木鱼的外部图片生成动作，以及图片画布编辑器的图片、改图、图标 spritesheet、UI 素材提取、角色动作、视频、音效和背景音乐生成。后续玩法和编辑器生成入口继续复用同一队列 Module，不再为每个入口发明独立队列。 - 第一版外部生成队列粒度固定为“单个用户动作对应单个 job”。例如草稿编译、结果页单槽重生、图集重生都各自入一个 job；job 内部可以串行或并行调用 provider、OSS、SpacetimeDB 写回，但不再拆成“提示词 / 生图 / 切图 / 去背景 / 持久化 / 回写”等阶段 job。阶段进度只作为 `request_payload_json` / 业务 session 的展示状态，不作为队列调度单位。 - 不调用外部图片 / 音频 / LLM provider 的动作继续 inline 执行，不为了统一排队而进入 `external_generation_job`。 @@ -48,7 +48,7 @@ BFF 只做鉴权、授权裁剪、字段脱敏和契约映射；队列事实仍 | --- | --- | | `job_id` | 主键，`extgen-` 前缀 UUID | | `dedupe_key` | 唯一键，建议为 `play/action/session/scope` | -| `job_kind` | 执行类型，当前拼图为 `puzzle_compile_draft`、`puzzle_generate_images`、`puzzle_generate_ui_background` | +| `job_kind` | 执行类型，当前覆盖 `puzzle_compile_draft`、`puzzle_generate_images`、`puzzle_generate_ui_background`、跳一跳 / 拼消消 / 敲木鱼生成动作，以及 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation`、`editor_background_music_generation` | | `owner_user_id` | 触发用户 | | `source_module` | 玩法或能力名，例如 `puzzle` | | `source_entity_id` | session/profile/work 等作用域 | @@ -161,6 +161,20 @@ controller 配置： 每个玩法迁移时必须同时接入业务写回 lease guard：worker 路径带 `external_generation_job_id / worker_id / lease_token`，inline 路径三项同时为空。过期 worker 不得写 session / work profile；业务失败态写回成功后才允许 job 进入 `failed`。 +### 图片画布编辑器 + +图片画布 `/editor/canvas` 下所有会调用外部生成 provider 的入口在 `queue` 模式下入 `external_generation_job`，HTTP handler 只返回 `queueState`： + +- `editor_image_generation`：普通图片、生成规范、角色形象、UI 设计图、宣发素材和图片快速编辑。 +- `editor_image_edit`：图片编辑 / 修改结果。 +- `editor_icon_spritesheet_generation`：图标素材 spritesheet 生成和拆分。 +- `editor_ui_design_asset_extraction`：UI 设计图红框素材提取。 +- `editor_character_animation_generation`：角色动作视频和帧素材生成。 +- `editor_video_generation`：画布视频生成和视频素材快速编辑。 +- `editor_sound_effect_generation` / `editor_background_music_generation`：画布音效与背景音乐。 + +画板结果的业务真相仍是 `editor_project_resource`、账号级 `editor_asset` 和 `editor_canvas.layers_json`。请求携带 `projectId + canvasCompletion` 时，worker 成功后读取当前项目 layout，用最新 generation dialog placeholder 或无 dialog 完成占位写入结果图层，并保存项目快照；前端轮询单 job 到 completed 后重新读取项目快照，不从队列 payload 或本地临时响应重建正式图层。生成器已被删除时，worker 只保留生成出的资源 / 素材记录，不把结果重新塞回画布。 + ## 验收 基础检查： diff --git a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md index 529aa8dff..b89419150 100644 --- a/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md +++ b/docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md @@ -245,7 +245,7 @@ npm run check:server-rs-ddd - Rust 结构体：`ExternalGenerationJob` - 源码：`server-rs/crates/spacetime-module/src/external_generation.rs` -- 用途：外部生成 worker 的持久任务队列；`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，`api-server` HTTP 角色只入队，`external-generation-worker` 角色通过 claim lease 领取、续租、执行，并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity，避免过期 worker 写 session / work profile；`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行，三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回，半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job；业务失败态写回成功后才能 fail job，失败态未写回时保留租约等待后续重领。 +- 用途：外部生成 worker 的持久任务队列；`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，`api-server` HTTP 角色只入队，`external-generation-worker` 角色通过 claim lease 领取、续租、执行，并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity，避免过期 worker 写 session / work profile；图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表，worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`，前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行，三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回，半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job；业务失败态写回成功后才能 fail job，失败态未写回时保留租约等待后续重领。 ### `ai_text_chunk` diff --git a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md index ac17e3b2d..6cffa142c 100644 --- a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md +++ b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md @@ -57,7 +57,7 @@ Windows 本地如果已在 `%LOCALAPPDATA%\Genarrative\ffmpeg\bin` 安装 FFmpeg 本地 `npm run dev` 和 `npm run dev:api-server` 默认保留 inline 开发体验：未显式设置 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，外部生成 handler 会同步复用 worker executor，完成后返回 `completed`，便于快速确认 provider、OSS 和 SpacetimeDB 写回链路。inline 不创建 `external_generation_job`，也不能验证 worker lease、队列等待展示或动态扩缩容。 -本地排查外部内容生成 worker 队列时，必须显式使用 queue，例如 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue GENARRATIVE_PROCESS_ROLE=all npm run dev:api-server`，让同一 Rust 进程同时监听 HTTP 并消费 `external_generation_job` 队列；更接近生产的验证应分别启动 `api`、`external-generation-worker` 和 `external-generation-controller`。生产默认 `GENARRATIVE_PROCESS_ROLE=api`，外部生成任务由独立 `GENARRATIVE_PROCESS_ROLE=external-generation-worker` 进程消费；生产与容器扩缩容验证保持 `queue`。当前进入持久队列的外部图片生成动作包括：拼图 `compile_puzzle_draft` / `generate_puzzle_images` / `generate_puzzle_ui_background`，跳一跳 `compile-draft` / `regenerate-tiles`，拼消消 `compile-draft` / `regenerate-atlas`，敲木鱼 `compile-draft` / `regenerate-hit-object`。非外部图片生成动作继续 inline，不进入队列。worker 数量为 0 时，HTTP 只返回 queued/running，不会兜底执行外部 provider。 +本地排查外部内容生成 worker 队列时，必须显式使用 queue，例如 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue GENARRATIVE_PROCESS_ROLE=all npm run dev:api-server`，让同一 Rust 进程同时监听 HTTP 并消费 `external_generation_job` 队列；更接近生产的验证应分别启动 `api`、`external-generation-worker` 和 `external-generation-controller`。生产默认 `GENARRATIVE_PROCESS_ROLE=api`，外部生成任务由独立 `GENARRATIVE_PROCESS_ROLE=external-generation-worker` 进程消费；生产与容器扩缩容验证保持 `queue`。当前进入持久队列的外部生成动作包括：拼图 `compile_puzzle_draft` / `generate_puzzle_images` / `generate_puzzle_ui_background`，跳一跳 `compile-draft` / `regenerate-tiles`，拼消消 `compile-draft` / `regenerate-atlas`，敲木鱼 `compile-draft` / `regenerate-hit-object`，以及图片画布 `editor_image_generation` / `editor_image_edit` / `editor_icon_spritesheet_generation` / `editor_ui_design_asset_extraction` / `editor_character_animation_generation` / `editor_video_generation` / `editor_sound_effect_generation` / `editor_background_music_generation`。非外部 provider 生成动作继续 inline，不进入队列。worker 数量为 0 时，HTTP 只返回 queued/running，不会兜底执行外部 provider。 `我的` 页签或排障面板展示队列等待时，只读取 BFF 队列接口：`GET /api/runtime/external-generation/queue-overview` 查看当前用户可见队列概览，`GET /api/runtime/external-generation/jobs/{jobId}` 查看单 job 状态。生成页 / 进度页不承接队列概览，只展示当前玩法业务进度；队列接口只提供等待 / 运行 / 失败 / 完成状态补充，最终草稿、作品和结果页仍要轮询对应玩法 session/detail 接口收敛到 ready 或 failed；不要直接查询 `external_generation_job` private table，也不要把 worker 内部 payload 暴露到前端。 @@ -387,7 +387,7 @@ Pingora current release 自审脚本 `scripts/ops/pingora-current-release-audit. 生产环境变量模板：`deploy/env/api-server.env.example`。真实密钥只放服务器，不提交 Git，不写入文档示例。 -`api-server` 进程角色由 `GENARRATIVE_PROCESS_ROLE` 控制：`api` 只监听 HTTP，`external-generation-worker` 只消费外部生成队列，`external-generation-controller` 只管理 worker systemd 实例，`all` 仅用于本地或临时 smoke，不隐式启动 controller。外部生成策略由 `GENARRATIVE_EXTERNAL_GENERATION_MODE` 控制；生产和容器压测默认保持 `queue`，本地 `npm run dev` 默认保留 `inline` 开发体验，只有显式配置 `queue` 才会落 `external_generation_job`。`inline` 只用于本地或低并发同步排查，HTTP handler 会直接复用 worker executor，完成后返回 `completed`，但不会落 `external_generation_job`，也不能通过增加 worker 进程扩吞吐。外部生成 worker 使用同一发布包和同一套 SpacetimeDB 配置，按实例数和 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_CONCURRENCY` 动态扩缩；生产默认由 `genarrative-external-generation-controller.service` 读取 `get_external_generation_queue_stats_and_return`，按 `claimable_pending + running_active + expired_running` 计算目标 worker 数，并对 `genarrative-external-generation-worker@N.service` 精确执行 `systemctl start/stop`。controller 参数模板是 `deploy/env/external-generation-controller.env.example`：默认保底 `MIN_WORKERS=1`、上限 `MAX_WORKERS=8`、每 worker 目标 `TARGET_JOBS_PER_WORKER=2`、`POLL_INTERVAL_MS=10000`、连续 `SCALE_DOWN_IDLE_ROUNDS=6` 轮完全空闲才缩容；缩容每轮只停止最高编号的一个实例，且不主动停止 `@1`。worker 收到 SIGINT/SIGTERM 后会停止 claim 新任务并等待当前任务完成；若进程被硬杀、机器断电或超过 systemd `TimeoutStopSec`，未完成任务才会在 lease 过期后由其它 worker 重领。每个 worker 实例应设置唯一 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_ID`，默认会用主机名和 pid 兜底；systemd 生产模板 `deploy/systemd/genarrative-external-generation-worker@.service` 会用 `%H-%i` 生成实例 ID，并把 tracking outbox 隔离到 `/var/lib/genarrative/tracking-outbox/%H-%i`。`Genarrative-Server-Provision` 会安装 worker 模板、controller unit 和两份专属 env 模板，默认 enable 首个 `genarrative-external-generation-worker@1.service` 与 `genarrative-external-generation-controller.service`；首次 API deploy 会在默认 worker pattern 下自动 `enable --now genarrative-external-generation-worker@1.service` 并等待 worker active，同时重启并验活 controller。手动兜底扩容仍可用 `systemctl start genarrative-external-generation-worker@2.service` / `@3.service`，缩容用 `systemctl stop genarrative-external-generation-worker@N.service`；controller 下轮会按队列压力修正到目标实例数。worker 专属参数模板是 `deploy/env/external-generation-worker.env.example`，密钥与 SpacetimeDB 连接仍复用 `/etc/genarrative/api-server.env`。API 发布脚本默认会重启并验活 `genarrative-external-generation-worker@*.service` 和 `genarrative-external-generation-controller.service`；若本次只发 HTTP 且不希望滚动 worker，可传 `--no-worker-services`，若不希望重启 controller 可传 `--no-worker-controller`。`GENARRATIVE_EXTERNAL_GENERATION_WORKER_POLL_INTERVAL_MS` 控制空队列轮询间隔，`GENARRATIVE_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS` 控制单次 lease，worker 会约每三分之一 lease、最长 30 秒续租；该值应覆盖一次心跳网络抖动窗口，不需要大于完整外部生成链路耗时。SpacetimeDB 使用自身事务时间计算 claim/renew/complete/fail，完成和失败回写还会校验 `lease_token` 与未过期 lease，避免同一 job 被过期 worker 覆盖。首版 worker 粒度是单动作单 job，不拆阶段 job；当前外部图片生成动作覆盖拼图、跳一跳、拼消消和敲木鱼，纯元信息保存、发布、试玩启动、运行态动作和公开读取继续 inline。当前生成业务失败只做用户重新触发，不做自动业务重试，避免 worker 退款和重试成功之间产生钱包账本漂移。 +`api-server` 进程角色由 `GENARRATIVE_PROCESS_ROLE` 控制：`api` 只监听 HTTP，`external-generation-worker` 只消费外部生成队列，`external-generation-controller` 只管理 worker systemd 实例，`all` 仅用于本地或临时 smoke，不隐式启动 controller。外部生成策略由 `GENARRATIVE_EXTERNAL_GENERATION_MODE` 控制；生产和容器压测默认保持 `queue`，本地 `npm run dev` 默认保留 `inline` 开发体验，只有显式配置 `queue` 才会落 `external_generation_job`。`inline` 只用于本地或低并发同步排查，HTTP handler 会直接复用 worker executor，完成后返回 `completed`，但不会落 `external_generation_job`，也不能通过增加 worker 进程扩吞吐。外部生成 worker 使用同一发布包和同一套 SpacetimeDB 配置，按实例数和 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_CONCURRENCY` 动态扩缩；生产默认由 `genarrative-external-generation-controller.service` 读取 `get_external_generation_queue_stats_and_return`，按 `claimable_pending + running_active + expired_running` 计算目标 worker 数，并对 `genarrative-external-generation-worker@N.service` 精确执行 `systemctl start/stop`。controller 参数模板是 `deploy/env/external-generation-controller.env.example`：默认保底 `MIN_WORKERS=1`、上限 `MAX_WORKERS=8`、每 worker 目标 `TARGET_JOBS_PER_WORKER=2`、`POLL_INTERVAL_MS=10000`、连续 `SCALE_DOWN_IDLE_ROUNDS=6` 轮完全空闲才缩容；缩容每轮只停止最高编号的一个实例，且不主动停止 `@1`。worker 收到 SIGINT/SIGTERM 后会停止 claim 新任务并等待当前任务完成；若进程被硬杀、机器断电或超过 systemd `TimeoutStopSec`，未完成任务才会在 lease 过期后由其它 worker 重领。每个 worker 实例应设置唯一 `GENARRATIVE_EXTERNAL_GENERATION_WORKER_ID`，默认会用主机名和 pid 兜底；systemd 生产模板 `deploy/systemd/genarrative-external-generation-worker@.service` 会用 `%H-%i` 生成实例 ID，并把 tracking outbox 隔离到 `/var/lib/genarrative/tracking-outbox/%H-%i`。`Genarrative-Server-Provision` 会安装 worker 模板、controller unit 和两份专属 env 模板，默认 enable 首个 `genarrative-external-generation-worker@1.service` 与 `genarrative-external-generation-controller.service`；首次 API deploy 会在默认 worker pattern 下自动 `enable --now genarrative-external-generation-worker@1.service` 并等待 worker active，同时重启并验活 controller。手动兜底扩容仍可用 `systemctl start genarrative-external-generation-worker@2.service` / `@3.service`，缩容用 `systemctl stop genarrative-external-generation-worker@N.service`；controller 下轮会按队列压力修正到目标实例数。worker 专属参数模板是 `deploy/env/external-generation-worker.env.example`，密钥与 SpacetimeDB 连接仍复用 `/etc/genarrative/api-server.env`。API 发布脚本默认会重启并验活 `genarrative-external-generation-worker@*.service` 和 `genarrative-external-generation-controller.service`；若本次只发 HTTP 且不希望滚动 worker，可传 `--no-worker-services`，若不希望重启 controller 可传 `--no-worker-controller`。`GENARRATIVE_EXTERNAL_GENERATION_WORKER_POLL_INTERVAL_MS` 控制空队列轮询间隔，`GENARRATIVE_EXTERNAL_GENERATION_WORKER_LEASE_SECONDS` 控制单次 lease，worker 会约每三分之一 lease、最长 30 秒续租；该值应覆盖一次心跳网络抖动窗口，不需要大于完整外部生成链路耗时。SpacetimeDB 使用自身事务时间计算 claim/renew/complete/fail，完成和失败回写还会校验 `lease_token` 与未过期 lease，避免同一 job 被过期 worker 覆盖。首版 worker 粒度是单动作单 job，不拆阶段 job；当前外部生成动作覆盖拼图、跳一跳、拼消消、敲木鱼和图片画布编辑器生成入口，纯元信息保存、发布、试玩启动、运行态动作和公开读取继续 inline。图片画布 worker 成功后由后端保存 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`，前端只轮询 job 并重新读取项目快照。当前生成业务失败只做用户重新触发，不做自动业务重试，避免 worker 退款和重试成功之间产生钱包账本漂移。 `Genarrative-Server-Provision` 会安装 systemd 模板和 Nginx 站点模板，不再安装 clang / lld / pkg-config / OpenSSL headers / sccache 等通用构建链依赖。因 VectorEngine 图片上游 POST 已改用 `libcurl`，当前 Linux release 构建出的 `api-server` 运行时需要 `OPENSSL_3.2.0` 符号；Ubuntu 24.04 apt 默认只提供 OpenSSL 3.0.x，不能直接满足该符号版本。Provision 会把 OpenSSL `3.2.0` 独立安装到 `/opt/genarrative/openssl-3.2.0`，校验官方 tarball SHA256，并只通过 `genarrative-api.service` 的 `LD_LIBRARY_PATH=/opt/genarrative/openssl-3.2.0/lib64:/opt/genarrative/openssl-3.2.0/lib` 让 api-server 使用，避免替换系统 OpenSSL 或影响 ssh / nginx / apt。Ubuntu / apt 目标机为完成这一步会安装 `build-essential`、`ca-certificates`、`curl`、`perl`、`tar` 等 OpenSSL 运行时自举工具；这只服务于独立 OpenSSL 运行时安装，不代表 provision 重新承担 api-server 构建职责。Ubuntu / apt 目标机会额外安装 `libnginx-mod-http-brotli-filter` 与 `libnginx-mod-http-brotli-static`，随后由 `scripts/jenkins-server-provision.sh` 通过临时 `nginx -t` 配置探测 Brotli 指令是否可用；该临时配置必须先 `include /etc/nginx/modules-enabled/*.conf`，因为 apt 安装的 Brotli 是动态模块，不会出现在普通 `nginx -V` 编译参数里。探测成功才在渲染后的 `deploy/nginx/genarrative.conf` / `genarrative-dev-http.conf` 中启用 Brotli，避免未安装模块的机器直接写入无效配置。Provision 写入 Genarrative Nginx 站点时会把 `/etc/nginx/sites-enabled/default*` 移到 `/etc/nginx/sites-disabled/`，避免 Debian / Certbot 默认站点继续占用 `genarrative.world` / `www.genarrative.world` 并在 `nginx -T` 中出现 `conflicting server name ... ignored`。如果 `nginx -t` 失败，脚本会恢复写入前的 Genarrative 配置和被移动的默认站点。 diff --git a/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md b/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md index 363ac3602..d8b7d5064 100644 --- a/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md +++ b/docs/【玩法创作】创作主页与项目入口改版计划-2026-06-18.md @@ -15,7 +15,7 @@ - 顶级导航中“创作”进入 `/creation`，“草稿”改为“项目”并进入 `/project`。 - 移动端隐藏“创作”和“项目”入口，只保留浏览和个人相关入口；移动端直达 `/creation` 时提示用户使用桌面端打开。 - 登录后在创作主页展示最近项目，并通过新建项目进入 `/editor/canvas?projectid=xxx`。 -- `陶泥儿精选` 展示真实账号素材瀑布流，不使用 mock 素材。 +- `陶泥儿精选` 只展示已经存在于用户图片画布项目中的画布生成素材包和素材，不使用账号级非项目素材、公开作品补充或 mock 素材。 - 现有 `/creation/` 玩法工作台、草稿、作品架、生成恢复和发布链路保持不变。 ## 页面结构 @@ -61,9 +61,9 @@ ### 陶泥儿精选 -`陶泥儿精选` 是页面底部的用户素材瀑布流，不承载玩法入口列表。瀑布流卡片按真实素材宽高设置预览比例，同一行允许出现不同高度卡片，不使用固定等高网格。创作入口配置仍继续来自 `/api/creation-entry/config`，供旧创作入口和具体 `/creation/` 工作台使用，但不作为本页精选区内容。 +`陶泥儿精选` 是页面底部的项目内画布生成素材瀑布流，不承载玩法入口列表。瀑布流卡片按真实素材宽高设置预览比例，同一行允许出现不同高度卡片，不使用固定等高网格。创作入口配置仍继续来自 `/api/creation-entry/config`，供旧创作入口和具体 `/creation/` 工作台使用，但不作为本页精选区内容。 -精选内容优先使用真实账号级素材和生成结果数据；未登录或账号素材为空时，可使用已公开作品中真实用户生成的图片作为瀑布流素材来源。两种来源都不使用 mock。若现有数据缺少提示词、作者名或成本字段，v1 显示保守占位，不伪造内容。 +精选内容只使用 `editor_project_resource` 中 `sourceType="generated"` 的项目资源，代表用户通过图片画布生成并已经落入某个项目的素材。账号级 `editor_asset` 只表示跨项目素材库，不单独作为精选来源；上传素材、公开作品图片和 `mock_generated` 资源都不进入精选。若现有数据缺少提示词、作者名或成本字段，v1 显示保守占位，不伪造内容。 Tab： @@ -100,9 +100,10 @@ Tab： 数据来源： -- 图片类素材优先读取账号级素材库和已落库生成结果。 -- 未登录或账号素材为空时，读取已公开作品中的真实用户生成图片作为精选素材补充，但卡片仍按素材瀑布流呈现，不展示玩法入口卡。 -- 音乐、音效、视频类素材只展示已存在的真实记录。 +- 读取用户项目列表中的 `editor_project_resource` 快照，只保留 `sourceType="generated"` 的画布生成资源。 +- 素材包、角色、UI、音乐、音效、视频和美宣都必须来自已存在项目资源；素材库里尚未放入任何项目的账号级素材不展示。 +- 未登录、项目为空或项目内没有画布生成资源时，不再用公开作品图片补充，只显示简洁空态。 +- 上传素材、公开作品图片、mock 资源和假组合都不进入精选。 - 不为了填满展示区创建假素材、假作者、假泥点成本或假组合关系。 - 暂无真实数据的 Tab 保留 Tab 入口，但内容区显示简洁空态。 @@ -113,7 +114,7 @@ Tab： - 最近项目继续使用编辑器项目接口：`listEditorProjects`、`createEditorProject` 和 `/editor/canvas?projectid=xxx`。 - 最近项目和项目页打开画布时，浏览器 history 只保留带 `projectid` 的最终画布路由。 - 项目封面逻辑复用 `/project` 项目卡已有的画布中心缩略图算法。 -- `陶泥儿精选` 读取账号级素材库；若字段缺失，前端只显示空值或隐藏对应元信息，不在前端推导业务真相。 +- `陶泥儿精选` 读取项目内画布生成资源；若字段缺失，前端只显示空值或隐藏对应元信息，不在前端推导业务真相。 - `/creation/` 的玩法工作台、草稿、生成页、结果页、发布、运行态和作品架链路保持原状。 ## 路由与导航 @@ -128,6 +129,7 @@ Tab： - 桌面端从 `/creation` 切换到 `/project` 后，“项目”页签进入选中态，“创作”页签必须恢复为普通默认态，不保留创作主页的主按钮强调表现。 - `/project` 项目页头部必须提供 `返回主站` 按钮，点击后切回平台首页 `/`，行为与桌面侧栏切回推荐页保持一致。 - 移动端导航列表不展示“创作”和“项目”。 +- 移动端首次进入或刷新 `/` 平台首页时显示欢迎弹窗，文案提示移动端仅支持作品展示，体验创作工具请使用电脑端访问；点击 `好` 关闭后，本次页面生命周期内切换下方页签再回到首页不再弹出。 - 移动端直接访问 `/creation` 时显示桌面端打开引导，不加载创作工具主页；`/creation/` 继续直达对应玩法工作台。`/project` 暂不新增移动端拦截。 ## 实施拆分 @@ -194,11 +196,12 @@ git diff --check - 桌面导航显示“项目”，不显示“草稿”。 - `/project` 头部点击“返回主站”回到 `/`，并退出项目页主内容。 - 移动视口底部不显示“创作”和“项目”。 +- 移动端进入 `/` 首页时显示 `欢迎` 弹窗，正文包含移动端仅支持作品展示和电脑端访问提示，按钮为 `好`。 - 移动端直达 `/creation` 显示“请在桌面端打开创作主页”，点击“返回首页”回到 `/`。 - “我的”页没有项目快捷入口。 -- 陶泥儿精选素材瀑布流不出现 mock 素材、假作者或假泥点成本。 +- 陶泥儿精选素材瀑布流不出现账号级非项目素材、公开作品补充、上传素材、mock 素材、假作者或假泥点成本。 - 陶泥儿精选素材卡按真实素材宽高呈现瀑布流，不退化为固定等高网格。 -- 陶泥儿精选不出现玩法入口卡；即使使用公开作品图片补充展示，也必须呈现为用户素材瀑布流。 +- 陶泥儿精选不出现玩法入口卡或公开作品图片补充展示。 ## 非目标 diff --git a/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md b/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md index 66c975cff..0dd894415 100644 --- a/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md +++ b/docs/【玩法创作】平台入口与玩法链路-2026-05-15.md @@ -202,7 +202,7 @@ RPG / 拼图等运行态存档仍以 `/api/profile/save-archives` 的后端列 跳一跳作品架删除入口必须走 `/api/creation/jump-hop/works/{profile_id}`，并通过 SpacetimeDB 同步删除 work profile、源 session、运行态 run 与事件，再刷新作品架和公开广场；不得只做前端本地隐藏。 -推荐页匿名游玩不再限定为跳一跳。移动端一级 `推荐` Tab 是内嵌运行态刷卡流，会自动选择推荐作品并启动对应玩法；桌面端首页不启动这套移动推荐运行态，而是渲染桌面发现壳，展示 `今日游戏`、`推荐`、`作品分类` 等桌面内容。推荐页候选顺序由前端轻量推荐算法 `platformRecommendation.ts` 统一生成：先按公开作品 key 去重，再使用公开读模型已有的精选来源、近 7 日游玩、点赞、改造、总游玩、发布时间新鲜度、封面和标签完整度做确定性评分，最后优先交错不同玩法类型；只要还有其它玩法候选，就不要连续推荐同一玩法，只有候选池已没有其它玩法时才允许同玩法相邻。该算法不得新增前端业务真相或绕过公开作品 read model。断点事实统一走 `platformEntryResponsive.ts` 的 `usePlatformDesktopLayout()`，平台壳和首页视图必须共用同一个判断，避免桌面发现页与移动推荐页同时挂载、重复触发请求或启动运行态。移动端推荐页拿到推荐作品列表后必须预加载每个作品的卡片封面、主封面和玩法兜底封面；启动或切换作品时先展示当前带玩法标签和标题的作品卡面遮罩，嵌入 runtime 在卡面下层加载，不得再从卡面闪切到另一层单独纯封面图。作品切换提交后，当前 runtime 遮罩接手已在屏幕上的卡面时必须瞬时贴合，不允许再执行“卡面到同一卡面”的淡入或重绘过渡；推荐页 runtime 必须通过统一 `ready` 门控等待对应运行态 run / profile、lazy runtime 组件和 runtime DOM 内图片资源都准备好，且必须持续观察后续新增图片、内联 `background-image` 和换签中的资源标记，不能只在首次挂载时扫描主图或封面；`ready` 返回 `true` 后才由外层放开游戏画面并只让卡面遮罩渐隐。遮罩层级必须高于并隔离下层 runtime，防止运行态 HUD、canvas 或高 `z-index` 子层穿透到封面上；ready 前不展示“加载中”文案，但封面内必须保留无文案加载动效或进度条，避免用户误以为卡片损坏，也不得把未准备好的运行态直接暴露给用户。切换推荐作品时，如果上一条作品的启动请求、退出收口或目标玩法 busy 状态尚未结束，应继续显示当前作品卡面遮罩并等待下一轮自动启动；只有目标作品启动明确失败时，才显示“作品暂时无法进入，请稍后再试。”这类失败态。推荐页内拼图通关后的同 run 相似作品推进不视为推荐作品切换，不能重新显示启动封面；如果需要跨公开作品进入下一关，则必须走推荐页统一切卡入口，不能复用拼图 runtime 的跨作品 handoff，也不能直接把当前 run 改写到另一个作品，`activeRecommendEntryKey` 只能由推荐页统一选择下一作品后更新。推荐页嵌入运行态启动时按真实身份分流：已登录用户或本地已有 access token 时继续使用账号 Bearer，但请求选项必须是 local auth impact，避免单卡 401 清空整站登录态；只有确认为匿名访客时才申请短期 Runtime Guest Token，并只把它作为局部请求头传给运行态客户端，不写入全局登录态、不触发 refresh，也不把匿名流量伪装成普通用户。当前覆盖矩阵为：跳一跳、视觉小说、抓大鹅 Match3D、方洞挑战、拼图、敲木鱼、大鱼吃小鱼、汪汪声浪。每个模板的启动请求、推荐页内后续运行态动作以及需要上报的 play/finish/leaderboard/next-level 类请求，都必须继续按该身份分流；公开读取入口仍可匿名读取，创作、个人作品、删除、发布、Remix 等账号/所有权动作仍保持普通用户鉴权。推荐 runtime 的 `none` / `background` / `runtime-guest` 请求计划和拼图 `default` / `isolated` runtime auth mode 由 `platformRecommendRuntimeAuthModel.ts` 统一判定，平台壳只负责读取 token、申请 Runtime Guest Token 和传递 request options。推荐 runtime 自动启动只由 `platformPublicGalleryFlow.ts` 输出 `noop` / `clear` / `start(entry)` 决策，平台壳只执行清空 state 或启动指定作品。 +推荐页匿名游玩不再限定为跳一跳。移动端一级 `推荐` Tab 是内嵌运行态刷卡流，会自动选择推荐作品并启动对应玩法；移动端首次进入或刷新 `/` 平台首页时弹出一次欢迎提示，说明移动端仅支持作品展示、体验创作工具需使用电脑端访问，用户点击 `好` 后关闭；本次页面生命周期内切换下方页签再回到首页不再弹出。桌面端首页不启动这套移动推荐运行态，而是渲染桌面发现壳，展示 `今日游戏`、`推荐`、`作品分类` 等桌面内容。推荐页候选顺序由前端轻量推荐算法 `platformRecommendation.ts` 统一生成：先按公开作品 key 去重，再使用公开读模型已有的精选来源、近 7 日游玩、点赞、改造、总游玩、发布时间新鲜度、封面和标签完整度做确定性评分，最后优先交错不同玩法类型；只要还有其它玩法候选，就不要连续推荐同一玩法，只有候选池已没有其它玩法时才允许同玩法相邻。该算法不得新增前端业务真相或绕过公开作品 read model。断点事实统一走 `platformEntryResponsive.ts` 的 `usePlatformDesktopLayout()`，平台壳和首页视图必须共用同一个判断，避免桌面发现页与移动推荐页同时挂载、重复触发请求或启动运行态。移动端推荐页拿到推荐作品列表后必须预加载每个作品的卡片封面、主封面和玩法兜底封面；启动或切换作品时先展示当前带玩法标签和标题的作品卡面遮罩，嵌入 runtime 在卡面下层加载，不得再从卡面闪切到另一层单独纯封面图。作品切换提交后，当前 runtime 遮罩接手已在屏幕上的卡面时必须瞬时贴合，不允许再执行“卡面到同一卡面”的淡入或重绘过渡；推荐页 runtime 必须通过统一 `ready` 门控等待对应运行态 run / profile、lazy runtime 组件和 runtime DOM 内图片资源都准备好，且必须持续观察后续新增图片、内联 `background-image` 和换签中的资源标记，不能只在首次挂载时扫描主图或封面；`ready` 返回 `true` 后才由外层放开游戏画面并只让卡面遮罩渐隐。遮罩层级必须高于并隔离下层 runtime，防止运行态 HUD、canvas 或高 `z-index` 子层穿透到封面上；ready 前不展示“加载中”文案，但封面内必须保留无文案加载动效或进度条，避免用户误以为卡片损坏，也不得把未准备好的运行态直接暴露给用户。切换推荐作品时，如果上一条作品的启动请求、退出收口或目标玩法 busy 状态尚未结束，应继续显示当前作品卡面遮罩并等待下一轮自动启动；只有目标作品启动明确失败时，才显示“作品暂时无法进入，请稍后再试。”这类失败态。推荐页内拼图通关后的同 run 相似作品推进不视为推荐作品切换，不能重新显示启动封面；如果需要跨公开作品进入下一关，则必须走推荐页统一切卡入口，不能复用拼图 runtime 的跨作品 handoff，也不能直接把当前 run 改写到另一个作品，`activeRecommendEntryKey` 只能由推荐页统一选择下一作品后更新。推荐页嵌入运行态启动时按真实身份分流：已登录用户或本地已有 access token 时继续使用账号 Bearer，但请求选项必须是 local auth impact，避免单卡 401 清空整站登录态；只有确认为匿名访客时才申请短期 Runtime Guest Token，并只把它作为局部请求头传给运行态客户端，不写入全局登录态、不触发 refresh，也不把匿名流量伪装成普通用户。当前覆盖矩阵为：跳一跳、视觉小说、抓大鹅 Match3D、方洞挑战、拼图、敲木鱼、大鱼吃小鱼、汪汪声浪。每个模板的启动请求、推荐页内后续运行态动作以及需要上报的 play/finish/leaderboard/next-level 类请求，都必须继续按该身份分流；公开读取入口仍可匿名读取，创作、个人作品、删除、发布、Remix 等账号/所有权动作仍保持普通用户鉴权。推荐 runtime 的 `none` / `background` / `runtime-guest` 请求计划和拼图 `default` / `isolated` runtime auth mode 由 `platformRecommendRuntimeAuthModel.ts` 统一判定，平台壳只负责读取 token、申请 Runtime Guest Token 和传递 request options。推荐 runtime 自动启动只由 `platformPublicGalleryFlow.ts` 输出 `noop` / `clear` / `start(entry)` 决策，平台壳只执行清空 state 或启动指定作品。 ## 敲木鱼 diff --git a/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md b/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md index ad2ae3fa2..a134012c5 100644 --- a/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md +++ b/docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md @@ -10,6 +10,7 @@ - 底部 AI 画布工具栏新增 `生成图标素材` 按钮。 - 点击后立即在画布中心创建图标素材占位图，不复用普通“单张空白图片”图标；占位图表现为一叠空白素材图标卡片。 +- 图标素材占位图使用 `360x360` 的画布展示尺寸和 `512x512` 的原始图集尺寸；面板中的模型、比例和尺寸仍按生成契约独立提交，不用通用图片生成的 `1K` 画布外框。 - 图标素材面板锚定在占位图下方，和现有生成输入框同一层级展示。 - 生成完成后删除占位态，先把后端返回的 spritesheet 原图作为 `assetKind: "icon-spritesheet"` 的图集图层放到画布，再把拆分出的每个独立图标素材作为画布图片图层铺开，图集与图标之间、图标彼此之间不重叠，并保留少量间距。 - 图标素材图层写入 `assetKind: "icon"`；图标规范图写入 `assetKind: "icon-spec"`，用于刷新后保留标签和限制点选来源。 diff --git a/public/branding/mobile-home-welcome-taonier-ip.png b/public/branding/mobile-home-welcome-taonier-ip.png new file mode 100644 index 000000000..acb8e7deb Binary files /dev/null and b/public/branding/mobile-home-welcome-taonier-ip.png differ diff --git a/server-rs/crates/api-server/src/app.rs b/server-rs/crates/api-server/src/app.rs index 55e3d04d7..e86aabf9a 100644 --- a/server-rs/crates/api-server/src/app.rs +++ b/server-rs/crates/api-server/src/app.rs @@ -283,7 +283,10 @@ mod tests { use tokio::net::TcpListener; use tower::ServiceExt; - use crate::{config::AppConfig, state::AppState}; + use crate::{ + config::{AppConfig, ExternalGenerationMode}, + state::AppState, + }; use super::{build_router, build_spacetime_unavailable_router}; @@ -1472,7 +1475,11 @@ mod tests { #[tokio::test] async fn editor_character_animation_accepts_character_image_body_above_default_limit() { - let state = AppState::new(AppConfig::default()).expect("state should build"); + let state = AppState::new(AppConfig { + external_generation_mode: ExternalGenerationMode::Inline, + ..AppConfig::default() + }) + .expect("state should build"); let seed_user = seed_phone_user_with_password(&state, "13800138190", TEST_PASSWORD).await; let token = sign_test_user_token(&state, &seed_user, "sess_editor_character_body"); let app = build_router(state); @@ -1554,7 +1561,11 @@ mod tests { #[tokio::test] async fn editor_ui_design_asset_extraction_accepts_image_body_above_default_limit() { - let state = AppState::new(AppConfig::default()).expect("state should build"); + let state = AppState::new(AppConfig { + external_generation_mode: ExternalGenerationMode::Inline, + ..AppConfig::default() + }) + .expect("state should build"); let seed_user = seed_phone_user_with_password(&state, "13800138188", TEST_PASSWORD).await; let token = sign_test_user_token(&state, &seed_user, "sess_editor_ui_extract_body"); let app = build_router(state); @@ -1599,7 +1610,11 @@ mod tests { #[tokio::test] async fn editor_image_generation_accepts_reference_body_above_default_limit() { - let state = AppState::new(AppConfig::default()).expect("state should build"); + let state = AppState::new(AppConfig { + external_generation_mode: ExternalGenerationMode::Inline, + ..AppConfig::default() + }) + .expect("state should build"); let seed_user = seed_phone_user_with_password(&state, "13800138028", TEST_PASSWORD).await; let token = sign_test_user_token(&state, &seed_user, "sess_editor_image_reference_body"); let app = build_router(state); diff --git a/server-rs/crates/api-server/src/character_animation_assets.rs b/server-rs/crates/api-server/src/character_animation_assets.rs index 3b552054d..fc3a325d3 100644 --- a/server-rs/crates/api-server/src/character_animation_assets.rs +++ b/server-rs/crates/api-server/src/character_animation_assets.rs @@ -1,4 +1,4 @@ -use std::{ +use std::{ collections::BTreeMap, fs, path::{Path, PathBuf}, @@ -55,6 +55,15 @@ use crate::{ build_character_animation_prompt, build_fallback_moderation_safe_animation_prompt, }, editor_generation_config::EditorGenerationPricingConfig, + editor_generation_queue::{ + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, + EditorGenerationQueuedResponse, editor_generation_queue_state, + editor_generation_source_entity_id, enqueue_editor_generation_job, + }, + editor_project::{ + EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item, + complete_editor_canvas_generation_with_items, + }, http_error::AppError, platform_errors::map_oss_error, prompt::role_asset_studio::{ @@ -525,11 +534,44 @@ pub async fn generate_editor_character_animation( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, Response> { + let Json(payload) = payload.map_err(|error| { + character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-character-animation", + "message": error.body_text(), + })), + ) + })?; + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + payload.source_layer_id.as_str(), + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + owner_user_id.as_str(), + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成角色动作", + &payload, + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } generate_editor_character_animation_for_owner( state, request_context, - authenticated.claims().user_id().to_string(), - payload, + owner_user_id, + Ok(Json(payload)), ) .await } @@ -549,6 +591,10 @@ pub(crate) async fn generate_editor_character_animation_for_owner( })), ) })?; + let project_id = payload.project_id.clone(); + let canvas_completion = payload.canvas_completion.clone(); + let generation_inputs = payload.generation_inputs.clone(); + let source_resource_id = payload.source_resource_id.clone(); let pricing = state.editor_generation_pricing().map_err(|error| { character_animation_error_response( @@ -613,6 +659,68 @@ pub(crate) async fn generate_editor_character_animation_for_owner( let (generated, frames) = result.map_err(|error| character_animation_error_response(&request_context, error))?; + let completed_project = if let Some(completion) = canvas_completion.as_ref() { + let frame_payload_json = serde_json::to_value(&frames).map_err(|error| { + character_animation_error_response( + &request_context, + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": "editor-character-animation", + "message": format!("角色动作序列帧无法写入画布：{error}"), + })), + ) + })?; + let preview_frame = frames.first(); + let layer_id = format!("layer-editor-character-animation-{task_id}"); + let item = build_editor_canvas_generated_layer_item( + completion, + EditorCanvasGeneratedLayerInput { + layer_id: layer_id.clone(), + resource_id: format!("local-resource-editor-character-animation-{task_id}"), + title: completion.title.trim().to_string(), + src: preview_frame + .map(|frame| frame.image_src.clone()) + .unwrap_or_else(|| generated.preview_video_path.clone()), + media_type: Some("image-sequence".to_string()), + thumbnail_src: preview_frame.map(|frame| frame.image_src.clone()), + original_width: preview_frame + .map(|frame| frame.width) + .unwrap_or(normalized.frame_width), + original_height: preview_frame + .map(|frame| frame.height) + .unwrap_or(normalized.frame_height), + display_width: completion.placeholder.width, + display_height: completion.placeholder.height, + source_type: "generated".to_string(), + prompt: Some(generated.submitted_prompt.clone()), + actual_prompt: Some(generated.submitted_prompt.clone()), + model: Some(EDITOR_CHARACTER_ANIMATION_MODEL.to_string()), + provider: Some("ark".to_string()), + task_id: Some(task_id.clone()), + object_key: None, + asset_object_id: None, + duration_seconds: Some(normalized.duration_seconds), + source_resource_id, + asset_kind: Some("character-animation".to_string()), + generation_inputs, + image_sequence_frames: Some(frame_payload_json), + preview_video_path: Some(generated.preview_video_path.clone()), + }, + ); + complete_editor_canvas_generation_with_items( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + canvas_completion.as_ref(), + vec![item], + Some(layer_id), + ) + .await + .map_err(|error| character_animation_error_response(&request_context, error))? + } else { + None + }; + let completed_project = + completed_project.and_then(|project| serde_json::to_value(project).ok()); Ok(json_success_body( Some(&request_context), @@ -629,6 +737,8 @@ pub(crate) async fn generate_editor_character_animation_for_owner( fps: normalized.fps, price_mud_points: normalized.price_mud_points, frames, + project: completed_project, + queue_state: None, }, )) } @@ -639,13 +749,38 @@ pub async fn generate_editor_video( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, Response> { - generate_editor_video_for_owner( - state, - request_context, - authenticated.claims().user_id().to_string(), - payload, - ) - .await + let Json(payload) = payload.map_err(|error| { + editor_video_error_response( + &request_context, + AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({ + "provider": "editor-video", + "message": error.body_text(), + })), + ) + })?; + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = + editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-video"); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + owner_user_id.as_str(), + EDITOR_VIDEO_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成视频", + &payload, + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + generate_editor_video_for_owner(state, request_context, owner_user_id, Ok(Json(payload))).await } pub(crate) async fn generate_editor_video_for_owner( @@ -663,6 +798,11 @@ pub(crate) async fn generate_editor_video_for_owner( })), ) })?; + let project_id = payload.project_id.clone(); + let canvas_completion = payload.canvas_completion.clone(); + let generation_inputs = payload.generation_inputs.clone(); + let source_resource_id = payload.source_resource_id.clone(); + let asset_kind = payload.asset_kind.clone(); let pricing = state.editor_generation_pricing().map_err(|error| { editor_video_error_response( @@ -698,6 +838,52 @@ pub(crate) async fn generate_editor_video_for_owner( ) .await .map_err(|error| editor_video_error_response(&request_context, error))?; + let completed_project = if let Some(completion) = canvas_completion.as_ref() { + let layer_id = format!("layer-editor-video-{task_id}"); + let item = build_editor_canvas_generated_layer_item( + completion, + EditorCanvasGeneratedLayerInput { + layer_id: layer_id.clone(), + resource_id: format!("local-resource-editor-video-{task_id}"), + title: completion.title.trim().to_string(), + src: generated.preview_video_path.clone(), + media_type: Some("video".to_string()), + thumbnail_src: None, + original_width: normalized.width, + original_height: normalized.height, + display_width: completion.placeholder.width, + display_height: completion.placeholder.height, + source_type: "generated".to_string(), + prompt: Some(normalized.prompt.clone()), + actual_prompt: Some(generated.submitted_prompt.clone()), + model: Some(normalized.model.clone()), + provider: Some("VectorEngine".to_string()), + task_id: Some(task_id.clone()), + object_key: generated.object_key.clone(), + asset_object_id: generated.asset_object_id.clone(), + duration_seconds: Some(normalized.duration_seconds), + source_resource_id, + asset_kind: Some(asset_kind.unwrap_or_else(|| "video".to_string())), + generation_inputs, + image_sequence_frames: None, + preview_video_path: None, + }, + ); + complete_editor_canvas_generation_with_items( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + canvas_completion.as_ref(), + vec![item], + Some(layer_id), + ) + .await + .map_err(|error| editor_video_error_response(&request_context, error))? + } else { + None + }; + let completed_project = + completed_project.and_then(|project| serde_json::to_value(project).ok()); Ok(json_success_body( Some(&request_context), @@ -717,6 +903,8 @@ pub(crate) async fn generate_editor_video_for_owner( duration_seconds: normalized.duration_seconds, resolution: normalized.resolution, price_mud_points: normalized.price_mud_points, + project: completed_project, + queue_state: None, }, )) } @@ -4909,6 +5097,10 @@ mod tests { frame_count: 48, duration_seconds: 6, model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, }) .expect("editor request should normalize"); @@ -4943,6 +5135,10 @@ mod tests { frame_count: 32, duration_seconds: 4, model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, }) .expect("editor request should normalize"); @@ -4966,6 +5162,10 @@ mod tests { frame_count: 48, duration_seconds: 4, model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, }) .expect_err("invalid frame/duration pair should fail"); @@ -5121,6 +5321,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("editor video request should normalize"); @@ -5146,6 +5351,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance fast request should normalize"); @@ -5168,6 +5378,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance standard request should normalize"); @@ -5190,6 +5405,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance standard 1080p request should normalize"); @@ -5213,6 +5433,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect_err("seedance fast should reject 1080p"); @@ -5235,6 +5460,11 @@ mod tests { "generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(), ], reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()], + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance multimodal request should normalize"); @@ -5266,6 +5496,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: vec!["data:video/mp4;base64,video".to_string()], reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect_err("Seedance reference videos must not accept Base64 data URLs"); @@ -5287,6 +5522,11 @@ mod tests { reference_image_srcs: vec!["data:image/png;base64,image".to_string()], reference_video_srcs: vec!["https://assets.example.com/reference.mp4".to_string()], reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()], + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance multimodal request should normalize"); @@ -5328,6 +5568,11 @@ mod tests { "generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(), ], reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("seedance objectKey request should normalize"); @@ -5363,6 +5608,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: vec!["data:audio/wav;base64,audio".to_string()], + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect_err("audio-only references should fail"); @@ -5404,7 +5654,6 @@ mod tests { ); } - fn assert_function_contains(source: &str, start: &str, end: &str, snippets: &[&str]) { let start_index = source .find(start) diff --git a/server-rs/crates/api-server/src/editor_generation_queue.rs b/server-rs/crates/api-server/src/editor_generation_queue.rs new file mode 100644 index 000000000..08355bdc6 --- /dev/null +++ b/server-rs/crates/api-server/src/editor_generation_queue.rs @@ -0,0 +1,105 @@ +use axum::http::StatusCode; +use serde::Serialize; +use serde_json::json; +use shared_contracts::external_generation::{ + ExternalGenerationJobStatus, ExternalGenerationJobStatusRecord, +}; +use shared_kernel::{build_prefixed_uuid_id, offset_datetime_to_unix_micros}; +use spacetime_client::{ExternalGenerationJobEnqueueRecordInput, ExternalGenerationJobRecord}; + +use crate::{http_error::AppError, request_context::RequestContext, state::AppState}; + +pub(crate) const EDITOR_IMAGE_GENERATION_JOB_KIND: &str = "editor_image_generation"; +pub(crate) const EDITOR_IMAGE_EDIT_JOB_KIND: &str = "editor_image_edit"; +pub(crate) const EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND: &str = + "editor_icon_spritesheet_generation"; +pub(crate) const EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND: &str = + "editor_ui_design_asset_extraction"; +pub(crate) const EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND: &str = + "editor_character_animation_generation"; +pub(crate) const EDITOR_VIDEO_GENERATION_JOB_KIND: &str = "editor_video_generation"; +pub(crate) const EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND: &str = "editor_sound_effect_generation"; +pub(crate) const EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND: &str = + "editor_background_music_generation"; + +pub(crate) const EDITOR_GENERATION_QUEUE_SOURCE_MODULE: &str = "editor-canvas"; +const EDITOR_GENERATION_QUEUE_PROVIDER: &str = "editor-generation-worker"; + +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub(crate) struct EditorGenerationQueuedResponse { + pub(crate) queue_state: ExternalGenerationJobStatusRecord, +} + +pub(crate) async fn enqueue_editor_generation_job( + state: &AppState, + _request_context: &RequestContext, + owner_user_id: &str, + job_kind: &str, + source_entity_id: impl Into, + request_label: impl Into, + payload: &T, +) -> Result +where + T: Serialize, +{ + let job_id = build_prefixed_uuid_id("extgen-"); + let request_payload_json = serde_json::to_string(payload).map_err(|error| { + AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({ + "provider": EDITOR_GENERATION_QUEUE_PROVIDER, + "message": format!("编辑器 worker 任务参数序列化失败：{error}"), + })) + })?; + let now_micros = current_utc_micros(); + state + .spacetime_client() + .enqueue_external_generation_job(ExternalGenerationJobEnqueueRecordInput { + dedupe_key: format!("editor-canvas:{job_kind}:{job_id}"), + job_id, + job_kind: job_kind.to_string(), + owner_user_id: owner_user_id.to_string(), + source_module: EDITOR_GENERATION_QUEUE_SOURCE_MODULE.to_string(), + source_entity_id: source_entity_id.into(), + request_label: request_label.into(), + request_payload_json, + max_attempts: 1, + available_at_micros: now_micros, + created_at_micros: now_micros, + }) + .await + .map_err(|error| { + AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({ + "provider": EDITOR_GENERATION_QUEUE_PROVIDER, + "message": error.to_string(), + })) + }) +} + +pub(crate) fn editor_generation_queue_state( + job: ExternalGenerationJobRecord, +) -> ExternalGenerationJobStatusRecord { + ExternalGenerationJobStatusRecord { + operation_id: job.job_id, + status: ExternalGenerationJobStatus::Queued, + phase_label: job.request_label, + phase_detail: "排队中。".to_string(), + progress: 8, + error: job.last_error_message, + updated_at_micros: job.updated_at_micros, + } +} + +pub(crate) fn editor_generation_source_entity_id( + project_id: Option<&str>, + fallback: &str, +) -> String { + project_id + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or(fallback) + .to_string() +} + +fn current_utc_micros() -> i64 { + offset_datetime_to_unix_micros(time::OffsetDateTime::now_utc()) +} diff --git a/server-rs/crates/api-server/src/editor_project.rs b/server-rs/crates/api-server/src/editor_project.rs index 5326c9610..a34393826 100644 --- a/server-rs/crates/api-server/src/editor_project.rs +++ b/server-rs/crates/api-server/src/editor_project.rs @@ -22,6 +22,10 @@ use platform_image::{ use platform_oss::{LegacyAssetPrefix, OssHeadObjectRequest, OssObjectAccess}; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; +use shared_contracts::assets::{ + EditorCanvasGenerationCompletionPayload as EditorCanvasGenerationCompletionRequest, + EditorCanvasGenerationPlaceholderPayload, +}; use shared_kernel::build_prefixed_uuid_id; use spacetime_client::{ EditorAssetCreateRecordInput, EditorAssetDeleteRecordInput, EditorAssetFolderCreateRecordInput, @@ -37,6 +41,12 @@ use crate::{ api_response::json_success_body, asset_billing::execute_billable_asset_operation_with_cost, auth::AuthenticatedAccessToken, + editor_generation_queue::{ + EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND, + EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, + EditorGenerationQueuedResponse, editor_generation_queue_state, + editor_generation_source_entity_id, enqueue_editor_generation_job, + }, generated_image_assets::{ GeneratedImageAssetAdapter, GeneratedImageAssetDataUrl, adapter::{GeneratedImageAssetAdapterMetadata, GeneratedImageAssetPersistInput}, @@ -162,7 +172,7 @@ pub struct EditorAssetUpdateRequest { folder_id: Option, } -#[derive(Debug, Deserialize)] +#[derive(Clone, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorImageGenerationRequest { pub(crate) prompt: String, @@ -181,26 +191,7 @@ pub struct EditorImageGenerationRequest { pub(crate) canvas_completion: Option, } -#[derive(Clone, Debug, Deserialize)] -#[serde(rename_all = "camelCase")] -pub struct EditorCanvasGenerationCompletionRequest { - pub(crate) dialog_id: String, - pub(crate) title: String, - pub(crate) placeholder: EditorCanvasGenerationPlaceholderPayload, -} - -#[derive(Clone, Debug, Deserialize)] -#[serde(rename_all = "camelCase")] -pub struct EditorCanvasGenerationPlaceholderPayload { - pub(crate) x: f64, - pub(crate) y: f64, - pub(crate) width: f64, - pub(crate) height: f64, - pub(crate) original_width: f64, - pub(crate) original_height: f64, -} - -#[derive(Debug, Deserialize)] +#[derive(Clone, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorImageEditRequest { pub(crate) prompt: String, @@ -211,9 +202,10 @@ pub struct EditorImageEditRequest { pub(crate) asset_folder_id: Option, pub(crate) asset_label: Option, pub(crate) source_resource_id: Option, + pub(crate) canvas_completion: Option, } -#[derive(Debug, Deserialize)] +#[derive(Clone, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorIconSpritesheetGenerationRequest { pub(crate) reference_image_src: String, @@ -224,9 +216,10 @@ pub struct EditorIconSpritesheetGenerationRequest { pub(crate) project_id: Option, pub(crate) generation_inputs: Option, pub(crate) asset_folder_id: Option, + pub(crate) canvas_completion: Option, } -#[derive(Debug, Deserialize)] +#[derive(Clone, Debug, Deserialize, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorUiDesignAssetExtractionRequest { pub(crate) source_image_src: String, @@ -236,6 +229,7 @@ pub struct EditorUiDesignAssetExtractionRequest { pub(crate) generation_inputs: Option, pub(crate) asset_folder_id: Option, pub(crate) spritesheet_label: Option, + pub(crate) canvas_completion: Option, } pub(crate) struct EditorGenerationCaller { @@ -341,6 +335,33 @@ struct PersistEditorGeneratedAssetInput { generation_inputs: Option, } +pub(crate) struct EditorCanvasGeneratedLayerInput { + pub(crate) layer_id: String, + pub(crate) resource_id: String, + pub(crate) title: String, + pub(crate) src: String, + pub(crate) media_type: Option, + pub(crate) thumbnail_src: Option, + pub(crate) original_width: u32, + pub(crate) original_height: u32, + pub(crate) display_width: f64, + pub(crate) display_height: f64, + pub(crate) source_type: String, + pub(crate) prompt: Option, + pub(crate) actual_prompt: Option, + pub(crate) model: Option, + pub(crate) provider: Option, + pub(crate) task_id: Option, + pub(crate) object_key: Option, + pub(crate) asset_object_id: Option, + pub(crate) duration_seconds: Option, + pub(crate) source_resource_id: Option, + pub(crate) asset_kind: Option, + pub(crate) generation_inputs: Option, + pub(crate) image_sequence_frames: Option, + pub(crate) preview_video_path: Option, +} + #[derive(Debug, Serialize)] #[serde(rename_all = "camelCase")] pub struct EditorImageGenerationResponse { @@ -386,6 +407,7 @@ pub struct EditorIconSpritesheetGenerationResponse { price_mud_points: u32, spritesheet_resource: Option, spritesheet_asset: Option, + project: Option, } #[derive(Debug, Serialize)] @@ -895,13 +917,30 @@ pub async fn generate_editor_image( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { - generate_editor_image_for_owner( - &state, - &request_context, - EditorGenerationCaller::from_authenticated(&authenticated), - payload, - ) - .await + let caller = EditorGenerationCaller::from_authenticated(&authenticated); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-image-generation", + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + caller.owner_user_id.as_str(), + EDITOR_IMAGE_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成图片", + &payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + generate_editor_image_for_owner(&state, &request_context, caller, payload).await } pub(crate) async fn generate_editor_image_for_owner( @@ -1329,13 +1368,28 @@ pub async fn edit_editor_image( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { - edit_editor_image_for_owner( - &state, - &request_context, - EditorGenerationCaller::from_authenticated(&authenticated), - payload, - ) - .await + let caller = EditorGenerationCaller::from_authenticated(&authenticated); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = + editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-image-edit"); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + caller.owner_user_id.as_str(), + EDITOR_IMAGE_EDIT_JOB_KIND, + source_entity_id, + "图片画布修改图片", + &payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + edit_editor_image_for_owner(&state, &request_context, caller, payload).await } pub(crate) async fn edit_editor_image_for_owner( @@ -1397,11 +1451,12 @@ pub(crate) async fn edit_editor_image_for_owner( image.mime_type, BASE64_STANDARD.encode(image.bytes.as_slice()) ); + let owner_user_id = caller.owner_user_id.clone(); let generated_asset = persist_editor_generated_asset( state, PersistEditorGeneratedAssetInput { - project_id: payload.project_id, - owner_user_id: caller.owner_user_id, + project_id: payload.project_id.clone(), + owner_user_id: owner_user_id.clone(), folder_id: payload.asset_folder_id, label: normalize_optional_string(payload.asset_label) .unwrap_or_else(|| "图片修改结果".to_string()), @@ -1421,6 +1476,14 @@ pub(crate) async fn edit_editor_image_for_owner( }, ) .await?; + let completed_project = complete_editor_canvas_generation( + state, + owner_user_id.as_str(), + payload.project_id.as_deref(), + payload.canvas_completion.as_ref(), + generated_asset.resource.as_ref(), + ) + .await?; Ok(json_success_body( Some(&request_context), @@ -1438,7 +1501,7 @@ pub(crate) async fn edit_editor_image_for_owner( task_id: generated.task_id, resource: generated_asset.resource, asset: generated_asset.asset, - project: None, + project: completed_project, }, )) } @@ -1449,13 +1512,30 @@ pub async fn generate_editor_icon_spritesheet( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { - generate_editor_icon_spritesheet_for_owner( - &state, - &request_context, - EditorGenerationCaller::from_authenticated(&authenticated), - payload, - ) - .await + let caller = EditorGenerationCaller::from_authenticated(&authenticated); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-icon-spritesheet-generation", + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + caller.owner_user_id.as_str(), + EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成图标素材", + &payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + generate_editor_icon_spritesheet_for_owner(&state, &request_context, caller, payload).await } pub(crate) async fn generate_editor_icon_spritesheet_for_owner( @@ -1615,6 +1695,25 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( ..icon_response }); } + let (canvas_items, primary_layer_id) = + if let Some(completion) = payload.canvas_completion.as_ref() { + build_icon_spritesheet_canvas_layer_items( + completion, + spritesheet_record.resource.as_ref(), + icon_image_srcs.as_slice(), + ) + } else { + (Vec::new(), None) + }; + let completed_project = complete_editor_canvas_generation_with_items( + state, + owner_user_id.as_str(), + payload.project_id.as_deref(), + payload.canvas_completion.as_ref(), + canvas_items, + primary_layer_id, + ) + .await?; Ok(json_success_body( Some(&request_context), @@ -1631,6 +1730,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner( price_mud_points: expected_price_mud_points, spritesheet_resource: spritesheet_record.resource, spritesheet_asset: spritesheet_record.asset, + project: completed_project, }, )) } @@ -1641,13 +1741,30 @@ pub async fn extract_editor_ui_design_assets( Extension(authenticated): Extension, Json(payload): Json, ) -> Result, AppError> { - extract_editor_ui_design_assets_for_owner( - &state, - &request_context, - EditorGenerationCaller::from_authenticated(&authenticated), - payload, - ) - .await + let caller = EditorGenerationCaller::from_authenticated(&authenticated); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-ui-design-asset-extraction", + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + caller.owner_user_id.as_str(), + EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, + source_entity_id, + "图片画布提取UI设计图素材", + &payload, + ) + .await?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + extract_editor_ui_design_assets_for_owner(&state, &request_context, caller, payload).await } pub(crate) async fn extract_editor_ui_design_assets_for_owner( @@ -1782,6 +1899,25 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner( ..icon_response }); } + let (canvas_items, primary_layer_id) = + if let Some(completion) = payload.canvas_completion.as_ref() { + build_icon_spritesheet_canvas_layer_items( + completion, + spritesheet_record.resource.as_ref(), + icon_image_srcs.as_slice(), + ) + } else { + (Vec::new(), None) + }; + let completed_project = complete_editor_canvas_generation_with_items( + state, + owner_user_id.as_str(), + payload.project_id.as_deref(), + payload.canvas_completion.as_ref(), + canvas_items, + primary_layer_id, + ) + .await?; Ok(json_success_body( Some(&request_context), @@ -1798,6 +1934,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner( price_mud_points: expected_price_mud_points, spritesheet_resource: spritesheet_record.resource, spritesheet_asset: spritesheet_record.asset, + project: completed_project, }, )) } @@ -1991,7 +2128,7 @@ async fn persist_editor_generated_asset( Ok(EditorGeneratedAssetRecord { resource, asset }) } -async fn complete_editor_canvas_generation( +pub(crate) async fn complete_editor_canvas_generation( state: &AppState, owner_user_id: &str, project_id: Option<&str>, @@ -2003,7 +2140,7 @@ async fn complete_editor_canvas_generation( return Ok(None); }; let project_id = project_id.trim(); - if project_id.is_empty() || completion.dialog_id.trim().is_empty() { + if project_id.is_empty() { return Ok(None); } @@ -2015,8 +2152,21 @@ async fn complete_editor_canvas_generation( }) .await .map_err(map_editor_project_error)?; - let completion_result = - apply_editor_canvas_generation_completion(project.layers, completion, resource)?; + let Some(placeholder) = + resolve_canvas_completion_placeholder(project.layers.clone(), completion)? + else { + return Ok(None); + }; + let layer_id = generated_canvas_layer_id(resource.resource_id.as_str()); + let item = + build_generated_canvas_layer_item(completion, &placeholder, resource, layer_id.as_str(), 0); + let completion_result = apply_editor_canvas_generation_items( + project.layers, + completion, + placeholder, + vec![item], + Some(layer_id), + )?; if !completion_result.changed { return Ok(None); } @@ -2035,69 +2185,214 @@ async fn complete_editor_canvas_generation( Ok(Some(editor_project_payload_from_record(saved))) } -fn apply_editor_canvas_generation_completion( +pub(crate) async fn complete_editor_canvas_generation_with_items( + state: &AppState, + owner_user_id: &str, + project_id: Option<&str>, + completion: Option<&EditorCanvasGenerationCompletionRequest>, + items: Vec, + primary_layer_id: Option, +) -> Result, AppError> { + let (Some(project_id), Some(completion)) = (project_id, completion) else { + return Ok(None); + }; + let project_id = project_id.trim(); + if project_id.is_empty() || items.is_empty() { + return Ok(None); + } + + let project = state + .spacetime_client() + .get_editor_project(EditorProjectGetRecordInput { + project_id: project_id.to_string(), + owner_user_id: owner_user_id.to_string(), + }) + .await + .map_err(map_editor_project_error)?; + let Some(placeholder) = + resolve_canvas_completion_placeholder(project.layers.clone(), completion)? + else { + return Ok(None); + }; + let items = + translate_editor_canvas_generation_items(items, &completion.placeholder, &placeholder); + let completion_result = apply_editor_canvas_generation_items( + project.layers, + completion, + placeholder, + items, + primary_layer_id, + )?; + if !completion_result.changed { + return Ok(None); + } + let saved = state + .spacetime_client() + .save_editor_project_layout(EditorProjectLayoutSaveRecordInput { + project_id: project_id.to_string(), + owner_user_id: owner_user_id.to_string(), + viewport: project.viewport.clone(), + layers_json: serialize_editor_layers(completion_result.layers)?, + updated_at_micros: current_utc_micros(), + }) + .await + .map_err(map_editor_project_error)?; + + Ok(Some(editor_project_payload_from_record(saved))) +} + +fn translate_editor_canvas_generation_items( + mut items: Vec, + submitted_placeholder: &EditorCanvasGenerationPlaceholderPayload, + resolved_placeholder: &EditorCanvasGenerationPlaceholderPayload, +) -> Vec { + let dx = resolved_placeholder.x - submitted_placeholder.x; + let dy = resolved_placeholder.y - submitted_placeholder.y; + if !dx.is_finite() || !dy.is_finite() || (dx == 0.0 && dy == 0.0) { + return items; + } + + for item in &mut items { + let Some(object) = item.as_object_mut() else { + continue; + }; + if let Some(x) = object.get("x").and_then(Value::as_f64) + && (x + dx).is_finite() + { + object.insert("x".to_string(), json!(x + dx)); + } + if let Some(y) = object.get("y").and_then(Value::as_f64) + && (y + dy).is_finite() + { + object.insert("y".to_string(), json!(y + dy)); + } + } + + items +} + +fn resolve_canvas_completion_placeholder( layers: Value, completion: &EditorCanvasGenerationCompletionRequest, - resource: &EditorProjectResourcePayload, +) -> Result, AppError> { + let dialog_id = normalized_canvas_completion_dialog_id(completion); + let items = match layers { + Value::Array(items) => items, + _ => Vec::new(), + }; + if let Some(dialog_id) = dialog_id.as_deref() { + return items + .iter() + .find(|item| generation_dialog_item_matches(item, dialog_id)) + .map(|item| { + generation_dialog_item_placeholder(item, dialog_id) + .unwrap_or_else(|| completion.placeholder.clone()) + }) + .map(Some) + .map_or(Ok(None), Ok); + } + Ok(Some(completion.placeholder.clone())) +} + +fn apply_editor_canvas_generation_items( + layers: Value, + completion: &EditorCanvasGenerationCompletionRequest, + _placeholder: EditorCanvasGenerationPlaceholderPayload, + generated_items: Vec, + primary_layer_id: Option, ) -> Result { let mut items = match layers { Value::Array(items) => items, _ => Vec::new(), }; - let Some(placeholder) = items - .iter() - .find(|item| generation_dialog_item_matches(item, completion.dialog_id.as_str())) - .map(|item| { - generation_dialog_item_placeholder(item, completion.dialog_id.as_str()) - .unwrap_or_else(|| completion.placeholder.clone()) - }) - else { + let dialog_id = normalized_canvas_completion_dialog_id(completion); + if let Some(dialog_id) = dialog_id.as_deref() + && !items + .iter() + .any(|item| generation_dialog_item_matches(item, dialog_id)) + { return Ok(EditorCanvasGenerationCompletionResult { layers: Value::Array(items), changed: false, }); - }; - let generated_layer_id = generated_canvas_layer_id(resource.resource_id.as_str()); - let completed_layer_id = if let Some(existing_layer_id) = items.iter().find_map(|item| { - let resource_id = item.get("resourceId").and_then(Value::as_str)?; - if resource_id == resource.resource_id { - item.get("layerId") - .and_then(Value::as_str) - .map(str::to_string) - } else { - None - } - }) { - existing_layer_id - } else { - let z_index = next_canvas_layer_z_index(&items); - items.push(build_generated_canvas_layer_item( - completion, - &placeholder, - resource, - generated_layer_id.as_str(), - z_index, - )); - generated_layer_id - }; + } - for item in &mut items { - if generation_dialog_item_matches(item, completion.dialog_id.as_str()) { - if let Some(dialog) = item.get_mut("dialog").and_then(Value::as_object_mut) { + let mut changed = false; + let mut next_z_index = next_canvas_layer_z_index(&items); + let mut completed_layer_id = primary_layer_id; + for mut item in generated_items { + let existing_layer_id = + item.get("resourceId") + .and_then(Value::as_str) + .and_then(|resource_id| { + items.iter().find_map(|existing| { + let existing_resource_id = + existing.get("resourceId").and_then(Value::as_str)?; + if existing_resource_id == resource_id { + existing + .get("layerId") + .and_then(Value::as_str) + .map(str::to_string) + } else { + None + } + }) + }); + if let Some(existing_layer_id) = existing_layer_id { + completed_layer_id = Some(existing_layer_id); + continue; + } + let needs_z_index = match item.get("zIndex").and_then(Value::as_i64) { + Some(value) => value <= 0, + None => true, + }; + if needs_z_index && let Some(object) = item.as_object_mut() { + object.insert("zIndex".to_string(), json!(next_z_index)); + next_z_index += 1; + } + if completed_layer_id.is_none() { + completed_layer_id = item + .get("layerId") + .and_then(Value::as_str) + .map(str::to_string); + } + items.push(item); + changed = true; + } + + if let Some(dialog_id) = dialog_id.as_deref() { + for item in &mut items { + if generation_dialog_item_matches(item, dialog_id) + && let Some(dialog) = item.get_mut("dialog").and_then(Value::as_object_mut) + { dialog.insert("status".to_string(), json!("idle")); dialog.insert("composerOpen".to_string(), json!(true)); - dialog.insert("generatedLayerId".to_string(), json!(completed_layer_id)); + if let Some(completed_layer_id) = completed_layer_id.as_deref() { + dialog.insert("generatedLayerId".to_string(), json!(completed_layer_id)); + } dialog.remove("errorMessage"); + changed = true; } } } Ok(EditorCanvasGenerationCompletionResult { layers: Value::Array(items), - changed: true, + changed, }) } +fn normalized_canvas_completion_dialog_id( + completion: &EditorCanvasGenerationCompletionRequest, +) -> Option { + completion + .dialog_id + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned) +} + fn build_generated_canvas_layer_item( completion: &EditorCanvasGenerationCompletionRequest, placeholder: &EditorCanvasGenerationPlaceholderPayload, @@ -2119,6 +2414,7 @@ fn build_generated_canvas_layer_item( "layerId": generated_layer_id, "resourceId": resource.resource_id, "title": completion.title.trim(), + "src": resource.image_src, "x": x, "y": y, "width": width, @@ -2136,6 +2432,52 @@ fn build_generated_canvas_layer_item( "objectKey": resource.object_key, "assetObjectId": resource.asset_object_id, "sourceResourceId": resource.source_resource_id, + "assetKind": resource.asset_kind, + "generationInputs": resource.generation_inputs, + }) +} + +pub(crate) fn build_editor_canvas_generated_layer_item( + completion: &EditorCanvasGenerationCompletionRequest, + input: EditorCanvasGeneratedLayerInput, +) -> Value { + let placeholder = &completion.placeholder; + let original_width = positive_f64(input.original_width as f64, placeholder.original_width); + let original_height = positive_f64(input.original_height as f64, placeholder.original_height); + let width = positive_f64(input.display_width, placeholder.width); + let height = positive_f64(input.display_height, placeholder.height); + let placeholder_width = positive_f64(placeholder.width, width); + let placeholder_height = positive_f64(placeholder.height, height); + let x = placeholder.x + placeholder_width / 2.0 - width / 2.0; + let y = placeholder.y + placeholder_height / 2.0 - height / 2.0; + + json!({ + "layerId": input.layer_id, + "resourceId": input.resource_id, + "title": input.title, + "src": input.src, + "mediaType": input.media_type, + "thumbnailSrc": input.thumbnail_src, + "x": x, + "y": y, + "width": width, + "height": height, + "originalWidth": original_width, + "originalHeight": original_height, + "sourceType": input.source_type, + "prompt": input.prompt, + "actualPrompt": input.actual_prompt, + "model": input.model, + "provider": input.provider, + "taskId": input.task_id, + "objectKey": input.object_key, + "assetObjectId": input.asset_object_id, + "durationSeconds": input.duration_seconds, + "sourceResourceId": input.source_resource_id, + "assetKind": input.asset_kind, + "generationInputs": input.generation_inputs, + "imageSequenceFrames": input.image_sequence_frames, + "previewVideoPath": input.preview_video_path, }) } @@ -2161,6 +2503,75 @@ fn generated_canvas_media_type(asset_kind: Option<&str>) -> &'static str { } } +fn build_icon_spritesheet_canvas_layer_items( + completion: &EditorCanvasGenerationCompletionRequest, + spritesheet_resource: Option<&EditorProjectResourcePayload>, + icons: &[EditorIconSpritesheetIconResponse], +) -> (Vec, Option) { + let mut items = Vec::new(); + let mut primary_layer_id = None; + let placeholder = completion.placeholder.clone(); + let Some(spritesheet_resource) = spritesheet_resource else { + return (items, primary_layer_id); + }; + + let spritesheet_layer_id = generated_canvas_layer_id(spritesheet_resource.resource_id.as_str()); + primary_layer_id = Some(spritesheet_layer_id.clone()); + let spritesheet_item = build_generated_canvas_layer_item( + completion, + &placeholder, + spritesheet_resource, + spritesheet_layer_id.as_str(), + 0, + ); + let sheet_width = spritesheet_item + .get("width") + .and_then(Value::as_f64) + .unwrap_or(placeholder.width.max(1.0)); + items.push(spritesheet_item); + + let mut cursor_x = placeholder.x + sheet_width + 32.0; + let mut cursor_y = placeholder.y; + let mut row_height = 0.0_f64; + let max_row_width = 560.0_f64; + let start_x = cursor_x; + let spacing = 24.0_f64; + + for icon in icons { + let Some(resource) = icon.resource.as_ref() else { + continue; + }; + let width = positive_f64(resource.width as f64, 128.0); + let height = positive_f64(resource.height as f64, 128.0); + if cursor_x > start_x && cursor_x + width - start_x > max_row_width { + cursor_x = start_x; + cursor_y += row_height + spacing; + row_height = 0.0; + } + let layer_id = generated_canvas_layer_id(resource.resource_id.as_str()); + let mut item = build_generated_canvas_layer_item( + completion, + &placeholder, + resource, + layer_id.as_str(), + 0, + ); + if let Some(object) = item.as_object_mut() { + object.insert("x".to_string(), json!(cursor_x)); + object.insert("y".to_string(), json!(cursor_y)); + object.insert("width".to_string(), json!(width)); + object.insert("height".to_string(), json!(height)); + object.insert("originalWidth".to_string(), json!(width)); + object.insert("originalHeight".to_string(), json!(height)); + } + cursor_x += width + spacing; + row_height = row_height.max(height); + items.push(item); + } + + (items, primary_layer_id) +} + fn next_canvas_layer_z_index(items: &[Value]) -> i64 { items .iter() @@ -2679,6 +3090,49 @@ mod tests { use super::*; use crate::config::AppConfig; + fn apply_editor_canvas_generation_completion( + layers: Value, + completion: &EditorCanvasGenerationCompletionRequest, + resource: &EditorProjectResourcePayload, + ) -> Result { + let placeholder = resolve_canvas_completion_placeholder(layers.clone(), completion)? + .unwrap_or_else(|| completion.placeholder.clone()); + let layer_id = generated_canvas_layer_id(resource.resource_id.as_str()); + let item = build_generated_canvas_layer_item( + completion, + &placeholder, + resource, + layer_id.as_str(), + 0, + ); + apply_editor_canvas_generation_items( + layers, + completion, + placeholder, + vec![item], + Some(layer_id), + ) + } + + fn apply_editor_canvas_generation_multi_items( + layers: Value, + completion: &EditorCanvasGenerationCompletionRequest, + items: Vec, + primary_layer_id: Option, + ) -> Result { + let placeholder = resolve_canvas_completion_placeholder(layers.clone(), completion)? + .unwrap_or_else(|| completion.placeholder.clone()); + let items = + translate_editor_canvas_generation_items(items, &completion.placeholder, &placeholder); + apply_editor_canvas_generation_items( + layers, + completion, + placeholder, + items, + primary_layer_id, + ) + } + #[test] fn editor_image_generation_size_keeps_quick_edit_canvas_ratio_presets() { assert_eq!(normalize_editor_image_generation_size(None), "1024x1024"); @@ -3003,7 +3457,7 @@ mod tests { } ]); let completion = EditorCanvasGenerationCompletionRequest { - dialog_id: "dialog-character".to_string(), + dialog_id: Some("dialog-character".to_string()), title: "角色形象 1".to_string(), placeholder: EditorCanvasGenerationPlaceholderPayload { x: 200.0, @@ -3062,9 +3516,8 @@ mod tests { assert_eq!(result_layer["zIndex"], json!(5)); assert_eq!(result_layer["mediaType"], json!("image")); assert_eq!( - result_layer.get("src"), - None, - "layout must not embed heavy image payloads" + result_layer["src"], + json!("data:image/png;base64,character") ); let dialog = items @@ -3103,7 +3556,7 @@ mod tests { } ]); let completion = EditorCanvasGenerationCompletionRequest { - dialog_id: "dialog-character".to_string(), + dialog_id: Some("dialog-character".to_string()), title: "角色形象 1".to_string(), placeholder: EditorCanvasGenerationPlaceholderPayload { x: 0.0, @@ -3183,7 +3636,7 @@ mod tests { } ]); let completion = EditorCanvasGenerationCompletionRequest { - dialog_id: "dialog-character".to_string(), + dialog_id: Some("dialog-character".to_string()), title: "角色形象 1".to_string(), placeholder: EditorCanvasGenerationPlaceholderPayload { x: 0.0, @@ -3229,6 +3682,71 @@ mod tests { assert_eq!(result_layer["y"], json!(-88.0)); } + #[test] + fn editor_canvas_generation_multi_item_completion_uses_latest_layout_placeholder() { + let layers = json!([ + { + "itemType": "generation-dialog", + "layerId": "generation-dialog:dialog-icons", + "resourceId": "generation-dialog:dialog-icons", + "dialog": { + "id": "dialog-icons", + "mode": "icon", + "prompt": "返回按钮", + "status": "generating", + "composerOpen": false, + "placeholder": { + "x": 300, + "y": 240, + "width": 200, + "height": 200, + "originalWidth": 200, + "originalHeight": 200 + } + } + } + ]); + let completion = EditorCanvasGenerationCompletionRequest { + dialog_id: Some("dialog-icons".to_string()), + title: "图标素材图集".to_string(), + placeholder: EditorCanvasGenerationPlaceholderPayload { + x: 100.0, + y: 80.0, + width: 200.0, + height: 200.0, + original_width: 200.0, + original_height: 200.0, + }, + }; + let item = json!({ + "layerId": "layer-icon-sheet", + "resourceId": "resource-icon-sheet", + "title": "图标素材图集", + "x": 150.0, + "y": 130.0, + "width": 100.0, + "height": 100.0 + }); + + let completed = apply_editor_canvas_generation_multi_items( + layers, + &completion, + vec![item], + Some("layer-icon-sheet".to_string()), + ) + .expect("multi-item completion should apply"); + let result_layer = completed + .layers + .as_array() + .expect("layers should stay array") + .iter() + .find(|item| item["resourceId"] == "resource-icon-sheet") + .expect("result layer should be inserted"); + + assert_eq!(result_layer["x"], json!(350.0)); + assert_eq!(result_layer["y"], json!(290.0)); + } + #[test] fn editor_canvas_generation_completion_skips_when_dialog_was_removed() { let layers = json!([ @@ -3239,7 +3757,7 @@ mod tests { } ]); let completion = EditorCanvasGenerationCompletionRequest { - dialog_id: "dialog-character".to_string(), + dialog_id: Some("dialog-character".to_string()), title: "角色形象 1".to_string(), placeholder: EditorCanvasGenerationPlaceholderPayload { x: 200.0, @@ -3338,8 +3856,7 @@ mod tests { 12 ); assert_eq!( - resolve_editor_icon_spritesheet_price(&state, Some("gpt-image-2"), Some("1K")) - .unwrap(), + resolve_editor_icon_spritesheet_price(&state, Some("gpt-image-2"), Some("1K")).unwrap(), 3 ); let response = EditorIconSpritesheetGenerationResponse { @@ -3366,6 +3883,7 @@ mod tests { ), spritesheet_resource: None, spritesheet_asset: None, + project: None, }; let payload = serde_json::to_value(response).expect("response should serialize"); @@ -3390,7 +3908,10 @@ mod tests { assert_eq!(generation_options.model, GPT_IMAGE_2_MODEL); assert_eq!(generation_options.size, "2048x2048"); assert_eq!( - resolve_editor_ui_design_asset_extraction_price(&state, Some(generation_options.image_size)) + resolve_editor_ui_design_asset_extraction_price( + &state, + Some(generation_options.image_size) + ) .unwrap(), 5 ); diff --git a/server-rs/crates/api-server/src/external_generation_worker.rs b/server-rs/crates/api-server/src/external_generation_worker.rs index fe110c2ed..ccc02ac3d 100644 --- a/server-rs/crates/api-server/src/external_generation_worker.rs +++ b/server-rs/crates/api-server/src/external_generation_worker.rs @@ -1,6 +1,6 @@ use std::{future::Future, io, pin::Pin, time::Duration}; -use axum::extract::FromRef; +use axum::{Json, extract::FromRef}; use serde_json::json; use shared_kernel::offset_datetime_to_unix_micros; use spacetime_client::{ @@ -15,6 +15,22 @@ use tokio::{ use tracing::{error, info, warn}; use crate::{ + character_animation_assets::{ + generate_editor_character_animation_for_owner, generate_editor_video_for_owner, + }, + editor_generation_queue::{ + EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, + EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND, + EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, + EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND, + }, + editor_project::{ + EditorGenerationCaller, EditorIconSpritesheetGenerationRequest, EditorImageEditRequest, + EditorImageGenerationRequest, EditorUiDesignAssetExtractionRequest, + edit_editor_image_for_owner, extract_editor_ui_design_assets_for_owner, + generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner, + }, jump_hop::{ JUMP_HOP_COMPILE_DRAFT_JOB_KIND, JumpHopCompileDraftWorkerPayload, execute_jump_hop_compile_draft_worker_job, @@ -31,6 +47,9 @@ use crate::{ }, request_context::RequestContext, state::{AppState, PuzzleApiState}, + vector_engine_audio_generation::{ + generate_editor_background_music_for_owner, generate_editor_sound_effect_for_owner, + }, wooden_fish::{ WOODEN_FISH_GENERATE_IMAGE_ASSETS_JOB_KIND, WoodenFishGenerateImageAssetsWorkerPayload, execute_wooden_fish_generate_image_assets_worker_job, @@ -536,6 +555,234 @@ async fn process_external_generation_job_once( } } } + EDITOR_IMAGE_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::( + job.request_payload_json.as_str(), + ) { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布生图任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_image_for_owner( + &state, + &request_context, + editor_generation_worker_caller(&job), + payload, + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(error) => { + let message = error.body_text(); + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_IMAGE_EDIT_JOB_KIND => { + let payload = match serde_json::from_str::( + job.request_payload_json.as_str(), + ) { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布改图任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match edit_editor_image_for_owner( + &state, + &request_context, + editor_generation_worker_caller(&job), + payload, + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(error) => { + let message = error.body_text(); + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::( + job.request_payload_json.as_str(), + ) { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布图标素材任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_icon_spritesheet_for_owner( + &state, + &request_context, + editor_generation_worker_caller(&job), + payload, + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(error) => { + let message = error.body_text(); + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND => { + let payload = match serde_json::from_str::( + job.request_payload_json.as_str(), + ) { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布 UI 素材提取任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match extract_editor_ui_design_assets_for_owner( + &state, + &request_context, + editor_generation_worker_caller(&job), + payload, + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(error) => { + let message = error.body_text(); + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorCharacterAnimationGenerateRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布角色动作任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_character_animation_for_owner( + state.clone(), + request_context, + job.owner_user_id.clone(), + Ok(Json(payload)), + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_VIDEO_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorVideoGenerateRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布视频生成任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_video_for_owner( + state.clone(), + request_context, + job.owner_user_id.clone(), + Ok(Json(payload)), + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorSoundEffectGenerateRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布音效生成任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_sound_effect_for_owner( + state.clone(), + request_context, + job.owner_user_id.clone(), + Ok(Json(payload)), + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } + EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND => { + let payload = match serde_json::from_str::< + shared_contracts::assets::EditorBackgroundMusicGenerateRequest, + >(job.request_payload_json.as_str()) + { + Ok(payload) => payload, + Err(error) => { + let message = format!("图片画布背景音乐生成任务参数解析失败：{error}"); + fail_job(&state, &worker_id, &job, message.clone()).await?; + return Err(message); + } + }; + let request_context = worker_request_context(&job); + match generate_editor_background_music_for_owner( + state.clone(), + request_context, + job.owner_user_id.clone(), + Ok(Json(payload)), + ) + .await + { + Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await, + Err(response) => { + let message = response_error_message(response).await; + fail_job(&state, &worker_id, &job, message.clone()).await?; + Err(message) + } + } + } unknown => { warn!( job_id = job.job_id, @@ -598,6 +845,43 @@ async fn fail_queue_job_after_worker_error( Ok(()) } +fn worker_request_context(job: &ExternalGenerationJobRecord) -> RequestContext { + RequestContext::new( + format!("external-generation-worker-{}", job.job_id), + format!("external-generation-worker {}", job.job_kind), + std::time::Duration::ZERO, + false, + ) +} + +fn editor_generation_worker_caller(job: &ExternalGenerationJobRecord) -> EditorGenerationCaller { + EditorGenerationCaller { + owner_user_id: job.owner_user_id.clone(), + audit_subject_user_id: Some(job.owner_user_id.clone()), + audit_project_id: Some(job.source_entity_id.clone()), + } +} + +async fn complete_editor_generation_job( + state: &AppState, + worker_id: &str, + job: &ExternalGenerationJobRecord, +) -> Result<(), String> { + complete_job( + state, + worker_id, + job, + Some( + json!({ + "sourceModule": job.source_module.clone(), + "sourceEntityId": job.source_entity_id.clone(), + }) + .to_string(), + ), + ) + .await +} + async fn complete_job( state: &AppState, worker_id: &str, diff --git a/server-rs/crates/api-server/src/main.rs b/server-rs/crates/api-server/src/main.rs index e9da1150a..529060bb9 100644 --- a/server-rs/crates/api-server/src/main.rs +++ b/server-rs/crates/api-server/src/main.rs @@ -37,6 +37,7 @@ mod custom_world_foundation_draft; mod custom_world_result_prompts; mod custom_world_rpg_draft_prompts; mod editor_generation_config; +mod editor_generation_queue; mod editor_project; mod edutainment_baby_drawing; mod edutainment_baby_object; diff --git a/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs b/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs index 07eff0393..0296deb8a 100644 --- a/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs +++ b/server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs @@ -11,9 +11,21 @@ use serde_json::json; use shared_contracts::{assets, creation_audio}; use crate::{ - api_response::json_success_body, auth::AuthenticatedAccessToken, - editor_generation_config::EditorGenerationPricingConfig, http_error::AppError, - request_context::RequestContext, state::AppState, + api_response::json_success_body, + auth::AuthenticatedAccessToken, + editor_generation_config::EditorGenerationPricingConfig, + editor_generation_queue::{ + EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, + EditorGenerationQueuedResponse, editor_generation_queue_state, + editor_generation_source_entity_id, enqueue_editor_generation_job, + }, + editor_project::{ + EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item, + complete_editor_canvas_generation_with_items, + }, + http_error::AppError, + request_context::RequestContext, + state::AppState, tracking::record_external_generation_run_after_success, }; @@ -128,13 +140,33 @@ pub async fn generate_editor_sound_effect( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, Response> { - generate_editor_sound_effect_for_owner( - state, - request_context, - authenticated.claims().user_id().to_string(), - payload, - ) - .await + let Json(payload) = parse_json_payload(&request_context, payload)?; + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-sound-effect", + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + owner_user_id.as_str(), + EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成音效", + &payload, + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } + generate_editor_sound_effect_for_owner(state, request_context, owner_user_id, Ok(Json(payload))) + .await } pub(crate) async fn generate_editor_sound_effect_for_owner( @@ -144,6 +176,9 @@ pub(crate) async fn generate_editor_sound_effect_for_owner( payload: Result, JsonRejection>, ) -> Result, Response> { let Json(payload) = parse_json_payload(&request_context, payload)?; + let project_id = payload.project_id.clone(); + let canvas_completion = payload.canvas_completion.clone(); + let generation_inputs = payload.generation_inputs.clone(); let pricing = state .editor_generation_pricing() .map_err(|error| { @@ -192,6 +227,52 @@ pub(crate) async fn generate_editor_sound_effect_for_owner( .audio_src .ok_or_else(|| vector_engine_bad_gateway("音效生成完成但缺少播放地址")) .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let completed_project = if let Some(completion) = canvas_completion.as_ref() { + let layer_id = format!("layer-editor-sound-effect-{}", generated.task_id); + let item = build_editor_canvas_generated_layer_item( + completion, + EditorCanvasGeneratedLayerInput { + layer_id: layer_id.clone(), + resource_id: format!("local-resource-editor-sound-effect-{}", generated.task_id), + title: completion.title.trim().to_string(), + src: audio_src.clone(), + media_type: Some("audio".to_string()), + thumbnail_src: None, + original_width: EDITOR_AUDIO_WIDTH, + original_height: EDITOR_AUDIO_HEIGHT, + display_width: completion.placeholder.width, + display_height: completion.placeholder.height, + source_type: "generated".to_string(), + prompt: Some(normalized.prompt.clone()), + actual_prompt: Some(normalized.prompt.clone()), + model: Some(normalized.model.clone()), + provider: Some(generated.provider.clone()), + task_id: Some(generated.task_id.clone()), + object_key: generated.object_key.clone(), + asset_object_id: generated.asset_object_id.clone(), + duration_seconds: Some(u32::from(normalized.duration)), + source_resource_id: None, + asset_kind: Some("sound-effect".to_string()), + generation_inputs, + image_sequence_frames: None, + preview_video_path: None, + }, + ); + complete_editor_canvas_generation_with_items( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + canvas_completion.as_ref(), + vec![item], + Some(layer_id), + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))? + } else { + None + }; + let completed_project = + completed_project.and_then(|project| serde_json::to_value(project).ok()); Ok(json_success_body( Some(&request_context), @@ -210,6 +291,8 @@ pub(crate) async fn generate_editor_sound_effect_for_owner( task_id: generated.task_id, price_mud_points: normalized.price_mud_points, audio_kind: "sound-effect".to_string(), + project: completed_project, + queue_state: None, }, )) } @@ -220,11 +303,36 @@ pub async fn generate_editor_background_music( Extension(authenticated): Extension, payload: Result, JsonRejection>, ) -> Result, Response> { + let Json(payload) = parse_json_payload(&request_context, payload)?; + let owner_user_id = authenticated.claims().user_id().to_string(); + if !state.config.external_generation_mode.is_inline() { + let source_entity_id = editor_generation_source_entity_id( + payload.project_id.as_deref(), + "editor-background-music", + ); + let queue_job = enqueue_editor_generation_job( + &state, + &request_context, + owner_user_id.as_str(), + EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, + source_entity_id, + "图片画布生成背景音乐", + &payload, + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + return Ok(json_success_body( + Some(&request_context), + EditorGenerationQueuedResponse { + queue_state: editor_generation_queue_state(queue_job), + }, + )); + } generate_editor_background_music_for_owner( state, request_context, - authenticated.claims().user_id().to_string(), - payload, + owner_user_id, + Ok(Json(payload)), ) .await } @@ -236,6 +344,9 @@ pub(crate) async fn generate_editor_background_music_for_owner( payload: Result, JsonRejection>, ) -> Result, Response> { let Json(payload) = parse_json_payload(&request_context, payload)?; + let project_id = payload.project_id.clone(); + let canvas_completion = payload.canvas_completion.clone(); + let generation_inputs = payload.generation_inputs.clone(); let pricing = state .editor_generation_pricing() .map_err(|error| { @@ -282,6 +393,55 @@ pub(crate) async fn generate_editor_background_music_for_owner( .audio_src .ok_or_else(|| vector_engine_bad_gateway("背景音乐生成完成但缺少播放地址")) .map_err(|error| error.into_response_with_context(Some(&request_context)))?; + let completed_project = if let Some(completion) = canvas_completion.as_ref() { + let layer_id = format!("layer-editor-background-music-{}", generated.task_id); + let item = build_editor_canvas_generated_layer_item( + completion, + EditorCanvasGeneratedLayerInput { + layer_id: layer_id.clone(), + resource_id: format!( + "local-resource-editor-background-music-{}", + generated.task_id + ), + title: completion.title.trim().to_string(), + src: audio_src.clone(), + media_type: Some("audio".to_string()), + thumbnail_src: None, + original_width: EDITOR_AUDIO_WIDTH, + original_height: EDITOR_AUDIO_HEIGHT, + display_width: completion.placeholder.width, + display_height: completion.placeholder.height, + source_type: "generated".to_string(), + prompt: Some(normalized.gpt_description_prompt.clone()), + actual_prompt: Some(normalized.gpt_description_prompt.clone()), + model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()), + provider: Some(generated.provider.clone()), + task_id: Some(generated.task_id.clone()), + object_key: generated.object_key.clone(), + asset_object_id: generated.asset_object_id.clone(), + duration_seconds: None, + source_resource_id: None, + asset_kind: Some("background-music".to_string()), + generation_inputs, + image_sequence_frames: None, + preview_video_path: None, + }, + ); + complete_editor_canvas_generation_with_items( + &state, + owner_user_id.as_str(), + project_id.as_deref(), + canvas_completion.as_ref(), + vec![item], + Some(layer_id), + ) + .await + .map_err(|error| error.into_response_with_context(Some(&request_context)))? + } else { + None + }; + let completed_project = + completed_project.and_then(|project| serde_json::to_value(project).ok()); Ok(json_success_body( Some(&request_context), @@ -300,6 +460,8 @@ pub(crate) async fn generate_editor_background_music_for_owner( task_id: generated.task_id, price_mud_points: normalized.price_mud_points, audio_kind: "background-music".to_string(), + project: completed_project, + queue_state: None, }, )) } diff --git a/server-rs/crates/api-server/src/vector_engine_audio_generation/tests.rs b/server-rs/crates/api-server/src/vector_engine_audio_generation/tests.rs index 50c3d7ac6..53913ee9d 100644 --- a/server-rs/crates/api-server/src/vector_engine_audio_generation/tests.rs +++ b/server-rs/crates/api-server/src/vector_engine_audio_generation/tests.rs @@ -109,6 +109,9 @@ fn editor_sound_effect_request_normalizes_prompt_duration_and_resolves_price() { prompt: " 金币掉落叮当声 ".to_string(), model: None, duration: 7, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("editor sound effect request should normalize"); @@ -125,6 +128,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() { prompt: "金币掉落叮当声".to_string(), model: Some(" audio1.0 ".to_string()), duration: 5, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("Vidu audio model should be accepted"); @@ -134,6 +140,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() { prompt: "金币掉落叮当声".to_string(), model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()), duration: 5, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect_err("Suno text-to-sound should be disabled for editor sound effects"); @@ -148,6 +157,9 @@ fn editor_sound_effect_request_rejects_duration_outside_vidu_range() { prompt: "按钮点击".to_string(), model: None, duration, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect_err("duration outside 2-10 seconds should fail"); @@ -183,6 +195,9 @@ fn editor_background_music_request_forces_instrumental_and_resolves_price() { normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest { gpt_description_prompt: " 森林冒险背景音乐 ".to_string(), make_instrumental: false, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("editor background music request should normalize"); @@ -197,6 +212,9 @@ fn editor_background_music_request_rejects_prompt_over_documented_limit() { normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest { gpt_description_prompt: "乐".repeat(201), make_instrumental: true, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect_err("Suno gpt_description_prompt should follow Apifox 200 char limit"); diff --git a/server-rs/crates/shared-contracts/src/assets.rs b/server-rs/crates/shared-contracts/src/assets.rs index 20b21ada0..42fe67d42 100644 --- a/server-rs/crates/shared-contracts/src/assets.rs +++ b/server-rs/crates/shared-contracts/src/assets.rs @@ -3,6 +3,8 @@ use std::collections::BTreeMap; use serde::{Deserialize, Serialize}; use serde_json::Value; +use crate::external_generation::ExternalGenerationJobStatusRecord; + #[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct CreateDirectUploadTicketRequest { @@ -304,7 +306,7 @@ pub struct CharacterAnimationGenerateResponse { pub preview_video_path: Option, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorCharacterAnimationGenerateRequest { pub source_layer_id: String, @@ -317,6 +319,14 @@ pub struct EditorCharacterAnimationGenerateRequest { pub frame_count: u32, pub duration_seconds: u32, pub model: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub source_resource_id: Option, } #[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] @@ -328,7 +338,7 @@ pub struct EditorCharacterAnimationFramePayload { pub height: u32, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorCharacterAnimationGenerateResponse { pub ok: bool, @@ -343,9 +353,13 @@ pub struct EditorCharacterAnimationGenerateResponse { pub frame_height: u32, pub fps: u32, pub price_mud_points: u32, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub queue_state: Option, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorVideoGenerateRequest { pub prompt: String, @@ -363,13 +377,23 @@ pub struct EditorVideoGenerateRequest { pub reference_video_srcs: Vec, #[serde(default)] pub reference_audio_srcs: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub source_resource_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asset_kind: Option, } fn default_editor_video_web_search_enabled() -> bool { true } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorVideoGenerateResponse { pub ok: bool, @@ -390,6 +414,30 @@ pub struct EditorVideoGenerateResponse { pub duration_seconds: u32, pub resolution: String, pub price_mud_points: u32, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub queue_state: Option, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCanvasGenerationPlaceholderPayload { + pub x: f64, + pub y: f64, + pub width: f64, + pub height: f64, + pub original_width: f64, + pub original_height: f64, +} + +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct EditorCanvasGenerationCompletionPayload { + #[serde(default, skip_serializing_if = "Option::is_none")] + pub dialog_id: Option, + pub title: String, + pub placeholder: EditorCanvasGenerationPlaceholderPayload, } #[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] @@ -430,23 +478,35 @@ pub struct EditorIconSpritesheetGenerateResponse { pub price_mud_points: u32, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorSoundEffectGenerateRequest { pub prompt: String, #[serde(default, skip_serializing_if = "Option::is_none")] pub model: Option, pub duration: u8, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorBackgroundMusicGenerateRequest { pub gpt_description_prompt: String, pub make_instrumental: bool, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project_id: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub canvas_completion: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub generation_inputs: Option, } -#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)] +#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] #[serde(rename_all = "camelCase")] pub struct EditorAudioGenerateResponse { pub ok: bool, @@ -466,6 +526,10 @@ pub struct EditorAudioGenerateResponse { pub task_id: String, pub price_mud_points: u32, pub audio_kind: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub project: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub queue_state: Option, } #[derive(Clone, Debug, Serialize, Deserialize, PartialEq)] @@ -992,6 +1056,21 @@ mod tests { frame_count: 48, duration_seconds: 6, model: "seedance2.0-fast".to_string(), + project_id: Some("editor-project-1".to_string()), + canvas_completion: Some(EditorCanvasGenerationCompletionPayload { + dialog_id: Some("dialog-1".to_string()), + title: "角色动作".to_string(), + placeholder: EditorCanvasGenerationPlaceholderPayload { + x: 10.0, + y: 20.0, + width: 420.0, + height: 420.0, + original_width: 1024.0, + original_height: 1024.0, + }, + }), + generation_inputs: None, + source_resource_id: Some("resource-source".to_string()), }) .expect("request should serialize"); @@ -1002,6 +1081,9 @@ mod tests { ); assert!(payload.get("priceMudPoints").is_none()); assert_eq!(payload["model"], json!("seedance2.0-fast")); + assert_eq!(payload["projectId"], json!("editor-project-1")); + assert_eq!(payload["canvasCompletion"]["dialogId"], json!("dialog-1")); + assert_eq!(payload["sourceResourceId"], json!("resource-source")); } #[test] @@ -1024,6 +1106,8 @@ mod tests { frame_height: 1024, fps: 8, price_mud_points: 40, + project: None, + queue_state: None, }) .expect("response should serialize"); @@ -1052,6 +1136,11 @@ mod tests { reference_image_srcs: Vec::new(), reference_video_srcs: Vec::new(), reference_audio_srcs: Vec::new(), + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("request should serialize"); @@ -1078,6 +1167,11 @@ mod tests { "generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(), ], reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()], + project_id: None, + canvas_completion: None, + generation_inputs: None, + source_resource_id: None, + asset_kind: None, }) .expect("request should serialize"); @@ -1100,7 +1194,9 @@ mod tests { let payload = serde_json::to_value(EditorVideoGenerateResponse { ok: true, video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(), - object_key: Some("generated-character-drafts/editor-videos/task-1/preview.mp4".to_string()), + object_key: Some( + "generated-character-drafts/editor-videos/task-1/preview.mp4".to_string(), + ), asset_object_id: Some("assetobj_video_1".to_string()), width: 1280, height: 720, @@ -1113,6 +1209,8 @@ mod tests { duration_seconds: 5, resolution: "480p".to_string(), price_mud_points: 100, + project: None, + queue_state: None, }) .expect("response should serialize"); @@ -1178,6 +1276,9 @@ mod tests { prompt: "金币掉落叮当声".to_string(), model: Some("audio1.0".to_string()), duration: 7, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("sound request should serialize"); assert_eq!(sound_payload["prompt"], json!("金币掉落叮当声")); @@ -1198,6 +1299,9 @@ mod tests { prompt: "按钮确认短促音".to_string(), model: None, duration: 5, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("sound request with unset model should serialize"); assert!(unset_model_payload.get("model").is_none()); @@ -1206,6 +1310,9 @@ mod tests { let music_payload = serde_json::to_value(EditorBackgroundMusicGenerateRequest { gpt_description_prompt: "森林冒险背景音乐".to_string(), make_instrumental: true, + project_id: None, + canvas_completion: None, + generation_inputs: None, }) .expect("background music request should serialize"); assert_eq!( @@ -1229,6 +1336,8 @@ mod tests { task_id: "sound-task-1".to_string(), price_mud_points: 10, audio_kind: "sound-effect".to_string(), + project: None, + queue_state: None, }) .expect("audio response should serialize"); assert_eq!( diff --git a/src/components/creation-home/CreationLandingView.test.tsx b/src/components/creation-home/CreationLandingView.test.tsx index ce026690d..31997e2d1 100644 --- a/src/components/creation-home/CreationLandingView.test.tsx +++ b/src/components/creation-home/CreationLandingView.test.tsx @@ -6,12 +6,10 @@ import type { ContextType } from 'react'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { AuthUiContext } from '../auth/AuthUiContext'; -import type { PlatformPublicGalleryCard } from '../rpg-entry/rpgEntryWorldPresentation'; import { CreationLandingView } from './CreationLandingView'; const listEditorProjectsMock = vi.hoisted(() => vi.fn()); const createEditorProjectMock = vi.hoisted(() => vi.fn()); -const loadEditorAssetLibraryMock = vi.hoisted(() => vi.fn()); type AuthValue = NonNullable>; @@ -48,7 +46,6 @@ function createAuthValue(overrides: Partial = {}): AuthValue { vi.mock('../../services/image-editor/editorProjectClient', () => ({ listEditorProjects: listEditorProjectsMock, createEditorProject: createEditorProjectMock, - loadEditorAssetLibrary: loadEditorAssetLibraryMock, })); const projectItems = [ @@ -72,12 +69,10 @@ const projectItems = [ type RenderCreationLandingOptions = { authValue?: AuthValue | null; - publicGalleryEntries?: PlatformPublicGalleryCard[]; }; function renderCreationLanding({ authValue = createAuthValue(), - publicGalleryEntries = [], }: RenderCreationLandingOptions = {}) { const result = render( @@ -85,7 +80,6 @@ function renderCreationLanding({ onOpenProject={vi.fn()} onOpenProjects={vi.fn()} onOpenCommunity={vi.fn()} - publicGalleryEntries={publicGalleryEntries} /> , ); @@ -97,12 +91,10 @@ describe('CreationLandingView', () => { afterEach(() => { listEditorProjectsMock.mockReset(); createEditorProjectMock.mockReset(); - loadEditorAssetLibraryMock.mockReset(); }); it('renders the Taonier creation landing modules without external branding', async () => { listEditorProjectsMock.mockResolvedValueOnce(projectItems); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ folders: [], assets: [] }); renderCreationLanding(); @@ -129,7 +121,6 @@ describe('CreationLandingView', () => { const onOpenProject = vi.fn(); const onOpenProjects = vi.fn(); listEditorProjectsMock.mockResolvedValueOnce(projectItems); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ folders: [], assets: [] }); render( @@ -173,7 +164,6 @@ describe('CreationLandingView', () => { const user = userEvent.setup(); const onOpenProject = vi.fn(); listEditorProjectsMock.mockResolvedValueOnce([]); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ folders: [], assets: [] }); createEditorProjectMock.mockResolvedValueOnce({ projectId: 'created-project', title: '未命名画布', @@ -203,7 +193,6 @@ describe('CreationLandingView', () => { const user = userEvent.setup(); const onOpenCommunity = vi.fn(); listEditorProjectsMock.mockResolvedValueOnce([]); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ folders: [], assets: [] }); render( @@ -221,30 +210,32 @@ describe('CreationLandingView', () => { expect(createEditorProjectMock).not.toHaveBeenCalled(); }); - it('renders real account assets in showcase tabs without mock rows', async () => { + it('renders generated project resources in showcase tabs without mock rows', async () => { const user = userEvent.setup(); - listEditorProjectsMock.mockResolvedValueOnce([]); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ - folders: [], - assets: [ - { - assetId: 'asset-character', - folderId: 'folder-1', - label: '角色英雄', - imageSrc: 'data:image/png;base64,character', - width: 512, - height: 512, - sourceType: 'generated', - prompt: 'character hero prompt', - actualPrompt: null, - model: 'character-model', - provider: 'provider', - taskId: 'task-1', - authorName: '创作者A', - priceMudPoints: 20, - }, - ], - }); + listEditorProjectsMock.mockResolvedValueOnce([ + { + ...projectItems[0]!, + resources: [ + { + resourceId: 'resource-character', + projectId: 'project-newest', + label: '角色英雄', + imageSrc: 'data:image/png;base64,character', + width: 512, + height: 512, + sourceType: 'generated', + prompt: 'character hero prompt', + actualPrompt: null, + model: 'character-model', + provider: 'provider', + taskId: 'task-1', + assetKind: 'character', + authorName: '创作者A', + priceMudPoints: 20, + }, + ], + }, + ]); renderCreationLanding(); @@ -260,56 +251,57 @@ describe('CreationLandingView', () => { it('opens a fullscreen asset preview modal with thumbnail navigation', async () => { const user = userEvent.setup(); - listEditorProjectsMock.mockResolvedValueOnce([]); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ - folders: [], - assets: [ - { - assetId: 'asset-character', - folderId: 'folder-1', - label: '角色英雄', - imageSrc: 'data:image/png;base64,character', - width: 512, - height: 512, - sourceType: 'generated', - prompt: 'character hero prompt', - actualPrompt: null, - model: 'character-model', - provider: 'provider', - taskId: 'task-1', - assetKind: 'character', - authorName: '创作者A', - priceMudPoints: 20, - }, - { - assetId: 'asset-animation', - folderId: 'folder-1', - label: '待机动画', - imageSrc: 'data:image/png;base64,animation', - width: 512, - height: 512, - sourceType: 'generated', - prompt: 'idle animation prompt', - actualPrompt: null, - model: 'seedance2.0-fast', - provider: 'ark', - taskId: 'task-2', - assetKind: 'character-animation', - priceMudPoints: 40, - generationInputs: { - fields: [{ title: '动作描述', value: '待机动作' }], - references: [ - { - title: '角色图片', - label: '角色英雄', - refType: 'asset', - refId: 'asset-character', - }, - ], + listEditorProjectsMock.mockResolvedValueOnce([ + { + ...projectItems[0]!, + resources: [ + { + resourceId: 'resource-character', + projectId: 'project-newest', + label: '角色英雄', + imageSrc: 'data:image/png;base64,character', + width: 512, + height: 512, + sourceType: 'generated', + prompt: 'character hero prompt', + actualPrompt: null, + model: 'character-model', + provider: 'provider', + taskId: 'task-1', + assetKind: 'character', + authorName: '创作者A', + priceMudPoints: 20, }, - }, - ], - }); + { + resourceId: 'resource-animation', + projectId: 'project-newest', + label: '待机动画', + imageSrc: 'data:image/png;base64,animation', + width: 512, + height: 512, + sourceType: 'generated', + prompt: 'idle animation prompt', + actualPrompt: null, + model: 'seedance2.0-fast', + provider: 'ark', + taskId: 'task-2', + assetKind: 'character-animation', + priceMudPoints: 40, + generationInputs: { + fields: [{ title: '动作描述', value: '待机动作' }], + references: [ + { + title: '角色图片', + label: '角色英雄', + refType: 'project-resource', + refId: 'resource-character', + }, + ], + }, + }, + ], + }, + ]); renderCreationLanding(); @@ -332,46 +324,67 @@ describe('CreationLandingView', () => { ).toBe('true'); }); - it('uses Taonier featured as the account asset waterfall instead of creation entries', async () => { - listEditorProjectsMock.mockResolvedValueOnce([]); - loadEditorAssetLibraryMock.mockResolvedValueOnce({ - folders: [], - assets: [ - { - assetId: 'asset-pack-a', - folderId: 'folder-1', - label: '素材包 A', - imageSrc: 'data:image/png;base64,a', - width: 512, - height: 640, - sourceType: 'generated', - prompt: '素材包提示词', - actualPrompt: null, - model: null, - provider: null, - taskId: null, - }, - { - assetId: 'asset-pack-b', - folderId: 'folder-1', - label: '素材包 B', - imageSrc: 'data:image/png;base64,b', - width: 512, - height: 512, - sourceType: 'generated', - prompt: '第二组素材', - actualPrompt: null, - model: null, - provider: null, - taskId: null, - }, - ], - }); + it('uses Taonier featured as the generated project resource waterfall instead of creation entries', async () => { + listEditorProjectsMock.mockResolvedValueOnce([ + { + ...projectItems[0]!, + resources: [ + { + resourceId: 'resource-pack-a', + projectId: 'project-newest', + label: '素材包 A', + imageSrc: 'data:image/png;base64,a', + width: 512, + height: 640, + sourceType: 'generated', + prompt: '素材包提示词', + actualPrompt: null, + model: null, + provider: null, + taskId: null, + }, + { + resourceId: 'resource-uploaded', + projectId: 'project-newest', + label: '上传素材', + imageSrc: 'data:image/png;base64,uploaded', + width: 512, + height: 512, + sourceType: 'uploaded', + prompt: null, + actualPrompt: null, + model: null, + provider: null, + taskId: null, + }, + ], + }, + { + ...projectItems[1]!, + resources: [ + { + resourceId: 'resource-pack-b', + projectId: 'project-old', + label: '素材包 B', + imageSrc: 'data:image/png;base64,b', + width: 512, + height: 512, + sourceType: 'generated', + prompt: '第二组素材', + actualPrompt: null, + model: null, + provider: null, + taskId: null, + }, + ], + }, + ]); const { container } = renderCreationLanding(); expect(await screen.findByText('素材包 A')).toBeTruthy(); expect(screen.getByText('素材包 B')).toBeTruthy(); + expect(screen.queryByText('上传素材')).toBeNull(); expect( screen.getByLabelText('用户素材瀑布流').className, ).toContain('creation-landing__asset-waterfall'); @@ -383,52 +396,16 @@ describe('CreationLandingView', () => { expect(screen.queryByText('拼图关卡创作')).toBeNull(); }); - it('falls back to real public user assets for the featured waterfall', async () => { - const publicAsset = { - sourceType: 'puzzle', - workId: 'work-public-asset', - profileId: 'profile-public-asset', - publicWorkCode: 'PZ-PUBLIC-ASSET', - ownerUserId: 'owner-public-asset', - authorDisplayName: '公开作者', - worldName: '公开素材组合', - subtitle: '公开素材副标题', - summaryText: '公开用户生成素材提示词', - coverImageSrc: 'https://assets.example.test/public-asset.png', - coverSlides: [ - { - id: 'slide-1', - imageSrc: 'https://assets.example.test/public-asset-slide.png', - label: '公开素材图', - }, - ], - themeTags: ['素材包'], - visibility: 'published', - publishedAt: '2026-06-18T09:00:00.000Z', - updatedAt: '2026-06-18T10:00:00.000Z', - } satisfies PlatformPublicGalleryCard; - - const { container } = renderCreationLanding({ + it('keeps the featured waterfall empty when project resources are empty', async () => { + renderCreationLanding({ authValue: createAuthValue({ user: null, canAccessProtectedData: false, }), - publicGalleryEntries: [publicAsset], }); - expect(await screen.findByText('公开素材组合')).toBeTruthy(); - const card = screen - .getByText('公开素材组合') - .closest('.creation-landing__asset-card'); - expect(card).toBeTruthy(); - expect( - within(card as HTMLElement).getByText('公开用户生成素材提示词'), - ).toBeTruthy(); - expect(within(card as HTMLElement).getByText('公开作者')).toBeTruthy(); - expect( - container.querySelector('.creation-landing__asset-waterfall'), - ).toBeTruthy(); - expect(screen.queryByText('暂无素材')).toBeNull(); + expect(await screen.findByText('暂无素材')).toBeTruthy(); + expect(screen.queryByLabelText('用户素材瀑布流')).toBeNull(); expect(screen.queryByText('精选入口')).toBeNull(); }); }); diff --git a/src/components/creation-home/CreationLandingView.tsx b/src/components/creation-home/CreationLandingView.tsx index 23c79b9f0..981a31f7b 100644 --- a/src/components/creation-home/CreationLandingView.tsx +++ b/src/components/creation-home/CreationLandingView.tsx @@ -4,8 +4,7 @@ import { useCallback, useEffect, useMemo, useState } from 'react'; import { createEditorProject, listEditorProjects, - loadEditorAssetLibrary, - type EditorAssetLibrarySnapshot, + type EditorProjectResourceSnapshot, type EditorProjectSnapshot, } from '../../services/image-editor/editorProjectClient'; import { ApiClientError } from '../../services/apiClient'; @@ -20,7 +19,6 @@ import { UnifiedModal } from '../common/UnifiedModal'; import { ProjectCanvasCover } from '../project/ProjectCanvasCover'; import { ResolvedAssetImage } from '../ResolvedAssetImage'; import { ResolvedAssetVideo } from '../ResolvedAssetVideo'; -import type { PlatformPublicGalleryCard } from '../rpg-entry/rpgEntryWorldPresentation'; import { buildCreationShowcaseItems, SHOWCASE_TABS, @@ -33,7 +31,6 @@ type CreationLandingViewProps = { onOpenProject: (projectId: string) => void; onOpenProjects: () => void; onOpenCommunity?: () => void; - publicGalleryEntries?: PlatformPublicGalleryCard[]; }; type CreationFeatureItem = { @@ -360,7 +357,6 @@ export function CreationLandingView({ onOpenProject, onOpenProjects, onOpenCommunity, - publicGalleryEntries = [], }: CreationLandingViewProps) { const authUi = useAuthUi(); const isAuthenticated = Boolean(authUi?.user); @@ -370,9 +366,9 @@ export function CreationLandingView({ const [isLoadingProjects, setIsLoadingProjects] = useState(false); const [isCreatingProject, setIsCreatingProject] = useState(false); const [projectError, setProjectError] = useState(null); - const [assetLibrary, setAssetLibrary] = - useState(null); - const [isLoadingAssets, setIsLoadingAssets] = useState(false); + const [projectShowcaseResources, setProjectShowcaseResources] = useState< + EditorProjectResourceSnapshot[] + >([]); const [assetError, setAssetError] = useState(null); const [activeShowcaseTab, setActiveShowcaseTab] = useState('packs'); @@ -383,21 +379,24 @@ export function CreationLandingView({ const refreshRecentProjects = useCallback(() => { if (!isAuthenticated) { setRecentProjects([]); + setProjectShowcaseResources([]); setProjectError(null); + setAssetError(null); return; } setIsLoadingProjects(true); setProjectError(null); + setAssetError(null); listEditorProjects() .then((projects) => { - setRecentProjects( - [...projects] - .sort( - (left, right) => - new Date(right.updatedAt).getTime() - - new Date(left.updatedAt).getTime(), - ) - .slice(0, 4), + const sortedProjects = [...projects].sort( + (left, right) => + new Date(right.updatedAt).getTime() - + new Date(left.updatedAt).getTime(), + ); + setRecentProjects(sortedProjects.slice(0, 4)); + setProjectShowcaseResources( + sortedProjects.flatMap((project) => project.resources ?? []), ); }) .catch((error: unknown) => { @@ -405,42 +404,20 @@ export function CreationLandingView({ authUi?.openLoginModal(refreshRecentProjects); return; } + const message = + error instanceof Error ? error.message : '读取最近项目失败'; setProjectError( - error instanceof Error ? error.message : '读取最近项目失败', + message, ); + setAssetError(message); }) .finally(() => setIsLoadingProjects(false)); }, [authUi, isAuthenticated]); - const refreshAssetLibrary = useCallback(() => { - if (!isAuthenticated) { - setAssetLibrary(null); - setAssetError(null); - return; - } - setIsLoadingAssets(true); - setAssetError(null); - loadEditorAssetLibrary() - .then(setAssetLibrary) - .catch((error: unknown) => { - if (isUnauthorizedError(error)) { - return; - } - setAssetError( - error instanceof Error ? error.message : '读取素材展示失败', - ); - }) - .finally(() => setIsLoadingAssets(false)); - }, [isAuthenticated]); - useEffect(() => { refreshRecentProjects(); }, [refreshRecentProjects]); - useEffect(() => { - refreshAssetLibrary(); - }, [refreshAssetLibrary]); - const createProject = useCallback(() => { if (authUi && !authUi.user) { authUi.openLoginModal(createProject); @@ -466,16 +443,14 @@ export function CreationLandingView({ () => buildCreationShowcaseItems({ activeTab: activeShowcaseTab, - assetLibrary, - publicGalleryEntries, + projectResources: projectShowcaseResources, fallbackAuthorName: authUi?.user?.displayName, }), - [activeShowcaseTab, assetLibrary, authUi?.user?.displayName, publicGalleryEntries], + [activeShowcaseTab, projectShowcaseResources, authUi?.user?.displayName], ); - const isShowcaseLoading = isAuthenticated && isLoadingAssets; - const showcaseEmptyText = - publicGalleryEntries.length > 0 ? '暂无匹配素材' : '暂无素材'; + const isShowcaseLoading = isAuthenticated && isLoadingProjects; + const showcaseEmptyText = '暂无素材'; const renderAssetShowcase = () => { if (assetError) { diff --git a/src/components/creation-home/creationShowcaseModel.test.ts b/src/components/creation-home/creationShowcaseModel.test.ts index 90ffd8089..30f6d8c96 100644 --- a/src/components/creation-home/creationShowcaseModel.test.ts +++ b/src/components/creation-home/creationShowcaseModel.test.ts @@ -1,17 +1,14 @@ import { describe, expect, it } from 'vitest'; -import type { - EditorAssetLibrarySnapshot, - EditorAssetSnapshot, -} from '../../services/image-editor/editorProjectClient'; +import type { EditorProjectResourceSnapshot } from '../../services/image-editor/editorProjectClient'; import { buildCreationShowcaseItems } from './creationShowcaseModel'; -function createAsset( - overrides: Partial & Record, -): EditorAssetSnapshot { +function createResource( + overrides: Partial & Record, +): EditorProjectResourceSnapshot { return { - assetId: 'asset-1', - folderId: 'folder-1', + resourceId: 'resource-1', + projectId: 'project-1', label: '素材', imageSrc: 'data:image/png;base64,asset', width: 512, @@ -23,31 +20,53 @@ function createAsset( provider: null, taskId: null, ...overrides, - }; -} - -function createLibrary(assets: EditorAssetSnapshot[]): EditorAssetLibrarySnapshot { - return { - folders: [], - assets, - }; + } as EditorProjectResourceSnapshot; } describe('creationShowcaseModel', () => { + it('only builds showcase items from generated project resources', () => { + const items = buildCreationShowcaseItems({ + activeTab: 'packs', + projectResources: [ + createResource({ + resourceId: 'generated-1', + label: '画布生成素材', + sourceType: 'generated', + prompt: '真实画布生成', + }), + createResource({ + resourceId: 'uploaded-1', + label: '上传素材', + sourceType: 'uploaded', + prompt: '上传不应展示', + }), + createResource({ + resourceId: 'mock-1', + label: 'Mock 素材', + sourceType: 'mock_generated', + prompt: 'Mock 不应展示', + }), + ], + }); + + expect(items).toHaveLength(1); + expect(items[0]?.label).toBe('画布生成素材'); + }); + it('groups pack items by the same referenced spec image and sums known cost', () => { const items = buildCreationShowcaseItems({ activeTab: 'packs', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'spec-1', + projectResources: [ + createResource({ + resourceId: 'spec-1', label: '角色规范图', assetKind: 'spec', prompt: '规范提示词', priceMudPoints: 5, authorName: '创作者A', }), - createAsset({ - assetId: 'character-1', + createResource({ + resourceId: 'character-1', label: '角色主图', assetKind: 'character', priceMudPoints: 12, @@ -57,14 +76,13 @@ describe('creationShowcaseModel', () => { { title: '角色规范', label: '角色规范图', - refType: 'asset', + refType: 'project-resource', refId: 'spec-1', }, ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); @@ -81,15 +99,15 @@ describe('creationShowcaseModel', () => { it('groups assets when references point to the source project resource', () => { const items = buildCreationShowcaseItems({ activeTab: 'packs', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'spec-1', + projectResources: [ + createResource({ + resourceId: 'spec-1', label: '项目规范图', assetKind: 'spec', sourceResourceId: 'resource-spec-1', }), - createAsset({ - assetId: 'character-1', + createResource({ + resourceId: 'character-1', label: '项目角色图', assetKind: 'character', generationInputs: { @@ -104,8 +122,7 @@ describe('creationShowcaseModel', () => { ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); @@ -118,15 +135,15 @@ describe('creationShowcaseModel', () => { it('groups character animations under the referenced character image', () => { const items = buildCreationShowcaseItems({ activeTab: 'characters', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'character-1', + projectResources: [ + createResource({ + resourceId: 'character-1', label: '游戏角色图', assetKind: 'character', prompt: '角色提示词', }), - createAsset({ - assetId: 'animation-1', + createResource({ + resourceId: 'animation-1', label: '待机动画', assetKind: 'character-animation', imageSrc: 'data:image/png;base64,animation', @@ -136,14 +153,13 @@ describe('creationShowcaseModel', () => { { title: '角色图片', label: '游戏角色图', - refType: 'asset', + refType: 'project-resource', refId: 'character-1', }, ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); @@ -157,15 +173,15 @@ describe('creationShowcaseModel', () => { it('groups UI design extraction assets under the referenced UI image', () => { const items = buildCreationShowcaseItems({ activeTab: 'ui', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'ui-design-1', + projectResources: [ + createResource({ + resourceId: 'ui-design-1', label: '游戏UI图', assetKind: 'ui-design', prompt: 'UI提示词', }), - createAsset({ - assetId: 'icon-sheet-1', + createResource({ + resourceId: 'icon-sheet-1', label: 'UI子素材图集', assetKind: 'icon-spritesheet', generationInputs: { @@ -174,14 +190,14 @@ describe('creationShowcaseModel', () => { { title: 'UI设计图', label: '游戏UI图', - refType: 'asset', + refType: 'project-resource', refId: 'ui-design-1', }, ], }, }), - createAsset({ - assetId: 'icon-1', + createResource({ + resourceId: 'icon-1', label: '开始按钮', assetKind: 'icon', generationInputs: { @@ -190,14 +206,13 @@ describe('creationShowcaseModel', () => { { title: 'UI设计图', label: '游戏UI图', - refType: 'asset', + refType: 'project-resource', refId: 'ui-design-1', }, ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); @@ -212,16 +227,16 @@ describe('creationShowcaseModel', () => { it('counts a spritesheet task only once when UI children share the same task', () => { const items = buildCreationShowcaseItems({ activeTab: 'ui', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'ui-design-1', + projectResources: [ + createResource({ + resourceId: 'ui-design-1', label: '游戏UI图', assetKind: 'ui-design', model: 'gemini-3.1-flash-image-preview', taskId: 'task-ui-design', }), - createAsset({ - assetId: 'icon-sheet-1', + createResource({ + resourceId: 'icon-sheet-1', label: 'UI子素材图集', assetKind: 'icon-spritesheet', model: 'gpt-image-2', @@ -232,14 +247,14 @@ describe('creationShowcaseModel', () => { { title: 'UI设计图', label: '游戏UI图', - refType: 'asset', + refType: 'project-resource', refId: 'ui-design-1', }, ], }, }), - createAsset({ - assetId: 'icon-1', + createResource({ + resourceId: 'icon-1', label: '开始按钮', assetKind: 'icon', model: 'gpt-image-2', @@ -250,34 +265,32 @@ describe('creationShowcaseModel', () => { { title: 'UI设计图', label: '游戏UI图', - refType: 'asset', + refType: 'project-resource', refId: 'ui-design-1', }, ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); - expect(items[0]?.cost).toBe('32泥点'); + expect(items[0]?.cost).toBe('15泥点'); }); it('keeps music assets as audio preview items', () => { const items = buildCreationShowcaseItems({ activeTab: 'music', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'music-1', + projectResources: [ + createResource({ + resourceId: 'music-1', label: '背景音乐', imageSrc: '/generated-editor-audios/music.mp3', assetKind: 'background-music', prompt: '舒缓背景音乐', priceMudPoints: '5', }), - ]), - publicGalleryEntries: [], + ], fallbackAuthorName: '当前用户', }); @@ -290,21 +303,20 @@ describe('creationShowcaseModel', () => { it('keeps character animation videos out of the marketing tab', () => { const items = buildCreationShowcaseItems({ activeTab: 'marketing', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'animation-1', + projectResources: [ + createResource({ + resourceId: 'animation-1', label: '角色动画', assetKind: 'character-animation', imageSrc: '/generated-editor-videos/animation.mp4', }), - createAsset({ - assetId: 'poster-1', + createResource({ + resourceId: 'poster-1', label: '上线美宣视频', assetKind: 'publication-material', imageSrc: '/generated-editor-videos/poster.mp4', }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); @@ -315,9 +327,9 @@ describe('creationShowcaseModel', () => { it('infers costs from generated asset kinds when stored price is absent', () => { const items = buildCreationShowcaseItems({ activeTab: 'characters', - assetLibrary: createLibrary([ - createAsset({ - assetId: 'character-1', + projectResources: [ + createResource({ + resourceId: 'character-1', label: '游戏角色图', imageSrc: '/generated-editor-assets/character.png', objectKey: 'generated-editor-assets/character.png', @@ -325,8 +337,8 @@ describe('creationShowcaseModel', () => { model: 'gpt-image-2', prompt: '角色提示词', }), - createAsset({ - assetId: 'animation-1', + createResource({ + resourceId: 'animation-1', label: '待机动画', assetKind: 'character-animation', imageSrc: '/generated-editor-videos/animation.mp4', @@ -339,18 +351,17 @@ describe('creationShowcaseModel', () => { { title: '角色图片', label: '游戏角色图', - refType: 'asset', + refType: 'project-resource', refId: 'character-1', }, ], }, }), - ]), - publicGalleryEntries: [], + ], }); expect(items).toHaveLength(1); - expect(items[0]?.cost).toBe('140泥点'); + expect(items[0]?.cost).toBe('123泥点'); expect(items[0]?.previews[0]?.objectKey).toBe( 'generated-editor-assets/character.png', ); diff --git a/src/components/creation-home/creationShowcaseModel.ts b/src/components/creation-home/creationShowcaseModel.ts index 2e6f299c9..bd5fa4a69 100644 --- a/src/components/creation-home/creationShowcaseModel.ts +++ b/src/components/creation-home/creationShowcaseModel.ts @@ -1,7 +1,7 @@ import type { EditorAssetGenerationInputReference, - EditorAssetLibrarySnapshot, EditorAssetSnapshot, + EditorProjectResourceSnapshot, } from '../../services/image-editor/editorProjectClient'; import { calculateCharacterAnimationPrice, @@ -15,12 +15,6 @@ import { DEFAULT_VIDEO_DURATION_SECONDS, DEFAULT_VIDEO_MODEL, } from '../image-editor/ImageCanvasGenerationModel'; -import { - resolvePlatformWorkAuthorDisplayName, - resolvePlatformWorldCoverImage, - resolvePlatformWorldCoverSlides, - type PlatformPublicGalleryCard, -} from '../rpg-entry/rpgEntryWorldPresentation'; export type ShowcaseTabId = | 'packs' @@ -91,6 +85,10 @@ function assetRecord(asset: EditorAssetSnapshot) { return asset as Record; } +function projectResourceRecord(resource: EditorProjectResourceSnapshot) { + return resource as unknown as Record; +} + function normalizeText(value: unknown) { return typeof value === 'string' && value.trim() ? value.trim() : ''; } @@ -162,6 +160,9 @@ function getSelfAssetKey(asset: EditorAssetSnapshot) { function getAssetReferenceKeys(asset: EditorAssetSnapshot) { const keys = [getSelfAssetKey(asset)]; + if (asset.assetId?.trim()) { + keys.push(`project-resource:${asset.assetId.trim()}`); + } if (asset.sourceResourceId?.trim()) { keys.push(`project-resource:${asset.sourceResourceId.trim()}`); } @@ -174,6 +175,61 @@ function getAssetReferenceKeys(asset: EditorAssetSnapshot) { return keys; } +function projectResourceToShowcaseAsset( + resource: EditorProjectResourceSnapshot, +): EditorAssetSnapshot { + const record = projectResourceRecord(resource); + const fallbackLabel = + normalizeText(record.label) || + normalizeText(record.title) || + resource.assetKind?.trim() || + '项目素材'; + + return { + ...record, + assetId: resource.resourceId, + folderId: `project:${resource.projectId}`, + label: fallbackLabel, + imageSrc: resource.imageSrc, + objectKey: resource.objectKey, + assetObjectId: resource.assetObjectId, + width: resource.width, + height: resource.height, + sourceType: resource.sourceType, + prompt: resource.prompt ?? null, + actualPrompt: resource.actualPrompt ?? null, + model: resource.model ?? null, + provider: resource.provider ?? null, + taskId: resource.taskId ?? null, + durationSeconds: resource.durationSeconds, + sourceResourceId: resource.sourceResourceId, + assetKind: resource.assetKind, + generationInputs: resource.generationInputs, + createdAt: resource.createdAt, + updatedAt: resource.updatedAt, + } as EditorAssetSnapshot; +} + +function buildGeneratedProjectResourceAssets( + resources: EditorProjectResourceSnapshot[], +) { + const seenResourceIds = new Set(); + + return resources + .filter( + (resource) => + resource.sourceType === 'generated' && resource.imageSrc.trim(), + ) + .filter((resource) => { + if (seenResourceIds.has(resource.resourceId)) { + return false; + } + seenResourceIds.add(resource.resourceId); + return true; + }) + .map(projectResourceToShowcaseAsset); +} + function findReference( asset: EditorAssetSnapshot, matcher: (reference: EditorAssetGenerationInputReference) => boolean, @@ -641,114 +697,21 @@ function toLibraryShowcaseItem( }; } -function matchesPublicGalleryTab( - entry: PlatformPublicGalleryCard, - tab: ShowcaseTabId, -) { - if (tab === 'packs') { - return true; - } - if (tab === 'games') { - return false; - } - - const themeTags = 'themeTags' in entry ? entry.themeTags : []; - const haystack = [ - entry.worldName, - entry.subtitle, - entry.summaryText, - entry.sourceType, - ...themeTags, - ] - .join(' ') - .toLowerCase(); - - if (tab === 'characters') { - return /角色|character|rpg|visual/u.test(haystack); - } - if (tab === 'ui') { - return /\bui\b|界面|图标|icon|puzzle/u.test(haystack); - } - if (tab === 'music') { - return /音乐|音效|audio|music|sound/u.test(haystack); - } - if (tab === 'marketing') { - return /美宣|宣传|poster|video|视觉|visual/u.test(haystack); - } - if (tab === 'effects') { - return /特效|effect|vfx/u.test(haystack); - } - if (tab === 'scenes') { - return /场景|scene|background|world|rpg/u.test(haystack); - } - return false; -} - -function toPublicGalleryShowcaseItems( - entries: PlatformPublicGalleryCard[], - tab: ShowcaseTabId, -): ShowcaseAssetItem[] { - const usedImageSrcs = new Set(); - const items: ShowcaseAssetItem[] = []; - - entries - .filter((entry) => matchesPublicGalleryTab(entry, tab)) - .forEach((entry) => { - const coverImageSrc = resolvePlatformWorldCoverImage(entry).trim(); - const imageCandidates = [ - ...resolvePlatformWorldCoverSlides(entry).map((slide) => ({ - src: slide.imageSrc.trim(), - label: slide.label?.trim() || entry.worldName, - })), - { src: coverImageSrc, label: entry.worldName }, - ].filter((candidate) => candidate.src); - - imageCandidates.forEach((candidate, index) => { - if (usedImageSrcs.has(candidate.src)) { - return; - } - usedImageSrcs.add(candidate.src); - items.push({ - id: `public:${entry.sourceType}:${entry.profileId}:${index}`, - label: index === 0 ? entry.worldName : `${entry.worldName} ${index + 1}`, - previews: [ - { - id: `public:${entry.profileId}:${index}`, - label: candidate.label, - src: candidate.src, - mediaType: 'image', - width: index === 0 ? 720 : 640, - height: index === 0 ? 480 : 720, - }, - ], - prompt: entry.summaryText || entry.subtitle || UNKNOWN_META_VALUE, - author: resolvePlatformWorkAuthorDisplayName(entry), - cost: UNKNOWN_META_VALUE, - }); - }); - }); - - return items.slice(0, 36); -} - export function buildCreationShowcaseItems({ activeTab, - assetLibrary, - publicGalleryEntries, + projectResources, fallbackAuthorName, }: { activeTab: ShowcaseTabId; - assetLibrary: EditorAssetLibrarySnapshot | null; - publicGalleryEntries: PlatformPublicGalleryCard[]; + projectResources: EditorProjectResourceSnapshot[]; fallbackAuthorName?: string | null; }) { - const libraryItems = getGroupsForTab(assetLibrary?.assets ?? [], activeTab) + const projectItems = getGroupsForTab( + buildGeneratedProjectResourceAssets(projectResources), + activeTab, + ) .map((group) => toLibraryShowcaseItem(group, fallbackAuthorName)) .filter((item): item is ShowcaseAssetItem => item !== null); - if (libraryItems.length > 0) { - return libraryItems.slice(0, 36); - } - - return toPublicGalleryShowcaseItems(publicGalleryEntries, activeTab); + return projectItems.slice(0, 36); } diff --git a/src/components/image-editor/ImageCanvasEditorModel.test.ts b/src/components/image-editor/ImageCanvasEditorModel.test.ts index 2c1f980c4..5f3a56dc6 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.test.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.test.ts @@ -27,9 +27,11 @@ describe('ImageCanvasEditorModel', () => { expect(canvasDisplayScaleToViewportScale(0.5)).toBe(0.25); expect(canvasDisplayScaleToViewportScale(1)).toBe(0.5); expect(canvasDisplayScaleToViewportScale(2)).toBe(1); + expect(canvasDisplayScaleToViewportScale(0.05)).toBe(0.025); expect(viewportScaleToCanvasDisplayScale(0.25)).toBe(0.5); expect(viewportScaleToCanvasDisplayScale(0.5)).toBe(1); expect(viewportScaleToCanvasDisplayScale(1)).toBe(2); + expect(formatCanvasDisplayScalePercent(0.025)).toBe('5%'); expect(formatCanvasDisplayScalePercent(0.5)).toBe('100%'); expect(formatCanvasDisplayScalePercent(Number.NaN)).toBe('100%'); expect( @@ -705,4 +707,59 @@ describe('ImageCanvasEditorModel', () => { horizontal: 250, }); }); + + it('keeps equal-spacing snap focused on nearby overlapping layers', () => { + const movingLayer: CanvasLayer = { + id: 'moving', + resourceId: 'resource-moving', + title: '移动图', + src: 'data:image/png;base64,moving', + x: 0, + y: 0, + width: 80, + height: 80, + originalWidth: 80, + originalHeight: 80, + zIndex: 1, + sourceType: 'uploaded', + }; + const firstLayer: CanvasLayer = { + ...movingLayer, + id: 'first', + resourceId: 'resource-first', + x: 100, + y: 160, + zIndex: 2, + }; + const secondLayer: CanvasLayer = { + ...movingLayer, + id: 'second', + resourceId: 'resource-second', + x: 260, + y: 160, + zIndex: 3, + }; + const offscreenLayers = Array.from({ length: 80 }, (_, index) => ({ + ...movingLayer, + id: `offscreen-${index}`, + resourceId: `resource-offscreen-${index}`, + x: 2400 + index * 96, + y: 1600, + zIndex: 10 + index, + })); + + const snapped = resolveSnappedLayerPosition( + movingLayer, + 417, + 160, + [movingLayer, ...offscreenLayers, firstLayer, secondLayer], + 1, + ); + + expect(snapped.x).toBe(420); + expect(snapped.guide).toEqual({ + vertical: 460, + horizontal: 160, + }); + }); }); diff --git a/src/components/image-editor/ImageCanvasEditorModel.ts b/src/components/image-editor/ImageCanvasEditorModel.ts index ea803eb85..33dd17702 100644 --- a/src/components/image-editor/ImageCanvasEditorModel.ts +++ b/src/components/image-editor/ImageCanvasEditorModel.ts @@ -30,13 +30,14 @@ export const EDITOR_ASSET_FOLDERS: EditorAssetFolder[] = [ export const CANVAS_WORLD_SIZE = 12000; export const CANVAS_WORLD_ORIGIN = CANVAS_WORLD_SIZE / 2; -export const MIN_SCALE = 0.24; +export const MIN_SCALE = 0.025; export const MAX_SCALE = 3.2; export const CANVAS_DISPLAY_SCALE_BASE = 0.5; export const TOOLBAR_HALF_WIDTH = 132; export const DEFAULT_CANVAS_SIZE = { width: 900, height: 640 }; export const SNAP_THRESHOLD_SCREEN_PX = 18; export const SNAP_DISTRIBUTION_OVERLAP_TOLERANCE = 1; +const SNAP_DISTRIBUTION_PAIR_LOOKAHEAD = 3; export const FIT_VIEW_PADDING = 10; export const MINIMAP_SIZE = { width: 132, height: 84 }; export const MINIMAP_PADDING = 8; @@ -1291,10 +1292,19 @@ function findNearestEqualSpacingSnap({ items: CanvasSnapItem[]; threshold: number; }): SnapCandidate | null { - const orderedItems = [...items].sort( - (firstItem, secondItem) => - getSnapItemStart(firstItem, axis) - getSnapItemStart(secondItem, axis), - ); + const orderedItems = items + .filter((item) => + snapItemsOverlapOnCrossAxis( + proposedCrossStart, + movingCrossSize, + item, + axis, + ), + ) + .sort( + (firstItem, secondItem) => + getSnapItemStart(firstItem, axis) - getSnapItemStart(secondItem, axis), + ); let nearest: SnapCandidate | null = null; for (let firstIndex = 0; firstIndex < orderedItems.length; firstIndex += 1) { @@ -1304,29 +1314,14 @@ function findNearestEqualSpacingSnap({ } for ( let secondIndex = firstIndex + 1; - secondIndex < orderedItems.length; + secondIndex < orderedItems.length && + secondIndex <= firstIndex + SNAP_DISTRIBUTION_PAIR_LOOKAHEAD; secondIndex += 1 ) { const secondItem = orderedItems[secondIndex]; if (!secondItem) { continue; } - if ( - !snapItemsOverlapOnCrossAxis( - proposedCrossStart, - movingCrossSize, - firstItem, - axis, - ) || - !snapItemsOverlapOnCrossAxis( - proposedCrossStart, - movingCrossSize, - secondItem, - axis, - ) - ) { - continue; - } const firstStart = getSnapItemStart(firstItem, axis); const firstEnd = getSnapItemEnd(firstItem, axis); diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts index 5b2a8d711..2fe74381b 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts @@ -118,7 +118,7 @@ describe('ImageCanvasGenerationDialogModel', () => { }); }); - it('creates character and icon generation drafts with model dimensions', () => { + it('creates character generation drafts with model dimensions and icon drafts with icon frame dimensions', () => { const canvasSize = { width: 960, height: 720 }; const viewport = { x: 0, y: 0, scale: 1 }; @@ -164,10 +164,12 @@ describe('ImageCanvasGenerationDialogModel', () => { '冻结按钮', ], placeholder: { - x: -32, - y: -152, - width: 1024, - height: 1024, + x: 300, + y: 180, + width: 360, + height: 360, + originalWidth: 512, + originalHeight: 512, }, }); }); diff --git a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts index 070daac06..989e45a10 100644 --- a/src/components/image-editor/ImageCanvasGenerationDialogModel.ts +++ b/src/components/image-editor/ImageCanvasGenerationDialogModel.ts @@ -31,6 +31,8 @@ import { DEFAULT_VIDEO_WEB_SEARCH_ENABLED, EDITOR_IMAGE_DIMENSION_OPTIONS, ICON_DESCRIPTION_LIMIT, + ICON_FRAME_DISPLAY_SIZE, + ICON_FRAME_ORIGINAL_SIZE, IMAGE_MODEL_GPT_IMAGE_2, inferEditorImageAspectRatio, inferEditorImageSizeLabel, @@ -263,11 +265,6 @@ export function createIconGenerationDialogDraft({ const worldCenter = getViewportWorldCenter({ canvasSize, viewport }); const normalizedImageModel = normalizeEditorImageModel(imageModel); const dimensionDefaults = resolveImageDimensionDefaults(normalizedImageModel); - const placeholderSize = resolveEditorImageGenerationPixelSize({ - model: normalizedImageModel, - aspectRatio: dimensionDefaults.aspectRatio, - imageSize: dimensionDefaults.imageSize, - }); return { mode: 'icon', prompt: formatIconDescriptionsText(DEFAULT_ICON_DESCRIPTIONS), @@ -279,12 +276,12 @@ export function createIconGenerationDialogDraft({ aspectRatio: dimensionDefaults.aspectRatio, imageSize: dimensionDefaults.imageSize, placeholder: { - x: worldCenter.x - placeholderSize.width / 2, - y: worldCenter.y - placeholderSize.height / 2, - width: placeholderSize.width, - height: placeholderSize.height, - originalWidth: placeholderSize.width, - originalHeight: placeholderSize.height, + x: worldCenter.x - ICON_FRAME_DISPLAY_SIZE.width / 2, + y: worldCenter.y - ICON_FRAME_DISPLAY_SIZE.height / 2, + width: ICON_FRAME_DISPLAY_SIZE.width, + height: ICON_FRAME_DISPLAY_SIZE.height, + originalWidth: ICON_FRAME_ORIGINAL_SIZE.width, + originalHeight: ICON_FRAME_ORIGINAL_SIZE.height, }, }; } diff --git a/src/components/image-editor/ImageCanvasInteractionModel.test.ts b/src/components/image-editor/ImageCanvasInteractionModel.test.ts index d461ca4cc..7356be03f 100644 --- a/src/components/image-editor/ImageCanvasInteractionModel.test.ts +++ b/src/components/image-editor/ImageCanvasInteractionModel.test.ts @@ -84,10 +84,10 @@ describe('ImageCanvasInteractionModel', () => { expect( fitViewportToLayers({ - layers: [createLayer({ width: 5000, height: 3000 })], + layers: [createLayer({ width: 50000, height: 30000 })], canvasSize: { width: 900, height: 640 }, })?.scale, - ).toBe(0.24); + ).toBe(0.025); }); it('maps client points to canvas and world coordinates with safe fallbacks', () => { @@ -142,6 +142,21 @@ describe('ImageCanvasInteractionModel', () => { expect(zoomedViewport.x).toBeCloseTo(-10); expect(zoomedViewport.y).toBeCloseTo(10); expect(zoomedViewport.scale).toBe(1.1); + + expect( + zoomViewportFromWheel({ + viewport: { x: 10, y: 20, scale: 0.03 }, + deltaY: 120, + screenPoint: { x: 210, y: 120 }, + }).scale, + ).toBe(0.027); + expect( + zoomViewportFromWheel({ + viewport: { x: 10, y: 20, scale: 0.027 }, + deltaY: 120, + screenPoint: { x: 210, y: 120 }, + }).scale, + ).toBe(0.025); }); it('selects layers intersecting the marquee in screen space', () => { diff --git a/src/components/image-editor/ImageCanvasWorldView.test.tsx b/src/components/image-editor/ImageCanvasWorldView.test.tsx index c119945c1..7f7c2b6da 100644 --- a/src/components/image-editor/ImageCanvasWorldView.test.tsx +++ b/src/components/image-editor/ImageCanvasWorldView.test.tsx @@ -130,6 +130,23 @@ describe('ImageCanvasWorldView', () => { expect(screen.queryByRole('status', { name: '' })).toBeNull(); }); + it('keeps the image layer frame visible while the image is loading', () => { + useResolvedAssetReadUrlMock.mockImplementation(() => ({ + resolvedUrl: '', + isResolving: true, + shouldResolve: true, + })); + + renderWorldView(); + + const layerButton = screen.getByRole('button', { name: '选择角色主图' }); + + expect( + layerButton.querySelector('.image-canvas-editor__layer-image-loading'), + ).toBeTruthy(); + expect(within(layerButton).queryByAltText('画布图片：角色主图')).toBeNull(); + }); + it('resolves image layers from objectKey before falling back to src', () => { useResolvedAssetReadUrlMock.mockImplementation( (_source: string, options?: { objectKey?: string | null }) => ({ @@ -660,6 +677,28 @@ describe('ImageCanvasWorldView', () => { ).toBe('4'); }); + it('does not inverse-scale the icon generator decorative tile when zoomed out', () => { + const dialog = createGenerationDialog({ + id: 'dialog-icon-decorative-scale', + mode: 'icon', + }); + + renderWorldView({ + viewport: { x: 0, y: 0, scale: 0.25 }, + canvasGenerationDialogs: [dialog], + generateDialog: null, + }); + + const frame = screen.getByRole('button', { name: '图标素材生成占位图' }); + const iconTile = frame.querySelector( + '.image-canvas-editor__generation-frame-icon', + ) as HTMLElement; + + expect( + iconTile.style.getPropertyValue('--image-canvas-editor-inverse-scale'), + ).toBe(''); + }); + it('uses a plain information icon for metadata corner actions', () => { const layer = createLayer(); const { props } = renderWorldView({ layers: [layer] }); diff --git a/src/components/image-editor/ImageCanvasWorldView.tsx b/src/components/image-editor/ImageCanvasWorldView.tsx index 9b9626ebf..473a82eea 100644 --- a/src/components/image-editor/ImageCanvasWorldView.tsx +++ b/src/components/image-editor/ImageCanvasWorldView.tsx @@ -629,17 +629,48 @@ function ImageCanvasImageLayer({ layer: CanvasLayer; mediaTransform?: string; }) { - const { resolvedUrl } = useResolvedAssetReadUrl(layer.src, { - objectKey: layer.objectKey, - refreshKey: layer.taskId ?? layer.resourceId, - }); + const { resolvedUrl, isResolving, shouldResolve } = useResolvedAssetReadUrl( + layer.src, + { + objectKey: layer.objectKey, + refreshKey: layer.taskId ?? layer.resourceId, + }, + ); + const [isLoaded, setIsLoaded] = useState(false); + + useEffect(() => { + setIsLoaded(false); + }, [resolvedUrl]); + + const isLoading = + isResolving || + (shouldResolve && !resolvedUrl) || + Boolean(resolvedUrl && !isLoaded); return ( - {`画布图片：${layer.title}`} + <> + {isLoading ? ( +