补齐画板视频封面持久化

生成画板视频时截取首帧并上传为封面

将视频生成结果写入素材库并保留 thumbnailSrc

同步后端契约、SpacetimeDB 绑定、前端素材库和画布层映射

补充相关测试与文档说明
This commit is contained in:
2026-06-28 00:22:59 +08:00
parent 5976c2b337
commit 63790e22ef
24 changed files with 348 additions and 65 deletions
@@ -1810,6 +1810,12 @@
"imageSrc": {
"type": "string"
},
"thumbnailSrc": {
"type": [
"string",
"null"
]
},
"objectKey": {
"type": [
"string",
@@ -124,7 +124,7 @@
## 2026-06-19 图片画布素材生成元数据上移到资源和素材
- 背景:角色、图标、UI 设计图、视频和音频等生成结果会在图片信息页展示用户可见输入快照;此前这些 `assetKind/generationInputs` 主要保存在画布 layer JSON 中,素材进入账号级素材库后跨项目复用和刷新恢复都依赖画布布局,不符合素材库作为账号级事实源的边界。
- 决策:普通图层的新保存不再把 `assetKind/generationInputs` 写入 `editor_canvas.layers_json``editor_project_resource` 保存项目画布资源快照的 `asset_kind/generation_inputs_json``editor_asset` 保存账号级素材的同名元数据;图片 / 图标 / UI 提取等生成 BFF 在请求携带 `projectId` / `assetFolderId` 时由后端创建新 resource / asset 并把快照回传前端,前端只用回包更新画布图层和素材栏,不再把同一生成结果二次调用保存接口。前端加载时优先从 resource / asset 恢复素材类别和生成输入快照,旧 layout 中的同名字段只作为历史兼容兜底。生成器对象本身仍作为 `itemType="generation-dialog"` 保存在画布布局中。
- 决策:普通图层的新保存不再把 `assetKind/generationInputs` 写入 `editor_canvas.layers_json``editor_project_resource` 保存项目画布资源快照的 `asset_kind/generation_inputs_json``editor_asset` 保存账号级素材的同名元数据和可选封面 `thumbnail_src`;图片 / 图标 / UI 提取等生成 BFF 在请求携带 `projectId` / `assetFolderId` 时由后端创建新 resource / asset 并把快照回传前端,前端只用回包更新画布图层和素材栏,不再把同一生成结果二次调用保存接口。生成视频由后端单独抽取首帧封面写入 `editor_asset.thumbnail_src` 和画布图层 `thumbnailSrc`,刷新素材库或从素材库拖回画布时继续作为视频 poster 使用。前端加载时优先从 resource / asset 恢复素材类别和生成输入快照,旧 layout 中的同名字段只作为历史兼容兜底。生成器对象本身仍作为 `itemType="generation-dialog"` 保存在画布布局中。
- 影响范围:`server-rs/crates/spacetime-module/src/editor_project_storage.rs``server-rs/crates/spacetime-client/src/mapper/editor_project.rs``server-rs/crates/api-server/src/editor_project.rs``src/services/image-editor/editorProjectClient.ts`、图片画布 hydrate / serialize / project persistence / asset library 代码和后端数据契约文档。
- 验证方式:运行 `npm run spacetime:generate``npm run check:spacetime-schema``npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/services/image-editor/editorProjectClient.test.ts``npm run typecheck``npm run check:encoding``git diff --check`,并按需补充 `cargo check -p spacetime-client -p api-server --manifest-path server-rs/Cargo.toml`
- 关联文档:`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md``docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`
@@ -2867,7 +2867,7 @@
## 2026-06-17 图片画布生成器快照纳入画布布局
- 背景:生成占位图和生成器对话框里包含用户输入、参数、参考图、占位框位置和生成结果绑定,刷新后丢失会让已生成图片无法回到 Lovart 式跟随编辑状态。
- 决策:生成器对象统一作为 `editor_canvas` 布局 JSON 的 `itemType: "generation-dialog"` 项保存,不新增表;成功生成后仍保留生成器快照和最后占位框位置,并通过 `generatedLayerId` 锚定到成品图层,渲染时不重复显示灰色占位框。图片类生成结果同步写入账号级素材库;视频结果当前只作为画布视频资源保存
- 决策:生成器对象统一作为 `editor_canvas` 布局 JSON 的 `itemType: "generation-dialog"` 项保存,不新增表;成功生成后仍保留生成器快照和最后占位框位置,并通过 `generatedLayerId` 锚定到成品图层,渲染时不重复显示灰色占位框。图片类生成视频结果同步写入账号级素材库;生成视频素材当前没有独立 poster 字段,素材栏以视频图标叠层展示
- 影响范围:图片画布 layout 序列化 / hydrate、生成工作流、生成器渲染、项目自动保存、素材库回填和编辑器技术方案。
- 验证方式:`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useCanvasGenerationDialogs.test.tsx src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/components/image-editor/ImageCanvasWorldView.test.tsx src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx``npm run typecheck``npm run check:encoding``git diff --check`、浏览器刷新 smoke。
- 关联文档:`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
@@ -44,13 +44,13 @@
- 新增 `editor_project` 表保存图片画布工程:`projectId``ownerUserId`、标题、创建时间和更新时间;历史 layout 字段暂保留为兼容列,不再作为权威画布数据。
- 新增 `editor_canvas` 表保存工程下的画布:`canvasId``projectId``ownerUserId`、标题、viewport、图层布局 JSON、创建时间和更新时间。当前编辑器使用项目默认画布,后续可扩展为一个 project 下多个 canvas。
- 新增 `editor_asset_folder` 表保存账号级素材文件夹:`folderId``ownerUserId`、名称、排序、折叠状态、系统默认标记、创建时间和更新时间。素材文件夹不归属于 project,同一个账号进入任一项目都能看到。
- 新增 `editor_asset` 表保存账号级素材:`assetId``ownerUserId``folderId`、名称、图片读取地址、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、`assetKind``generationInputs`、创建时间和更新时间。素材只跟账号走,不跟 project 走;角色、图标、UI 设计图、视频和音频等生成结果的用户可见输入快照随素材保存。
- 新增 `editor_asset` 表保存账号级素材:`assetId``ownerUserId``folderId`、名称、图片读取地址、可选封面 `thumbnailSrc`OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、`assetKind``generationInputs`、创建时间和更新时间。素材只跟账号走,不跟 project 走;角色、图标、UI 设计图、视频和音频等生成结果的用户可见输入快照随素材保存。
- `editor_project_resource` 表保存工程画布引用过的资源快照:`resourceId``projectId``ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind``generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource,图层只引用 resourceId;图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource,并把 `resourceId` 随生成响应返回给前端。图片生成请求如果同时携带 `canvasCompletion`(生成器 `dialogId`、标题和占位框,或无 dialog 的右侧完成占位),BFF / worker 在生成成功后必须直接读取当前项目布局,优先使用最新 `generation-dialog` 占位框位置;只有当前布局仍存在对应 `generation-dialog` 时才插入轻量结果图层、把生成器标记为 `idle` 并写入 `generatedLayerId`,沿用后端当前 viewport 保存布局,再返回或刷新最新项目快照;前端只应用该快照刷新显示,不把生成完成态作为本地业务真相,也不在项目加载时根据资源行推断完成态。有项目上下文但后端没有返回项目快照时,前端不得本地补结果图层,只保留当前生成器交互状态等待下一次项目刷新。
- 项目封面图是画布当前视口栅格化后的静态快照资源,不在项目列表页临时重放 `layers + viewport`。前端在项目加载后和防抖保存 layout 时生成 320x240 PNG,走私有 OSS / asset object 上传,再创建 `editor_project_resource`,其中 `assetKind="project-cover-snapshot"``sourceType="uploaded"``/project``/creation` 最近项目卡只读取最新封面快照资源渲染;没有封面快照时显示普通项目占位,不回退为实时画布组合。
- 图片、音频、视频和角色动画帧文件本体继续走 OSS / asset object;浏览器读取私有 generated 对象统一经 `/api/assets/read-url` 换签,签名 URL 可在 session 内复用,但不得作为持久化真相。`/api/assets/read-url` 属于页面展示层高频后台请求,前端统一在 `assetReadUrlService` 内做同 key pending 去重、session 缓存和跨组件节流;UI 设计切片、角色动画帧或大量素材恢复时不得绕过该服务并发换签,否则单页可在同一秒内打满发布入口 `genarrative_api_rps` burst。
- 登录态上传和生成结果必须先落 OSS / asset object,再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/<objectKey>"``objectKey``assetObjectId`;未登录演示态可以在内存里使用 Data URL 预览,但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*``data:video/*``data:audio/*``blob:`。旧数据读取时如果已有 `objectKey``imageSrc` 归一成 `/<objectKey>`;没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行;没有当前工程 ID 时才创建账号级 `editor_asset` 行,随后把对应 `resourceId``assetId` 写入参考图临时状态,生成请求仍使用临时状态中的图片源或 `objectKey`
- 资源表保存资源和素材级元数据;图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组:普通图层按 `layerId/resourceId` 保存,生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存,不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON;刷新时优先从 `editor_project_resource` 恢复,旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`;宣发素材生成器还必须保存并恢复 `publicationWorkflowId``publicationGameInfo``publicationReferences`,避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label,不保存图片 Data URL、signed URL 或 `objectKey`;刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照,只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引,只允许保存 `{ title, label, refType, refId }`;生成接口所需的图片 Data URL、signed URL 或 `objectKey` 只存在于提交前的临时参考图状态和请求体字段,不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存,刷新与新建图层均按 `Resolution``originalWidth/originalHeight`)原分辨率显示。图层组第一版是画布内布局语义,不单独建表。
- 图片类生成结果除作为 `editor_project_resource` 和画布图层保存外,还要写入账号级 `editor_asset` 素材库;该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`,前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照;`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底,不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后,右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额,不做本地乐观扣减。视频结果当前只保存为画布视频资源,不进入图片素材库
- 图片类生成视频结果除作为 `editor_project_resource` 和画布图层保存外,还要写入账号级 `editor_asset` 素材库;该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`,前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照;`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底,不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后,右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额,不做本地乐观扣减。生成视频会单独抽取首帧封面并写入 `thumbnailSrc`,素材栏和拖回画布时沿用该封面作为 poster
- 前端不直接订阅 SpacetimeDB,统一通过 api-server 的 `/api/editor/projects*` BFF 读写。
- 工程刷新恢复可先应用 session 级轻量项目快照缓存,让画布和素材 chrome 尽快显示;缓存快照必须排除 `data:*` / `blob:` 内联媒体,且在后端项目快照返回前不得触发自动保存。后端快照回来后覆盖本地缓存显示并恢复正常保存队列。
- 未登录用户可以使用本地演示态,但不触发工程自动保存;真实图片生成 / 修改需要登录。编辑器 API 请求允许使用 refresh cookie 静默补 access token,但 401 / 403 只在编辑器局部提示登录,不清空整站登录态,也不把后端 requestId 直接作为生图弹窗主文案。
@@ -76,7 +76,7 @@
- `POST /api/editor/icon-spritesheets/generations`:按图标规范图和素材描述数组生成 spritesheet,再由后端切分为独立透明图标。请求支持 `model``aspectRatio``imageSize``priceMudPoints``projectId``assetFolderId``generationInputs``priceMudPoints` 必须来自编辑器生成计费配置中对应生图模型的尺寸档位(如 `nanobanana2``0.5K / 1K / 2K``gpt-image-2``1K / 2K`),后端用 `editor_generation_config` 校验后才调用上游;`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize``0.5K``"512"``gpt-image-2``/v1/images/edits`。后端把 spritesheet 和拆分后的 icon 都保存为 project resource / 账号素材,并随响应返回对应快照。
- `POST /api/editor/ui-designs/assets/extractions`:以前端已绘入红色框选轮廓的 UI 设计图 Data URL 作为参考图,固定 `gpt-image-2` 和提示词 `仅提取被红色框框选的素材并整理成spritesheet,图集背景必须使用单一纯绿色 #00FF00 / RGB(0,255,0) 绿幕,方便后续扣除背景;素材自身不要出现绿色描边、绿色底板或绿色阴影。` 生成素材 spritesheet,再按连通域自动拆分为 `素材 1..N`,返回结构复用图标 spritesheet 响应。请求必须携带 `aspectRatio: "1:1"``imageSize: "1K" | "2K"``priceMudPoints`;框选数量不超过 6 个时前端按 `1:1·1K` 与 gpt-image-2 1K 价格提交,超过 6 个时按 `1:1·2K` 与 2K 价格提交。后端必须在调用上游前校验比例、尺寸和泥点价格,只允许 `1:1 / 1K / 2K`。请求可携带 `projectId``assetFolderId``generationInputs``spritesheetLabel`,后端保存 spritesheet / 拆分素材并返回对应 resource / asset 快照;前端必须把 spritesheet 原图与拆分素材都加入画布。
- `POST /api/editor/images/edits`:按提示词和当前图片 Data URL 调用 VectorEngine edits,返回新的生成图片元数据;请求携带 project / asset 上下文时由后端创建新 resource / asset,前端只消费响应快照。
- `POST /api/editor/videos/generations`:按视频描述、模型、比例、时长、分辨率、模式、声音、默认联网搜索标记和泥点价格生成视频。前端可选模型为 `seedance2.0-fast``seedance2.0``kling3.0``kling3.0-omni`,默认 `seedance2.0-fast`;后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`,将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`,两者不得混用。后端允许 6 类比例、4 到 15 秒整数、`480p / 720p / 1080p`,并拒绝 `seedance2.0-fast + 1080p``sound=on/off` 映射 Ark `generate_audio=true/false`。后端复用 Ark / VectorEngine content generation task 轮询链路,下载最终视频并持久化到 OSS返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`
- `POST /api/editor/videos/generations`:按视频描述、模型、比例、时长、分辨率、模式、声音、默认联网搜索标记和泥点价格生成视频。前端可选模型为 `seedance2.0-fast``seedance2.0``kling3.0``kling3.0-omni`,默认 `seedance2.0-fast`;后端必须将 `seedance2.0-fast` 映射到 `doubao-seedance-2-0-fast-260128`,将 `seedance2.0` 映射到 `doubao-seedance-2-0-260128`,两者不得混用。后端允许 6 类比例、4 到 15 秒整数、`480p / 720p / 1080p`,并拒绝 `seedance2.0-fast + 1080p``sound=on/off` 映射 Ark `generate_audio=true/false`。后端复用 Ark / VectorEngine content generation task 轮询链路,下载最终视频并持久化到 OSS;请求携带 `projectId` / `assetFolderId` 时同步创建 project resource / 账号素材并返回 `project` / `asset` 快照,基础响应返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`
所有写接口都必须校验 Bearer 登录态和 owner;接口只返回当前用户有权读取的工程与资源。
@@ -482,7 +482,7 @@ npm run check:server-rs-ddd
- Rust 结构体:`EditorAsset`
- 源码:`server-rs/crates/spacetime-module/src/editor_project_storage.rs`
- 说明:图片画布账号级素材表,保存用户上传 / 生成素材的名称、文件夹、图片读取地址、OSS 引用、尺寸、来源类型、prompt、provider、task、`asset_kind``generation_inputs_json` 和可选 `source_resource_id`。素材在同一账号的所有项目中可见;图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照,若同次生成也创建了 `editor_project_resource`,则把该 `resource_id` 写入 `source_resource_id`。素材库快照通过 `source_resource_id` 回查对应 project resource 的 `public_showcase_enabled`,供左侧素材菜单展示和切换公开状态;公开开关本身仍只落在 `editor_project_resource`,不在账号素材表复制真相。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId,画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。
- 说明:图片画布账号级素材表,保存用户上传 / 生成素材的名称、文件夹、图片读取地址、可选封面 `thumbnail_src`OSS 引用、尺寸、来源类型、prompt、provider、task、`asset_kind``generation_inputs_json` 和可选 `source_resource_id`。素材在同一账号的所有项目中可见;图片 / 图标 / UI 提取等生成 BFF 在请求携带 `asset_folder_id` 时负责创建账号级生成素材并返回 asset 快照,若同次生成也创建了 `editor_project_resource`,则把该 `resource_id` 写入 `source_resource_id`生成视频会抽取首帧封面写入 `thumbnail_src`,素材库和再次放入画布时用它作为 video poster。素材库快照通过 `source_resource_id` 回查对应 project resource 的 `public_showcase_enabled`,供左侧素材菜单展示和切换公开状态;公开开关本身仍只落在 `editor_project_resource`,不在账号素材表复制真相。素材放入画布时复制为 `editor_project_resource` 并由图层引用 resourceId,画布从 resource / asset 级元数据恢复素材类别和用户可见生成输入快照。
- 索引:`by_editor_asset_owner_user_id``by_editor_asset_folder_id`
### `inventory_slot`
@@ -61,8 +61,9 @@ use crate::{
editor_generation_source_entity_id, enqueue_editor_generation_job,
},
editor_project::{
EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item,
complete_editor_canvas_generation_with_items,
EditorCanvasGeneratedLayerInput, PersistEditorGeneratedAssetInput,
build_editor_canvas_generated_layer_item, complete_editor_canvas_generation_with_items,
persist_editor_generated_asset,
},
http_error::AppError,
platform_errors::map_oss_error,
@@ -104,6 +105,7 @@ const EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX: &str = "生成游戏角色动画
const EDITOR_VIDEO_ASSET_KIND: &str = "editor_video";
const EDITOR_VIDEO_ENTITY_KIND: &str = "editor_canvas";
const EDITOR_VIDEO_SLOT: &str = "video_preview";
const EDITOR_VIDEO_COVER_SLOT: &str = "video_cover";
const EDITOR_VIDEO_MODEL_SEEDANCE_2: &str = "seedance2.0";
const EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST: &str = "seedance2.0-fast";
const EDITOR_VIDEO_MODEL_KLING_3: &str = "kling3.0";
@@ -151,6 +153,7 @@ struct PersistedEditorVideo {
legacy_public_path: String,
object_key: String,
asset_object_id: String,
thumbnail_src: String,
}
pub async fn list_character_animation_templates(
@@ -828,6 +831,8 @@ pub(crate) async fn generate_editor_video_for_owner(
let generation_inputs = payload.generation_inputs.clone();
let source_resource_id = payload.source_resource_id.clone();
let asset_kind = payload.asset_kind.clone();
let asset_folder_id = payload.asset_folder_id.clone();
let asset_label = payload.asset_label.clone();
let pricing = state.editor_generation_pricing().map_err(|error| {
editor_video_error_response(
@@ -863,17 +868,53 @@ pub(crate) async fn generate_editor_video_for_owner(
)
.await
.map_err(|error| editor_video_error_response(&request_context, error))?;
let image_src = generated.preview_video_path.clone();
let effective_asset_kind = asset_kind.clone().unwrap_or_else(|| "video".to_string());
let generated_asset = persist_editor_generated_asset(
&state,
PersistEditorGeneratedAssetInput {
project_id: project_id.clone(),
owner_user_id: owner_user_id.clone(),
folder_id: asset_folder_id,
label: asset_label
.and_then(|value| {
let trimmed = value.trim().to_string();
(!trimmed.is_empty()).then_some(trimmed)
})
.unwrap_or_else(|| "生成视频".to_string()),
image_src: image_src.clone(),
object_key: generated.object_key.clone(),
asset_object_id: generated.asset_object_id.clone(),
width: normalized.width,
height: normalized.height,
prompt: normalized.prompt.clone(),
actual_prompt: Some(generated.submitted_prompt.clone()),
model: normalized.model.clone(),
provider: "VectorEngine".to_string(),
task_id: task_id.clone(),
source_resource_id: source_resource_id.clone(),
asset_kind: Some(effective_asset_kind.clone()),
generation_inputs: generation_inputs.clone(),
thumbnail_src: generated.thumbnail_src.clone(),
},
)
.await
.map_err(|error| editor_video_error_response(&request_context, error))?;
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
let layer_id = format!("layer-editor-video-{task_id}");
let item = build_editor_canvas_generated_layer_item(
completion,
EditorCanvasGeneratedLayerInput {
layer_id: layer_id.clone(),
resource_id: format!("local-resource-editor-video-{task_id}"),
resource_id: generated_asset
.resource
.as_ref()
.map(|resource| resource.resource_id.clone())
.unwrap_or_else(|| format!("local-resource-editor-video-{task_id}")),
title: completion.title.trim().to_string(),
src: generated.preview_video_path.clone(),
src: image_src.clone(),
media_type: Some("video".to_string()),
thumbnail_src: None,
thumbnail_src: generated.thumbnail_src.clone(),
original_width: normalized.width,
original_height: normalized.height,
display_width: completion.placeholder.width,
@@ -888,7 +929,7 @@ pub(crate) async fn generate_editor_video_for_owner(
asset_object_id: generated.asset_object_id.clone(),
duration_seconds: Some(normalized.duration_seconds),
source_resource_id,
asset_kind: Some(asset_kind.unwrap_or_else(|| "video".to_string())),
asset_kind: Some(effective_asset_kind.clone()),
generation_inputs,
image_sequence_frames: None,
preview_video_path: None,
@@ -909,12 +950,16 @@ pub(crate) async fn generate_editor_video_for_owner(
};
let completed_project =
completed_project.and_then(|project| serde_json::to_value(project).ok());
let generated_asset = generated_asset
.asset
.and_then(|asset| serde_json::to_value(asset).ok());
Ok(json_success_body(
Some(&request_context),
EditorVideoGenerateResponse {
ok: true,
video_src: generated.preview_video_path,
thumbnail_src: generated.thumbnail_src,
object_key: generated.object_key,
asset_object_id: generated.asset_object_id,
width: normalized.width,
@@ -929,6 +974,7 @@ pub(crate) async fn generate_editor_video_for_owner(
resolution: normalized.resolution,
price_mud_points: normalized.price_mud_points,
project: completed_project,
asset: generated_asset,
queue_state: None,
},
))
@@ -1124,13 +1170,12 @@ pub async fn resolve_role_asset_workflow(
})?;
let cache_scope_id = trim_optional_text(payload.cache_scope_id.as_deref());
let cache = match load_workflow_cache(&state, character_id.as_str(), cache_scope_id.as_deref())
.await
{
Ok(cache) => cache,
Err(error) if is_missing_oss_config_error(&error) => None,
Err(error) => return Err(character_animation_error_response(&request_context, error)),
};
let cache =
match load_workflow_cache(&state, character_id.as_str(), cache_scope_id.as_deref()).await {
Ok(cache) => cache,
Err(error) if is_missing_oss_config_error(&error) => None,
Err(error) => return Err(character_animation_error_response(&request_context, error)),
};
let workflow = build_role_asset_workflow(payload.role, cache.as_ref());
Ok(json_success_body(
@@ -1441,6 +1486,7 @@ async fn request_image_to_video_preview(
preview_video_path,
object_key: None,
asset_object_id: None,
thumbnail_src: None,
upstream_task_id,
submitted_prompt,
moderation_fallback_applied,
@@ -1735,6 +1781,7 @@ async fn request_editor_character_animation_preview(
preview_video_path,
object_key: None,
asset_object_id: None,
thumbnail_src: None,
upstream_task_id,
submitted_prompt: prompt.to_string(),
moderation_fallback_applied: false,
@@ -1763,6 +1810,7 @@ async fn request_editor_video_preview(
preview_video_path: persisted.legacy_public_path,
object_key: Some(persisted.object_key),
asset_object_id: Some(persisted.asset_object_id),
thumbnail_src: Some(persisted.thumbnail_src),
upstream_task_id,
submitted_prompt: request.prompt.clone(),
moderation_fallback_applied: false,
@@ -1820,6 +1868,9 @@ async fn put_generated_editor_video(
preview_payload: MediaPayload,
) -> Result<PersistedEditorVideo, AppError> {
let content_type = preview_payload.mime_type.clone();
let thumbnail_src =
put_generated_editor_video_cover(state, owner_user_id, task_id, request, &preview_payload)
.await?;
let put_result = put_character_animation_object(
state,
LegacyAssetPrefix::CharacterDrafts,
@@ -1853,9 +1904,65 @@ async fn put_generated_editor_video(
legacy_public_path: put_result.legacy_public_path,
object_key: put_result.object_key,
asset_object_id: confirmed.record.asset_object_id,
thumbnail_src,
})
}
async fn put_generated_editor_video_cover(
state: &AppState,
owner_user_id: &str,
task_id: &str,
request: &NormalizedEditorVideoRequest,
preview_payload: &MediaPayload,
) -> Result<String, AppError> {
let temp_dir = create_animation_temp_dir()?;
let input_path = temp_dir.join(format!("preview.{}", preview_payload.extension));
let output_path = temp_dir.join("cover.png");
fs::write(&input_path, &preview_payload.bytes).map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": "editor-video",
"message": format!("写入画板视频封面临时视频失败:{error}"),
}))
})?;
let extraction_result = (|| {
let settings = resolve_backend_frame_extraction_settings(state);
extract_video_frame_to_png(&input_path, &output_path, 0.0, &settings)?;
let cover_bytes = fs::read(&output_path).map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": "editor-video",
"message": format!("读取画板视频首帧封面失败:{error}"),
}))
})?;
Ok::<_, AppError>(cover_bytes)
})();
let _ = fs::remove_dir_all(&temp_dir);
let cover_bytes = extraction_result?;
let put_result = put_character_animation_object(
state,
LegacyAssetPrefix::CharacterDrafts,
vec![
"editor-videos".to_string(),
sanitize_storage_segment(request.model.as_str(), "model"),
task_id.to_string(),
],
"cover.png".to_string(),
"image/png".to_string(),
cover_bytes,
build_asset_metadata(
EDITOR_VIDEO_ASSET_KIND,
owner_user_id,
EDITOR_VIDEO_ENTITY_KIND,
task_id,
EDITOR_VIDEO_COVER_SLOT,
request.model.as_str(),
),
)
.await?;
Ok(put_result.legacy_public_path)
}
async fn create_editor_ark_image_to_video_task(
http_client: &reqwest::Client,
settings: &EditorCharacterAnimationSettings,
@@ -4925,6 +5032,7 @@ struct GeneratedAnimationPreview {
preview_video_path: String,
object_key: Option<String>,
asset_object_id: Option<String>,
thumbnail_src: Option<String>,
upstream_task_id: String,
submitted_prompt: String,
moderation_fallback_applied: bool,
@@ -5363,6 +5471,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("editor video request should normalize");
@@ -5393,6 +5503,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance fast request should normalize");
@@ -5420,6 +5532,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance standard request should normalize");
@@ -5447,6 +5561,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance standard 1080p request should normalize");
@@ -5475,6 +5591,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect_err("seedance fast should reject 1080p");
@@ -5502,6 +5620,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance multimodal request should normalize");
@@ -5538,6 +5658,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect_err("Seedance reference videos must not accept Base64 data URLs");
@@ -5564,6 +5686,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance multimodal request should normalize");
@@ -5610,6 +5734,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("seedance objectKey request should normalize");
@@ -5650,6 +5776,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect_err("audio-only references should fail");
@@ -339,9 +339,9 @@ pub struct EditorAssetResponse {
}
#[derive(Debug)]
struct EditorGeneratedAssetRecord {
resource: Option<EditorProjectResourcePayload>,
asset: Option<EditorAssetPayload>,
pub(crate) struct EditorGeneratedAssetRecord {
pub(crate) resource: Option<EditorProjectResourcePayload>,
pub(crate) asset: Option<EditorAssetPayload>,
}
struct EditorCanvasGenerationCompletionResult {
@@ -350,24 +350,25 @@ struct EditorCanvasGenerationCompletionResult {
}
#[derive(Debug)]
struct PersistEditorGeneratedAssetInput {
project_id: Option<String>,
owner_user_id: String,
folder_id: Option<String>,
label: String,
image_src: String,
object_key: Option<String>,
asset_object_id: Option<String>,
width: u32,
height: u32,
prompt: String,
actual_prompt: Option<String>,
model: String,
provider: String,
task_id: String,
source_resource_id: Option<String>,
asset_kind: Option<String>,
generation_inputs: Option<Value>,
pub(crate) struct PersistEditorGeneratedAssetInput {
pub(crate) project_id: Option<String>,
pub(crate) owner_user_id: String,
pub(crate) folder_id: Option<String>,
pub(crate) label: String,
pub(crate) image_src: String,
pub(crate) object_key: Option<String>,
pub(crate) asset_object_id: Option<String>,
pub(crate) width: u32,
pub(crate) height: u32,
pub(crate) prompt: String,
pub(crate) actual_prompt: Option<String>,
pub(crate) model: String,
pub(crate) provider: String,
pub(crate) task_id: String,
pub(crate) source_resource_id: Option<String>,
pub(crate) asset_kind: Option<String>,
pub(crate) generation_inputs: Option<Value>,
pub(crate) thumbnail_src: Option<String>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
@@ -479,7 +480,7 @@ pub struct EditorCanvasPayload {
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct EditorProjectResourcePayload {
resource_id: String,
pub(crate) resource_id: String,
project_id: String,
owner_user_id: String,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -545,6 +546,7 @@ pub struct EditorAssetPayload {
generation_inputs: Option<Value>,
source_resource_id: Option<String>,
public_showcase_enabled: Option<bool>,
thumbnail_src: Option<String>,
created_at: String,
updated_at: String,
}
@@ -966,6 +968,7 @@ pub async fn create_editor_asset(
generation_inputs_json,
source_resource_id: None,
now_micros: current_utc_micros(),
thumbnail_src: None,
})
.await
.map_err(map_editor_project_error)?;
@@ -1300,6 +1303,7 @@ pub(crate) async fn generate_editor_image_for_owner(
source_resource_id: payload.source_resource_id,
asset_kind: payload.asset_kind,
generation_inputs: payload.generation_inputs,
thumbnail_src: None,
},
)
.await?;
@@ -1658,6 +1662,7 @@ pub(crate) async fn edit_editor_image_for_owner(
source_resource_id: payload.source_resource_id,
asset_kind: payload.asset_kind,
generation_inputs: payload.generation_inputs,
thumbnail_src: None,
},
)
.await?;
@@ -1856,6 +1861,7 @@ pub(crate) async fn generate_editor_icon_spritesheet_for_owner(
source_resource_id: None,
asset_kind: Some("icon-spritesheet".to_string()),
generation_inputs: generation_inputs.clone(),
thumbnail_src: None,
},
)
.await?;
@@ -2088,6 +2094,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
source_resource_id: None,
asset_kind: Some("icon-spritesheet".to_string()),
generation_inputs: generation_inputs.clone(),
thumbnail_src: None,
},
)
.await?;
@@ -2137,6 +2144,7 @@ pub(crate) async fn extract_editor_ui_design_assets_for_owner(
source_resource_id: None,
asset_kind: Some("icon".to_string()),
generation_inputs: generation_inputs.clone(),
thumbnail_src: None,
},
)
.await?;
@@ -2570,6 +2578,7 @@ pub(crate) fn editor_asset_payload_from_record(record: EditorAssetRecord) -> Edi
.map(sanitize_editor_payload_inline_media),
source_resource_id: record.source_resource_id,
public_showcase_enabled: record.public_showcase_enabled,
thumbnail_src: normalize_optional_string(record.thumbnail_src),
created_at: record.created_at,
updated_at: record.updated_at,
}
@@ -2692,7 +2701,7 @@ fn fill_missing_media_identity_field(
}
}
async fn persist_editor_generated_asset(
pub(crate) async fn persist_editor_generated_asset(
state: &AppState,
input: PersistEditorGeneratedAssetInput,
) -> Result<EditorGeneratedAssetRecord, AppError> {
@@ -2760,6 +2769,7 @@ async fn persist_editor_generated_asset(
generation_inputs_json,
source_resource_id: created_resource_id,
now_micros,
thumbnail_src: normalize_optional_string(input.thumbnail_src),
})
.await
.map_err(map_editor_project_error)?;
@@ -492,6 +492,7 @@ pub async fn create_external_editor_asset(
generation_inputs_json,
source_resource_id: None,
now_micros: current_utc_micros(),
thumbnail_src: None,
})
.await
.map_err(map_editor_project_error)?;
@@ -387,6 +387,10 @@ pub struct EditorVideoGenerateRequest {
pub source_resource_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub asset_kind: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub asset_folder_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub asset_label: Option<String>,
}
fn default_editor_video_web_search_enabled() -> bool {
@@ -399,6 +403,8 @@ pub struct EditorVideoGenerateResponse {
pub ok: bool,
pub video_src: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub thumbnail_src: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub object_key: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub asset_object_id: Option<String>,
@@ -417,6 +423,8 @@ pub struct EditorVideoGenerateResponse {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub asset: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
}
@@ -1141,6 +1149,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("request should serialize");
@@ -1172,6 +1182,8 @@ mod tests {
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
asset_folder_id: None,
asset_label: None,
})
.expect("request should serialize");
@@ -1194,6 +1206,7 @@ mod tests {
let payload = serde_json::to_value(EditorVideoGenerateResponse {
ok: true,
video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(),
thumbnail_src: Some("/generated-editor-videos/task-1/cover.png".to_string()),
object_key: Some(
"generated-character-drafts/editor-videos/task-1/preview.mp4".to_string(),
),
@@ -1210,6 +1223,7 @@ mod tests {
resolution: "480p".to_string(),
price_mud_points: 100,
project: None,
asset: None,
queue_state: None,
})
.expect("response should serialize");
@@ -1218,6 +1232,10 @@ mod tests {
payload["videoSrc"],
json!("/generated-editor-videos/task-1/preview.mp4")
);
assert_eq!(
payload["thumbnailSrc"],
json!("/generated-editor-videos/task-1/cover.png")
);
assert_eq!(
payload["objectKey"],
json!("generated-character-drafts/editor-videos/task-1/preview.mp4")
@@ -94,6 +94,7 @@ pub struct EditorAssetRecord {
pub generation_inputs: Option<serde_json::Value>,
pub source_resource_id: Option<String>,
pub public_showcase_enabled: Option<bool>,
pub thumbnail_src: Option<String>,
pub created_at: String,
pub updated_at: String,
}
@@ -232,6 +233,7 @@ pub struct EditorAssetCreateRecordInput {
pub generation_inputs_json: Option<String>,
pub source_resource_id: Option<String>,
pub now_micros: i64,
pub thumbnail_src: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq)]
@@ -442,6 +444,7 @@ impl From<EditorAssetCreateRecordInput> for crate::module_bindings::EditorAssetC
generation_inputs_json: input.generation_inputs_json,
source_resource_id: input.source_resource_id,
now_micros: input.now_micros,
thumbnail_src: input.thumbnail_src,
}
}
}
@@ -756,6 +759,7 @@ fn map_editor_asset_snapshot(
generation_inputs,
source_resource_id: snapshot.source_resource_id,
public_showcase_enabled: snapshot.public_showcase_enabled,
thumbnail_src: snapshot.thumbnail_src,
created_at: format_timestamp_micros(snapshot.created_at_micros),
updated_at: format_timestamp_micros(snapshot.updated_at_micros),
})
@@ -1,7 +1,7 @@
// THIS FILE IS AUTOMATICALLY GENERATED BY SPACETIMEDB. EDITS TO THIS FILE
// WILL NOT BE SAVED. MODIFY TABLES IN YOUR MODULE SOURCE CODE INSTEAD.
// This was generated using spacetimedb cli version 2.5.0 (commit 19cbf2b219ac9fd7fdca6c5e2c61a3a2f0319515).
// This was generated using spacetimedb cli version 2.5.0 (commit ca16958ef0a5f8c816700d2255a0b20ecacff901).
#![allow(unused, clippy::all)]
use spacetimedb_sdk::__codegen::{self as __sdk, __lib, __sats, __ws};
@@ -26,6 +26,7 @@ pub struct EditorAssetCreateInput {
pub generation_inputs_json: Option<String>,
pub source_resource_id: Option<String>,
pub now_micros: i64,
pub thumbnail_src: Option<String>,
}
impl __sdk::InModule for EditorAssetCreateInput {
@@ -27,6 +27,7 @@ pub struct EditorAssetSnapshot {
pub public_showcase_enabled: Option<bool>,
pub created_at_micros: i64,
pub updated_at_micros: i64,
pub thumbnail_src: Option<String>,
}
impl __sdk::InModule for EditorAssetSnapshot {
@@ -27,6 +27,7 @@ pub struct EditorAsset {
pub asset_kind: Option<String>,
pub generation_inputs_json: Option<String>,
pub source_resource_id: Option<String>,
pub thumbnail_src: Option<String>,
}
impl __sdk::InModule for EditorAsset {
@@ -57,6 +58,7 @@ pub struct EditorAssetCols {
pub asset_kind: __sdk::__query_builder::Col<EditorAsset, Option<String>>,
pub generation_inputs_json: __sdk::__query_builder::Col<EditorAsset, Option<String>>,
pub source_resource_id: __sdk::__query_builder::Col<EditorAsset, Option<String>>,
pub thumbnail_src: __sdk::__query_builder::Col<EditorAsset, Option<String>>,
}
impl __sdk::__query_builder::HasCols for EditorAsset {
@@ -86,6 +88,7 @@ impl __sdk::__query_builder::HasCols for EditorAsset {
"generation_inputs_json",
),
source_resource_id: __sdk::__query_builder::Col::new(table_name, "source_resource_id"),
thumbnail_src: __sdk::__query_builder::Col::new(table_name, "thumbnail_src"),
}
}
}
@@ -124,6 +124,8 @@ pub struct EditorAsset {
generation_inputs_json: Option<String>,
#[default(None::<String>)]
source_resource_id: Option<String>,
#[default(None::<String>)]
thumbnail_src: Option<String>,
}
#[derive(Clone, Debug, PartialEq, SpacetimeType)]
@@ -282,6 +284,7 @@ pub struct EditorAssetSnapshot {
pub public_showcase_enabled: Option<bool>,
pub created_at_micros: i64,
pub updated_at_micros: i64,
pub thumbnail_src: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)]
@@ -342,6 +345,7 @@ pub struct EditorAssetCreateInput {
pub generation_inputs_json: Option<String>,
pub source_resource_id: Option<String>,
pub now_micros: i64,
pub thumbnail_src: Option<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)]
@@ -1267,6 +1271,7 @@ fn create_editor_asset(
asset_kind: normalize_optional(input.asset_kind),
generation_inputs_json: normalize_optional(input.generation_inputs_json),
source_resource_id: normalize_optional(input.source_resource_id),
thumbnail_src: normalize_optional(input.thumbnail_src),
});
ctx.db
.editor_asset()
@@ -1315,6 +1320,7 @@ fn update_editor_asset(
asset_kind: asset.asset_kind,
generation_inputs_json: asset.generation_inputs_json,
source_resource_id: asset.source_resource_id,
thumbnail_src: asset.thumbnail_src,
});
ctx.db
.editor_asset()
@@ -1359,6 +1365,7 @@ fn repair_editor_asset_media(
asset_kind: asset.asset_kind,
generation_inputs_json: asset.generation_inputs_json,
source_resource_id: asset.source_resource_id,
thumbnail_src: asset.thumbnail_src,
});
ctx.db
@@ -1726,6 +1733,7 @@ fn asset_snapshot_from_row(ctx: &ReducerContext, row: EditorAsset) -> EditorAsse
public_showcase_enabled,
created_at_micros: row.created_at.to_micros_since_unix_epoch(),
updated_at_micros: row.updated_at.to_micros_since_unix_epoch(),
thumbnail_src: row.thumbnail_src,
}
}
@@ -1847,12 +1855,7 @@ fn project_resource_input_copies_source_media(
.find(&source_resource_id.to_string())
.filter(|source_resource| source_resource.owner_user_id == owner_user_id)
.is_some_and(|source_resource| {
project_resource_matches_media(
&source_resource,
asset_object_id,
object_key,
image_src,
)
project_resource_matches_media(&source_resource, asset_object_id, object_key, image_src)
})
}
@@ -1883,12 +1886,7 @@ fn find_reusable_project_resource_for_input(
.as_deref()
.map(str::trim)
.is_some_and(|value| value == source_resource_id))
&& project_resource_matches_media(
resource,
asset_object_id,
object_key,
image_src,
)
&& project_resource_matches_media(resource, asset_object_id, object_key, image_src)
})
.collect::<Vec<_>>();
matches.sort_by(|left, right| {
@@ -1934,12 +1932,7 @@ fn matching_media_ref(left: &str, right: &str) -> bool {
}
fn normalize_media_ref(value: &str) -> String {
let trimmed_value = value
.trim()
.split('?')
.next()
.unwrap_or("")
.trim();
let trimmed_value = value.trim().split('?').next().unwrap_or("").trim();
let path_value = if let Some(protocol_index) = trimmed_value.find("://") {
let after_protocol = &trimmed_value[protocol_index + 3..];
after_protocol
@@ -1408,6 +1408,9 @@ fn normalize_migration_row(table_name: &str, value: &serde_json::Value) -> serde
object
.entry("source_resource_id".to_string())
.or_insert(serde_json::Value::Null);
object
.entry("thumbnail_src".to_string())
.or_insert(serde_json::Value::Null);
}
}
next_value
@@ -102,6 +102,8 @@ describe('ImageCanvasEditorModel', () => {
folderId: 'project',
label: '开场动画.mp4',
imageSrc: '/generated-character-drafts/editor/asset-library/video/开场动画.mp4',
thumbnailSrc:
'/generated-character-drafts/editor/asset-library/video/开场动画-cover.png',
width: 560,
height: 315,
sourceType: 'uploaded',
@@ -112,7 +114,11 @@ describe('ImageCanvasEditorModel', () => {
});
expect(library.assets[0]).toMatchObject({ mediaType: 'audio' });
expect(library.assets[1]).toMatchObject({ mediaType: 'video' });
expect(library.assets[1]).toMatchObject({
mediaType: 'video',
thumbnailSrc:
'/generated-character-drafts/editor/asset-library/video/开场动画-cover.png',
});
expect(
createLayerFromAsset(
library.assets[0]!,
@@ -139,6 +145,7 @@ describe('ImageCanvasEditorModel', () => {
persisted: true,
objectKey: 'oss/asset-1.png',
assetObjectId: 'object-1',
thumbnailSrc: '/generated-character-drafts/editor/asset-1-cover.png',
sourceResourceId: 'resource-source-1',
};
@@ -158,6 +165,7 @@ describe('ImageCanvasEditorModel', () => {
originalHeight: 480,
objectKey: 'oss/asset-1.png',
assetObjectId: 'object-1',
thumbnailSrc: '/generated-character-drafts/editor/asset-1-cover.png',
sourceAssetId: 'asset-1',
sourceResourceId: 'resource-source-1',
});
@@ -172,6 +172,7 @@ export function createLayerFromAsset(
originalHeight: asset.height,
zIndex: index + 10,
sourceType: asset.sourceType,
thumbnailSrc: asset.thumbnailSrc,
prompt: asset.prompt,
actualPrompt: asset.actualPrompt,
model: asset.model,
@@ -710,6 +711,7 @@ export function mapAssetLibrarySnapshot(
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
thumbnailSrc: asset.thumbnailSrc ?? undefined,
sourceResourceId: asset.sourceResourceId ?? null,
publicShowcaseEnabled: asset.publicShowcaseEnabled ?? null,
assetKind: canvasAssetKindOrNull(asset.assetKind),
@@ -647,6 +647,7 @@ export function ImageCanvasEditorView() {
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
thumbnailSrc: asset.thumbnailSrc ?? undefined,
durationSeconds: asset.durationSeconds ?? undefined,
assetKind: canvasAssetKindOrNull(asset.assetKind),
generationInputs: generationInputsOrNull(asset.generationInputs),
@@ -519,6 +519,21 @@ describe('ImageCanvasGenerationLayerModel', () => {
provider: 'VectorEngine',
taskId: 'video-task-1',
priceMudPoints: 40,
durationSeconds: 5,
thumbnailSrc: '/generated-character-drafts/editor-videos/cover.png',
asset: {
assetId: 'asset-video-bff',
folderId: 'user-1:asset-folder:project',
label: '生成视频 15',
imageSrc: '/generated-character-drafts/editor-videos/video.mp4',
thumbnailSrc: '/generated-character-drafts/editor-videos/cover.png',
objectKey: 'generated-character-drafts/editor-videos/video.mp4',
assetObjectId: 'assetobj-video-1',
width: 1280,
height: 720,
sourceType: 'generated',
assetKind: 'video',
},
},
generatedIndex: 15,
title: '生成视频 15',
@@ -534,7 +549,10 @@ describe('ImageCanvasGenerationLayerModel', () => {
assetKind: 'video',
objectKey: 'generated-character-drafts/editor-videos/video.mp4',
assetObjectId: 'assetobj-video-1',
thumbnailSrc: '/generated-character-drafts/editor-videos/cover.png',
durationSeconds: 5,
});
expect(layer.generatedAssetSnapshot?.assetId).toBe('asset-video-bff');
});
it('creates character animation result layers as playable image sequences', () => {
@@ -438,9 +438,12 @@ export function createVideoResultLayer({
taskId: generated.taskId,
objectKey: generated.objectKey,
assetObjectId: generated.assetObjectId,
thumbnailSrc: generated.thumbnailSrc ?? generated.asset?.thumbnailSrc ?? undefined,
sourceResourceId: sourceLayer?.resourceId,
groupId: sourceLayer?.groupId,
generationInputs,
durationSeconds: generated.durationSeconds,
generatedAssetSnapshot: generated.asset ?? undefined,
};
}
@@ -161,6 +161,7 @@ function SubmissionWorkflowHarness({
initialCharacterAnimationPanel = null,
initialLayers = [createLayer()],
projectId = undefined,
upsertGeneratedAsset = undefined,
applyProjectSnapshot = undefined,
onWalletBalanceMayHaveChanged = undefined,
}: {
@@ -170,6 +171,9 @@ function SubmissionWorkflowHarness({
initialCharacterAnimationPanel?: CharacterAnimationPanelState | null;
initialLayers?: CanvasLayer[];
projectId?: string | null;
upsertGeneratedAsset?: Parameters<
typeof useImageCanvasGenerationSubmissionWorkflow
>[0]['upsertGeneratedAsset'];
applyProjectSnapshot?: Parameters<
typeof useImageCanvasGenerationSubmissionWorkflow
>[0]['applyProjectSnapshot'];
@@ -245,6 +249,7 @@ function SubmissionWorkflowHarness({
},
projectId,
assetFolderId: projectId ? 'project' : undefined,
upsertGeneratedAsset,
applyProjectSnapshot,
onWalletBalanceMayHaveChanged,
});
@@ -758,11 +763,39 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
});
it('submits video quick edits through the video API with the source video', async () => {
const applyProjectSnapshot = vi.fn();
const upsertGeneratedAsset = vi.fn();
generateEditorVideoMock.mockResolvedValueOnce(
createVideoGenerated({ prompt: '把当前图改成雨天' }),
createVideoGenerated({
prompt: '把当前图改成雨天',
project: {
projectId: 'editor-project-1',
title: '项目',
coverImageSrc: null,
viewport: { x: 0, y: 0, scale: 1 },
layers: [],
resources: [],
createdAt: '2026-06-27T00:00:00.000Z',
updatedAt: '2026-06-27T00:00:00.000Z',
},
asset: {
assetId: 'asset-video-quick-edit',
folderId: 'user-1:asset-folder:project',
label: '源视频 快速编辑',
imageSrc: 'https://assets.example.test/generated-video.mp4',
thumbnailSrc: 'https://assets.example.test/generated-video-cover.png',
width: 854,
height: 480,
sourceType: 'generated',
assetKind: 'video',
},
}),
);
render(
<SubmissionWorkflowHarness
projectId="editor-project-1"
applyProjectSnapshot={applyProjectSnapshot}
upsertGeneratedAsset={upsertGeneratedAsset}
initialLayers={[
createLayer({
title: '源视频',
@@ -811,14 +844,22 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
referenceVideoSrcs: [
'generated-character-drafts/editor/videos/source.mp4',
],
assetFolderId: 'project',
assetLabel: '源视频 快速编辑',
}),
);
});
expect(generateEditorImageMock).not.toHaveBeenCalled();
expect(generateEditorCharacterAnimationMock).not.toHaveBeenCalled();
await waitFor(() => {
expect(screen.getByTestId('layers').textContent).toContain(
'layer-video-1:源视频 快速编辑:resource-source:video:video:https://assets.example.test/generated-video.mp4',
expect(applyProjectSnapshot).toHaveBeenCalledWith(
expect.objectContaining({ projectId: 'editor-project-1' }),
);
expect(upsertGeneratedAsset).toHaveBeenCalledWith(
expect.objectContaining({
assetId: 'asset-video-quick-edit',
thumbnailSrc: 'https://assets.example.test/generated-video-cover.png',
}),
);
});
});
@@ -1136,6 +1136,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({
generationInputs,
sourceResourceId: quickEditSourceLayer.resourceId,
assetKind: quickEditSourceLayer.assetKind,
assetFolderId,
assetLabel: `${quickEditSourceLayer.title} 快速编辑`,
...(projectId && quickEditCanvasCompletionPlaceholder
? {
canvasCompletion: {
@@ -1160,6 +1162,9 @@ export function useImageCanvasGenerationSubmissionWorkflow({
}
if (generated.project && applyProjectSnapshot) {
applyProjectSnapshot(generated.project);
if (generated.asset) {
upsertGeneratedAsset?.(generated.asset);
}
return;
}
addVideoResultLayer(
@@ -1324,6 +1329,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
openCanvasGenerationDialog,
projectId,
assetFolderId,
upsertGeneratedAsset,
quickEditPanel,
quickEditSourceLayer,
quickEditSelectionState,
@@ -1476,6 +1482,8 @@ export function useImageCanvasGenerationSubmissionWorkflow({
...submissionPlan.input,
projectId,
generationInputs: submissionPlan.result.generationInputs,
assetFolderId,
assetLabel: submissionPlan.result.title,
...(projectId && canvasCompletionPlaceholder
? {
canvasCompletion: {
@@ -1500,6 +1508,9 @@ export function useImageCanvasGenerationSubmissionWorkflow({
}
if (generated.project && applyProjectSnapshot) {
applyProjectSnapshot(generated.project);
if (generated.asset) {
upsertGeneratedAsset?.(generated.asset);
}
return;
}
addVideoResultLayer(
@@ -1097,6 +1097,18 @@ describe('editorProjectClient', () => {
taskId: 'video-task-1',
durationSeconds: 5,
resolution: '480p',
thumbnailSrc: '/generated-editor-videos/video-task-1/cover.png',
asset: {
assetId: 'asset-video-1',
folderId: 'user-1:asset-folder:project',
label: '生成视频 1',
imageSrc: '/generated-editor-videos/video-task-1/preview.mp4',
thumbnailSrc: '/generated-editor-videos/video-task-1/cover.png',
width: 1280,
height: 720,
sourceType: 'generated',
assetKind: 'video',
},
});
const result = await generateEditorVideo({
@@ -1107,9 +1119,18 @@ describe('editorProjectClient', () => {
resolution: '480p',
mode: 'std',
sound: 'off',
assetFolderId: 'project',
assetLabel: '生成视频 1',
});
expect(result.taskId).toBe('video-task-1');
expect(result.thumbnailSrc).toBe(
'/generated-editor-videos/video-task-1/cover.png',
);
expect(result.asset?.assetId).toBe('asset-video-1');
expect(result.asset?.thumbnailSrc).toBe(
'/generated-editor-videos/video-task-1/cover.png',
);
expect(requestJsonMock).toHaveBeenCalledWith(
'/api/editor/videos/generations',
expect.objectContaining({
@@ -1124,6 +1145,8 @@ describe('editorProjectClient', () => {
mode: 'std',
sound: 'off',
webSearchEnabled: true,
assetFolderId: 'project',
assetLabel: '生成视频 1',
}),
}),
'生成视频失败',
@@ -97,6 +97,7 @@ export type EditorAssetSnapshot = {
folderId: string;
label: string;
imageSrc: string;
thumbnailSrc?: string | null;
objectKey?: string | null;
assetObjectId?: string | null;
width: number;
@@ -329,10 +330,13 @@ export type EditorVideoGenerationInput = {
generationInputs?: EditorAssetGenerationInputs | null;
sourceResourceId?: string | null;
assetKind?: string | null;
assetFolderId?: string | null;
assetLabel?: string | null;
};
export type EditorVideoGenerationResult = {
videoSrc: string;
thumbnailSrc?: string | null;
objectKey?: string | null;
assetObjectId?: string | null;
width: number;
@@ -343,8 +347,10 @@ export type EditorVideoGenerationResult = {
model: string;
provider: string;
taskId: string;
durationSeconds: number;
priceMudPoints?: number;
project?: EditorProjectSnapshot | null;
asset?: EditorAssetSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
@@ -912,6 +918,8 @@ export async function generateEditorVideo(input: EditorVideoGenerationInput) {
? { sourceResourceId: input.sourceResourceId }
: {}),
...(input.assetKind ? { assetKind: input.assetKind } : {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
...(input.assetLabel ? { assetLabel: input.assetLabel } : {}),
}),
'生成视频失败',
{