接入画板生成视频功能
新增画板底部生成视频入口、Lovart 风格面板、视频图层渲染与元数据展示。 接入 /api/editor/videos/generations 契约与后端 Ark/VectorEngine 视频任务链路。 统一编辑器生成类泥点配置，并补充 UI 设计图、参考图与生成面板结构测试。 更新编辑器技术方案、生成类面板方案和 Hermes 共享决策/踩坑记录。
This commit is contained in:
@@ -2263,13 +2263,13 @@
|
||||
- 验证方式：新增或增删素材描述项时，面板宽度应随项数变化；图标素材规范入口应呈现参考卡视觉而非纯文本按钮；移动端下仍应固定在底部锚定，不出现内部滚动条。
|
||||
- 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-16 图片画布图标素材采用 nanobanana2 spritesheet + 后端连通域拆分
|
||||
## 2026-06-16 图片画布图标素材与角色生成支持双图片模型
|
||||
|
||||
- 背景：图片画布需要一次生成多枚 UI 图标素材，并保证生成后能按用户输入顺序命名、拆成独立透明素材铺回画布。
|
||||
- 决策：底部 `生成图标素材` 入口创建一叠空白图标占位和独立面板；图标规范参考图只允许绑定 `assetKind="icon-spec"`。前端提交 `/api/editor/icon-spritesheets/generations`，后端固定使用 VectorEngine `gemini-3.1-flash-image-preview`，`<=25` 个描述用 `512x512`，`>25` 个描述用 `1024x1024`，先生成绿幕 1:1 spritesheet，再由 `platform-image` 绿幕去背并按 8 邻域连通域从上到下、从左到右拆分。成品图标图层写入 `assetKind="icon"`。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/services/image-editor/editorProjectClient.ts`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/platform-image/src/generated_asset_sheets/sheet.rs`、图片画布技术方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`cargo test --manifest-path server-rs/Cargo.toml -p platform-image generated_asset_sheets::sheet::tests -- --nocapture`、`cargo test --manifest-path server-rs/Cargo.toml -p api-server editor_project -- --nocapture`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
- 背景：图片画布需要一次生成多枚 UI 图标素材，并保证生成后能按用户输入顺序命名、拆成独立透明素材铺回画布；角色形象生成也需要和图标素材共用同一套图片模型选择、比例和大小口径。
|
||||
- 决策：底部 `生成图标素材` 入口创建一叠空白图标占位和独立面板；图标规范参考图只允许绑定 `assetKind="icon-spec"`。`生成角色形象` 与 `生成图标素材` 均支持 VectorEngine `gemini-3.1-flash-image-preview`（UI 显示 `nanobanana2`）和 `gpt-image-2`，默认 `nanobanana2`，用户在两类面板中切换过模型后下一次打开继续沿用上次模型。前端提交 `model`、`aspectRatio`、`imageSize`；后端不再按图标数量分 `512x512/1024x1024`，而是按模型归一尺寸：`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，把参考图写成 `inline_data`，并在 `generationConfig.imageConfig` 写入比例和大小，`0.5K` 传 `"512"`；`gpt-image-2` 无参考图走 generations，有参考图走 edits，按文档支持的 `size` 字符串映射。图标素材仍先生成绿幕 spritesheet，再由 `platform-image` 绿幕去背并按 8 邻域连通域从上到下、从左到右拆分。成品图标图层写入 `assetKind="icon"`，角色图层写入 `assetKind="character"`。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/useImageCanvasGenerationWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/platform-image/src/vector_engine/*`、`server-rs/crates/platform-image/src/generated_asset_sheets/sheet.rs`、图片画布技术方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/ImageCanvasGenerationPlacementModel.test.ts src/services/image-editor/editorProjectClient.test.ts`、`cargo test -p api-server editor_generation_dimensions_follow_model_options --manifest-path server-rs/Cargo.toml`、`cargo test -p platform-image nanobanana_generate_content --manifest-path server-rs/Cargo.toml`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-16 图片画布生成面板与浮层层级收口
|
||||
|
||||
@@ -2294,3 +2294,11 @@
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、图片画布 layout hydrate、新建 / 上传 / 生成 / 快速编辑 / 图标素材生成结果铺回画布逻辑，以及图片画布技术方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "hydrates canvas images from Resolution instead of saved Size|opens generated image info from the corner button and creates a real right-side edit result|shows image resolution on hover"`。
|
||||
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-17 图片画布底部生成视频接入 Lovart 面板
|
||||
|
||||
- 背景：编辑器画板底部工具栏需要新增 `生成视频`，并和现有 Lovart 式生成类面板、泥点展示、占位图和画布结果图层保持一致。
|
||||
- 决策：`生成视频` 点击后创建独立视频生成占位和极简面板，提交 `POST /api/editor/videos/generations`；首期支持 `seedance2.0`、`seedance2.0-fast`、`kling3.0`、`kling3.0-omni`、`veo3.1`、`veo3.1-fast`，固定文字转视频、`16:9`、标准模式和静音。后端复用 Ark / VectorEngine content generation task 轮询链路，下载视频后持久化到 OSS。生成结果在画布中写入 `mediaType="video"` 与 `assetKind="video"`，图片信息弹窗按视频显示为 `视频信息` / `视频类型`。生成类泥点价格统一走 `editor_generation_config`，视频和角色动画均为 480p 每秒 10 泥点、720p 每秒 20 泥点。
|
||||
- 影响范围：图片画布生成工作流、前端 editorProjectClient、`shared-contracts`、`api-server` 视频生成 BFF、编辑器技术方案和生成类面板方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "opens the bottom generate video panel"`、`npm run test -- src/components/image-editor/ImageCanvasMetadataModalView.test.tsx`、`npm run test -- src/services/image-editor/editorProjectClient.test.ts`、`cargo test -p shared-contracts editor_video --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_video --manifest-path server-rs/Cargo.toml`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
@@ -55,6 +55,14 @@
|
||||
- 验证：测试断言菜单不包含在底部工具栏 / 参考图行里，并且生成面板打开时底部 `AI画布工具栏` 仍存在。
|
||||
- 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`。
|
||||
|
||||
## 图片编辑器规范图片面板不要脱离统一生成 shell
|
||||
|
||||
- 现象：生成 UI 设计图或新建图标素材规范时，面板参考图、输入区和底部生成按钮相对生成图片 / 生成角色 / 生成视频错位；图标素材规范甚至可能缺少首行参考图入口。
|
||||
- 原因：规范、UI 设计图等面板虽然都属于生成类入口，但 JSX 和 CSS 曾各自维护 `spec-footer`、局部 field wrapper 或缺省参考区，导致后续改造只覆盖普通图片 / 角色 / 视频，规范图片类面板结构漂移。
|
||||
- 处理：生成规范下的角色形象规范、UI 素材规范、图标素材规范、自定义规范，以及生成 UI 设计图，都必须复用 `image-canvas-editor__generation-composer image-canvas-editor__generation-composer--image` 外层 shell；首行统一 `image-canvas-editor__generation-ref`，底部统一 `image-canvas-editor__generation-composer-footer` + `image-canvas-editor__generation-submit`。多字段内容只在中央字段区保持紧凑，不单独发明 footer 或省略参考区。
|
||||
- 验证：`npm test -- src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx src/components/image-editor/ImageCanvasEditorView.test.tsx -t "生成UI设计图|生成规范|visible titles|图标素材规范|character spec"`。
|
||||
- 关联：`src/components/image-editor/ImageCanvasGenerationComposerView.tsx`、`src/index.css`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`。
|
||||
|
||||
## 图片编辑器生成占位图在生成中也要使用最新拖拽位置
|
||||
|
||||
- 现象：用户在图片编辑器里提交生成后继续拖动画布占位图，预览框可以移动，但生成完成后的真实图片仍落回提交瞬间的旧位置。
|
||||
|
||||
@@ -12,13 +12,13 @@
|
||||
- 编辑器左侧为图片素材栏，可展开 / 收起；移动端优先保持素材栏可折叠。
|
||||
- 中央画布支持背景拖拽平移、滚轮缩放、缩放百分比菜单、显示所有元素和固定比例缩放。
|
||||
- 画布左下角提供 Lovart 式状态控件：背景色圆点、素材 / 图层入口、小地图开关；小地图显示图层缩略分布和当前视口框，点击小地图执行显示所有元素。
|
||||
- 画布中的图片可展示、悬浮显示图片 Resolution 尺寸与边框，点击后在图片上方显示浮动工具栏；图片不再维护独立展示 `Size` 字段，画布显示宽高统一取 `originalWidth/originalHeight`（图片信息中的 `Resolution`）。
|
||||
- 画布中的图片可展示、悬浮显示图片 Resolution 尺寸与边框，点击后在图片上方显示浮动工具栏；图片右上角素材类型标签、查看信息按钮和悬浮尺寸标签在画布缩小时必须按 viewport 反向缩放，保持屏幕可读尺寸；查看信息按钮使用圆形 `i` 图标，不使用中括号或花括号样式。图片不再维护独立展示 `Size` 字段，画布显示宽高统一取 `originalWidth/originalHeight`（图片信息中的 `Resolution`）。
|
||||
- 默认工具为选择模式；底部工具栏采用 AI 画布工作流工具组：选择、抓手、上传、生成、局部修改 / 蒙版、文字、形状 / 标注、导出。
|
||||
- 鼠标中键拖拽始终平移画布；长按 Space 临时进入抓手模式，松开后恢复原工具。
|
||||
- 图片拖拽时显示水平 / 垂直吸附参考线，吸附到其它图层或画板的边缘与中心线。
|
||||
- 生成资源右上角显示元数据按钮，点击打开独立元数据窗口。图片信息页不展示后端组装后的生图 Prompt，也不提供复制 Prompt；只展示该图片生成时用户在面板里提交的输入快照，包括普通生成提示词、规范表单字段、角色设定、图标素材描述、修改要求，以及角色形象规范 / 常规参考图 / 图标素材规范 / 修改参考图等参考图卡片。旧数据或上传图片没有输入快照时显示 `-`，禁止回退展示内部 Prompt。
|
||||
- 对生成资源执行修改时，在右侧创建新的生成结果图层，并自动调整视图显示原图和新图。
|
||||
- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine `gpt-image-2`：纯文本生成走 `/api/editor/images/generations`，基于当前生成图的修改走 `/api/editor/images/edits`。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成工具后先在画布中心创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图落在最新占位框位置，输入框继续跟随新生成图；点击所有图片生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。快速编辑和修改图片在调用后端前必须把当前图层图片源读取为图片 Data URL，来源可以是本地上传 Data URL、站内 public 图片、历史 `/generated-*` 路径或可读取的 OSS generated URL；后端仍只接收图片 Data URL，不把普通 URL 直接透传到 VectorEngine edits。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。
|
||||
- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine。普通生成、生成规范和快速编辑保留既有 `gpt-image-2` 路径；`生成角色形象` 与 `生成图标素材` 支持 `nanobanana2`（`gemini-3.1-flash-image-preview`）和 `gpt-image-2`，默认 `nanobanana2`，并在两类面板之间沿用用户上次选择的模型。`nanobanana2` 走 `/v1beta/models/{model}:generateContent`，请求体写入 `generationConfig.imageConfig.aspectRatio/imageSize`；`gpt-image-2` 走 `/v1/images/generations` 或 `/v1/images/edits`，请求体按 VectorEngine 文档映射 `size`。纯文本生成走 `/api/editor/images/generations`，基于当前生成图的修改走 `/api/editor/images/edits`。`生成视频` 走 `/api/editor/videos/generations`，支持 Seedance 2.0 / Seedance 2.0 Fast / Kling 3.0 / Kling 3.0 Omni / Veo 3.1 / Veo 3.1 Fast，首期固定文字转视频、`16:9`、标准模式、静音，生成结果以视频图层加入画布。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成工具后先在画布中心创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图或视频落在最新占位框位置，输入框继续跟随新生成图层；占位图失焦时隐藏高亮边框、左上角生成器名称和右上角原始尺寸，重新聚焦时再显示，且名称 / 尺寸在画布缩小时按 viewport 反向缩放保持屏幕尺寸稳定；点击所有图片 / 视频生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。快速编辑和修改图片在调用后端前必须把当前图层图片源读取为图片 Data URL，来源可以是本地上传 Data URL、站内 public 图片、历史 `/generated-*` 路径或可读取的 OSS generated URL；后端仍只接收图片 Data URL，不把普通 URL 直接透传到 VectorEngine edits。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。
|
||||
- 底部生成类按钮每次点击都必须创建独立的画布生成对象；新建规范、角色形象或图标素材时，只切换当前编辑面板，不得销毁此前尚未生成或已生成后的其它生成对象状态。归档为非当前编辑对象的生成占位仍可拖动、删除和等待异步完成，完成 / 失败回写必须按生成对象 ID 读取最新占位状态，不能使用提交瞬间的旧快照。
|
||||
|
||||
## 交互规则
|
||||
@@ -62,8 +62,10 @@
|
||||
- `POST /api/editor/assets`：批量或单个创建账号级素材，支持按钮上传和拖拽上传后的 data URL / 后续 OSS 元数据。
|
||||
- `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。
|
||||
- `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。
|
||||
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine `gpt-image-2` 生成图片；携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；接口返回 data URL、尺寸、prompt、model、provider 和 taskId。
|
||||
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine 生成图片；角色生成可携带 `model`、`aspectRatio`、`imageSize` 和 `referenceImageSrcs`。`nanobanana2` 参考图作为 `inline_data` 进入 `generateContent`，`gpt-image-2` 参考图进入 edits。携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；接口返回 data URL、尺寸、prompt、model、provider 和 taskId。
|
||||
- `POST /api/editor/icon-spritesheets/generations`：按图标素材规范图和素材描述数组生成 spritesheet，再由后端切分为独立透明图标。请求支持 `model`、`aspectRatio`、`imageSize`；`nanobanana2` 走原生 `generateContent` 并写入 `generationConfig.imageConfig.aspectRatio/imageSize`，`0.5K` 传 `"512"`；`gpt-image-2` 走 `/v1/images/edits`，后端把 UI 尺寸归一为文档支持的 `1024x1024`、`1024x1536`、`1536x1024`、`2048x2048`、`2048x1152` 等 `size` 字符串。
|
||||
- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据。
|
||||
- `POST /api/editor/videos/generations`：按视频描述、模型、比例、时长、分辨率、模式、声音和泥点价格生成视频。请求模型支持 `seedance2.0`、`seedance2.0-fast`、`kling3.0`、`kling3.0-omni`、`veo3.1`、`veo3.1-fast`；后端复用 Ark / VectorEngine content generation task 轮询链路，下载最终视频并持久化到 OSS，返回 `videoSrc`、尺寸、prompt、model、provider、taskId、durationSeconds、resolution 和 `priceMudPoints`。
|
||||
|
||||
所有写接口都必须校验 Bearer 登录态和 owner；接口只返回当前用户有权读取的工程与资源。
|
||||
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# 生成类面板 Lovart 统一改造方案
|
||||
|
||||
日期：`2026-06-17`
|
||||
|
||||
## 范围
|
||||
|
||||
图片画布编辑器内以下生成类面板统一对齐 Lovart 式极简创作工具风格：
|
||||
|
||||
- `生成图片`
|
||||
- `生成规范`
|
||||
- `生成角色形象`
|
||||
- `生成图标素材`
|
||||
- `生成UI设计图`
|
||||
- `生成视频`
|
||||
- `角色动画生成面板`
|
||||
|
||||
## 统一布局
|
||||
|
||||
1. 参考图区域永远位于面板第一行。
|
||||
2. 参考图使用统一参考图卡片：
|
||||
- 普通参考图：灰蓝色图片图标。
|
||||
- 规范参考图：紫色剪贴板图标。
|
||||
- 图标素材规范：绿色图标图标。
|
||||
- UI 图标规范：琥珀色图标图标。
|
||||
- 视频参考图 / 角色视频源图：深色视频或角色图标。
|
||||
3. 参考图卡片尽量少文字；必要文字写在图标块内或短标签内，不写规则说明。
|
||||
4. 普通参考图支持连续追加：已有图缩略图后始终保留一个 `+` 入口。点击入口只弹出“从画布中选择 / 上传图片”来源选项，不再直接打开系统文件选择器；生成图片、生成视频、角色常规参考图等同类参考图入口都遵循同一交互。
|
||||
5. 单文本输入面板不显示文本框标题，用问题式 placeholder：
|
||||
- 生成图片：`今天想生成什么画面？`
|
||||
- 生成角色：`你希望角色如何设计？`
|
||||
- 生成 UI：`你希望这个 UI 长什么样？`
|
||||
- 生成视频：`你希望生成什么视频？`
|
||||
- 角色动画：`你希望角色做什么动作？`
|
||||
6. 多输入框面板必须保留每个字段标题和输入框边界，例如生成规范、图标素材多描述。
|
||||
7. 生成规范下的角色形象规范、UI素材规范、图标素材规范和自定义规范都使用同一生成类 shell：首行参考图区域、中央字段区、底部生成按钮区，不再出现缺首行参考区或单独 footer 样式。
|
||||
|
||||
## 参数交互
|
||||
|
||||
画面比例、大小尺寸、模型、分辨率、时长等生成参数统一为 Lovart 式底部胶囊按钮：
|
||||
|
||||
```text
|
||||
左下：[比例 · 尺寸 ˄] 右下：[模型 ˄] [生成 · 图标 12]
|
||||
```
|
||||
|
||||
- 面板内不显示 `画面比例`、`大小尺寸`、`模型` 等字段标题，只显示当前选择值。
|
||||
- 图片类面板把画面比例和大小尺寸合并成一个左下角选项框；视频 / 角色动画把比例、时长、清晰度合并成一个左下角选项框。
|
||||
- 模型选项框和生成按钮位于右下角。
|
||||
- 点击后在父级生成面板内向上弹出子面板；父级面板隐藏或销毁时，子面板同步销毁。
|
||||
- 子面板展示时，点击父级面板中除当前子面板和选项触发器以外的任意区域即可收起，不额外显示右上角关闭按钮。
|
||||
- 弹出面板内可分组展示字段，点击某个选项只更新当前字段和选中样式，不关闭弹出面板，方便连续修改。
|
||||
- 模型子面板每行固定一个模型，不用方框包裹模型名；模型名不换行，前置对应模型类型图标，选中项在模型名后用对号标记。
|
||||
- 比例选项卡片内展示对应比例的线框。
|
||||
- 父级面板、底部选项框、弹出子面板字号保持一致。
|
||||
- 底部组合值使用 `·` 分隔，例如 `16:9 · 4秒 · 480p`。
|
||||
- 底部参数热区与生成按钮等高，默认不显示阴影；悬停显示轻量阴影；箭头默认向下，展开后旋转向上。
|
||||
- 生成图片和生成视频文本输入框紧贴参考图下方，取消旧网格预留导致的空白高度。
|
||||
- 不再在底部常驻展开全部可选项。
|
||||
|
||||
## 泥点显示
|
||||
|
||||
- 本次消耗泥点必须显示在生成按钮内部。
|
||||
- `泥点` 文本不在 UI 中显示，改用图标 + 数值，例如 `生成 ✦ 12`。
|
||||
- 泥点配置统一收口到 `api-server` 的编辑器生成配置模块；前端只保留与后端配置同名的展示兜底，后续可接接口动态下发。
|
||||
|
||||
## 第一版计费配置
|
||||
|
||||
```text
|
||||
生成图片：12 泥点
|
||||
生成规范：5 泥点
|
||||
生成角色形象：12 泥点
|
||||
生成图标素材：12 泥点
|
||||
生成UI设计图：12 泥点
|
||||
生成视频：480p 每秒 10 泥点，720p 每秒 20 泥点
|
||||
角色动画：480p 每秒 10 泥点，720p 每秒 20 泥点
|
||||
```
|
||||
|
||||
## 生成视频模型与接口
|
||||
|
||||
- 底部 `生成视频` 面板提交到 `POST /api/editor/videos/generations`。
|
||||
- 首期只开放文字生成视频，固定 `16:9`、静音、标准模式；结果作为 `mediaType="video"`、`assetKind="video"` 的视频图层加入画布。
|
||||
- 支持模型：
|
||||
- `seedance2.0`
|
||||
- `seedance2.0-fast`
|
||||
- `kling3.0`
|
||||
- `kling3.0-omni`
|
||||
- 不展示 Veo 模型入口。
|
||||
- `veo3.1`
|
||||
- `veo3.1-fast`
|
||||
- 后端复用现有 Ark / VectorEngine content generation task 轮询链路，并把生成视频持久化到 OSS；缺少 `ARK_CHARACTER_VIDEO_BASE_URL` 或 `ARK_CHARACTER_VIDEO_API_KEY` 时 fail-closed 返回配置错误。
|
||||
|
||||
## 验收
|
||||
|
||||
- 所有生成类面板首行都是参考图区域。
|
||||
- 比例 / 尺寸 / 模型不再平铺全部选项；比例与尺寸合并为左下角当前值按钮，模型与生成按钮位于右下角。
|
||||
- 单文本输入面板不显示字段标题，placeholder 是问题式文案。
|
||||
- 多文本输入面板字段标题和边界仍清晰。
|
||||
- 生成按钮内能看到图标化泥点消耗数值。
|
||||
- 弹出选项面板点击选项后保持打开，可连续修改多个字段。
|
||||
- 规范面板比图片生成面板更紧凑，字段间距和输入高度更小，但外层 shell、首行参考图和底部按钮区必须继续对齐生成图片 / 生成角色 / 生成视频。
|
||||
- 后端存在独立编辑器生成计费配置文件，角色动画价格校验使用该配置。
|
||||
- 生成视频结果以视频图层加入画布，画布媒体元素标记为 `画布视频：生成视频 N`。
|
||||
@@ -0,0 +1,53 @@
|
||||
# 【编辑器】画板UI设计图生成入口设计
|
||||
|
||||
日期：2026-06-17
|
||||
|
||||
## 入口与画布状态
|
||||
|
||||
- 底部 AI 画布工具栏新增 `生成UI设计图`。
|
||||
- 点击后立即在画布中新建 `UI设计图生成占位图`，不复用普通新建图片的空白样式。
|
||||
- 占位图默认 16:9 展示，生成成功后替换为 `assetKind: "ui-design"` 的画布图层。
|
||||
|
||||
## 生成面板
|
||||
|
||||
- 占位图下方打开独立生成面板，标题为 `生成UI设计图`。
|
||||
- 面板复用普通 `生成图片` / `生成角色形象` / `生成视频` 的生成类 shell：首行参考图区域、中央单文本输入、底部参数与生成按钮区。
|
||||
- 面板第一个模块为 `图标素材规范`，并放在首行参考图区域。
|
||||
- 点击图标素材规范卡片后，在卡片旁弹出来源菜单：
|
||||
- `从画布中选择`
|
||||
- `新建图标素材规范`
|
||||
- `上传图片`
|
||||
- `从画布中选择` 只接受 `assetKind: "icon-spec"` 的图层；普通图片、其他类别图层和不携带标签的图片不绑定。
|
||||
- `新建图标素材规范` 复用现有图标素材规范生成表单。
|
||||
- `上传图片` 仅绑定到当前 UI 设计面板的图标素材规范参考，不自动添加为画布图层。
|
||||
|
||||
## 生成参数
|
||||
|
||||
- 支持自定义画面比例和大小尺寸。
|
||||
- 模型固定为 `gpt-image-2`，面板不提供切换到其他模型的能力。
|
||||
- 默认画面比例为 `16:9`，默认大小为 `1K`。
|
||||
|
||||
## 提示词契约
|
||||
|
||||
后端收到 `kind: "ui-design"` 时固定拼接：
|
||||
|
||||
```text
|
||||
生成玩法UI原型图
|
||||
【用户输入】<用户输入>
|
||||
```
|
||||
|
||||
如果用户设置了图标素材规范参考图，则追加：
|
||||
|
||||
```text
|
||||
参考图1为图标素材规范，请在UI图标、按钮符号、描边、材质、圆角、阴影和状态层级上严格遵循参考图1的素材规范。
|
||||
```
|
||||
|
||||
生成请求固定使用 `gpt-image-2`。有参考图时走图片编辑请求；无参考图时走图片生成请求。
|
||||
|
||||
## 验收点
|
||||
|
||||
- 点击 `生成UI设计图` 后出现 UI 设计占位图和独立生成面板。
|
||||
- 面板第一模块为图标素材规范，来源菜单包含三个动作。
|
||||
- 从画布选择时只能绑定图标素材规范图片。
|
||||
- 请求参数包含 `kind: "ui-design"`、`model: "gpt-image-2"`、比例、大小与可选参考图。
|
||||
- 生成图层信息面板展示 `用户输入` 与 `图标素材规范`。
|
||||
@@ -39,11 +39,13 @@
|
||||
- 请求字段：
|
||||
- `referenceImageSrc`：图标素材规范 Data URL。
|
||||
- `iconDescriptions`：过滤空文本后的图标描述数组，`1..100`。
|
||||
- `model`：固定 `gemini-3.1-flash-image-preview`。
|
||||
- 后端根据图标数量选择尺寸：
|
||||
- `<=25` 个：`512x512`，即 0.5K 1:1。
|
||||
- `>25` 个：`1024x1024`，即 1K 1:1。
|
||||
- 后端使用 VectorEngine 图片编辑接口，把 `referenceImageSrc` 作为参考图 1，模型固定传 `gemini-3.1-flash-image-preview`。
|
||||
- `model`：支持 `gemini-3.1-flash-image-preview`（UI 显示 `nanobanana2`）和 `gpt-image-2`，默认 `nanobanana2`。
|
||||
- `aspectRatio`：按 `x:y` 展示，选项跟随模型。
|
||||
- `imageSize`：按 `0.5K / 1K / 2K` 展示，选项跟随模型。
|
||||
- 模型与尺寸选项：
|
||||
- `nanobanana2`：比例 `1:1 / 2:3 / 3:2 / 9:16 / 16:9`；大小 `0.5K / 1K / 2K`。后端走 `/v1beta/models/{model}:generateContent`，把图标素材规范图作为 `inline_data`，并把 `aspectRatio` / `imageSize` 写入 `generationConfig.imageConfig`；`0.5K` 按 VectorEngine 文档传 `"512"`。
|
||||
- `gpt-image-2`：比例 `1:1 / 2:3 / 3:2 / 9:16 / 16:9`；大小 `1K / 2K`。后端走 `/v1/images/edits`，把图标素材规范图作为 multipart `image`，按 `size` 映射：`1K 1:1 -> 1024x1024`、`1K 2:3/9:16 -> 1024x1536`、`1K 3:2/16:9 -> 1536x1024`、`2K 1:1 -> 2048x2048`、`2K 3:2/16:9 -> 2048x1152`；文档未列出 `2K` 竖版，`2K 2:3/9:16` 后端回落到 `1024x1536`。
|
||||
- 用户在角色或图标素材面板中切换过模型后，下一次打开这两类面板继续使用上次模型。
|
||||
- Prompt 固定为：
|
||||
|
||||
```text
|
||||
@@ -72,6 +74,6 @@
|
||||
- 点击 `生成图标素材` 后出现一叠空白图标占位和图标素材面板。
|
||||
- `图标素材规范 -> 从画布中选择` 只能选择图标素材规范图，点击普通图片或角色规范图不会绑定。
|
||||
- 默认 6 个素材描述会进入 prompt；新增描述最多到 100 个。
|
||||
- `<=25` 个描述提交时后端请求尺寸为 `512x512`；`>25` 个描述提交时后端请求尺寸为 `1024x1024`。
|
||||
- VectorEngine 请求体的 `model` 为 `gemini-3.1-flash-image-preview`。
|
||||
- 默认打开图标素材面板时选中 `nanobanana2 / 1:1 / 1K`；模型切换后，角色和图标素材面板之间沿用上次选择的模型。
|
||||
- 图标素材生成请求必须带 `model`、`aspectRatio` 和 `imageSize`；`nanobanana2` 请求体必须包含 `generationConfig.imageConfig.aspectRatio/imageSize`，`gpt-image-2` 请求必须包含文档映射后的 `size`。
|
||||
- 生成成功后画布出现按描述命名的多个透明图标素材图层，图层之间不重叠。
|
||||
|
||||
@@ -32,6 +32,16 @@
|
||||
4. 左下角展示画面比例和大小选择按钮。
|
||||
5. 右下角展示模型选择和生成按钮。
|
||||
|
||||
## 普通生成面板视觉口径
|
||||
|
||||
普通 `生成图片`、`生成角色形象`、`生成UI设计图` 等锚定在图片下方的生成面板统一采用 Lovart 式极简创作工具风格：
|
||||
|
||||
- PC 端优先保持面板锚定在占位图或目标图片下方，宽度允许比旧版略宽，避免比例、尺寸、模型和生成按钮挤成一行难以阅读。
|
||||
- 面板只展示创作必需输入：参考图、提示词、比例、大小、模型和生成按钮；不在 UI 内铺说明性规则文案。
|
||||
- 视觉以白色半透明面板、轻边框、低阴影、低饱和选项按钮和清晰黑色主按钮为主，减少游戏式厚重装饰，贴近 Lovart 的极简画布工具感。
|
||||
- 普通 `生成图片` 面板的比例、大小和模型必须使用真实选项按钮，不再使用占位式参数按钮或弹出“建设中”提示。
|
||||
- 移动端仍可固定在底部工具栏上方并允许内部滚动，保证不遮挡底部 AI 工具栏和画布操作。
|
||||
|
||||
## 生成与参考图契约
|
||||
|
||||
- 前端提交角色生成时，使用 `POST /api/editor/images/generations`。
|
||||
@@ -39,8 +49,14 @@
|
||||
- 角色形象规范与常规参考图作为 `referenceImageSrcs` 传入，顺序固定为：
|
||||
1. 角色形象规范图。
|
||||
2. 常规参考图列表。
|
||||
- 当前请求尺寸沿用编辑器普通生成默认值；比例和大小按钮先复用现有占位交互。
|
||||
- 后端如果收到参考图，则走带多参考图的图片编辑/参考图生成链路；没有参考图时走纯文本生成链路。
|
||||
- 请求同时提交 `model`、`aspectRatio` 和 `imageSize`：
|
||||
- `model` 支持 `gemini-3.1-flash-image-preview`（UI 显示 `nanobanana2`）和 `gpt-image-2`，默认 `nanobanana2`。
|
||||
- 用户在角色或图标素材面板中切换过模型后，下一次打开这两类面板继续使用上次模型。
|
||||
- 比例按 `x:y` 展示；大小按 `0.5K / 1K / 2K` 展示。
|
||||
- 尺寸选项来源以 VectorEngine 接入文档为准：
|
||||
- `nanobanana2`：比例 `1:1 / 2:3 / 3:2 / 9:16 / 16:9`；大小 `0.5K / 1K / 2K`。后端走 `/v1beta/models/{model}:generateContent`，把比例写入 `generationConfig.imageConfig.aspectRatio`，把大小写入 `generationConfig.imageConfig.imageSize`；其中 `0.5K` 按文档传 `"512"`。
|
||||
- `gpt-image-2`：比例 `1:1 / 2:3 / 3:2 / 9:16 / 16:9`；大小 `1K / 2K`。后端走 `/v1/images/generations` 或 `/v1/images/edits`，按文档尺寸映射：`1K 1:1 -> 1024x1024`、`1K 2:3/9:16 -> 1024x1536`、`1K 3:2/16:9 -> 1536x1024`、`2K 1:1 -> 2048x2048`、`2K 3:2/16:9 -> 2048x1152`；文档未列出 `2K` 竖版，`2K 2:3/9:16` 后端回落到 `1024x1536`。
|
||||
- 后端如果收到参考图，`nanobanana2` 把参考图作为 `inline_data` 传入原生 `generateContent`；`gpt-image-2` 走带多参考图的图片编辑链路。没有参考图时按所选模型走纯文本生成链路。
|
||||
- `kind = "character"` 时，后端不直接把前端文本当完整生图提示词，而是把文本作为 `角色设定` 填入固定提示词骨架：
|
||||
|
||||
```text
|
||||
@@ -52,11 +68,18 @@
|
||||
|
||||
## 生成规范参考图
|
||||
|
||||
- `生成规范 -> 角色形象规范`、`UI素材规范`、`自定义规范` 的设定面板支持上传 1 张参考图；`图标素材规范` 继续使用后续图标素材生成面板里的专用规范图链路，不在这里重复新增入口。
|
||||
- `生成规范 -> 角色形象规范`、`UI素材规范`、`图标素材规范`、`自定义规范` 的设定面板都支持上传 1 张参考图，并统一放在面板首行参考图区域。
|
||||
- 参考图入口只展示字段标题、缩略图或上传图标、文件名，不把参考规则说明铺在 UI 上。
|
||||
- 提交生成规范时，若存在参考图，前端必须把参考图作为 `referenceImageSrcs[0]` 提交到 `/api/editor/images/generations`，并在生图提示词开头自动追加“参考图生成规范”语义：要求模型参考图 1 的构图、风格、材质、色彩、形状语言和视觉层级生成规范图，但不要复制参考图中的文字、水印或无关背景。
|
||||
- 生成结果的信息快照必须记录该参考图，标题为 `参考图`，便于后续在图片信息面板回看生成输入。
|
||||
|
||||
## 新建生成图落点避让
|
||||
|
||||
- 普通生成、生成规范、生成角色形象和生成图标素材在创建画布占位图前，必须先检测占位图矩形是否与画布中已有可见图片图层或已有生成占位图重叠。
|
||||
- 若当前屏幕中心对应的画板位置可用，则占位图仍创建在该中心；若重叠，则以当前屏幕中心对应画板位置为原点，按距离由近到远查找不重叠候选位置。
|
||||
- 候选占位图与相邻图片或占位图之间保留 `32px` 画板间距；隐藏图层不阻挡新建落点。
|
||||
- 选中落点后保持当前缩放比例不变，将视口中心移动到新占位图中心，确保用户创建后立即看到新图和生成面板。
|
||||
|
||||
## 图片信息展示
|
||||
|
||||
- 图片信息弹窗标题固定为 `图片信息`，不拼接图片 / 图层名称。
|
||||
@@ -78,13 +101,16 @@
|
||||
- `角色形象规范` 与 `上传常规参考图` 入口是带预览视觉块的参考图卡片，不是无样式文字。
|
||||
- `从画布中选择` 后点击已有画布图片可绑定为角色形象规范，`Esc` 可退出点选状态。
|
||||
- 上传常规参考图后缩略图右下角显示序号。
|
||||
- 输入角色设定并生成时，请求包含 `kind: "character"`、角色设定 prompt 和参考图数组。
|
||||
- 输入角色设定并生成时，请求包含 `kind: "character"`、角色设定 prompt、参考图数组、`model`、`aspectRatio` 和 `imageSize`。
|
||||
- 默认打开角色生成面板时选中 `nanobanana2 / 1:1 / 1K`；切换到 `gpt-image-2` 后再次打开角色或图标素材面板应沿用该模型。
|
||||
- 生成成功后在占位图位置创建 `assetKind: "character"` 图层，右上角显示 `角色` 标签，布局保存包含该字段。
|
||||
|
||||
## 当前落地记录
|
||||
|
||||
- 前端画板已接入 `生成角色形象` 底部入口、角色占位图、角色面板、画布点选规范图、上传规范图、上传常规参考图和序号角标。
|
||||
- 画布生成类入口已统一接入新建占位图落点避让：优先使用当前屏幕中心对应画板位置，重叠时自动选择最近的不重叠位置，并将视口中心移动到新占位图。
|
||||
- 角色生成提交统一走 `/api/editor/images/generations`，按 `角色形象规范 -> 常规参考图` 顺序传 `referenceImageSrcs`，并写入 `assetKind: "character"`。
|
||||
- 角色和图标素材生成已接入 `nanobanana2` / `gpt-image-2` 模型切换、上次模型记忆，以及按模型归一的比例 / 大小尺寸；`nanobanana2` 使用原生 `generateContent` 的 `imageConfig.aspectRatio/imageSize`，`gpt-image-2` 使用文档列出的 `size` 字符串。
|
||||
- 角色生成后端已按固定 prompt 骨架补入 `角色设定`，并在生成成功后自动执行绿幕去背、写入 `generated-character-drafts/editor/character-images/<taskId>/image.png` 路径下的 OSS 私有对象，返回的 `objectKey` / `assetObjectId` 会随画板资源记录保存。
|
||||
- `Esc` 只退出角色规范画布点选状态，不关闭角色生成面板。
|
||||
- 已补充回归测试覆盖角色形象生成、点选退出、角色动画入口隔离和快速编辑入口。
|
||||
|
||||
@@ -42,7 +42,8 @@ use shared_contracts::assets::{
|
||||
CharacterVisualDraftPayload, CharacterWorkflowCacheGetResponse, CharacterWorkflowCachePayload,
|
||||
CharacterWorkflowCacheSaveRequest, CharacterWorkflowCacheSaveResponse,
|
||||
EditorCharacterAnimationFramePayload, EditorCharacterAnimationGenerateRequest,
|
||||
EditorCharacterAnimationGenerateResponse,
|
||||
EditorCharacterAnimationGenerateResponse, EditorVideoGenerateRequest,
|
||||
EditorVideoGenerateResponse,
|
||||
};
|
||||
use spacetime_client::SpacetimeClientError;
|
||||
|
||||
@@ -87,6 +88,15 @@ const ARK_VIDEO_TASK_POLL_INTERVAL_MS: u64 = 5_000;
|
||||
const EDITOR_CHARACTER_ANIMATION_MODEL: &str = "seedance2.0";
|
||||
const EDITOR_CHARACTER_ANIMATION_ASSET_KIND: &str = "editor_character_animation";
|
||||
const EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX: &str = "生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。\n动作描述：";
|
||||
const EDITOR_VIDEO_ASSET_KIND: &str = "editor_video";
|
||||
const EDITOR_VIDEO_ENTITY_KIND: &str = "editor_canvas";
|
||||
const EDITOR_VIDEO_SLOT: &str = "video_preview";
|
||||
const EDITOR_VIDEO_MODEL_SEEDANCE_2: &str = "seedance2.0";
|
||||
const EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST: &str = "seedance2.0-fast";
|
||||
const EDITOR_VIDEO_MODEL_KLING_3: &str = "kling3.0";
|
||||
const EDITOR_VIDEO_MODEL_KLING_3_OMNI: &str = "kling3.0-omni";
|
||||
const EDITOR_VIDEO_MODEL_VEO_3_1: &str = "veo3.1";
|
||||
const EDITOR_VIDEO_MODEL_VEO_3_1_FAST: &str = "veo3.1-fast";
|
||||
|
||||
const BUILT_IN_MOTION_TEMPLATES: [MotionTemplate; 4] = [
|
||||
MotionTemplate {
|
||||
@@ -575,6 +585,63 @@ pub async fn generate_editor_character_animation(
|
||||
))
|
||||
}
|
||||
|
||||
pub async fn generate_editor_video(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
payload: Result<Json<EditorVideoGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = payload.map_err(|error| {
|
||||
editor_video_error_response(
|
||||
&request_context,
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-video",
|
||||
"message": error.body_text(),
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
|
||||
let normalized =
|
||||
normalize_editor_video_request(payload).map_err(|error| {
|
||||
editor_video_error_response(&request_context, error)
|
||||
})?;
|
||||
let settings = require_editor_video_settings(&state, normalized.model.as_str()).map_err(
|
||||
|error| editor_video_error_response(&request_context, error),
|
||||
)?;
|
||||
let http_client = build_upstream_http_client(settings.ark.request_timeout_ms)
|
||||
.map_err(|error| editor_video_error_response(&request_context, error))?;
|
||||
let task_id = generate_ai_task_id(current_utc_micros());
|
||||
|
||||
let generated = request_editor_video_preview(
|
||||
&state,
|
||||
&http_client,
|
||||
&settings,
|
||||
"editor-video",
|
||||
task_id.as_str(),
|
||||
&normalized,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| editor_video_error_response(&request_context, error))?;
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorVideoGenerateResponse {
|
||||
ok: true,
|
||||
video_src: generated.preview_video_path,
|
||||
width: normalized.width,
|
||||
height: normalized.height,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: normalized.prompt,
|
||||
actual_prompt: Some(generated.submitted_prompt),
|
||||
model: normalized.model,
|
||||
provider: "VectorEngine".to_string(),
|
||||
task_id,
|
||||
duration_seconds: normalized.duration_seconds,
|
||||
resolution: normalized.resolution,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
},
|
||||
))
|
||||
}
|
||||
|
||||
pub async fn get_character_animation_job(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
@@ -1371,6 +1438,114 @@ async fn request_editor_character_animation_preview(
|
||||
})
|
||||
}
|
||||
|
||||
async fn request_editor_video_preview(
|
||||
state: &AppState,
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorVideoSettings,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<GeneratedAnimationPreview, AppError> {
|
||||
let upstream_task_id =
|
||||
create_editor_text_to_video_task(http_client, settings, request).await?;
|
||||
let video_url =
|
||||
wait_for_ark_content_generation_task(http_client, &settings.ark, upstream_task_id.as_str())
|
||||
.await?;
|
||||
let preview_payload =
|
||||
download_generated_video(http_client, video_url.as_str(), "下载画板生成视频失败。").await?;
|
||||
let preview_video_path =
|
||||
put_generated_editor_video(state, owner_user_id, task_id, request, preview_payload).await?;
|
||||
|
||||
Ok(GeneratedAnimationPreview {
|
||||
preview_video_path,
|
||||
upstream_task_id,
|
||||
submitted_prompt: request.prompt.clone(),
|
||||
moderation_fallback_applied: false,
|
||||
})
|
||||
}
|
||||
|
||||
async fn create_editor_text_to_video_task(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorVideoSettings,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<String, AppError> {
|
||||
let response = http_client
|
||||
.post(format!("{}/contents/generations/tasks", settings.ark.base_url))
|
||||
.header(
|
||||
reqwest::header::AUTHORIZATION,
|
||||
format!("Bearer {}", settings.ark.api_key),
|
||||
)
|
||||
.header(reqwest::header::CONTENT_TYPE, "application/json")
|
||||
.json(&json!({
|
||||
"model": request.provider_model,
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": request.prompt,
|
||||
}
|
||||
],
|
||||
"resolution": request.resolution,
|
||||
"ratio": request.aspect_ratio,
|
||||
"duration": request.duration_seconds,
|
||||
"mode": "std",
|
||||
"watermark": false,
|
||||
}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|error| {
|
||||
map_character_animation_upstream_error(format!("请求画板视频服务失败：{error}"))
|
||||
})?;
|
||||
|
||||
let status = response.status();
|
||||
let body = response.text().await.map_err(|error| {
|
||||
map_character_animation_upstream_error(format!("读取画板视频任务响应失败：{error}"))
|
||||
})?;
|
||||
if !status.is_success() {
|
||||
return Err(parse_animation_upstream_error(
|
||||
body.as_str(),
|
||||
"创建画板视频任务失败。",
|
||||
));
|
||||
}
|
||||
let payload = parse_animation_json_payload(body.as_str(), "创建画板视频任务失败。")?;
|
||||
extract_animation_task_id(&payload.payload).ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "editor-video",
|
||||
"message": "画板视频任务未返回任务 id。",
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
async fn put_generated_editor_video(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
preview_payload: MediaPayload,
|
||||
) -> Result<String, AppError> {
|
||||
let put_result = put_character_animation_object(
|
||||
state,
|
||||
LegacyAssetPrefix::CharacterDrafts,
|
||||
vec![
|
||||
"editor-videos".to_string(),
|
||||
sanitize_storage_segment(request.model.as_str(), "model"),
|
||||
task_id.to_string(),
|
||||
],
|
||||
format!("preview.{}", preview_payload.extension),
|
||||
preview_payload.mime_type,
|
||||
preview_payload.bytes,
|
||||
build_asset_metadata(
|
||||
EDITOR_VIDEO_ASSET_KIND,
|
||||
owner_user_id,
|
||||
EDITOR_VIDEO_ENTITY_KIND,
|
||||
task_id,
|
||||
EDITOR_VIDEO_SLOT,
|
||||
request.model.as_str(),
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
Ok(put_result.legacy_public_path)
|
||||
}
|
||||
|
||||
async fn create_editor_ark_image_to_video_task(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorCharacterAnimationSettings,
|
||||
@@ -2131,7 +2306,10 @@ fn normalize_editor_character_animation_request(
|
||||
let frame_count = normalize_editor_character_animation_frame_count(payload.frame_count)?;
|
||||
let duration_seconds =
|
||||
normalize_editor_character_animation_duration(payload.duration_seconds, frame_count)?;
|
||||
let expected_price = calculate_editor_character_animation_price(resolution, duration_seconds);
|
||||
let expected_price = crate::editor_generation_config::editor_character_animation_mud_points(
|
||||
resolution,
|
||||
duration_seconds,
|
||||
);
|
||||
if payload.price_mud_points != expected_price {
|
||||
return Err(editor_character_animation_bad_request(format!(
|
||||
"priceMudPoints 与分辨率和时长不一致，应为 {expected_price}。"
|
||||
@@ -2212,6 +2390,145 @@ fn require_editor_character_animation_settings(
|
||||
})
|
||||
}
|
||||
|
||||
fn normalize_editor_video_request(
|
||||
payload: EditorVideoGenerateRequest,
|
||||
) -> Result<NormalizedEditorVideoRequest, AppError> {
|
||||
let prompt = payload.prompt.trim().chars().take(4000).collect::<String>();
|
||||
if prompt.is_empty() {
|
||||
return Err(editor_video_bad_request("视频描述不能为空。"));
|
||||
}
|
||||
let model = normalize_editor_video_model(payload.model.as_str())?;
|
||||
let aspect_ratio = normalize_editor_video_aspect_ratio(payload.aspect_ratio.as_str())?;
|
||||
let resolution = normalize_editor_video_resolution(payload.resolution.as_str())?;
|
||||
let duration_seconds = normalize_editor_video_duration(payload.duration_seconds)?;
|
||||
if payload.mode.trim() != "std" {
|
||||
return Err(editor_video_bad_request("mode 只支持 std。"));
|
||||
}
|
||||
if payload.sound.trim() != "off" {
|
||||
return Err(editor_video_bad_request("sound 只支持 off。"));
|
||||
}
|
||||
let expected_price = crate::editor_generation_config::editor_video_generation_mud_points(
|
||||
resolution,
|
||||
duration_seconds,
|
||||
);
|
||||
if payload.price_mud_points != expected_price {
|
||||
return Err(editor_video_bad_request(format!(
|
||||
"priceMudPoints 与分辨率和时长不一致，应为 {expected_price}。"
|
||||
)));
|
||||
}
|
||||
let (width, height) = resolve_editor_video_size(aspect_ratio, resolution);
|
||||
|
||||
Ok(NormalizedEditorVideoRequest {
|
||||
prompt,
|
||||
model: model.to_string(),
|
||||
provider_model: resolve_editor_video_provider_model(model).to_string(),
|
||||
aspect_ratio: aspect_ratio.to_string(),
|
||||
resolution: resolution.to_string(),
|
||||
duration_seconds,
|
||||
price_mud_points: expected_price,
|
||||
width,
|
||||
height,
|
||||
})
|
||||
}
|
||||
|
||||
fn normalize_editor_video_model(value: &str) -> Result<&'static str, AppError> {
|
||||
match value.trim() {
|
||||
EDITOR_VIDEO_MODEL_SEEDANCE_2 => Ok(EDITOR_VIDEO_MODEL_SEEDANCE_2),
|
||||
EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST => Ok(EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST),
|
||||
EDITOR_VIDEO_MODEL_KLING_3 => Ok(EDITOR_VIDEO_MODEL_KLING_3),
|
||||
EDITOR_VIDEO_MODEL_KLING_3_OMNI => Ok(EDITOR_VIDEO_MODEL_KLING_3_OMNI),
|
||||
EDITOR_VIDEO_MODEL_VEO_3_1 => Ok(EDITOR_VIDEO_MODEL_VEO_3_1),
|
||||
EDITOR_VIDEO_MODEL_VEO_3_1_FAST => Ok(EDITOR_VIDEO_MODEL_VEO_3_1_FAST),
|
||||
_ => Err(editor_video_bad_request(
|
||||
"model 只支持 seedance2.0、seedance2.0-fast、kling3.0、kling3.0-omni、veo3.1、veo3.1-fast。",
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_editor_video_provider_model(model: &str) -> &str {
|
||||
match model {
|
||||
// 中文注释：Seedance 2.0 Fast 复用平台已有 fast seedance 上游模型；标准版按产品 ID 透传。
|
||||
EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST => CHARACTER_ANIMATION_MODEL,
|
||||
_ => model,
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_video_aspect_ratio(value: &str) -> Result<&'static str, AppError> {
|
||||
match value.trim() {
|
||||
"16:9" => Ok("16:9"),
|
||||
_ => Err(editor_video_bad_request("aspectRatio 只支持 16:9。")),
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_video_resolution(value: &str) -> Result<&'static str, AppError> {
|
||||
match value.trim() {
|
||||
"480p" => Ok("480p"),
|
||||
"720p" => Ok("720p"),
|
||||
_ => Err(editor_video_bad_request("resolution 只支持 480p 或 720p。")),
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_video_duration(value: u32) -> Result<u32, AppError> {
|
||||
match value {
|
||||
4 | 5 => Ok(value),
|
||||
_ => Err(editor_video_bad_request("durationSeconds 只支持 4 或 5。")),
|
||||
}
|
||||
}
|
||||
|
||||
fn resolve_editor_video_size(aspect_ratio: &str, resolution: &str) -> (u32, u32) {
|
||||
match (aspect_ratio, resolution) {
|
||||
("16:9", "720p") => (1280, 720),
|
||||
_ => (854, 480),
|
||||
}
|
||||
}
|
||||
|
||||
fn require_editor_video_settings(
|
||||
state: &AppState,
|
||||
model: &str,
|
||||
) -> Result<EditorVideoSettings, AppError> {
|
||||
let base_url = state
|
||||
.config
|
||||
.ark_character_video_base_url
|
||||
.trim()
|
||||
.trim_end_matches('/');
|
||||
if base_url.is_empty() {
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "ark",
|
||||
"reason": "ARK_CHARACTER_VIDEO_BASE_URL 未配置",
|
||||
})),
|
||||
);
|
||||
}
|
||||
let api_key = state
|
||||
.config
|
||||
.ark_character_video_api_key
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "ark",
|
||||
"reason": "ARK_CHARACTER_VIDEO_API_KEY 未配置",
|
||||
}))
|
||||
})?;
|
||||
|
||||
Ok(EditorVideoSettings {
|
||||
ark: ArkVideoSettings {
|
||||
base_url: base_url.to_string(),
|
||||
api_key: api_key.to_string(),
|
||||
request_timeout_ms: state.config.ark_character_video_request_timeout_ms.max(1),
|
||||
model: resolve_editor_video_provider_model(model).to_string(),
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
fn editor_video_bad_request(message: impl Into<String>) -> AppError {
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-video",
|
||||
"message": message.into(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn build_editor_character_animation_prompt(prompt_text: &str) -> String {
|
||||
format!(
|
||||
"{}\n{}",
|
||||
@@ -2272,11 +2589,6 @@ fn normalize_editor_character_animation_duration(
|
||||
}
|
||||
}
|
||||
|
||||
fn calculate_editor_character_animation_price(resolution: &str, duration_seconds: u32) -> u32 {
|
||||
let per_second = if resolution == "720p" { 20 } else { 10 };
|
||||
per_second * duration_seconds
|
||||
}
|
||||
|
||||
fn resolve_editor_character_animation_provider_ratio(
|
||||
ratio: &str,
|
||||
source_width: u32,
|
||||
@@ -3792,6 +4104,10 @@ fn character_animation_error_response(
|
||||
error.into_response_with_context(Some(request_context))
|
||||
}
|
||||
|
||||
fn editor_video_error_response(request_context: &RequestContext, error: AppError) -> Response {
|
||||
error.into_response_with_context(Some(request_context))
|
||||
}
|
||||
|
||||
pub(crate) struct MotionTemplate {
|
||||
pub(crate) id: &'static str,
|
||||
pub(crate) label: &'static str,
|
||||
@@ -3836,6 +4152,10 @@ struct EditorCharacterAnimationSettings {
|
||||
duration_seconds: u32,
|
||||
}
|
||||
|
||||
struct EditorVideoSettings {
|
||||
ark: ArkVideoSettings,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct NormalizedEditorCharacterAnimationRequest {
|
||||
source_layer_id: String,
|
||||
@@ -3851,6 +4171,19 @@ struct NormalizedEditorCharacterAnimationRequest {
|
||||
fps: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct NormalizedEditorVideoRequest {
|
||||
prompt: String,
|
||||
model: String,
|
||||
provider_model: String,
|
||||
aspect_ratio: String,
|
||||
resolution: String,
|
||||
duration_seconds: u32,
|
||||
price_mud_points: u32,
|
||||
width: u32,
|
||||
height: u32,
|
||||
}
|
||||
|
||||
struct GeneratedAnimationPreview {
|
||||
preview_video_path: String,
|
||||
upstream_task_id: String,
|
||||
@@ -4112,7 +4445,71 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_price_depends_on_resolution_and_duration() {
|
||||
assert_eq!(calculate_editor_character_animation_price("480p", 4), 40);
|
||||
assert_eq!(calculate_editor_character_animation_price("720p", 6), 120);
|
||||
assert_eq!(
|
||||
crate::editor_generation_config::editor_character_animation_mud_points("480p", 4),
|
||||
40
|
||||
);
|
||||
assert_eq!(
|
||||
crate::editor_generation_config::editor_character_animation_mud_points("720p", 6),
|
||||
120
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_normalizes_lovart_model_contract() {
|
||||
let normalized = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: " 让角色向镜头挥手 ".to_string(),
|
||||
model: EDITOR_VIDEO_MODEL_KLING_3_OMNI.to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
duration_seconds: 5,
|
||||
resolution: "480p".to_string(),
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
price_mud_points: 50,
|
||||
})
|
||||
.expect("editor video request should normalize");
|
||||
|
||||
assert_eq!(normalized.prompt, "让角色向镜头挥手");
|
||||
assert_eq!(normalized.model, EDITOR_VIDEO_MODEL_KLING_3_OMNI);
|
||||
assert_eq!(normalized.provider_model, EDITOR_VIDEO_MODEL_KLING_3_OMNI);
|
||||
assert_eq!(normalized.width, 854);
|
||||
assert_eq!(normalized.height, 480);
|
||||
assert_eq!(normalized.price_mud_points, 50);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_seedance_fast_uses_existing_fast_seedance_model() {
|
||||
let normalized = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: "快速生成镜头推进。".to_string(),
|
||||
model: EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST.to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
duration_seconds: 4,
|
||||
resolution: "720p".to_string(),
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
price_mud_points: 80,
|
||||
})
|
||||
.expect("seedance fast request should normalize");
|
||||
|
||||
assert_eq!(normalized.provider_model, CHARACTER_ANIMATION_MODEL);
|
||||
assert_eq!(normalized.width, 1280);
|
||||
assert_eq!(normalized.height, 720);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_rejects_price_mismatch() {
|
||||
let error = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: "生成镜头。".to_string(),
|
||||
model: EDITOR_VIDEO_MODEL_VEO_3_1.to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
duration_seconds: 5,
|
||||
resolution: "480p".to_string(),
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
price_mud_points: 40,
|
||||
})
|
||||
.expect_err("wrong price should fail");
|
||||
|
||||
assert!(error.body_text().contains("priceMudPoints"));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
/// 图片画布编辑器生成类能力的泥点配置。
|
||||
///
|
||||
/// 中文注释：先用 api-server 静态配置收口价格事实源，避免继续把价格散落在
|
||||
/// 前端常量和具体 handler 内；后续若接后台配置，可只替换本模块读取来源。
|
||||
const EDITOR_IMAGE_GENERATION_MUD_POINTS: u32 = 12;
|
||||
const EDITOR_SPEC_GENERATION_MUD_POINTS: u32 = 5;
|
||||
const EDITOR_CHARACTER_IMAGE_GENERATION_MUD_POINTS: u32 = 12;
|
||||
const EDITOR_ICON_SPRITESHEET_GENERATION_MUD_POINTS: u32 = 12;
|
||||
const EDITOR_UI_DESIGN_GENERATION_MUD_POINTS: u32 = 12;
|
||||
|
||||
pub(crate) const EDITOR_VIDEO_GENERATION_480P_MUD_POINTS_PER_SECOND: u32 = 10;
|
||||
pub(crate) const EDITOR_VIDEO_GENERATION_720P_MUD_POINTS_PER_SECOND: u32 = 20;
|
||||
|
||||
pub(crate) const EDITOR_CHARACTER_ANIMATION_480P_MUD_POINTS_PER_SECOND: u32 = 10;
|
||||
pub(crate) const EDITOR_CHARACTER_ANIMATION_720P_MUD_POINTS_PER_SECOND: u32 = 20;
|
||||
|
||||
pub(crate) fn editor_image_generation_mud_points(kind: Option<&str>) -> u32 {
|
||||
match kind.map(str::trim) {
|
||||
Some("spec") => EDITOR_SPEC_GENERATION_MUD_POINTS,
|
||||
Some("character") => EDITOR_CHARACTER_IMAGE_GENERATION_MUD_POINTS,
|
||||
Some("icon") => EDITOR_ICON_SPRITESHEET_GENERATION_MUD_POINTS,
|
||||
Some("ui-design") => EDITOR_UI_DESIGN_GENERATION_MUD_POINTS,
|
||||
_ => EDITOR_IMAGE_GENERATION_MUD_POINTS,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn editor_video_generation_mud_points(resolution: &str, duration_seconds: u32) -> u32 {
|
||||
let per_second = if resolution == "720p" {
|
||||
EDITOR_VIDEO_GENERATION_720P_MUD_POINTS_PER_SECOND
|
||||
} else {
|
||||
EDITOR_VIDEO_GENERATION_480P_MUD_POINTS_PER_SECOND
|
||||
};
|
||||
per_second * duration_seconds
|
||||
}
|
||||
|
||||
pub(crate) fn editor_character_animation_mud_points(
|
||||
resolution: &str,
|
||||
duration_seconds: u32,
|
||||
) -> u32 {
|
||||
let per_second = if resolution == "720p" {
|
||||
EDITOR_CHARACTER_ANIMATION_720P_MUD_POINTS_PER_SECOND
|
||||
} else {
|
||||
EDITOR_CHARACTER_ANIMATION_480P_MUD_POINTS_PER_SECOND
|
||||
};
|
||||
per_second * duration_seconds
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_price_uses_configured_resolution_rates() {
|
||||
assert_eq!(editor_character_animation_mud_points("480p", 4), 40);
|
||||
assert_eq!(editor_character_animation_mud_points("720p", 6), 120);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_generation_price_uses_configured_resolution_rates() {
|
||||
assert_eq!(editor_video_generation_mud_points("480p", 4), 40);
|
||||
assert_eq!(editor_video_generation_mud_points("720p", 5), 100);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_image_generation_price_uses_configured_kind_rates() {
|
||||
assert_eq!(editor_image_generation_mud_points(None), 12);
|
||||
assert_eq!(editor_image_generation_mud_points(Some("spec")), 5);
|
||||
assert_eq!(editor_image_generation_mud_points(Some("character")), 12);
|
||||
assert_eq!(editor_image_generation_mud_points(Some("icon")), 12);
|
||||
assert_eq!(editor_image_generation_mud_points(Some("ui-design")), 12);
|
||||
}
|
||||
}
|
||||
@@ -42,7 +42,8 @@ use crate::{
|
||||
openai_image_generation::{
|
||||
DownloadedOpenAiImage, GPT_IMAGE_2_MODEL, OpenAiReferenceImage,
|
||||
build_openai_image_http_client, create_openai_image_edit_with_references,
|
||||
create_openai_image_edit_with_references_and_model, create_openai_image_generation,
|
||||
create_openai_image_edit_with_references_and_model,
|
||||
create_openai_image_generation_with_model, create_openai_nanobanana_generate_content,
|
||||
require_openai_image_settings,
|
||||
},
|
||||
platform_errors::map_oss_error,
|
||||
@@ -57,9 +58,7 @@ const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-";
|
||||
const EDITOR_LAYOUT_MAX_BYTES: usize = 256 * 1024;
|
||||
const EDITOR_PROJECT_DEFAULT_TITLE: &str = "未命名画布";
|
||||
const EDITOR_IMAGE_GENERATION_SIZE: &str = "1024x1024";
|
||||
const EDITOR_ICON_SPRITESHEET_MODEL: &str = "gemini-3.1-flash-image-preview";
|
||||
const EDITOR_ICON_SPRITESHEET_SMALL_SIZE: &str = "512x512";
|
||||
const EDITOR_ICON_SPRITESHEET_LARGE_SIZE: &str = "1024x1024";
|
||||
const EDITOR_IMAGE_MODEL_NANOBANANA2: &str = "gemini-3.1-flash-image-preview";
|
||||
const EDITOR_ICON_DESCRIPTION_LIMIT: usize = 100;
|
||||
const EDITOR_CHARACTER_IMAGE_ASSET_KIND: &str = "editor_character_image";
|
||||
const EDITOR_CHARACTER_IMAGE_ENTITY_KIND: &str = "editor_project";
|
||||
@@ -155,6 +154,8 @@ pub struct EditorImageGenerationRequest {
|
||||
size: Option<String>,
|
||||
kind: Option<String>,
|
||||
model: Option<String>,
|
||||
aspect_ratio: Option<String>,
|
||||
image_size: Option<String>,
|
||||
reference_image_srcs: Option<Vec<String>>,
|
||||
}
|
||||
|
||||
@@ -171,6 +172,8 @@ pub struct EditorIconSpritesheetGenerationRequest {
|
||||
reference_image_src: String,
|
||||
icon_descriptions: Vec<String>,
|
||||
model: Option<String>,
|
||||
aspect_ratio: Option<String>,
|
||||
image_size: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
@@ -238,7 +241,7 @@ pub struct EditorImageGenerationResponse {
|
||||
source_type: &'static str,
|
||||
prompt: String,
|
||||
actual_prompt: Option<String>,
|
||||
model: &'static str,
|
||||
model: String,
|
||||
provider: &'static str,
|
||||
task_id: String,
|
||||
}
|
||||
@@ -757,12 +760,33 @@ pub async fn generate_editor_image(
|
||||
);
|
||||
}
|
||||
|
||||
let image_size = normalize_editor_image_generation_size(payload.size.as_deref());
|
||||
let _requested_model = payload.model.as_deref();
|
||||
let generation_options = normalize_editor_generation_options(
|
||||
payload.model.as_deref(),
|
||||
payload.aspect_ratio.as_deref(),
|
||||
payload.image_size.as_deref(),
|
||||
);
|
||||
let has_dimension_options = payload.aspect_ratio.is_some() || payload.image_size.is_some();
|
||||
let legacy_size = normalize_editor_image_generation_size(payload.size.as_deref());
|
||||
let image_size = if has_dimension_options {
|
||||
Cow::Owned(generation_options.size.clone())
|
||||
} else {
|
||||
legacy_size
|
||||
};
|
||||
let normalized_kind = payload.kind.as_deref().map(str::trim);
|
||||
let _configured_price_mud_points =
|
||||
crate::editor_generation_config::editor_image_generation_mud_points(normalized_kind);
|
||||
let is_character_generation = matches!(normalized_kind, Some("character"));
|
||||
let is_ui_design_generation = matches!(normalized_kind, Some("ui-design"));
|
||||
let submitted_prompt = if is_character_generation {
|
||||
build_editor_character_image_prompt(role_setting.as_str())
|
||||
} else if is_ui_design_generation {
|
||||
build_editor_ui_design_prompt(
|
||||
role_setting.as_str(),
|
||||
payload
|
||||
.reference_image_srcs
|
||||
.as_ref()
|
||||
.is_some_and(|references| references.iter().any(|source| !source.trim().is_empty())),
|
||||
)
|
||||
} else {
|
||||
role_setting.clone()
|
||||
};
|
||||
@@ -770,6 +794,7 @@ pub async fn generate_editor_image(
|
||||
Some("character") => "图片画布生成角色形象",
|
||||
Some("spec") => "图片画布生成规范",
|
||||
Some("quick-edit") => "图片画布快速编辑图片",
|
||||
Some("ui-design") => "图片画布生成UI设计图",
|
||||
_ => "图片画布生成图片",
|
||||
};
|
||||
let reference_sources = payload
|
||||
@@ -787,10 +812,46 @@ pub async fn generate_editor_image(
|
||||
);
|
||||
let http_client = build_openai_image_http_client(&settings)?;
|
||||
let negative_prompt = Some("文字、水印、边框、按钮、UI 控件、低清晰度、变形主体");
|
||||
let generated = if reference_sources.is_empty() {
|
||||
create_openai_image_generation(
|
||||
let reference_images = if reference_sources.is_empty() {
|
||||
Vec::new()
|
||||
} else {
|
||||
reference_sources
|
||||
.iter()
|
||||
.map(|source| parse_editor_reference_image(source.as_str()))
|
||||
.collect::<Result<Vec<_>, _>>()?
|
||||
};
|
||||
let generation_options = if is_ui_design_generation {
|
||||
normalize_editor_generation_options(
|
||||
Some(GPT_IMAGE_2_MODEL),
|
||||
payload.aspect_ratio.as_deref(),
|
||||
payload.image_size.as_deref(),
|
||||
)
|
||||
} else {
|
||||
generation_options
|
||||
};
|
||||
let image_size = if is_ui_design_generation {
|
||||
Cow::Owned(generation_options.size.clone())
|
||||
} else {
|
||||
image_size
|
||||
};
|
||||
let generated = if generation_options.model == EDITOR_IMAGE_MODEL_NANOBANANA2 {
|
||||
create_openai_nanobanana_generate_content(
|
||||
&http_client,
|
||||
&settings,
|
||||
generation_options.model,
|
||||
submitted_prompt.as_str(),
|
||||
negative_prompt,
|
||||
generation_options.aspect_ratio,
|
||||
generation_options.provider_image_size,
|
||||
reference_images.as_slice(),
|
||||
failure_context,
|
||||
)
|
||||
.await?
|
||||
} else if reference_images.is_empty() {
|
||||
create_openai_image_generation_with_model(
|
||||
&http_client,
|
||||
&settings,
|
||||
generation_options.model,
|
||||
submitted_prompt.as_str(),
|
||||
negative_prompt,
|
||||
image_size.as_ref(),
|
||||
@@ -800,13 +861,10 @@ pub async fn generate_editor_image(
|
||||
)
|
||||
.await?
|
||||
} else {
|
||||
let reference_images = reference_sources
|
||||
.iter()
|
||||
.map(|source| parse_editor_reference_image(source.as_str()))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
create_openai_image_edit_with_references(
|
||||
create_openai_image_edit_with_references_and_model(
|
||||
&http_client,
|
||||
&settings,
|
||||
generation_options.model,
|
||||
submitted_prompt.as_str(),
|
||||
negative_prompt,
|
||||
image_size.as_ref(),
|
||||
@@ -863,7 +921,7 @@ pub async fn generate_editor_image(
|
||||
source_type: "generated",
|
||||
prompt: role_setting,
|
||||
actual_prompt: generated.actual_prompt,
|
||||
model: GPT_IMAGE_2_MODEL,
|
||||
model: generation_options.model.to_string(),
|
||||
provider: "VectorEngine",
|
||||
task_id: generated.task_id,
|
||||
},
|
||||
@@ -895,6 +953,96 @@ fn is_editor_custom_image_size(value: &str) -> bool {
|
||||
(64..=4096).contains(&width) && (64..=4096).contains(&height)
|
||||
}
|
||||
|
||||
fn normalize_editor_generation_options(
|
||||
model: Option<&str>,
|
||||
aspect_ratio: Option<&str>,
|
||||
image_size: Option<&str>,
|
||||
) -> EditorGenerationOptions {
|
||||
let normalized_model = match model.map(str::trim).filter(|value| !value.is_empty()) {
|
||||
Some(GPT_IMAGE_2_MODEL) => GPT_IMAGE_2_MODEL,
|
||||
Some(EDITOR_IMAGE_MODEL_NANOBANANA2) => EDITOR_IMAGE_MODEL_NANOBANANA2,
|
||||
// 中文注释：未显式传模型的旧普通生成、快速编辑和生成规范继续走 gpt-image-2；
|
||||
// 角色 / 图标素材入口由前端显式传入 nanobanana2 默认值。
|
||||
None => GPT_IMAGE_2_MODEL,
|
||||
_ => EDITOR_IMAGE_MODEL_NANOBANANA2,
|
||||
};
|
||||
let aspect_ratio = normalize_editor_generation_aspect_ratio(aspect_ratio);
|
||||
let image_size = normalize_editor_generation_image_size(normalized_model, image_size);
|
||||
let size = editor_generation_size_for_model(normalized_model, aspect_ratio, image_size);
|
||||
let provider_image_size =
|
||||
editor_generation_provider_image_size_for_model(normalized_model, image_size);
|
||||
|
||||
EditorGenerationOptions {
|
||||
model: normalized_model,
|
||||
aspect_ratio,
|
||||
image_size,
|
||||
size,
|
||||
provider_image_size,
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_generation_aspect_ratio(aspect_ratio: Option<&str>) -> &'static str {
|
||||
match aspect_ratio
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
{
|
||||
Some("2:3") => "2:3",
|
||||
Some("3:2") => "3:2",
|
||||
Some("9:16") => "9:16",
|
||||
Some("16:9") => "16:9",
|
||||
_ => "1:1",
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_generation_image_size(model: &str, image_size: Option<&str>) -> &'static str {
|
||||
match (
|
||||
model,
|
||||
image_size.map(str::trim).filter(|value| !value.is_empty()),
|
||||
) {
|
||||
(EDITOR_IMAGE_MODEL_NANOBANANA2, Some("0.5K")) => "0.5K",
|
||||
(_, Some("2K")) => "2K",
|
||||
_ => "1K",
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_generation_size_for_model(model: &str, aspect_ratio: &str, image_size: &str) -> String {
|
||||
if model == EDITOR_IMAGE_MODEL_NANOBANANA2 {
|
||||
return match image_size {
|
||||
// 中文注释：VectorEngine 的 nanobanana2 文档要求 0.5K 传入 512。
|
||||
"0.5K" => "512",
|
||||
"2K" => "2048",
|
||||
_ => "1024",
|
||||
}
|
||||
.to_string();
|
||||
}
|
||||
|
||||
match (image_size, aspect_ratio) {
|
||||
("2K", "1:1") => "2048x2048",
|
||||
// 中文注释：gpt-image-2 文档未列出 2K 竖版，竖版选择回落到文档明确支持的 1K 竖版。
|
||||
("2K", "2:3") | ("2K", "9:16") => "1024x1536",
|
||||
("2K", "16:9") | ("2K", "3:2") => "2048x1152",
|
||||
("1K", "2:3") | ("1K", "9:16") => "1024x1536",
|
||||
("1K", "3:2") | ("1K", "16:9") => "1536x1024",
|
||||
_ => "1024x1024",
|
||||
}
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn editor_generation_provider_image_size_for_model(
|
||||
model: &str,
|
||||
image_size: &'static str,
|
||||
) -> &'static str {
|
||||
if model == EDITOR_IMAGE_MODEL_NANOBANANA2 {
|
||||
return match image_size {
|
||||
"0.5K" => "512",
|
||||
"2K" => "2K",
|
||||
_ => "1K",
|
||||
};
|
||||
}
|
||||
|
||||
image_size
|
||||
}
|
||||
|
||||
pub async fn edit_editor_image(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
@@ -955,7 +1103,7 @@ pub async fn edit_editor_image(
|
||||
source_type: "generated",
|
||||
prompt,
|
||||
actual_prompt: generated.actual_prompt,
|
||||
model: GPT_IMAGE_2_MODEL,
|
||||
model: GPT_IMAGE_2_MODEL.to_string(),
|
||||
provider: "VectorEngine",
|
||||
task_id: generated.task_id,
|
||||
},
|
||||
@@ -977,14 +1125,12 @@ pub async fn generate_editor_icon_spritesheet(
|
||||
"message": "图标素材规范必须是图片 Data URL。",
|
||||
}))
|
||||
})?;
|
||||
let model = payload
|
||||
.model
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.unwrap_or(EDITOR_ICON_SPRITESHEET_MODEL)
|
||||
.to_string();
|
||||
let size = editor_icon_spritesheet_size_for_count(icon_descriptions.len());
|
||||
let generation_options = normalize_editor_generation_options(
|
||||
payload.model.as_deref(),
|
||||
payload.aspect_ratio.as_deref(),
|
||||
payload.image_size.as_deref(),
|
||||
);
|
||||
let size = generation_options.size.as_str();
|
||||
let prompt = build_editor_icon_spritesheet_prompt(&icon_descriptions);
|
||||
|
||||
let settings = require_openai_image_settings(&state)?.with_external_api_audit_context(
|
||||
@@ -993,18 +1139,33 @@ pub async fn generate_editor_icon_spritesheet(
|
||||
None,
|
||||
);
|
||||
let http_client = build_openai_image_http_client(&settings)?;
|
||||
let generated = create_openai_image_edit_with_references_and_model(
|
||||
&http_client,
|
||||
&settings,
|
||||
model.as_str(),
|
||||
prompt.as_str(),
|
||||
None,
|
||||
size,
|
||||
1,
|
||||
&[reference_image],
|
||||
"图片画布生成图标素材 spritesheet",
|
||||
)
|
||||
.await?;
|
||||
let generated = if generation_options.model == EDITOR_IMAGE_MODEL_NANOBANANA2 {
|
||||
create_openai_nanobanana_generate_content(
|
||||
&http_client,
|
||||
&settings,
|
||||
generation_options.model,
|
||||
prompt.as_str(),
|
||||
None,
|
||||
generation_options.aspect_ratio,
|
||||
generation_options.provider_image_size,
|
||||
&[reference_image],
|
||||
"图片画布生成图标素材 spritesheet",
|
||||
)
|
||||
.await?
|
||||
} else {
|
||||
create_openai_image_edit_with_references_and_model(
|
||||
&http_client,
|
||||
&settings,
|
||||
generation_options.model,
|
||||
prompt.as_str(),
|
||||
None,
|
||||
size,
|
||||
1,
|
||||
&[reference_image],
|
||||
"图片画布生成图标素材 spritesheet",
|
||||
)
|
||||
.await?
|
||||
};
|
||||
let image = generated.images.into_iter().next().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "vector-engine",
|
||||
@@ -1045,7 +1206,7 @@ pub async fn generate_editor_icon_spritesheet(
|
||||
icon_image_srcs,
|
||||
prompt,
|
||||
actual_prompt: generated.actual_prompt,
|
||||
model,
|
||||
model: generation_options.model.to_string(),
|
||||
provider: "VectorEngine",
|
||||
task_id: generated.task_id,
|
||||
},
|
||||
@@ -1249,6 +1410,17 @@ fn build_editor_icon_spritesheet_prompt(icon_descriptions: &[String]) -> String
|
||||
)
|
||||
}
|
||||
|
||||
fn build_editor_ui_design_prompt(user_input: &str, has_icon_spec_reference: bool) -> String {
|
||||
let mut prompt = vec![
|
||||
"生成玩法UI原型图".to_string(),
|
||||
format!("【用户输入】{}", user_input.trim()),
|
||||
];
|
||||
if has_icon_spec_reference {
|
||||
prompt.push("参考图1为图标素材规范，请在UI图标、按钮符号、描边、材质、圆角、阴影和状态层级上严格遵循参考图1的素材规范。".to_string());
|
||||
}
|
||||
prompt.join("\n")
|
||||
}
|
||||
|
||||
fn build_editor_character_image_prompt(role_setting: &str) -> String {
|
||||
vec![
|
||||
"严格基于图1的角色美术视觉规范指导中的美术风格、角色头身比、角色朝向等特征生成游戏角色形象图。画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止生成美术视觉规范、出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。".to_string(),
|
||||
@@ -1295,14 +1467,6 @@ fn prepare_editor_character_image_for_response(
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_icon_spritesheet_size_for_count(icon_count: usize) -> &'static str {
|
||||
if icon_count <= 25 {
|
||||
EDITOR_ICON_SPRITESHEET_SMALL_SIZE
|
||||
} else {
|
||||
EDITOR_ICON_SPRITESHEET_LARGE_SIZE
|
||||
}
|
||||
}
|
||||
|
||||
fn data_url_from_image_bytes(mime_type: &str, bytes: &[u8]) -> String {
|
||||
format!(
|
||||
"data:{};base64,{}",
|
||||
@@ -1322,6 +1486,15 @@ fn editor_icon_response_from_slice(
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
struct EditorGenerationOptions {
|
||||
model: &'static str,
|
||||
aspect_ratio: &'static str,
|
||||
image_size: &'static str,
|
||||
size: String,
|
||||
provider_image_size: &'static str,
|
||||
}
|
||||
|
||||
struct PersistedEditorGeneratedImage {
|
||||
object_key: String,
|
||||
asset_object_id: String,
|
||||
@@ -1548,6 +1721,68 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_generation_dimensions_follow_model_options() {
|
||||
let default_generation = normalize_editor_generation_options(None, Some("1:1"), Some("1K"));
|
||||
assert_eq!(default_generation.model, GPT_IMAGE_2_MODEL);
|
||||
assert_eq!(default_generation.size, "1024x1024");
|
||||
|
||||
let nanobanana = normalize_editor_generation_options(
|
||||
Some(EDITOR_IMAGE_MODEL_NANOBANANA2),
|
||||
Some("1:1"),
|
||||
Some("0.5K"),
|
||||
);
|
||||
assert_eq!(nanobanana.model, EDITOR_IMAGE_MODEL_NANOBANANA2);
|
||||
assert_eq!(nanobanana.size, "512");
|
||||
assert_eq!(nanobanana.aspect_ratio, "1:1");
|
||||
assert_eq!(nanobanana.image_size, "0.5K");
|
||||
assert_eq!(nanobanana.provider_image_size, "512");
|
||||
|
||||
let gpt = normalize_editor_generation_options(Some("gpt-image-2"), Some("2:3"), Some("1K"));
|
||||
assert_eq!(gpt.model, GPT_IMAGE_2_MODEL);
|
||||
assert_eq!(gpt.size, "1024x1536");
|
||||
assert_eq!(gpt.aspect_ratio, "2:3");
|
||||
assert_eq!(gpt.image_size, "1K");
|
||||
assert_eq!(gpt.provider_image_size, "1K");
|
||||
|
||||
let gpt_landscape_2k =
|
||||
normalize_editor_generation_options(Some("gpt-image-2"), Some("16:9"), Some("2K"));
|
||||
assert_eq!(gpt_landscape_2k.model, GPT_IMAGE_2_MODEL);
|
||||
assert_eq!(gpt_landscape_2k.size, "2048x1152");
|
||||
assert_eq!(gpt_landscape_2k.aspect_ratio, "16:9");
|
||||
assert_eq!(gpt_landscape_2k.image_size, "2K");
|
||||
|
||||
let gpt_portrait_2k_fallback =
|
||||
normalize_editor_generation_options(Some("gpt-image-2"), Some("9:16"), Some("2K"));
|
||||
assert_eq!(gpt_portrait_2k_fallback.model, GPT_IMAGE_2_MODEL);
|
||||
assert_eq!(gpt_portrait_2k_fallback.size, "1024x1536");
|
||||
assert_eq!(gpt_portrait_2k_fallback.aspect_ratio, "9:16");
|
||||
assert_eq!(gpt_portrait_2k_fallback.image_size, "2K");
|
||||
|
||||
let fallback = normalize_editor_generation_options(
|
||||
Some("unknown-model"),
|
||||
Some("bad-ratio"),
|
||||
Some("bad-size"),
|
||||
);
|
||||
assert_eq!(fallback.model, EDITOR_IMAGE_MODEL_NANOBANANA2);
|
||||
assert_eq!(fallback.size, "1024");
|
||||
assert_eq!(fallback.aspect_ratio, "1:1");
|
||||
assert_eq!(fallback.image_size, "1K");
|
||||
assert_eq!(fallback.provider_image_size, "1K");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_ui_design_prompt_uses_fixed_user_input_block_and_optional_icon_spec() {
|
||||
let prompt = build_editor_ui_design_prompt("二消玩法，主界面和结算弹窗", true);
|
||||
|
||||
assert!(prompt.contains("生成玩法UI原型图"));
|
||||
assert!(prompt.contains("【用户输入】二消玩法，主界面和结算弹窗"));
|
||||
assert!(prompt.contains("参考图1为图标素材规范"));
|
||||
|
||||
let no_reference_prompt = build_editor_ui_design_prompt("只要战斗HUD", false);
|
||||
assert!(!no_reference_prompt.contains("参考图1为图标素材规范"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_image_prompt_appends_user_role_setting() {
|
||||
let prompt = build_editor_character_image_prompt("菜市场卖菜大妈");
|
||||
@@ -1631,8 +1866,6 @@ mod tests {
|
||||
assert!(prompt.contains("参考图1的图标素材规范"));
|
||||
assert!(prompt.contains("纯绿幕背景方便扣除背景"));
|
||||
assert!(prompt.contains("返回按钮、设置按钮、下一关按钮"));
|
||||
assert_eq!(editor_icon_spritesheet_size_for_count(25), "512x512");
|
||||
assert_eq!(editor_icon_spritesheet_size_for_count(26), "1024x1024");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -36,6 +36,7 @@ mod custom_world_asset_prompts;
|
||||
mod custom_world_foundation_draft;
|
||||
mod custom_world_result_prompts;
|
||||
mod custom_world_rpg_draft_prompts;
|
||||
mod editor_generation_config;
|
||||
mod editor_project;
|
||||
mod edutainment_baby_drawing;
|
||||
mod edutainment_baby_object;
|
||||
|
||||
@@ -16,7 +16,7 @@ use crate::{
|
||||
assets::get_asset_history,
|
||||
auth::require_bearer_auth,
|
||||
character_animation_assets::{
|
||||
generate_character_animation, generate_editor_character_animation,
|
||||
generate_character_animation, generate_editor_character_animation, generate_editor_video,
|
||||
get_character_animation_job, get_character_workflow_cache, import_character_animation_video,
|
||||
list_character_animation_templates,
|
||||
publish_character_animation, put_role_asset_workflow, resolve_role_asset_workflow,
|
||||
@@ -461,6 +461,13 @@ fn play_flow_support_router(state: AppState) -> Router<AppState> {
|
||||
EDITOR_CHARACTER_ANIMATION_BODY_LIMIT_BYTES,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/videos/generations",
|
||||
post(generate_editor_video).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/assets/character-animation/jobs/{task_id}",
|
||||
get(get_character_animation_job),
|
||||
|
||||
@@ -3,7 +3,9 @@ use platform_image::{
|
||||
DownloadedImage, GeneratedImages, PlatformImageError, PlatformImageStatusHint, ReferenceImage,
|
||||
VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings, build_vector_engine_image_http_client,
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_edit_with_references_and_model, create_vector_engine_image_generation,
|
||||
create_vector_engine_image_edit_with_references_and_model,
|
||||
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
|
||||
create_vector_engine_nanobanana_generate_content,
|
||||
};
|
||||
#[cfg(test)]
|
||||
use platform_image::{
|
||||
@@ -159,6 +161,94 @@ pub(crate) async fn create_openai_image_generation(
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) async fn create_openai_image_generation_with_model(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &OpenAiImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
reference_images: &[String],
|
||||
failure_context: &str,
|
||||
) -> Result<OpenAiGeneratedImages, AppError> {
|
||||
let started_at_micros = current_utc_micros();
|
||||
let request_payload = json!({
|
||||
"model": model,
|
||||
"size": size,
|
||||
"candidateCount": candidate_count,
|
||||
"promptChars": prompt.chars().count(),
|
||||
"negativePromptChars": negative_prompt.map(str::chars).map(Iterator::count),
|
||||
"referenceImageCount": reference_images.len(),
|
||||
});
|
||||
let result = create_vector_engine_image_generation_with_model(
|
||||
http_client,
|
||||
&settings.provider_settings(),
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
candidate_count,
|
||||
reference_images,
|
||||
failure_context,
|
||||
)
|
||||
.await;
|
||||
map_platform_image_result(
|
||||
settings,
|
||||
result,
|
||||
"image_generation",
|
||||
failure_context,
|
||||
request_payload,
|
||||
started_at_micros,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) async fn create_openai_nanobanana_generate_content(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &OpenAiImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
aspect_ratio: &str,
|
||||
image_size: &str,
|
||||
reference_images: &[OpenAiReferenceImage],
|
||||
failure_context: &str,
|
||||
) -> Result<OpenAiGeneratedImages, AppError> {
|
||||
let started_at_micros = current_utc_micros();
|
||||
let request_payload = json!({
|
||||
"model": model,
|
||||
"aspectRatio": aspect_ratio,
|
||||
"imageSize": image_size,
|
||||
"promptChars": prompt.chars().count(),
|
||||
"negativePromptChars": negative_prompt.map(str::chars).map(Iterator::count),
|
||||
"referenceImageCount": reference_images.len(),
|
||||
});
|
||||
let result = create_vector_engine_nanobanana_generate_content(
|
||||
http_client,
|
||||
&settings.provider_settings(),
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
aspect_ratio,
|
||||
image_size,
|
||||
reference_images,
|
||||
failure_context,
|
||||
)
|
||||
.await;
|
||||
map_platform_image_result(
|
||||
settings,
|
||||
result,
|
||||
"nanobanana_generate_content",
|
||||
failure_context,
|
||||
request_payload,
|
||||
started_at_micros,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
pub(crate) async fn create_openai_image_edit(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &OpenAiImageSettings,
|
||||
|
||||
@@ -7,8 +7,11 @@ pub use vector_engine::{
|
||||
PlatformImageFailureAudit, PlatformImageStatusHint, ReferenceImage,
|
||||
VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
|
||||
build_vector_engine_image_http_client, build_vector_engine_image_request_body,
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
build_vector_engine_nanobanana_generate_content_request_body, create_vector_engine_image_edit,
|
||||
create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_edit_with_references_and_model,
|
||||
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
|
||||
download_remote_image, vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
create_vector_engine_nanobanana_generate_content, download_remote_image,
|
||||
vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
vector_engine_nanobanana_generate_content_url,
|
||||
};
|
||||
|
||||
@@ -14,9 +14,10 @@ use super::{
|
||||
image_source::resolve_reference_images,
|
||||
request::{
|
||||
build_vector_engine_image_edit_request_log_params,
|
||||
build_vector_engine_image_request_body_with_model, normalize_image_size,
|
||||
build_vector_engine_image_request_body_with_model,
|
||||
build_vector_engine_nanobanana_generate_content_request_body, normalize_image_size,
|
||||
normalize_vector_engine_image_model, vector_engine_images_edit_url,
|
||||
vector_engine_images_generation_url,
|
||||
vector_engine_images_generation_url, vector_engine_nanobanana_generate_content_url,
|
||||
},
|
||||
response::handle_vector_engine_response,
|
||||
types::{GeneratedImages, ReferenceImage, VectorEngineImageSettings},
|
||||
@@ -181,6 +182,144 @@ pub async fn create_vector_engine_image_generation_with_model(
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn create_vector_engine_nanobanana_generate_content(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &VectorEngineImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
aspect_ratio: &str,
|
||||
image_size: &str,
|
||||
reference_images: &[ReferenceImage],
|
||||
failure_context: &str,
|
||||
) -> Result<GeneratedImages, PlatformImageError> {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
let request_url = vector_engine_nanobanana_generate_content_url(settings, model);
|
||||
let request_body = build_vector_engine_nanobanana_generate_content_request_body(
|
||||
prompt,
|
||||
negative_prompt,
|
||||
aspect_ratio,
|
||||
image_size,
|
||||
reference_images,
|
||||
);
|
||||
let reference_image_count = reference_images.iter().take(14).count();
|
||||
let reference_image_bytes_total: usize = reference_images
|
||||
.iter()
|
||||
.take(14)
|
||||
.map(|image| image.bytes.len())
|
||||
.sum();
|
||||
let request_params = serde_json::json!({
|
||||
"model": model,
|
||||
"promptChars": prompt.trim().chars().count(),
|
||||
"negativePromptChars": negative_prompt
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.map(str::chars)
|
||||
.map(Iterator::count)
|
||||
.unwrap_or_default(),
|
||||
"aspectRatio": aspect_ratio,
|
||||
"imageSize": image_size,
|
||||
"referenceImageCount": reference_image_count,
|
||||
"referenceImageBytesTotal": reference_image_bytes_total,
|
||||
});
|
||||
let started_at = std::time::Instant::now();
|
||||
let mut attempt = 1;
|
||||
let response = loop {
|
||||
match send_vector_engine_json_request_with_curl(
|
||||
request_url.as_str(),
|
||||
settings.api_key.as_str(),
|
||||
&request_body,
|
||||
settings.request_timeout_ms,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(response) => {
|
||||
if should_retry_vector_engine_upstream_status(response.status, attempt) {
|
||||
retry_vector_engine_upstream_status_after_delay(
|
||||
"nanobanana_generate_content",
|
||||
request_url.as_str(),
|
||||
attempt,
|
||||
response.status,
|
||||
response.body.as_str(),
|
||||
started_at.elapsed().as_millis() as u64,
|
||||
Some(prompt.chars().count()),
|
||||
Some(reference_image_count),
|
||||
Some(&request_params),
|
||||
)
|
||||
.await;
|
||||
attempt += 1;
|
||||
continue;
|
||||
}
|
||||
break response;
|
||||
}
|
||||
Err(error) => {
|
||||
if should_retry_vector_engine_curl_send_error(&error, attempt) {
|
||||
retry_vector_engine_send_after_delay(
|
||||
"nanobanana_generate_content",
|
||||
request_url.as_str(),
|
||||
"request_send",
|
||||
attempt,
|
||||
error.is_timeout(),
|
||||
error.is_connect() || error.is_transient_transport(),
|
||||
true,
|
||||
false,
|
||||
error.to_string().as_str(),
|
||||
started_at.elapsed().as_millis() as u64,
|
||||
Some(prompt.chars().count()),
|
||||
Some(reference_image_count),
|
||||
Some(&request_params),
|
||||
)
|
||||
.await;
|
||||
attempt += 1;
|
||||
continue;
|
||||
}
|
||||
return Err(map_curl_error(
|
||||
format!("{failure_context}：创建 nanobanana2 图片生成任务失败").as_str(),
|
||||
request_url.as_str(),
|
||||
"request_send",
|
||||
error,
|
||||
started_at.elapsed().as_millis() as u64,
|
||||
Some(prompt.chars().count()),
|
||||
Some(reference_image_count),
|
||||
Some(&request_params),
|
||||
));
|
||||
}
|
||||
}
|
||||
};
|
||||
let response_status = response.status;
|
||||
tracing::info!(
|
||||
provider = VECTOR_ENGINE_PROVIDER,
|
||||
endpoint = %request_url,
|
||||
status = response_status,
|
||||
image_model = model,
|
||||
prompt_chars = prompt.chars().count(),
|
||||
aspect_ratio,
|
||||
image_size,
|
||||
reference_image_count,
|
||||
reference_image_bytes_total,
|
||||
request_params = %request_params,
|
||||
attempt,
|
||||
elapsed_ms = started_at.elapsed().as_millis() as u64,
|
||||
failure_context,
|
||||
"VectorEngine nanobanana2 图片生成 HTTP 返回"
|
||||
);
|
||||
let response_text = response.body;
|
||||
handle_vector_engine_response(
|
||||
http_client,
|
||||
request_url.as_str(),
|
||||
response_status,
|
||||
response_text.as_str(),
|
||||
failure_context,
|
||||
started_at.elapsed().as_millis() as u64,
|
||||
Some(prompt.chars().count()),
|
||||
Some(reference_image_count),
|
||||
1,
|
||||
"vector-engine-nanobanana",
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
pub async fn create_vector_engine_image_edit(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &VectorEngineImageSettings,
|
||||
|
||||
@@ -16,13 +16,16 @@ pub use client::{
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_edit_with_references_and_model,
|
||||
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
|
||||
create_vector_engine_nanobanana_generate_content,
|
||||
};
|
||||
pub use constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER};
|
||||
pub use error::{PlatformImageError, PlatformImageStatusHint};
|
||||
pub use image_source::download_remote_image;
|
||||
pub use request::{
|
||||
build_vector_engine_image_request_body, build_vector_engine_image_request_body_with_model,
|
||||
normalize_image_size, vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
build_vector_engine_nanobanana_generate_content_request_body, normalize_image_size,
|
||||
vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
vector_engine_nanobanana_generate_content_url,
|
||||
};
|
||||
pub use transport::build_vector_engine_image_http_client;
|
||||
pub use types::{DownloadedImage, GeneratedImages, ReferenceImage, VectorEngineImageSettings};
|
||||
|
||||
@@ -88,9 +88,48 @@ pub(super) fn extract_image_urls(payload: &Value) -> Vec<String> {
|
||||
pub(super) fn extract_b64_images(payload: &Value) -> Vec<String> {
|
||||
let mut values = Vec::new();
|
||||
collect_strings_by_key(payload, "b64_json", &mut values);
|
||||
collect_inline_image_data(payload, &mut values);
|
||||
values
|
||||
}
|
||||
|
||||
fn collect_inline_image_data(value: &Value, results: &mut Vec<String>) {
|
||||
match value {
|
||||
Value::Array(entries) => {
|
||||
for entry in entries {
|
||||
collect_inline_image_data(entry, results);
|
||||
}
|
||||
}
|
||||
Value::Object(object) => {
|
||||
for key in ["inlineData", "inline_data"] {
|
||||
if let Some(Value::Object(inline_data)) = object.get(key) {
|
||||
let mime_type = inline_data
|
||||
.get("mimeType")
|
||||
.or_else(|| inline_data.get("mime_type"))
|
||||
.and_then(Value::as_str)
|
||||
.map(str::trim)
|
||||
.unwrap_or("image/png")
|
||||
.to_ascii_lowercase();
|
||||
if !mime_type.is_empty() && !mime_type.starts_with("image/") {
|
||||
continue;
|
||||
}
|
||||
if let Some(data) = inline_data
|
||||
.get("data")
|
||||
.and_then(Value::as_str)
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
{
|
||||
results.push(data.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
for nested_value in object.values() {
|
||||
collect_inline_image_data(nested_value, results);
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn parse_api_error_message(raw_text: &str, fallback_message: &str) -> String {
|
||||
if raw_text.trim().is_empty() {
|
||||
return fallback_message.to_string();
|
||||
|
||||
@@ -32,10 +32,7 @@ pub fn build_vector_engine_image_request_body_with_model(
|
||||
) -> Value {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
let body = Map::from_iter([
|
||||
(
|
||||
"model".to_string(),
|
||||
Value::String(model.to_string()),
|
||||
),
|
||||
("model".to_string(), Value::String(model.to_string())),
|
||||
(
|
||||
"prompt".to_string(),
|
||||
Value::String(build_prompt_with_negative(prompt, negative_prompt)),
|
||||
@@ -50,6 +47,42 @@ pub fn build_vector_engine_image_request_body_with_model(
|
||||
Value::Object(body)
|
||||
}
|
||||
|
||||
pub fn build_vector_engine_nanobanana_generate_content_request_body(
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
aspect_ratio: &str,
|
||||
image_size: &str,
|
||||
reference_images: &[ReferenceImage],
|
||||
) -> Value {
|
||||
let prompt = build_prompt_with_negative(prompt, negative_prompt);
|
||||
let mut parts = vec![json!({ "text": prompt })];
|
||||
for reference_image in reference_images.iter().take(14) {
|
||||
parts.push(json!({
|
||||
"inline_data": {
|
||||
"mime_type": reference_image.mime_type,
|
||||
"data": base64::Engine::encode(
|
||||
&base64::engine::general_purpose::STANDARD,
|
||||
reference_image.bytes.as_slice()
|
||||
),
|
||||
},
|
||||
}));
|
||||
}
|
||||
|
||||
json!({
|
||||
"contents": [{
|
||||
"role": "user",
|
||||
"parts": parts,
|
||||
}],
|
||||
"generationConfig": {
|
||||
"responseModalities": ["IMAGE"],
|
||||
"imageConfig": {
|
||||
"aspectRatio": normalize_nanobanana_aspect_ratio(aspect_ratio),
|
||||
"imageSize": normalize_nanobanana_image_size(image_size),
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
pub fn normalize_vector_engine_image_model(model: &str) -> &str {
|
||||
match model.trim() {
|
||||
"" => GPT_IMAGE_2_MODEL,
|
||||
@@ -71,6 +104,31 @@ pub fn normalize_image_size(size: &str) -> String {
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn normalize_nanobanana_aspect_ratio(aspect_ratio: &str) -> &str {
|
||||
match aspect_ratio.trim() {
|
||||
"2:3" => "2:3",
|
||||
"3:2" => "3:2",
|
||||
"3:4" => "3:4",
|
||||
"4:3" => "4:3",
|
||||
"4:5" => "4:5",
|
||||
"5:4" => "5:4",
|
||||
"9:16" => "9:16",
|
||||
"16:9" => "16:9",
|
||||
"21:9" => "21:9",
|
||||
_ => "1:1",
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_nanobanana_image_size(image_size: &str) -> &str {
|
||||
match image_size.trim() {
|
||||
// 中文注释：nanobanana / Gemini 3.1 的 0.5K 在 VectorEngine 文档中要求传 512。
|
||||
"512" | "0.5K" => "512",
|
||||
"2K" => "2K",
|
||||
"4K" => "4K",
|
||||
_ => "1K",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn vector_engine_images_generation_url(settings: &VectorEngineImageSettings) -> String {
|
||||
if settings.base_url.ends_with("/v1") {
|
||||
format!("{}/images/generations", settings.base_url)
|
||||
@@ -87,6 +145,21 @@ pub fn vector_engine_images_edit_url(settings: &VectorEngineImageSettings) -> St
|
||||
}
|
||||
}
|
||||
|
||||
pub fn vector_engine_nanobanana_generate_content_url(
|
||||
settings: &VectorEngineImageSettings,
|
||||
model: &str,
|
||||
) -> String {
|
||||
let base_url = settings
|
||||
.base_url
|
||||
.trim_end_matches("/v1")
|
||||
.trim_end_matches('/');
|
||||
format!(
|
||||
"{}/v1beta/models/{}:generateContent",
|
||||
base_url,
|
||||
normalize_vector_engine_image_model(model)
|
||||
)
|
||||
}
|
||||
|
||||
pub(crate) fn build_vector_engine_image_edit_request_log_params(
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
|
||||
@@ -66,8 +66,10 @@ mod tests {
|
||||
assert_eq!(reference.mime_type, "image/png");
|
||||
assert_eq!(reference.bytes, b"\x89PNG\r\n\x1A\nrest");
|
||||
|
||||
let image = decode_generated_image_base64(BASE64_STANDARD.encode(b"\x89PNG\r\n\x1A\nrest").as_str())
|
||||
.expect("base64 image should decode");
|
||||
let image = decode_generated_image_base64(
|
||||
BASE64_STANDARD.encode(b"\x89PNG\r\n\x1A\nrest").as_str(),
|
||||
)
|
||||
.expect("base64 image should decode");
|
||||
assert_eq!(image.extension, "png");
|
||||
assert_eq!(image.mime_type, "image/png");
|
||||
assert_eq!(image.bytes, b"\x89PNG\r\n\x1A\nrest");
|
||||
@@ -121,10 +123,22 @@ mod tests {
|
||||
audit: Some(audit.clone()),
|
||||
};
|
||||
|
||||
assert_eq!(invalid_config.status_hint(), PlatformImageStatusHint::ServiceUnavailable);
|
||||
assert_eq!(invalid_request.status_hint(), PlatformImageStatusHint::BadRequest);
|
||||
assert_eq!(request_error.status_hint(), PlatformImageStatusHint::GatewayTimeout);
|
||||
assert_eq!(upstream_timeout.status_hint(), PlatformImageStatusHint::GatewayTimeout);
|
||||
assert_eq!(
|
||||
invalid_config.status_hint(),
|
||||
PlatformImageStatusHint::ServiceUnavailable
|
||||
);
|
||||
assert_eq!(
|
||||
invalid_request.status_hint(),
|
||||
PlatformImageStatusHint::BadRequest
|
||||
);
|
||||
assert_eq!(
|
||||
request_error.status_hint(),
|
||||
PlatformImageStatusHint::GatewayTimeout
|
||||
);
|
||||
assert_eq!(
|
||||
upstream_timeout.status_hint(),
|
||||
PlatformImageStatusHint::GatewayTimeout
|
||||
);
|
||||
assert_eq!(
|
||||
PlatformImageError::MissingImage {
|
||||
provider: VECTOR_ENGINE_PROVIDER,
|
||||
@@ -137,7 +151,10 @@ mod tests {
|
||||
|
||||
let audit_ref = upstream_timeout.audit().expect("audit should be preserved");
|
||||
assert_eq!(audit_ref.provider, VECTOR_ENGINE_PROVIDER);
|
||||
assert_eq!(audit_ref.endpoint, "https://vector.example/v1/images/generations");
|
||||
assert_eq!(
|
||||
audit_ref.endpoint,
|
||||
"https://vector.example/v1/images/generations"
|
||||
);
|
||||
assert_eq!(audit_ref.status_code, Some(504));
|
||||
assert_eq!(audit_ref.status_class, Some("5xx"));
|
||||
assert!(audit_ref.timeout);
|
||||
@@ -158,7 +175,27 @@ mod tests {
|
||||
{"url": "https://example.com/b.png"}
|
||||
],
|
||||
"nested": {
|
||||
"b64_json": ["YWJj", "ZGVm"]
|
||||
"b64_json": ["YWJj", "ZGVm"],
|
||||
"parts": [
|
||||
{
|
||||
"inlineData": {
|
||||
"mimeType": "image/png",
|
||||
"data": "aW1hZ2UtMQ=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"inline_data": {
|
||||
"mime_type": "image/jpeg",
|
||||
"data": "aW1hZ2UtMg=="
|
||||
}
|
||||
},
|
||||
{
|
||||
"inlineData": {
|
||||
"mimeType": "text/plain",
|
||||
"data": "bm90LWltYWdl"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
});
|
||||
|
||||
@@ -171,7 +208,12 @@ mod tests {
|
||||
);
|
||||
assert_eq!(
|
||||
extract_b64_images(&payload),
|
||||
vec!["YWJj".to_string(), "ZGVm".to_string()]
|
||||
vec![
|
||||
"YWJj".to_string(),
|
||||
"ZGVm".to_string(),
|
||||
"aW1hZ2UtMQ==".to_string(),
|
||||
"aW1hZ2UtMg==".to_string(),
|
||||
]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
use platform_image::vector_engine::{
|
||||
GPT_IMAGE_2_MODEL, ReferenceImage, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
|
||||
build_vector_engine_image_http_client, build_vector_engine_image_request_body,
|
||||
build_vector_engine_image_request_body_with_model, create_vector_engine_image_edit,
|
||||
create_vector_engine_image_generation,
|
||||
build_vector_engine_image_request_body_with_model,
|
||||
build_vector_engine_nanobanana_generate_content_request_body, create_vector_engine_image_edit,
|
||||
create_vector_engine_image_generation, create_vector_engine_nanobanana_generate_content,
|
||||
vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
vector_engine_nanobanana_generate_content_url,
|
||||
};
|
||||
use std::{
|
||||
sync::{
|
||||
@@ -69,6 +71,60 @@ fn vector_engine_request_body_can_use_nanobanana2_model() {
|
||||
assert_eq!(body["n"], 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vector_engine_request_body_can_use_nanobanana2_half_k() {
|
||||
let body = build_vector_engine_image_request_body_with_model(
|
||||
"gemini-3.1-flash-image-preview",
|
||||
"生成图标 spritesheet",
|
||||
None,
|
||||
"512",
|
||||
1,
|
||||
&[],
|
||||
);
|
||||
|
||||
assert_eq!(body["model"], "gemini-3.1-flash-image-preview");
|
||||
assert_eq!(body["size"], "512");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nanobanana_generate_content_body_carries_aspect_ratio_and_image_size() {
|
||||
let body = build_vector_engine_nanobanana_generate_content_request_body(
|
||||
"生成角色图",
|
||||
Some("文字、水印"),
|
||||
"2:3",
|
||||
"512",
|
||||
&[],
|
||||
);
|
||||
|
||||
assert_eq!(body["contents"][0]["role"], "user");
|
||||
assert_eq!(
|
||||
body["contents"][0]["parts"][0]["text"],
|
||||
"生成角色图\n避免：文字、水印"
|
||||
);
|
||||
assert_eq!(body["generationConfig"]["responseModalities"][0], "IMAGE");
|
||||
assert_eq!(
|
||||
body["generationConfig"]["imageConfig"]["aspectRatio"],
|
||||
"2:3"
|
||||
);
|
||||
assert_eq!(body["generationConfig"]["imageConfig"]["imageSize"], "512");
|
||||
assert!(body.get("model").is_none());
|
||||
assert!(body.get("n").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nanobanana_generate_content_url_uses_model_path() {
|
||||
let settings = VectorEngineImageSettings {
|
||||
base_url: "https://vector.example/v1".to_string(),
|
||||
api_key: "test-key".to_string(),
|
||||
request_timeout_ms: 1_000,
|
||||
};
|
||||
|
||||
assert_eq!(
|
||||
vector_engine_nanobanana_generate_content_url(&settings, "gemini-3.1-flash-image-preview"),
|
||||
"https://vector.example/v1beta/models/gemini-3.1-flash-image-preview:generateContent"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn vector_engine_image_edit_retries_send_timeout_once_and_succeeds() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0")
|
||||
@@ -136,6 +192,73 @@ async fn vector_engine_image_edit_retries_send_timeout_once_and_succeeds() {
|
||||
server.abort();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn nanobanana_generate_content_posts_native_body_and_reads_inline_data() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0")
|
||||
.await
|
||||
.expect("mock server should bind");
|
||||
let server_addr = listener
|
||||
.local_addr()
|
||||
.expect("mock server address should be readable");
|
||||
let server = tokio::spawn(async move {
|
||||
let Ok((mut stream, _)) = listener.accept().await else {
|
||||
return;
|
||||
};
|
||||
let mut request = Vec::new();
|
||||
let mut buffer = [0_u8; 4096];
|
||||
loop {
|
||||
let Ok(read) = stream.read(&mut buffer).await else {
|
||||
return;
|
||||
};
|
||||
if read == 0 {
|
||||
return;
|
||||
}
|
||||
request.extend_from_slice(&buffer[..read]);
|
||||
if request.windows(4).any(|window| window == b"\r\n\r\n") {
|
||||
break;
|
||||
}
|
||||
}
|
||||
let request_text = String::from_utf8_lossy(request.as_slice());
|
||||
assert!(
|
||||
request_text.contains("/v1beta/models/gemini-3.1-flash-image-preview:generateContent")
|
||||
);
|
||||
assert!(request_text.contains("\"aspectRatio\":\"2:3\""));
|
||||
assert!(request_text.contains("\"imageSize\":\"512\""));
|
||||
|
||||
let body = r#"{"candidates":[{"content":{"parts":[{"inlineData":{"mimeType":"image/png","data":"iVBORw0KGgpyZXN0"}}]}}]}"#;
|
||||
let response = format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{}",
|
||||
body.len(),
|
||||
body
|
||||
);
|
||||
let _ = stream.write_all(response.as_bytes()).await;
|
||||
});
|
||||
let settings = VectorEngineImageSettings {
|
||||
base_url: format!("http://{}", server_addr),
|
||||
api_key: "test-key".to_string(),
|
||||
request_timeout_ms: 1_000,
|
||||
};
|
||||
let client = build_vector_engine_image_http_client(&settings).expect("client should build");
|
||||
|
||||
let generated = create_vector_engine_nanobanana_generate_content(
|
||||
&client,
|
||||
&settings,
|
||||
"gemini-3.1-flash-image-preview",
|
||||
"生成角色图",
|
||||
Some("文字、水印"),
|
||||
"2:3",
|
||||
"512",
|
||||
&[],
|
||||
"测试 nanobanana",
|
||||
)
|
||||
.await
|
||||
.expect("nanobanana response should parse");
|
||||
|
||||
assert_eq!(generated.images.len(), 1);
|
||||
assert_eq!(generated.images[0].mime_type, "image/png");
|
||||
server.abort();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn vector_engine_image_generation_retries_upstream_502_once_and_succeeds() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0")
|
||||
|
||||
@@ -346,6 +346,38 @@ pub struct EditorCharacterAnimationGenerateResponse {
|
||||
pub price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorVideoGenerateRequest {
|
||||
pub prompt: String,
|
||||
pub model: String,
|
||||
pub aspect_ratio: String,
|
||||
pub duration_seconds: u32,
|
||||
pub resolution: String,
|
||||
pub mode: String,
|
||||
pub sound: String,
|
||||
pub price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorVideoGenerateResponse {
|
||||
pub ok: bool,
|
||||
pub video_src: String,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
pub source_type: String,
|
||||
pub prompt: String,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub actual_prompt: Option<String>,
|
||||
pub model: String,
|
||||
pub provider: String,
|
||||
pub task_id: String,
|
||||
pub duration_seconds: u32,
|
||||
pub resolution: String,
|
||||
pub price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct CharacterAnimationDraftPayload {
|
||||
@@ -908,6 +940,53 @@ mod tests {
|
||||
assert_eq!(payload["fps"], json!(8));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_request_supports_lovart_model_contract() {
|
||||
let payload = serde_json::to_value(EditorVideoGenerateRequest {
|
||||
prompt: "让角色向镜头挥手".to_string(),
|
||||
model: "kling3.0-omni".to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
duration_seconds: 5,
|
||||
resolution: "480p".to_string(),
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
price_mud_points: 50,
|
||||
})
|
||||
.expect("request should serialize");
|
||||
|
||||
assert_eq!(payload["aspectRatio"], json!("16:9"));
|
||||
assert_eq!(payload["durationSeconds"], json!(5));
|
||||
assert_eq!(payload["priceMudPoints"], json!(50));
|
||||
assert_eq!(payload["model"], json!("kling3.0-omni"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_response_uses_canvas_video_shape() {
|
||||
let payload = serde_json::to_value(EditorVideoGenerateResponse {
|
||||
ok: true,
|
||||
video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(),
|
||||
width: 1280,
|
||||
height: 720,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: "让角色向镜头挥手".to_string(),
|
||||
actual_prompt: Some("让角色向镜头挥手".to_string()),
|
||||
model: "kling3.0-omni".to_string(),
|
||||
provider: "VectorEngine".to_string(),
|
||||
task_id: "task-1".to_string(),
|
||||
duration_seconds: 5,
|
||||
resolution: "480p".to_string(),
|
||||
price_mud_points: 50,
|
||||
})
|
||||
.expect("response should serialize");
|
||||
|
||||
assert_eq!(
|
||||
payload["videoSrc"],
|
||||
json!("/generated-editor-videos/task-1/preview.mp4")
|
||||
);
|
||||
assert_eq!(payload["sourceType"], json!("generated"));
|
||||
assert_eq!(payload["durationSeconds"], json!(5));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn character_workflow_cache_response_keeps_legacy_shape() {
|
||||
let payload = serde_json::to_value(CharacterWorkflowCacheSaveResponse {
|
||||
|
||||
@@ -150,6 +150,7 @@ export function serializeLayer(
|
||||
originalHeight: layer.originalHeight,
|
||||
zIndex: layer.zIndex,
|
||||
sourceType: layer.sourceType,
|
||||
mediaType: layer.mediaType,
|
||||
prompt: layer.prompt,
|
||||
actualPrompt: layer.actualPrompt,
|
||||
model: layer.model,
|
||||
@@ -207,6 +208,7 @@ export function hydrateLayer(
|
||||
sourceType: isCanvasSourceType(snapshot.sourceType)
|
||||
? snapshot.sourceType
|
||||
: 'uploaded',
|
||||
mediaType: snapshot.mediaType === 'video' ? 'video' : 'image',
|
||||
prompt: stringOrNull(snapshot.prompt),
|
||||
actualPrompt: stringOrNull(snapshot.actualPrompt),
|
||||
model: stringOrNull(snapshot.model),
|
||||
@@ -406,7 +408,9 @@ export function canvasAssetKindOrNull(value: unknown): CanvasAssetKind | null {
|
||||
return value === 'spec' ||
|
||||
value === 'character' ||
|
||||
value === 'icon' ||
|
||||
value === 'icon-spec'
|
||||
value === 'icon-spec' ||
|
||||
value === 'ui-design' ||
|
||||
value === 'video'
|
||||
? value
|
||||
: null;
|
||||
}
|
||||
|
||||
@@ -7,12 +7,21 @@ import type {
|
||||
|
||||
export type CanvasSourceType = 'uploaded' | 'generated' | 'mock_generated';
|
||||
|
||||
export type CanvasAssetKind = 'spec' | 'character' | 'icon' | 'icon-spec';
|
||||
export type CanvasAssetKind =
|
||||
| 'spec'
|
||||
| 'character'
|
||||
| 'icon'
|
||||
| 'icon-spec'
|
||||
| 'ui-design'
|
||||
| 'video';
|
||||
|
||||
export type CanvasMediaType = 'image' | 'video';
|
||||
|
||||
export type EditorAsset = {
|
||||
id: string;
|
||||
label: string;
|
||||
src: string;
|
||||
mediaType?: CanvasMediaType;
|
||||
width: number;
|
||||
height: number;
|
||||
folderId: string;
|
||||
@@ -52,6 +61,7 @@ export type CanvasLayer = {
|
||||
resourceId: string;
|
||||
title: string;
|
||||
src: string;
|
||||
mediaType?: CanvasMediaType;
|
||||
x: number;
|
||||
y: number;
|
||||
width: number;
|
||||
@@ -89,9 +99,11 @@ export type CanvasTool =
|
||||
| 'hand'
|
||||
| 'upload'
|
||||
| 'generate'
|
||||
| 'video'
|
||||
| 'spec'
|
||||
| 'character'
|
||||
| 'icon'
|
||||
| 'ui-design'
|
||||
| 'text'
|
||||
| 'shape'
|
||||
| 'export';
|
||||
@@ -124,7 +136,14 @@ export type CharacterReferenceImage = {
|
||||
|
||||
export type GenerateDialogState = {
|
||||
id?: string;
|
||||
mode: 'generate' | 'edit' | 'spec' | 'character' | 'icon';
|
||||
mode:
|
||||
| 'generate'
|
||||
| 'edit'
|
||||
| 'spec'
|
||||
| 'character'
|
||||
| 'icon'
|
||||
| 'ui-design'
|
||||
| 'video';
|
||||
prompt: string;
|
||||
status: 'idle' | 'generating' | 'failed';
|
||||
composerOpen?: boolean;
|
||||
@@ -133,11 +152,19 @@ export type GenerateDialogState = {
|
||||
specType?: SpecGenerationType;
|
||||
specValues?: SpecFormValues;
|
||||
specReference?: CharacterReferenceImage | null;
|
||||
generationReferences?: CharacterReferenceImage[];
|
||||
characterSpecReference?: CharacterReferenceImage | null;
|
||||
characterReferences?: CharacterReferenceImage[];
|
||||
iconSpecReference?: CharacterReferenceImage | null;
|
||||
iconDescriptions?: string[];
|
||||
uiDesignSpecReference?: CharacterReferenceImage | null;
|
||||
imageModel?: string;
|
||||
videoModel?: string;
|
||||
videoAspectRatio?: string;
|
||||
videoResolution?: string;
|
||||
videoDurationSeconds?: 4 | 5;
|
||||
videoMode?: 'std';
|
||||
videoSound?: 'off';
|
||||
aspectRatio?: string;
|
||||
imageSize?: string;
|
||||
errorMessage?: string;
|
||||
@@ -219,10 +246,12 @@ export type CharacterAnimationPanelState = {
|
||||
|
||||
export type UploadTarget =
|
||||
| 'asset'
|
||||
| 'generation-reference'
|
||||
| 'spec-reference'
|
||||
| 'character-spec'
|
||||
| 'character-reference'
|
||||
| 'icon-spec';
|
||||
| 'icon-spec'
|
||||
| 'ui-design-icon-spec';
|
||||
|
||||
export type SnapGuide = {
|
||||
vertical?: number;
|
||||
|
||||
@@ -0,0 +1,296 @@
|
||||
/* @vitest-environment jsdom */
|
||||
|
||||
import { fireEvent, render, screen, within } from '@testing-library/react';
|
||||
import {
|
||||
createRef,
|
||||
type ComponentProps,
|
||||
type Dispatch,
|
||||
type SetStateAction,
|
||||
} from 'react';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import type { GenerateDialogState } from './ImageCanvasEditorTypes';
|
||||
import { ImageCanvasGenerationComposerView } from './ImageCanvasGenerationComposerView';
|
||||
|
||||
function mockStateSetter<T>() {
|
||||
return vi.fn() as unknown as Dispatch<SetStateAction<T>>;
|
||||
}
|
||||
|
||||
function renderComposer(
|
||||
generateDialog: GenerateDialogState,
|
||||
overrides: Partial<
|
||||
ComponentProps<typeof ImageCanvasGenerationComposerView>
|
||||
> = {},
|
||||
) {
|
||||
const props: ComponentProps<typeof ImageCanvasGenerationComposerView> = {
|
||||
specToolWrapRef: createRef(),
|
||||
characterSpecButtonRef: createRef(),
|
||||
characterReferenceButtonRef: createRef(),
|
||||
generationReferenceButtonRef: createRef(),
|
||||
iconSpecButtonRef: createRef(),
|
||||
isSpecMenuOpen: false,
|
||||
isCharacterSpecMenuOpen: false,
|
||||
isCharacterReferenceMenuOpen: false,
|
||||
isGenerationReferenceMenuOpen: false,
|
||||
isIconSpecMenuOpen: false,
|
||||
isUiDesignSpecMenuOpen: false,
|
||||
isPickingCharacterSpecFromCanvas: false,
|
||||
isPickingCharacterReferenceFromCanvas: false,
|
||||
isPickingGenerationReferenceFromCanvas: false,
|
||||
isPickingIconSpecFromCanvas: false,
|
||||
isPickingUiDesignSpecFromCanvas: false,
|
||||
generateDialog,
|
||||
generationComposerStyle: { left: 320, top: 240 },
|
||||
iconComposerStyle: { left: 320, top: 240, width: '32rem' },
|
||||
quickEditPanel: null,
|
||||
quickEditSourceLayer: null,
|
||||
quickEditPanelStyle: null,
|
||||
quickEditSizeOptions: ['1024x1024'],
|
||||
quickEditModelOptions: [{ label: 'gpt-image-2', value: 'gpt-image-2' }],
|
||||
characterAnimationPanel: null,
|
||||
characterAnimationSourceLayer: null,
|
||||
characterAnimationPanelStyle: null,
|
||||
characterAnimationPrice: 40,
|
||||
setGenerateDialog: mockStateSetter<GenerateDialogState | null>(),
|
||||
setQuickEditPanel:
|
||||
mockStateSetter<
|
||||
ComponentProps<
|
||||
typeof ImageCanvasGenerationComposerView
|
||||
>['quickEditPanel']
|
||||
>(),
|
||||
setCharacterAnimationPanel:
|
||||
mockStateSetter<
|
||||
ComponentProps<
|
||||
typeof ImageCanvasGenerationComposerView
|
||||
>['characterAnimationPanel']
|
||||
>(),
|
||||
setIsCharacterSpecMenuOpen: mockStateSetter<boolean>(),
|
||||
setIsCharacterReferenceMenuOpen: mockStateSetter<boolean>(),
|
||||
setIsGenerationReferenceMenuOpen: mockStateSetter<boolean>(),
|
||||
setIsIconSpecMenuOpen: mockStateSetter<boolean>(),
|
||||
setIsUiDesignSpecMenuOpen: mockStateSetter<boolean>(),
|
||||
setIsPickingCharacterSpecFromCanvas: mockStateSetter<boolean>(),
|
||||
setIsPickingCharacterReferenceFromCanvas: mockStateSetter<boolean>(),
|
||||
setIsPickingGenerationReferenceFromCanvas: mockStateSetter<boolean>(),
|
||||
setIsPickingIconSpecFromCanvas: mockStateSetter<boolean>(),
|
||||
setIsPickingUiDesignSpecFromCanvas: mockStateSetter<boolean>(),
|
||||
onOpenSpecDialog: vi.fn(),
|
||||
onRequestUpload: vi.fn(),
|
||||
onSubmitImageGeneration: vi.fn(),
|
||||
onSubmitIconSpritesheetGeneration: vi.fn(),
|
||||
onSubmitQuickEdit: vi.fn(),
|
||||
onSubmitCharacterAnimation: vi.fn(),
|
||||
onCloseGenerateComposer: vi.fn(),
|
||||
onUpdateSpecFormValue: vi.fn(),
|
||||
onUpdateIconDescription: vi.fn(),
|
||||
onAddIconDescription: vi.fn(),
|
||||
onUpdateCharacterAnimationDuration: vi.fn(),
|
||||
onRememberImageModel: vi.fn(),
|
||||
...overrides,
|
||||
};
|
||||
|
||||
return render(<ImageCanvasGenerationComposerView {...props} />);
|
||||
}
|
||||
|
||||
describe('ImageCanvasGenerationComposerView', () => {
|
||||
it('让生成UI设计图面板复用普通图片生成面板的纵向结构', () => {
|
||||
renderComposer({
|
||||
mode: 'ui-design',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
uiDesignSpecReference: null,
|
||||
imageModel: 'gpt-image-2',
|
||||
aspectRatio: '16:9',
|
||||
imageSize: '1K',
|
||||
});
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成UI设计图' });
|
||||
expect(panel.className).toContain(
|
||||
'image-canvas-editor__generation-composer',
|
||||
);
|
||||
expect(panel.className).toContain(
|
||||
'image-canvas-editor__generation-composer--image',
|
||||
);
|
||||
expect(
|
||||
within(panel).getByRole('button', { name: 'UI设计图标素材规范' })
|
||||
.parentElement?.className,
|
||||
).toContain('image-canvas-editor__generation-ref');
|
||||
expect(
|
||||
within(panel).getByRole('textbox', { name: 'UI设计要求' }).className,
|
||||
).toContain('image-canvas-editor__generation-prompt');
|
||||
expect(
|
||||
panel.querySelector('.image-canvas-editor__generation-composer-footer'),
|
||||
).toBeTruthy();
|
||||
});
|
||||
|
||||
it('让生成规范图片面板复用生成类面板 shell 和底部按钮结构', () => {
|
||||
renderComposer({
|
||||
mode: 'spec',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
specType: 'character',
|
||||
specValues: {
|
||||
playSetting: '战棋类RPG玩法',
|
||||
artStyle: '像素风',
|
||||
bodyRatio: '3',
|
||||
characterView: '右向斜侧身站姿',
|
||||
customPrompt: '',
|
||||
},
|
||||
specReference: null,
|
||||
});
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成规范' });
|
||||
expect(panel.className).toContain(
|
||||
'image-canvas-editor__generation-composer',
|
||||
);
|
||||
expect(panel.className).toContain(
|
||||
'image-canvas-editor__generation-composer--image',
|
||||
);
|
||||
expect(
|
||||
within(panel).getByRole('button', { name: '参考图' }).parentElement
|
||||
?.className,
|
||||
).toContain('image-canvas-editor__generation-ref');
|
||||
expect(
|
||||
panel.querySelector('.image-canvas-editor__generation-composer-footer'),
|
||||
).toBeTruthy();
|
||||
expect(
|
||||
within(panel).getByRole('button', { name: '提交生成规范' }).className,
|
||||
).toContain('image-canvas-editor__generation-submit');
|
||||
});
|
||||
|
||||
it('让图标素材规范生成面板也保留首行参考区', () => {
|
||||
renderComposer({
|
||||
mode: 'spec',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
specType: 'icon',
|
||||
specValues: {
|
||||
playSetting: '休闲小游戏',
|
||||
artStyle: '清爽卡通',
|
||||
bodyRatio: '3',
|
||||
characterView: '右向斜侧身站姿',
|
||||
customPrompt: '',
|
||||
},
|
||||
specReference: null,
|
||||
});
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成规范' });
|
||||
expect(
|
||||
within(panel).getByRole('button', { name: '参考图' }).parentElement
|
||||
?.className,
|
||||
).toContain('image-canvas-editor__generation-ref');
|
||||
});
|
||||
it('生成图片参考图点击先弹来源菜单，不直接打开上传', () => {
|
||||
const onRequestUpload = vi.fn();
|
||||
const setIsGenerationReferenceMenuOpen = vi.fn();
|
||||
|
||||
renderComposer(
|
||||
{
|
||||
mode: 'generate',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
generationReferences: [],
|
||||
},
|
||||
{
|
||||
isGenerationReferenceMenuOpen: true,
|
||||
onRequestUpload,
|
||||
setIsGenerationReferenceMenuOpen,
|
||||
},
|
||||
);
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成图片' });
|
||||
fireEvent.click(within(panel).getByRole('button', { name: '添加参考图' }));
|
||||
|
||||
expect(onRequestUpload).not.toHaveBeenCalled();
|
||||
expect(setIsGenerationReferenceMenuOpen).toHaveBeenCalled();
|
||||
const menu = screen.getByRole('menu', { name: '参考图来源' });
|
||||
expect(
|
||||
within(menu).getByRole('menuitem', { name: '从画布中选择' }),
|
||||
).toBeTruthy();
|
||||
expect(
|
||||
within(menu).getByRole('menuitem', { name: '上传图片' }),
|
||||
).toBeTruthy();
|
||||
});
|
||||
|
||||
it('生成视频参考图点击先弹来源菜单，不直接打开上传', () => {
|
||||
const onRequestUpload = vi.fn();
|
||||
const setIsGenerationReferenceMenuOpen = vi.fn();
|
||||
|
||||
renderComposer(
|
||||
{
|
||||
mode: 'video',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
generationReferences: [],
|
||||
videoModel: 'seedance2.0',
|
||||
videoAspectRatio: '16:9',
|
||||
videoDurationSeconds: 4,
|
||||
videoResolution: '480p',
|
||||
videoMode: 'std',
|
||||
videoSound: 'off',
|
||||
},
|
||||
{
|
||||
isGenerationReferenceMenuOpen: true,
|
||||
onRequestUpload,
|
||||
setIsGenerationReferenceMenuOpen,
|
||||
},
|
||||
);
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成视频' });
|
||||
fireEvent.click(
|
||||
within(panel).getByRole('button', { name: '添加视频参考图' }),
|
||||
);
|
||||
|
||||
expect(onRequestUpload).not.toHaveBeenCalled();
|
||||
expect(setIsGenerationReferenceMenuOpen).toHaveBeenCalled();
|
||||
const menu = screen.getByRole('menu', { name: '参考图来源' });
|
||||
expect(
|
||||
within(menu).getByRole('menuitem', { name: '从画布中选择' }),
|
||||
).toBeTruthy();
|
||||
expect(
|
||||
within(menu).getByRole('menuitem', { name: '上传图片' }),
|
||||
).toBeTruthy();
|
||||
});
|
||||
it('生成规范参考图点击先弹来源菜单，不直接打开上传', () => {
|
||||
const onRequestUpload = vi.fn();
|
||||
const setIsGenerationReferenceMenuOpen = vi.fn();
|
||||
|
||||
renderComposer(
|
||||
{
|
||||
mode: 'spec',
|
||||
prompt: '',
|
||||
status: 'idle',
|
||||
composerOpen: true,
|
||||
specType: 'character',
|
||||
specValues: {
|
||||
playSetting: '战棋类RPG玩法',
|
||||
artStyle: '像素风',
|
||||
bodyRatio: '3',
|
||||
characterView: '右向斜侧身站姿',
|
||||
customPrompt: '',
|
||||
},
|
||||
specReference: null,
|
||||
},
|
||||
{
|
||||
isGenerationReferenceMenuOpen: true,
|
||||
onRequestUpload,
|
||||
setIsGenerationReferenceMenuOpen,
|
||||
},
|
||||
);
|
||||
|
||||
const panel = screen.getByRole('dialog', { name: '生成规范' });
|
||||
fireEvent.click(within(panel).getByRole('button', { name: '参考图' }));
|
||||
|
||||
expect(onRequestUpload).not.toHaveBeenCalled();
|
||||
expect(setIsGenerationReferenceMenuOpen).toHaveBeenCalled();
|
||||
const menu = screen.getByRole('menu', { name: '参考图来源' });
|
||||
expect(within(menu).getByRole('menuitem', { name: '从画布中选择' })).toBeTruthy();
|
||||
expect(within(menu).getByRole('menuitem', { name: '上传图片' })).toBeTruthy();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,6 +2,7 @@ import type {
|
||||
EditorIconSpritesheetGenerationResult,
|
||||
EditorIconSpritesheetIconResult,
|
||||
EditorImageGenerationResult,
|
||||
EditorVideoGenerationResult,
|
||||
} from '../../services/image-editor/editorProjectClient';
|
||||
import { resolveLayerResolutionSize } from './ImageCanvasEditorModel';
|
||||
import { ICON_FRAME_DISPLAY_SIZE } from './ImageCanvasGenerationModel';
|
||||
@@ -43,6 +44,16 @@ type IconSpritesheetResultLayerOptions = {
|
||||
frame?: GenerateDialogState['placeholder'];
|
||||
};
|
||||
|
||||
type VideoResultLayerOptions = {
|
||||
generated: EditorVideoGenerationResult;
|
||||
generatedIndex: number;
|
||||
title: string;
|
||||
canvasSize: CanvasSize;
|
||||
viewport: CanvasViewport;
|
||||
generationInputs: CanvasGenerationInputs;
|
||||
frame?: GenerateDialogState['placeholder'];
|
||||
};
|
||||
|
||||
function getViewportWorldCenter({
|
||||
canvasSize,
|
||||
viewport,
|
||||
@@ -235,3 +246,53 @@ export function createIconSpritesheetResultLayers({
|
||||
return layer;
|
||||
});
|
||||
}
|
||||
|
||||
export function createVideoResultLayer({
|
||||
generated,
|
||||
generatedIndex,
|
||||
title,
|
||||
canvasSize,
|
||||
viewport,
|
||||
generationInputs,
|
||||
frame,
|
||||
}: VideoResultLayerOptions): CanvasLayer {
|
||||
const originalWidth = generated.width || 1280;
|
||||
const originalHeight = generated.height || 720;
|
||||
const { width, height } = resolveLayerResolutionSize(
|
||||
originalWidth,
|
||||
originalHeight,
|
||||
{ width: 560, height: 315 },
|
||||
);
|
||||
const worldCenter = getViewportWorldCenter({ canvasSize, viewport });
|
||||
const frameX =
|
||||
frame && frame.width > 0
|
||||
? frame.x + frame.width / 2 - width / 2
|
||||
: undefined;
|
||||
const frameY =
|
||||
frame && frame.height > 0
|
||||
? frame.y + frame.height / 2 - height / 2
|
||||
: undefined;
|
||||
|
||||
return {
|
||||
id: `layer-video-${generatedIndex}`,
|
||||
resourceId: `local-resource-video-${generatedIndex}`,
|
||||
title,
|
||||
src: generated.videoSrc,
|
||||
mediaType: 'video',
|
||||
assetKind: 'video',
|
||||
x: frameX ?? worldCenter.x - width / 2,
|
||||
y: frameY ?? worldCenter.y - height / 2,
|
||||
width,
|
||||
height,
|
||||
originalWidth,
|
||||
originalHeight,
|
||||
zIndex: generatedIndex + 10,
|
||||
sourceType: generated.sourceType,
|
||||
prompt: generated.prompt,
|
||||
actualPrompt: generated.actualPrompt ?? generated.prompt,
|
||||
model: generated.model,
|
||||
provider: generated.provider,
|
||||
taskId: generated.taskId,
|
||||
generationInputs,
|
||||
};
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user