新增编辑器生成规范、生成角色形象、生成图标素材等功能
新增编辑器生成规范、生成角色形象、生成图标素材等功能
This commit is contained in:
@@ -2246,3 +2246,35 @@
|
||||
- 影响范围：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`、`server-rs/crates/spacetime-client/src/editor_project.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`src/services/image-editor/editorProjectClient.ts`、`src/components/image-editor/ImageCanvasEditorView.tsx`、后端数据契约文档和图片画布前端技术方案。
|
||||
- 验证方式：`npm run spacetime:generate -- --rust-only`、`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:spacetime-schema`、`npm run check:encoding`、`cargo check -p spacetime-client -p api-server --manifest-path server-rs/Cargo.toml`、`git diff --check`。
|
||||
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`。
|
||||
|
||||
## 2026-06-15 图片画布角色图层新增动画生成入口
|
||||
|
||||
- 背景：图片画布已有角色形象图层标记 `assetKind="character"`，需要只对角色图片开放动画生成，不让普通素材误触发角色动画链路。
|
||||
- 决策：角色动画入口只由画布图层 `assetKind="character"` 控制，在图片上方浮动工具条和右键菜单显示 `生成动画`；非角色图层不展示入口。点击后打开独立 `角色动画生成面板`，桌面端锚定到图片右侧，移动端按底部面板承接。前端固定提交 `seedance2.0`、分辨率 / 比例 / 帧数 / 时长 / 价格字段；后端经 `/api/editor/character-animations/generations` 使用角色图作为首帧和尾帧生成视频，并立即抽取 32 / 40 / 48 帧、绿幕去背后写入 OSS。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/services/image-editor/editorProjectClient.ts`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/shared-contracts/src/assets.rs`、图片画布技术方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-16 图片画布图标素材面板采用 Lovart 式参考卡与横向增宽布局
|
||||
|
||||
- 背景：图标素材生成面板里，规范入口与素材描述项过于平铺，且子面板内部采用滑动列表，和 Lovart 风格画布的参考卡 / 物料卡不一致。
|
||||
- 决策：`生成图标素材` 面板不使用内部纵向滚动列表；每新增一个素材描述项就让面板整体增宽，保持描述项横向卡片一眼可扫。图标素材规范入口改为 Lovart 式参考卡：缩略图、名称、绑定状态和轻量动作分区分开呈现，独立菜单只负责来源切换，不再承载说明文案。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/index.css`、图标素材生成专项设计文档。
|
||||
- 验证方式：新增或增删素材描述项时，面板宽度应随项数变化；图标素材规范入口应呈现参考卡视觉而非纯文本按钮；移动端下仍应固定在底部锚定，不出现内部滚动条。
|
||||
- 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-16 图片画布图标素材采用 nanobanana2 spritesheet + 后端连通域拆分
|
||||
|
||||
- 背景：图片画布需要一次生成多枚 UI 图标素材，并保证生成后能按用户输入顺序命名、拆成独立透明素材铺回画布。
|
||||
- 决策：底部 `生成图标素材` 入口创建一叠空白图标占位和独立面板；图标规范参考图只允许绑定 `assetKind="icon-spec"`。前端提交 `/api/editor/icon-spritesheets/generations`，后端固定使用 VectorEngine `gemini-3.1-flash-image-preview`，`<=25` 个描述用 `512x512`，`>25` 个描述用 `1024x1024`，先生成绿幕 1:1 spritesheet，再由 `platform-image` 绿幕去背并按 8 邻域连通域从上到下、从左到右拆分。成品图标图层写入 `assetKind="icon"`。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/services/image-editor/editorProjectClient.ts`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/platform-image/src/generated_asset_sheets/sheet.rs`、图片画布技术方案。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`cargo test --manifest-path server-rs/Cargo.toml -p platform-image generated_asset_sheets::sheet::tests -- --nocapture`、`cargo test --manifest-path server-rs/Cargo.toml -p api-server editor_project -- --nocapture`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 2026-06-16 图片画布生成面板与浮层层级收口
|
||||
|
||||
- 背景：图片画布底部工具栏和角色参考图行都存在局部滚动 / 裁切容器，生成规范菜单和角色规范来源菜单如果仍内嵌在触发按钮附近，会被边界遮挡；同时生成类面板打开后隐藏底部工具栏会破坏连续创作节奏。
|
||||
- 决策：`生成规范`、`角色形象规范来源` 和 `图标素材规范来源` 菜单统一通过页面级 fixed portal 渲染到 `document.body`，触发按钮只提供定位锚点；点击 `生成工具`、`生成角色形象` 或 `生成图标素材` 后底部 AI 工具栏保持可见。点击画布空白区域只关闭当前生成面板并清除图片选中样式，不删除新建的占位图。角色面板中的 `角色形象规范` 与 `上传常规参考图` 入口统一改为 Lovart 式参考图卡片。
|
||||
- 影响范围：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/index.css`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`、图片画布前端技术方案和角色形象生成设计文档。
|
||||
- 验证方式：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联文档：`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# 踩坑与排障记录
|
||||
# 踩坑与排障记录
|
||||
|
||||
> 用途：记录已验证、未来很可能再次遇到的问题。每条都应包含现象、原因、处理方式和验证方式。
|
||||
|
||||
@@ -15,6 +15,54 @@
|
||||
- 关联：相关文件、文档、提交或 Issue
|
||||
```
|
||||
|
||||
## 图片编辑器底部生成按钮不要复用单一画布生成状态
|
||||
|
||||
- 现象：图片画布里先新建一个“生成规范”占位，再点击“生成角色形象”或其它底部生成入口，前一个规范占位和面板状态被销毁。
|
||||
- 原因：底部普通生成、规范、角色和图标素材曾共用单个 `generateDialog` 状态；后一次点击直接覆盖该状态，等同把前一个画布生成对象卸载。
|
||||
- 处理：底部生成类入口每次点击都创建独立 generation dialog id；当前 active 对象只负责显示编辑面板，旧对象归档为 inactive 后仍保留占位和生成逻辑状态。生成完成 / 失败回写、生成中拖拽和删除都必须按 dialog id 读取 active + inactive 中的最新对象，不能回退到提交瞬间的旧占位快照。
|
||||
- 验证：`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps existing generation placeholders"` 应断言规范占位和角色占位可同时存在；`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps archived generation logic"` 应断言旧对象归档后拖动，占位完成回写仍落在最新位置。
|
||||
- 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 图片编辑器生成中设定面板不要和预览框绑成同一可见性
|
||||
|
||||
- 现象：图片编辑器里点击生成后，有时设定面板没收起，有时连画布上的占位预览一起消失，看起来像“生成中界面掉了”。
|
||||
- 原因：生成中状态只收了 composer 可见性，或把占位框和设定面板共用了同一段条件渲染；面板隐藏后把 placeholder 也一起卸掉，就会丢掉 Lovart 式生成中预览。
|
||||
- 处理：进入 `generating` 后只隐藏设定面板，保留占位框和生成中状态胶囊；面板外观、预览框和结果图层分开控制，不共用同一个 `composerOpen` 条件。
|
||||
- 验证：对应测试应断言生成按钮点击后 `dialog` 消失但 `image-canvas-editor__generation-frame--generating` 仍然存在。
|
||||
- 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`。
|
||||
|
||||
## 图片画布快速编辑不要直接提交普通图片 URL
|
||||
|
||||
- 现象：图片画布快速编辑站内示例图、历史 generated 图或 OSS generated 图时，后端返回 `修改图片参考图必须是图片 Data URL。`。
|
||||
- 原因：快速编辑直接把图层 `src` 塞进 `/api/editor/images/generations` 的 `referenceImageSrcs`；默认示例图和部分持久化图层的 `src` 是 `/creation-type-references/*.webp`、`/generated-*` 或 OSS URL，而 `api-server` 的编辑参考图解析只接收 `data:image/*;base64,...`。
|
||||
- 处理：前端统一通过 `resolveEditorImageReferenceDataUrl(...)` 在提交前读取图片字节并转成图片 Data URL；Data URL 原样透传，`/generated-*` 和 generated OSS URL 走 `/api/assets/read-bytes` 避免 CORS，普通 public 路径直接 fetch。
|
||||
- 验证：`npm run test -- src/services/image-editor/editorImageReference.test.ts src/components/image-editor/ImageCanvasEditorView.test.tsx -t "editorImageReference|converts non-data-url quick edit source images before submitting references"`。
|
||||
- 关联：`src/services/image-editor/editorImageReference.ts`、`src/components/image-editor/ImageCanvasEditorView.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 图片编辑器生成类菜单要挂到页面级 portal
|
||||
|
||||
- 现象：底部 `生成规范` 菜单、角色面板里的 `角色形象规范` 来源菜单点击后像没有弹出来，实际被按钮所在的局部滚动容器挡住了。
|
||||
- 原因：菜单仍然渲染在底部工具栏或参考图横向滚动行内部，父容器带 `overflow`，弹层无法越出边界。
|
||||
- 处理：这类轻量菜单统一用页面级 fixed portal 挂到 `document.body`，位置根据触发按钮的 `getBoundingClientRect()` 计算；底部 AI 工具栏在生成面板打开时仍保持可见，不要整栏隐藏。
|
||||
- 验证：测试断言菜单不包含在底部工具栏 / 参考图行里，并且生成面板打开时底部 `AI画布工具栏` 仍存在。
|
||||
- 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`。
|
||||
|
||||
## 图片编辑器生成占位图在生成中也要使用最新拖拽位置
|
||||
|
||||
- 现象：用户在图片编辑器里提交生成后继续拖动画布占位图，预览框可以移动，但生成完成后的真实图片仍落回提交瞬间的旧位置。
|
||||
- 原因：生成提交函数闭包里保存了旧的 `dialog.placeholder` 快照；如果完成回包仍用这个快照创建图层，就会丢失生成中期间的拖拽坐标。若 `handleGenerationFramePointerDown` 又按 `status === 'generating'` 拦截，则生成中占位图完全不能拖动。
|
||||
- 处理：生成占位图的 pointer down 不因 `generating` 禁止；普通图片、规范图、角色图和图标素材回包创建图层时，都从当前 `generateDialogRef.current.placeholder` 读取最新占位位置，失败后保留的占位图也继续走同一拖拽链路。
|
||||
- 验证：`npm test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps the generation placeholder draggable while the image is generating"`。
|
||||
- 关联：`src/components/image-editor/ImageCanvasEditorView.tsx`、`src/components/image-editor/ImageCanvasEditorView.test.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## Windows 本地 dev 不要把 RUSTC_WRAPPER 绕过写成 rustc
|
||||
|
||||
- 现象：Windows 上执行 `npm run dev:api-server` 时，api-server 在 Cargo 启动阶段失败，日志出现 `error: multiple input filenames provided (first two filenames are ... rustc.exe and -)`，`/healthz` 无法访问。
|
||||
- 原因：`server-rs/.cargo/config.toml` 默认配置 `rustc-wrapper = "sccache"`；本地 dev 脚本为了绕过损坏的 sccache 需要覆盖 wrapper。Windows 下如果把 `RUSTC_WRAPPER` 设置为 `rustc`，Cargo 会按 wrapper 协议调用 `rustc <真实rustc路径> - ...`，真实 rustc 把 wrapper 传入的 rustc 路径和 stdin `-` 都当输入文件。
|
||||
- 处理：Windows 本地 dev 脚本应把 `RUSTC_WRAPPER` 和 `CARGO_BUILD_RUSTC_WRAPPER` 显式设为空字符串，让 Cargo 覆盖项目配置并直连真实 rustc；Linux 保持 `/usr/bin/env` 绕过 sccache。
|
||||
- 验证：`npm run test -- scripts/dev.test.ts -t "Windows 下本地 dev Rust env 用空 wrapper 覆盖项目 sccache"`，并用 `npm run dev:api-server` 拉起后访问实际 api 端口的 `/healthz` 返回 200。
|
||||
- 关联：`scripts/dev.mjs`、`scripts/dev.test.ts`、`docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`。
|
||||
|
||||
## 外部生成 worker 业务失败重试会撞上钱包扣退费幂等
|
||||
|
||||
- 现象：同一个外部生成 job 如果第一次业务失败后退款，再用同一个业务资源 ID 自动重试并成功，钱包 `consume` ledger 可能因为同 ID 已存在而跳过，最终出现“失败已退、成功不再扣”的余额漂移。
|
||||
|
||||
106
docs/superpowers/plans/【编辑器】图片画布生成对象独立化修复计划-2026-06-16.md
Normal file
106
docs/superpowers/plans/【编辑器】图片画布生成对象独立化修复计划-2026-06-16.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# 图片画布生成对象独立化修复 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking.
|
||||
|
||||
**Goal:** 修复图片画布底部生成按钮复用单一状态导致后创建对象销毁前一个对象的问题。
|
||||
|
||||
**Architecture:** 保留现有图片画布组件结构，先以回归测试锁定“规范占位 + 角色占位可并存”。实现上把生成占位状态从单个 `generateDialog` 扩展为 active dialog + inactive dialog 列表；每次新建生成对象只新增一个 dialog 实例，旧实例保留占位和逻辑状态，只有当前 active 实例渲染编辑面板。
|
||||
|
||||
**Tech Stack:** React、TypeScript、Vitest、Testing Library。
|
||||
|
||||
---
|
||||
|
||||
### Task 1: 补充失败回归测试
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/src/components/image-editor/ImageCanvasEditorView.test.tsx`
|
||||
|
||||
- [x] **Step 1: Write the failing test**
|
||||
|
||||
在 `keeps the bottom AI toolbar visible while generation panels are open` 附近新增测试：
|
||||
|
||||
```tsx
|
||||
it('keeps existing generation placeholders when another bottom generation object is created', () => {
|
||||
render(<ImageCanvasEditorView />);
|
||||
|
||||
const bottomToolbar = screen.getByRole('toolbar', { name: 'AI画布工具栏' });
|
||||
fireEvent.click(
|
||||
within(bottomToolbar).getByRole('button', { name: '生成规范' }),
|
||||
);
|
||||
fireEvent.click(screen.getByRole('menuitem', { name: '角色形象规范' }));
|
||||
|
||||
expect(screen.getByLabelText('规范生成占位图')).toBeTruthy();
|
||||
expect(screen.getByRole('dialog', { name: '生成规范' })).toBeTruthy();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: '生成角色形象' }));
|
||||
|
||||
expect(screen.getByLabelText('规范生成占位图')).toBeTruthy();
|
||||
expect(screen.getByLabelText('角色生成占位图')).toBeTruthy();
|
||||
expect(screen.getByRole('dialog', { name: '生成角色形象' })).toBeTruthy();
|
||||
});
|
||||
```
|
||||
|
||||
- [x] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps existing generation placeholders"`
|
||||
|
||||
Expected: FAIL because only the latest placeholder remains.
|
||||
|
||||
### Task 2: 实现最小独立生成对象状态
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/src/components/image-editor/ImageCanvasEditorView.tsx`
|
||||
|
||||
- [x] **Step 1: Add stable dialog ids and inactive dialog state**
|
||||
|
||||
Add `id: string` to `GenerateDialogState`; add `generationDialogCounterRef`; add `inactiveGenerateDialogs` state. Provide helpers to create ids, archive current active dialog before replacing it, update active/inactive dialogs by id, and list all canvas generation dialogs.
|
||||
|
||||
- [x] **Step 2: Update bottom generation openers**
|
||||
|
||||
Change `openGenerateDialog`、`openSpecDialog`、`openCharacterGenerationDialog`、`openIconGenerationDialog` so each call archives the current active canvas generation dialog and sets a newly created active dialog. Edit modal remains single active dialog and does not archive.
|
||||
|
||||
- [x] **Step 3: Render all placeholders**
|
||||
|
||||
Replace the single placeholder render block with a map over inactive dialogs plus active dialog. Only active dialog shows composer; inactive dialogs remain visible and can be clicked to reactivate their own panel.
|
||||
|
||||
- [x] **Step 4: Keep actions scoped to active dialog**
|
||||
|
||||
Keep submit/update/upload/pick actions operating on active dialog only. Adjust delete, drag, blur, and generated-layer cleanup so they update or remove only the matching active/inactive dialog.
|
||||
|
||||
- [x] **Step 5: Keep archived async generation writeback scoped by dialog id**
|
||||
|
||||
当一个生成对象已经进入 `generating`，随后用户再创建第二个生成对象并把第一个对象归档为 inactive 时，第一个对象仍可能继续被拖拽或等待异步完成。完成回写必须按 `dialog.id` 从 active + inactive 的最新状态读取占位图，不能使用提交瞬间的旧 `placeholder` 快照。
|
||||
|
||||
回归测试：
|
||||
|
||||
```bash
|
||||
npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps archived generation logic"
|
||||
```
|
||||
|
||||
### Task 3: 验证并更新文档
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
|
||||
- Optional Modify: `C:/Genarrative/.hermes/shared-memory/pitfalls.md`
|
||||
|
||||
- [x] **Step 1: Run focused tests**
|
||||
|
||||
Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "keeps existing generation placeholders|opens character spec generation form|opens icon asset generation panel|removes the active character generation placeholder"`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [x] **Step 2: Run full image editor test**
|
||||
|
||||
Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [x] **Step 3: Run encoding check**
|
||||
|
||||
Run: `npm run check:encoding`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [x] **Step 4: Document behavior**
|
||||
|
||||
Add one sentence to the image canvas editor technical plan: bottom generation buttons create independent canvas generation objects; creating a new one must not destroy previous placeholders or generated-object logic.
|
||||
128
docs/superpowers/plans/【编辑器】画板角色动画生成实施计划-2026-06-15.md
Normal file
128
docs/superpowers/plans/【编辑器】画板角色动画生成实施计划-2026-06-15.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# 画板角色动画生成 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** 在图片画布编辑器中，仅对角色图片提供角色动画生成入口，并通过后端 seedance2.0 链路生成视频、抽帧、去绿幕并持久化到 OSS。
|
||||
|
||||
**Architecture:** 前端在 `ImageCanvasEditorView` 中基于图层 `assetKind === "character"` 控制悬浮按钮和右键菜单，打开锚定到图片右侧的独立动画生成面板。前端 service 调用新增编辑器角色动画 API，后端复用 `character_animation_assets.rs` 中现有视频生成、抽帧、绿幕去背、OSS 写入能力，避免新建平行资产系统。
|
||||
|
||||
**Tech Stack:** React + TypeScript + Vitest；Rust Axum `api-server`；现有 `shared-contracts` 资产 DTO；Aliyun OSS 资产持久化；VectorEngine/Ark seedance2.0 角色动画链路。
|
||||
|
||||
---
|
||||
|
||||
### Task 1: 文档补充
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`
|
||||
|
||||
- [ ] **Step 1: 补充角色动画生成章节**
|
||||
- 明确仅 `assetKind: "character"` 图层展示入口。
|
||||
- 明确右侧独立面板字段、预设动作、价格、模型、抽帧和 OSS 存储口径。
|
||||
- 明确非角色图层不展示按钮。
|
||||
|
||||
- [ ] **Step 2: 运行编码检查**
|
||||
- Run: `npm run check:encoding`
|
||||
- Expected: PASS 或仅与本任务无关的既有问题。
|
||||
|
||||
### Task 2: 前端失败测试
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/src/components/image-editor/ImageCanvasEditorView.test.tsx`
|
||||
- Modify: `C:/Genarrative/src/services/image-editor/editorProjectClient.test.ts`
|
||||
|
||||
- [ ] **Step 1: 写失败测试**
|
||||
- 测试角色图层显示悬浮 / 右键 `生成动画`。
|
||||
- 测试非角色图层不显示 `生成动画`。
|
||||
- 测试面板提交请求包含 `sourceLayerId`、`sourceImageSrc`、prompt、resolution、ratio、frameCount、durationSeconds、priceMudPoints、model。
|
||||
|
||||
- [ ] **Step 2: 验证 RED**
|
||||
- Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`
|
||||
- Expected: FAIL，失败原因是功能/API 尚未实现。
|
||||
|
||||
### Task 3: 前端实现
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/src/services/image-editor/editorProjectClient.ts`
|
||||
- Modify: `C:/Genarrative/src/components/image-editor/ImageCanvasEditorView.tsx`
|
||||
- Modify: `C:/Genarrative/src/index.css`
|
||||
|
||||
- [ ] **Step 1: 新增 service 类型和请求函数**
|
||||
- `generateEditorCharacterAnimation(input)` 调用 `/api/editor/character-animations/generations`。
|
||||
- 限定 model 固定为 `seedance2.0` 的回包展示字段。
|
||||
|
||||
- [ ] **Step 2: 扩展图层 assetKind**
|
||||
- `CanvasLayer.assetKind` 支持 `'character' | 'spec' | null`。
|
||||
- hydrate / serialize / 图片类型展示跟随扩展。
|
||||
|
||||
- [ ] **Step 3: 加入口和面板**
|
||||
- 角色图层悬浮工具条和右键菜单显示 `生成动画`。
|
||||
- 面板锚定图片右侧，字段按设计实现，文本框 maxLength=4000。
|
||||
- 价格通过 `resolution * durationSeconds` 计算。
|
||||
|
||||
- [ ] **Step 4: 验证 GREEN**
|
||||
- Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`
|
||||
- Expected: PASS。
|
||||
|
||||
### Task 4: 后端失败测试
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/server-rs/crates/shared-contracts/src/assets.rs`
|
||||
- Modify: `C:/Genarrative/server-rs/crates/api-server/src/character_animation_assets.rs`
|
||||
- Modify: `C:/Genarrative/server-rs/crates/api-server/src/modules/play_flow.rs` 或现有 editor router 文件（按现有路由事实选择）
|
||||
|
||||
- [ ] **Step 1: 写 DTO / prompt / plan 单测**
|
||||
- 验证请求 480p/720p、32/40/48 帧、比例枚举、模型固定 seedance2.0。
|
||||
- 验证构造 prompt 包含用户给定固定骨架与动作描述。
|
||||
- 验证价格计算：480p 每秒 10，720p 每秒 20。
|
||||
|
||||
- [ ] **Step 2: 验证 RED**
|
||||
- Run: `cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`
|
||||
- Expected: FAIL，失败原因是 helper 或 handler 尚未实现。
|
||||
|
||||
### Task 5: 后端实现
|
||||
|
||||
**Files:**
|
||||
- Modify: `C:/Genarrative/server-rs/crates/shared-contracts/src/assets.rs`
|
||||
- Modify: `C:/Genarrative/server-rs/crates/api-server/src/character_animation_assets.rs`
|
||||
- Modify: `C:/Genarrative/server-rs/crates/api-server/src/modules/play_flow.rs` 或现有 editor router 文件
|
||||
|
||||
- [ ] **Step 1: 新增编辑器角色动画 DTO**
|
||||
- 请求字段：sourceLayerId/sourceImageSrc/promptText/resolution/ratio/frameCount/durationSeconds/sourceWidth/sourceHeight。
|
||||
- 响应字段：taskId/model/prompt/previewVideoPath/frames/priceMudPoints。
|
||||
|
||||
- [ ] **Step 2: 新增 handler**
|
||||
- 校验 prompt 1..4000、resolution、ratio、frameCount 与 durationSeconds 组合。
|
||||
- sourceImageSrc 作为首尾帧参考。
|
||||
- 调用现有 seedance image-to-video 逻辑生成预览视频。
|
||||
- 调用现有抽帧 + 绿幕去背 + OSS 持久化逻辑输出帧。
|
||||
|
||||
- [ ] **Step 3: 路由接入**
|
||||
- `POST /api/editor/character-animations/generations`。
|
||||
- 保持走 play_flow 创作/游玩支撑主干或 editor 路由现有聚合，不回到 `app.rs` 平行挂载。
|
||||
|
||||
- [ ] **Step 4: 验证 GREEN**
|
||||
- Run: `cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`
|
||||
- Expected: PASS。
|
||||
|
||||
### Task 6: 总验证与收口
|
||||
|
||||
**Files:**
|
||||
- All modified files.
|
||||
|
||||
- [ ] **Step 1: 定向前端测试**
|
||||
- Run: `npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx src/services/image-editor/editorProjectClient.test.ts`
|
||||
- Expected: PASS。
|
||||
|
||||
- [ ] **Step 2: 定向后端测试 / check**
|
||||
- Run: `cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`
|
||||
- Run: `cargo check -p api-server --manifest-path server-rs/Cargo.toml`
|
||||
- Expected: PASS。
|
||||
|
||||
- [ ] **Step 3: 类型与编码**
|
||||
- Run: `npm run typecheck`
|
||||
- Run: `npm run check:encoding`
|
||||
- Expected: PASS。
|
||||
|
||||
- [ ] **Step 4: 检查 diff**
|
||||
- Run: `git diff --check`
|
||||
- Expected: PASS。
|
||||
@@ -18,7 +18,8 @@
|
||||
- 图片拖拽时显示水平 / 垂直吸附参考线，吸附到其它图层或画板的边缘与中心线。
|
||||
- 生成资源右上角显示元数据按钮，点击打开独立元数据窗口。
|
||||
- 对生成资源执行修改时，在右侧创建新的生成结果图层，并自动调整视图显示原图和新图。
|
||||
- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine `gpt-image-2`：纯文本生成走 `/api/editor/images/generations`，基于当前生成图的修改走 `/api/editor/images/edits`。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成工具后先在画布中心创建选中的灰色占位框，输入框跟随占位框显示；提交成功后真实生成图落在占位框位置，输入框继续跟随新生成图；基于已有生成图的修改仍通过轻量弹窗承载。前端不持有 provider 密钥；上游失败或配置缺失时只在当前生成输入框展示失败，不创建 mock 成功图。
|
||||
- 图片生成 / 修改统一经 api-server BFF 接入 VectorEngine `gpt-image-2`：纯文本生成走 `/api/editor/images/generations`，基于当前生成图的修改走 `/api/editor/images/edits`。纯文本生成入口采用 Lovart 式画布内占位图 + 锚定生成输入框：点击生成工具后先在画布中心创建选中的灰色占位框，输入框跟随占位框显示；待生成、生成中和失败后保留的占位图都必须继续支持拖动，生成完成时真实生成图落在最新占位框位置，输入框继续跟随新生成图；点击所有图片生成入口并确认请求开始后，必须隐藏对应设置面板，只保留画布内占位图或原图预览，并在预览上显示 Lovart 式生成中遮罩，避免“面板仍占屏”或“预览一起消失”。快速编辑和修改图片在调用后端前必须把当前图层图片源读取为图片 Data URL，来源可以是本地上传 Data URL、站内 public 图片、历史 `/generated-*` 路径或可读取的 OSS generated URL；后端仍只接收图片 Data URL，不把普通 URL 直接透传到 VectorEngine edits。前端不持有 provider 密钥；上游失败或配置缺失时恢复当前生成设置面板展示失败，不创建 mock 成功图。
|
||||
- 底部生成类按钮每次点击都必须创建独立的画布生成对象；新建规范、角色形象或图标素材时，只切换当前编辑面板，不得销毁此前尚未生成或已生成后的其它生成对象状态。归档为非当前编辑对象的生成占位仍可拖动、删除和等待异步完成，完成 / 失败回写必须按生成对象 ID 读取最新占位状态，不能使用提交瞬间的旧快照。
|
||||
|
||||
## 交互规则
|
||||
|
||||
@@ -61,8 +62,8 @@
|
||||
- `POST /api/editor/assets`：批量或单个创建账号级素材，支持按钮上传和拖拽上传后的 data URL / 后续 OSS 元数据。
|
||||
- `PATCH /api/editor/assets/{assetId}`：重命名素材或移动素材到文件夹。
|
||||
- `DELETE /api/editor/assets/{assetId}`：删除素材。已放入画布的 project resource 不被级联删除，避免旧画布丢图。
|
||||
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine `gpt-image-2` 生成图片，返回 data URL、尺寸、prompt、model、provider 和 taskId。
|
||||
- `POST /api/editor/images/edits`：按提示词和当前生成图 Data URL 调用 VectorEngine edits，返回新的生成图片元数据。
|
||||
- `POST /api/editor/images/generations`：按提示词调用 VectorEngine `gpt-image-2` 生成图片；携带参考图的快速编辑也走该接口，前端必须把参考图源预读成图片 Data URL 后放入 `referenceImageSrcs`；接口返回 data URL、尺寸、prompt、model、provider 和 taskId。
|
||||
- `POST /api/editor/images/edits`：按提示词和当前图片 Data URL 调用 VectorEngine edits，返回新的生成图片元数据。
|
||||
|
||||
所有写接口都必须校验 Bearer 登录态和 owner；接口只返回当前用户有权读取的工程与资源。
|
||||
|
||||
@@ -82,8 +83,11 @@
|
||||
- 默认选择模式；底部工具栏能切换工具；中键拖拽和 Space 临时抓手都能平移画布。
|
||||
- 拖拽图片接近其它图片边缘或中心时显示吸附线，并保存吸附后的最终布局。
|
||||
- 生成工具点击后显示画布内 `Image Generator` 占位框和跟随占位框的生成输入框，生成失败保留占位和输入状态，生成成功后在占位位置创建真实图层，并让输入框继续跟随该生成图。
|
||||
- 生成类入口打开画布内面板时，底部 AI 工具栏必须保持可见；`生成规范`、角色 / 图标规范来源这类轻量菜单通过页面级 fixed portal 渲染，不能留在底部工具栏或参考图横向滚动容器内部，避免被局部 `overflow` 裁切。
|
||||
- 点击生成、生成规范、生成角色形象或生成图标素材后创建的占位图可继续保留；点击画布空白区域让当前图片或占位图失焦时，关闭当前生成面板并移除图片选中样式，但不删除占位图本身。
|
||||
- 生成资源显示元数据按钮，元数据窗口展示来源、prompt、model、provider、task、尺寸和 OSS 引用。
|
||||
- 修改生成资源后，右侧出现新生成结果图层，并自动 fit 原图 + 新图。
|
||||
- 快速编辑站内 public 示例图、历史 generated 图或 OSS generated 图时，前端先读取成 `data:image/*;base64,...` 再提交，后端不得再收到 `/creation-type-references/*`、`/generated-*` 或 OSS URL 作为 `referenceImageSrcs/sourceImageSrc`。
|
||||
- 素材文件夹可以新建、折叠、重命名和删除；删除普通文件夹后，其素材移动到“项目素材”。
|
||||
- 上传按钮和拖拽上传都支持多文件；拖到文件夹或该文件夹内素材时进入目标文件夹；拖到画布时进入默认文件夹并在投放点创建画布图层。
|
||||
- 素材面板支持选择模式框选，一次选中多个素材，并可批量移动或删除上传素材。
|
||||
|
||||
@@ -49,6 +49,8 @@ Linux 本机多用户并发开发时，`npm run dev` 和 `npm run dev:*` 单模
|
||||
|
||||
后端日志默认写入 `logs/api-server/`。后端 API smoke 使用 `npm run dev:api-server` 并检查 `/healthz`；需要确认实例可接生产流量时检查 `/readyz`。不要使用旧 `api-server:maincloud` 或任何 `GENARRATIVE_SPACETIME_MAINCLOUD_*` 口径。
|
||||
|
||||
Windows 本地 `npm run dev` / `npm run dev:api-server` 会用空的 `RUSTC_WRAPPER` / `CARGO_BUILD_RUSTC_WRAPPER` 覆盖 `server-rs/.cargo/config.toml` 里的 `sccache`，从而直连真实 `rustc`。不要把 wrapper 绕过值写成 `rustc`；Cargo 会按 wrapper 协议调用 `rustc <真实rustc路径> - ...`，最终报 `multiple input filenames provided` 并导致 api-server 无法启动。排查本地启动失败时，先看 dev 日志是否出现该错误，再确认脚本注入的 wrapper 为空。
|
||||
|
||||
开发态 `npm run dev` 与 `npm run dev:api-server` 会默认注入 `GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED=true`，因此密码登录在本地开发环境可直接注册未知手机号账号；生产环境仍按 `api-server` 配置默认关闭该开关。
|
||||
|
||||
本地 `npm run dev` 和 `npm run dev:api-server` 默认保留 inline 开发体验：未显式设置 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时，外部生成 handler 会同步复用 worker executor，完成后返回 `completed`，便于快速确认 provider、OSS 和 SpacetimeDB 写回链路。inline 不创建 `external_generation_job`，也不能验证 worker lease、队列等待展示或动态扩缩容。
|
||||
|
||||
77
docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md
Normal file
77
docs/【编辑器】画板图标素材生成入口设计-2026-06-15.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# 画板图标素材生成入口设计
|
||||
|
||||
日期：`2026-06-15`
|
||||
|
||||
## 背景
|
||||
|
||||
图片画布编辑器已有普通图片生成、生成规范、生成角色形象和角色动画入口。本次新增 `生成图标素材`，用于一次输入多条图标素材描述，先生成一张绿幕 spritesheet，再自动去背、按连通域拆分为独立图标素材并铺到画布。
|
||||
|
||||
## 入口与画布表现
|
||||
|
||||
- 底部 AI 画布工具栏新增 `生成图标素材` 按钮。
|
||||
- 点击后立即在画布中心创建图标素材占位图，不复用普通“单张空白图片”图标；占位图表现为一叠空白素材图标卡片。
|
||||
- 图标素材面板锚定在占位图下方，和现有生成输入框同一层级展示。
|
||||
- 生成完成后删除占位态，把拆分出的每个独立图标素材作为画布图片图层铺开，图标之间不重叠，并保留少量间距。
|
||||
- 图标素材图层写入 `assetKind: "icon"`；图标素材规范图写入 `assetKind: "icon-spec"`，用于刷新后保留标签和限制点选来源。
|
||||
|
||||
## 面板结构
|
||||
|
||||
1. 第一模块为 `图标素材规范`。
|
||||
- 点击后弹出菜单：`从画布中选择`、`新建图标素材规范`、`上传图片`。
|
||||
- `从画布中选择` 进入画布点选状态，只允许选择 `assetKind: "icon-spec"` 的图标素材规范图片；其它图片点击无效。
|
||||
- `新建图标素材规范` 复用生成规范表单，规格类型为 `图标素材规范`，生成成功后图层标记为 `icon-spec`。
|
||||
- `上传图片` 使用现有本地图片上传入口，上传图只绑定到本次面板，不自动放入画布。
|
||||
2. 第二模块为素材描述列表。
|
||||
- 每个文本框输入一个素材描述。
|
||||
- 默认填入：`返回按钮`、`设置按钮`、`下一关按钮`、`提示按钮`、`原图按钮`、`冻结按钮`。
|
||||
- 可以继续添加新的素材描述框，最多 `100` 个。
|
||||
- 生成时过滤空文本，按面板从上到下顺序作为 prompt 的素材清单。
|
||||
|
||||
## 面板外观
|
||||
|
||||
- 图标素材面板不使用内部纵向滑动列表；素材描述项按横向卡片铺开，新增一项就让面板整体更宽，保持列表一眼可扫。
|
||||
- 图标素材规范入口采用 Lovart 式参考卡：左侧预览缩略图，中间显示当前绑定名称，右侧显示绑定状态和三个轻量动作入口，不再只是两行文字平铺。
|
||||
- 规范卡的 `从画布中选择 / 新建图标素材规范 / 上传图片` 继续保留独立菜单，但菜单只负责来源切换，不承载说明文案。
|
||||
|
||||
## 生成契约
|
||||
|
||||
- 前端提交到 `POST /api/editor/icon-spritesheets/generations`。
|
||||
- 请求字段：
|
||||
- `referenceImageSrc`：图标素材规范 Data URL。
|
||||
- `iconDescriptions`：过滤空文本后的图标描述数组，`1..100`。
|
||||
- `model`：固定 `gemini-3.1-flash-image-preview`。
|
||||
- 后端根据图标数量选择尺寸：
|
||||
- `<=25` 个：`512x512`，即 0.5K 1:1。
|
||||
- `>25` 个：`1024x1024`，即 1K 1:1。
|
||||
- 后端使用 VectorEngine 图片编辑接口，把 `referenceImageSrc` 作为参考图 1，模型固定传 `gemini-3.1-flash-image-preview`。
|
||||
- Prompt 固定为：
|
||||
|
||||
```text
|
||||
参考图1的图标素材规范，纯绿幕背景方便扣除背景，禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet：
|
||||
|
||||
<素材描述按中文顿号拼接>
|
||||
```
|
||||
|
||||
## 去背与拆分
|
||||
|
||||
- 后端收到 spritesheet 后先复用 `platform-image::generated_asset_sheets` 的绿幕去背能力。
|
||||
- 去背后基于 alpha 可见像素执行 8 邻域连通域检测。
|
||||
- 连通域按从上到下、从左到右排序；排序结果依次绑定面板中的素材描述名称。
|
||||
- 每个连通域裁剪时保留少量透明边距，并输出独立透明 PNG Data URL。
|
||||
- 若模型返回的连通域数量少于素材描述数量，接口返回失败，不在画布上铺半成品，避免名称和图标错位。
|
||||
|
||||
## 前端铺放规则
|
||||
|
||||
- 第一张图标放在原占位图左上附近。
|
||||
- 后续图标按行铺开，图标之间保留约 `24px` 世界坐标间距。
|
||||
- 每个图标图层标题使用对应素材描述文本。
|
||||
- 生成成功后关闭图标素材面板，选中第一张图标素材，并打开图层面板。
|
||||
|
||||
## 验收
|
||||
|
||||
- 点击 `生成图标素材` 后出现一叠空白图标占位和图标素材面板。
|
||||
- `图标素材规范 -> 从画布中选择` 只能选择图标素材规范图，点击普通图片或角色规范图不会绑定。
|
||||
- 默认 6 个素材描述会进入 prompt；新增描述最多到 100 个。
|
||||
- `<=25` 个描述提交时后端请求尺寸为 `512x512`；`>25` 个描述提交时后端请求尺寸为 `1024x1024`。
|
||||
- VectorEngine 请求体的 `model` 为 `gemini-3.1-flash-image-preview`。
|
||||
- 生成成功后画布出现按描述命名的多个透明图标素材图层，图层之间不重叠。
|
||||
123
docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md
Normal file
123
docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# 画板角色形象生成入口设计
|
||||
|
||||
日期：`2026-06-15`
|
||||
|
||||
## 背景
|
||||
|
||||
图片画布编辑器已有普通图片生成与“生成规范”能力。本次新增“生成角色形象”入口，用于在同一画布内生成标注为“角色”的单张角色形象图片，并支持绑定角色形象规范与常规参考图。
|
||||
|
||||
## 入口与画布表现
|
||||
|
||||
- 底部 AI 画布工具栏新增 `生成角色形象` 按钮。
|
||||
- 点击后在画布中心创建一张空白图片占位图，面板锚定在占位图下方，视觉风格复用现有生成新图片面板；底部 AI 工具栏继续保持可见，不因角色面板打开而隐藏。
|
||||
- 占位图与生成成功后的图片右上角都覆盖 `角色` 标签。
|
||||
- 角色图层在布局快照中写入 `assetKind: "character"`，刷新后仍显示 `角色` 标签。
|
||||
|
||||
## 面板结构
|
||||
|
||||
角色生成面板只包含创作必需输入；每个输入框、参考图入口和选项按钮都必须展示对应中文字段标题，不只依赖 placeholder、按钮文案或 aria-label：
|
||||
|
||||
1. 第一项参考图入口为 `角色形象规范`。
|
||||
- 入口必须采用 Lovart 式参考图卡片：左侧小预览 / 图标，中间短标题，右侧仅保留必要状态，不把说明性规则文案铺在 UI 上。
|
||||
- 点击后弹出菜单：`从画布中选择`、`新建角色形象规范`、`上传图片`。
|
||||
- 来源菜单通过页面级 fixed portal 渲染，层级高于角色面板与参考图横向滚动区，不能被 `.image-canvas-editor__character-reference-row` 裁切。
|
||||
- `从画布中选择` 进入画布点选状态，点击已有图片后把该图绑定为角色形象规范；按 `Esc` 退出点选状态。
|
||||
- `新建角色形象规范` 复用当前 `生成规范 -> 角色形象规范` 流程。
|
||||
- `上传图片` 使用现有本地图片上传入口。
|
||||
2. 规范入口后方是常规参考图入口。
|
||||
- `上传常规参考图` 同样使用 Lovart 式参考图卡片，不只显示一段文字按钮。
|
||||
- 上传后的每张常规参考图以缩略图展示。
|
||||
- 每张常规参考图右下角显示大号序号，从 `1` 开始递增。
|
||||
3. 唯一文本框为 `角色设定`。
|
||||
4. 左下角展示画面比例和大小选择按钮。
|
||||
5. 右下角展示模型选择和生成按钮。
|
||||
|
||||
## 生成与参考图契约
|
||||
|
||||
- 前端提交角色生成时，使用 `POST /api/editor/images/generations`。
|
||||
- `kind` 传 `character`，用于后端日志 / 审计语义识别。
|
||||
- 角色形象规范与常规参考图作为 `referenceImageSrcs` 传入，顺序固定为：
|
||||
1. 角色形象规范图。
|
||||
2. 常规参考图列表。
|
||||
- 当前请求尺寸沿用编辑器普通生成默认值；比例和大小按钮先复用现有占位交互。
|
||||
- 后端如果收到参考图，则走带多参考图的图片编辑/参考图生成链路；没有参考图时走纯文本生成链路。
|
||||
- `kind = "character"` 时，后端不直接把前端文本当完整生图提示词，而是把文本作为 `角色设定` 填入固定提示词骨架：
|
||||
|
||||
```text
|
||||
基于图1的角色美术视觉规范指导生成游戏角色形象图。画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止生成美术视觉规范、出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
角色设定：<用户输入的角色设定>
|
||||
```
|
||||
|
||||
- 角色图生成完成后，后端必须先对返回图片执行绿幕 / 近白背景去背，并统一输出透明背景 PNG；随后写入 OSS 私有对象，并确认 `asset_object`。接口回包仍返回透明 PNG Data URL 供画板立即显示，同时返回 `objectKey` / `assetObjectId`，前端创建图层和画板资源记录时必须保存这两个字段。
|
||||
|
||||
## 可访问性与状态
|
||||
|
||||
- 点选状态下画布显示状态提示 `请选择画布中的图片作为角色形象规范，按 Esc 退出`。
|
||||
- 已绑定的角色形象规范入口显示所选图片标题。
|
||||
- 生成中禁用参考图入口、文本框和按钮。
|
||||
- 点击画布空白区域让当前占位图或图片失焦时，关闭角色生成面板并移除图片选中样式；占位图本身保持可重新打开。删除图层逻辑沿用现有图层删除能力。
|
||||
|
||||
## 验收
|
||||
|
||||
- 点击 `生成角色形象` 后出现角色占位图、角色标签和角色生成面板。
|
||||
- 角色生成面板打开时底部 AI 工具栏仍可见；点击画布空白区域后面板关闭，当前图片不再显示选中边框。
|
||||
- `角色形象规范` 与 `上传常规参考图` 入口是带预览视觉块的参考图卡片，不是无样式文字。
|
||||
- `从画布中选择` 后点击已有画布图片可绑定为角色形象规范，`Esc` 可退出点选状态。
|
||||
- 上传常规参考图后缩略图右下角显示序号。
|
||||
- 输入角色设定并生成时，请求包含 `kind: "character"`、角色设定 prompt 和参考图数组。
|
||||
- 生成成功后在占位图位置创建 `assetKind: "character"` 图层，右上角显示 `角色` 标签，布局保存包含该字段。
|
||||
|
||||
## 当前落地记录
|
||||
|
||||
- 前端画板已接入 `生成角色形象` 底部入口、角色占位图、角色面板、画布点选规范图、上传规范图、上传常规参考图和序号角标。
|
||||
- 角色生成提交统一走 `/api/editor/images/generations`，按 `角色形象规范 -> 常规参考图` 顺序传 `referenceImageSrcs`，并写入 `assetKind: "character"`。
|
||||
- 角色生成后端已按固定 prompt 骨架补入 `角色设定`，并在生成成功后自动执行绿幕去背、写入 `generated-character-drafts/editor/character-images/<taskId>/image.png` 路径下的 OSS 私有对象，返回的 `objectKey` / `assetObjectId` 会随画板资源记录保存。
|
||||
- `Esc` 只退出角色规范画布点选状态，不关闭角色生成面板。
|
||||
- 已补充回归测试覆盖角色形象生成、点选退出、角色动画入口隔离和快速编辑入口。
|
||||
- 本次验证命令：
|
||||
- `npm test -- src/components/image-editor/ImageCanvasEditorView.test.tsx`
|
||||
- `npm test -- src/services/image-editor/editorProjectClient.test.ts`
|
||||
- `npm run typecheck -- --pretty false`
|
||||
|
||||
## 角色动画生成
|
||||
|
||||
### 入口规则
|
||||
|
||||
- 仅 `assetKind: "character"` 的角色图片图层显示 `生成动画` 入口。
|
||||
- 入口同时出现在图片上方悬浮工具条和图片右键功能列表中。
|
||||
- 非角色图片不得显示 `生成动画`，也不得通过右键菜单触发角色动画生成。
|
||||
- 点击 `生成动画` 后，在角色图片右侧打开独立的 `角色动画生成面板`；面板不在当前图片下方展开。
|
||||
|
||||
### 面板字段
|
||||
|
||||
1. 第一模块为 `动画描述` 文本框，最多输入 `4000` 字。
|
||||
2. 文本框下方提供预设动作提示词按钮：`待机`、`行走`、`奔跑`、`跳跃`、`攻击`、`受击`、`倒下`。点击后把对应动作文本写入动画描述文本框。
|
||||
3. 其它设置包括：
|
||||
- 分辨率：`480p`、`720p`。
|
||||
- 画面比例：默认 `与角色图片保持同尺寸`，可选 `1:1`、`4:3`、`16:9`、`9:16`、`3:4`。
|
||||
- 时长：`32帧·4秒`、`40帧·5秒`、`48帧·6秒`。
|
||||
4. 模型固定使用 `seedance2.0`，前端不提供模型切换。
|
||||
5. 生成按钮上方显示本次生成文本摘要和生成价格：
|
||||
- `480p` 每秒 `10` 泥点。
|
||||
- `720p` 每秒 `20` 泥点。
|
||||
|
||||
### Prompt 与生成契约
|
||||
|
||||
- 前端提交到 `POST /api/editor/character-animations/generations`。
|
||||
- 请求必须带上角色图片来源、原始尺寸、动画描述、分辨率、画面比例、帧数和时长。
|
||||
- 后端使用角色图片作为首帧和尾帧参考，模型固定映射到 seedance2.0 对应后端模型。
|
||||
- 后端 prompt 使用以下固定骨架，并把面板输入追加到 `动作描述：` 后：
|
||||
|
||||
```text
|
||||
生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。
|
||||
动作描述：
|
||||
<用户输入的动画描述>
|
||||
```
|
||||
|
||||
### 抽帧与 OSS 存储
|
||||
|
||||
- 视频生成完成后，后端按面板选择抽取对应帧数：`32`、`40` 或 `48`。
|
||||
- 每帧必须执行绿幕去背，输出透明背景 PNG。
|
||||
- 抽帧结果写入 OSS，并返回帧路径、帧尺寸、帧数、fps、预览视频路径、模型、价格和实际 prompt。
|
||||
- 画板前端首版只展示生成完成结果摘要，不把帧序列自动铺到画布上；后续若要展示逐帧图层，必须继续复用画布图层与素材库资源模型。
|
||||
|
||||
@@ -37,7 +37,10 @@ const manifestPath = resolve(serverRsDir, 'Cargo.toml');
|
||||
const modulePath = resolve(serverRsDir, 'crates/spacetime-module');
|
||||
const viteCliPath = resolve(repoRoot, 'scripts/vite-cli.mjs');
|
||||
const adminWebDir = resolve(repoRoot, 'apps/admin-web');
|
||||
const LOCAL_DEV_RUSTC_WRAPPER_BYPASS = process.platform === 'win32' ? 'rustc' : '/usr/bin/env';
|
||||
function resolveLocalDevRustcWrapperBypass() {
|
||||
// Windows 下不能把 rustc 自身当成 Cargo wrapper；空值会覆盖仓库 .cargo/config.toml 中的 sccache。
|
||||
return process.platform === 'win32' ? '' : '/usr/bin/env';
|
||||
}
|
||||
|
||||
const SERVICE_NAMES = ['spacetime', 'api-server', 'web', 'admin-web'];
|
||||
const SERVICE_ALIASES = new Map([
|
||||
@@ -423,8 +426,9 @@ function buildLocalRustProcessEnv(env, options = {}) {
|
||||
return mergedEnv;
|
||||
}
|
||||
|
||||
mergedEnv.RUSTC_WRAPPER = LOCAL_DEV_RUSTC_WRAPPER_BYPASS;
|
||||
mergedEnv.CARGO_BUILD_RUSTC_WRAPPER = LOCAL_DEV_RUSTC_WRAPPER_BYPASS;
|
||||
const rustcWrapperBypass = resolveLocalDevRustcWrapperBypass();
|
||||
mergedEnv.RUSTC_WRAPPER = rustcWrapperBypass;
|
||||
mergedEnv.CARGO_BUILD_RUSTC_WRAPPER = rustcWrapperBypass;
|
||||
if (options.log !== false) {
|
||||
console.warn(
|
||||
'[dev:rust] 本地 dev 构建绕过项目 sccache wrapper，避免缓存进程异常阻断启动。',
|
||||
|
||||
@@ -213,6 +213,31 @@ describe('dev scheduler Rust build env', () => {
|
||||
expect(env.RUSTC_WRAPPER).toBe('custom-wrapper');
|
||||
expect(env.CARGO_BUILD_RUSTC_WRAPPER).toBe('custom-wrapper');
|
||||
});
|
||||
|
||||
test('Windows 下本地 dev Rust env 用空 wrapper 覆盖项目 sccache', () => {
|
||||
const originalPlatform = Object.getOwnPropertyDescriptor(process, 'platform');
|
||||
Object.defineProperty(process, 'platform', {
|
||||
configurable: true,
|
||||
value: 'win32',
|
||||
});
|
||||
|
||||
try {
|
||||
const env = buildLocalRustProcessEnv(
|
||||
{
|
||||
RUSTC_WRAPPER: 'sccache',
|
||||
CARGO_BUILD_RUSTC_WRAPPER: 'sccache',
|
||||
},
|
||||
{log: false},
|
||||
);
|
||||
|
||||
expect(env.RUSTC_WRAPPER).toBe('');
|
||||
expect(env.CARGO_BUILD_RUSTC_WRAPPER).toBe('');
|
||||
} finally {
|
||||
if (originalPlatform) {
|
||||
Object.defineProperty(process, 'platform', originalPlatform);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('dev scheduler stack state file', () => {
|
||||
|
||||
@@ -41,6 +41,8 @@ use shared_contracts::assets::{
|
||||
CharacterRoleAssetWorkflowResolveRequest, CharacterRoleAssetWorkflowResponse,
|
||||
CharacterVisualDraftPayload, CharacterWorkflowCacheGetResponse, CharacterWorkflowCachePayload,
|
||||
CharacterWorkflowCacheSaveRequest, CharacterWorkflowCacheSaveResponse,
|
||||
EditorCharacterAnimationFramePayload, EditorCharacterAnimationGenerateRequest,
|
||||
EditorCharacterAnimationGenerateResponse,
|
||||
};
|
||||
use spacetime_client::SpacetimeClientError;
|
||||
|
||||
@@ -82,6 +84,9 @@ const FIXED_ARK_CHARACTER_VIDEO_RESOLUTION: &str = "480p";
|
||||
const FIXED_ARK_CHARACTER_VIDEO_RATIO: &str = "1:1";
|
||||
const FIXED_ARK_CHARACTER_VIDEO_DURATION_SECONDS: u32 = 4;
|
||||
const ARK_VIDEO_TASK_POLL_INTERVAL_MS: u64 = 5_000;
|
||||
const EDITOR_CHARACTER_ANIMATION_MODEL: &str = "seedance2.0";
|
||||
const EDITOR_CHARACTER_ANIMATION_ASSET_KIND: &str = "editor_character_animation";
|
||||
const EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX: &str = "生成游戏角色动画，参考图作为首帧和尾帧，画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。\n动作描述：";
|
||||
|
||||
const BUILT_IN_MOTION_TEMPLATES: [MotionTemplate; 4] = [
|
||||
MotionTemplate {
|
||||
@@ -489,6 +494,87 @@ pub async fn generate_character_animation(
|
||||
))
|
||||
}
|
||||
|
||||
pub async fn generate_editor_character_animation(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
payload: Result<Json<EditorCharacterAnimationGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = payload.map_err(|error| {
|
||||
character_animation_error_response(
|
||||
&request_context,
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-character-animation",
|
||||
"message": error.body_text(),
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
|
||||
let normalized = normalize_editor_character_animation_request(payload)
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let settings = require_editor_character_animation_settings(&state, &normalized)
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let extraction_settings = resolve_backend_frame_extraction_settings(&state);
|
||||
let http_client = build_upstream_http_client(settings.ark.request_timeout_ms)
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let owner_user_id = "editor-character-animation".to_string();
|
||||
let task_id = generate_ai_task_id(current_utc_micros());
|
||||
|
||||
let result = async {
|
||||
let source_data_url = resolve_media_source_as_data_url(
|
||||
&state,
|
||||
&http_client,
|
||||
normalized.source_image_src.as_str(),
|
||||
"sourceImageSrc",
|
||||
)
|
||||
.await?;
|
||||
let generated = request_editor_character_animation_preview(
|
||||
&state,
|
||||
&http_client,
|
||||
&settings,
|
||||
owner_user_id.as_str(),
|
||||
normalized.source_layer_id.as_str(),
|
||||
task_id.as_str(),
|
||||
normalized.prompt.as_str(),
|
||||
source_data_url.as_str(),
|
||||
)
|
||||
.await?;
|
||||
let frames = extract_and_persist_editor_character_animation_frames(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
normalized.source_layer_id.as_str(),
|
||||
task_id.as_str(),
|
||||
generated.preview_video_path.as_str(),
|
||||
&normalized,
|
||||
&extraction_settings,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok::<_, AppError>((generated, frames))
|
||||
}
|
||||
.await;
|
||||
|
||||
let (generated, frames) = result
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorCharacterAnimationGenerateResponse {
|
||||
ok: true,
|
||||
task_id,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
prompt: generated.submitted_prompt,
|
||||
preview_video_path: generated.preview_video_path,
|
||||
frame_count: frames.len() as u32,
|
||||
duration_seconds: normalized.duration_seconds,
|
||||
frame_width: normalized.frame_width,
|
||||
frame_height: normalized.frame_height,
|
||||
fps: normalized.fps,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
frames,
|
||||
},
|
||||
))
|
||||
}
|
||||
|
||||
pub async fn get_character_animation_job(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
@@ -1248,6 +1334,167 @@ async fn put_generated_preview_video(
|
||||
Ok(put_result.legacy_public_path)
|
||||
}
|
||||
|
||||
async fn request_editor_character_animation_preview(
|
||||
state: &AppState,
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorCharacterAnimationSettings,
|
||||
owner_user_id: &str,
|
||||
source_layer_id: &str,
|
||||
task_id: &str,
|
||||
prompt: &str,
|
||||
source_frame_data_url: &str,
|
||||
) -> Result<GeneratedAnimationPreview, AppError> {
|
||||
let upstream_task_id =
|
||||
create_editor_ark_image_to_video_task(http_client, settings, prompt, source_frame_data_url)
|
||||
.await?;
|
||||
let video_url =
|
||||
wait_for_ark_content_generation_task(http_client, &settings.ark, upstream_task_id.as_str())
|
||||
.await?;
|
||||
let preview_payload =
|
||||
download_generated_video(http_client, video_url.as_str(), "下载画板角色动画视频失败。")
|
||||
.await?;
|
||||
let preview_video_path = put_generated_preview_video(
|
||||
state,
|
||||
owner_user_id,
|
||||
source_layer_id,
|
||||
"editor-character-animation",
|
||||
task_id,
|
||||
preview_payload,
|
||||
)
|
||||
.await?;
|
||||
|
||||
Ok(GeneratedAnimationPreview {
|
||||
preview_video_path,
|
||||
upstream_task_id,
|
||||
submitted_prompt: prompt.to_string(),
|
||||
moderation_fallback_applied: false,
|
||||
})
|
||||
}
|
||||
|
||||
async fn create_editor_ark_image_to_video_task(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorCharacterAnimationSettings,
|
||||
prompt: &str,
|
||||
source_frame_data_url: &str,
|
||||
) -> Result<String, AppError> {
|
||||
let response = http_client
|
||||
.post(format!("{}/contents/generations/tasks", settings.ark.base_url))
|
||||
.header(
|
||||
reqwest::header::AUTHORIZATION,
|
||||
format!("Bearer {}", settings.ark.api_key),
|
||||
)
|
||||
.header(reqwest::header::CONTENT_TYPE, "application/json")
|
||||
.json(&json!({
|
||||
"model": settings.ark.model,
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": prompt,
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": source_frame_data_url,
|
||||
},
|
||||
"role": "first_frame",
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": source_frame_data_url,
|
||||
},
|
||||
"role": "last_frame",
|
||||
}
|
||||
],
|
||||
"resolution": settings.resolution,
|
||||
"ratio": settings.ratio,
|
||||
"duration": settings.duration_seconds,
|
||||
"watermark": false,
|
||||
}))
|
||||
.send()
|
||||
.await
|
||||
.map_err(|error| {
|
||||
map_character_animation_upstream_error(format!("请求 Ark 视频服务失败：{error}"))
|
||||
})?;
|
||||
|
||||
let status = response.status();
|
||||
let body = response.text().await.map_err(|error| {
|
||||
map_character_animation_upstream_error(format!("读取 Ark 视频任务响应失败：{error}"))
|
||||
})?;
|
||||
if !status.is_success() {
|
||||
return Err(parse_animation_upstream_error(
|
||||
body.as_str(),
|
||||
"创建画板角色动画视频任务失败。",
|
||||
));
|
||||
}
|
||||
let payload = parse_animation_json_payload(body.as_str(), "创建画板角色动画视频任务失败。")?;
|
||||
extract_animation_task_id(&payload.payload).ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "ark",
|
||||
"message": "画板角色动画视频任务未返回任务 id。",
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
async fn extract_and_persist_editor_character_animation_frames(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
source_layer_id: &str,
|
||||
task_id: &str,
|
||||
preview_video_path: &str,
|
||||
request: &NormalizedEditorCharacterAnimationRequest,
|
||||
extraction_settings: &BackendFrameExtractionSettings,
|
||||
) -> Result<Vec<EditorCharacterAnimationFramePayload>, AppError> {
|
||||
let plan = AnimationFrameExtractionPlan {
|
||||
frame_count: request.frame_count,
|
||||
apply_chroma_key: true,
|
||||
sample_start_ratio: 0.0,
|
||||
sample_end_ratio: 1.0,
|
||||
};
|
||||
let finalized_frames = extract_animation_frames_from_preview_video(
|
||||
state,
|
||||
preview_video_path,
|
||||
request.frame_width,
|
||||
request.frame_height,
|
||||
extraction_settings,
|
||||
&plan,
|
||||
)
|
||||
.await?;
|
||||
|
||||
let mut frame_payloads = Vec::with_capacity(finalized_frames.len());
|
||||
for (index, frame) in finalized_frames.into_iter().enumerate() {
|
||||
let put_result = put_character_animation_object(
|
||||
state,
|
||||
LegacyAssetPrefix::Animations,
|
||||
vec![
|
||||
"editor".to_string(),
|
||||
sanitize_storage_segment(source_layer_id, "layer"),
|
||||
task_id.to_string(),
|
||||
],
|
||||
format!("frame{:02}.{}", index + 1, frame.extension),
|
||||
frame.mime_type,
|
||||
frame.bytes,
|
||||
build_asset_metadata(
|
||||
EDITOR_CHARACTER_ANIMATION_ASSET_KIND,
|
||||
owner_user_id,
|
||||
"editor_layer",
|
||||
source_layer_id,
|
||||
"animation_frame",
|
||||
"editor-character-animation",
|
||||
),
|
||||
)
|
||||
.await?;
|
||||
frame_payloads.push(EditorCharacterAnimationFramePayload {
|
||||
frame_index: index as u32 + 1,
|
||||
image_src: put_result.legacy_public_path,
|
||||
width: request.frame_width,
|
||||
height: request.frame_height,
|
||||
});
|
||||
}
|
||||
|
||||
Ok(frame_payloads)
|
||||
}
|
||||
|
||||
async fn publish_animation_set(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
@@ -1864,6 +2111,235 @@ fn resolve_character_animation_model(payload: &CharacterAnimationGenerateRequest
|
||||
normalize_required_text(candidate, CHARACTER_ANIMATION_MODEL)
|
||||
}
|
||||
|
||||
fn normalize_editor_character_animation_request(
|
||||
payload: EditorCharacterAnimationGenerateRequest,
|
||||
) -> Result<NormalizedEditorCharacterAnimationRequest, AppError> {
|
||||
let source_layer_id = normalize_required_text(payload.source_layer_id.as_str(), "");
|
||||
if source_layer_id.is_empty() {
|
||||
return Err(editor_character_animation_bad_request(
|
||||
"sourceLayerId 不能为空。",
|
||||
));
|
||||
}
|
||||
let source_image_src = trim_optional_text(Some(payload.source_image_src.as_str()))
|
||||
.ok_or_else(|| editor_character_animation_bad_request("sourceImageSrc 不能为空。"))?;
|
||||
let prompt_text = payload.prompt_text.trim().chars().take(4000).collect::<String>();
|
||||
if prompt_text.is_empty() {
|
||||
return Err(editor_character_animation_bad_request("动画描述不能为空。"));
|
||||
}
|
||||
let resolution = normalize_editor_character_animation_resolution(payload.resolution.as_str())?;
|
||||
let ratio = normalize_editor_character_animation_ratio(payload.ratio.as_str())?;
|
||||
let frame_count = normalize_editor_character_animation_frame_count(payload.frame_count)?;
|
||||
let duration_seconds =
|
||||
normalize_editor_character_animation_duration(payload.duration_seconds, frame_count)?;
|
||||
let expected_price = calculate_editor_character_animation_price(resolution, duration_seconds);
|
||||
if payload.price_mud_points != expected_price {
|
||||
return Err(editor_character_animation_bad_request(format!(
|
||||
"priceMudPoints 与分辨率和时长不一致，应为 {expected_price}。"
|
||||
)));
|
||||
}
|
||||
if payload.model.trim() != EDITOR_CHARACTER_ANIMATION_MODEL {
|
||||
return Err(editor_character_animation_bad_request(
|
||||
"model 必须固定为 seedance2.0。",
|
||||
));
|
||||
}
|
||||
let (frame_width, frame_height) = resolve_editor_character_animation_frame_size(
|
||||
payload.source_width,
|
||||
payload.source_height,
|
||||
ratio,
|
||||
resolution,
|
||||
);
|
||||
let prompt = build_editor_character_animation_prompt(prompt_text.as_str());
|
||||
|
||||
Ok(NormalizedEditorCharacterAnimationRequest {
|
||||
source_layer_id,
|
||||
source_image_src,
|
||||
prompt,
|
||||
resolution: resolution.to_string(),
|
||||
ratio: resolve_editor_character_animation_provider_ratio(
|
||||
ratio,
|
||||
payload.source_width,
|
||||
payload.source_height,
|
||||
),
|
||||
frame_count,
|
||||
duration_seconds,
|
||||
price_mud_points: expected_price,
|
||||
frame_width,
|
||||
frame_height,
|
||||
fps: (frame_count / duration_seconds).max(1),
|
||||
})
|
||||
}
|
||||
|
||||
fn require_editor_character_animation_settings(
|
||||
state: &AppState,
|
||||
request: &NormalizedEditorCharacterAnimationRequest,
|
||||
) -> Result<EditorCharacterAnimationSettings, AppError> {
|
||||
let base_url = state
|
||||
.config
|
||||
.ark_character_video_base_url
|
||||
.trim()
|
||||
.trim_end_matches('/');
|
||||
if base_url.is_empty() {
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "ark",
|
||||
"reason": "ARK_CHARACTER_VIDEO_BASE_URL 未配置",
|
||||
})),
|
||||
);
|
||||
}
|
||||
let api_key = state
|
||||
.config
|
||||
.ark_character_video_api_key
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "ark",
|
||||
"reason": "ARK_CHARACTER_VIDEO_API_KEY 未配置",
|
||||
}))
|
||||
})?;
|
||||
// 中文注释：画板角色动画入口产品侧固定为 seedance2.0，不继承通用角色视频环境模型覆盖。
|
||||
Ok(EditorCharacterAnimationSettings {
|
||||
ark: ArkVideoSettings {
|
||||
base_url: base_url.to_string(),
|
||||
api_key: api_key.to_string(),
|
||||
request_timeout_ms: state.config.ark_character_video_request_timeout_ms.max(1),
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
},
|
||||
resolution: request.resolution.clone(),
|
||||
ratio: request.ratio.clone(),
|
||||
duration_seconds: request.duration_seconds,
|
||||
})
|
||||
}
|
||||
|
||||
fn build_editor_character_animation_prompt(prompt_text: &str) -> String {
|
||||
format!(
|
||||
"{}\n{}",
|
||||
EDITOR_CHARACTER_ANIMATION_PROMPT_PREFIX,
|
||||
prompt_text.trim()
|
||||
)
|
||||
}
|
||||
|
||||
fn normalize_editor_character_animation_resolution(value: &str) -> Result<&'static str, AppError> {
|
||||
match value.trim() {
|
||||
"480p" => Ok("480p"),
|
||||
"720p" => Ok("720p"),
|
||||
_ => Err(editor_character_animation_bad_request(
|
||||
"resolution 只支持 480p 或 720p。",
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_character_animation_ratio(value: &str) -> Result<&'static str, AppError> {
|
||||
match value.trim() {
|
||||
"same" => Ok("same"),
|
||||
"1:1" => Ok("1:1"),
|
||||
"4:3" => Ok("4:3"),
|
||||
"16:9" => Ok("16:9"),
|
||||
"9:16" => Ok("9:16"),
|
||||
"3:4" => Ok("3:4"),
|
||||
_ => Err(editor_character_animation_bad_request(
|
||||
"ratio 只支持 same、1:1、4:3、16:9、9:16、3:4。",
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_character_animation_frame_count(value: u32) -> Result<u32, AppError> {
|
||||
match value {
|
||||
32 | 40 | 48 => Ok(value),
|
||||
_ => Err(editor_character_animation_bad_request(
|
||||
"frameCount 只支持 32、40、48。",
|
||||
)),
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_editor_character_animation_duration(
|
||||
value: u32,
|
||||
frame_count: u32,
|
||||
) -> Result<u32, AppError> {
|
||||
let expected = match frame_count {
|
||||
32 => 4,
|
||||
40 => 5,
|
||||
48 => 6,
|
||||
_ => 0,
|
||||
};
|
||||
if value == expected {
|
||||
Ok(value)
|
||||
} else {
|
||||
Err(editor_character_animation_bad_request(
|
||||
"durationSeconds 必须与帧数组合为 32/4、40/5 或 48/6。",
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
fn calculate_editor_character_animation_price(resolution: &str, duration_seconds: u32) -> u32 {
|
||||
let per_second = if resolution == "720p" { 20 } else { 10 };
|
||||
per_second * duration_seconds
|
||||
}
|
||||
|
||||
fn resolve_editor_character_animation_provider_ratio(
|
||||
ratio: &str,
|
||||
source_width: u32,
|
||||
source_height: u32,
|
||||
) -> String {
|
||||
if ratio != "same" {
|
||||
return ratio.to_string();
|
||||
}
|
||||
if source_width == 0 || source_height == 0 {
|
||||
return "1:1".to_string();
|
||||
}
|
||||
let normalized = source_width as f32 / source_height as f32;
|
||||
[
|
||||
("1:1", 1.0f32),
|
||||
("4:3", 4.0 / 3.0),
|
||||
("16:9", 16.0 / 9.0),
|
||||
("9:16", 9.0 / 16.0),
|
||||
("3:4", 3.0 / 4.0),
|
||||
]
|
||||
.into_iter()
|
||||
.min_by(|(_, left), (_, right)| {
|
||||
(normalized - *left)
|
||||
.abs()
|
||||
.partial_cmp(&(normalized - *right).abs())
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
})
|
||||
.map(|(value, _)| value.to_string())
|
||||
.unwrap_or_else(|| "1:1".to_string())
|
||||
}
|
||||
|
||||
fn resolve_editor_character_animation_frame_size(
|
||||
source_width: u32,
|
||||
source_height: u32,
|
||||
ratio: &str,
|
||||
resolution: &str,
|
||||
) -> (u32, u32) {
|
||||
let long_edge = if resolution == "720p" { 720 } else { 480 };
|
||||
let (ratio_width, ratio_height) = match ratio {
|
||||
"same" if source_width > 0 && source_height > 0 => (source_width, source_height),
|
||||
"4:3" => (4, 3),
|
||||
"16:9" => (16, 9),
|
||||
"9:16" => (9, 16),
|
||||
"3:4" => (3, 4),
|
||||
_ => (1, 1),
|
||||
};
|
||||
if ratio_width >= ratio_height {
|
||||
let height = ((long_edge as f32 * ratio_height as f32 / ratio_width as f32).round() as u32)
|
||||
.max(1);
|
||||
(long_edge, height)
|
||||
} else {
|
||||
let width = ((long_edge as f32 * ratio_width as f32 / ratio_height as f32).round() as u32)
|
||||
.max(1);
|
||||
(width, long_edge)
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_character_animation_bad_request(message: impl Into<String>) -> AppError {
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-character-animation",
|
||||
"message": message.into(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn build_animation_generate_result_payload(generated: &CharacterAnimationGeneratedDraft) -> Value {
|
||||
match generated.preview_video_path.as_ref() {
|
||||
Some(preview_video_path) => json!({
|
||||
@@ -3353,6 +3829,28 @@ struct ArkVideoSettings {
|
||||
model: String,
|
||||
}
|
||||
|
||||
struct EditorCharacterAnimationSettings {
|
||||
ark: ArkVideoSettings,
|
||||
resolution: String,
|
||||
ratio: String,
|
||||
duration_seconds: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct NormalizedEditorCharacterAnimationRequest {
|
||||
source_layer_id: String,
|
||||
source_image_src: String,
|
||||
prompt: String,
|
||||
resolution: String,
|
||||
ratio: String,
|
||||
frame_count: u32,
|
||||
duration_seconds: u32,
|
||||
price_mud_points: u32,
|
||||
frame_width: u32,
|
||||
frame_height: u32,
|
||||
fps: u32,
|
||||
}
|
||||
|
||||
struct GeneratedAnimationPreview {
|
||||
preview_video_path: String,
|
||||
upstream_task_id: String,
|
||||
@@ -3548,4 +4046,73 @@ mod tests {
|
||||
|
||||
assert_eq!(resolve_character_animation_model(&payload), "wan-move");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_normalizes_seedance_request_contract() {
|
||||
let normalized =
|
||||
normalize_editor_character_animation_request(EditorCharacterAnimationGenerateRequest {
|
||||
source_layer_id: " layer-hero ".to_string(),
|
||||
source_image_src: "/generated-characters/hero/master.png".to_string(),
|
||||
source_width: 768,
|
||||
source_height: 1024,
|
||||
prompt_text: "待机呼吸，轻微摆动。".to_string(),
|
||||
resolution: "720p".to_string(),
|
||||
ratio: "same".to_string(),
|
||||
frame_count: 48,
|
||||
duration_seconds: 6,
|
||||
price_mud_points: 120,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
})
|
||||
.expect("editor request should normalize");
|
||||
|
||||
assert_eq!(normalized.source_layer_id, "layer-hero");
|
||||
assert_eq!(normalized.frame_count, 48);
|
||||
assert_eq!(normalized.duration_seconds, 6);
|
||||
assert_eq!(normalized.price_mud_points, 120);
|
||||
assert_eq!(normalized.fps, 8);
|
||||
assert_eq!(normalized.frame_width, 540);
|
||||
assert_eq!(normalized.frame_height, 720);
|
||||
assert_eq!(normalized.ratio, "3:4");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_rejects_invalid_frame_duration_pair() {
|
||||
let error =
|
||||
normalize_editor_character_animation_request(EditorCharacterAnimationGenerateRequest {
|
||||
source_layer_id: "layer-hero".to_string(),
|
||||
source_image_src: "/generated-characters/hero/master.png".to_string(),
|
||||
source_width: 1024,
|
||||
source_height: 1024,
|
||||
prompt_text: "奔跑".to_string(),
|
||||
resolution: "480p".to_string(),
|
||||
ratio: "1:1".to_string(),
|
||||
frame_count: 48,
|
||||
duration_seconds: 4,
|
||||
price_mud_points: 40,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
})
|
||||
.expect_err("invalid frame/duration pair should fail");
|
||||
|
||||
assert!(
|
||||
error
|
||||
.body_text()
|
||||
.contains("durationSeconds 必须与帧数组合")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_builds_required_green_screen_prompt() {
|
||||
let prompt = build_editor_character_animation_prompt("行走两步后回到站姿。");
|
||||
|
||||
assert!(prompt.contains("生成游戏角色动画"));
|
||||
assert!(prompt.contains("参考图作为首帧和尾帧"));
|
||||
assert!(prompt.contains("背景固定为纯绿色绿幕"));
|
||||
assert!(prompt.contains("动作描述：\n行走两步后回到站姿。"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_price_depends_on_resolution_and_duration() {
|
||||
assert_eq!(calculate_editor_character_animation_price("480p", 4), 40);
|
||||
assert_eq!(calculate_editor_character_animation_price("720p", 6), 120);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,18 +1,31 @@
|
||||
use std::{borrow::Cow, collections::BTreeMap};
|
||||
|
||||
use axum::{
|
||||
Json,
|
||||
extract::{Extension, Path, State},
|
||||
http::StatusCode,
|
||||
};
|
||||
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64_STANDARD};
|
||||
use module_assets::{
|
||||
AssetObjectAccessPolicy, AssetObjectFieldError, build_asset_object_upsert_input,
|
||||
generate_asset_object_id,
|
||||
};
|
||||
use platform_image::{
|
||||
DownloadedImage,
|
||||
generated_asset_sheets::{
|
||||
GeneratedAssetSheetConnectedIcon, slice_generated_icon_spritesheet_by_connected_components,
|
||||
},
|
||||
};
|
||||
use platform_oss::{LegacyAssetPrefix, OssHeadObjectRequest, OssObjectAccess};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{Value, json};
|
||||
use shared_kernel::build_prefixed_uuid_id;
|
||||
use spacetime_client::{
|
||||
EditorAssetCreateRecordInput, EditorAssetDeleteRecordInput, EditorAssetFolderCreateRecordInput,
|
||||
EditorAssetFolderDeleteRecordInput, EditorAssetFolderRecord, EditorAssetFolderUpdateRecordInput,
|
||||
EditorAssetLibraryRecord, EditorAssetRecord, EditorAssetUpdateRecordInput, EditorCanvasRecord,
|
||||
EditorCanvasViewportRecord, EditorProjectCreateRecordInput, EditorProjectDeleteRecordInput,
|
||||
EditorProjectGetRecordInput,
|
||||
EditorAssetFolderDeleteRecordInput, EditorAssetFolderRecord,
|
||||
EditorAssetFolderUpdateRecordInput, EditorAssetLibraryRecord, EditorAssetRecord,
|
||||
EditorAssetUpdateRecordInput, EditorCanvasRecord, EditorCanvasViewportRecord,
|
||||
EditorProjectCreateRecordInput, EditorProjectDeleteRecordInput, EditorProjectGetRecordInput,
|
||||
EditorProjectLayoutSaveRecordInput, EditorProjectRecord, EditorProjectRenameRecordInput,
|
||||
EditorProjectResourceCreateRecordInput, EditorProjectResourceRecord, SpacetimeClientError,
|
||||
};
|
||||
@@ -20,12 +33,19 @@ use spacetime_client::{
|
||||
use crate::{
|
||||
api_response::json_success_body,
|
||||
auth::AuthenticatedAccessToken,
|
||||
generated_image_assets::{
|
||||
GeneratedImageAssetAdapter, GeneratedImageAssetDataUrl,
|
||||
adapter::{GeneratedImageAssetAdapterMetadata, GeneratedImageAssetPersistInput},
|
||||
normalize_generated_image_asset_mime,
|
||||
},
|
||||
http_error::AppError,
|
||||
openai_image_generation::{
|
||||
GPT_IMAGE_2_MODEL, OpenAiReferenceImage, build_openai_image_http_client,
|
||||
create_openai_image_edit_with_references, create_openai_image_generation,
|
||||
DownloadedOpenAiImage, GPT_IMAGE_2_MODEL, OpenAiReferenceImage,
|
||||
build_openai_image_http_client, create_openai_image_edit_with_references,
|
||||
create_openai_image_edit_with_references_and_model, create_openai_image_generation,
|
||||
require_openai_image_settings,
|
||||
},
|
||||
platform_errors::map_oss_error,
|
||||
request_context::RequestContext,
|
||||
state::AppState,
|
||||
};
|
||||
@@ -37,6 +57,13 @@ const EDITOR_ASSET_ID_PREFIX: &str = "editor-asset-";
|
||||
const EDITOR_LAYOUT_MAX_BYTES: usize = 256 * 1024;
|
||||
const EDITOR_PROJECT_DEFAULT_TITLE: &str = "未命名画布";
|
||||
const EDITOR_IMAGE_GENERATION_SIZE: &str = "1024x1024";
|
||||
const EDITOR_ICON_SPRITESHEET_MODEL: &str = "gemini-3.1-flash-image-preview";
|
||||
const EDITOR_ICON_SPRITESHEET_SMALL_SIZE: &str = "512x512";
|
||||
const EDITOR_ICON_SPRITESHEET_LARGE_SIZE: &str = "1024x1024";
|
||||
const EDITOR_ICON_DESCRIPTION_LIMIT: usize = 100;
|
||||
const EDITOR_CHARACTER_IMAGE_ASSET_KIND: &str = "editor_character_image";
|
||||
const EDITOR_CHARACTER_IMAGE_ENTITY_KIND: &str = "editor_project";
|
||||
const EDITOR_CHARACTER_IMAGE_SLOT: &str = "character";
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
@@ -125,6 +152,10 @@ pub struct EditorAssetUpdateRequest {
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorImageGenerationRequest {
|
||||
prompt: String,
|
||||
size: Option<String>,
|
||||
kind: Option<String>,
|
||||
model: Option<String>,
|
||||
reference_image_srcs: Option<Vec<String>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
@@ -134,6 +165,14 @@ pub struct EditorImageEditRequest {
|
||||
source_image_src: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorIconSpritesheetGenerationRequest {
|
||||
reference_image_src: String,
|
||||
icon_descriptions: Vec<String>,
|
||||
model: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorProjectResponse {
|
||||
@@ -192,6 +231,8 @@ pub struct EditorAssetResponse {
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorImageGenerationResponse {
|
||||
image_src: String,
|
||||
object_key: Option<String>,
|
||||
asset_object_id: Option<String>,
|
||||
width: u32,
|
||||
height: u32,
|
||||
source_type: &'static str,
|
||||
@@ -202,6 +243,29 @@ pub struct EditorImageGenerationResponse {
|
||||
task_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorIconSpritesheetIconResponse {
|
||||
name: String,
|
||||
image_src: String,
|
||||
width: u32,
|
||||
height: u32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorIconSpritesheetGenerationResponse {
|
||||
spritesheet_image_src: String,
|
||||
spritesheet_width: u32,
|
||||
spritesheet_height: u32,
|
||||
icon_image_srcs: Vec<EditorIconSpritesheetIconResponse>,
|
||||
prompt: String,
|
||||
actual_prompt: Option<String>,
|
||||
model: String,
|
||||
provider: &'static str,
|
||||
task_id: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorProjectPayload {
|
||||
@@ -683,8 +747,8 @@ pub async fn generate_editor_image(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
Json(payload): Json<EditorImageGenerationRequest>,
|
||||
) -> Result<Json<Value>, AppError> {
|
||||
let prompt = payload.prompt.trim().to_string();
|
||||
if prompt.is_empty() {
|
||||
let role_setting = payload.prompt.trim().to_string();
|
||||
if role_setting.is_empty() {
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-image-generation",
|
||||
@@ -693,33 +757,93 @@ pub async fn generate_editor_image(
|
||||
);
|
||||
}
|
||||
|
||||
let image_size = normalize_editor_image_generation_size(payload.size.as_deref());
|
||||
let _requested_model = payload.model.as_deref();
|
||||
let normalized_kind = payload.kind.as_deref().map(str::trim);
|
||||
let is_character_generation = matches!(normalized_kind, Some("character"));
|
||||
let submitted_prompt = if is_character_generation {
|
||||
build_editor_character_image_prompt(role_setting.as_str())
|
||||
} else {
|
||||
role_setting.clone()
|
||||
};
|
||||
let failure_context = match normalized_kind {
|
||||
Some("character") => "图片画布生成角色形象",
|
||||
Some("spec") => "图片画布生成规范",
|
||||
Some("quick-edit") => "图片画布快速编辑图片",
|
||||
_ => "图片画布生成图片",
|
||||
};
|
||||
let reference_sources = payload
|
||||
.reference_image_srcs
|
||||
.unwrap_or_default()
|
||||
.into_iter()
|
||||
.map(|source| source.trim().to_string())
|
||||
.filter(|source| !source.is_empty())
|
||||
.take(5)
|
||||
.collect::<Vec<_>>();
|
||||
let settings = require_openai_image_settings(&state)?.with_external_api_audit_context(
|
||||
&request_context,
|
||||
Some(authenticated.claims().user_id().to_string()),
|
||||
None,
|
||||
);
|
||||
let http_client = build_openai_image_http_client(&settings)?;
|
||||
let generated = create_openai_image_generation(
|
||||
&http_client,
|
||||
&settings,
|
||||
prompt.as_str(),
|
||||
Some("文字、水印、边框、按钮、UI 控件、低清晰度、变形主体"),
|
||||
EDITOR_IMAGE_GENERATION_SIZE,
|
||||
1,
|
||||
&[],
|
||||
"图片画布生成图片",
|
||||
)
|
||||
.await?;
|
||||
let image = generated.images.into_iter().next().ok_or_else(|| {
|
||||
let negative_prompt = Some("文字、水印、边框、按钮、UI 控件、低清晰度、变形主体");
|
||||
let generated = if reference_sources.is_empty() {
|
||||
create_openai_image_generation(
|
||||
&http_client,
|
||||
&settings,
|
||||
submitted_prompt.as_str(),
|
||||
negative_prompt,
|
||||
image_size.as_ref(),
|
||||
1,
|
||||
&[],
|
||||
failure_context,
|
||||
)
|
||||
.await?
|
||||
} else {
|
||||
let reference_images = reference_sources
|
||||
.iter()
|
||||
.map(|source| parse_editor_reference_image(source.as_str()))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
create_openai_image_edit_with_references(
|
||||
&http_client,
|
||||
&settings,
|
||||
submitted_prompt.as_str(),
|
||||
negative_prompt,
|
||||
image_size.as_ref(),
|
||||
1,
|
||||
reference_images.as_slice(),
|
||||
failure_context,
|
||||
)
|
||||
.await?
|
||||
};
|
||||
let mut image = generated.images.into_iter().next().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "vector-engine",
|
||||
"message": "VectorEngine 未返回图片",
|
||||
}))
|
||||
})?;
|
||||
if is_character_generation {
|
||||
image = prepare_editor_character_image_for_response(image);
|
||||
}
|
||||
|
||||
let (width, height) = image::load_from_memory(image.bytes.as_slice())
|
||||
.map(|image| (image.width(), image.height()))
|
||||
.unwrap_or((1024, 1024));
|
||||
let persisted = if is_character_generation {
|
||||
Some(
|
||||
persist_editor_character_image(
|
||||
&state,
|
||||
authenticated.claims().user_id(),
|
||||
generated.task_id.as_str(),
|
||||
&image,
|
||||
submitted_prompt.as_str(),
|
||||
generated.actual_prompt.as_deref(),
|
||||
)
|
||||
.await?,
|
||||
)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let image_src = format!(
|
||||
"data:{};base64,{}",
|
||||
image.mime_type,
|
||||
@@ -730,10 +854,14 @@ pub async fn generate_editor_image(
|
||||
Some(&request_context),
|
||||
EditorImageGenerationResponse {
|
||||
image_src,
|
||||
object_key: persisted.as_ref().map(|asset| asset.object_key.clone()),
|
||||
asset_object_id: persisted
|
||||
.as_ref()
|
||||
.map(|asset| asset.asset_object_id.clone()),
|
||||
width,
|
||||
height,
|
||||
source_type: "generated",
|
||||
prompt,
|
||||
prompt: role_setting,
|
||||
actual_prompt: generated.actual_prompt,
|
||||
model: GPT_IMAGE_2_MODEL,
|
||||
provider: "VectorEngine",
|
||||
@@ -742,6 +870,31 @@ pub async fn generate_editor_image(
|
||||
))
|
||||
}
|
||||
|
||||
fn normalize_editor_image_generation_size(size: Option<&str>) -> Cow<'static, str> {
|
||||
match size.map(str::trim).filter(|value| !value.is_empty()) {
|
||||
Some("1024x1024") | Some("1024*1024") | Some("1:1") => Cow::Borrowed("1024x1024"),
|
||||
Some("1536x1024") | Some("1536*1024") | Some("16:9") => Cow::Borrowed("1536x1024"),
|
||||
Some("2048x1152") | Some("2048*1152") | Some("1920x1080") | Some("1920*1080")
|
||||
| Some("2k-16:9") => Cow::Borrowed("2048x1152"),
|
||||
Some("1024x1536") | Some("1024*1536") | Some("9:16") => Cow::Borrowed("1024x1536"),
|
||||
Some(value) if is_editor_custom_image_size(value) => Cow::Owned(value.to_string()),
|
||||
_ => Cow::Borrowed(EDITOR_IMAGE_GENERATION_SIZE),
|
||||
}
|
||||
}
|
||||
|
||||
fn is_editor_custom_image_size(value: &str) -> bool {
|
||||
let Some((width, height)) = value.split_once('x') else {
|
||||
return false;
|
||||
};
|
||||
let Ok(width) = width.parse::<u32>() else {
|
||||
return false;
|
||||
};
|
||||
let Ok(height) = height.parse::<u32>() else {
|
||||
return false;
|
||||
};
|
||||
(64..=4096).contains(&width) && (64..=4096).contains(&height)
|
||||
}
|
||||
|
||||
pub async fn edit_editor_image(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
@@ -795,6 +948,8 @@ pub async fn edit_editor_image(
|
||||
Some(&request_context),
|
||||
EditorImageGenerationResponse {
|
||||
image_src,
|
||||
object_key: None,
|
||||
asset_object_id: None,
|
||||
width,
|
||||
height,
|
||||
source_type: "generated",
|
||||
@@ -807,6 +962,96 @@ pub async fn edit_editor_image(
|
||||
))
|
||||
}
|
||||
|
||||
pub async fn generate_editor_icon_spritesheet(
|
||||
State(state): State<AppState>,
|
||||
Extension(request_context): Extension<RequestContext>,
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
Json(payload): Json<EditorIconSpritesheetGenerationRequest>,
|
||||
) -> Result<Json<Value>, AppError> {
|
||||
let icon_descriptions = normalize_icon_descriptions(payload.icon_descriptions)?;
|
||||
let reference_image = parse_editor_reference_image(payload.reference_image_src.as_str())
|
||||
.map_err(|error| {
|
||||
error.with_details(json!({
|
||||
"provider": "editor-icon-spritesheet",
|
||||
"field": "referenceImageSrc",
|
||||
"message": "图标素材规范必须是图片 Data URL。",
|
||||
}))
|
||||
})?;
|
||||
let model = payload
|
||||
.model
|
||||
.as_deref()
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.unwrap_or(EDITOR_ICON_SPRITESHEET_MODEL)
|
||||
.to_string();
|
||||
let size = editor_icon_spritesheet_size_for_count(icon_descriptions.len());
|
||||
let prompt = build_editor_icon_spritesheet_prompt(&icon_descriptions);
|
||||
|
||||
let settings = require_openai_image_settings(&state)?.with_external_api_audit_context(
|
||||
&request_context,
|
||||
Some(authenticated.claims().user_id().to_string()),
|
||||
None,
|
||||
);
|
||||
let http_client = build_openai_image_http_client(&settings)?;
|
||||
let generated = create_openai_image_edit_with_references_and_model(
|
||||
&http_client,
|
||||
&settings,
|
||||
model.as_str(),
|
||||
prompt.as_str(),
|
||||
None,
|
||||
size,
|
||||
1,
|
||||
&[reference_image],
|
||||
"图片画布生成图标素材 spritesheet",
|
||||
)
|
||||
.await?;
|
||||
let image = generated.images.into_iter().next().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "vector-engine",
|
||||
"message": "VectorEngine 未返回图标 spritesheet",
|
||||
}))
|
||||
})?;
|
||||
let (spritesheet_width, spritesheet_height) = image::load_from_memory(image.bytes.as_slice())
|
||||
.map(|image| (image.width(), image.height()))
|
||||
.unwrap_or((512, 512));
|
||||
let source = DownloadedImage {
|
||||
bytes: image.bytes.clone(),
|
||||
mime_type: image.mime_type.clone(),
|
||||
extension: image.extension.clone(),
|
||||
};
|
||||
let icon_slices = slice_generated_icon_spritesheet_by_connected_components(
|
||||
&source,
|
||||
icon_descriptions.as_slice(),
|
||||
)
|
||||
.map_err(|error| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "editor-icon-spritesheet",
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
})?;
|
||||
let spritesheet_image_src =
|
||||
data_url_from_image_bytes(image.mime_type.as_str(), image.bytes.as_slice());
|
||||
let icon_image_srcs = icon_slices
|
||||
.into_iter()
|
||||
.map(editor_icon_response_from_slice)
|
||||
.collect();
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorIconSpritesheetGenerationResponse {
|
||||
spritesheet_image_src,
|
||||
spritesheet_width,
|
||||
spritesheet_height,
|
||||
icon_image_srcs,
|
||||
prompt,
|
||||
actual_prompt: generated.actual_prompt,
|
||||
model,
|
||||
provider: "VectorEngine",
|
||||
task_id: generated.task_id,
|
||||
},
|
||||
))
|
||||
}
|
||||
|
||||
fn editor_project_payload_from_record(record: EditorProjectRecord) -> EditorProjectPayload {
|
||||
let canvas = editor_canvas_payload_from_record(record.canvas);
|
||||
EditorProjectPayload {
|
||||
@@ -957,6 +1202,229 @@ fn normalize_optional_string(value: Option<String>) -> Option<String> {
|
||||
.filter(|item| !item.is_empty())
|
||||
}
|
||||
|
||||
fn sanitize_editor_storage_segment(value: &str, fallback: &str) -> String {
|
||||
let normalized = value
|
||||
.trim()
|
||||
.chars()
|
||||
.map(|character| match character {
|
||||
'a'..='z' | '0'..='9' | '-' | '_' => character,
|
||||
'A'..='Z' => character.to_ascii_lowercase(),
|
||||
_ => '-',
|
||||
})
|
||||
.collect::<String>()
|
||||
.split('-')
|
||||
.filter(|part| !part.is_empty())
|
||||
.collect::<Vec<_>>()
|
||||
.join("-");
|
||||
if normalized.is_empty() {
|
||||
fallback.to_string()
|
||||
} else {
|
||||
normalized
|
||||
}
|
||||
}
|
||||
|
||||
fn normalize_icon_descriptions(descriptions: Vec<String>) -> Result<Vec<String>, AppError> {
|
||||
let normalized = descriptions
|
||||
.into_iter()
|
||||
.map(|description| description.trim().to_string())
|
||||
.filter(|description| !description.is_empty())
|
||||
.take(EDITOR_ICON_DESCRIPTION_LIMIT.saturating_add(1))
|
||||
.collect::<Vec<_>>();
|
||||
if normalized.is_empty() || normalized.len() > EDITOR_ICON_DESCRIPTION_LIMIT {
|
||||
return Err(
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-icon-spritesheet",
|
||||
"field": "iconDescriptions",
|
||||
"message": "图标素材描述数量必须在 1 到 100 个之间。",
|
||||
})),
|
||||
);
|
||||
}
|
||||
Ok(normalized)
|
||||
}
|
||||
|
||||
fn build_editor_icon_spritesheet_prompt(icon_descriptions: &[String]) -> String {
|
||||
format!(
|
||||
"参考图1的图标素材规范，纯绿幕背景方便扣除背景，禁止出现文字，保证每个图标素材的所有内容区域是完全连通的。按照以下的素材的顺序从上到下从左到右依次生成并整理成一张spritesheet：\n\n{}",
|
||||
icon_descriptions.join("、")
|
||||
)
|
||||
}
|
||||
|
||||
fn build_editor_character_image_prompt(role_setting: &str) -> String {
|
||||
vec![
|
||||
"基于图1的角色美术视觉规范指导生成游戏角色形象图。画面中心构图，角色主体完整置于画面中央，禁止镜头透视，禁止特写。背景固定为纯绿色绿幕，只作为抠像底色，禁止生成美术视觉规范、出现建筑、室内布景、风景、地面道具、漂浮物、烟雾叙事元素、文字或其他角色以外的场景内容。".to_string(),
|
||||
format!("角色设定：{}", role_setting.trim()),
|
||||
]
|
||||
.join("\n")
|
||||
}
|
||||
|
||||
fn prepare_editor_character_image_for_response(
|
||||
image: DownloadedOpenAiImage,
|
||||
) -> DownloadedOpenAiImage {
|
||||
let source_bytes = if image.mime_type == "image/png" {
|
||||
image.bytes
|
||||
} else {
|
||||
match image::load_from_memory(image.bytes.as_slice()) {
|
||||
Ok(decoded) => {
|
||||
let mut encoded = std::io::Cursor::new(Vec::new());
|
||||
if decoded
|
||||
.write_to(&mut encoded, image::ImageFormat::Png)
|
||||
.is_err()
|
||||
{
|
||||
return image;
|
||||
}
|
||||
encoded.into_inner()
|
||||
}
|
||||
Err(_) => return image,
|
||||
}
|
||||
};
|
||||
|
||||
let processed =
|
||||
crate::character_visual_assets::try_apply_background_alpha_to_png(source_bytes.as_slice());
|
||||
let Some(bytes) = processed else {
|
||||
return DownloadedOpenAiImage {
|
||||
bytes: source_bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
};
|
||||
};
|
||||
|
||||
DownloadedOpenAiImage {
|
||||
bytes,
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
}
|
||||
}
|
||||
|
||||
fn editor_icon_spritesheet_size_for_count(icon_count: usize) -> &'static str {
|
||||
if icon_count <= 25 {
|
||||
EDITOR_ICON_SPRITESHEET_SMALL_SIZE
|
||||
} else {
|
||||
EDITOR_ICON_SPRITESHEET_LARGE_SIZE
|
||||
}
|
||||
}
|
||||
|
||||
fn data_url_from_image_bytes(mime_type: &str, bytes: &[u8]) -> String {
|
||||
format!(
|
||||
"data:{};base64,{}",
|
||||
mime_type,
|
||||
BASE64_STANDARD.encode(bytes)
|
||||
)
|
||||
}
|
||||
|
||||
fn editor_icon_response_from_slice(
|
||||
icon: GeneratedAssetSheetConnectedIcon,
|
||||
) -> EditorIconSpritesheetIconResponse {
|
||||
EditorIconSpritesheetIconResponse {
|
||||
name: icon.name,
|
||||
image_src: data_url_from_image_bytes("image/png", icon.bytes.as_slice()),
|
||||
width: icon.width,
|
||||
height: icon.height,
|
||||
}
|
||||
}
|
||||
|
||||
struct PersistedEditorGeneratedImage {
|
||||
object_key: String,
|
||||
asset_object_id: String,
|
||||
}
|
||||
|
||||
async fn persist_editor_character_image(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
task_id: &str,
|
||||
image: &DownloadedOpenAiImage,
|
||||
prompt: &str,
|
||||
actual_prompt: Option<&str>,
|
||||
) -> Result<PersistedEditorGeneratedImage, AppError> {
|
||||
let oss_client = state.oss_client().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "aliyun-oss",
|
||||
"reason": "OSS 未完成环境变量配置",
|
||||
}))
|
||||
})?;
|
||||
let prepared =
|
||||
GeneratedImageAssetAdapter::prepare_put_object(GeneratedImageAssetPersistInput {
|
||||
prefix: LegacyAssetPrefix::CharacterDrafts,
|
||||
path_segments: vec![
|
||||
"editor".to_string(),
|
||||
"character-images".to_string(),
|
||||
sanitize_editor_storage_segment(task_id, "task"),
|
||||
],
|
||||
file_stem: "image".to_string(),
|
||||
image: GeneratedImageAssetDataUrl {
|
||||
format: normalize_generated_image_asset_mime(image.mime_type.as_str()),
|
||||
bytes: image.bytes.clone(),
|
||||
},
|
||||
access: OssObjectAccess::Private,
|
||||
metadata: GeneratedImageAssetAdapterMetadata {
|
||||
asset_kind: Some(EDITOR_CHARACTER_IMAGE_ASSET_KIND.to_string()),
|
||||
owner_user_id: Some(owner_user_id.to_string()),
|
||||
entity_kind: Some(EDITOR_CHARACTER_IMAGE_ENTITY_KIND.to_string()),
|
||||
entity_id: Some(task_id.to_string()),
|
||||
slot: Some(EDITOR_CHARACTER_IMAGE_SLOT.to_string()),
|
||||
provider: Some("vector-engine".to_string()),
|
||||
task_id: Some(task_id.to_string()),
|
||||
},
|
||||
extra_metadata: BTreeMap::from([(
|
||||
"source".to_string(),
|
||||
"image-canvas-editor".to_string(),
|
||||
)]),
|
||||
})
|
||||
.map_err(|error| {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": "generated-image-assets",
|
||||
"message": format!("准备画板角色形象 OSS 上传请求失败：{error:?}"),
|
||||
}))
|
||||
})?;
|
||||
let persisted_mime_type = prepared.format.mime_type.clone();
|
||||
let http_client = reqwest::Client::new();
|
||||
let put_result = oss_client
|
||||
.put_object(&http_client, prepared.request)
|
||||
.await
|
||||
.map_err(|error| map_oss_error(error, "aliyun-oss"))?;
|
||||
let head = oss_client
|
||||
.head_object(
|
||||
&http_client,
|
||||
OssHeadObjectRequest {
|
||||
object_key: put_result.object_key.clone(),
|
||||
},
|
||||
)
|
||||
.await
|
||||
.map_err(|error| map_oss_error(error, "aliyun-oss"))?;
|
||||
let now_micros = current_utc_micros();
|
||||
let asset_object = state
|
||||
.spacetime_client()
|
||||
.confirm_asset_object(
|
||||
build_asset_object_upsert_input(
|
||||
generate_asset_object_id(now_micros),
|
||||
head.bucket,
|
||||
head.object_key.clone(),
|
||||
AssetObjectAccessPolicy::Private,
|
||||
head.content_type.or(Some(persisted_mime_type)),
|
||||
head.content_length,
|
||||
Some(actual_prompt.unwrap_or(prompt).to_string()),
|
||||
EDITOR_CHARACTER_IMAGE_ASSET_KIND.to_string(),
|
||||
Some(task_id.to_string()),
|
||||
Some(owner_user_id.to_string()),
|
||||
None,
|
||||
Some(task_id.to_string()),
|
||||
now_micros,
|
||||
)
|
||||
.map_err(map_editor_asset_field_error)?,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "spacetimedb",
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
})?;
|
||||
|
||||
Ok(PersistedEditorGeneratedImage {
|
||||
object_key: head.object_key,
|
||||
asset_object_id: asset_object.asset_object_id,
|
||||
})
|
||||
}
|
||||
|
||||
fn parse_editor_reference_image(source: &str) -> Result<OpenAiReferenceImage, AppError> {
|
||||
let Some((header, data)) = source.trim().split_once(',') else {
|
||||
return Err(
|
||||
@@ -1023,12 +1491,11 @@ fn map_editor_project_error(error: SpacetimeClientError) -> AppError {
|
||||
"message": message,
|
||||
}))
|
||||
}
|
||||
SpacetimeClientError::Runtime(message) => {
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
SpacetimeClientError::Runtime(message) => AppError::from_status(StatusCode::BAD_REQUEST)
|
||||
.with_details(json!({
|
||||
"provider": "editor-project",
|
||||
"message": message,
|
||||
}))
|
||||
}
|
||||
})),
|
||||
other => AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": "spacetimedb",
|
||||
"message": other.to_string(),
|
||||
@@ -1036,6 +1503,13 @@ fn map_editor_project_error(error: SpacetimeClientError) -> AppError {
|
||||
}
|
||||
}
|
||||
|
||||
fn map_editor_asset_field_error(error: AssetObjectFieldError) -> AppError {
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "asset-object",
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
}
|
||||
|
||||
fn current_utc_micros() -> i64 {
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
@@ -1044,3 +1518,136 @@ fn current_utc_micros() -> i64 {
|
||||
.expect("system clock should be after unix epoch");
|
||||
i64::try_from(duration.as_micros()).expect("current unix micros should fit in i64")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn editor_image_generation_size_keeps_quick_edit_canvas_ratio_presets() {
|
||||
assert_eq!(normalize_editor_image_generation_size(None), "1024x1024");
|
||||
assert_eq!(
|
||||
normalize_editor_image_generation_size(Some("1536x1024")),
|
||||
"1536x1024"
|
||||
);
|
||||
assert_eq!(
|
||||
normalize_editor_image_generation_size(Some("1024x1536")),
|
||||
"1024x1536"
|
||||
);
|
||||
assert_eq!(
|
||||
normalize_editor_image_generation_size(Some("2048x1152")),
|
||||
"2048x1152"
|
||||
);
|
||||
assert_eq!(
|
||||
normalize_editor_image_generation_size(Some("640x640")),
|
||||
"640x640"
|
||||
);
|
||||
assert_eq!(
|
||||
normalize_editor_image_generation_size(Some("bad-size")),
|
||||
"1024x1024"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_image_prompt_appends_user_role_setting() {
|
||||
let prompt = build_editor_character_image_prompt("菜市场卖菜大妈");
|
||||
|
||||
assert!(prompt.contains("基于图1的角色美术视觉规范指导生成游戏角色形象图。"));
|
||||
assert!(prompt.contains("背景固定为纯绿色绿幕"));
|
||||
assert!(prompt.contains("禁止镜头透视"));
|
||||
assert!(prompt.contains("角色设定：菜市场卖菜大妈"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_image_postprocess_removes_green_screen_background() {
|
||||
let width = 12;
|
||||
let height = 12;
|
||||
let mut image = image::RgbaImage::from_pixel(width, height, image::Rgba([0, 255, 0, 255]));
|
||||
for y in 4..8 {
|
||||
for x in 4..8 {
|
||||
image.put_pixel(x, y, image::Rgba([188, 82, 45, 255]));
|
||||
}
|
||||
}
|
||||
let mut encoded = std::io::Cursor::new(Vec::new());
|
||||
image::DynamicImage::ImageRgba8(image)
|
||||
.write_to(&mut encoded, image::ImageFormat::Png)
|
||||
.expect("test image should encode");
|
||||
|
||||
let processed = prepare_editor_character_image_for_response(DownloadedOpenAiImage {
|
||||
bytes: encoded.into_inner(),
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
});
|
||||
let decoded = image::load_from_memory(processed.bytes.as_slice())
|
||||
.expect("processed image should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(processed.mime_type, "image/png");
|
||||
assert_eq!(processed.extension, "png");
|
||||
assert_eq!(decoded.get_pixel(0, 0).0[3], 0);
|
||||
assert_eq!(decoded.get_pixel(5, 5).0[3], 255);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_image_postprocess_converts_non_png_green_screen_to_transparent_png() {
|
||||
let width = 12;
|
||||
let height = 12;
|
||||
let mut image =
|
||||
image::RgbImage::from_pixel(width, height, image::Rgb([0_u8, 255_u8, 0_u8]));
|
||||
for y in 4..8 {
|
||||
for x in 4..8 {
|
||||
image.put_pixel(x, y, image::Rgb([188, 82, 45]));
|
||||
}
|
||||
}
|
||||
let mut encoded = std::io::Cursor::new(Vec::new());
|
||||
image::DynamicImage::ImageRgb8(image)
|
||||
.write_to(&mut encoded, image::ImageFormat::Jpeg)
|
||||
.expect("test image should encode");
|
||||
|
||||
let processed = prepare_editor_character_image_for_response(DownloadedOpenAiImage {
|
||||
bytes: encoded.into_inner(),
|
||||
mime_type: "image/jpeg".to_string(),
|
||||
extension: "jpg".to_string(),
|
||||
});
|
||||
let decoded = image::load_from_memory(processed.bytes.as_slice())
|
||||
.expect("processed image should decode")
|
||||
.to_rgba8();
|
||||
|
||||
assert_eq!(processed.mime_type, "image/png");
|
||||
assert_eq!(processed.extension, "png");
|
||||
assert_eq!(decoded.get_pixel(0, 0).0[3], 0);
|
||||
assert_eq!(decoded.get_pixel(5, 5).0[3], 255);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_icon_spritesheet_prompt_uses_ordered_descriptions_and_size_tiers() {
|
||||
let descriptions = vec![
|
||||
"返回按钮".to_string(),
|
||||
"设置按钮".to_string(),
|
||||
"下一关按钮".to_string(),
|
||||
];
|
||||
let prompt = build_editor_icon_spritesheet_prompt(&descriptions);
|
||||
|
||||
assert!(prompt.contains("参考图1的图标素材规范"));
|
||||
assert!(prompt.contains("纯绿幕背景方便扣除背景"));
|
||||
assert!(prompt.contains("返回按钮、设置按钮、下一关按钮"));
|
||||
assert_eq!(editor_icon_spritesheet_size_for_count(25), "512x512");
|
||||
assert_eq!(editor_icon_spritesheet_size_for_count(26), "1024x1024");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_icon_description_validation_filters_empty_and_rejects_more_than_limit() {
|
||||
let descriptions = normalize_icon_descriptions(vec![
|
||||
" 返回按钮 ".to_string(),
|
||||
" ".to_string(),
|
||||
"设置按钮".to_string(),
|
||||
])
|
||||
.expect("valid descriptions should pass");
|
||||
assert_eq!(descriptions, vec!["返回按钮", "设置按钮"]);
|
||||
|
||||
let too_many = (0..101)
|
||||
.map(|index| format!("图标{index}"))
|
||||
.collect::<Vec<_>>();
|
||||
assert!(normalize_icon_descriptions(too_many).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -744,6 +744,7 @@ mod tests {
|
||||
started_at: Some("2026-06-03T00:00:00Z".to_string()),
|
||||
completed_at: None,
|
||||
updated_at: "2026-06-03T00:00:00Z".to_string(),
|
||||
updated_at_micros: 1_780_000_000_000_000,
|
||||
lease_token: lease_token.map(ToOwned::to_owned),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -9,9 +9,9 @@ use crate::{
|
||||
create_editor_asset, create_editor_asset_folder, create_editor_project,
|
||||
create_editor_project_resource, delete_editor_asset, delete_editor_asset_folder,
|
||||
delete_editor_project, edit_editor_image, generate_editor_image,
|
||||
get_editor_asset_library, get_editor_project, list_editor_projects,
|
||||
load_recent_editor_project, rename_editor_project, save_editor_project_layout,
|
||||
update_editor_asset, update_editor_asset_folder,
|
||||
generate_editor_icon_spritesheet, get_editor_asset_library, get_editor_project,
|
||||
list_editor_projects, load_recent_editor_project, rename_editor_project,
|
||||
save_editor_project_layout, update_editor_asset, update_editor_asset_folder,
|
||||
},
|
||||
state::AppState,
|
||||
};
|
||||
@@ -111,4 +111,11 @@ pub fn router(state: AppState) -> Router<AppState> {
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/icon-spritesheets/generations",
|
||||
post(generate_editor_icon_spritesheet).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -16,8 +16,9 @@ use crate::{
|
||||
assets::get_asset_history,
|
||||
auth::require_bearer_auth,
|
||||
character_animation_assets::{
|
||||
generate_character_animation, get_character_animation_job, get_character_workflow_cache,
|
||||
import_character_animation_video, list_character_animation_templates,
|
||||
generate_character_animation, generate_editor_character_animation,
|
||||
get_character_animation_job, get_character_workflow_cache, import_character_animation_video,
|
||||
list_character_animation_templates,
|
||||
publish_character_animation, put_role_asset_workflow, resolve_role_asset_workflow,
|
||||
save_character_workflow_cache,
|
||||
},
|
||||
@@ -452,6 +453,10 @@ fn play_flow_support_router(state: AppState) -> Router<AppState> {
|
||||
"/api/assets/character-animation/generate",
|
||||
post(generate_character_animation),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/character-animations/generations",
|
||||
post(generate_editor_character_animation),
|
||||
)
|
||||
.route(
|
||||
"/api/assets/character-animation/jobs/{task_id}",
|
||||
get(get_character_animation_job),
|
||||
|
||||
@@ -3,7 +3,7 @@ use platform_image::{
|
||||
DownloadedImage, GeneratedImages, PlatformImageError, PlatformImageStatusHint, ReferenceImage,
|
||||
VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings, build_vector_engine_image_http_client,
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_generation,
|
||||
create_vector_engine_image_edit_with_references_and_model, create_vector_engine_image_generation,
|
||||
};
|
||||
#[cfg(test)]
|
||||
use platform_image::{
|
||||
@@ -236,6 +236,49 @@ pub(crate) async fn create_openai_image_edit_with_references(
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub(crate) async fn create_openai_image_edit_with_references_and_model(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &OpenAiImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
reference_images: &[OpenAiReferenceImage],
|
||||
failure_context: &str,
|
||||
) -> Result<OpenAiGeneratedImages, AppError> {
|
||||
let started_at_micros = current_utc_micros();
|
||||
let request_payload = json!({
|
||||
"model": model,
|
||||
"size": size,
|
||||
"promptChars": prompt.chars().count(),
|
||||
"negativePromptChars": negative_prompt.map(str::chars).map(Iterator::count),
|
||||
"referenceImageCount": reference_images.len(),
|
||||
});
|
||||
let result = create_vector_engine_image_edit_with_references_and_model(
|
||||
http_client,
|
||||
&settings.provider_settings(),
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
candidate_count,
|
||||
reference_images,
|
||||
failure_context,
|
||||
)
|
||||
.await;
|
||||
map_platform_image_result(
|
||||
settings,
|
||||
result,
|
||||
"image_edit_with_references",
|
||||
failure_context,
|
||||
request_payload,
|
||||
started_at_micros,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) fn build_openai_image_request_body(
|
||||
prompt: &str,
|
||||
|
||||
@@ -16,7 +16,9 @@ pub use persist::{
|
||||
};
|
||||
pub use prompt::{GeneratedAssetSheetPromptInput, build_generated_asset_sheet_prompt};
|
||||
pub use sheet::{
|
||||
GeneratedAssetSheetSliceImage, crop_generated_asset_sheet_view_edge_matte,
|
||||
GeneratedAssetSheetConnectedIcon, GeneratedAssetSheetSliceImage,
|
||||
crop_generated_asset_sheet_view_edge_matte,
|
||||
crop_generated_asset_sheet_view_edge_matte_with_options, slice_generated_asset_sheet,
|
||||
slice_generated_asset_sheet_two_items_per_row,
|
||||
slice_generated_icon_spritesheet_by_connected_components,
|
||||
};
|
||||
|
||||
@@ -132,6 +132,25 @@ pub fn slice_generated_asset_sheet_two_items_per_row(
|
||||
Ok(slices)
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, PartialEq, Eq)]
|
||||
pub struct GeneratedAssetSheetConnectedIcon {
|
||||
pub name: String,
|
||||
pub bytes: Vec<u8>,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
}
|
||||
|
||||
pub fn slice_generated_icon_spritesheet_by_connected_components(
|
||||
image: &crate::DownloadedImage,
|
||||
icon_names: &[String],
|
||||
) -> Result<Vec<GeneratedAssetSheetConnectedIcon>, GeneratedAssetSheetError> {
|
||||
let source = image::load_from_memory(image.bytes.as_slice()).map_err(|error| {
|
||||
GeneratedAssetSheetError::decode_image(format!("图标 spritesheet 解码失败：{error}"))
|
||||
})?;
|
||||
let source = apply_generated_asset_sheet_green_screen_alpha(source);
|
||||
slice_generated_icon_spritesheet_rgba_by_connected_components(source, icon_names)
|
||||
}
|
||||
|
||||
pub fn crop_generated_asset_sheet_view_edge_matte(
|
||||
image: image::DynamicImage,
|
||||
) -> image::DynamicImage {
|
||||
@@ -141,6 +160,207 @@ pub fn crop_generated_asset_sheet_view_edge_matte(
|
||||
)
|
||||
}
|
||||
|
||||
fn slice_generated_icon_spritesheet_rgba_by_connected_components(
|
||||
source: image::DynamicImage,
|
||||
icon_names: &[String],
|
||||
) -> Result<Vec<GeneratedAssetSheetConnectedIcon>, GeneratedAssetSheetError> {
|
||||
let image = source.to_rgba8();
|
||||
let (width, height) = image.dimensions();
|
||||
let pixel_count = (width as usize).saturating_mul(height as usize);
|
||||
if pixel_count == 0 {
|
||||
return Err(GeneratedAssetSheetError::invalid_request(
|
||||
"图标 spritesheet 尺寸为空。",
|
||||
));
|
||||
}
|
||||
|
||||
let mut visited = vec![false; pixel_count];
|
||||
let mut components = Vec::<GeneratedAssetSheetCellBounds>::new();
|
||||
for y in 0..height {
|
||||
for x in 0..width {
|
||||
let pixel_index = (y as usize)
|
||||
.saturating_mul(width as usize)
|
||||
.saturating_add(x as usize);
|
||||
if visited[pixel_index] || image.get_pixel(x, y).0[3] == 0 {
|
||||
continue;
|
||||
}
|
||||
components.push(flood_fill_generated_icon_component(
|
||||
&image,
|
||||
&mut visited,
|
||||
width,
|
||||
height,
|
||||
x,
|
||||
y,
|
||||
));
|
||||
}
|
||||
}
|
||||
|
||||
components.sort_by_key(|bounds| (bounds.y0, bounds.x0));
|
||||
if components.len() < icon_names.len() {
|
||||
return Err(GeneratedAssetSheetError::invalid_request(format!(
|
||||
"图标 spritesheet 连通域数量不足：需要 {} 个，实际 {} 个。",
|
||||
icon_names.len(),
|
||||
components.len()
|
||||
)));
|
||||
}
|
||||
|
||||
let mut icons = Vec::with_capacity(icon_names.len());
|
||||
for (name, bounds) in icon_names.iter().zip(components.into_iter()) {
|
||||
let pad_x = (bounds.width() / 12).clamp(4, 16);
|
||||
let pad_y = (bounds.height() / 12).clamp(4, 16);
|
||||
let crop = GeneratedAssetSheetCellBounds {
|
||||
x0: bounds.x0.saturating_sub(pad_x),
|
||||
y0: bounds.y0.saturating_sub(pad_y),
|
||||
x1: bounds.x1.saturating_add(pad_x).min(width),
|
||||
y1: bounds.y1.saturating_add(pad_y).min(height),
|
||||
};
|
||||
let cropped = image::imageops::crop_imm(
|
||||
&image,
|
||||
crop.x0,
|
||||
crop.y0,
|
||||
crop.width(),
|
||||
crop.height(),
|
||||
)
|
||||
.to_image();
|
||||
let mut cursor = std::io::Cursor::new(Vec::new());
|
||||
image::DynamicImage::ImageRgba8(cropped)
|
||||
.write_to(&mut cursor, ImageFormat::Png)
|
||||
.map_err(|error| {
|
||||
GeneratedAssetSheetError::encode_image(format!(
|
||||
"图标 spritesheet 切割失败：{error}"
|
||||
))
|
||||
})?;
|
||||
icons.push(GeneratedAssetSheetConnectedIcon {
|
||||
name: name.clone(),
|
||||
bytes: cursor.into_inner(),
|
||||
width: crop.width(),
|
||||
height: crop.height(),
|
||||
});
|
||||
}
|
||||
|
||||
Ok(icons)
|
||||
}
|
||||
|
||||
fn flood_fill_generated_icon_component(
|
||||
image: &image::RgbaImage,
|
||||
visited: &mut [bool],
|
||||
width: u32,
|
||||
height: u32,
|
||||
start_x: u32,
|
||||
start_y: u32,
|
||||
) -> GeneratedAssetSheetCellBounds {
|
||||
let mut queue = vec![(start_x, start_y)];
|
||||
let mut queue_index = 0usize;
|
||||
let start_index = (start_y as usize)
|
||||
.saturating_mul(width as usize)
|
||||
.saturating_add(start_x as usize);
|
||||
visited[start_index] = true;
|
||||
let mut bounds = GeneratedAssetSheetCellBounds {
|
||||
x0: start_x,
|
||||
y0: start_y,
|
||||
x1: start_x.saturating_add(1),
|
||||
y1: start_y.saturating_add(1),
|
||||
};
|
||||
|
||||
while queue_index < queue.len() {
|
||||
let (x, y) = queue[queue_index];
|
||||
queue_index += 1;
|
||||
bounds.x0 = bounds.x0.min(x);
|
||||
bounds.y0 = bounds.y0.min(y);
|
||||
bounds.x1 = bounds.x1.max(x.saturating_add(1));
|
||||
bounds.y1 = bounds.y1.max(y.saturating_add(1));
|
||||
|
||||
for next_y in y.saturating_sub(1)..=(y.saturating_add(1).min(height.saturating_sub(1))) {
|
||||
for next_x in x.saturating_sub(1)..=(x.saturating_add(1).min(width.saturating_sub(1))) {
|
||||
if next_x == x && next_y == y {
|
||||
continue;
|
||||
}
|
||||
let next_index = (next_y as usize)
|
||||
.saturating_mul(width as usize)
|
||||
.saturating_add(next_x as usize);
|
||||
if visited[next_index] || image.get_pixel(next_x, next_y).0[3] == 0 {
|
||||
continue;
|
||||
}
|
||||
visited[next_index] = true;
|
||||
queue.push((next_x, next_y));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
bounds
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use image::{ImageBuffer, Rgba};
|
||||
|
||||
fn encode_png(image: image::RgbaImage) -> Vec<u8> {
|
||||
let mut cursor = std::io::Cursor::new(Vec::new());
|
||||
image::DynamicImage::ImageRgba8(image)
|
||||
.write_to(&mut cursor, ImageFormat::Png)
|
||||
.expect("png should encode");
|
||||
cursor.into_inner()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn slices_icon_spritesheet_by_connected_components_in_reading_order() {
|
||||
let mut sheet: image::RgbaImage =
|
||||
ImageBuffer::from_pixel(96, 64, Rgba([0, 255, 0, 255]));
|
||||
for y in 10..24 {
|
||||
for x in 12..28 {
|
||||
sheet.put_pixel(x, y, Rgba([240, 80, 80, 255]));
|
||||
}
|
||||
}
|
||||
for y in 32..46 {
|
||||
for x in 52..70 {
|
||||
sheet.put_pixel(x, y, Rgba([80, 120, 240, 255]));
|
||||
}
|
||||
}
|
||||
|
||||
let source = crate::DownloadedImage {
|
||||
bytes: encode_png(sheet),
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
};
|
||||
let icons = slice_generated_icon_spritesheet_by_connected_components(
|
||||
&source,
|
||||
&["返回按钮".to_string(), "设置按钮".to_string()],
|
||||
)
|
||||
.expect("icons should slice");
|
||||
|
||||
assert_eq!(icons.len(), 2);
|
||||
assert_eq!(icons[0].name, "返回按钮");
|
||||
assert_eq!(icons[1].name, "设置按钮");
|
||||
assert!(icons[0].width >= 16);
|
||||
assert!(icons[0].height >= 14);
|
||||
assert!(image::load_from_memory(icons[0].bytes.as_slice()).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_when_connected_components_are_fewer_than_icon_names() {
|
||||
let mut sheet: image::RgbaImage =
|
||||
ImageBuffer::from_pixel(48, 48, Rgba([0, 255, 0, 255]));
|
||||
for y in 12..24 {
|
||||
for x in 12..24 {
|
||||
sheet.put_pixel(x, y, Rgba([240, 80, 80, 255]));
|
||||
}
|
||||
}
|
||||
let source = crate::DownloadedImage {
|
||||
bytes: encode_png(sheet),
|
||||
mime_type: "image/png".to_string(),
|
||||
extension: "png".to_string(),
|
||||
};
|
||||
|
||||
let error = slice_generated_icon_spritesheet_by_connected_components(
|
||||
&source,
|
||||
&["返回按钮".to_string(), "设置按钮".to_string()],
|
||||
)
|
||||
.expect_err("missing component should fail");
|
||||
|
||||
assert!(error.to_string().contains("连通域数量不足"));
|
||||
}
|
||||
}
|
||||
|
||||
pub fn crop_generated_asset_sheet_view_edge_matte_with_options(
|
||||
image: image::DynamicImage,
|
||||
options: GeneratedAssetSheetAlphaOptions,
|
||||
|
||||
@@ -8,6 +8,7 @@ pub use vector_engine::{
|
||||
VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
|
||||
build_vector_engine_image_http_client, build_vector_engine_image_request_body,
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_generation, download_remote_image, vector_engine_images_edit_url,
|
||||
vector_engine_images_generation_url,
|
||||
create_vector_engine_image_edit_with_references_and_model,
|
||||
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
|
||||
download_remote_image, vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
};
|
||||
|
||||
@@ -13,8 +13,10 @@ use super::{
|
||||
error::PlatformImageError,
|
||||
image_source::resolve_reference_images,
|
||||
request::{
|
||||
build_vector_engine_image_edit_request_log_params, build_vector_engine_image_request_body,
|
||||
normalize_image_size, vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
build_vector_engine_image_edit_request_log_params,
|
||||
build_vector_engine_image_request_body_with_model, normalize_image_size,
|
||||
normalize_vector_engine_image_model, vector_engine_images_edit_url,
|
||||
vector_engine_images_generation_url,
|
||||
},
|
||||
response::handle_vector_engine_response,
|
||||
types::{GeneratedImages, ReferenceImage, VectorEngineImageSettings},
|
||||
@@ -31,12 +33,40 @@ pub async fn create_vector_engine_image_generation(
|
||||
reference_images: &[String],
|
||||
failure_context: &str,
|
||||
) -> Result<GeneratedImages, PlatformImageError> {
|
||||
create_vector_engine_image_generation_with_model(
|
||||
http_client,
|
||||
settings,
|
||||
GPT_IMAGE_2_MODEL,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
candidate_count,
|
||||
reference_images,
|
||||
failure_context,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn create_vector_engine_image_generation_with_model(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &VectorEngineImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
reference_images: &[String],
|
||||
failure_context: &str,
|
||||
) -> Result<GeneratedImages, PlatformImageError> {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
if !reference_images.is_empty() {
|
||||
let resolved_references =
|
||||
resolve_reference_images(http_client, reference_images, failure_context).await?;
|
||||
return create_vector_engine_image_edit_with_references(
|
||||
return create_vector_engine_image_edit_with_references_and_model(
|
||||
http_client,
|
||||
settings,
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
@@ -49,7 +79,8 @@ pub async fn create_vector_engine_image_generation(
|
||||
|
||||
let request_url = vector_engine_images_generation_url(settings);
|
||||
let normalized_size = normalize_image_size(size);
|
||||
let request_body = build_vector_engine_image_request_body(
|
||||
let request_body = build_vector_engine_image_request_body_with_model(
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
normalized_size.as_str(),
|
||||
@@ -125,6 +156,7 @@ pub async fn create_vector_engine_image_generation(
|
||||
provider = VECTOR_ENGINE_PROVIDER,
|
||||
endpoint = %request_url,
|
||||
status = response_status,
|
||||
image_model = model,
|
||||
prompt_chars = prompt.chars().count(),
|
||||
size = %normalized_size,
|
||||
reference_image_count = reference_images.len(),
|
||||
@@ -181,6 +213,33 @@ pub async fn create_vector_engine_image_edit_with_references(
|
||||
reference_images: &[ReferenceImage],
|
||||
failure_context: &str,
|
||||
) -> Result<GeneratedImages, PlatformImageError> {
|
||||
create_vector_engine_image_edit_with_references_and_model(
|
||||
http_client,
|
||||
settings,
|
||||
GPT_IMAGE_2_MODEL,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
candidate_count,
|
||||
reference_images,
|
||||
failure_context,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub async fn create_vector_engine_image_edit_with_references_and_model(
|
||||
http_client: &reqwest::Client,
|
||||
settings: &VectorEngineImageSettings,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
reference_images: &[ReferenceImage],
|
||||
failure_context: &str,
|
||||
) -> Result<GeneratedImages, PlatformImageError> {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
if reference_images.is_empty() {
|
||||
return Err(PlatformImageError::InvalidRequest {
|
||||
provider: VECTOR_ENGINE_PROVIDER,
|
||||
@@ -191,6 +250,7 @@ pub async fn create_vector_engine_image_edit_with_references(
|
||||
let request_url = vector_engine_images_edit_url(settings);
|
||||
let normalized_size = normalize_image_size(size);
|
||||
let request_params = build_vector_engine_image_edit_request_log_params(
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
normalized_size.as_str(),
|
||||
@@ -208,7 +268,7 @@ pub async fn create_vector_engine_image_edit_with_references(
|
||||
tracing::info!(
|
||||
provider = VECTOR_ENGINE_PROVIDER,
|
||||
endpoint = %request_url,
|
||||
image_model = GPT_IMAGE_2_MODEL,
|
||||
image_model = model,
|
||||
size = %normalized_size,
|
||||
candidate_count = candidate_count.clamp(1, 4),
|
||||
requested_candidate_count = candidate_count,
|
||||
@@ -230,6 +290,7 @@ pub async fn create_vector_engine_image_edit_with_references(
|
||||
match send_vector_engine_multipart_edit_request_with_curl(
|
||||
request_url.as_str(),
|
||||
settings.api_key.as_str(),
|
||||
model,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
normalized_size.as_str(),
|
||||
|
||||
@@ -8,7 +8,7 @@ use serde_json::Value;
|
||||
|
||||
use super::{
|
||||
audit::build_failure_audit,
|
||||
constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER},
|
||||
constants::VECTOR_ENGINE_PROVIDER,
|
||||
error::PlatformImageError,
|
||||
request::build_prompt_with_negative,
|
||||
types::ReferenceImage,
|
||||
@@ -115,6 +115,7 @@ pub(crate) async fn send_vector_engine_json_request_with_curl(
|
||||
pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
|
||||
request_url: &str,
|
||||
api_key: &str,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
normalized_size: &str,
|
||||
@@ -124,6 +125,7 @@ pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
|
||||
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
|
||||
let request_url = request_url.to_string();
|
||||
let api_key = api_key.to_string();
|
||||
let model = model.to_string();
|
||||
let prompt = prompt.to_string();
|
||||
let negative_prompt = negative_prompt.map(str::to_string);
|
||||
let normalized_size = normalized_size.to_string();
|
||||
@@ -132,6 +134,7 @@ pub(crate) async fn send_vector_engine_multipart_edit_request_with_curl(
|
||||
send_multipart_edit_request_with_curl_blocking(
|
||||
request_url.as_str(),
|
||||
api_key.as_str(),
|
||||
model.as_str(),
|
||||
prompt.as_str(),
|
||||
negative_prompt.as_deref(),
|
||||
normalized_size.as_str(),
|
||||
@@ -230,6 +233,7 @@ fn send_json_request_with_curl_blocking(
|
||||
fn send_multipart_edit_request_with_curl_blocking(
|
||||
request_url: &str,
|
||||
api_key: &str,
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
normalized_size: &str,
|
||||
@@ -239,7 +243,7 @@ fn send_multipart_edit_request_with_curl_blocking(
|
||||
) -> Result<VectorEngineCurlResponse, VectorEngineCurlError> {
|
||||
let mut form = Form::new();
|
||||
form.part("model")
|
||||
.contents(GPT_IMAGE_2_MODEL.as_bytes())
|
||||
.contents(model.as_bytes())
|
||||
.add()?;
|
||||
form.part("prompt")
|
||||
.contents(build_prompt_with_negative(prompt, negative_prompt).as_bytes())
|
||||
@@ -295,7 +299,7 @@ fn perform_curl_request(mut easy: Easy) -> Result<VectorEngineCurlResponse, curl
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::vector_engine::types::ReferenceImage;
|
||||
use crate::vector_engine::{constants::GPT_IMAGE_2_MODEL, types::ReferenceImage};
|
||||
use tokio::{
|
||||
io::{AsyncReadExt, AsyncWriteExt},
|
||||
net::TcpListener,
|
||||
@@ -330,6 +334,7 @@ mod tests {
|
||||
let response = send_vector_engine_multipart_edit_request_with_curl(
|
||||
format!("{base_url}/v1/images/edits").as_str(),
|
||||
"test-key",
|
||||
GPT_IMAGE_2_MODEL,
|
||||
"测试提示词",
|
||||
None,
|
||||
"1024x1024",
|
||||
|
||||
@@ -14,14 +14,15 @@ mod util;
|
||||
pub use audit::PlatformImageFailureAudit;
|
||||
pub use client::{
|
||||
create_vector_engine_image_edit, create_vector_engine_image_edit_with_references,
|
||||
create_vector_engine_image_generation,
|
||||
create_vector_engine_image_edit_with_references_and_model,
|
||||
create_vector_engine_image_generation, create_vector_engine_image_generation_with_model,
|
||||
};
|
||||
pub use constants::{GPT_IMAGE_2_MODEL, VECTOR_ENGINE_GPT_IMAGE_2_MODEL, VECTOR_ENGINE_PROVIDER};
|
||||
pub use error::{PlatformImageError, PlatformImageStatusHint};
|
||||
pub use image_source::download_remote_image;
|
||||
pub use request::{
|
||||
build_vector_engine_image_request_body, normalize_image_size, vector_engine_images_edit_url,
|
||||
vector_engine_images_generation_url,
|
||||
build_vector_engine_image_request_body, build_vector_engine_image_request_body_with_model,
|
||||
normalize_image_size, vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
};
|
||||
pub use transport::build_vector_engine_image_http_client;
|
||||
pub use types::{DownloadedImage, GeneratedImages, ReferenceImage, VectorEngineImageSettings};
|
||||
|
||||
@@ -12,10 +12,29 @@ pub fn build_vector_engine_image_request_body(
|
||||
candidate_count: u32,
|
||||
_reference_images: &[String],
|
||||
) -> Value {
|
||||
build_vector_engine_image_request_body_with_model(
|
||||
GPT_IMAGE_2_MODEL,
|
||||
prompt,
|
||||
negative_prompt,
|
||||
size,
|
||||
candidate_count,
|
||||
_reference_images,
|
||||
)
|
||||
}
|
||||
|
||||
pub fn build_vector_engine_image_request_body_with_model(
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
_reference_images: &[String],
|
||||
) -> Value {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
let body = Map::from_iter([
|
||||
(
|
||||
"model".to_string(),
|
||||
Value::String(GPT_IMAGE_2_MODEL.to_string()),
|
||||
Value::String(model.to_string()),
|
||||
),
|
||||
(
|
||||
"prompt".to_string(),
|
||||
@@ -31,11 +50,20 @@ pub fn build_vector_engine_image_request_body(
|
||||
Value::Object(body)
|
||||
}
|
||||
|
||||
pub fn normalize_vector_engine_image_model(model: &str) -> &str {
|
||||
match model.trim() {
|
||||
"" => GPT_IMAGE_2_MODEL,
|
||||
value => value,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn normalize_image_size(size: &str) -> String {
|
||||
match size.trim() {
|
||||
"1024*1024" | "1024x1024" | "1:1" => "1024x1024",
|
||||
"1280*720" | "1280x720" | "1600*900" | "1600x900" | "16:9" | "1536x1024" | "2048x1152"
|
||||
| "2k" => "1536x1024",
|
||||
"1280*720" | "1280x720" | "1600*900" | "1600x900" | "16:9" | "1536x1024" | "2k" => {
|
||||
"1536x1024"
|
||||
}
|
||||
"1920*1080" | "1920x1080" | "2048*1152" | "2048x1152" | "2k-16:9" => "2048x1152",
|
||||
"1024*1536" | "1024x1536" | "9:16" => "1024x1536",
|
||||
value if !value.is_empty() => value,
|
||||
_ => "1024x1024",
|
||||
@@ -60,12 +88,14 @@ pub fn vector_engine_images_edit_url(settings: &VectorEngineImageSettings) -> St
|
||||
}
|
||||
|
||||
pub(crate) fn build_vector_engine_image_edit_request_log_params(
|
||||
model: &str,
|
||||
prompt: &str,
|
||||
negative_prompt: Option<&str>,
|
||||
size: &str,
|
||||
candidate_count: u32,
|
||||
reference_images: &[ReferenceImage],
|
||||
) -> Value {
|
||||
let model = normalize_vector_engine_image_model(model);
|
||||
let prompt = prompt.trim();
|
||||
let negative_prompt = negative_prompt
|
||||
.map(str::trim)
|
||||
@@ -91,7 +121,7 @@ pub(crate) fn build_vector_engine_image_edit_request_log_params(
|
||||
.sum();
|
||||
|
||||
json!({
|
||||
"model": GPT_IMAGE_2_MODEL,
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"negativePrompt": negative_prompt.unwrap_or_default(),
|
||||
"promptChars": prompt.chars().count(),
|
||||
@@ -125,6 +155,7 @@ mod tests {
|
||||
#[test]
|
||||
fn edit_request_log_params_include_reference_image_sizes_without_secrets_or_bytes() {
|
||||
let params = build_vector_engine_image_edit_request_log_params(
|
||||
GPT_IMAGE_2_MODEL,
|
||||
" 拼图参考图重绘 ",
|
||||
Some(" 文字，水印 "),
|
||||
"1024x1024",
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
use platform_image::vector_engine::{
|
||||
GPT_IMAGE_2_MODEL, ReferenceImage, VECTOR_ENGINE_PROVIDER, VectorEngineImageSettings,
|
||||
build_vector_engine_image_http_client, build_vector_engine_image_request_body,
|
||||
create_vector_engine_image_edit, create_vector_engine_image_generation,
|
||||
build_vector_engine_image_request_body_with_model, create_vector_engine_image_edit,
|
||||
create_vector_engine_image_generation,
|
||||
vector_engine_images_edit_url, vector_engine_images_generation_url,
|
||||
};
|
||||
use std::{
|
||||
@@ -43,6 +44,31 @@ fn vector_engine_module_exposes_provider_protocol_helpers() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vector_engine_normalizes_2k_landscape_spec_size() {
|
||||
let body = build_vector_engine_image_request_body("生成规范图", None, "2048x1152", 1, &[]);
|
||||
|
||||
assert_eq!(body["model"], GPT_IMAGE_2_MODEL);
|
||||
assert_eq!(body["size"], "2048x1152");
|
||||
assert_eq!(body["n"], 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn vector_engine_request_body_can_use_nanobanana2_model() {
|
||||
let body = build_vector_engine_image_request_body_with_model(
|
||||
"gemini-3.1-flash-image-preview",
|
||||
"生成图标 spritesheet",
|
||||
None,
|
||||
"512x512",
|
||||
1,
|
||||
&[],
|
||||
);
|
||||
|
||||
assert_eq!(body["model"], "gemini-3.1-flash-image-preview");
|
||||
assert_eq!(body["size"], "512x512");
|
||||
assert_eq!(body["n"], 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn vector_engine_image_edit_retries_send_timeout_once_and_succeeds() {
|
||||
let listener = TcpListener::bind("127.0.0.1:0")
|
||||
|
||||
@@ -304,6 +304,48 @@ pub struct CharacterAnimationGenerateResponse {
|
||||
pub preview_video_path: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCharacterAnimationGenerateRequest {
|
||||
pub source_layer_id: String,
|
||||
pub source_image_src: String,
|
||||
pub source_width: u32,
|
||||
pub source_height: u32,
|
||||
pub prompt_text: String,
|
||||
pub resolution: String,
|
||||
pub ratio: String,
|
||||
pub frame_count: u32,
|
||||
pub duration_seconds: u32,
|
||||
pub price_mud_points: u32,
|
||||
pub model: String,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCharacterAnimationFramePayload {
|
||||
pub frame_index: u32,
|
||||
pub image_src: String,
|
||||
pub width: u32,
|
||||
pub height: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCharacterAnimationGenerateResponse {
|
||||
pub ok: bool,
|
||||
pub task_id: String,
|
||||
pub model: String,
|
||||
pub prompt: String,
|
||||
pub preview_video_path: String,
|
||||
pub frames: Vec<EditorCharacterAnimationFramePayload>,
|
||||
pub frame_count: u32,
|
||||
pub duration_seconds: u32,
|
||||
pub frame_width: u32,
|
||||
pub frame_height: u32,
|
||||
pub fps: u32,
|
||||
pub price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct CharacterAnimationDraftPayload {
|
||||
@@ -815,6 +857,57 @@ mod tests {
|
||||
assert_eq!(payload["draftId"], json!("animation-import-1"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_request_uses_expected_camel_case_shape() {
|
||||
let payload = serde_json::to_value(EditorCharacterAnimationGenerateRequest {
|
||||
source_layer_id: "layer-1".to_string(),
|
||||
source_image_src: "/generated-characters/hero/master.png".to_string(),
|
||||
source_width: 768,
|
||||
source_height: 1024,
|
||||
prompt_text: "待机呼吸".to_string(),
|
||||
resolution: "720p".to_string(),
|
||||
ratio: "same".to_string(),
|
||||
frame_count: 48,
|
||||
duration_seconds: 6,
|
||||
price_mud_points: 120,
|
||||
model: "seedance2.0".to_string(),
|
||||
})
|
||||
.expect("request should serialize");
|
||||
|
||||
assert_eq!(payload["sourceLayerId"], json!("layer-1"));
|
||||
assert_eq!(payload["sourceImageSrc"], json!("/generated-characters/hero/master.png"));
|
||||
assert_eq!(payload["priceMudPoints"], json!(120));
|
||||
assert_eq!(payload["model"], json!("seedance2.0"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_response_includes_frames_and_preview_video() {
|
||||
let payload = serde_json::to_value(EditorCharacterAnimationGenerateResponse {
|
||||
ok: true,
|
||||
task_id: "task-1".to_string(),
|
||||
model: "seedance2.0".to_string(),
|
||||
prompt: "生成游戏角色动画".to_string(),
|
||||
preview_video_path: "/generated-character-drafts/editor/layer/preview.mp4".to_string(),
|
||||
frames: vec![EditorCharacterAnimationFramePayload {
|
||||
frame_index: 1,
|
||||
image_src: "/generated-animations/editor/layer/frame01.png".to_string(),
|
||||
width: 768,
|
||||
height: 1024,
|
||||
}],
|
||||
frame_count: 1,
|
||||
duration_seconds: 4,
|
||||
frame_width: 768,
|
||||
frame_height: 1024,
|
||||
fps: 8,
|
||||
price_mud_points: 40,
|
||||
})
|
||||
.expect("response should serialize");
|
||||
|
||||
assert_eq!(payload["previewVideoPath"], json!("/generated-character-drafts/editor/layer/preview.mp4"));
|
||||
assert_eq!(payload["frames"][0]["imageSrc"], json!("/generated-animations/editor/layer/frame01.png"));
|
||||
assert_eq!(payload["fps"], json!(8));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn character_workflow_cache_response_keeps_legacy_shape() {
|
||||
let payload = serde_json::to_value(CharacterWorkflowCacheSaveResponse {
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import type { ButtonHTMLAttributes, ReactNode } from 'react';
|
||||
import type { ButtonHTMLAttributes, CSSProperties, ReactNode } from 'react';
|
||||
|
||||
type PlatformFloatingMenuProps = {
|
||||
children: ReactNode;
|
||||
className?: string;
|
||||
label?: string;
|
||||
placement?: 'bottom-start' | 'bottom-end' | 'top-start' | 'top-end';
|
||||
style?: CSSProperties;
|
||||
};
|
||||
|
||||
type PlatformFloatingMenuItemProps = Omit<
|
||||
@@ -24,6 +25,7 @@ export function PlatformFloatingMenu({
|
||||
className,
|
||||
label,
|
||||
placement = 'top-end',
|
||||
style,
|
||||
}: PlatformFloatingMenuProps) {
|
||||
return (
|
||||
<div
|
||||
@@ -36,6 +38,7 @@ export function PlatformFloatingMenu({
|
||||
.join(' ')}
|
||||
role="menu"
|
||||
aria-label={label}
|
||||
style={style}
|
||||
>
|
||||
{children}
|
||||
</div>
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
1085
src/index.css
1085
src/index.css
File diff suppressed because it is too large
Load Diff
33
src/services/image-editor/editorImageReference.test.ts
Normal file
33
src/services/image-editor/editorImageReference.test.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { resolveEditorImageReferenceDataUrl } from './editorImageReference';
|
||||
|
||||
describe('editorImageReference', () => {
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('passes through base64 image data urls without fetching', async () => {
|
||||
vi.spyOn(globalThis, 'fetch');
|
||||
|
||||
await expect(
|
||||
resolveEditorImageReferenceDataUrl('data:image/png;base64,c291cmNl'),
|
||||
).resolves.toBe('data:image/png;base64,c291cmNl');
|
||||
expect(globalThis.fetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('converts public image paths to base64 image data urls', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue(
|
||||
new Response(new Uint8Array([104, 101, 108, 108, 111]), {
|
||||
status: 200,
|
||||
headers: {
|
||||
'Content-Type': 'image/webp',
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(
|
||||
resolveEditorImageReferenceDataUrl('/creation-type-references/puzzle.webp'),
|
||||
).resolves.toBe('data:image/webp;base64,aGVsbG8=');
|
||||
});
|
||||
});
|
||||
48
src/services/image-editor/editorImageReference.ts
Normal file
48
src/services/image-editor/editorImageReference.ts
Normal file
@@ -0,0 +1,48 @@
|
||||
import { readAssetBytes } from '../assetReadUrlService';
|
||||
|
||||
function normalizeImageContentType(contentType: string | null) {
|
||||
const mimeType = contentType?.split(';')[0]?.trim().toLowerCase() ?? '';
|
||||
return mimeType.startsWith('image/') ? mimeType : 'image/png';
|
||||
}
|
||||
|
||||
function encodeBytesAsBase64(bytes: Uint8Array) {
|
||||
let binary = '';
|
||||
const chunkSize = 0x8000;
|
||||
for (let offset = 0; offset < bytes.length; offset += chunkSize) {
|
||||
binary += String.fromCharCode(...bytes.subarray(offset, offset + chunkSize));
|
||||
}
|
||||
return btoa(binary);
|
||||
}
|
||||
|
||||
/**
|
||||
* 图片画布的生成编辑接口只接收图片 Data URL。
|
||||
* 画布里既有上传 / 生成图，也有内置 public 参考图和历史 generated 路径；
|
||||
* 提交给后端前统一读取成 Data URL，避免后端误收到普通 URL 后进入 Data URL 校验错误。
|
||||
*/
|
||||
export async function resolveEditorImageReferenceDataUrl(
|
||||
source: string,
|
||||
signal?: AbortSignal,
|
||||
) {
|
||||
const normalizedSource = source.trim();
|
||||
if (!normalizedSource) {
|
||||
throw new Error('图片参考图不能为空');
|
||||
}
|
||||
if (normalizedSource.startsWith('data:image/')) {
|
||||
return normalizedSource;
|
||||
}
|
||||
if (normalizedSource.startsWith('data:')) {
|
||||
throw new Error('图片参考图必须是图片 Data URL');
|
||||
}
|
||||
|
||||
const response = await readAssetBytes(normalizedSource, {
|
||||
signal,
|
||||
expireSeconds: 300,
|
||||
});
|
||||
const mimeType = normalizeImageContentType(response.headers.get('Content-Type'));
|
||||
const bytes = new Uint8Array(await response.arrayBuffer());
|
||||
if (bytes.byteLength <= 0) {
|
||||
throw new Error('图片参考图为空');
|
||||
}
|
||||
|
||||
return `data:${mimeType};base64,${encodeBytesAsBase64(bytes)}`;
|
||||
}
|
||||
@@ -9,9 +9,11 @@ import {
|
||||
deleteEditorAssetFolder,
|
||||
deleteEditorProject,
|
||||
editEditorImage,
|
||||
generateEditorCharacterAnimation,
|
||||
generateEditorIconSpritesheet,
|
||||
generateEditorImage,
|
||||
loadEditorAssetLibrary,
|
||||
listEditorProjects,
|
||||
loadEditorAssetLibrary,
|
||||
loadEditorProject,
|
||||
loadOrCreateRecentEditorProject,
|
||||
renameEditorProject,
|
||||
@@ -117,11 +119,15 @@ describe('editorProjectClient', () => {
|
||||
projectId: 'editor-project-1',
|
||||
title: '默认画布',
|
||||
viewport: { x: 12, y: 24, scale: 0.5 },
|
||||
layers: [{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 }],
|
||||
layers: [
|
||||
{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 },
|
||||
],
|
||||
updatedAt: '2026-06-12T00:00:00.000Z',
|
||||
},
|
||||
viewport: { x: 12, y: 24, scale: 0.5 },
|
||||
layers: [{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 }],
|
||||
layers: [
|
||||
{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 },
|
||||
],
|
||||
resources: [],
|
||||
updatedAt: '2026-06-12T00:00:00.000Z',
|
||||
},
|
||||
@@ -139,7 +145,9 @@ describe('editorProjectClient', () => {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
viewport: { x: 12, y: 24, scale: 0.5 },
|
||||
layers: [{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 }],
|
||||
layers: [
|
||||
{ layerId: 'layer-1', resourceId: 'resource-1', x: 10, y: 20 },
|
||||
],
|
||||
}),
|
||||
}),
|
||||
'保存图片画布工程失败',
|
||||
@@ -559,6 +567,158 @@ describe('editorProjectClient', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('generates icon spritesheets through the dedicated backend BFF', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
spritesheetImageSrc: 'data:image/png;base64,sheet',
|
||||
spritesheetWidth: 512,
|
||||
spritesheetHeight: 512,
|
||||
iconImageSrcs: [
|
||||
{
|
||||
name: '返回按钮',
|
||||
imageSrc: 'data:image/png;base64,back',
|
||||
width: 96,
|
||||
height: 96,
|
||||
},
|
||||
],
|
||||
prompt: '图标素材 prompt',
|
||||
actualPrompt: '图标素材 prompt',
|
||||
model: 'gemini-3.1-flash-image-preview',
|
||||
provider: 'VectorEngine',
|
||||
taskId: 'icon-spritesheet-task-1',
|
||||
});
|
||||
|
||||
const result = await generateEditorIconSpritesheet({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
iconDescriptions: ['返回按钮', '设置按钮'],
|
||||
});
|
||||
|
||||
expect(result.taskId).toBe('icon-spritesheet-task-1');
|
||||
expect(requestJsonMock).toHaveBeenCalledWith(
|
||||
'/api/editor/icon-spritesheets/generations',
|
||||
expect.objectContaining({
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
iconDescriptions: ['返回按钮', '设置按钮'],
|
||||
model: 'gemini-3.1-flash-image-preview',
|
||||
}),
|
||||
}),
|
||||
'生成图标素材失败',
|
||||
expect.objectContaining({
|
||||
clearAuthOnUnauthorized: false,
|
||||
notifyAuthStateChange: false,
|
||||
timeoutMs: 1_200_000,
|
||||
retry: { maxRetries: 0 },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('passes spec generation size and kind to the backend BFF', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
imageSrc: 'data:image/png;base64,spec',
|
||||
width: 2048,
|
||||
height: 1152,
|
||||
sourceType: 'generated',
|
||||
prompt: '生成规范图',
|
||||
actualPrompt: '生成规范图',
|
||||
model: 'gpt-image-2',
|
||||
provider: 'VectorEngine',
|
||||
taskId: 'vector-spec-1',
|
||||
});
|
||||
|
||||
const result = await generateEditorImage({
|
||||
prompt: '生成规范图',
|
||||
size: '2048x1152',
|
||||
kind: 'spec',
|
||||
model: 'gpt-image-2',
|
||||
});
|
||||
|
||||
expect(result.width).toBe(2048);
|
||||
expect(requestJsonMock).toHaveBeenCalledWith(
|
||||
'/api/editor/images/generations',
|
||||
expect.objectContaining({
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt: '生成规范图',
|
||||
size: '2048x1152',
|
||||
kind: 'spec',
|
||||
model: 'gpt-image-2',
|
||||
}),
|
||||
}),
|
||||
'生成图片失败',
|
||||
expect.objectContaining({
|
||||
timeoutMs: 1_200_000,
|
||||
retry: { maxRetries: 0 },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('generates editor character animations through the backend BFF', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
taskId: 'character-animation-1',
|
||||
model: 'seedance2.0',
|
||||
prompt: '生成游戏角色动画\n动作描述：\n待机',
|
||||
previewVideoPath: '/generated-character-drafts/editor/preview.mp4',
|
||||
frames: [
|
||||
{
|
||||
frameIndex: 1,
|
||||
imageSrc: '/generated-character-drafts/editor/frame01.png',
|
||||
width: 1024,
|
||||
height: 1024,
|
||||
},
|
||||
],
|
||||
frameCount: 32,
|
||||
durationSeconds: 4,
|
||||
fps: 8,
|
||||
priceMudPoints: 40,
|
||||
});
|
||||
|
||||
const result = await generateEditorCharacterAnimation({
|
||||
sourceLayerId: 'layer-character',
|
||||
sourceImageSrc: 'data:image/png;base64,character',
|
||||
sourceWidth: 1024,
|
||||
sourceHeight: 1024,
|
||||
promptText: '待机',
|
||||
resolution: '480p',
|
||||
ratio: 'same',
|
||||
frameCount: 32,
|
||||
durationSeconds: 4,
|
||||
priceMudPoints: 40,
|
||||
model: 'seedance2.0',
|
||||
});
|
||||
|
||||
expect(result.taskId).toBe('character-animation-1');
|
||||
expect(requestJsonMock).toHaveBeenCalledWith(
|
||||
'/api/editor/character-animations/generations',
|
||||
expect.objectContaining({
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourceLayerId: 'layer-character',
|
||||
sourceImageSrc: 'data:image/png;base64,character',
|
||||
sourceWidth: 1024,
|
||||
sourceHeight: 1024,
|
||||
promptText: '待机',
|
||||
resolution: '480p',
|
||||
ratio: 'same',
|
||||
frameCount: 32,
|
||||
durationSeconds: 4,
|
||||
priceMudPoints: 40,
|
||||
model: 'seedance2.0',
|
||||
}),
|
||||
}),
|
||||
'生成角色动画失败',
|
||||
expect.objectContaining({
|
||||
clearAuthOnUnauthorized: false,
|
||||
notifyAuthStateChange: false,
|
||||
timeoutMs: 1_200_000,
|
||||
retry: { maxRetries: 0 },
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('edits editor images through the backend BFF', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
imageSrc: 'data:image/png;base64,edited',
|
||||
@@ -575,6 +735,8 @@ describe('editorProjectClient', () => {
|
||||
const result = await editEditorImage({
|
||||
prompt: '把画面改成黄昏光线',
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
size: '1024x1024',
|
||||
model: 'gpt-image-2',
|
||||
});
|
||||
|
||||
expect(result.taskId).toBe('vector-edit-1');
|
||||
@@ -586,6 +748,8 @@ describe('editorProjectClient', () => {
|
||||
body: JSON.stringify({
|
||||
prompt: '把画面改成黄昏光线',
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
size: '1024x1024',
|
||||
model: 'gpt-image-2',
|
||||
}),
|
||||
}),
|
||||
'修改图片失败',
|
||||
|
||||
@@ -4,6 +4,11 @@ const EDITOR_PROJECT_API_BASE = '/api/editor/projects';
|
||||
const EDITOR_ASSET_API_BASE = '/api/editor/assets';
|
||||
const EDITOR_IMAGE_GENERATION_API = '/api/editor/images/generations';
|
||||
const EDITOR_IMAGE_EDIT_API = '/api/editor/images/edits';
|
||||
const EDITOR_ICON_SPRITESHEET_GENERATION_API =
|
||||
'/api/editor/icon-spritesheets/generations';
|
||||
const EDITOR_CHARACTER_ANIMATION_GENERATION_API =
|
||||
'/api/editor/character-animations/generations';
|
||||
const EDITOR_ICON_SPRITESHEET_MODEL = 'gemini-3.1-flash-image-preview';
|
||||
const DEFAULT_PROJECT_TITLE = '未命名画布';
|
||||
const EDITOR_PROJECT_REQUEST_OPTIONS = {
|
||||
clearAuthOnUnauthorized: false,
|
||||
@@ -81,15 +86,29 @@ export type EditorAssetLibrarySnapshot = {
|
||||
|
||||
export type EditorImageGenerationInput = {
|
||||
prompt: string;
|
||||
size?: string;
|
||||
kind?: 'spec' | 'character' | 'quick-edit';
|
||||
model?: string;
|
||||
referenceImageSrcs?: string[];
|
||||
};
|
||||
|
||||
export type EditorIconSpritesheetGenerationInput = {
|
||||
referenceImageSrc: string;
|
||||
iconDescriptions: string[];
|
||||
model?: string;
|
||||
};
|
||||
|
||||
export type EditorImageEditInput = {
|
||||
prompt: string;
|
||||
sourceImageSrc: string;
|
||||
size?: string;
|
||||
model?: string;
|
||||
};
|
||||
|
||||
export type EditorImageGenerationResult = {
|
||||
imageSrc: string;
|
||||
objectKey?: string | null;
|
||||
assetObjectId?: string | null;
|
||||
width: number;
|
||||
height: number;
|
||||
sourceType: 'generated';
|
||||
@@ -100,6 +119,72 @@ export type EditorImageGenerationResult = {
|
||||
taskId: string;
|
||||
};
|
||||
|
||||
export type EditorIconSpritesheetIconResult = {
|
||||
name: string;
|
||||
imageSrc: string;
|
||||
width: number;
|
||||
height: number;
|
||||
};
|
||||
|
||||
export type EditorIconSpritesheetGenerationResult = {
|
||||
spritesheetImageSrc: string;
|
||||
spritesheetWidth: number;
|
||||
spritesheetHeight: number;
|
||||
iconImageSrcs: EditorIconSpritesheetIconResult[];
|
||||
prompt: string;
|
||||
actualPrompt?: string | null;
|
||||
model: string;
|
||||
provider: string;
|
||||
taskId: string;
|
||||
};
|
||||
|
||||
export type EditorCharacterAnimationResolution = '480p' | '720p';
|
||||
|
||||
export type EditorCharacterAnimationRatio =
|
||||
| 'same'
|
||||
| '1:1'
|
||||
| '4:3'
|
||||
| '16:9'
|
||||
| '9:16'
|
||||
| '3:4';
|
||||
|
||||
export type EditorCharacterAnimationFrameCount = 32 | 40 | 48;
|
||||
|
||||
export type EditorCharacterAnimationDurationSeconds = 4 | 5 | 6;
|
||||
|
||||
export type EditorCharacterAnimationGenerationInput = {
|
||||
sourceLayerId: string;
|
||||
sourceImageSrc: string;
|
||||
sourceWidth: number;
|
||||
sourceHeight: number;
|
||||
promptText: string;
|
||||
resolution: EditorCharacterAnimationResolution;
|
||||
ratio: EditorCharacterAnimationRatio;
|
||||
frameCount: EditorCharacterAnimationFrameCount;
|
||||
durationSeconds: EditorCharacterAnimationDurationSeconds;
|
||||
priceMudPoints: number;
|
||||
model: 'seedance2.0';
|
||||
};
|
||||
|
||||
export type EditorCharacterAnimationFrameResult = {
|
||||
frameIndex: number;
|
||||
imageSrc: string;
|
||||
width: number;
|
||||
height: number;
|
||||
};
|
||||
|
||||
export type EditorCharacterAnimationGenerationResult = {
|
||||
taskId: string;
|
||||
model: 'seedance2.0';
|
||||
prompt: string;
|
||||
previewVideoPath: string;
|
||||
frames: EditorCharacterAnimationFrameResult[];
|
||||
frameCount: number;
|
||||
durationSeconds: number;
|
||||
fps: number;
|
||||
priceMudPoints: number;
|
||||
};
|
||||
|
||||
export type EditorProjectSnapshot = {
|
||||
projectId: string;
|
||||
title: string;
|
||||
@@ -194,6 +279,8 @@ type EditorAssetResponse = {
|
||||
};
|
||||
|
||||
type EditorImageGenerationResponse = EditorImageGenerationResult;
|
||||
type EditorIconSpritesheetGenerationResponse =
|
||||
EditorIconSpritesheetGenerationResult;
|
||||
|
||||
function jsonRequest(method: 'POST' | 'PATCH', body: Record<string, unknown>) {
|
||||
return {
|
||||
@@ -222,10 +309,14 @@ export async function loadRecentEditorProject() {
|
||||
);
|
||||
}
|
||||
|
||||
export async function createEditorProject(input: EditorProjectCreateInput = {}) {
|
||||
export async function createEditorProject(
|
||||
input: EditorProjectCreateInput = {},
|
||||
) {
|
||||
const response = await requestJson<EditorProjectResponse>(
|
||||
EDITOR_PROJECT_API_BASE,
|
||||
jsonRequest('POST', { title: input.title?.trim() || DEFAULT_PROJECT_TITLE }),
|
||||
jsonRequest('POST', {
|
||||
title: input.title?.trim() || DEFAULT_PROJECT_TITLE,
|
||||
}),
|
||||
'创建图片画布工程失败',
|
||||
EDITOR_PROJECT_REQUEST_OPTIONS,
|
||||
);
|
||||
@@ -309,7 +400,10 @@ export async function loadEditorAssetLibrary() {
|
||||
return response.library;
|
||||
}
|
||||
|
||||
export async function createEditorAssetFolder(label: string, sortOrder?: number) {
|
||||
export async function createEditorAssetFolder(
|
||||
label: string,
|
||||
sortOrder?: number,
|
||||
) {
|
||||
const response = await requestJson<EditorAssetFolderResponse>(
|
||||
`${EDITOR_ASSET_API_BASE}/folders`,
|
||||
jsonRequest('POST', { label, sortOrder }),
|
||||
@@ -352,7 +446,10 @@ export async function createEditorAsset(input: EditorAssetCreateInput) {
|
||||
return response.asset;
|
||||
}
|
||||
|
||||
export async function updateEditorAsset(assetId: string, input: EditorAssetUpdateInput) {
|
||||
export async function updateEditorAsset(
|
||||
assetId: string,
|
||||
input: EditorAssetUpdateInput,
|
||||
) {
|
||||
const response = await requestJson<EditorAssetResponse>(
|
||||
`${EDITOR_ASSET_API_BASE}/${encodeURIComponent(assetId)}`,
|
||||
jsonRequest('PATCH', input),
|
||||
@@ -375,7 +472,15 @@ export async function deleteEditorAsset(assetId: string) {
|
||||
export async function generateEditorImage(input: EditorImageGenerationInput) {
|
||||
return requestJson<EditorImageGenerationResponse>(
|
||||
EDITOR_IMAGE_GENERATION_API,
|
||||
jsonRequest('POST', { prompt: input.prompt }),
|
||||
jsonRequest('POST', {
|
||||
prompt: input.prompt,
|
||||
...(input.size ? { size: input.size } : {}),
|
||||
...(input.kind ? { kind: input.kind } : {}),
|
||||
...(input.model ? { model: input.model } : {}),
|
||||
...(input.referenceImageSrcs?.length
|
||||
? { referenceImageSrcs: input.referenceImageSrcs }
|
||||
: {}),
|
||||
}),
|
||||
'生成图片失败',
|
||||
{
|
||||
...EDITOR_PROJECT_REQUEST_OPTIONS,
|
||||
@@ -387,12 +492,35 @@ export async function generateEditorImage(input: EditorImageGenerationInput) {
|
||||
);
|
||||
}
|
||||
|
||||
export async function generateEditorIconSpritesheet(
|
||||
input: EditorIconSpritesheetGenerationInput,
|
||||
) {
|
||||
return requestJson<EditorIconSpritesheetGenerationResponse>(
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_API,
|
||||
jsonRequest('POST', {
|
||||
referenceImageSrc: input.referenceImageSrc,
|
||||
iconDescriptions: input.iconDescriptions,
|
||||
model: input.model?.trim() || EDITOR_ICON_SPRITESHEET_MODEL,
|
||||
}),
|
||||
'生成图标素材失败',
|
||||
{
|
||||
...EDITOR_PROJECT_REQUEST_OPTIONS,
|
||||
timeoutMs: 1_200_000,
|
||||
retry: {
|
||||
maxRetries: 0,
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
export async function editEditorImage(input: EditorImageEditInput) {
|
||||
return requestJson<EditorImageGenerationResponse>(
|
||||
EDITOR_IMAGE_EDIT_API,
|
||||
jsonRequest('POST', {
|
||||
prompt: input.prompt,
|
||||
sourceImageSrc: input.sourceImageSrc,
|
||||
...(input.size ? { size: input.size } : {}),
|
||||
...(input.model ? { model: input.model } : {}),
|
||||
}),
|
||||
'修改图片失败',
|
||||
{
|
||||
@@ -404,3 +532,20 @@ export async function editEditorImage(input: EditorImageEditInput) {
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
export async function generateEditorCharacterAnimation(
|
||||
input: EditorCharacterAnimationGenerationInput,
|
||||
) {
|
||||
return requestJson<EditorCharacterAnimationGenerationResult>(
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_API,
|
||||
jsonRequest('POST', input),
|
||||
'生成角色动画失败',
|
||||
{
|
||||
...EDITOR_PROJECT_REQUEST_OPTIONS,
|
||||
timeoutMs: 1_200_000,
|
||||
retry: {
|
||||
maxRetries: 0,
|
||||
},
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user