画板生成统一接入Worker队列
新增图片画布外部生成队列封装和 worker job 分发 将画板图片、改图、图标、UI 素材、角色动作、视频和音频生成切到队列模式 前端提交生成任务后轮询 queueState 并应用后端项目快照 补充共享契约、测试和 worker 化文档说明
This commit is contained in:
@@ -2632,3 +2632,11 @@
|
||||
- 2026-06-22 补充:所有明确设置绿幕用于后续抠图的 prompt 都必须固定写明 `#00FF00 / RGB(0,255,0)`,不能只写“纯绿色绿幕”或“接近 #00FF00”;编辑器角色图通用抠图额外开启暗绿 / 灰绿绿幕背景识别,只作为生成模型偏离标准亮绿时的兜底。该宽松识别只参与从画布边缘连通扩散出的背景清理,不参与全图断开绿色区域删除,避免误伤角色衣物或纹理。
|
||||
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
|
||||
- 验证方式:`cargo test -p api-server --manifest-path server-rs/Cargo.toml editor_character_image_general_cutout`、`cargo test -p platform-image --manifest-path server-rs/Cargo.toml generated_asset_sheet_muted_green_alpha_requires_explicit_option`、`cargo check -p api-server --manifest-path server-rs/Cargo.toml`、`npm run check:encoding`、`git diff --check`。
|
||||
|
||||
## 2026-06-22 图片画布外部生成统一改走 worker 队列
|
||||
|
||||
- 背景:图片画布的图片、改图、图标素材、UI 素材提取、角色动作、视频和音频生成都可能长时间等待外部 provider;如果继续由 HTTP handler 同步执行,生产只能扩 API 进程,不能独立扩生成吞吐。
|
||||
- 决策:`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画板所有外部 provider 生成入口统一入 `external_generation_job`,job kind 使用 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation`。worker 成功后由后端写 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`;前端只轮询 BFF job 状态并重新读取项目快照,不从队列 payload 或本地临时状态重建完成图层。
|
||||
- 补充:带 `dialogId` 的 `canvasCompletion` 必须读取后端当前 layout 中的最新 generation dialog placeholder;等待期间用户移动占位时,结果层要跟随最新占位。无 dialog 的重绘 / UI 素材提取等入口使用明确的右侧完成占位;生成器已删除时不把结果重新塞回画布。
|
||||
- 影响范围:`server-rs/crates/api-server/src/editor_generation_queue.rs`、`server-rs/crates/api-server/src/external_generation_worker.rs`、`server-rs/crates/api-server/src/editor_project.rs`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`。
|
||||
- 验证方式:`cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_canvas_generation --manifest-path server-rs/Cargo.toml`、`cargo test -p shared-contracts --manifest-path server-rs/Cargo.toml`、`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorProjectClient.test.ts`、`npm run typecheck`、`npm run check:encoding`、`git diff --check`。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 外部生成 Worker 化方案
|
||||
|
||||
更新时间:`2026-06-12`
|
||||
更新时间:`2026-06-22`
|
||||
|
||||
## 背景
|
||||
|
||||
@@ -13,7 +13,7 @@
|
||||
- 多个 worker 进程通过 SpacetimeDB 任务表抢占任务,依赖 lease 超时恢复,支持按进程数和单进程并发动态缩扩容。
|
||||
- 本地或小流量同步排查可显式启用 `inline` 模式,由 HTTP handler 复用同一 worker executor 同步执行并返回 `completed`;该模式不创建队列任务,也不具备 worker 横向扩容能力。
|
||||
- SpacetimeDB reducer / procedure 只做任务状态流转,不做网络、文件系统或外部 provider I/O。
|
||||
- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`;本轮扩展到跳一跳、拼消消和敲木鱼的外部图片生成动作。后续玩法继续复用同一队列 Module,不再为每个玩法发明独立队列。
|
||||
- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`,跳一跳、拼消消和敲木鱼的外部图片生成动作,以及图片画布编辑器的图片、改图、图标 spritesheet、UI 素材提取、角色动作、视频、音效和背景音乐生成。后续玩法和编辑器生成入口继续复用同一队列 Module,不再为每个入口发明独立队列。
|
||||
- 第一版外部生成队列粒度固定为“单个用户动作对应单个 job”。例如草稿编译、结果页单槽重生、图集重生都各自入一个 job;job 内部可以串行或并行调用 provider、OSS、SpacetimeDB 写回,但不再拆成“提示词 / 生图 / 切图 / 去背景 / 持久化 / 回写”等阶段 job。阶段进度只作为 `request_payload_json` / 业务 session 的展示状态,不作为队列调度单位。
|
||||
- 不调用外部图片 / 音频 / LLM provider 的动作继续 inline 执行,不为了统一排队而进入 `external_generation_job`。
|
||||
|
||||
@@ -48,7 +48,7 @@ BFF 只做鉴权、授权裁剪、字段脱敏和契约映射;队列事实仍
|
||||
| --- | --- |
|
||||
| `job_id` | 主键,`extgen-` 前缀 UUID |
|
||||
| `dedupe_key` | 唯一键,建议为 `play/action/session/scope` |
|
||||
| `job_kind` | 执行类型,当前拼图为 `puzzle_compile_draft`、`puzzle_generate_images`、`puzzle_generate_ui_background` |
|
||||
| `job_kind` | 执行类型,当前覆盖 `puzzle_compile_draft`、`puzzle_generate_images`、`puzzle_generate_ui_background`、跳一跳 / 拼消消 / 敲木鱼生成动作,以及 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation`、`editor_background_music_generation` |
|
||||
| `owner_user_id` | 触发用户 |
|
||||
| `source_module` | 玩法或能力名,例如 `puzzle` |
|
||||
| `source_entity_id` | session/profile/work 等作用域 |
|
||||
@@ -161,6 +161,20 @@ controller 配置:
|
||||
|
||||
每个玩法迁移时必须同时接入业务写回 lease guard:worker 路径带 `external_generation_job_id / worker_id / lease_token`,inline 路径三项同时为空。过期 worker 不得写 session / work profile;业务失败态写回成功后才允许 job 进入 `failed`。
|
||||
|
||||
### 图片画布编辑器
|
||||
|
||||
图片画布 `/editor/canvas` 下所有会调用外部生成 provider 的入口在 `queue` 模式下入 `external_generation_job`,HTTP handler 只返回 `queueState`:
|
||||
|
||||
- `editor_image_generation`:普通图片、生成规范、角色形象、UI 设计图、宣发素材和图片快速编辑。
|
||||
- `editor_image_edit`:图片编辑 / 修改结果。
|
||||
- `editor_icon_spritesheet_generation`:图标素材 spritesheet 生成和拆分。
|
||||
- `editor_ui_design_asset_extraction`:UI 设计图红框素材提取。
|
||||
- `editor_character_animation_generation`:角色动作视频和帧素材生成。
|
||||
- `editor_video_generation`:画布视频生成和视频素材快速编辑。
|
||||
- `editor_sound_effect_generation` / `editor_background_music_generation`:画布音效与背景音乐。
|
||||
|
||||
画板结果的业务真相仍是 `editor_project_resource`、账号级 `editor_asset` 和 `editor_canvas.layers_json`。请求携带 `projectId + canvasCompletion` 时,worker 成功后读取当前项目 layout,用最新 generation dialog placeholder 或无 dialog 完成占位写入结果图层,并保存项目快照;前端轮询单 job 到 completed 后重新读取项目快照,不从队列 payload 或本地临时响应重建正式图层。生成器已被删除时,worker 只保留生成出的资源 / 素材记录,不把结果重新塞回画布。
|
||||
|
||||
## 验收
|
||||
|
||||
基础检查:
|
||||
|
||||
@@ -245,7 +245,7 @@ npm run check:server-rs-ddd
|
||||
|
||||
- Rust 结构体:`ExternalGenerationJob`
|
||||
- 源码:`server-rs/crates/spacetime-module/src/external_generation.rs`
|
||||
- 用途:外部生成 worker 的持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
|
||||
- 用途:外部生成 worker 的持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft`、`generate_puzzle_images` 与 `generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile;图片画布编辑器的 `editor_image_generation`、`editor_image_edit`、`editor_icon_spritesheet_generation`、`editor_ui_design_asset_extraction`、`editor_character_animation_generation`、`editor_video_generation`、`editor_sound_effect_generation` 和 `editor_background_music_generation` 复用同一队列表,worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`,前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
|
||||
|
||||
### `ai_text_chunk`
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -283,7 +283,10 @@ mod tests {
|
||||
use tokio::net::TcpListener;
|
||||
use tower::ServiceExt;
|
||||
|
||||
use crate::{config::AppConfig, state::AppState};
|
||||
use crate::{
|
||||
config::{AppConfig, ExternalGenerationMode},
|
||||
state::AppState,
|
||||
};
|
||||
|
||||
use super::{build_router, build_spacetime_unavailable_router};
|
||||
|
||||
@@ -1472,7 +1475,11 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_character_animation_accepts_character_image_body_above_default_limit() {
|
||||
let state = AppState::new(AppConfig::default()).expect("state should build");
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138190", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_character_body");
|
||||
let app = build_router(state);
|
||||
@@ -1554,7 +1561,11 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_ui_design_asset_extraction_accepts_image_body_above_default_limit() {
|
||||
let state = AppState::new(AppConfig::default()).expect("state should build");
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138188", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_ui_extract_body");
|
||||
let app = build_router(state);
|
||||
@@ -1599,7 +1610,11 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_image_generation_accepts_reference_body_above_default_limit() {
|
||||
let state = AppState::new(AppConfig::default()).expect("state should build");
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138028", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_image_reference_body");
|
||||
let app = build_router(state);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
use std::{
|
||||
use std::{
|
||||
collections::BTreeMap,
|
||||
fs,
|
||||
path::{Path, PathBuf},
|
||||
@@ -55,6 +55,15 @@ use crate::{
|
||||
build_character_animation_prompt, build_fallback_moderation_safe_animation_prompt,
|
||||
},
|
||||
editor_generation_config::EditorGenerationPricingConfig,
|
||||
editor_generation_queue::{
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
|
||||
EditorGenerationQueuedResponse, editor_generation_queue_state,
|
||||
editor_generation_source_entity_id, enqueue_editor_generation_job,
|
||||
},
|
||||
editor_project::{
|
||||
EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item,
|
||||
complete_editor_canvas_generation_with_items,
|
||||
},
|
||||
http_error::AppError,
|
||||
platform_errors::map_oss_error,
|
||||
prompt::role_asset_studio::{
|
||||
@@ -525,11 +534,44 @@ pub async fn generate_editor_character_animation(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
payload: Result<Json<EditorCharacterAnimationGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = payload.map_err(|error| {
|
||||
character_animation_error_response(
|
||||
&request_context,
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-character-animation",
|
||||
"message": error.body_text(),
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let source_entity_id = editor_generation_source_entity_id(
|
||||
payload.project_id.as_deref(),
|
||||
payload.source_layer_id.as_str(),
|
||||
);
|
||||
let queue_job = enqueue_editor_generation_job(
|
||||
&state,
|
||||
&request_context,
|
||||
owner_user_id.as_str(),
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成角色动作",
|
||||
&payload,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
return Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorGenerationQueuedResponse {
|
||||
queue_state: editor_generation_queue_state(queue_job),
|
||||
},
|
||||
));
|
||||
}
|
||||
generate_editor_character_animation_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
authenticated.claims().user_id().to_string(),
|
||||
payload,
|
||||
owner_user_id,
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
}
|
||||
@@ -549,6 +591,10 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let project_id = payload.project_id.clone();
|
||||
let canvas_completion = payload.canvas_completion.clone();
|
||||
let generation_inputs = payload.generation_inputs.clone();
|
||||
let source_resource_id = payload.source_resource_id.clone();
|
||||
|
||||
let pricing = state.editor_generation_pricing().map_err(|error| {
|
||||
character_animation_error_response(
|
||||
@@ -613,6 +659,68 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
|
||||
|
||||
let (generated, frames) =
|
||||
result.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
|
||||
let frame_payload_json = serde_json::to_value(&frames).map_err(|error| {
|
||||
character_animation_error_response(
|
||||
&request_context,
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": "editor-character-animation",
|
||||
"message": format!("角色动作序列帧无法写入画布:{error}"),
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let preview_frame = frames.first();
|
||||
let layer_id = format!("layer-editor-character-animation-{task_id}");
|
||||
let item = build_editor_canvas_generated_layer_item(
|
||||
completion,
|
||||
EditorCanvasGeneratedLayerInput {
|
||||
layer_id: layer_id.clone(),
|
||||
resource_id: format!("local-resource-editor-character-animation-{task_id}"),
|
||||
title: completion.title.trim().to_string(),
|
||||
src: preview_frame
|
||||
.map(|frame| frame.image_src.clone())
|
||||
.unwrap_or_else(|| generated.preview_video_path.clone()),
|
||||
media_type: Some("image-sequence".to_string()),
|
||||
thumbnail_src: preview_frame.map(|frame| frame.image_src.clone()),
|
||||
original_width: preview_frame
|
||||
.map(|frame| frame.width)
|
||||
.unwrap_or(normalized.frame_width),
|
||||
original_height: preview_frame
|
||||
.map(|frame| frame.height)
|
||||
.unwrap_or(normalized.frame_height),
|
||||
display_width: completion.placeholder.width,
|
||||
display_height: completion.placeholder.height,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: Some(generated.submitted_prompt.clone()),
|
||||
actual_prompt: Some(generated.submitted_prompt.clone()),
|
||||
model: Some(EDITOR_CHARACTER_ANIMATION_MODEL.to_string()),
|
||||
provider: Some("ark".to_string()),
|
||||
task_id: Some(task_id.clone()),
|
||||
object_key: None,
|
||||
asset_object_id: None,
|
||||
duration_seconds: Some(normalized.duration_seconds),
|
||||
source_resource_id,
|
||||
asset_kind: Some("character-animation".to_string()),
|
||||
generation_inputs,
|
||||
image_sequence_frames: Some(frame_payload_json),
|
||||
preview_video_path: Some(generated.preview_video_path.clone()),
|
||||
},
|
||||
);
|
||||
complete_editor_canvas_generation_with_items(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
project_id.as_deref(),
|
||||
canvas_completion.as_ref(),
|
||||
vec![item],
|
||||
Some(layer_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let completed_project =
|
||||
completed_project.and_then(|project| serde_json::to_value(project).ok());
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
@@ -629,6 +737,8 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
|
||||
fps: normalized.fps,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
frames,
|
||||
project: completed_project,
|
||||
queue_state: None,
|
||||
},
|
||||
))
|
||||
}
|
||||
@@ -639,13 +749,38 @@ pub async fn generate_editor_video(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
payload: Result<Json<EditorVideoGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
generate_editor_video_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
authenticated.claims().user_id().to_string(),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
let Json(payload) = payload.map_err(|error| {
|
||||
editor_video_error_response(
|
||||
&request_context,
|
||||
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
|
||||
"provider": "editor-video",
|
||||
"message": error.body_text(),
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let source_entity_id =
|
||||
editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-video");
|
||||
let queue_job = enqueue_editor_generation_job(
|
||||
&state,
|
||||
&request_context,
|
||||
owner_user_id.as_str(),
|
||||
EDITOR_VIDEO_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成视频",
|
||||
&payload,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
return Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorGenerationQueuedResponse {
|
||||
queue_state: editor_generation_queue_state(queue_job),
|
||||
},
|
||||
));
|
||||
}
|
||||
generate_editor_video_for_owner(state, request_context, owner_user_id, Ok(Json(payload))).await
|
||||
}
|
||||
|
||||
pub(crate) async fn generate_editor_video_for_owner(
|
||||
@@ -663,6 +798,11 @@ pub(crate) async fn generate_editor_video_for_owner(
|
||||
})),
|
||||
)
|
||||
})?;
|
||||
let project_id = payload.project_id.clone();
|
||||
let canvas_completion = payload.canvas_completion.clone();
|
||||
let generation_inputs = payload.generation_inputs.clone();
|
||||
let source_resource_id = payload.source_resource_id.clone();
|
||||
let asset_kind = payload.asset_kind.clone();
|
||||
|
||||
let pricing = state.editor_generation_pricing().map_err(|error| {
|
||||
editor_video_error_response(
|
||||
@@ -698,6 +838,52 @@ pub(crate) async fn generate_editor_video_for_owner(
|
||||
)
|
||||
.await
|
||||
.map_err(|error| editor_video_error_response(&request_context, error))?;
|
||||
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
|
||||
let layer_id = format!("layer-editor-video-{task_id}");
|
||||
let item = build_editor_canvas_generated_layer_item(
|
||||
completion,
|
||||
EditorCanvasGeneratedLayerInput {
|
||||
layer_id: layer_id.clone(),
|
||||
resource_id: format!("local-resource-editor-video-{task_id}"),
|
||||
title: completion.title.trim().to_string(),
|
||||
src: generated.preview_video_path.clone(),
|
||||
media_type: Some("video".to_string()),
|
||||
thumbnail_src: None,
|
||||
original_width: normalized.width,
|
||||
original_height: normalized.height,
|
||||
display_width: completion.placeholder.width,
|
||||
display_height: completion.placeholder.height,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: Some(normalized.prompt.clone()),
|
||||
actual_prompt: Some(generated.submitted_prompt.clone()),
|
||||
model: Some(normalized.model.clone()),
|
||||
provider: Some("VectorEngine".to_string()),
|
||||
task_id: Some(task_id.clone()),
|
||||
object_key: generated.object_key.clone(),
|
||||
asset_object_id: generated.asset_object_id.clone(),
|
||||
duration_seconds: Some(normalized.duration_seconds),
|
||||
source_resource_id,
|
||||
asset_kind: Some(asset_kind.unwrap_or_else(|| "video".to_string())),
|
||||
generation_inputs,
|
||||
image_sequence_frames: None,
|
||||
preview_video_path: None,
|
||||
},
|
||||
);
|
||||
complete_editor_canvas_generation_with_items(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
project_id.as_deref(),
|
||||
canvas_completion.as_ref(),
|
||||
vec![item],
|
||||
Some(layer_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| editor_video_error_response(&request_context, error))?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let completed_project =
|
||||
completed_project.and_then(|project| serde_json::to_value(project).ok());
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
@@ -717,6 +903,8 @@ pub(crate) async fn generate_editor_video_for_owner(
|
||||
duration_seconds: normalized.duration_seconds,
|
||||
resolution: normalized.resolution,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
project: completed_project,
|
||||
queue_state: None,
|
||||
},
|
||||
))
|
||||
}
|
||||
@@ -4909,6 +5097,10 @@ mod tests {
|
||||
frame_count: 48,
|
||||
duration_seconds: 6,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
})
|
||||
.expect("editor request should normalize");
|
||||
|
||||
@@ -4943,6 +5135,10 @@ mod tests {
|
||||
frame_count: 32,
|
||||
duration_seconds: 4,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
})
|
||||
.expect("editor request should normalize");
|
||||
|
||||
@@ -4966,6 +5162,10 @@ mod tests {
|
||||
frame_count: 48,
|
||||
duration_seconds: 4,
|
||||
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
})
|
||||
.expect_err("invalid frame/duration pair should fail");
|
||||
|
||||
@@ -5121,6 +5321,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("editor video request should normalize");
|
||||
|
||||
@@ -5146,6 +5351,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance fast request should normalize");
|
||||
|
||||
@@ -5168,6 +5378,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance standard request should normalize");
|
||||
|
||||
@@ -5190,6 +5405,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance standard 1080p request should normalize");
|
||||
|
||||
@@ -5213,6 +5433,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect_err("seedance fast should reject 1080p");
|
||||
|
||||
@@ -5235,6 +5460,11 @@ mod tests {
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance multimodal request should normalize");
|
||||
|
||||
@@ -5266,6 +5496,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: vec!["data:video/mp4;base64,video".to_string()],
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect_err("Seedance reference videos must not accept Base64 data URLs");
|
||||
|
||||
@@ -5287,6 +5522,11 @@ mod tests {
|
||||
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
|
||||
reference_video_srcs: vec!["https://assets.example.com/reference.mp4".to_string()],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance multimodal request should normalize");
|
||||
|
||||
@@ -5328,6 +5568,11 @@ mod tests {
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("seedance objectKey request should normalize");
|
||||
|
||||
@@ -5363,6 +5608,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: vec!["data:audio/wav;base64,audio".to_string()],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect_err("audio-only references should fail");
|
||||
|
||||
@@ -5404,7 +5654,6 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
fn assert_function_contains(source: &str, start: &str, end: &str, snippets: &[&str]) {
|
||||
let start_index = source
|
||||
.find(start)
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
use axum::http::StatusCode;
|
||||
use serde::Serialize;
|
||||
use serde_json::json;
|
||||
use shared_contracts::external_generation::{
|
||||
ExternalGenerationJobStatus, ExternalGenerationJobStatusRecord,
|
||||
};
|
||||
use shared_kernel::{build_prefixed_uuid_id, offset_datetime_to_unix_micros};
|
||||
use spacetime_client::{ExternalGenerationJobEnqueueRecordInput, ExternalGenerationJobRecord};
|
||||
|
||||
use crate::{http_error::AppError, request_context::RequestContext, state::AppState};
|
||||
|
||||
pub(crate) const EDITOR_IMAGE_GENERATION_JOB_KIND: &str = "editor_image_generation";
|
||||
pub(crate) const EDITOR_IMAGE_EDIT_JOB_KIND: &str = "editor_image_edit";
|
||||
pub(crate) const EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND: &str =
|
||||
"editor_icon_spritesheet_generation";
|
||||
pub(crate) const EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND: &str =
|
||||
"editor_ui_design_asset_extraction";
|
||||
pub(crate) const EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND: &str =
|
||||
"editor_character_animation_generation";
|
||||
pub(crate) const EDITOR_VIDEO_GENERATION_JOB_KIND: &str = "editor_video_generation";
|
||||
pub(crate) const EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND: &str = "editor_sound_effect_generation";
|
||||
pub(crate) const EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND: &str =
|
||||
"editor_background_music_generation";
|
||||
|
||||
pub(crate) const EDITOR_GENERATION_QUEUE_SOURCE_MODULE: &str = "editor-canvas";
|
||||
const EDITOR_GENERATION_QUEUE_PROVIDER: &str = "editor-generation-worker";
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub(crate) struct EditorGenerationQueuedResponse {
|
||||
pub(crate) queue_state: ExternalGenerationJobStatusRecord,
|
||||
}
|
||||
|
||||
pub(crate) async fn enqueue_editor_generation_job<T>(
|
||||
state: &AppState,
|
||||
_request_context: &RequestContext,
|
||||
owner_user_id: &str,
|
||||
job_kind: &str,
|
||||
source_entity_id: impl Into<String>,
|
||||
request_label: impl Into<String>,
|
||||
payload: &T,
|
||||
) -> Result<ExternalGenerationJobRecord, AppError>
|
||||
where
|
||||
T: Serialize,
|
||||
{
|
||||
let job_id = build_prefixed_uuid_id("extgen-");
|
||||
let request_payload_json = serde_json::to_string(payload).map_err(|error| {
|
||||
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
|
||||
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
|
||||
"message": format!("编辑器 worker 任务参数序列化失败:{error}"),
|
||||
}))
|
||||
})?;
|
||||
let now_micros = current_utc_micros();
|
||||
state
|
||||
.spacetime_client()
|
||||
.enqueue_external_generation_job(ExternalGenerationJobEnqueueRecordInput {
|
||||
dedupe_key: format!("editor-canvas:{job_kind}:{job_id}"),
|
||||
job_id,
|
||||
job_kind: job_kind.to_string(),
|
||||
owner_user_id: owner_user_id.to_string(),
|
||||
source_module: EDITOR_GENERATION_QUEUE_SOURCE_MODULE.to_string(),
|
||||
source_entity_id: source_entity_id.into(),
|
||||
request_label: request_label.into(),
|
||||
request_payload_json,
|
||||
max_attempts: 1,
|
||||
available_at_micros: now_micros,
|
||||
created_at_micros: now_micros,
|
||||
})
|
||||
.await
|
||||
.map_err(|error| {
|
||||
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
|
||||
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
|
||||
"message": error.to_string(),
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
pub(crate) fn editor_generation_queue_state(
|
||||
job: ExternalGenerationJobRecord,
|
||||
) -> ExternalGenerationJobStatusRecord {
|
||||
ExternalGenerationJobStatusRecord {
|
||||
operation_id: job.job_id,
|
||||
status: ExternalGenerationJobStatus::Queued,
|
||||
phase_label: job.request_label,
|
||||
phase_detail: "排队中。".to_string(),
|
||||
progress: 8,
|
||||
error: job.last_error_message,
|
||||
updated_at_micros: job.updated_at_micros,
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn editor_generation_source_entity_id(
|
||||
project_id: Option<&str>,
|
||||
fallback: &str,
|
||||
) -> String {
|
||||
project_id
|
||||
.map(str::trim)
|
||||
.filter(|value| !value.is_empty())
|
||||
.unwrap_or(fallback)
|
||||
.to_string()
|
||||
}
|
||||
|
||||
fn current_utc_micros() -> i64 {
|
||||
offset_datetime_to_unix_micros(time::OffsetDateTime::now_utc())
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,6 @@
|
||||
use std::{future::Future, io, pin::Pin, time::Duration};
|
||||
|
||||
use axum::extract::FromRef;
|
||||
use axum::{Json, extract::FromRef};
|
||||
use serde_json::json;
|
||||
use shared_kernel::offset_datetime_to_unix_micros;
|
||||
use spacetime_client::{
|
||||
@@ -15,6 +15,22 @@ use tokio::{
|
||||
use tracing::{error, info, warn};
|
||||
|
||||
use crate::{
|
||||
character_animation_assets::{
|
||||
generate_editor_character_animation_for_owner, generate_editor_video_for_owner,
|
||||
},
|
||||
editor_generation_queue::{
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND,
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND,
|
||||
EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
|
||||
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
|
||||
},
|
||||
editor_project::{
|
||||
EditorGenerationCaller, EditorIconSpritesheetGenerationRequest, EditorImageEditRequest,
|
||||
EditorImageGenerationRequest, EditorUiDesignAssetExtractionRequest,
|
||||
edit_editor_image_for_owner, extract_editor_ui_design_assets_for_owner,
|
||||
generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner,
|
||||
},
|
||||
jump_hop::{
|
||||
JUMP_HOP_COMPILE_DRAFT_JOB_KIND, JumpHopCompileDraftWorkerPayload,
|
||||
execute_jump_hop_compile_draft_worker_job,
|
||||
@@ -31,6 +47,9 @@ use crate::{
|
||||
},
|
||||
request_context::RequestContext,
|
||||
state::{AppState, PuzzleApiState},
|
||||
vector_engine_audio_generation::{
|
||||
generate_editor_background_music_for_owner, generate_editor_sound_effect_for_owner,
|
||||
},
|
||||
wooden_fish::{
|
||||
WOODEN_FISH_GENERATE_IMAGE_ASSETS_JOB_KIND, WoodenFishGenerateImageAssetsWorkerPayload,
|
||||
execute_wooden_fish_generate_image_assets_worker_job,
|
||||
@@ -536,6 +555,234 @@ async fn process_external_generation_job_once(
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_IMAGE_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<EditorImageGenerationRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
) {
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布生图任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_image_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
editor_generation_worker_caller(&job),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(error) => {
|
||||
let message = error.body_text();
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_IMAGE_EDIT_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<EditorImageEditRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
) {
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布改图任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match edit_editor_image_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
editor_generation_worker_caller(&job),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(error) => {
|
||||
let message = error.body_text();
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<EditorIconSpritesheetGenerationRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
) {
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布图标素材任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_icon_spritesheet_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
editor_generation_worker_caller(&job),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(error) => {
|
||||
let message = error.body_text();
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<EditorUiDesignAssetExtractionRequest>(
|
||||
job.request_payload_json.as_str(),
|
||||
) {
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布 UI 素材提取任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match extract_editor_ui_design_assets_for_owner(
|
||||
&state,
|
||||
&request_context,
|
||||
editor_generation_worker_caller(&job),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(error) => {
|
||||
let message = error.body_text();
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<
|
||||
shared_contracts::assets::EditorCharacterAnimationGenerateRequest,
|
||||
>(job.request_payload_json.as_str())
|
||||
{
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布角色动作任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_character_animation_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
job.owner_user_id.clone(),
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(response) => {
|
||||
let message = response_error_message(response).await;
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_VIDEO_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<
|
||||
shared_contracts::assets::EditorVideoGenerateRequest,
|
||||
>(job.request_payload_json.as_str())
|
||||
{
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布视频生成任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_video_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
job.owner_user_id.clone(),
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(response) => {
|
||||
let message = response_error_message(response).await;
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<
|
||||
shared_contracts::assets::EditorSoundEffectGenerateRequest,
|
||||
>(job.request_payload_json.as_str())
|
||||
{
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布音效生成任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_sound_effect_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
job.owner_user_id.clone(),
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(response) => {
|
||||
let message = response_error_message(response).await;
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND => {
|
||||
let payload = match serde_json::from_str::<
|
||||
shared_contracts::assets::EditorBackgroundMusicGenerateRequest,
|
||||
>(job.request_payload_json.as_str())
|
||||
{
|
||||
Ok(payload) => payload,
|
||||
Err(error) => {
|
||||
let message = format!("图片画布背景音乐生成任务参数解析失败:{error}");
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
return Err(message);
|
||||
}
|
||||
};
|
||||
let request_context = worker_request_context(&job);
|
||||
match generate_editor_background_music_for_owner(
|
||||
state.clone(),
|
||||
request_context,
|
||||
job.owner_user_id.clone(),
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
|
||||
Err(response) => {
|
||||
let message = response_error_message(response).await;
|
||||
fail_job(&state, &worker_id, &job, message.clone()).await?;
|
||||
Err(message)
|
||||
}
|
||||
}
|
||||
}
|
||||
unknown => {
|
||||
warn!(
|
||||
job_id = job.job_id,
|
||||
@@ -598,6 +845,43 @@ async fn fail_queue_job_after_worker_error(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn worker_request_context(job: &ExternalGenerationJobRecord) -> RequestContext {
|
||||
RequestContext::new(
|
||||
format!("external-generation-worker-{}", job.job_id),
|
||||
format!("external-generation-worker {}", job.job_kind),
|
||||
std::time::Duration::ZERO,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
fn editor_generation_worker_caller(job: &ExternalGenerationJobRecord) -> EditorGenerationCaller {
|
||||
EditorGenerationCaller {
|
||||
owner_user_id: job.owner_user_id.clone(),
|
||||
audit_subject_user_id: Some(job.owner_user_id.clone()),
|
||||
audit_project_id: Some(job.source_entity_id.clone()),
|
||||
}
|
||||
}
|
||||
|
||||
async fn complete_editor_generation_job(
|
||||
state: &AppState,
|
||||
worker_id: &str,
|
||||
job: &ExternalGenerationJobRecord,
|
||||
) -> Result<(), String> {
|
||||
complete_job(
|
||||
state,
|
||||
worker_id,
|
||||
job,
|
||||
Some(
|
||||
json!({
|
||||
"sourceModule": job.source_module.clone(),
|
||||
"sourceEntityId": job.source_entity_id.clone(),
|
||||
})
|
||||
.to_string(),
|
||||
),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn complete_job(
|
||||
state: &AppState,
|
||||
worker_id: &str,
|
||||
|
||||
@@ -37,6 +37,7 @@ mod custom_world_foundation_draft;
|
||||
mod custom_world_result_prompts;
|
||||
mod custom_world_rpg_draft_prompts;
|
||||
mod editor_generation_config;
|
||||
mod editor_generation_queue;
|
||||
mod editor_project;
|
||||
mod edutainment_baby_drawing;
|
||||
mod edutainment_baby_object;
|
||||
|
||||
@@ -11,9 +11,21 @@ use serde_json::json;
|
||||
use shared_contracts::{assets, creation_audio};
|
||||
|
||||
use crate::{
|
||||
api_response::json_success_body, auth::AuthenticatedAccessToken,
|
||||
editor_generation_config::EditorGenerationPricingConfig, http_error::AppError,
|
||||
request_context::RequestContext, state::AppState,
|
||||
api_response::json_success_body,
|
||||
auth::AuthenticatedAccessToken,
|
||||
editor_generation_config::EditorGenerationPricingConfig,
|
||||
editor_generation_queue::{
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
|
||||
EditorGenerationQueuedResponse, editor_generation_queue_state,
|
||||
editor_generation_source_entity_id, enqueue_editor_generation_job,
|
||||
},
|
||||
editor_project::{
|
||||
EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item,
|
||||
complete_editor_canvas_generation_with_items,
|
||||
},
|
||||
http_error::AppError,
|
||||
request_context::RequestContext,
|
||||
state::AppState,
|
||||
tracking::record_external_generation_run_after_success,
|
||||
};
|
||||
|
||||
@@ -128,13 +140,33 @@ pub async fn generate_editor_sound_effect(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
payload: Result<Json<assets::EditorSoundEffectGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
generate_editor_sound_effect_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
authenticated.claims().user_id().to_string(),
|
||||
payload,
|
||||
)
|
||||
.await
|
||||
let Json(payload) = parse_json_payload(&request_context, payload)?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let source_entity_id = editor_generation_source_entity_id(
|
||||
payload.project_id.as_deref(),
|
||||
"editor-sound-effect",
|
||||
);
|
||||
let queue_job = enqueue_editor_generation_job(
|
||||
&state,
|
||||
&request_context,
|
||||
owner_user_id.as_str(),
|
||||
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成音效",
|
||||
&payload,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
return Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorGenerationQueuedResponse {
|
||||
queue_state: editor_generation_queue_state(queue_job),
|
||||
},
|
||||
));
|
||||
}
|
||||
generate_editor_sound_effect_for_owner(state, request_context, owner_user_id, Ok(Json(payload)))
|
||||
.await
|
||||
}
|
||||
|
||||
pub(crate) async fn generate_editor_sound_effect_for_owner(
|
||||
@@ -144,6 +176,9 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
|
||||
payload: Result<Json<assets::EditorSoundEffectGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = parse_json_payload(&request_context, payload)?;
|
||||
let project_id = payload.project_id.clone();
|
||||
let canvas_completion = payload.canvas_completion.clone();
|
||||
let generation_inputs = payload.generation_inputs.clone();
|
||||
let pricing = state
|
||||
.editor_generation_pricing()
|
||||
.map_err(|error| {
|
||||
@@ -192,6 +227,52 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
|
||||
.audio_src
|
||||
.ok_or_else(|| vector_engine_bad_gateway("音效生成完成但缺少播放地址"))
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
|
||||
let layer_id = format!("layer-editor-sound-effect-{}", generated.task_id);
|
||||
let item = build_editor_canvas_generated_layer_item(
|
||||
completion,
|
||||
EditorCanvasGeneratedLayerInput {
|
||||
layer_id: layer_id.clone(),
|
||||
resource_id: format!("local-resource-editor-sound-effect-{}", generated.task_id),
|
||||
title: completion.title.trim().to_string(),
|
||||
src: audio_src.clone(),
|
||||
media_type: Some("audio".to_string()),
|
||||
thumbnail_src: None,
|
||||
original_width: EDITOR_AUDIO_WIDTH,
|
||||
original_height: EDITOR_AUDIO_HEIGHT,
|
||||
display_width: completion.placeholder.width,
|
||||
display_height: completion.placeholder.height,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: Some(normalized.prompt.clone()),
|
||||
actual_prompt: Some(normalized.prompt.clone()),
|
||||
model: Some(normalized.model.clone()),
|
||||
provider: Some(generated.provider.clone()),
|
||||
task_id: Some(generated.task_id.clone()),
|
||||
object_key: generated.object_key.clone(),
|
||||
asset_object_id: generated.asset_object_id.clone(),
|
||||
duration_seconds: Some(u32::from(normalized.duration)),
|
||||
source_resource_id: None,
|
||||
asset_kind: Some("sound-effect".to_string()),
|
||||
generation_inputs,
|
||||
image_sequence_frames: None,
|
||||
preview_video_path: None,
|
||||
},
|
||||
);
|
||||
complete_editor_canvas_generation_with_items(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
project_id.as_deref(),
|
||||
canvas_completion.as_ref(),
|
||||
vec![item],
|
||||
Some(layer_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let completed_project =
|
||||
completed_project.and_then(|project| serde_json::to_value(project).ok());
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
@@ -210,6 +291,8 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
|
||||
task_id: generated.task_id,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
audio_kind: "sound-effect".to_string(),
|
||||
project: completed_project,
|
||||
queue_state: None,
|
||||
},
|
||||
))
|
||||
}
|
||||
@@ -220,11 +303,36 @@ pub async fn generate_editor_background_music(
|
||||
Extension(authenticated): Extension<AuthenticatedAccessToken>,
|
||||
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = parse_json_payload(&request_context, payload)?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let source_entity_id = editor_generation_source_entity_id(
|
||||
payload.project_id.as_deref(),
|
||||
"editor-background-music",
|
||||
);
|
||||
let queue_job = enqueue_editor_generation_job(
|
||||
&state,
|
||||
&request_context,
|
||||
owner_user_id.as_str(),
|
||||
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
|
||||
source_entity_id,
|
||||
"图片画布生成背景音乐",
|
||||
&payload,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
return Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
EditorGenerationQueuedResponse {
|
||||
queue_state: editor_generation_queue_state(queue_job),
|
||||
},
|
||||
));
|
||||
}
|
||||
generate_editor_background_music_for_owner(
|
||||
state,
|
||||
request_context,
|
||||
authenticated.claims().user_id().to_string(),
|
||||
payload,
|
||||
owner_user_id,
|
||||
Ok(Json(payload)),
|
||||
)
|
||||
.await
|
||||
}
|
||||
@@ -236,6 +344,9 @@ pub(crate) async fn generate_editor_background_music_for_owner(
|
||||
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
|
||||
) -> Result<Json<Value>, Response> {
|
||||
let Json(payload) = parse_json_payload(&request_context, payload)?;
|
||||
let project_id = payload.project_id.clone();
|
||||
let canvas_completion = payload.canvas_completion.clone();
|
||||
let generation_inputs = payload.generation_inputs.clone();
|
||||
let pricing = state
|
||||
.editor_generation_pricing()
|
||||
.map_err(|error| {
|
||||
@@ -282,6 +393,55 @@ pub(crate) async fn generate_editor_background_music_for_owner(
|
||||
.audio_src
|
||||
.ok_or_else(|| vector_engine_bad_gateway("背景音乐生成完成但缺少播放地址"))
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
|
||||
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
|
||||
let layer_id = format!("layer-editor-background-music-{}", generated.task_id);
|
||||
let item = build_editor_canvas_generated_layer_item(
|
||||
completion,
|
||||
EditorCanvasGeneratedLayerInput {
|
||||
layer_id: layer_id.clone(),
|
||||
resource_id: format!(
|
||||
"local-resource-editor-background-music-{}",
|
||||
generated.task_id
|
||||
),
|
||||
title: completion.title.trim().to_string(),
|
||||
src: audio_src.clone(),
|
||||
media_type: Some("audio".to_string()),
|
||||
thumbnail_src: None,
|
||||
original_width: EDITOR_AUDIO_WIDTH,
|
||||
original_height: EDITOR_AUDIO_HEIGHT,
|
||||
display_width: completion.placeholder.width,
|
||||
display_height: completion.placeholder.height,
|
||||
source_type: "generated".to_string(),
|
||||
prompt: Some(normalized.gpt_description_prompt.clone()),
|
||||
actual_prompt: Some(normalized.gpt_description_prompt.clone()),
|
||||
model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()),
|
||||
provider: Some(generated.provider.clone()),
|
||||
task_id: Some(generated.task_id.clone()),
|
||||
object_key: generated.object_key.clone(),
|
||||
asset_object_id: generated.asset_object_id.clone(),
|
||||
duration_seconds: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: Some("background-music".to_string()),
|
||||
generation_inputs,
|
||||
image_sequence_frames: None,
|
||||
preview_video_path: None,
|
||||
},
|
||||
);
|
||||
complete_editor_canvas_generation_with_items(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
project_id.as_deref(),
|
||||
canvas_completion.as_ref(),
|
||||
vec![item],
|
||||
Some(layer_id),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| error.into_response_with_context(Some(&request_context)))?
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let completed_project =
|
||||
completed_project.and_then(|project| serde_json::to_value(project).ok());
|
||||
|
||||
Ok(json_success_body(
|
||||
Some(&request_context),
|
||||
@@ -300,6 +460,8 @@ pub(crate) async fn generate_editor_background_music_for_owner(
|
||||
task_id: generated.task_id,
|
||||
price_mud_points: normalized.price_mud_points,
|
||||
audio_kind: "background-music".to_string(),
|
||||
project: completed_project,
|
||||
queue_state: None,
|
||||
},
|
||||
))
|
||||
}
|
||||
|
||||
@@ -109,6 +109,9 @@ fn editor_sound_effect_request_normalizes_prompt_duration_and_resolves_price() {
|
||||
prompt: " 金币掉落叮当声 ".to_string(),
|
||||
model: None,
|
||||
duration: 7,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("editor sound effect request should normalize");
|
||||
|
||||
@@ -125,6 +128,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() {
|
||||
prompt: "金币掉落叮当声".to_string(),
|
||||
model: Some(" audio1.0 ".to_string()),
|
||||
duration: 5,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("Vidu audio model should be accepted");
|
||||
|
||||
@@ -134,6 +140,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() {
|
||||
prompt: "金币掉落叮当声".to_string(),
|
||||
model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()),
|
||||
duration: 5,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect_err("Suno text-to-sound should be disabled for editor sound effects");
|
||||
|
||||
@@ -148,6 +157,9 @@ fn editor_sound_effect_request_rejects_duration_outside_vidu_range() {
|
||||
prompt: "按钮点击".to_string(),
|
||||
model: None,
|
||||
duration,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect_err("duration outside 2-10 seconds should fail");
|
||||
|
||||
@@ -183,6 +195,9 @@ fn editor_background_music_request_forces_instrumental_and_resolves_price() {
|
||||
normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: " 森林冒险背景音乐 ".to_string(),
|
||||
make_instrumental: false,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("editor background music request should normalize");
|
||||
|
||||
@@ -197,6 +212,9 @@ fn editor_background_music_request_rejects_prompt_over_documented_limit() {
|
||||
normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: "乐".repeat(201),
|
||||
make_instrumental: true,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect_err("Suno gpt_description_prompt should follow Apifox 200 char limit");
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ use std::collections::BTreeMap;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::external_generation::ExternalGenerationJobStatusRecord;
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct CreateDirectUploadTicketRequest {
|
||||
@@ -304,7 +306,7 @@ pub struct CharacterAnimationGenerateResponse {
|
||||
pub preview_video_path: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCharacterAnimationGenerateRequest {
|
||||
pub source_layer_id: String,
|
||||
@@ -317,6 +319,14 @@ pub struct EditorCharacterAnimationGenerateRequest {
|
||||
pub frame_count: u32,
|
||||
pub duration_seconds: u32,
|
||||
pub model: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project_id: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub generation_inputs: Option<Value>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub source_resource_id: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
@@ -328,7 +338,7 @@ pub struct EditorCharacterAnimationFramePayload {
|
||||
pub height: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCharacterAnimationGenerateResponse {
|
||||
pub ok: bool,
|
||||
@@ -343,9 +353,13 @@ pub struct EditorCharacterAnimationGenerateResponse {
|
||||
pub frame_height: u32,
|
||||
pub fps: u32,
|
||||
pub price_mud_points: u32,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project: Option<Value>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorVideoGenerateRequest {
|
||||
pub prompt: String,
|
||||
@@ -363,13 +377,23 @@ pub struct EditorVideoGenerateRequest {
|
||||
pub reference_video_srcs: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub reference_audio_srcs: Vec<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project_id: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub generation_inputs: Option<Value>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub source_resource_id: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub asset_kind: Option<String>,
|
||||
}
|
||||
|
||||
fn default_editor_video_web_search_enabled() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorVideoGenerateResponse {
|
||||
pub ok: bool,
|
||||
@@ -390,6 +414,30 @@ pub struct EditorVideoGenerateResponse {
|
||||
pub duration_seconds: u32,
|
||||
pub resolution: String,
|
||||
pub price_mud_points: u32,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project: Option<Value>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCanvasGenerationPlaceholderPayload {
|
||||
pub x: f64,
|
||||
pub y: f64,
|
||||
pub width: f64,
|
||||
pub height: f64,
|
||||
pub original_width: f64,
|
||||
pub original_height: f64,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorCanvasGenerationCompletionPayload {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub dialog_id: Option<String>,
|
||||
pub title: String,
|
||||
pub placeholder: EditorCanvasGenerationPlaceholderPayload,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
@@ -430,23 +478,35 @@ pub struct EditorIconSpritesheetGenerateResponse {
|
||||
pub price_mud_points: u32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorSoundEffectGenerateRequest {
|
||||
pub prompt: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub model: Option<String>,
|
||||
pub duration: u8,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project_id: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub generation_inputs: Option<Value>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorBackgroundMusicGenerateRequest {
|
||||
pub gpt_description_prompt: String,
|
||||
pub make_instrumental: bool,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project_id: Option<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub generation_inputs: Option<Value>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct EditorAudioGenerateResponse {
|
||||
pub ok: bool,
|
||||
@@ -466,6 +526,10 @@ pub struct EditorAudioGenerateResponse {
|
||||
pub task_id: String,
|
||||
pub price_mud_points: u32,
|
||||
pub audio_kind: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub project: Option<Value>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
|
||||
@@ -992,6 +1056,21 @@ mod tests {
|
||||
frame_count: 48,
|
||||
duration_seconds: 6,
|
||||
model: "seedance2.0-fast".to_string(),
|
||||
project_id: Some("editor-project-1".to_string()),
|
||||
canvas_completion: Some(EditorCanvasGenerationCompletionPayload {
|
||||
dialog_id: Some("dialog-1".to_string()),
|
||||
title: "角色动作".to_string(),
|
||||
placeholder: EditorCanvasGenerationPlaceholderPayload {
|
||||
x: 10.0,
|
||||
y: 20.0,
|
||||
width: 420.0,
|
||||
height: 420.0,
|
||||
original_width: 1024.0,
|
||||
original_height: 1024.0,
|
||||
},
|
||||
}),
|
||||
generation_inputs: None,
|
||||
source_resource_id: Some("resource-source".to_string()),
|
||||
})
|
||||
.expect("request should serialize");
|
||||
|
||||
@@ -1002,6 +1081,9 @@ mod tests {
|
||||
);
|
||||
assert!(payload.get("priceMudPoints").is_none());
|
||||
assert_eq!(payload["model"], json!("seedance2.0-fast"));
|
||||
assert_eq!(payload["projectId"], json!("editor-project-1"));
|
||||
assert_eq!(payload["canvasCompletion"]["dialogId"], json!("dialog-1"));
|
||||
assert_eq!(payload["sourceResourceId"], json!("resource-source"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -1024,6 +1106,8 @@ mod tests {
|
||||
frame_height: 1024,
|
||||
fps: 8,
|
||||
price_mud_points: 40,
|
||||
project: None,
|
||||
queue_state: None,
|
||||
})
|
||||
.expect("response should serialize");
|
||||
|
||||
@@ -1052,6 +1136,11 @@ mod tests {
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("request should serialize");
|
||||
|
||||
@@ -1078,6 +1167,11 @@ mod tests {
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
})
|
||||
.expect("request should serialize");
|
||||
|
||||
@@ -1100,7 +1194,9 @@ mod tests {
|
||||
let payload = serde_json::to_value(EditorVideoGenerateResponse {
|
||||
ok: true,
|
||||
video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(),
|
||||
object_key: Some("generated-character-drafts/editor-videos/task-1/preview.mp4".to_string()),
|
||||
object_key: Some(
|
||||
"generated-character-drafts/editor-videos/task-1/preview.mp4".to_string(),
|
||||
),
|
||||
asset_object_id: Some("assetobj_video_1".to_string()),
|
||||
width: 1280,
|
||||
height: 720,
|
||||
@@ -1113,6 +1209,8 @@ mod tests {
|
||||
duration_seconds: 5,
|
||||
resolution: "480p".to_string(),
|
||||
price_mud_points: 100,
|
||||
project: None,
|
||||
queue_state: None,
|
||||
})
|
||||
.expect("response should serialize");
|
||||
|
||||
@@ -1178,6 +1276,9 @@ mod tests {
|
||||
prompt: "金币掉落叮当声".to_string(),
|
||||
model: Some("audio1.0".to_string()),
|
||||
duration: 7,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("sound request should serialize");
|
||||
assert_eq!(sound_payload["prompt"], json!("金币掉落叮当声"));
|
||||
@@ -1198,6 +1299,9 @@ mod tests {
|
||||
prompt: "按钮确认短促音".to_string(),
|
||||
model: None,
|
||||
duration: 5,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("sound request with unset model should serialize");
|
||||
assert!(unset_model_payload.get("model").is_none());
|
||||
@@ -1206,6 +1310,9 @@ mod tests {
|
||||
let music_payload = serde_json::to_value(EditorBackgroundMusicGenerateRequest {
|
||||
gpt_description_prompt: "森林冒险背景音乐".to_string(),
|
||||
make_instrumental: true,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
})
|
||||
.expect("background music request should serialize");
|
||||
assert_eq!(
|
||||
@@ -1229,6 +1336,8 @@ mod tests {
|
||||
task_id: "sound-task-1".to_string(),
|
||||
price_mud_points: 10,
|
||||
audio_kind: "sound-effect".to_string(),
|
||||
project: None,
|
||||
queue_state: None,
|
||||
})
|
||||
.expect("audio response should serialize");
|
||||
assert_eq!(
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,4 +1,5 @@
|
||||
import { requestJson } from '../apiClient';
|
||||
import type { ExternalGenerationJobStatusRecord } from '../../../packages/shared/src/contracts/externalGeneration';
|
||||
|
||||
const EDITOR_PROJECT_API_BASE = '/api/editor/projects';
|
||||
const EDITOR_ASSET_API_BASE = '/api/editor/assets';
|
||||
@@ -135,7 +136,7 @@ export type EditorImageGenerationInput = {
|
||||
};
|
||||
|
||||
export type EditorCanvasGenerationCompletionInput = {
|
||||
dialogId: string;
|
||||
dialogId?: string;
|
||||
title: string;
|
||||
placeholder: {
|
||||
x: number;
|
||||
@@ -156,6 +157,7 @@ export type EditorIconSpritesheetGenerationInput = {
|
||||
projectId?: string | null;
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
assetFolderId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
};
|
||||
|
||||
export type EditorUiDesignAssetExtractionInput = {
|
||||
@@ -166,6 +168,7 @@ export type EditorUiDesignAssetExtractionInput = {
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
assetFolderId?: string | null;
|
||||
spritesheetLabel?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
};
|
||||
|
||||
export type EditorImageEditInput = {
|
||||
@@ -179,6 +182,7 @@ export type EditorImageEditInput = {
|
||||
assetFolderId?: string | null;
|
||||
assetLabel?: string | null;
|
||||
sourceResourceId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
};
|
||||
|
||||
export type EditorImageGenerationResult = {
|
||||
@@ -196,6 +200,7 @@ export type EditorImageGenerationResult = {
|
||||
resource?: EditorProjectResourceSnapshot | null;
|
||||
asset?: EditorAssetSnapshot | null;
|
||||
project?: EditorProjectSnapshot | null;
|
||||
queueState?: ExternalGenerationJobStatusRecord | null;
|
||||
};
|
||||
|
||||
export type EditorIconSpritesheetIconResult = {
|
||||
@@ -220,6 +225,8 @@ export type EditorIconSpritesheetGenerationResult = {
|
||||
priceMudPoints: number;
|
||||
spritesheetResource?: EditorProjectResourceSnapshot | null;
|
||||
spritesheetAsset?: EditorAssetSnapshot | null;
|
||||
project?: EditorProjectSnapshot | null;
|
||||
queueState?: ExternalGenerationJobStatusRecord | null;
|
||||
};
|
||||
|
||||
export type EditorCharacterAnimationResolution = '480p' | '720p';
|
||||
@@ -247,6 +254,10 @@ export type EditorCharacterAnimationGenerationInput = {
|
||||
frameCount: EditorCharacterAnimationFrameCount;
|
||||
durationSeconds: EditorCharacterAnimationDurationSeconds;
|
||||
model: 'seedance2.0-fast';
|
||||
projectId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
sourceResourceId?: string | null;
|
||||
};
|
||||
|
||||
export type EditorCharacterAnimationFrameResult = {
|
||||
@@ -266,6 +277,8 @@ export type EditorCharacterAnimationGenerationResult = {
|
||||
durationSeconds: number;
|
||||
fps: number;
|
||||
priceMudPoints: number;
|
||||
project?: EditorProjectSnapshot | null;
|
||||
queueState?: ExternalGenerationJobStatusRecord | null;
|
||||
};
|
||||
|
||||
export type EditorVideoModel =
|
||||
@@ -298,6 +311,11 @@ export type EditorVideoGenerationInput = {
|
||||
referenceImageSrcs?: string[];
|
||||
referenceVideoSrcs?: string[];
|
||||
referenceAudioSrcs?: string[];
|
||||
projectId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
sourceResourceId?: string | null;
|
||||
assetKind?: string | null;
|
||||
};
|
||||
|
||||
export type EditorVideoGenerationResult = {
|
||||
@@ -313,17 +331,25 @@ export type EditorVideoGenerationResult = {
|
||||
provider: string;
|
||||
taskId: string;
|
||||
priceMudPoints?: number;
|
||||
project?: EditorProjectSnapshot | null;
|
||||
queueState?: ExternalGenerationJobStatusRecord | null;
|
||||
};
|
||||
|
||||
export type EditorSoundEffectGenerationInput = {
|
||||
prompt: string;
|
||||
model: 'audio1.0';
|
||||
duration: number;
|
||||
projectId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
};
|
||||
|
||||
export type EditorBackgroundMusicGenerationInput = {
|
||||
gptDescriptionPrompt: string;
|
||||
makeInstrumental: true;
|
||||
projectId?: string | null;
|
||||
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
|
||||
generationInputs?: EditorAssetGenerationInputs | null;
|
||||
};
|
||||
|
||||
export type EditorAudioGenerationResult = {
|
||||
@@ -341,6 +367,8 @@ export type EditorAudioGenerationResult = {
|
||||
priceMudPoints: number;
|
||||
audioKind: 'sound-effect' | 'background-music';
|
||||
durationSeconds?: number | null;
|
||||
project?: EditorProjectSnapshot | null;
|
||||
queueState?: ExternalGenerationJobStatusRecord | null;
|
||||
};
|
||||
|
||||
export type EditorGenerationPricingUnit = 'perGeneration' | 'perSecond';
|
||||
@@ -699,6 +727,9 @@ export async function generateEditorIconSpritesheet(
|
||||
? { generationInputs: input.generationInputs }
|
||||
: {}),
|
||||
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
}),
|
||||
'生成图标素材失败',
|
||||
{
|
||||
@@ -727,6 +758,9 @@ export async function extractEditorUiDesignAssets(
|
||||
: {}),
|
||||
...(input.aspectRatio ? { aspectRatio: input.aspectRatio } : {}),
|
||||
...(input.imageSize ? { imageSize: input.imageSize } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
}),
|
||||
'提取UI设计图素材失败',
|
||||
{
|
||||
@@ -756,6 +790,9 @@ export async function editEditorImage(input: EditorImageEditInput) {
|
||||
...(input.sourceResourceId
|
||||
? { sourceResourceId: input.sourceResourceId }
|
||||
: {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
}),
|
||||
'修改图片失败',
|
||||
{
|
||||
@@ -772,7 +809,19 @@ export async function generateEditorCharacterAnimation(
|
||||
) {
|
||||
return requestJson<EditorCharacterAnimationGenerationResult>(
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_API,
|
||||
jsonRequest('POST', input),
|
||||
jsonRequest('POST', {
|
||||
...input,
|
||||
...(input.projectId ? { projectId: input.projectId } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
...(input.generationInputs
|
||||
? { generationInputs: input.generationInputs }
|
||||
: {}),
|
||||
...(input.sourceResourceId
|
||||
? { sourceResourceId: input.sourceResourceId }
|
||||
: {}),
|
||||
}),
|
||||
'生成角色动画失败',
|
||||
{
|
||||
timeoutMs: 1_200_000,
|
||||
@@ -804,6 +853,17 @@ export async function generateEditorVideo(input: EditorVideoGenerationInput) {
|
||||
...(input.referenceAudioSrcs?.length
|
||||
? { referenceAudioSrcs: input.referenceAudioSrcs }
|
||||
: {}),
|
||||
...(input.projectId ? { projectId: input.projectId } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
...(input.generationInputs
|
||||
? { generationInputs: input.generationInputs }
|
||||
: {}),
|
||||
...(input.sourceResourceId
|
||||
? { sourceResourceId: input.sourceResourceId }
|
||||
: {}),
|
||||
...(input.assetKind ? { assetKind: input.assetKind } : {}),
|
||||
}),
|
||||
'生成视频失败',
|
||||
{
|
||||
@@ -824,6 +884,13 @@ export async function generateEditorSoundEffect(
|
||||
prompt: input.prompt,
|
||||
model: input.model,
|
||||
duration: input.duration,
|
||||
...(input.projectId ? { projectId: input.projectId } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
...(input.generationInputs
|
||||
? { generationInputs: input.generationInputs }
|
||||
: {}),
|
||||
}),
|
||||
'生成游戏音效失败',
|
||||
{
|
||||
@@ -843,6 +910,13 @@ export async function generateEditorBackgroundMusic(
|
||||
jsonRequest('POST', {
|
||||
gptDescriptionPrompt: input.gptDescriptionPrompt,
|
||||
makeInstrumental: input.makeInstrumental,
|
||||
...(input.projectId ? { projectId: input.projectId } : {}),
|
||||
...(input.canvasCompletion
|
||||
? { canvasCompletion: input.canvasCompletion }
|
||||
: {}),
|
||||
...(input.generationInputs
|
||||
? { generationInputs: input.generationInputs }
|
||||
: {}),
|
||||
}),
|
||||
'生成游戏背景音乐失败',
|
||||
{
|
||||
|
||||
Reference in New Issue
Block a user