画板生成统一接入Worker队列

新增图片画布外部生成队列封装和 worker job 分发

将画板图片、改图、图标、UI 素材、角色动作、视频和音频生成切到队列模式

前端提交生成任务后轮询 queueState 并应用后端项目快照

补充共享契约、测试和 worker 化文档说明
This commit is contained in:
2026-06-23 01:07:33 +08:00
parent 7d3e652b53
commit 0579741f64
15 changed files with 2103 additions and 176 deletions
@@ -2632,3 +2632,11 @@
- 2026-06-22 补充:所有明确设置绿幕用于后续抠图的 prompt 都必须固定写明 `#00FF00 / RGB(0,255,0)`,不能只写“纯绿色绿幕”或“接近 #00FF00”;编辑器角色图通用抠图额外开启暗绿 / 灰绿绿幕背景识别,只作为生成模型偏离标准亮绿时的兜底。该宽松识别只参与从画布边缘连通扩散出的背景清理,不参与全图断开绿色区域删除,避免误伤角色衣物或纹理。
- 影响范围:`server-rs/crates/api-server/src/editor_project.rs``docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`
- 验证方式:`cargo test -p api-server --manifest-path server-rs/Cargo.toml editor_character_image_general_cutout``cargo test -p platform-image --manifest-path server-rs/Cargo.toml generated_asset_sheet_muted_green_alpha_requires_explicit_option``cargo check -p api-server --manifest-path server-rs/Cargo.toml``npm run check:encoding``git diff --check`
## 2026-06-22 图片画布外部生成统一改走 worker 队列
- 背景:图片画布的图片、改图、图标素材、UI 素材提取、角色动作、视频和音频生成都可能长时间等待外部 provider;如果继续由 HTTP handler 同步执行,生产只能扩 API 进程,不能独立扩生成吞吐。
- 决策:`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画板所有外部 provider 生成入口统一入 `external_generation_job`job kind 使用 `editor_image_generation``editor_image_edit``editor_icon_spritesheet_generation``editor_ui_design_asset_extraction``editor_character_animation_generation``editor_video_generation``editor_sound_effect_generation``editor_background_music_generation`。worker 成功后由后端写 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`;前端只轮询 BFF job 状态并重新读取项目快照,不从队列 payload 或本地临时状态重建完成图层。
- 补充:带 `dialogId``canvasCompletion` 必须读取后端当前 layout 中的最新 generation dialog placeholder;等待期间用户移动占位时,结果层要跟随最新占位。无 dialog 的重绘 / UI 素材提取等入口使用明确的右侧完成占位;生成器已删除时不把结果重新塞回画布。
- 影响范围:`server-rs/crates/api-server/src/editor_generation_queue.rs``server-rs/crates/api-server/src/external_generation_worker.rs``server-rs/crates/api-server/src/editor_project.rs``server-rs/crates/api-server/src/character_animation_assets.rs``server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs``src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts``src/services/image-editor/editorProjectClient.ts`
- 验证方式:`cargo test -p api-server external_generation_worker --manifest-path server-rs/Cargo.toml``cargo test -p api-server editor_canvas_generation --manifest-path server-rs/Cargo.toml``cargo test -p shared-contracts --manifest-path server-rs/Cargo.toml``npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorProjectClient.test.ts``npm run typecheck``npm run check:encoding``git diff --check`
@@ -1,6 +1,6 @@
# 外部生成 Worker 化方案
更新时间:`2026-06-12`
更新时间:`2026-06-22`
## 背景
@@ -13,7 +13,7 @@
- 多个 worker 进程通过 SpacetimeDB 任务表抢占任务,依赖 lease 超时恢复,支持按进程数和单进程并发动态缩扩容。
- 本地或小流量同步排查可显式启用 `inline` 模式,由 HTTP handler 复用同一 worker executor 同步执行并返回 `completed`;该模式不创建队列任务,也不具备 worker 横向扩容能力。
- SpacetimeDB reducer / procedure 只做任务状态流转,不做网络、文件系统或外部 provider I/O。
- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`;本轮扩展到跳一跳、拼消消和敲木鱼的外部图片生成动作。后续玩法继续复用同一队列 Module,不再为每个玩法发明独立队列。
- 已接入拼图 `compile_puzzle_draft`、结果页 `generate_puzzle_images` 与结果页 `generate_puzzle_ui_background`跳一跳、拼消消和敲木鱼的外部图片生成动作,以及图片画布编辑器的图片、改图、图标 spritesheet、UI 素材提取、角色动作、视频、音效和背景音乐生成。后续玩法和编辑器生成入口继续复用同一队列 Module,不再为每个入口发明独立队列。
- 第一版外部生成队列粒度固定为“单个用户动作对应单个 job”。例如草稿编译、结果页单槽重生、图集重生都各自入一个 job;job 内部可以串行或并行调用 provider、OSS、SpacetimeDB 写回,但不再拆成“提示词 / 生图 / 切图 / 去背景 / 持久化 / 回写”等阶段 job。阶段进度只作为 `request_payload_json` / 业务 session 的展示状态,不作为队列调度单位。
- 不调用外部图片 / 音频 / LLM provider 的动作继续 inline 执行,不为了统一排队而进入 `external_generation_job`
@@ -48,7 +48,7 @@ BFF 只做鉴权、授权裁剪、字段脱敏和契约映射;队列事实仍
| --- | --- |
| `job_id` | 主键,`extgen-` 前缀 UUID |
| `dedupe_key` | 唯一键,建议为 `play/action/session/scope` |
| `job_kind` | 执行类型,当前拼图为 `puzzle_compile_draft``puzzle_generate_images``puzzle_generate_ui_background` |
| `job_kind` | 执行类型,当前覆盖 `puzzle_compile_draft``puzzle_generate_images``puzzle_generate_ui_background`、跳一跳 / 拼消消 / 敲木鱼生成动作,以及 `editor_image_generation``editor_image_edit``editor_icon_spritesheet_generation``editor_ui_design_asset_extraction``editor_character_animation_generation``editor_video_generation``editor_sound_effect_generation``editor_background_music_generation` |
| `owner_user_id` | 触发用户 |
| `source_module` | 玩法或能力名,例如 `puzzle` |
| `source_entity_id` | session/profile/work 等作用域 |
@@ -161,6 +161,20 @@ controller 配置:
每个玩法迁移时必须同时接入业务写回 lease guardworker 路径带 `external_generation_job_id / worker_id / lease_token`,inline 路径三项同时为空。过期 worker 不得写 session / work profile;业务失败态写回成功后才允许 job 进入 `failed`
### 图片画布编辑器
图片画布 `/editor/canvas` 下所有会调用外部生成 provider 的入口在 `queue` 模式下入 `external_generation_job`HTTP handler 只返回 `queueState`
- `editor_image_generation`:普通图片、生成规范、角色形象、UI 设计图、宣发素材和图片快速编辑。
- `editor_image_edit`:图片编辑 / 修改结果。
- `editor_icon_spritesheet_generation`:图标素材 spritesheet 生成和拆分。
- `editor_ui_design_asset_extraction`UI 设计图红框素材提取。
- `editor_character_animation_generation`:角色动作视频和帧素材生成。
- `editor_video_generation`:画布视频生成和视频素材快速编辑。
- `editor_sound_effect_generation` / `editor_background_music_generation`:画布音效与背景音乐。
画板结果的业务真相仍是 `editor_project_resource`、账号级 `editor_asset``editor_canvas.layers_json`。请求携带 `projectId + canvasCompletion` 时,worker 成功后读取当前项目 layout,用最新 generation dialog placeholder 或无 dialog 完成占位写入结果图层,并保存项目快照;前端轮询单 job 到 completed 后重新读取项目快照,不从队列 payload 或本地临时响应重建正式图层。生成器已被删除时,worker 只保留生成出的资源 / 素材记录,不把结果重新塞回画布。
## 验收
基础检查:
@@ -245,7 +245,7 @@ npm run check:server-rs-ddd
- Rust 结构体:`ExternalGenerationJob`
- 源码:`server-rs/crates/spacetime-module/src/external_generation.rs`
- 用途:外部生成 worker 的持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft``generate_puzzle_images``generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
- 用途:外部生成 worker 的持久任务队列;`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,`api-server` HTTP 角色只入队,`external-generation-worker` 角色通过 claim lease 领取、续租、执行,并用 `lease_token` 栅栏回写完成 / 失败。拼图 `compile_puzzle_draft` 的前置 `compile_puzzle_agent_draft``generate_puzzle_images``generate_puzzle_ui_background` 的业务写回也在对应 SpacetimeDB transaction 内校验 `job_id + worker_id + lease_token`、job kind、owner 和 source entity,避免过期 worker 写 session / work profile图片画布编辑器的 `editor_image_generation``editor_image_edit``editor_icon_spritesheet_generation``editor_ui_design_asset_extraction``editor_character_animation_generation``editor_video_generation``editor_sound_effect_generation``editor_background_music_generation` 复用同一队列表,worker 成功后经 `api-server` facade 写入 `editor_project_resource` / `editor_asset` / `editor_canvas.layers_json`,前端只通过 BFF job 状态轮询和项目快照读取恢复完成态。`GENARRATIVE_EXTERNAL_GENERATION_MODE=inline` 时不创建该队列行,三个 external generation guard 字段必须同时为空才允许 api-server 受控同步写回,半空 guard 仍会拒绝。worker 成功写回业务事实后才能 complete job;业务失败态写回成功后才能 fail job,失败态未写回时保留租约等待后续重领。
### `ai_text_chunk`
File diff suppressed because one or more lines are too long
+19 -4
View File
@@ -283,7 +283,10 @@ mod tests {
use tokio::net::TcpListener;
use tower::ServiceExt;
use crate::{config::AppConfig, state::AppState};
use crate::{
config::{AppConfig, ExternalGenerationMode},
state::AppState,
};
use super::{build_router, build_spacetime_unavailable_router};
@@ -1472,7 +1475,11 @@ mod tests {
#[tokio::test]
async fn editor_character_animation_accepts_character_image_body_above_default_limit() {
let state = AppState::new(AppConfig::default()).expect("state should build");
let state = AppState::new(AppConfig {
external_generation_mode: ExternalGenerationMode::Inline,
..AppConfig::default()
})
.expect("state should build");
let seed_user = seed_phone_user_with_password(&state, "13800138190", TEST_PASSWORD).await;
let token = sign_test_user_token(&state, &seed_user, "sess_editor_character_body");
let app = build_router(state);
@@ -1554,7 +1561,11 @@ mod tests {
#[tokio::test]
async fn editor_ui_design_asset_extraction_accepts_image_body_above_default_limit() {
let state = AppState::new(AppConfig::default()).expect("state should build");
let state = AppState::new(AppConfig {
external_generation_mode: ExternalGenerationMode::Inline,
..AppConfig::default()
})
.expect("state should build");
let seed_user = seed_phone_user_with_password(&state, "13800138188", TEST_PASSWORD).await;
let token = sign_test_user_token(&state, &seed_user, "sess_editor_ui_extract_body");
let app = build_router(state);
@@ -1599,7 +1610,11 @@ mod tests {
#[tokio::test]
async fn editor_image_generation_accepts_reference_body_above_default_limit() {
let state = AppState::new(AppConfig::default()).expect("state should build");
let state = AppState::new(AppConfig {
external_generation_mode: ExternalGenerationMode::Inline,
..AppConfig::default()
})
.expect("state should build");
let seed_user = seed_phone_user_with_password(&state, "13800138028", TEST_PASSWORD).await;
let token = sign_test_user_token(&state, &seed_user, "sess_editor_image_reference_body");
let app = build_router(state);
@@ -1,4 +1,4 @@
use std::{
use std::{
collections::BTreeMap,
fs,
path::{Path, PathBuf},
@@ -55,6 +55,15 @@ use crate::{
build_character_animation_prompt, build_fallback_moderation_safe_animation_prompt,
},
editor_generation_config::EditorGenerationPricingConfig,
editor_generation_queue::{
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
EditorGenerationQueuedResponse, editor_generation_queue_state,
editor_generation_source_entity_id, enqueue_editor_generation_job,
},
editor_project::{
EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item,
complete_editor_canvas_generation_with_items,
},
http_error::AppError,
platform_errors::map_oss_error,
prompt::role_asset_studio::{
@@ -525,11 +534,44 @@ pub async fn generate_editor_character_animation(
Extension(authenticated): Extension<AuthenticatedAccessToken>,
payload: Result<Json<EditorCharacterAnimationGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
let Json(payload) = payload.map_err(|error| {
character_animation_error_response(
&request_context,
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "editor-character-animation",
"message": error.body_text(),
})),
)
})?;
let owner_user_id = authenticated.claims().user_id().to_string();
if !state.config.external_generation_mode.is_inline() {
let source_entity_id = editor_generation_source_entity_id(
payload.project_id.as_deref(),
payload.source_layer_id.as_str(),
);
let queue_job = enqueue_editor_generation_job(
&state,
&request_context,
owner_user_id.as_str(),
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成角色动作",
&payload,
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
return Ok(json_success_body(
Some(&request_context),
EditorGenerationQueuedResponse {
queue_state: editor_generation_queue_state(queue_job),
},
));
}
generate_editor_character_animation_for_owner(
state,
request_context,
authenticated.claims().user_id().to_string(),
payload,
owner_user_id,
Ok(Json(payload)),
)
.await
}
@@ -549,6 +591,10 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
})),
)
})?;
let project_id = payload.project_id.clone();
let canvas_completion = payload.canvas_completion.clone();
let generation_inputs = payload.generation_inputs.clone();
let source_resource_id = payload.source_resource_id.clone();
let pricing = state.editor_generation_pricing().map_err(|error| {
character_animation_error_response(
@@ -613,6 +659,68 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
let (generated, frames) =
result.map_err(|error| character_animation_error_response(&request_context, error))?;
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
let frame_payload_json = serde_json::to_value(&frames).map_err(|error| {
character_animation_error_response(
&request_context,
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": "editor-character-animation",
"message": format!("角色动作序列帧无法写入画布:{error}"),
})),
)
})?;
let preview_frame = frames.first();
let layer_id = format!("layer-editor-character-animation-{task_id}");
let item = build_editor_canvas_generated_layer_item(
completion,
EditorCanvasGeneratedLayerInput {
layer_id: layer_id.clone(),
resource_id: format!("local-resource-editor-character-animation-{task_id}"),
title: completion.title.trim().to_string(),
src: preview_frame
.map(|frame| frame.image_src.clone())
.unwrap_or_else(|| generated.preview_video_path.clone()),
media_type: Some("image-sequence".to_string()),
thumbnail_src: preview_frame.map(|frame| frame.image_src.clone()),
original_width: preview_frame
.map(|frame| frame.width)
.unwrap_or(normalized.frame_width),
original_height: preview_frame
.map(|frame| frame.height)
.unwrap_or(normalized.frame_height),
display_width: completion.placeholder.width,
display_height: completion.placeholder.height,
source_type: "generated".to_string(),
prompt: Some(generated.submitted_prompt.clone()),
actual_prompt: Some(generated.submitted_prompt.clone()),
model: Some(EDITOR_CHARACTER_ANIMATION_MODEL.to_string()),
provider: Some("ark".to_string()),
task_id: Some(task_id.clone()),
object_key: None,
asset_object_id: None,
duration_seconds: Some(normalized.duration_seconds),
source_resource_id,
asset_kind: Some("character-animation".to_string()),
generation_inputs,
image_sequence_frames: Some(frame_payload_json),
preview_video_path: Some(generated.preview_video_path.clone()),
},
);
complete_editor_canvas_generation_with_items(
&state,
owner_user_id.as_str(),
project_id.as_deref(),
canvas_completion.as_ref(),
vec![item],
Some(layer_id),
)
.await
.map_err(|error| character_animation_error_response(&request_context, error))?
} else {
None
};
let completed_project =
completed_project.and_then(|project| serde_json::to_value(project).ok());
Ok(json_success_body(
Some(&request_context),
@@ -629,6 +737,8 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
fps: normalized.fps,
price_mud_points: normalized.price_mud_points,
frames,
project: completed_project,
queue_state: None,
},
))
}
@@ -639,13 +749,38 @@ pub async fn generate_editor_video(
Extension(authenticated): Extension<AuthenticatedAccessToken>,
payload: Result<Json<EditorVideoGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
generate_editor_video_for_owner(
state,
request_context,
authenticated.claims().user_id().to_string(),
payload,
)
.await
let Json(payload) = payload.map_err(|error| {
editor_video_error_response(
&request_context,
AppError::from_status(StatusCode::BAD_REQUEST).with_details(json!({
"provider": "editor-video",
"message": error.body_text(),
})),
)
})?;
let owner_user_id = authenticated.claims().user_id().to_string();
if !state.config.external_generation_mode.is_inline() {
let source_entity_id =
editor_generation_source_entity_id(payload.project_id.as_deref(), "editor-video");
let queue_job = enqueue_editor_generation_job(
&state,
&request_context,
owner_user_id.as_str(),
EDITOR_VIDEO_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成视频",
&payload,
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
return Ok(json_success_body(
Some(&request_context),
EditorGenerationQueuedResponse {
queue_state: editor_generation_queue_state(queue_job),
},
));
}
generate_editor_video_for_owner(state, request_context, owner_user_id, Ok(Json(payload))).await
}
pub(crate) async fn generate_editor_video_for_owner(
@@ -663,6 +798,11 @@ pub(crate) async fn generate_editor_video_for_owner(
})),
)
})?;
let project_id = payload.project_id.clone();
let canvas_completion = payload.canvas_completion.clone();
let generation_inputs = payload.generation_inputs.clone();
let source_resource_id = payload.source_resource_id.clone();
let asset_kind = payload.asset_kind.clone();
let pricing = state.editor_generation_pricing().map_err(|error| {
editor_video_error_response(
@@ -698,6 +838,52 @@ pub(crate) async fn generate_editor_video_for_owner(
)
.await
.map_err(|error| editor_video_error_response(&request_context, error))?;
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
let layer_id = format!("layer-editor-video-{task_id}");
let item = build_editor_canvas_generated_layer_item(
completion,
EditorCanvasGeneratedLayerInput {
layer_id: layer_id.clone(),
resource_id: format!("local-resource-editor-video-{task_id}"),
title: completion.title.trim().to_string(),
src: generated.preview_video_path.clone(),
media_type: Some("video".to_string()),
thumbnail_src: None,
original_width: normalized.width,
original_height: normalized.height,
display_width: completion.placeholder.width,
display_height: completion.placeholder.height,
source_type: "generated".to_string(),
prompt: Some(normalized.prompt.clone()),
actual_prompt: Some(generated.submitted_prompt.clone()),
model: Some(normalized.model.clone()),
provider: Some("VectorEngine".to_string()),
task_id: Some(task_id.clone()),
object_key: generated.object_key.clone(),
asset_object_id: generated.asset_object_id.clone(),
duration_seconds: Some(normalized.duration_seconds),
source_resource_id,
asset_kind: Some(asset_kind.unwrap_or_else(|| "video".to_string())),
generation_inputs,
image_sequence_frames: None,
preview_video_path: None,
},
);
complete_editor_canvas_generation_with_items(
&state,
owner_user_id.as_str(),
project_id.as_deref(),
canvas_completion.as_ref(),
vec![item],
Some(layer_id),
)
.await
.map_err(|error| editor_video_error_response(&request_context, error))?
} else {
None
};
let completed_project =
completed_project.and_then(|project| serde_json::to_value(project).ok());
Ok(json_success_body(
Some(&request_context),
@@ -717,6 +903,8 @@ pub(crate) async fn generate_editor_video_for_owner(
duration_seconds: normalized.duration_seconds,
resolution: normalized.resolution,
price_mud_points: normalized.price_mud_points,
project: completed_project,
queue_state: None,
},
))
}
@@ -4909,6 +5097,10 @@ mod tests {
frame_count: 48,
duration_seconds: 6,
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
})
.expect("editor request should normalize");
@@ -4943,6 +5135,10 @@ mod tests {
frame_count: 32,
duration_seconds: 4,
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
})
.expect("editor request should normalize");
@@ -4966,6 +5162,10 @@ mod tests {
frame_count: 48,
duration_seconds: 4,
model: EDITOR_CHARACTER_ANIMATION_MODEL.to_string(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
})
.expect_err("invalid frame/duration pair should fail");
@@ -5121,6 +5321,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("editor video request should normalize");
@@ -5146,6 +5351,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance fast request should normalize");
@@ -5168,6 +5378,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance standard request should normalize");
@@ -5190,6 +5405,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance standard 1080p request should normalize");
@@ -5213,6 +5433,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect_err("seedance fast should reject 1080p");
@@ -5235,6 +5460,11 @@ mod tests {
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
],
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance multimodal request should normalize");
@@ -5266,6 +5496,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: vec!["data:video/mp4;base64,video".to_string()],
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect_err("Seedance reference videos must not accept Base64 data URLs");
@@ -5287,6 +5522,11 @@ mod tests {
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
reference_video_srcs: vec!["https://assets.example.com/reference.mp4".to_string()],
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance multimodal request should normalize");
@@ -5328,6 +5568,11 @@ mod tests {
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
],
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("seedance objectKey request should normalize");
@@ -5363,6 +5608,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: vec!["data:audio/wav;base64,audio".to_string()],
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect_err("audio-only references should fail");
@@ -5404,7 +5654,6 @@ mod tests {
);
}
fn assert_function_contains(source: &str, start: &str, end: &str, snippets: &[&str]) {
let start_index = source
.find(start)
@@ -0,0 +1,105 @@
use axum::http::StatusCode;
use serde::Serialize;
use serde_json::json;
use shared_contracts::external_generation::{
ExternalGenerationJobStatus, ExternalGenerationJobStatusRecord,
};
use shared_kernel::{build_prefixed_uuid_id, offset_datetime_to_unix_micros};
use spacetime_client::{ExternalGenerationJobEnqueueRecordInput, ExternalGenerationJobRecord};
use crate::{http_error::AppError, request_context::RequestContext, state::AppState};
pub(crate) const EDITOR_IMAGE_GENERATION_JOB_KIND: &str = "editor_image_generation";
pub(crate) const EDITOR_IMAGE_EDIT_JOB_KIND: &str = "editor_image_edit";
pub(crate) const EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND: &str =
"editor_icon_spritesheet_generation";
pub(crate) const EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND: &str =
"editor_ui_design_asset_extraction";
pub(crate) const EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND: &str =
"editor_character_animation_generation";
pub(crate) const EDITOR_VIDEO_GENERATION_JOB_KIND: &str = "editor_video_generation";
pub(crate) const EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND: &str = "editor_sound_effect_generation";
pub(crate) const EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND: &str =
"editor_background_music_generation";
pub(crate) const EDITOR_GENERATION_QUEUE_SOURCE_MODULE: &str = "editor-canvas";
const EDITOR_GENERATION_QUEUE_PROVIDER: &str = "editor-generation-worker";
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub(crate) struct EditorGenerationQueuedResponse {
pub(crate) queue_state: ExternalGenerationJobStatusRecord,
}
pub(crate) async fn enqueue_editor_generation_job<T>(
state: &AppState,
_request_context: &RequestContext,
owner_user_id: &str,
job_kind: &str,
source_entity_id: impl Into<String>,
request_label: impl Into<String>,
payload: &T,
) -> Result<ExternalGenerationJobRecord, AppError>
where
T: Serialize,
{
let job_id = build_prefixed_uuid_id("extgen-");
let request_payload_json = serde_json::to_string(payload).map_err(|error| {
AppError::from_status(StatusCode::INTERNAL_SERVER_ERROR).with_details(json!({
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
"message": format!("编辑器 worker 任务参数序列化失败:{error}"),
}))
})?;
let now_micros = current_utc_micros();
state
.spacetime_client()
.enqueue_external_generation_job(ExternalGenerationJobEnqueueRecordInput {
dedupe_key: format!("editor-canvas:{job_kind}:{job_id}"),
job_id,
job_kind: job_kind.to_string(),
owner_user_id: owner_user_id.to_string(),
source_module: EDITOR_GENERATION_QUEUE_SOURCE_MODULE.to_string(),
source_entity_id: source_entity_id.into(),
request_label: request_label.into(),
request_payload_json,
max_attempts: 1,
available_at_micros: now_micros,
created_at_micros: now_micros,
})
.await
.map_err(|error| {
AppError::from_status(StatusCode::BAD_GATEWAY).with_details(json!({
"provider": EDITOR_GENERATION_QUEUE_PROVIDER,
"message": error.to_string(),
}))
})
}
pub(crate) fn editor_generation_queue_state(
job: ExternalGenerationJobRecord,
) -> ExternalGenerationJobStatusRecord {
ExternalGenerationJobStatusRecord {
operation_id: job.job_id,
status: ExternalGenerationJobStatus::Queued,
phase_label: job.request_label,
phase_detail: "排队中。".to_string(),
progress: 8,
error: job.last_error_message,
updated_at_micros: job.updated_at_micros,
}
}
pub(crate) fn editor_generation_source_entity_id(
project_id: Option<&str>,
fallback: &str,
) -> String {
project_id
.map(str::trim)
.filter(|value| !value.is_empty())
.unwrap_or(fallback)
.to_string()
}
fn current_utc_micros() -> i64 {
offset_datetime_to_unix_micros(time::OffsetDateTime::now_utc())
}
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,6 @@
use std::{future::Future, io, pin::Pin, time::Duration};
use axum::extract::FromRef;
use axum::{Json, extract::FromRef};
use serde_json::json;
use shared_kernel::offset_datetime_to_unix_micros;
use spacetime_client::{
@@ -15,6 +15,22 @@ use tokio::{
use tracing::{error, info, warn};
use crate::{
character_animation_assets::{
generate_editor_character_animation_for_owner, generate_editor_video_for_owner,
},
editor_generation_queue::{
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND,
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND, EDITOR_IMAGE_EDIT_JOB_KIND,
EDITOR_IMAGE_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND, EDITOR_VIDEO_GENERATION_JOB_KIND,
},
editor_project::{
EditorGenerationCaller, EditorIconSpritesheetGenerationRequest, EditorImageEditRequest,
EditorImageGenerationRequest, EditorUiDesignAssetExtractionRequest,
edit_editor_image_for_owner, extract_editor_ui_design_assets_for_owner,
generate_editor_icon_spritesheet_for_owner, generate_editor_image_for_owner,
},
jump_hop::{
JUMP_HOP_COMPILE_DRAFT_JOB_KIND, JumpHopCompileDraftWorkerPayload,
execute_jump_hop_compile_draft_worker_job,
@@ -31,6 +47,9 @@ use crate::{
},
request_context::RequestContext,
state::{AppState, PuzzleApiState},
vector_engine_audio_generation::{
generate_editor_background_music_for_owner, generate_editor_sound_effect_for_owner,
},
wooden_fish::{
WOODEN_FISH_GENERATE_IMAGE_ASSETS_JOB_KIND, WoodenFishGenerateImageAssetsWorkerPayload,
execute_wooden_fish_generate_image_assets_worker_job,
@@ -536,6 +555,234 @@ async fn process_external_generation_job_once(
}
}
}
EDITOR_IMAGE_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<EditorImageGenerationRequest>(
job.request_payload_json.as_str(),
) {
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布生图任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_image_for_owner(
&state,
&request_context,
editor_generation_worker_caller(&job),
payload,
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(error) => {
let message = error.body_text();
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_IMAGE_EDIT_JOB_KIND => {
let payload = match serde_json::from_str::<EditorImageEditRequest>(
job.request_payload_json.as_str(),
) {
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布改图任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match edit_editor_image_for_owner(
&state,
&request_context,
editor_generation_worker_caller(&job),
payload,
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(error) => {
let message = error.body_text();
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_ICON_SPRITESHEET_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<EditorIconSpritesheetGenerationRequest>(
job.request_payload_json.as_str(),
) {
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布图标素材任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_icon_spritesheet_for_owner(
&state,
&request_context,
editor_generation_worker_caller(&job),
payload,
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(error) => {
let message = error.body_text();
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_UI_DESIGN_ASSET_EXTRACTION_JOB_KIND => {
let payload = match serde_json::from_str::<EditorUiDesignAssetExtractionRequest>(
job.request_payload_json.as_str(),
) {
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布 UI 素材提取任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match extract_editor_ui_design_assets_for_owner(
&state,
&request_context,
editor_generation_worker_caller(&job),
payload,
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(error) => {
let message = error.body_text();
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_CHARACTER_ANIMATION_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<
shared_contracts::assets::EditorCharacterAnimationGenerateRequest,
>(job.request_payload_json.as_str())
{
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布角色动作任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_character_animation_for_owner(
state.clone(),
request_context,
job.owner_user_id.clone(),
Ok(Json(payload)),
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(response) => {
let message = response_error_message(response).await;
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_VIDEO_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<
shared_contracts::assets::EditorVideoGenerateRequest,
>(job.request_payload_json.as_str())
{
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布视频生成任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_video_for_owner(
state.clone(),
request_context,
job.owner_user_id.clone(),
Ok(Json(payload)),
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(response) => {
let message = response_error_message(response).await;
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<
shared_contracts::assets::EditorSoundEffectGenerateRequest,
>(job.request_payload_json.as_str())
{
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布音效生成任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_sound_effect_for_owner(
state.clone(),
request_context,
job.owner_user_id.clone(),
Ok(Json(payload)),
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(response) => {
let message = response_error_message(response).await;
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND => {
let payload = match serde_json::from_str::<
shared_contracts::assets::EditorBackgroundMusicGenerateRequest,
>(job.request_payload_json.as_str())
{
Ok(payload) => payload,
Err(error) => {
let message = format!("图片画布背景音乐生成任务参数解析失败:{error}");
fail_job(&state, &worker_id, &job, message.clone()).await?;
return Err(message);
}
};
let request_context = worker_request_context(&job);
match generate_editor_background_music_for_owner(
state.clone(),
request_context,
job.owner_user_id.clone(),
Ok(Json(payload)),
)
.await
{
Ok(_) => complete_editor_generation_job(&state, &worker_id, &job).await,
Err(response) => {
let message = response_error_message(response).await;
fail_job(&state, &worker_id, &job, message.clone()).await?;
Err(message)
}
}
}
unknown => {
warn!(
job_id = job.job_id,
@@ -598,6 +845,43 @@ async fn fail_queue_job_after_worker_error(
Ok(())
}
fn worker_request_context(job: &ExternalGenerationJobRecord) -> RequestContext {
RequestContext::new(
format!("external-generation-worker-{}", job.job_id),
format!("external-generation-worker {}", job.job_kind),
std::time::Duration::ZERO,
false,
)
}
fn editor_generation_worker_caller(job: &ExternalGenerationJobRecord) -> EditorGenerationCaller {
EditorGenerationCaller {
owner_user_id: job.owner_user_id.clone(),
audit_subject_user_id: Some(job.owner_user_id.clone()),
audit_project_id: Some(job.source_entity_id.clone()),
}
}
async fn complete_editor_generation_job(
state: &AppState,
worker_id: &str,
job: &ExternalGenerationJobRecord,
) -> Result<(), String> {
complete_job(
state,
worker_id,
job,
Some(
json!({
"sourceModule": job.source_module.clone(),
"sourceEntityId": job.source_entity_id.clone(),
})
.to_string(),
),
)
.await
}
async fn complete_job(
state: &AppState,
worker_id: &str,
+1
View File
@@ -37,6 +37,7 @@ mod custom_world_foundation_draft;
mod custom_world_result_prompts;
mod custom_world_rpg_draft_prompts;
mod editor_generation_config;
mod editor_generation_queue;
mod editor_project;
mod edutainment_baby_drawing;
mod edutainment_baby_object;
@@ -11,9 +11,21 @@ use serde_json::json;
use shared_contracts::{assets, creation_audio};
use crate::{
api_response::json_success_body, auth::AuthenticatedAccessToken,
editor_generation_config::EditorGenerationPricingConfig, http_error::AppError,
request_context::RequestContext, state::AppState,
api_response::json_success_body,
auth::AuthenticatedAccessToken,
editor_generation_config::EditorGenerationPricingConfig,
editor_generation_queue::{
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND, EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
EditorGenerationQueuedResponse, editor_generation_queue_state,
editor_generation_source_entity_id, enqueue_editor_generation_job,
},
editor_project::{
EditorCanvasGeneratedLayerInput, build_editor_canvas_generated_layer_item,
complete_editor_canvas_generation_with_items,
},
http_error::AppError,
request_context::RequestContext,
state::AppState,
tracking::record_external_generation_run_after_success,
};
@@ -128,13 +140,33 @@ pub async fn generate_editor_sound_effect(
Extension(authenticated): Extension<AuthenticatedAccessToken>,
payload: Result<Json<assets::EditorSoundEffectGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
generate_editor_sound_effect_for_owner(
state,
request_context,
authenticated.claims().user_id().to_string(),
payload,
)
.await
let Json(payload) = parse_json_payload(&request_context, payload)?;
let owner_user_id = authenticated.claims().user_id().to_string();
if !state.config.external_generation_mode.is_inline() {
let source_entity_id = editor_generation_source_entity_id(
payload.project_id.as_deref(),
"editor-sound-effect",
);
let queue_job = enqueue_editor_generation_job(
&state,
&request_context,
owner_user_id.as_str(),
EDITOR_SOUND_EFFECT_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成音效",
&payload,
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
return Ok(json_success_body(
Some(&request_context),
EditorGenerationQueuedResponse {
queue_state: editor_generation_queue_state(queue_job),
},
));
}
generate_editor_sound_effect_for_owner(state, request_context, owner_user_id, Ok(Json(payload)))
.await
}
pub(crate) async fn generate_editor_sound_effect_for_owner(
@@ -144,6 +176,9 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
payload: Result<Json<assets::EditorSoundEffectGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
let Json(payload) = parse_json_payload(&request_context, payload)?;
let project_id = payload.project_id.clone();
let canvas_completion = payload.canvas_completion.clone();
let generation_inputs = payload.generation_inputs.clone();
let pricing = state
.editor_generation_pricing()
.map_err(|error| {
@@ -192,6 +227,52 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
.audio_src
.ok_or_else(|| vector_engine_bad_gateway("音效生成完成但缺少播放地址"))
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
let layer_id = format!("layer-editor-sound-effect-{}", generated.task_id);
let item = build_editor_canvas_generated_layer_item(
completion,
EditorCanvasGeneratedLayerInput {
layer_id: layer_id.clone(),
resource_id: format!("local-resource-editor-sound-effect-{}", generated.task_id),
title: completion.title.trim().to_string(),
src: audio_src.clone(),
media_type: Some("audio".to_string()),
thumbnail_src: None,
original_width: EDITOR_AUDIO_WIDTH,
original_height: EDITOR_AUDIO_HEIGHT,
display_width: completion.placeholder.width,
display_height: completion.placeholder.height,
source_type: "generated".to_string(),
prompt: Some(normalized.prompt.clone()),
actual_prompt: Some(normalized.prompt.clone()),
model: Some(normalized.model.clone()),
provider: Some(generated.provider.clone()),
task_id: Some(generated.task_id.clone()),
object_key: generated.object_key.clone(),
asset_object_id: generated.asset_object_id.clone(),
duration_seconds: Some(u32::from(normalized.duration)),
source_resource_id: None,
asset_kind: Some("sound-effect".to_string()),
generation_inputs,
image_sequence_frames: None,
preview_video_path: None,
},
);
complete_editor_canvas_generation_with_items(
&state,
owner_user_id.as_str(),
project_id.as_deref(),
canvas_completion.as_ref(),
vec![item],
Some(layer_id),
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?
} else {
None
};
let completed_project =
completed_project.and_then(|project| serde_json::to_value(project).ok());
Ok(json_success_body(
Some(&request_context),
@@ -210,6 +291,8 @@ pub(crate) async fn generate_editor_sound_effect_for_owner(
task_id: generated.task_id,
price_mud_points: normalized.price_mud_points,
audio_kind: "sound-effect".to_string(),
project: completed_project,
queue_state: None,
},
))
}
@@ -220,11 +303,36 @@ pub async fn generate_editor_background_music(
Extension(authenticated): Extension<AuthenticatedAccessToken>,
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
let Json(payload) = parse_json_payload(&request_context, payload)?;
let owner_user_id = authenticated.claims().user_id().to_string();
if !state.config.external_generation_mode.is_inline() {
let source_entity_id = editor_generation_source_entity_id(
payload.project_id.as_deref(),
"editor-background-music",
);
let queue_job = enqueue_editor_generation_job(
&state,
&request_context,
owner_user_id.as_str(),
EDITOR_BACKGROUND_MUSIC_GENERATION_JOB_KIND,
source_entity_id,
"图片画布生成背景音乐",
&payload,
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
return Ok(json_success_body(
Some(&request_context),
EditorGenerationQueuedResponse {
queue_state: editor_generation_queue_state(queue_job),
},
));
}
generate_editor_background_music_for_owner(
state,
request_context,
authenticated.claims().user_id().to_string(),
payload,
owner_user_id,
Ok(Json(payload)),
)
.await
}
@@ -236,6 +344,9 @@ pub(crate) async fn generate_editor_background_music_for_owner(
payload: Result<Json<assets::EditorBackgroundMusicGenerateRequest>, JsonRejection>,
) -> Result<Json<Value>, Response> {
let Json(payload) = parse_json_payload(&request_context, payload)?;
let project_id = payload.project_id.clone();
let canvas_completion = payload.canvas_completion.clone();
let generation_inputs = payload.generation_inputs.clone();
let pricing = state
.editor_generation_pricing()
.map_err(|error| {
@@ -282,6 +393,55 @@ pub(crate) async fn generate_editor_background_music_for_owner(
.audio_src
.ok_or_else(|| vector_engine_bad_gateway("背景音乐生成完成但缺少播放地址"))
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
let completed_project = if let Some(completion) = canvas_completion.as_ref() {
let layer_id = format!("layer-editor-background-music-{}", generated.task_id);
let item = build_editor_canvas_generated_layer_item(
completion,
EditorCanvasGeneratedLayerInput {
layer_id: layer_id.clone(),
resource_id: format!(
"local-resource-editor-background-music-{}",
generated.task_id
),
title: completion.title.trim().to_string(),
src: audio_src.clone(),
media_type: Some("audio".to_string()),
thumbnail_src: None,
original_width: EDITOR_AUDIO_WIDTH,
original_height: EDITOR_AUDIO_HEIGHT,
display_width: completion.placeholder.width,
display_height: completion.placeholder.height,
source_type: "generated".to_string(),
prompt: Some(normalized.gpt_description_prompt.clone()),
actual_prompt: Some(normalized.gpt_description_prompt.clone()),
model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()),
provider: Some(generated.provider.clone()),
task_id: Some(generated.task_id.clone()),
object_key: generated.object_key.clone(),
asset_object_id: generated.asset_object_id.clone(),
duration_seconds: None,
source_resource_id: None,
asset_kind: Some("background-music".to_string()),
generation_inputs,
image_sequence_frames: None,
preview_video_path: None,
},
);
complete_editor_canvas_generation_with_items(
&state,
owner_user_id.as_str(),
project_id.as_deref(),
canvas_completion.as_ref(),
vec![item],
Some(layer_id),
)
.await
.map_err(|error| error.into_response_with_context(Some(&request_context)))?
} else {
None
};
let completed_project =
completed_project.and_then(|project| serde_json::to_value(project).ok());
Ok(json_success_body(
Some(&request_context),
@@ -300,6 +460,8 @@ pub(crate) async fn generate_editor_background_music_for_owner(
task_id: generated.task_id,
price_mud_points: normalized.price_mud_points,
audio_kind: "background-music".to_string(),
project: completed_project,
queue_state: None,
},
))
}
@@ -109,6 +109,9 @@ fn editor_sound_effect_request_normalizes_prompt_duration_and_resolves_price() {
prompt: " 金币掉落叮当声 ".to_string(),
model: None,
duration: 7,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("editor sound effect request should normalize");
@@ -125,6 +128,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() {
prompt: "金币掉落叮当声".to_string(),
model: Some(" audio1.0 ".to_string()),
duration: 5,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("Vidu audio model should be accepted");
@@ -134,6 +140,9 @@ fn editor_sound_effect_request_accepts_only_vidu_audio_model() {
prompt: "金币掉落叮当声".to_string(),
model: Some(platform_audio::SUNO_DEFAULT_MODEL.to_string()),
duration: 5,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect_err("Suno text-to-sound should be disabled for editor sound effects");
@@ -148,6 +157,9 @@ fn editor_sound_effect_request_rejects_duration_outside_vidu_range() {
prompt: "按钮点击".to_string(),
model: None,
duration,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect_err("duration outside 2-10 seconds should fail");
@@ -183,6 +195,9 @@ fn editor_background_music_request_forces_instrumental_and_resolves_price() {
normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: " 森林冒险背景音乐 ".to_string(),
make_instrumental: false,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("editor background music request should normalize");
@@ -197,6 +212,9 @@ fn editor_background_music_request_rejects_prompt_over_documented_limit() {
normalize_editor_background_music_request(assets::EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: "".repeat(201),
make_instrumental: true,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect_err("Suno gpt_description_prompt should follow Apifox 200 char limit");
+117 -8
View File
@@ -3,6 +3,8 @@ use std::collections::BTreeMap;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use crate::external_generation::ExternalGenerationJobStatusRecord;
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct CreateDirectUploadTicketRequest {
@@ -304,7 +306,7 @@ pub struct CharacterAnimationGenerateResponse {
pub preview_video_path: Option<String>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorCharacterAnimationGenerateRequest {
pub source_layer_id: String,
@@ -317,6 +319,14 @@ pub struct EditorCharacterAnimationGenerateRequest {
pub frame_count: u32,
pub duration_seconds: u32,
pub model: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generation_inputs: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_resource_id: Option<String>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
@@ -328,7 +338,7 @@ pub struct EditorCharacterAnimationFramePayload {
pub height: u32,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorCharacterAnimationGenerateResponse {
pub ok: bool,
@@ -343,9 +353,13 @@ pub struct EditorCharacterAnimationGenerateResponse {
pub frame_height: u32,
pub fps: u32,
pub price_mud_points: u32,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorVideoGenerateRequest {
pub prompt: String,
@@ -363,13 +377,23 @@ pub struct EditorVideoGenerateRequest {
pub reference_video_srcs: Vec<String>,
#[serde(default)]
pub reference_audio_srcs: Vec<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generation_inputs: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub source_resource_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub asset_kind: Option<String>,
}
fn default_editor_video_web_search_enabled() -> bool {
true
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorVideoGenerateResponse {
pub ok: bool,
@@ -390,6 +414,30 @@ pub struct EditorVideoGenerateResponse {
pub duration_seconds: u32,
pub resolution: String,
pub price_mud_points: u32,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorCanvasGenerationPlaceholderPayload {
pub x: f64,
pub y: f64,
pub width: f64,
pub height: f64,
pub original_width: f64,
pub original_height: f64,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorCanvasGenerationCompletionPayload {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub dialog_id: Option<String>,
pub title: String,
pub placeholder: EditorCanvasGenerationPlaceholderPayload,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
@@ -430,23 +478,35 @@ pub struct EditorIconSpritesheetGenerateResponse {
pub price_mud_points: u32,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorSoundEffectGenerateRequest {
pub prompt: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model: Option<String>,
pub duration: u8,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generation_inputs: Option<Value>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorBackgroundMusicGenerateRequest {
pub gpt_description_prompt: String,
pub make_instrumental: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project_id: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub canvas_completion: Option<EditorCanvasGenerationCompletionPayload>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub generation_inputs: Option<Value>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct EditorAudioGenerateResponse {
pub ok: bool,
@@ -466,6 +526,10 @@ pub struct EditorAudioGenerateResponse {
pub task_id: String,
pub price_mud_points: u32,
pub audio_kind: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub project: Option<Value>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub queue_state: Option<ExternalGenerationJobStatusRecord>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
@@ -992,6 +1056,21 @@ mod tests {
frame_count: 48,
duration_seconds: 6,
model: "seedance2.0-fast".to_string(),
project_id: Some("editor-project-1".to_string()),
canvas_completion: Some(EditorCanvasGenerationCompletionPayload {
dialog_id: Some("dialog-1".to_string()),
title: "角色动作".to_string(),
placeholder: EditorCanvasGenerationPlaceholderPayload {
x: 10.0,
y: 20.0,
width: 420.0,
height: 420.0,
original_width: 1024.0,
original_height: 1024.0,
},
}),
generation_inputs: None,
source_resource_id: Some("resource-source".to_string()),
})
.expect("request should serialize");
@@ -1002,6 +1081,9 @@ mod tests {
);
assert!(payload.get("priceMudPoints").is_none());
assert_eq!(payload["model"], json!("seedance2.0-fast"));
assert_eq!(payload["projectId"], json!("editor-project-1"));
assert_eq!(payload["canvasCompletion"]["dialogId"], json!("dialog-1"));
assert_eq!(payload["sourceResourceId"], json!("resource-source"));
}
#[test]
@@ -1024,6 +1106,8 @@ mod tests {
frame_height: 1024,
fps: 8,
price_mud_points: 40,
project: None,
queue_state: None,
})
.expect("response should serialize");
@@ -1052,6 +1136,11 @@ mod tests {
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("request should serialize");
@@ -1078,6 +1167,11 @@ mod tests {
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
],
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
project_id: None,
canvas_completion: None,
generation_inputs: None,
source_resource_id: None,
asset_kind: None,
})
.expect("request should serialize");
@@ -1100,7 +1194,9 @@ mod tests {
let payload = serde_json::to_value(EditorVideoGenerateResponse {
ok: true,
video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(),
object_key: Some("generated-character-drafts/editor-videos/task-1/preview.mp4".to_string()),
object_key: Some(
"generated-character-drafts/editor-videos/task-1/preview.mp4".to_string(),
),
asset_object_id: Some("assetobj_video_1".to_string()),
width: 1280,
height: 720,
@@ -1113,6 +1209,8 @@ mod tests {
duration_seconds: 5,
resolution: "480p".to_string(),
price_mud_points: 100,
project: None,
queue_state: None,
})
.expect("response should serialize");
@@ -1178,6 +1276,9 @@ mod tests {
prompt: "金币掉落叮当声".to_string(),
model: Some("audio1.0".to_string()),
duration: 7,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("sound request should serialize");
assert_eq!(sound_payload["prompt"], json!("金币掉落叮当声"));
@@ -1198,6 +1299,9 @@ mod tests {
prompt: "按钮确认短促音".to_string(),
model: None,
duration: 5,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("sound request with unset model should serialize");
assert!(unset_model_payload.get("model").is_none());
@@ -1206,6 +1310,9 @@ mod tests {
let music_payload = serde_json::to_value(EditorBackgroundMusicGenerateRequest {
gpt_description_prompt: "森林冒险背景音乐".to_string(),
make_instrumental: true,
project_id: None,
canvas_completion: None,
generation_inputs: None,
})
.expect("background music request should serialize");
assert_eq!(
@@ -1229,6 +1336,8 @@ mod tests {
task_id: "sound-task-1".to_string(),
price_mud_points: 10,
audio_kind: "sound-effect".to_string(),
project: None,
queue_state: None,
})
.expect("audio response should serialize");
assert_eq!(
File diff suppressed because it is too large Load Diff
@@ -1,4 +1,5 @@
import { requestJson } from '../apiClient';
import type { ExternalGenerationJobStatusRecord } from '../../../packages/shared/src/contracts/externalGeneration';
const EDITOR_PROJECT_API_BASE = '/api/editor/projects';
const EDITOR_ASSET_API_BASE = '/api/editor/assets';
@@ -135,7 +136,7 @@ export type EditorImageGenerationInput = {
};
export type EditorCanvasGenerationCompletionInput = {
dialogId: string;
dialogId?: string;
title: string;
placeholder: {
x: number;
@@ -156,6 +157,7 @@ export type EditorIconSpritesheetGenerationInput = {
projectId?: string | null;
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
};
export type EditorUiDesignAssetExtractionInput = {
@@ -166,6 +168,7 @@ export type EditorUiDesignAssetExtractionInput = {
generationInputs?: EditorAssetGenerationInputs | null;
assetFolderId?: string | null;
spritesheetLabel?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
};
export type EditorImageEditInput = {
@@ -179,6 +182,7 @@ export type EditorImageEditInput = {
assetFolderId?: string | null;
assetLabel?: string | null;
sourceResourceId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
};
export type EditorImageGenerationResult = {
@@ -196,6 +200,7 @@ export type EditorImageGenerationResult = {
resource?: EditorProjectResourceSnapshot | null;
asset?: EditorAssetSnapshot | null;
project?: EditorProjectSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
export type EditorIconSpritesheetIconResult = {
@@ -220,6 +225,8 @@ export type EditorIconSpritesheetGenerationResult = {
priceMudPoints: number;
spritesheetResource?: EditorProjectResourceSnapshot | null;
spritesheetAsset?: EditorAssetSnapshot | null;
project?: EditorProjectSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
export type EditorCharacterAnimationResolution = '480p' | '720p';
@@ -247,6 +254,10 @@ export type EditorCharacterAnimationGenerationInput = {
frameCount: EditorCharacterAnimationFrameCount;
durationSeconds: EditorCharacterAnimationDurationSeconds;
model: 'seedance2.0-fast';
projectId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
generationInputs?: EditorAssetGenerationInputs | null;
sourceResourceId?: string | null;
};
export type EditorCharacterAnimationFrameResult = {
@@ -266,6 +277,8 @@ export type EditorCharacterAnimationGenerationResult = {
durationSeconds: number;
fps: number;
priceMudPoints: number;
project?: EditorProjectSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
export type EditorVideoModel =
@@ -298,6 +311,11 @@ export type EditorVideoGenerationInput = {
referenceImageSrcs?: string[];
referenceVideoSrcs?: string[];
referenceAudioSrcs?: string[];
projectId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
generationInputs?: EditorAssetGenerationInputs | null;
sourceResourceId?: string | null;
assetKind?: string | null;
};
export type EditorVideoGenerationResult = {
@@ -313,17 +331,25 @@ export type EditorVideoGenerationResult = {
provider: string;
taskId: string;
priceMudPoints?: number;
project?: EditorProjectSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
export type EditorSoundEffectGenerationInput = {
prompt: string;
model: 'audio1.0';
duration: number;
projectId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
generationInputs?: EditorAssetGenerationInputs | null;
};
export type EditorBackgroundMusicGenerationInput = {
gptDescriptionPrompt: string;
makeInstrumental: true;
projectId?: string | null;
canvasCompletion?: EditorCanvasGenerationCompletionInput | null;
generationInputs?: EditorAssetGenerationInputs | null;
};
export type EditorAudioGenerationResult = {
@@ -341,6 +367,8 @@ export type EditorAudioGenerationResult = {
priceMudPoints: number;
audioKind: 'sound-effect' | 'background-music';
durationSeconds?: number | null;
project?: EditorProjectSnapshot | null;
queueState?: ExternalGenerationJobStatusRecord | null;
};
export type EditorGenerationPricingUnit = 'perGeneration' | 'perSecond';
@@ -699,6 +727,9 @@ export async function generateEditorIconSpritesheet(
? { generationInputs: input.generationInputs }
: {}),
...(input.assetFolderId ? { assetFolderId: input.assetFolderId } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
}),
'生成图标素材失败',
{
@@ -727,6 +758,9 @@ export async function extractEditorUiDesignAssets(
: {}),
...(input.aspectRatio ? { aspectRatio: input.aspectRatio } : {}),
...(input.imageSize ? { imageSize: input.imageSize } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
}),
'提取UI设计图素材失败',
{
@@ -756,6 +790,9 @@ export async function editEditorImage(input: EditorImageEditInput) {
...(input.sourceResourceId
? { sourceResourceId: input.sourceResourceId }
: {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
}),
'修改图片失败',
{
@@ -772,7 +809,19 @@ export async function generateEditorCharacterAnimation(
) {
return requestJson<EditorCharacterAnimationGenerationResult>(
EDITOR_CHARACTER_ANIMATION_GENERATION_API,
jsonRequest('POST', input),
jsonRequest('POST', {
...input,
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.sourceResourceId
? { sourceResourceId: input.sourceResourceId }
: {}),
}),
'生成角色动画失败',
{
timeoutMs: 1_200_000,
@@ -804,6 +853,17 @@ export async function generateEditorVideo(input: EditorVideoGenerationInput) {
...(input.referenceAudioSrcs?.length
? { referenceAudioSrcs: input.referenceAudioSrcs }
: {}),
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
...(input.sourceResourceId
? { sourceResourceId: input.sourceResourceId }
: {}),
...(input.assetKind ? { assetKind: input.assetKind } : {}),
}),
'生成视频失败',
{
@@ -824,6 +884,13 @@ export async function generateEditorSoundEffect(
prompt: input.prompt,
model: input.model,
duration: input.duration,
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
}),
'生成游戏音效失败',
{
@@ -843,6 +910,13 @@ export async function generateEditorBackgroundMusic(
jsonRequest('POST', {
gptDescriptionPrompt: input.gptDescriptionPrompt,
makeInstrumental: input.makeInstrumental,
...(input.projectId ? { projectId: input.projectId } : {}),
...(input.canvasCompletion
? { canvasCompletion: input.canvasCompletion }
: {}),
...(input.generationInputs
? { generationInputs: input.generationInputs }
: {}),
}),
'生成游戏背景音乐失败',
{