修复画板生成面板与音乐音效链路

统一画板生成类面板的参数选项、热区交互、底部按钮和输入框布局。

新增并完善画板生成音乐入口、音效和背景音乐面板以及音频生成提交流程。

按 VectorEngine Suno 音效文档调整音效请求体、任务 ID 解析、轮询和音频地址解析。

补充 Seedance 视频生成版权限制报错的中文提示。

更新编辑器相关技术文档和回归测试。
This commit is contained in:
2026-06-18 20:24:52 +08:00
parent eb855644d5
commit b04b6cee2b
28 changed files with 995 additions and 107 deletions
@@ -87,7 +87,7 @@
- 默认选择模式;底部工具栏能切换工具;中键拖拽和 Space 临时抓手都能平移画布。
- 拖拽图片接近其它图片边缘或中心时显示吸附线,并保存吸附后的最终布局。
- 生成图片点击后显示画布内 `Image Generator` 占位框和跟随占位框的生成输入框,生成失败保留占位和输入状态,生成成功后在占位位置创建真实图层,并让输入框继续跟随该生成图。
- 生成中的占位图聚焦后支持键盘 `Delete` / `Backspace` 删除,不新增可见删除按钮;删除后对应异步回写必须按生成器 ID 判空并丢弃,不能把已删除素材重新落回画布。
- 生成中的占位图聚焦后支持键盘 `Delete` / `Backspace` 删除,不新增可见删除按钮;删除后对应异步回写必须按生成器 ID 判空并丢弃,不能把已删除素材重新落回画布。音乐 / 音频生成占位和已生成音频图层同样必须支持键盘删除。
- 生成器快照刷新后必须恢复;待生成、生成中、失败和已生成后跟随成品图层的生成器都不能因为刷新丢失输入、参数、参考图或占位框位置。
- 生成类入口打开画布内面板时,底部 AI 工具栏必须保持可见;`生成规范`、角色 / 图标规范来源、角色常规参考图来源这类轻量菜单通过页面级 fixed portal 渲染,不能留在底部工具栏或参考图横向滚动容器内部,避免被局部 `overflow` 裁切。角色形象规范和常规参考图来源菜单必须向上弹出;常规参考图点击后先选择“从画布中选择”或“上传图片”,从画布取图时只绑定参考图,不触发普通画布图层选中、聚焦、面板隐藏或拖拽逻辑,绑定后退出画布选择状态。所有生成面板参考图槽位统一为方形图标组件;角色形象规范槽位只显示规范 logo 和 `角色规范` 四字,绑定来源标题只保留给可访问名称、悬浮 title 和图片信息。已有参考图槽位只有在 hover / focus 时显示右上角 `×`,点击后只解绑对应参考图。
- 生成规范类图片面板底部必须以禁用态参数按钮显示 `16:9 · 2K``gpt-image-2`,视觉对齐可编辑面板参数控件,提交到 `/api/editor/images/generations` 时也固定携带这些参数。
@@ -24,7 +24,7 @@
- `type`:合成选项,只支持 `one-shot` / `loop`UI 显示为 `单次` / `循环`,不展示字段标题。
- `tempo`:可空 BPM 数值;默认不填为 `null`,用户可通过拖拉条或数字输入设置,范围 `1-300`
- `type``tempo` 合并为一个 Lovart 式底部参数按钮;无 BPM 时按钮显示 `单次` / `循环`,有 BPM 时显示 `单次·120BPM` 这类组合值。
- 提交时映射到 VectorEngine 游戏音效接口字段:`sound``type``tempo`
- 提交到 VectorEngine 时映射为 Suno 音效模式字段:`mv``task: "sound"``metadata_params.sound``metadata_params.type``metadata_params.tempo``metadata_params.key: null``mv` 后端固定使用默认 Suno 模型,不在 UI 中展示
### 生成游戏背景音乐
@@ -94,7 +94,10 @@ POST /api/editor/audios/background-music/generations
-`platform-audio` 增加编辑器专用 body builder 和 submit 函数:
- 背景音乐 body 使用 `mv``gpt_description_prompt``make_instrumental`
- Suno 音乐接口路径固定为 `/suno/submit/music``VECTOR_ENGINE_BASE_URL` 即使配置为带 `/v1` 的图片接口根,也要在 `platform-audio` 中归一为根路径后再拼接,避免误请求 `/v1/suno/submit/music`
- 音效 body 使用 `sound``type``tempo`,其中 `tempo``1-300` 的 BPM 或 `null`
- 音效 body 使用 Suno 音效契约:`mv``task: "sound"``metadata_params`,其中 `metadata_params.tempo``1-300` 的 BPM 或 `null``metadata_params.key` 固定 `null`
- 编辑器音效提交和轮询都使用 Suno 路径,提交 `/suno/submit/music`,查询 `/suno/fetch/{taskId}`;旧 Vidu `/ent/v2/text2audio` 仅保留给历史视觉小说音效链路。
- Suno 提交成功后的任务 ID 兼容从 `data` 字符串、`data.task_id``data.taskId``data.id``task_id``taskId``id` 中读取,避免音效接口只返回 `data` 时误报“上游未返回任务 ID”。
- VectorEngine 音频响应的 `code` 需要兼容 `"success"``"ok"``"0"``"200"` 以及数字 `0` / `200`;HTTP 非 2xx 时后端错误信息应透出安全的上游状态和短响应摘要,避免前端只显示笼统提交失败。
-`api-server` 增加编辑器音频 BFF
- `/api/editor/audios/sound-effects/generations`
- `/api/editor/audios/background-music/generations`
@@ -3592,7 +3592,7 @@ fn parse_animation_api_error_message(raw_text: &str, fallback_message: &str) ->
.map(str::trim)
.filter(|value| !value.is_empty())
{
return message.to_string();
return localize_animation_api_error_message(message);
}
if let Some(message) = parsed
.get("message")
@@ -3600,10 +3600,25 @@ fn parse_animation_api_error_message(raw_text: &str, fallback_message: &str) ->
.map(str::trim)
.filter(|value| !value.is_empty())
{
return message.to_string();
return localize_animation_api_error_message(message);
}
}
raw_text.trim().to_string()
localize_animation_api_error_message(raw_text.trim())
}
fn localize_animation_api_error_message(message: &str) -> String {
let normalized = message.to_ascii_lowercase();
if normalized.contains("output video")
&& normalized.contains("copyright")
&& normalized.contains("restriction")
{
return "生成失败:输出视频可能涉及版权限制,请调整提示词或更换参考素材后重试。"
.to_string();
}
if normalized.contains("copyright") && normalized.contains("restriction") {
return "生成失败:内容可能涉及版权限制,请调整提示词或更换参考素材后重试。".to_string();
}
message.to_string()
}
fn parse_animation_upstream_error(raw_text: &str, fallback_message: &str) -> AppError {
@@ -4979,6 +4994,37 @@ mod tests {
);
}
#[test]
fn seedance_copyright_restriction_error_is_localized() {
let raw = json!({
"error": {
"message": "The request failed because the output video may be related to copyright restrictions. Request id: 0217817823139540000000000000000000ffffac191c1d86f05"
}
})
.to_string();
let message = parse_animation_api_error_message(&raw, "创建画板视频任务失败。");
assert_eq!(
message,
"生成失败:输出视频可能涉及版权限制,请调整提示词或更换参考素材后重试。"
);
assert!(!message.contains("copyright"));
assert!(!message.contains("Request id"));
}
#[test]
fn raw_seedance_copyright_restriction_error_is_localized() {
let message = parse_animation_api_error_message(
"The request failed because the output video may be related to copyright restrictions. Request id: abc",
"创建画板视频任务失败。",
);
assert_eq!(
message,
"生成失败:输出视频可能涉及版权限制,请调整提示词或更换参考素材后重试。"
);
}
#[test]
fn editor_video_rejects_price_mismatch() {
let error = normalize_editor_video_request(EditorVideoGenerateRequest {
@@ -19,7 +19,7 @@ use crate::{
use super::{
clock::{current_utc_iso_text, current_utc_micros},
errors::{map_platform_audio_error, parse_json_payload, vector_engine_bad_gateway},
publish::wait_for_generated_audio_asset,
publish::{wait_for_generated_audio_asset, wait_for_generated_audio_asset_with_task_kind},
settings::require_vector_engine_audio_settings,
tasks::create_sound_effect_task_response,
types::{AudioAssetBindingTarget, AudioAssetSlot, GeneratedCreationAudioTarget},
@@ -122,16 +122,18 @@ pub async fn generate_editor_sound_effect(
sound: normalized.sound.clone(),
sound_type: normalized.sound_type.clone(),
tempo: normalized.tempo,
model: None,
},
)
.await
.map_err(map_platform_audio_error)
.map_err(|error| error.into_response_with_context(Some(&request_context)))?;
let generated = wait_for_generated_audio_asset(
let generated = wait_for_generated_audio_asset_with_task_kind(
&state,
authenticated.claims().user_id(),
task.task_id.clone(),
AudioAssetSlot::SoundEffect,
task.kind,
build_editor_audio_target(&task.task_id, "sound_effect", "editor_sound_effect"),
)
.await
@@ -151,7 +153,7 @@ pub async fn generate_editor_sound_effect(
source_type: "generated".to_string(),
prompt: normalized.sound.clone(),
actual_prompt: Some(normalized.sound),
model: platform_audio::VIDU_AUDIO_MODEL.to_string(),
model: platform_audio::SUNO_DEFAULT_MODEL.to_string(),
provider: generated.provider,
task_id: generated.task_id,
price_mud_points: normalized.price_mud_points,
@@ -25,7 +25,8 @@ pub(super) async fn persist_generated_audio_asset(
http_client: &reqwest::Client,
owner_user_id: &str,
task_id: &str,
slot: AudioAssetSlot,
_slot: AudioAssetSlot,
task_kind: platform_audio::AudioTaskKind,
target: AudioAssetBindingTarget,
audio: DownloadedAudio,
) -> Result<PersistedAudioAsset, AppError> {
@@ -41,7 +42,7 @@ pub(super) async fn persist_generated_audio_asset(
platform_audio::prepare_generated_audio_put_request(GeneratedAudioPersistInput {
owner_user_id: owner_user_id.to_string(),
task_id: task_id.to_string(),
task_kind: slot.task_kind(),
task_kind,
target: GeneratedAudioPersistTarget {
entity_kind: target.entity_kind.clone(),
entity_id: target.entity_id.clone(),
@@ -23,6 +23,25 @@ pub(super) async fn publish_generated_audio_asset(
task_id: String,
slot: AudioAssetSlot,
target: AudioAssetBindingTarget,
) -> Result<creation_audio::GeneratedAudioAssetResponse, AppError> {
publish_generated_audio_asset_with_task_kind(
state,
owner_user_id,
task_id,
slot,
slot.task_kind(),
target,
)
.await
}
pub(super) async fn publish_generated_audio_asset_with_task_kind(
state: &AppState,
owner_user_id: &str,
task_id: String,
slot: AudioAssetSlot,
task_kind: platform_audio::AudioTaskKind,
target: AudioAssetBindingTarget,
) -> Result<creation_audio::GeneratedAudioAssetResponse, AppError> {
let task_id = platform_audio::normalize_limited_text(&task_id, "taskId", 160)
.map_err(map_platform_audio_error)?;
@@ -33,7 +52,7 @@ pub(super) async fn publish_generated_audio_asset(
platform_audio::resolve_audio_task_download_urls(
&http_client,
&settings,
slot.task_kind(),
task_kind,
&task_id,
)
.await
@@ -43,7 +62,7 @@ pub(super) async fn publish_generated_audio_asset(
return Ok(creation_audio::GeneratedAudioAssetResponse {
kind: slot.creation_contract_kind(),
task_id,
provider: slot.provider().to_string(),
provider: task_kind.provider().to_string(),
status: status.clone(),
asset_object_id: None,
asset_kind: None,
@@ -71,16 +90,20 @@ pub(super) async fn publish_generated_audio_asset(
billing_asset_id.as_str(),
points_cost,
async {
let audio =
platform_audio::download_generated_audio(&http_client, &audio_url, slot.provider())
.await
.map_err(map_platform_audio_error)?;
let audio = platform_audio::download_generated_audio(
&http_client,
&audio_url,
task_kind.provider(),
)
.await
.map_err(map_platform_audio_error)?;
persist_generated_audio_asset(
state,
&http_client,
owner_user_id,
&task_id,
slot,
task_kind,
target.clone(),
audio,
)
@@ -92,7 +115,7 @@ pub(super) async fn publish_generated_audio_asset(
Ok(creation_audio::GeneratedAudioAssetResponse {
kind: slot.creation_contract_kind(),
task_id,
provider: slot.provider().to_string(),
provider: task_kind.provider().to_string(),
status: "completed".to_string(),
asset_object_id: Some(persisted.asset_object_id),
asset_kind: Some(target.asset_kind),
@@ -106,14 +129,34 @@ pub(super) async fn wait_for_generated_audio_asset(
task_id: String,
slot: AudioAssetSlot,
target: AudioAssetBindingTarget,
) -> Result<creation_audio::GeneratedAudioAssetResponse, AppError> {
wait_for_generated_audio_asset_with_task_kind(
state,
owner_user_id,
task_id,
slot,
slot.task_kind(),
target,
)
.await
}
pub(super) async fn wait_for_generated_audio_asset_with_task_kind(
state: &AppState,
owner_user_id: &str,
task_id: String,
slot: AudioAssetSlot,
task_kind: platform_audio::AudioTaskKind,
target: AudioAssetBindingTarget,
) -> Result<creation_audio::GeneratedAudioAssetResponse, AppError> {
let mut latest_status = String::new();
for _ in 0..40 {
let response = publish_generated_audio_asset(
let response = publish_generated_audio_asset_with_task_kind(
state,
owner_user_id,
task_id.clone(),
slot,
task_kind,
target.clone(),
)
.await?;
@@ -124,15 +124,14 @@ fn editor_sound_effect_request_accepts_null_tempo_and_rejects_invalid_type() {
#[test]
fn editor_sound_effect_request_rejects_bpm_outside_documented_range() {
for tempo in [0, 301] {
let error = normalize_editor_sound_effect_request(
assets::EditorSoundEffectGenerateRequest {
let error =
normalize_editor_sound_effect_request(assets::EditorSoundEffectGenerateRequest {
sound: "按钮点击".to_string(),
sound_type: "one-shot".to_string(),
tempo: Some(tempo),
price_mud_points: 10,
},
)
.expect_err("BPM outside 1-300 should fail");
})
.expect_err("BPM outside 1-300 should fail");
assert!(error.to_string().contains("1-300"));
}
@@ -46,10 +46,6 @@ impl AudioAssetSlot {
}
}
pub(super) fn provider(self) -> &'static str {
self.task_kind().provider()
}
pub(super) fn asset_kind(self) -> &'static str {
match self {
Self::BackgroundMusic => MUSIC_ASSET_KIND,
+175 -42
View File
@@ -4,7 +4,8 @@ use reqwest::header;
use serde_json::Value;
use crate::response::{
extract_audio_urls, extract_string_by_path, find_first_string_by_key, normalize_task_status,
extract_audio_urls, extract_string_by_path, extract_suno_clip_ids, find_first_string_by_key,
is_failed_task_status, normalize_task_status,
};
use crate::{
AudioError, AudioTaskKind, AudioTaskResponse, BackgroundMusicTaskRequest,
@@ -43,12 +44,9 @@ pub async fn submit_background_music_task(
"提交 Suno 背景音乐任务失败",
)
.await?;
let task_id = extract_string_by_path(&response, &["data"])
.or_else(|| find_first_string_by_key(&response, "task_id"))
.or_else(|| find_first_string_by_key(&response, "taskId"))
.ok_or_else(|| {
AudioError::missing_audio("提交 Suno 背景音乐任务失败:上游未返回任务 ID")
})?;
let task_id = extract_submit_task_id(&response).ok_or_else(|| {
AudioError::missing_audio("提交 Suno 背景音乐任务失败:上游未返回任务 ID")
})?;
Ok(AudioTaskResponse {
kind: AudioTaskKind::BackgroundMusic,
@@ -72,8 +70,7 @@ pub async fn submit_sound_effect_task(
"提交 Vidu 音效任务失败",
)
.await?;
let task_id = find_first_string_by_key(&response, "task_id")
.or_else(|| find_first_string_by_key(&response, "taskId"))
let task_id = extract_submit_task_id(&response)
.ok_or_else(|| AudioError::missing_audio("提交 Vidu 音效任务失败:上游未返回任务 ID"))?;
let status = find_first_string_by_key(&response, "state").unwrap_or_else(|| "created".into());
@@ -99,12 +96,9 @@ pub async fn submit_editor_background_music_task(
"提交编辑器背景音乐任务失败",
)
.await?;
let task_id = extract_string_by_path(&response, &["data"])
.or_else(|| find_first_string_by_key(&response, "task_id"))
.or_else(|| find_first_string_by_key(&response, "taskId"))
.ok_or_else(|| {
AudioError::missing_audio("提交编辑器背景音乐任务失败:上游未返回任务 ID")
})?;
let task_id = extract_submit_task_id(&response).ok_or_else(|| {
AudioError::missing_audio("提交编辑器背景音乐任务失败:上游未返回任务 ID")
})?;
Ok(AudioTaskResponse {
kind: AudioTaskKind::BackgroundMusic,
@@ -123,20 +117,19 @@ pub async fn submit_editor_sound_effect_task(
let response = post_vector_engine_json(
http_client,
settings,
AudioTaskKind::SoundEffect.submit_path(),
AudioTaskKind::SunoSoundEffect.submit_path(),
body,
"提交编辑器音效任务失败",
)
.await?;
let task_id = find_first_string_by_key(&response, "task_id")
.or_else(|| find_first_string_by_key(&response, "taskId"))
let task_id = extract_submit_task_id(&response)
.ok_or_else(|| AudioError::missing_audio("提交编辑器音效任务失败:上游未返回任务 ID"))?;
let status = find_first_string_by_key(&response, "state").unwrap_or_else(|| "created".into());
Ok(AudioTaskResponse {
kind: AudioTaskKind::SoundEffect,
kind: AudioTaskKind::SunoSoundEffect,
task_id,
provider: AudioTaskKind::SoundEffect.provider().to_string(),
provider: AudioTaskKind::SunoSoundEffect.provider().to_string(),
status,
})
}
@@ -154,6 +147,7 @@ async fn fetch_audio_task_payload(
match kind {
AudioTaskKind::BackgroundMusic => "查询 Suno 背景音乐任务失败",
AudioTaskKind::SoundEffect => "查询 Vidu 音效任务失败",
AudioTaskKind::SunoSoundEffect => "查询 Suno 音效任务失败",
},
)
.await
@@ -174,27 +168,58 @@ pub async fn resolve_audio_task_download_urls(
.unwrap_or(""),
);
let mut audio_urls = extract_audio_urls(&task_payload);
if kind == AudioTaskKind::BackgroundMusic && audio_urls.is_empty() {
if let Some(clip_id) = extract_string_by_path(&task_payload, &["data"]).and_then(|value| {
if value.trim().is_empty() {
None
} else {
Some(value)
}
}) {
let wav_payload = get_vector_engine_json(
if matches!(
kind,
AudioTaskKind::BackgroundMusic | AudioTaskKind::SunoSoundEffect
) && audio_urls.is_empty()
{
let clip_ids = extract_suno_clip_ids(&task_payload);
let has_clip_ids = !clip_ids.is_empty();
let mut latest_wav_error = None;
for clip_id in clip_ids {
let wav_payload = match get_vector_engine_json(
http_client,
settings,
&format!("/suno/act/wav/{}", urlencoding::encode(clip_id.as_str())),
"获取 Suno wav 音频失败",
match kind {
AudioTaskKind::SunoSoundEffect => "获取 Suno 音效 wav 失败",
_ => "获取 Suno wav 音频失败",
},
)
.await?;
.await
{
Ok(payload) => {
latest_wav_error = None;
payload
}
Err(error) => {
latest_wav_error = Some(error);
continue;
}
};
audio_urls = extract_audio_urls(&wav_payload);
if !audio_urls.is_empty() {
break;
}
}
if audio_urls.is_empty()
&& let Some(error) = latest_wav_error
{
return Err(error);
}
// 中文注释:Suno 任务完成后可能先返回 clip id/suno/act/wav/{clip_id}
// 的 wav_file_url 仍为 null。此时不能把任务判成失败,让上层轮询继续等待。
if should_wait_for_suno_wav_audio(&status, has_clip_ids, &audio_urls) {
return Ok(("processing".to_string(), audio_urls));
}
}
Ok((status, audio_urls))
}
fn should_wait_for_suno_wav_audio(status: &str, has_clip_ids: bool, audio_urls: &[String]) -> bool {
audio_urls.is_empty() && has_clip_ids && !is_failed_task_status(status)
}
async fn get_vector_engine_json(
http_client: &reqwest::Client,
settings: &VectorEngineAudioSettings,
@@ -280,7 +305,7 @@ async fn parse_vector_engine_response(
})?;
if !status.is_success() {
return Err(AudioError::upstream(
failure_context.to_string(),
build_upstream_error_message(failure_context, status.as_u16(), raw_text.as_str()),
status.as_u16(),
truncate_raw(raw_text.as_str()),
));
@@ -292,18 +317,12 @@ async fn parse_vector_engine_response(
truncate_raw(raw_text.as_str()),
)
})?;
if let Some(code) = payload.get("code").and_then(Value::as_str)
&& !matches!(
code.trim().to_ascii_lowercase().as_str(),
"success" | "succeeded" | "ok"
)
if let Some(code) = payload.get("code")
&& !is_success_response_code(code)
{
return Err(AudioError::upstream(
payload
.get("message")
.and_then(Value::as_str)
.unwrap_or(failure_context)
.to_string(),
extract_upstream_payload_message(&payload)
.unwrap_or_else(|| failure_context.to_string()),
status.as_u16(),
truncate_raw(raw_text.as_str()),
));
@@ -329,6 +348,62 @@ fn truncate_raw(raw_text: &str) -> String {
raw_text.chars().take(800).collect()
}
fn extract_submit_task_id(response: &Value) -> Option<String> {
// 中文注释:Suno 音频提交接口文档未声明响应结构,实际可能直接把任务 ID 放在 data 字符串或 data 对象中。
extract_string_by_path(response, &["data"])
.or_else(|| extract_string_by_path(response, &["data", "task_id"]))
.or_else(|| extract_string_by_path(response, &["data", "taskId"]))
.or_else(|| extract_string_by_path(response, &["data", "id"]))
.or_else(|| find_first_string_by_key(response, "task_id"))
.or_else(|| find_first_string_by_key(response, "taskId"))
.or_else(|| find_first_string_by_key(response, "id"))
.map(|value| value.trim().to_string())
.filter(|value| !value.is_empty())
}
fn build_upstream_error_message(failure_context: &str, status: u16, raw_text: &str) -> String {
let excerpt = truncate_raw(raw_text);
if excerpt.trim().is_empty() {
return format!("{failure_context}:上游 HTTP {status}");
}
format!("{failure_context}:上游 HTTP {status}{excerpt}")
}
fn is_success_response_code(code: &Value) -> bool {
match code {
Value::String(code) => matches!(
code.trim().to_ascii_lowercase().as_str(),
"success" | "succeeded" | "ok" | "0" | "200"
),
Value::Number(code) => code.as_i64().is_some_and(|code| code == 0 || code == 200),
_ => false,
}
}
fn extract_upstream_payload_message(payload: &Value) -> Option<String> {
payload
.get("message")
.and_then(Value::as_str)
.map(str::trim)
.filter(|message| !message.is_empty())
.map(ToOwned::to_owned)
.or_else(|| {
payload
.get("error")
.and_then(|value| match value {
Value::String(message) => Some(message.as_str()),
Value::Object(object) => object
.get("message")
.and_then(Value::as_str)
.or_else(|| object.get("detail").and_then(Value::as_str)),
_ => None,
})
.map(str::trim)
.filter(|message| !message.is_empty())
.map(ToOwned::to_owned)
})
}
#[cfg(test)]
mod tests {
use super::*;
@@ -365,4 +440,62 @@ mod tests {
"https://api.vectorengine.cn/suno/submit/music"
);
}
#[test]
fn vector_engine_audio_success_codes_accept_numeric_and_string_values() {
for code in [
Value::String("success".to_string()),
Value::String("200".to_string()),
Value::String("0".to_string()),
Value::from(200),
Value::from(0),
] {
assert!(is_success_response_code(&code), "{code}");
}
assert!(!is_success_response_code(&Value::String(
"invalid_request".to_string()
)));
}
#[test]
fn submit_task_id_extracts_suno_data_string_and_nested_ids() {
assert_eq!(
extract_submit_task_id(&serde_json::json!({
"code": 200,
"data": "suno-task-1"
})),
Some("suno-task-1".to_string())
);
assert_eq!(
extract_submit_task_id(&serde_json::json!({
"code": 200,
"data": {
"id": "suno-task-2"
}
})),
Some("suno-task-2".to_string())
);
assert_eq!(
extract_submit_task_id(&serde_json::json!({
"taskId": "legacy-task-1"
})),
Some("legacy-task-1".to_string())
);
}
#[test]
fn suno_clip_ids_without_wav_url_keep_task_pending() {
assert!(should_wait_for_suno_wav_audio(
"completed",
true,
&Vec::new()
));
assert!(!should_wait_for_suno_wav_audio("failed", true, &Vec::new()));
assert!(!should_wait_for_suno_wav_audio(
"completed",
true,
&["https://cdn.example.test/audio.wav".to_string()]
));
}
}
+10 -3
View File
@@ -84,10 +84,17 @@ pub fn build_editor_sound_effect_task_body(
return Err(AudioError::invalid_request("tempo BPM 必须在 1-300 之间"));
}
}
let model = normalize_optional_text(request.model.as_deref())
.unwrap_or_else(|| SUNO_DEFAULT_MODEL.to_string());
Ok(json!({
"sound": sound,
"type": sound_type,
"tempo": request.tempo,
"mv": model,
"task": "sound",
"metadata_params": {
"sound": sound,
"type": sound_type,
"tempo": request.tempo,
"key": null,
},
}))
}
@@ -37,6 +37,21 @@ pub fn extract_audio_urls(payload: &Value) -> Vec<String> {
deduped
}
pub(crate) fn extract_suno_clip_ids(payload: &Value) -> Vec<String> {
let mut clip_ids = Vec::new();
if let Some(data) = payload.get("data") {
collect_suno_clip_ids_from_data(data, &mut clip_ids);
}
let mut deduped = Vec::new();
for clip_id in clip_ids {
if !deduped.contains(&clip_id) {
deduped.push(clip_id);
}
}
deduped
}
pub(crate) fn find_first_string_by_key(value: &Value, target_key: &str) -> Option<String> {
match value {
Value::Object(object) => {
@@ -67,6 +82,34 @@ pub(crate) fn extract_string_by_path(value: &Value, path: &[&str]) -> Option<Str
current.as_str().map(str::trim).map(ToOwned::to_owned)
}
fn collect_suno_clip_ids_from_data(value: &Value, output: &mut Vec<String>) {
match value {
Value::Object(object) => {
for (key, value) in object {
// 中文注释:对象里的普通状态 / 任务字段不能按裸字符串当作 clip id,只接受明确的音频 id 字段。
if let Some(raw) = value.as_str()
&& looks_like_suno_clip_id_key(key)
&& is_plausible_suno_clip_id(raw)
{
output.push(raw.trim().to_string());
}
if value.is_object() || value.is_array() {
collect_suno_clip_ids_from_data(value, output);
}
}
}
Value::Array(items) => {
for item in items {
collect_suno_clip_ids_from_data(item, output);
}
}
Value::String(raw) if is_plausible_suno_clip_id(raw) => {
output.push(raw.trim().to_string());
}
_ => {}
}
}
fn collect_audio_url_strings(value: &Value, output: &mut Vec<String>) {
match value {
Value::Object(object) => {
@@ -92,6 +135,19 @@ fn collect_audio_url_strings(value: &Value, output: &mut Vec<String>) {
}
}
fn looks_like_suno_clip_id_key(key: &str) -> bool {
let normalized = key.trim().to_ascii_lowercase().replace(['_', '-'], "");
matches!(normalized.as_str(), "id" | "clipid" | "audioid" | "songid")
}
fn is_plausible_suno_clip_id(value: &str) -> bool {
let value = value.trim();
!value.is_empty()
&& value.len() <= 180
&& !looks_like_http_url(value)
&& !value.chars().any(char::is_whitespace)
}
fn looks_like_audio_url_key(key: &str) -> bool {
let normalized = key.trim().to_ascii_lowercase();
normalized.contains("audio")
@@ -123,3 +179,44 @@ fn looks_like_audio_url(value: &str) -> bool {
|| value.ends_with(".webm")
|| value.ends_with(".flac")
}
#[cfg(test)]
mod tests {
use super::*;
use serde_json::json;
#[test]
fn extracts_suno_clip_ids_from_object_or_array_data() {
let payload = json!({
"code": "success",
"data": [
{ "id": "clip-a", "status": "complete" },
{ "clip_id": "clip-b", "audio_url": "" },
{ "task_id": "task-should-not-be-treated-as-clip" },
{ "nested": { "audioId": "clip-c" } }
]
});
assert_eq!(
extract_suno_clip_ids(&payload),
vec![
"clip-a".to_string(),
"clip-b".to_string(),
"clip-c".to_string()
]
);
}
#[test]
fn extracts_suno_clip_id_from_string_data() {
let payload = json!({
"code": 0,
"data": "clip-only"
});
assert_eq!(
extract_suno_clip_ids(&payload),
vec!["clip-only".to_string()]
);
}
}
@@ -2,6 +2,7 @@
pub enum AudioTaskKind {
BackgroundMusic,
SoundEffect,
SunoSoundEffect,
}
impl AudioTaskKind {
@@ -9,6 +10,7 @@ impl AudioTaskKind {
match self {
Self::BackgroundMusic => VECTOR_ENGINE_SUNO_PROVIDER,
Self::SoundEffect => VECTOR_ENGINE_VIDU_PROVIDER,
Self::SunoSoundEffect => VECTOR_ENGINE_SUNO_PROVIDER,
}
}
@@ -16,6 +18,7 @@ impl AudioTaskKind {
match self {
Self::BackgroundMusic => "/suno/submit/music",
Self::SoundEffect => "/ent/v2/text2audio",
Self::SunoSoundEffect => "/suno/submit/music",
}
}
@@ -25,6 +28,7 @@ impl AudioTaskKind {
Self::SoundEffect => {
format!("/ent/v2/tasks/{}/creations", urlencoding::encode(task_id))
}
Self::SunoSoundEffect => format!("/suno/fetch/{}", urlencoding::encode(task_id)),
}
}
@@ -32,6 +36,7 @@ impl AudioTaskKind {
match self {
Self::BackgroundMusic => "background-music",
Self::SoundEffect => "sound-effect",
Self::SunoSoundEffect => "sound-effect",
}
}
}
@@ -64,6 +69,7 @@ pub struct EditorSoundEffectTaskRequest {
pub sound: String,
pub sound_type: String,
pub tempo: Option<u16>,
pub model: Option<String>,
}
#[derive(Clone, Debug)]
@@ -132,12 +132,16 @@ fn editor_sound_effect_request_body_uses_sound_type_and_tempo() {
sound: " 金币掉落叮当声 ".to_string(),
sound_type: " one-shot ".to_string(),
tempo: Some(120),
model: None,
})
.expect("editor sound effect body should be valid");
assert_eq!(body["sound"], "金币掉落叮当声");
assert_eq!(body["type"], "one-shot");
assert_eq!(body["tempo"], 120);
assert_eq!(body["mv"], SUNO_DEFAULT_MODEL);
assert_eq!(body["task"], "sound");
assert_eq!(body["metadata_params"]["sound"], "金币掉落叮当声");
assert_eq!(body["metadata_params"]["type"], "one-shot");
assert_eq!(body["metadata_params"]["tempo"], 120);
assert_eq!(body["metadata_params"]["key"], serde_json::Value::Null);
assert!(body.get("prompt").is_none());
}
@@ -147,11 +151,13 @@ fn editor_sound_effect_request_body_keeps_null_tempo() {
sound: "按钮确认短促音".to_string(),
sound_type: "loop".to_string(),
tempo: None,
model: Some(" chirp-v5-5 ".to_string()),
})
.expect("unset BPM should serialize as null");
assert_eq!(body["type"], "loop");
assert_eq!(body["tempo"], serde_json::Value::Null);
assert_eq!(body["mv"], "chirp-v5-5");
assert_eq!(body["metadata_params"]["type"], "loop");
assert_eq!(body["metadata_params"]["tempo"], serde_json::Value::Null);
}
#[test]
@@ -160,6 +166,7 @@ fn editor_sound_effect_request_body_rejects_invalid_type_and_tempo() {
sound: "按钮确认短促音".to_string(),
sound_type: "ambient".to_string(),
tempo: Some(120),
model: None,
})
.expect_err("platform layer should reject unsupported sound type");
assert!(invalid_type.message().contains("one-shot"));
@@ -169,6 +176,7 @@ fn editor_sound_effect_request_body_rejects_invalid_type_and_tempo() {
sound: "按钮确认短促音".to_string(),
sound_type: "one-shot".to_string(),
tempo: Some(tempo),
model: None,
})
.expect_err("platform layer should reject BPM outside 1-300");
assert!(invalid_tempo.message().contains("1-300"));
@@ -153,11 +153,18 @@ export function ImageCanvasCharacterAnimationPanelView({
: currentPanel,
);
};
const closePanel = () => {
setCharacterAnimationPanel((currentPanel) =>
currentPanel && isGenerating
? currentPanel
: null,
);
};
return (
<>
<form
className="image-canvas-editor__character-animation-panel"
className="image-canvas-editor__character-animation-panel image-canvas-editor__generation-composer image-canvas-editor__generation-composer--image image-canvas-editor__generation-composer--character-animation"
style={style}
role="dialog"
aria-label="角色动画生成面板"
@@ -189,7 +196,7 @@ export function ImageCanvasCharacterAnimationPanelView({
icon={X}
variant="surfaceFloating"
disabled={isGenerating}
onClick={() => setCharacterAnimationPanel(null)}
onClick={closePanel}
/>
<PlatformTextField
variant="textarea"
@@ -305,6 +305,37 @@ export function hydrateCanvasGenerationDialog(
: undefined,
videoMode: snapshot.videoMode === 'std' ? 'std' : undefined,
videoSound: snapshot.videoSound === 'off' ? 'off' : undefined,
characterAnimationResolution:
snapshot.characterAnimationResolution === '480p' ||
snapshot.characterAnimationResolution === '720p'
? snapshot.characterAnimationResolution
: undefined,
characterAnimationRatio:
snapshot.characterAnimationRatio === 'same' ||
snapshot.characterAnimationRatio === '1:1' ||
snapshot.characterAnimationRatio === '4:3' ||
snapshot.characterAnimationRatio === '16:9' ||
snapshot.characterAnimationRatio === '9:16' ||
snapshot.characterAnimationRatio === '3:4'
? snapshot.characterAnimationRatio
: undefined,
characterAnimationFrameCount:
snapshot.characterAnimationFrameCount === 32 ||
snapshot.characterAnimationFrameCount === 40 ||
snapshot.characterAnimationFrameCount === 48
? snapshot.characterAnimationFrameCount
: undefined,
characterAnimationDurationSeconds:
snapshot.characterAnimationDurationSeconds === 4 ||
snapshot.characterAnimationDurationSeconds === 5 ||
snapshot.characterAnimationDurationSeconds === 6
? snapshot.characterAnimationDurationSeconds
: undefined,
characterAnimationResult:
snapshot.characterAnimationResult &&
typeof snapshot.characterAnimationResult === 'object'
? snapshot.characterAnimationResult
: undefined,
soundType:
snapshot.soundType === 'loop' || snapshot.soundType === 'one-shot'
? snapshot.soundType
@@ -576,6 +607,7 @@ export function generationInputsOrNull(
export function canvasAssetKindOrNull(value: unknown): CanvasAssetKind | null {
return value === 'spec' ||
value === 'character' ||
value === 'character-animation' ||
value === 'icon' ||
value === 'icon-spec' ||
value === 'ui-design' ||
@@ -609,6 +641,7 @@ function isCanvasGenerationDialogMode(
value === 'character' ||
value === 'icon' ||
value === 'ui-design' ||
value === 'character-animation' ||
value === 'video' ||
value === 'audio-sound-effect' ||
value === 'audio-background-music'
@@ -11,6 +11,7 @@ export type CanvasSourceType = 'uploaded' | 'generated' | 'mock_generated';
export type CanvasAssetKind =
| 'spec'
| 'character'
| 'character-animation'
| 'icon'
| 'icon-spec'
| 'ui-design'
@@ -150,6 +151,7 @@ export type GenerateDialogState = {
| 'character'
| 'icon'
| 'ui-design'
| 'character-animation'
| 'video'
| 'audio-sound-effect'
| 'audio-background-music';
@@ -174,6 +176,11 @@ export type GenerateDialogState = {
videoDurationSeconds?: 4 | 5;
videoMode?: 'std';
videoSound?: 'off';
characterAnimationResolution?: EditorCharacterAnimationResolution;
characterAnimationRatio?: EditorCharacterAnimationRatio;
characterAnimationFrameCount?: EditorCharacterAnimationFrameCount;
characterAnimationDurationSeconds?: 4 | 5 | 6;
characterAnimationResult?: EditorCharacterAnimationGenerationResult;
soundType?: 'one-shot' | 'loop';
// 中文注释:音效 tempo 改为可空 BPM,null 表示用户不指定节奏。
soundTempo?: number | null;
@@ -604,6 +604,63 @@ describe('ImageCanvasEditorView', () => {
expect(screen.getByAltText('画布图片:大鱼素材')).toBeTruthy();
});
it('deletes the selected audio layer with Backspace from the audio control focus', async () => {
loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({
projectId: 'editor-project-audio-delete',
title: '音频删除画布',
viewport: { x: 0, y: 0, scale: 1 },
layers: [
{
layerId: 'layer-audio',
resourceId: 'resource-audio',
title: '背景音乐',
src: '/generated-character-drafts/editor-audios/bgm.mp3',
mediaType: 'audio',
assetKind: 'background-music',
x: 120,
y: 140,
width: 420,
height: 120,
originalWidth: 420,
originalHeight: 120,
zIndex: 2,
sourceType: 'generated',
},
{
layerId: 'layer-reference',
resourceId: 'resource-reference',
title: '参考图片',
src: '/creation-type-references/puzzle.webp',
x: 620,
y: 140,
width: 320,
height: 320,
originalWidth: 320,
originalHeight: 320,
zIndex: 1,
sourceType: 'uploaded',
},
],
resources: [],
updatedAt: '2026-06-18T00:00:00.000Z',
});
render(<ImageCanvasEditorView />);
const audioControl = await screen.findByLabelText('画布音频:背景音乐');
fireEvent.pointerDown(audioControl, {
button: 0,
pointerId: 53,
clientX: 160,
clientY: 180,
});
await act(async () => {
fireEvent.keyDown(audioControl, { key: 'Backspace', code: 'Backspace' });
});
expect(screen.queryByLabelText('画布音频:背景音乐')).toBeNull();
expect(screen.getByAltText('画布图片:参考图片')).toBeTruthy();
});
it('blocks the browser context menu inside the editor workspace', () => {
render(<ImageCanvasEditorView />);
@@ -295,7 +295,7 @@ export function ImageCanvasEditorView() {
isCanvasGenerationComposerVisible(currentDialog)
? {
...currentDialog,
composerOpen: currentDialog.generatedLayerId === layerId,
composerOpen: false,
}
: currentDialog,
);
@@ -12,6 +12,8 @@ import type {
import {
AUDIO_FRAME_DISPLAY_SIZE,
AUDIO_FRAME_ORIGINAL_SIZE,
CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE,
CHARACTER_ANIMATION_FRAME_ORIGINAL_SIZE,
CHARACTER_ANIMATION_DURATION_OPTIONS,
CHARACTER_FRAME_DISPLAY_SIZE,
CHARACTER_FRAME_ORIGINAL_SIZE,
@@ -406,6 +408,40 @@ export function createCharacterAnimationPanelDraft(
};
}
export function createCharacterAnimationGenerationDialogDraft({
canvasSize,
viewport,
layer,
}: {
canvasSize: CanvasSize;
viewport: CanvasViewport;
layer: CanvasLayer;
}): Omit<CanvasGenerationDialogState, 'id'> | null {
if (layer.assetKind !== 'character') {
return null;
}
const worldCenter = getViewportWorldCenter({ canvasSize, viewport });
return {
mode: 'character-animation',
sourceLayerId: layer.id,
prompt: '',
status: 'idle',
composerOpen: true,
characterAnimationResolution: '480p',
characterAnimationRatio: 'same',
characterAnimationFrameCount: 32,
characterAnimationDurationSeconds: 4,
placeholder: {
x: worldCenter.x - CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE.width / 2,
y: worldCenter.y - CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE.height / 2,
width: CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE.width,
height: CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE.height,
originalWidth: CHARACTER_ANIMATION_FRAME_ORIGINAL_SIZE.width,
originalHeight: CHARACTER_ANIMATION_FRAME_ORIGINAL_SIZE.height,
},
};
}
export function assignCharacterSpecReference(
dialog: GenerateDialogState | null,
layer: CanvasLayer,
@@ -567,6 +603,9 @@ export function hideGeneratedLayerComposerAfterBlur(
dialog?.mode === 'spec' ||
dialog?.mode === 'character' ||
dialog?.mode === 'icon' ||
dialog?.mode === 'ui-design' ||
dialog?.mode === 'character-animation' ||
dialog?.mode === 'video' ||
dialog?.mode === 'audio-sound-effect' ||
dialog?.mode === 'audio-background-music') &&
dialog.status !== 'generating'
@@ -581,6 +620,12 @@ export function closeGenerateComposerDialog(
dialog: GenerateDialogState | null,
): GenerateDialogState | null {
return dialog?.mode === 'generate' ||
dialog?.mode === 'spec' ||
dialog?.mode === 'character' ||
dialog?.mode === 'icon' ||
dialog?.mode === 'ui-design' ||
dialog?.mode === 'character-animation' ||
dialog?.mode === 'video' ||
dialog?.mode === 'audio-sound-effect' ||
dialog?.mode === 'audio-background-music'
? {
@@ -1,4 +1,5 @@
import type {
EditorCharacterAnimationGenerationResult,
EditorAudioGenerationResult,
EditorIconSpritesheetGenerationResult,
EditorIconSpritesheetIconResult,
@@ -66,6 +67,16 @@ type AudioResultLayerOptions = {
frame?: GenerateDialogState['placeholder'];
};
type CharacterAnimationResultLayerOptions = {
generated: EditorCharacterAnimationGenerationResult;
generatedIndex: number;
title: string;
canvasSize: CanvasSize;
viewport: CanvasViewport;
generationInputs: CanvasGenerationInputs;
frame?: GenerateDialogState['placeholder'];
};
function getViewportWorldCenter({
canvasSize,
viewport,
@@ -359,3 +370,56 @@ export function createAudioResultLayer({
generationInputs,
};
}
export function createCharacterAnimationResultLayer({
generated,
generatedIndex,
title,
canvasSize,
viewport,
generationInputs,
frame,
}: CharacterAnimationResultLayerOptions): CanvasLayer | null {
const previewFrame = generated.frames[0];
if (!previewFrame) {
return null;
}
const originalWidth = previewFrame.width || 1024;
const originalHeight = previewFrame.height || 1024;
const { width, height } = resolveLayerResolutionSize(
originalWidth,
originalHeight,
{ width: 420, height: 420 },
);
const worldCenter = getViewportWorldCenter({ canvasSize, viewport });
const frameX =
frame && frame.width > 0
? frame.x + frame.width / 2 - width / 2
: undefined;
const frameY =
frame && frame.height > 0
? frame.y + frame.height / 2 - height / 2
: undefined;
return {
id: `layer-character-animation-${generatedIndex}`,
resourceId: `local-resource-character-animation-${generatedIndex}`,
title,
src: previewFrame.imageSrc,
x: frameX ?? worldCenter.x - width / 2,
y: frameY ?? worldCenter.y - height / 2,
width,
height,
originalWidth,
originalHeight,
zIndex: generatedIndex + 10,
sourceType: 'generated',
prompt: generated.prompt,
actualPrompt: generated.prompt,
model: generated.model,
provider: 'ark',
taskId: generated.taskId,
assetKind: 'character-animation',
generationInputs,
};
}
@@ -39,6 +39,14 @@ export const SPEC_FRAME_ORIGINAL_SIZE = { width: 2048, height: 1152 };
export const SPEC_FRAME_DISPLAY_SIZE = { width: 560, height: 315 };
export const CHARACTER_FRAME_ORIGINAL_SIZE = { width: 2048, height: 2048 };
export const CHARACTER_FRAME_DISPLAY_SIZE = { width: 420, height: 420 };
export const CHARACTER_ANIMATION_FRAME_ORIGINAL_SIZE = {
width: 1024,
height: 1024,
};
export const CHARACTER_ANIMATION_FRAME_DISPLAY_SIZE = {
width: 420,
height: 420,
};
export const ICON_FRAME_ORIGINAL_SIZE = { width: 512, height: 512 };
export const ICON_FRAME_DISPLAY_SIZE = { width: 360, height: 360 };
export const UI_DESIGN_FRAME_ORIGINAL_SIZE = { width: 2048, height: 1152 };
@@ -251,6 +259,9 @@ export function getLayerKindLabel(layer: CanvasLayer) {
if (layer.assetKind === 'character') {
return '角色';
}
if (layer.assetKind === 'character-animation') {
return '动作';
}
if (layer.assetKind === 'icon') {
return '图标';
}
@@ -275,6 +286,22 @@ export function getLayerKindLabel(layer: CanvasLayer) {
return null;
}
export function buildCharacterAnimationGenerationInputs(
promptText: string,
sourceLayer: CanvasLayer,
): CanvasGenerationInputs {
return {
fields: createGenerationInputField('动作描述', promptText),
references: [
{
title: '角色图片',
label: sourceLayer.title,
src: sourceLayer.objectKey?.trim() || sourceLayer.src,
},
],
};
}
export function formatLayerImageType(layer: CanvasLayer) {
if (layer.assetKind === 'spec') {
return '规范图片';
@@ -282,6 +309,9 @@ export function formatLayerImageType(layer: CanvasLayer) {
if (layer.assetKind === 'character') {
return '角色图片';
}
if (layer.assetKind === 'character-animation') {
return '角色动作图片';
}
if (layer.assetKind === 'icon') {
return '图标素材图片';
}
@@ -550,6 +580,7 @@ export function isCanvasGenerationDialog(
dialog.mode === 'character' ||
dialog.mode === 'icon' ||
dialog.mode === 'ui-design' ||
dialog.mode === 'character-animation' ||
dialog.mode === 'video' ||
dialog.mode === 'audio-sound-effect' ||
dialog.mode === 'audio-background-music'),
@@ -574,6 +605,9 @@ export function getGenerationFrameAriaLabel(
if (dialog.mode === 'video') {
return '视频生成占位图';
}
if (dialog.mode === 'character-animation') {
return '角色动作生成占位图';
}
if (dialog.mode === 'audio-sound-effect') {
return '音效生成占位图';
}
@@ -599,6 +633,9 @@ export function getGenerationFrameLabel(dialog: CanvasGenerationDialogState) {
if (dialog.mode === 'video') {
return 'Video Generator';
}
if (dialog.mode === 'character-animation') {
return 'Action Generator';
}
if (dialog.mode === 'audio-sound-effect') {
return 'Sound Generator';
}
@@ -203,6 +203,9 @@ export function isCanvasGenerationComposerVisible(
dialog?.mode === 'character' ||
dialog?.mode === 'icon' ||
dialog?.mode === 'ui-design' ||
dialog?.mode === 'video'
dialog?.mode === 'character-animation' ||
dialog?.mode === 'video' ||
dialog?.mode === 'audio-sound-effect' ||
dialog?.mode === 'audio-background-music'
);
}
@@ -33,6 +33,11 @@ const CANVAS_GENERATION_DIALOG_MODES = new Set([
'spec',
'character',
'icon',
'ui-design',
'character-animation',
'video',
'audio-sound-effect',
'audio-background-music',
]);
function isFiniteNumber(value: unknown): value is number {
@@ -219,14 +224,14 @@ export function createLayerDragStart({
export function updateGenerateDialogForLayerPointerDown(
dialog: GenerateDialogState | null,
layerId: string,
_layerId: string,
): GenerateDialogState | null {
if (!hasCanvasGenerationDialogMode(dialog)) {
return dialog;
}
return {
...dialog,
composerOpen: dialog.generatedLayerId === layerId,
composerOpen: false,
};
}
@@ -1,4 +1,12 @@
import { Film, ImageIcon, Info, Music2, UserRound, Volume2 } from 'lucide-react';
import {
Film,
ImageIcon,
Info,
Music2,
PersonStanding,
UserRound,
Volume2,
} from 'lucide-react';
import type {
CSSProperties,
MouseEvent as ReactMouseEvent,
@@ -34,6 +42,15 @@ function getGenerationPlaceholderMeta(dialog: CanvasGenerationDialogState) {
icon: <UserRound className="h-8 w-8" />,
labelIcon: <UserRound className="h-4 w-4" />,
};
case 'character-animation':
return {
frameClassName:
'image-canvas-editor__generation-frame--character-animation',
badgeClassName: 'character-animation',
badgeLabel: '动作',
icon: <PersonStanding className="h-8 w-8" />,
labelIcon: <PersonStanding className="h-4 w-4" />,
};
case 'spec':
return {
frameClassName: 'image-canvas-editor__generation-frame--spec',
@@ -418,7 +435,12 @@ export function ImageCanvasWorldView({
{(generateDialog?.mode === 'generate' ||
generateDialog?.mode === 'spec' ||
generateDialog?.mode === 'character' ||
generateDialog?.mode === 'icon') &&
generateDialog?.mode === 'icon' ||
generateDialog?.mode === 'ui-design' ||
generateDialog?.mode === 'character-animation' ||
generateDialog?.mode === 'video' ||
generateDialog?.mode === 'audio-sound-effect' ||
generateDialog?.mode === 'audio-background-music') &&
generateDialog.status === 'generating' &&
generationComposerStyle ? (
<PlatformStatusMessage
@@ -29,12 +29,14 @@ import type {
} from './ImageCanvasEditorTypes';
import {
createAudioResultLayer,
createCharacterAnimationResultLayer,
createGeneratedResultLayer,
createIconSpritesheetResultLayers,
createQuickEditResultLayer,
createVideoResultLayer,
} from './ImageCanvasGenerationLayerModel';
import {
buildCharacterAnimationGenerationInputs,
buildEditGenerationInputs,
isCanvasGenerationDialog,
resolveImageGenerationErrorMessage,
@@ -60,6 +62,7 @@ type GenerationSubmissionWorkflowOptions = {
quickEditSourceLayer: CanvasLayer | null;
setQuickEditPanel: Dispatch<SetStateAction<QuickEditPanelState | null>>;
characterAnimationPanel: CharacterAnimationPanelState | null;
characterAnimationDialog: CanvasGenerationDialogState | null;
characterAnimationSourceLayer: CanvasLayer | null;
setCharacterAnimationPanel: Dispatch<
SetStateAction<CharacterAnimationPanelState | null>
@@ -91,6 +94,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
quickEditSourceLayer,
setQuickEditPanel,
characterAnimationPanel,
characterAnimationDialog,
characterAnimationSourceLayer,
setCharacterAnimationPanel,
setGenerateDialog,
@@ -385,6 +389,62 @@ export function useImageCanvasGenerationSubmissionWorkflow({
],
);
const addCharacterAnimationResultLayer = useCallback(
(
generated: Parameters<
typeof createCharacterAnimationResultLayer
>[0]['generated'],
title: string,
generationInputs: CanvasGenerationInputs,
frame?: GenerateDialogState['placeholder'],
dialogId?: string,
) => {
if (dialogId && !hasCanvasGenerationDialogById(dialogId)) {
return;
}
layerCounterRef.current += 1;
const generatedIndex = layerCounterRef.current;
const nextLayer = createCharacterAnimationResultLayer({
generated,
generatedIndex,
title,
canvasSize,
viewport,
generationInputs,
frame,
});
if (!nextLayer) {
return;
}
addGeneratedLayersToCanvas([nextLayer]);
selectSingleLayer(nextLayer.id);
setActiveSidebarPanel('layers');
if (dialogId) {
updateCanvasGenerationDialogById(dialogId, (currentDialog) => ({
...currentDialog,
status: 'idle',
composerOpen: true,
generatedLayerId: nextLayer.id,
characterAnimationResult: generated,
errorMessage: undefined,
}));
}
setActiveTool('select');
},
[
addGeneratedLayersToCanvas,
canvasSize,
layerCounterRef,
selectSingleLayer,
setActiveSidebarPanel,
setActiveTool,
updateCanvasGenerationDialogById,
hasCanvasGenerationDialogById,
viewport,
],
);
const submitIconSpritesheetGeneration = useCallback(
async (dialog: GenerateDialogState) => {
if (dialog.mode !== 'icon') {
@@ -625,6 +685,22 @@ export function useImageCanvasGenerationSubmissionWorkflow({
panel: characterAnimationPanel,
sourceLayer: characterAnimationSourceLayer,
});
const canvasDialog = characterAnimationDialog;
if (canvasDialog?.mode === 'character-animation') {
updateCanvasGenerationDialogById(canvasDialog.id, (currentDialog) => ({
...currentDialog,
prompt: submissionPlan.promptText,
characterAnimationResolution: characterAnimationPanel.resolution,
characterAnimationRatio: characterAnimationPanel.ratio,
characterAnimationFrameCount: characterAnimationPanel.frameCount,
characterAnimationDurationSeconds:
characterAnimationPanel.durationSeconds,
characterAnimationResult: undefined,
status: 'generating',
composerOpen: false,
errorMessage: undefined,
}));
}
const nextPanel = {
...characterAnimationPanel,
promptText: submissionPlan.promptText,
@@ -632,12 +708,27 @@ export function useImageCanvasGenerationSubmissionWorkflow({
errorMessage: undefined,
result: undefined,
};
setCharacterAnimationPanel(nextPanel);
if (!canvasDialog) {
setCharacterAnimationPanel(nextPanel);
}
try {
const result = await generateEditorCharacterAnimation(
submissionPlan.input,
);
if (canvasDialog) {
addCharacterAnimationResultLayer(
result,
'角色动作',
buildCharacterAnimationGenerationInputs(
submissionPlan.promptText,
characterAnimationSourceLayer,
),
getGeneratingDialogPlaceholder(canvasDialog),
canvasDialog.id,
);
return;
}
setCharacterAnimationPanel((currentPanel) =>
currentPanel
? {
@@ -648,6 +739,24 @@ export function useImageCanvasGenerationSubmissionWorkflow({
: currentPanel,
);
} catch (error) {
if (canvasDialog) {
updateCanvasGenerationDialogById(canvasDialog.id, (currentDialog) => ({
...currentDialog,
prompt: submissionPlan.promptText,
characterAnimationResolution: characterAnimationPanel.resolution,
characterAnimationRatio: characterAnimationPanel.ratio,
characterAnimationFrameCount: characterAnimationPanel.frameCount,
characterAnimationDurationSeconds:
characterAnimationPanel.durationSeconds,
status: 'failed',
composerOpen: true,
errorMessage:
error instanceof Error && error.message.trim()
? error.message
: '生成角色动画失败',
}));
return;
}
setCharacterAnimationPanel((currentPanel) =>
currentPanel
? {
@@ -662,9 +771,13 @@ export function useImageCanvasGenerationSubmissionWorkflow({
);
}
}, [
addCharacterAnimationResultLayer,
characterAnimationDialog,
characterAnimationPanel,
characterAnimationSourceLayer,
getGeneratingDialogPlaceholder,
setCharacterAnimationPanel,
updateCanvasGenerationDialogById,
]);
return useMemo(

Some files were not shown because too many files have changed in this diff Show More