GenarrativeAI/Genarrative
11
0
Fork 0
Code Issues 24 Pull Requests 7 Actions 1 Packages Projects 1 Releases Wiki Activity

扩展画布生成视频参数

Browse Source 
生成视频面板支持多比例、4到15秒时长滑杆和静音开关
生成视频清晰度增加1080p并限制Fast模型不可选
生成视频请求默认携带联网搜索标记但不在前端展示
后端校验视频比例、时长、清晰度和声音参数并映射音频生成
更新编辑器视频参数文档和项目决策记录
This commit is contained in:
段舒康
2026-06-21 20:15:07 +08:00
parent 945b813334
commit 928aecdcb1
17 changed files with 420 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/project-memory/shared-memory/decision-log.md
+8
View File
@@ -72,6 +72,14 @@
- 验证方式：运行 `npx vitest run src/components/image-editor/useImageCanvasUploadWorkflow.test.tsx src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/services/image-editor/editorReferenceUploadClient.test.ts --reporter verbose`、`cargo test -p api-server editor_video --manifest-path server-rs/Cargo.toml`、`cargo test -p shared-contracts editor_video_request_supports_seedance_multimodal_references --manifest-path server-rs/Cargo.toml`，并执行 `npm run typecheck`、`npm run check:encoding`、`git diff --check`。
- 关联文档：`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`、火山 Seedance 2.0 任务创建文档。
## 2026-06-21 图片画布生成视频参数扩展
- 背景：编辑器画布生成视频需要开放更多 Lovart 式参数，同时保留模型能力边界；`seedance2.0-fast` 不支持 `1080p`，联网搜索暂没有可确认的 Ark 视频生成 body 字段。
- 决策：生成视频参数面板支持比例 `16:9 / 9:16 / 1:1 / 4:3 / 3:4 / 21:9`，时长为 4 到 15 秒整数 slider，清晰度支持 `480p / 720p / 1080p`；`seedance2.0-fast` 不展示可用 `1080p`，从其它模型的 `1080p` 切回 Fast 时自动降到 `720p`，后端也拒绝 `seedance2.0-fast + 1080p`。静音只作为一个 toggle 展示，默认有声并映射 `sound=on` / Ark `generate_audio=true`；关闭静音时传 `sound=off` / `generate_audio=false`。`webSearchEnabled` 默认随请求提交为 `true`，但前端不展示联网搜索开关，后端当前只接收契约字段，不向 Ark 透传未知参数。
- 影响范围：图片画布生成视频面板、生成提交模型、画布项目快照恢复、`editorProjectClient`、`shared-contracts`、`api-server` 编辑器视频 BFF、编辑器技术文档。
- 验证方式：运行 `npm run test -- src/components/image-editor/ImageCanvasGenerationModel.test.ts src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx src/services/image-editor/editorProjectClient.test.ts src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts`、`cargo test -p shared-contracts editor_video --manifest-path server-rs/Cargo.toml`、`cargo test -p api-server editor_video --manifest-path server-rs/Cargo.toml`，并执行 `npm run typecheck`、`npm run check:encoding`。
- 关联文档：`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`。
## 2026-06-18 图片画布生成音乐入口作为音频图层接入
- 背景：图片画布底部生成工具需要补齐游戏音效和游戏背景音乐生成，既要复用现有 Lovart 式画布生成器快照、占位避让和持久化，又不能把音频能力并入图片素材库或视觉小说专用音频开关。
docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md
+2 -2
View File
File diff suppressed because one or more lines are too long
docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md
+5 -5
View File
@@ -94,7 +94,7 @@
- 待生成占位的空白样式按生成类型区分：视频使用视频图标和视频角标，角色形象使用角色图标和角色角标，角色动作使用角色动作图标和动作角标，音效使用音效图标和音效角标，背景音乐使用音乐图标和背景音乐角标。
- 待生成占位的图标语义必须与底部入口或触发入口保持一致：生成图片用图片图标，生成规范用规范图标，生成角色形象用角色图标，生成图标素材用图标网格，生成 UI 设计图用应用窗口，宣发素材用宣发图标，快速编辑用闪光图标，不能默认全部回退为图片图标。
- 图片类待生成占位尺寸必须与面板当前比例和尺寸同步：普通图片、角色形象、图标素材、UI 设计图按当前 `aspectRatio + imageSize` 计算像素尺寸；生成规范固定为 `16:9 · 2K`，占位为 `2048 x 1152`；宣发素材按 workflow 输出尺寸创建占位。
- 视频待生成占位必须与面板当前比例和清晰度同步：默认 `16:9 · 480p` 为 `854 x 480`，切换到 `720p` 后同步为 `1280 x 720`；调整参数时保持占位中心点不变。
- 视频待生成占位必须与面板当前比例和清晰度同步：默认 `16:9 · 480p` 为 `854 x 480`，切换比例、`720p` 或 `1080p` 后按比例和清晰度重算偶数宽度；调整参数时保持占位中心点不变。
- 面板中用户修改比例、尺寸或清晰度后，已有空白待生成占位立即同步更新 `width / height / originalWidth / originalHeight`，且保持中心点不跳动。
- 快速编辑点击生成后不在原图上播放生成中遮罩，而是立即创建独立 `Quick Edit Generator` 画布生成占位并播放生成中动画；生成成功后结果落在该占位框位置，失败时占位标记失败并恢复快速编辑面板。
@@ -119,7 +119,7 @@
```text
生成图片 / 生成角色形象 / 生成图标素材 / 生成UI设计图 / 宣发素材 / 快速编辑 / 重绘：nanobanana2 为 12 泥点，gpt-image-2 为 20 泥点
生成规范：固定 gpt-image-2，5 泥点
生成视频：seedance2.0-fast 为 480p 每秒 10 / 720p 每秒 20；seedance2.0 为 480p 每秒 12 / 720p 每秒 24；kling3.0 为 480p 每秒 15 / 720p 每秒 30；kling3.0-omni 为 480p 每秒 20 / 720p 每秒 40
生成视频：seedance2.0-fast 为 480p 每秒 10 / 720p 每秒 20，不支持 1080p；seedance2.0 为 480p 每秒 12 / 720p 每秒 24 / 1080p 每秒 48；kling3.0 为 480p 每秒 15 / 720p 每秒 30 / 1080p 每秒 60；kling3.0-omni 为 480p 每秒 20 / 720p 每秒 40 / 1080p 每秒 80
角色动画：480p 每秒 10 泥点，720p 每秒 20 泥点
生成音效：10 泥点
生成背景音乐：5 泥点
@@ -139,7 +139,7 @@
## 生成视频模型与接口
- 底部 `生成视频` 面板提交到 `POST /api/editor/videos/generations`。
- 固定 `16:9`、静音、标准模式；结果作为 `mediaType="video"`、`assetKind="video"` 的视频图层加入画布，并在画布中使用原生 `<video>` 播放控件渲染，不再走图片元素。
- 比例可选 `16:9`、`9:16`、`1:1`、`4:3`、`3:4`、`21:9`；时长为 4 到 15 秒整数 slider；清晰度支持 `480p`、`720p`、`1080p`，但 `seedance2.0-fast` 不支持 `1080p`。静音为单个 toggle，默认有声；`webSearchEnabled` 默认提交为 `true` 但不在前端显示，后端当前不透传未知 Ark 字段。结果作为 `mediaType="video"`、`assetKind="video"` 的视频图层加入画布，并在画布中使用原生 `<video>` 播放控件渲染，不再走图片元素。
- 支持模型：
- `seedance2.0-fast`
- `seedance2.0`
@@ -159,7 +159,7 @@
- 参考音频支持 URL / Base64 / `asset://` / 画板资源路径；格式为 wav、mp3，单个文件不超过 15MB，单个时长 [2, 15] 秒，最多 3 段且总时长不超过 15 秒。
- 前端上传参考视频 / 参考音频必须走 `/api/assets/direct-upload-tickets` 直传 OSS，再 `/api/assets/objects/confirm` 入库；前端状态保存 signed URL 供预览，同时保存 `objectKey` / `assetObjectId`，提交生成时优先使用 `objectKey`。
- 后端接收画板资源 `objectKey` 后统一重新签名为 Ark 可读 URL，再按文档构造 `content`：`image_url` + `role=reference_image`、`video_url` + `role=reference_video`、`audio_url` + `role=reference_audio`。
- 画板生成视频当前固定无声，因此 Ark 请求必须显式带 `generate_audio:false`，不能只依赖前端 `sound=off` 或上游默认值。
- 画板生成视频按静音 toggle 映射 Ark `generate_audio`：`sound=on` 时传 `true`，`sound=off` 时传 `false`，不能只依赖上游默认值。
- Seedance 2.0 文档提示不支持直接上传含真人人脸的参考图 / 视频；当前画板尚未做真人脸授权证明、来源声明或服务端人脸拦截，后续开放真人素材前必须补授权/来源确认链路。
## 验收
@@ -188,4 +188,4 @@
- 生成游戏背景音乐面板右下角固定模型胶囊显示 `Suno` 并紧贴生成按钮；`make_instrumental` 不在 UI 中展示。
- 生成视频结果以视频图层加入画布，画布媒体元素标记为 `画布视频：生成视频 N`。
- 生成器输入、参数、参考图和占位框在刷新后仍存在；已生成对象的生成器面板继续跟随成品图层。
- Seedance 2.0 参考视频 / 音频上传不产生 `data:video/*` / 大体积音频 JSON；提交请求优先携带 `objectKey`，后端 Ark body 含 `generate_audio:false` 且不含未确认的 `mode` 字段。
- Seedance 2.0 参考视频 / 音频上传不产生 `data:video/*` / 大体积音频 JSON；提交请求优先携带 `objectKey`，后端 Ark body 按 `sound` 映射 `generate_audio`，且不含未确认的 `mode` 字段。
server-rs/crates/api-server/src/character_animation_assets.rs
+104 -13
View File
@@ -2504,8 +2504,11 @@ fn normalize_editor_video_request(
if payload.mode.trim() != "std" {
return Err(editor_video_bad_request("mode 只支持 std。"));
}
if payload.sound.trim() != "off" {
return Err(editor_video_bad_request("sound 只支持 off。"));
let generate_audio = normalize_editor_video_sound(payload.sound.as_str())?;
if model == EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST && resolution == "1080p" {
return Err(editor_video_bad_request(
"seedance2.0-fast 不支持 1080p 清晰度。",
));
}
let expected_price = crate::editor_generation_config::editor_video_model_generation_mud_points(
Some(model),
@@ -2532,6 +2535,7 @@ fn normalize_editor_video_request(
aspect_ratio: aspect_ratio.to_string(),
resolution: resolution.to_string(),
duration_seconds,
generate_audio,
price_mud_points: expected_price,
width,
height,
@@ -2552,8 +2556,7 @@ fn build_editor_video_ark_request_body(
"resolution": request.resolution,
"ratio": request.aspect_ratio,
"duration": request.duration_seconds,
// 中文注释：Seedance 2.0 文档里声音生成默认为开启；画板当前视频入口固定无声，必须显式关闭。
"generate_audio": false,
"generate_audio": request.generate_audio,
"watermark": false,
}))
}
@@ -2821,7 +2824,14 @@ fn resolve_editor_video_provider_model(model: &str) -> &str {
fn normalize_editor_video_aspect_ratio(value: &str) -> Result<&'static str, AppError> {
match value.trim() {
"16:9" => Ok("16:9"),
_ => Err(editor_video_bad_request("aspectRatio 只支持 16:9。")),
"9:16" => Ok("9:16"),
"1:1" => Ok("1:1"),
"4:3" => Ok("4:3"),
"3:4" => Ok("3:4"),
"21:9" => Ok("21:9"),
_ => Err(editor_video_bad_request(
"aspectRatio 只支持 16:9、9:16、1:1、4:3、3:4 或 21:9。",
)),
}
}
@@ -2829,22 +2839,48 @@ fn normalize_editor_video_resolution(value: &str) -> Result<&'static str, AppErr
match value.trim() {
"480p" => Ok("480p"),
"720p" => Ok("720p"),
_ => Err(editor_video_bad_request("resolution 只支持 480p 或 720p。")),
"1080p" => Ok("1080p"),
_ => Err(editor_video_bad_request(
"resolution 只支持 480p、720p 或 1080p。",
)),
}
}
fn normalize_editor_video_duration(value: u32) -> Result<u32, AppError> {
match value {
4 | 5 => Ok(value),
_ => Err(editor_video_bad_request("durationSeconds 只支持 4 或 5。")),
if (4..=15).contains(&value) {
Ok(value)
} else {
Err(editor_video_bad_request(
"durationSeconds 只支持 4 到 15 秒的整数。",
))
}
}
fn normalize_editor_video_sound(value: &str) -> Result<bool, AppError> {
match value.trim() {
"on" => Ok(true),
"off" => Ok(false),
_ => Err(editor_video_bad_request("sound 只支持 on 或 off。")),
}
}
fn resolve_editor_video_size(aspect_ratio: &str, resolution: &str) -> (u32, u32) {
match (aspect_ratio, resolution) {
("16:9", "720p") => (1280, 720),
_ => (854, 480),
}
let height = match resolution {
"1080p" => 1080,
"720p" => 720,
_ => 480,
};
let (ratio_width, ratio_height) = match aspect_ratio {
"9:16" => (9, 16),
"1:1" => (1, 1),
"4:3" => (4, 3),
"3:4" => (3, 4),
"21:9" => (21, 9),
_ => (16, 9),
};
let width = ((height * ratio_width) + ratio_height / 2) / ratio_height;
let even_width = if width % 2 == 0 { width } else { width + 1 };
(even_width, height)
}
fn require_editor_video_settings(
@@ -4591,6 +4627,7 @@ struct NormalizedEditorVideoRequest {
aspect_ratio: String,
resolution: String,
duration_seconds: u32,
generate_audio: bool,
price_mud_points: u32,
width: u32,
height: u32,
@@ -4946,6 +4983,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 100,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
@@ -4971,6 +5009,7 @@ mod tests {
resolution: "720p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 80,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
@@ -4993,6 +5032,7 @@ mod tests {
resolution: "720p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 96,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
@@ -5005,6 +5045,51 @@ mod tests {
assert_ne!(normalized.provider_model, CHARACTER_ANIMATION_MODEL);
}
#[test]
fn editor_video_supports_extra_ratios_duration_range_and_non_fast_1080p() {
let normalized = normalize_editor_video_request(EditorVideoGenerateRequest {
prompt: "生成竖版电影感镜头。".to_string(),
model: EDITOR_VIDEO_MODEL_SEEDANCE_2.to_string(),
aspect_ratio: "9:16".to_string(),
duration_seconds: 15,
resolution: "1080p".to_string(),
mode: "std".to_string(),
sound: "on".to_string(),
web_search_enabled: true,
price_mud_points: 720,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
})
.expect("seedance standard 1080p request should normalize");
assert_eq!(normalized.aspect_ratio, "9:16");
assert_eq!(normalized.width, 608);
assert_eq!(normalized.height, 1080);
assert!(normalized.generate_audio);
}
#[test]
fn editor_video_rejects_seedance_fast_1080p() {
let error = normalize_editor_video_request(EditorVideoGenerateRequest {
prompt: "快速生成 1080p 镜头。".to_string(),
model: EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST.to_string(),
aspect_ratio: "16:9".to_string(),
duration_seconds: 4,
resolution: "1080p".to_string(),
mode: "std".to_string(),
sound: "on".to_string(),
web_search_enabled: true,
price_mud_points: 160,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
reference_audio_srcs: Vec::new(),
})
.expect_err("seedance fast should reject 1080p");
assert!(error.body_text().contains("seedance2.0-fast 不支持 1080p"));
}
#[test]
fn editor_video_normalizes_seedance_multimodal_references() {
let normalized = normalize_editor_video_request(EditorVideoGenerateRequest {
@@ -5015,6 +5100,7 @@ mod tests {
resolution: "720p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 100,
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
reference_video_srcs: vec![
@@ -5048,6 +5134,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 40,
reference_image_srcs: Vec::new(),
reference_video_srcs: vec!["data:video/mp4;base64,video".to_string()],
@@ -5069,6 +5156,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 40,
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
reference_video_srcs: vec!["https://assets.example.com/reference.mp4".to_string()],
@@ -5108,6 +5196,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 40,
reference_image_srcs: Vec::new(),
reference_video_srcs: vec![
@@ -5145,6 +5234,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 48,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
@@ -5200,6 +5290,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 40,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
server-rs/crates/api-server/src/editor_generation_config.rs
+10 -8
View File
@@ -12,6 +12,7 @@ const EDITOR_SPEC_MODEL_GPT_IMAGE_2_MUD_POINTS: u32 = 5;
pub(crate) const EDITOR_VIDEO_GENERATION_480P_MUD_POINTS_PER_SECOND: u32 = 10;
pub(crate) const EDITOR_VIDEO_GENERATION_720P_MUD_POINTS_PER_SECOND: u32 = 20;
pub(crate) const EDITOR_VIDEO_GENERATION_1080P_MUD_POINTS_PER_SECOND: u32 = 40;
pub(crate) const EDITOR_CHARACTER_ANIMATION_480P_MUD_POINTS_PER_SECOND: u32 = 10;
pub(crate) const EDITOR_CHARACTER_ANIMATION_720P_MUD_POINTS_PER_SECOND: u32 = 20;
@@ -72,19 +73,20 @@ fn editor_video_model_resolution_mud_points_per_second(
) => model.unwrap_or_default().trim(),
_ => EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST,
};
let (rate_480p, rate_720p) = match normalized_model {
EDITOR_VIDEO_MODEL_SEEDANCE_2 => (12, 24),
EDITOR_VIDEO_MODEL_KLING_3 => (15, 30),
EDITOR_VIDEO_MODEL_KLING_3_OMNI => (20, 40),
let (rate_480p, rate_720p, rate_1080p) = match normalized_model {
EDITOR_VIDEO_MODEL_SEEDANCE_2 => (12, 24, 48),
EDITOR_VIDEO_MODEL_KLING_3 => (15, 30, 60),
EDITOR_VIDEO_MODEL_KLING_3_OMNI => (20, 40, 80),
_ => (
EDITOR_VIDEO_GENERATION_480P_MUD_POINTS_PER_SECOND,
EDITOR_VIDEO_GENERATION_720P_MUD_POINTS_PER_SECOND,
EDITOR_VIDEO_GENERATION_1080P_MUD_POINTS_PER_SECOND,
),
};
if resolution == "720p" {
rate_720p
} else {
rate_480p
match resolution {
"1080p" => rate_1080p,
"720p" => rate_720p,
_ => rate_480p,
}
}
server-rs/crates/shared-contracts/src/assets.rs
+9
View File
@@ -356,6 +356,8 @@ pub struct EditorVideoGenerateRequest {
pub resolution: String,
pub mode: String,
pub sound: String,
#[serde(default = "default_editor_video_web_search_enabled")]
pub web_search_enabled: bool,
pub price_mud_points: u32,
#[serde(default)]
pub reference_image_srcs: Vec<String>,
@@ -365,6 +367,10 @@ pub struct EditorVideoGenerateRequest {
pub reference_audio_srcs: Vec<String>,
}
fn default_editor_video_web_search_enabled() -> bool {
true
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
pub struct EditorVideoGenerateResponse {
@@ -1048,6 +1054,7 @@ mod tests {
resolution: "480p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 100,
reference_image_srcs: Vec::new(),
reference_video_srcs: Vec::new(),
@@ -1059,6 +1066,7 @@ mod tests {
assert_eq!(payload["durationSeconds"], json!(5));
assert_eq!(payload["priceMudPoints"], json!(100));
assert_eq!(payload["model"], json!("kling3.0-omni"));
assert_eq!(payload["webSearchEnabled"], json!(true));
}
#[test]
@@ -1071,6 +1079,7 @@ mod tests {
resolution: "720p".to_string(),
mode: "std".to_string(),
sound: "off".to_string(),
web_search_enabled: true,
price_mud_points: 100,
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
reference_video_srcs: vec![
src/components/image-editor/ImageCanvasEditorModel.ts
+25 -6
View File
@@ -419,18 +419,26 @@ export function hydrateCanvasGenerationDialog(
typeof snapshot.videoModel === 'string'
? snapshot.videoModel
: undefined,
videoAspectRatio: stringOrUndefined(snapshot.videoAspectRatio),
videoAspectRatio: videoAspectRatioOrUndefined(snapshot.videoAspectRatio),
videoResolution:
snapshot.videoResolution === '480p' || snapshot.videoResolution === '720p'
snapshot.videoResolution === '480p' ||
snapshot.videoResolution === '720p' ||
snapshot.videoResolution === '1080p'
? snapshot.videoResolution
: undefined,
videoDurationSeconds:
snapshot.videoDurationSeconds === 4 ||
snapshot.videoDurationSeconds === 5
? snapshot.videoDurationSeconds
typeof snapshot.videoDurationSeconds === 'number'
? Math.min(15, Math.max(4, Math.round(snapshot.videoDurationSeconds)))
: undefined,
videoMode: snapshot.videoMode === 'std' ? 'std' : undefined,
videoSound: snapshot.videoSound === 'off' ? 'off' : undefined,
videoSound:
snapshot.videoSound === 'on' || snapshot.videoSound === 'off'
? snapshot.videoSound
: undefined,
videoWebSearchEnabled:
typeof snapshot.videoWebSearchEnabled === 'boolean'
? snapshot.videoWebSearchEnabled
: undefined,
characterAnimationResolution:
snapshot.characterAnimationResolution === '480p' ||
snapshot.characterAnimationResolution === '720p'
@@ -697,6 +705,17 @@ export function stringOrUndefined(value: unknown) {
return typeof value === 'string' && value.trim() ? value : undefined;
}
function videoAspectRatioOrUndefined(value: unknown) {
return value === '16:9' ||
value === '9:16' ||
value === '1:1' ||
value === '4:3' ||
value === '3:4' ||
value === '21:9'
? value
: undefined;
}
function hydrateCharacterAnimationResult(
value: unknown,
): EditorCharacterAnimationGenerationResult | undefined {
src/components/image-editor/ImageCanvasEditorTypes.ts
+8 -4
View File
@@ -4,7 +4,10 @@ import type {
EditorCharacterAnimationGenerationResult,
EditorCharacterAnimationRatio,
EditorCharacterAnimationResolution,
EditorVideoAspectRatio,
EditorVideoModel,
EditorVideoResolution,
EditorVideoSoundMode,
} from '../../services/image-editor/editorProjectClient';
export type CanvasSourceType = 'uploaded' | 'generated' | 'mock_generated';
@@ -201,11 +204,12 @@ export type GenerateDialogState = {
uiDesignSpecReference?: CharacterReferenceImage | null;
imageModel?: string;
videoModel?: EditorVideoModel;
videoAspectRatio?: string;
videoResolution?: '480p' | '720p';
videoDurationSeconds?: 4 | 5;
videoAspectRatio?: EditorVideoAspectRatio;
videoResolution?: EditorVideoResolution;
videoDurationSeconds?: number;
videoMode?: 'std';
videoSound?: 'off';
videoSound?: EditorVideoSoundMode;
videoWebSearchEnabled?: boolean;
characterAnimationResolution?: EditorCharacterAnimationResolution;
characterAnimationRatio?: EditorCharacterAnimationRatio;
characterAnimationFrameCount?: EditorCharacterAnimationFrameCount;
src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx
+18 -4
View File
@@ -491,7 +491,9 @@ describe('ImageCanvasGenerationComposerView', () => {
const panel = screen.getByRole('dialog', { name: '生成视频' });
expect(
within(panel).getByRole('button', { name: '视频参数 16:9 · 4秒 · 480p' }),
within(panel).getByRole('button', {
name: '视频参数 16:9 · 4秒 · 480p',
}),
).toBeTruthy();
expect(
within(panel).getByRole('button', { name: '模型 Seedance 2.0 Fast' }),
@@ -501,13 +503,18 @@ describe('ImageCanvasGenerationComposerView', () => {
);
fireEvent.click(
within(panel).getByRole('button', { name: '视频参数 16:9 · 4秒 · 480p' }),
within(panel).getByRole('button', {
name: '视频参数 16:9 · 4秒 · 480p',
}),
);
const paramsPanel = screen.getByRole('menu', {
name: '视频参数选项',
});
fireEvent.click(within(paramsPanel).getByRole('button', { name: '时长 5秒' }));
fireEvent.change(within(paramsPanel).getByRole('slider', { name: '视频时长' }), {
target: { value: '5' },
});
fireEvent.click(within(paramsPanel).getByRole('button', { name: '清晰度 720p' }));
fireEvent.click(within(paramsPanel).getByRole('button', { name: '静音' }));
expect(
screen.getByRole('menu', { name: '视频参数选项' }),
@@ -517,7 +524,14 @@ describe('ImageCanvasGenerationComposerView', () => {
expect(screen.getByLabelText('当前视频占位').textContent).toBe(
'-160:0:1280:720:1280:720',
);
expect(screen.getByRole('button', { name: '视频参数 16:9 · 5秒 · 720p' })).toBeTruthy();
expect(
screen.getByRole('button', { name: '视频参数 16:9 · 5秒 · 720p' }),
).toBeTruthy();
expect(
within(paramsPanel)
.getByRole('button', { name: '清晰度 1080p' })
.hasAttribute('disabled'),
).toBe(true);
expect(screen.getByRole('button', { name: '生成视频' }).textContent).toBe(
'生成100泥点',
);
src/components/image-editor/ImageCanvasGenerationComposerView.tsx
+99 -51
View File
@@ -42,10 +42,16 @@ import {
DEFAULT_BACKGROUND_MUSIC_MODEL,
DEFAULT_SOUND_EFFECT_DURATION_SECONDS,
DEFAULT_SOUND_EFFECT_MODEL,
DEFAULT_VIDEO_ASPECT_RATIO,
DEFAULT_VIDEO_DURATION_SECONDS,
DEFAULT_VIDEO_SOUND,
DEFAULT_VIDEO_WEB_SEARCH_ENABLED,
EDITOR_VIDEO_ASPECT_RATIO_OPTIONS,
EDITOR_SOUND_EFFECT_MODEL_OPTIONS,
SOUND_EFFECT_DURATION_OPTIONS,
EDITOR_VIDEO_DURATION_OPTIONS,
EDITOR_VIDEO_MODEL_OPTIONS,
EDITOR_VIDEO_RESOLUTION_OPTIONS,
VIDEO_MODEL_SEEDANCE_2_FAST,
resizeGenerationPlaceholderToVideoSelection,
SPEC_TYPE_LABEL,
} from './ImageCanvasGenerationModel';
@@ -277,8 +283,14 @@ function ImageCanvasVideoGenerationComposerView({
const [openPanel, setOpenPanel] = useState<'params' | 'model' | null>(null);
const paramsButtonRef = useRef<HTMLButtonElement | null>(null);
const modelButtonRef = useRef<HTMLButtonElement | null>(null);
const normalizeVideoDuration = (value: number) =>
Math.min(15, Math.max(4, Math.round(value)));
const resolution = dialog.videoResolution ?? '480p';
const durationSeconds = dialog.videoDurationSeconds ?? 4;
const durationSeconds = normalizeVideoDuration(
dialog.videoDurationSeconds ?? DEFAULT_VIDEO_DURATION_SECONDS,
);
const soundMode = dialog.videoSound ?? DEFAULT_VIDEO_SOUND;
const aspectRatio = dialog.videoAspectRatio ?? DEFAULT_VIDEO_ASPECT_RATIO;
const currentModel =
EDITOR_VIDEO_MODEL_OPTIONS.find(
(item) => item.value === dialog.videoModel,
@@ -319,6 +331,10 @@ function ImageCanvasVideoGenerationComposerView({
const updateVideoModel = (model: GenerateDialogState['videoModel']) => {
updateVideoDialog({
videoModel: model,
videoResolution:
model === VIDEO_MODEL_SEEDANCE_2_FAST && dialog.videoResolution === '1080p'
? '720p'
: dialog.videoResolution,
...(isSeedanceVideoModel(model) ? {} : { generationReferences: [] }),
});
};
@@ -384,7 +400,7 @@ function ImageCanvasVideoGenerationComposerView({
<PlatformInlineOptionButton
ref={paramsButtonRef}
className="image-canvas-editor__option-cluster image-canvas-editor__option-cluster--dimensions"
aria-label={`视频参数 16:9 · ${durationSeconds}秒 · ${resolution}`}
aria-label={`视频参数 ${aspectRatio} · ${durationSeconds}秒 · ${resolution}`}
aria-expanded={openPanel === 'params'}
disabled={isGenerating}
trailingIcon={<ChevronDown className="h-3 w-3" />}
@@ -394,7 +410,8 @@ function ImageCanvasVideoGenerationComposerView({
)
}
>
16:9 · {durationSeconds}秒 · {resolution}
{aspectRatio} · {durationSeconds}秒 · {resolution} ·{' '}
{soundMode === 'off' ? '静音' : '有声'}
</PlatformInlineOptionButton>
{openPanel === 'params'
? renderEditorPortal(
@@ -413,46 +430,46 @@ function ImageCanvasVideoGenerationComposerView({
<span className="image-canvas-editor__option-popover-title">
比例
</span>
<div className="image-canvas-editor__option-popover-items image-canvas-editor__option-popover-items--card">
<VideoOptionChoice
selected
disabled={isGenerating}
className="image-canvas-editor__option-popover-choice--ratio"
ariaLabel="比例 16:9"
>
<span
className="image-canvas-editor__ratio-wireframe"
data-ratio="16:9"
aria-hidden="true"
/>
<span>16:9</span>
</VideoOptionChoice>
<div className="image-canvas-editor__option-popover-items">
{EDITOR_VIDEO_ASPECT_RATIO_OPTIONS.map((option) => (
<VideoOptionChoice
key={option}
selected={aspectRatio === option}
disabled={isGenerating}
ariaLabel={`比例 ${option}`}
onClick={() =>
updateVideoDialog({
videoAspectRatio: option,
})
}
>
{option}
</VideoOptionChoice>
))}
</div>
</section>
<section className="image-canvas-editor__option-popover-section">
<span className="image-canvas-editor__option-popover-title">
时长
</span>
<div className="image-canvas-editor__option-popover-items">
{EDITOR_VIDEO_DURATION_OPTIONS.map((option) => {
const nextDuration =
Number(option.value) === 5 ? 5 : 4;
return (
<VideoOptionChoice
key={option.value}
selected={durationSeconds === nextDuration}
disabled={isGenerating}
ariaLabel={`时长 ${option.label}`}
onClick={() =>
updateVideoDialog({
videoDurationSeconds: nextDuration,
})
}
>
{option.label}
</VideoOptionChoice>
);
})}
<div className="image-canvas-editor__video-duration-slider">
<input
type="range"
min={4}
max={15}
step={1}
value={durationSeconds}
disabled={isGenerating}
aria-label="视频时长"
onChange={(event) =>
updateVideoDialog({
videoDurationSeconds: normalizeVideoDuration(
Number(event.target.value),
),
})
}
/>
<span>{durationSeconds}秒</span>
</div>
</section>
<section className="image-canvas-editor__option-popover-section">
@@ -460,19 +477,50 @@ function ImageCanvasVideoGenerationComposerView({
清晰度
</span>
<div className="image-canvas-editor__option-popover-items">
{(['480p', '720p'] as const).map((option) => (
<VideoOptionChoice
key={option}
selected={resolution === option}
disabled={isGenerating}
ariaLabel={`清晰度 ${option}`}
onClick={() =>
updateVideoDialog({ videoResolution: option })
}
>
{option}
</VideoOptionChoice>
))}
{EDITOR_VIDEO_RESOLUTION_OPTIONS.map((option) => {
const available =
currentModel.value !== VIDEO_MODEL_SEEDANCE_2_FAST ||
option !== '1080p';
return (
<VideoOptionChoice
key={option}
selected={resolution === option}
disabled={isGenerating || !available}
ariaLabel={`清晰度 ${option}`}
onClick={() =>
available
? updateVideoDialog({
videoResolution: option,
})
: undefined
}
>
{option}
</VideoOptionChoice>
);
})}
</div>
</section>
<section className="image-canvas-editor__option-popover-section">
<span className="image-canvas-editor__option-popover-title">
声音
</span>
<div className="image-canvas-editor__video-toggle-row">
<button
type="button"
className="image-canvas-editor__video-toggle"
aria-pressed={soundMode === 'off'}
disabled={isGenerating}
onClick={() =>
updateVideoDialog({
videoSound:
soundMode === 'off' ? 'on' : 'off',
})
}
>
<span aria-hidden="true" />
<strong>静音</strong>
</button>
</div>
</section>
</div>
src/components/image-editor/ImageCanvasGenerationDialogModel.test.ts
+2 -1
View File
@@ -187,7 +187,8 @@ describe('ImageCanvasGenerationDialogModel', () => {
videoResolution: '480p',
videoDurationSeconds: 4,
videoMode: 'std',
videoSound: 'off',
videoSound: 'on',
videoWebSearchEnabled: true,
placeholder: {
x: 53,
y: 120,
src/components/image-editor/ImageCanvasGenerationDialogModel.ts
+9 -4
View File
@@ -24,7 +24,11 @@ import {
DEFAULT_PUBLICATION_GAME_INFO,
DEFAULT_SOUND_EFFECT_MODEL,
DEFAULT_SPEC_FORM_VALUES,
DEFAULT_VIDEO_ASPECT_RATIO,
DEFAULT_VIDEO_DURATION_SECONDS,
DEFAULT_VIDEO_MODEL,
DEFAULT_VIDEO_SOUND,
DEFAULT_VIDEO_WEB_SEARCH_ENABLED,
EDITOR_IMAGE_DIMENSION_OPTIONS,
ICON_DESCRIPTION_LIMIT,
normalizeEditorImageModel,
@@ -320,7 +324,7 @@ export function createVideoGenerationDialogDraft({
}): Omit<CanvasGenerationDialogState, 'id'> {
const worldCenter = getViewportWorldCenter({ canvasSize, viewport });
const placeholderSize = resolveEditorVideoGenerationPixelSize({
aspectRatio: '16:9',
aspectRatio: DEFAULT_VIDEO_ASPECT_RATIO,
resolution: '480p',
});
return {
@@ -330,11 +334,12 @@ export function createVideoGenerationDialogDraft({
composerOpen: true,
generationReferences: [],
videoModel: DEFAULT_VIDEO_MODEL,
videoAspectRatio: '16:9',
videoAspectRatio: DEFAULT_VIDEO_ASPECT_RATIO,
videoResolution: '480p',
videoDurationSeconds: 4,
videoDurationSeconds: DEFAULT_VIDEO_DURATION_SECONDS,
videoMode: 'std',
videoSound: 'off',
videoSound: DEFAULT_VIDEO_SOUND,
videoWebSearchEnabled: DEFAULT_VIDEO_WEB_SEARCH_ENABLED,
placeholder: {
x: worldCenter.x - placeholderSize.width / 2,
y: worldCenter.y - placeholderSize.height / 2,
src/components/image-editor/ImageCanvasGenerationModel.ts
+22 -5
View File
@@ -231,9 +231,10 @@ export function resolveEditorVideoGenerationPixelSize({
resolution,
}: {
aspectRatio: string | null | undefined;
resolution: '480p' | '720p' | string | null | undefined;
resolution: '480p' | '720p' | '1080p' | string | null | undefined;
}): EditorImageFrameSize {
const height = resolution === '720p' ? 720 : 480;
const height =
resolution === '1080p' ? 1080 : resolution === '720p' ? 720 : 480;
return {
width: resolveEvenVideoWidth(height, aspectRatio),
height,
@@ -285,18 +286,22 @@ export const DEFAULT_BACKGROUND_MUSIC_MODEL = BACKGROUND_MUSIC_MODEL_SUNO;
const EDITOR_VIDEO_RESOLUTION_RATE_CONFIG = {
'480p': EDITOR_GENERATION_MUD_POINT_CONFIG.videoPerSecond['480p'],
'720p': EDITOR_GENERATION_MUD_POINT_CONFIG.videoPerSecond['720p'],
'1080p': EDITOR_GENERATION_MUD_POINT_CONFIG.videoPerSecond['720p'] * 2,
} as const;
const EDITOR_VIDEO_STANDARD_RESOLUTION_RATE_CONFIG = {
'480p': 12,
'720p': 24,
'1080p': 48,
} as const;
const EDITOR_VIDEO_KLING_RESOLUTION_RATE_CONFIG = {
'480p': 15,
'720p': 30,
'1080p': 60,
} as const;
const EDITOR_VIDEO_KLING_OMNI_RESOLUTION_RATE_CONFIG = {
'480p': 20,
'720p': 40,
'1080p': 80,
} as const;
export const EDITOR_VIDEO_MODEL_MUD_POINT_CONFIG = {
[VIDEO_MODEL_SEEDANCE_2_FAST]: EDITOR_VIDEO_RESOLUTION_RATE_CONFIG,
@@ -365,9 +370,21 @@ export const EDITOR_VIDEO_MODEL_OPTIONS = [
{ label: 'Kling 3.0 Omni', value: VIDEO_MODEL_KLING_3_OMNI },
] as const;
export const EDITOR_VIDEO_DURATION_OPTIONS = [
{ label: '4秒', value: '4' },
{ label: '5秒', value: '5' },
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
] as const;
export const EDITOR_VIDEO_ASPECT_RATIO_OPTIONS = [
'16:9',
'9:16',
'1:1',
'4:3',
'3:4',
'21:9',
] as const;
export const EDITOR_VIDEO_RESOLUTION_OPTIONS = ['480p', '720p', '1080p'] as const;
export const DEFAULT_VIDEO_ASPECT_RATIO = '16:9';
export const DEFAULT_VIDEO_DURATION_SECONDS = 4;
export const DEFAULT_VIDEO_SOUND = 'on';
export const DEFAULT_VIDEO_WEB_SEARCH_ENABLED = true;
export const DEFAULT_SPEC_FORM_VALUES: Record<
SpecGenerationType,
@@ -603,7 +620,7 @@ export function calculateCharacterAnimationPrice(
export function calculateEditorVideoPrice(
model: string | null | undefined,
resolution: '480p' | '720p',
resolution: '480p' | '720p' | '1080p',
durationSeconds: number,
) {
const normalizedModel = (model ?? DEFAULT_VIDEO_MODEL).trim();
src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts
+15 -3
View File
@@ -37,7 +37,11 @@ import {
DEFAULT_SOUND_EFFECT_DURATION_SECONDS,
DEFAULT_SOUND_EFFECT_MODEL,
DEFAULT_SPEC_FORM_VALUES,
DEFAULT_VIDEO_ASPECT_RATIO,
DEFAULT_VIDEO_DURATION_SECONDS,
DEFAULT_VIDEO_MODEL,
DEFAULT_VIDEO_SOUND,
DEFAULT_VIDEO_WEB_SEARCH_ENABLED,
ICON_DESCRIPTION_LIMIT,
normalizeEditorImageModel,
resolveCharacterAnimationSourceImageSrc,
@@ -346,8 +350,15 @@ export function buildImageGenerationSubmissionPlan({
if (dialog.mode === 'video') {
const resolution = dialog.videoResolution ?? '480p';
const durationSeconds = dialog.videoDurationSeconds ?? 4;
const durationSeconds =
typeof dialog.videoDurationSeconds === 'number'
? Math.min(15, Math.max(4, Math.round(dialog.videoDurationSeconds)))
: DEFAULT_VIDEO_DURATION_SECONDS;
const model = dialog.videoModel ?? DEFAULT_VIDEO_MODEL;
const aspectRatio = dialog.videoAspectRatio ?? DEFAULT_VIDEO_ASPECT_RATIO;
const sound = dialog.videoSound ?? DEFAULT_VIDEO_SOUND;
const webSearchEnabled =
dialog.videoWebSearchEnabled ?? DEFAULT_VIDEO_WEB_SEARCH_ENABLED;
const seedanceReferences = isSeedanceVideoModel(model)
? buildSeedanceVideoReferenceInput(dialog.generationReferences ?? [])
: {
@@ -361,11 +372,12 @@ export function buildImageGenerationSubmissionPlan({
input: {
prompt: normalizedPrompt,
model,
aspectRatio: '16:9',
aspectRatio,
durationSeconds,
resolution,
mode: 'std',
sound: 'off',
sound,
webSearchEnabled,
priceMudPoints: calculateEditorVideoPrice(
model,
resolution,
src/index.css
+72
View File
@@ -6311,6 +6311,78 @@ button.image-canvas-editor__reference-chip:disabled {
accent-color: #4f46e5;
}
.image-canvas-editor__video-duration-slider {
display: grid;
grid-template-columns: minmax(7rem, 1fr) 3.6rem;
align-items: center;
gap: 0.56rem;
}
.image-canvas-editor__video-duration-slider input[type='range'] {
width: 100%;
accent-color: #111827;
}
.image-canvas-editor__video-duration-slider span {
color: #111827;
font-size: 0.82rem;
font-weight: 760;
text-align: right;
}
.image-canvas-editor__video-toggle-row {
display: flex;
flex-wrap: wrap;
gap: 0.46rem;
}
.image-canvas-editor__video-toggle {
display: inline-flex;
min-height: 2.25rem;
align-items: center;
gap: 0.48rem;
border: 1px solid rgba(15, 23, 42, 0.1);
border-radius: 999px;
background: #ffffff;
padding: 0 0.78rem;
color: #111827;
font-size: 0.82rem;
font-weight: 760;
}
.image-canvas-editor__video-toggle span {
position: relative;
width: 1.7rem;
height: 0.96rem;
border-radius: 999px;
background: #cbd5e1;
}
.image-canvas-editor__video-toggle span::after {
position: absolute;
top: 0.16rem;
left: 0.16rem;
width: 0.64rem;
height: 0.64rem;
border-radius: 999px;
background: #ffffff;
content: '';
transition: transform 0.16s ease;
}
.image-canvas-editor__video-toggle[aria-pressed='true'] {
border-color: #202124;
box-shadow: inset 0 0 0 1px #202124;
}
.image-canvas-editor__video-toggle[aria-pressed='true'] span {
background: #111827;
}
.image-canvas-editor__video-toggle[aria-pressed='true'] span::after {
transform: translateX(0.74rem);
}
.image-canvas-editor__audio-bpm-control input[type='number'] {
min-width: 0;
border: 1px solid rgba(15, 23, 42, 0.1);
src/services/image-editor/editorProjectClient.test.ts
+2
View File
@@ -963,6 +963,7 @@ describe('editorProjectClient', () => {
resolution: '480p',
mode: 'std',
sound: 'off',
webSearchEnabled: true,
priceMudPoints: 100,
}),
}),
@@ -1022,6 +1023,7 @@ describe('editorProjectClient', () => {
resolution: '720p',
mode: 'std',
sound: 'off',
webSearchEnabled: true,
priceMudPoints: 100,
referenceImageSrcs: ['data:image/png;base64,image'],
referenceVideoSrcs: [
src/services/image-editor/editorProjectClient.ts
+10 -4
View File
@@ -259,14 +259,19 @@ export type EditorVideoModel =
| 'veo3.1'
| 'veo3.1-fast';
export type EditorVideoResolution = '480p' | '720p' | '1080p';
export type EditorVideoSoundMode = 'on' | 'off';
export type EditorVideoAspectRatio = '16:9' | '9:16' | '1:1' | '4:3' | '3:4' | '21:9';
export type EditorVideoGenerationInput = {
prompt: string;
model: EditorVideoModel;
aspectRatio: '16:9';
durationSeconds: 4 | 5;
resolution: '480p' | '720p';
aspectRatio: EditorVideoAspectRatio;
durationSeconds: number;
resolution: EditorVideoResolution;
mode: 'std';
sound: 'off';
sound: EditorVideoSoundMode;
webSearchEnabled?: boolean;
priceMudPoints: number;
referenceImageSrcs?: string[];
referenceVideoSrcs?: string[];
@@ -739,6 +744,7 @@ export async function generateEditorVideo(input: EditorVideoGenerationInput) {
resolution: input.resolution,
mode: input.mode,
sound: input.sound,
webSearchEnabled: input.webSearchEnabled ?? true,
priceMudPoints: input.priceMudPoints,
...(input.referenceImageSrcs?.length
? { referenceImageSrcs: input.referenceImageSrcs }