收紧编辑器媒体引用契约
禁止当前画布与外部编辑器接口接收 Data URL 和 Blob URL 生成请求统一先上传 OSS 并使用稳定资源引用 角色动画和视频签名前校验资源归属并支持资源与素材 ID 同步前后端 256KB 引用限制及相关文档测试
This commit is contained in:
@@ -240,21 +240,21 @@
|
||||
- 验证:`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/ImageCanvasGenerationComposerView.test.tsx src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx --reporter verbose`。
|
||||
- 关联:`src/components/image-editor/ImageCanvasEditorModel.ts`、`src/components/image-editor/ImageCanvasPublicationMaterialsDemoPanelView.tsx`、`src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx`。
|
||||
|
||||
## 图片画布快速编辑不要直接提交普通图片 URL
|
||||
## 图片画布生成请求不要直接提交临时媒体源
|
||||
|
||||
- 现象:图片画布快速编辑站内示例图、历史 generated 图或 OSS generated 图时,后端返回 `修改图片参考图必须是图片 Data URL。`。
|
||||
- 原因:快速编辑直接把图层 `src` 塞进 `/api/editor/images/generations` 的 `referenceImageSrcs`;默认示例图和部分持久化图层的 `src` 是 `/creation-type-references/*.webp`、`/generated-*` 或 OSS URL,而 `api-server` 的编辑参考图解析只接收 `data:image/*;base64,...`。
|
||||
- 处理:前端统一通过 `resolveEditorImageReferenceDataUrl(...)` 在提交前读取图片字节并转成图片 Data URL;Data URL 原样透传,`/generated-*` 和 generated OSS URL 先走 `/api/assets/read-url` 换签后由浏览器直读 OSS,直读失败时才 fallback 到 `/api/assets/read-bytes`,普通 public 路径直接 fetch。
|
||||
- 验证:`npm run test -- src/services/image-editor/editorImageReference.test.ts src/components/image-editor/ImageCanvasEditorView.test.tsx -t "editorImageReference|converts non-data-url quick edit source images before submitting references"`。
|
||||
- 关联:`src/services/image-editor/editorImageReference.ts`、`src/components/image-editor/ImageCanvasEditorView.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
- 现象:图片画布快速编辑、参考生成、去背景或角色动画提交 Data URL / Blob URL 时,前端或后端返回“必须先上传 OSS”。
|
||||
- 原因:Data URL / Blob URL 体积大且不能作为持久引用;如果写入外部生成队列,请求体会膨胀,worker 也无法稳定复用浏览器临时资源。
|
||||
- 处理:提交前优先复用图层已有 `objectKey`、项目资源 ID 或素材 ID;未登记的本地图片和普通 public 图片路径先通过 `resolveEditorGenerationMediaReference(...)` 读取并上传 OSS,再把稳定引用交给生成接口。Data URL 只允许用于浏览器内压缩、标注等临时处理,不能进入 API 请求、队列载荷或项目持久化。
|
||||
- 验证:`npm run test -- src/services/image-editor/editorProjectClient.test.ts src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx`;`cargo test -p api-server inline_data_url --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联:`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`src/services/image-editor/editorProjectClient.ts`、`server-rs/crates/api-server/src/editor_generation_queue.rs`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 图片编辑器角色动画不要默认提交大图 Data URL
|
||||
## 图片编辑器角色动画必须提交稳定图片引用
|
||||
|
||||
- 现象:图片编辑器里对角色图点击 `生成动画` 后,后端返回 `Failed to buffer the request body: length limit exceeded`,请求还没进入角色动画 handler。
|
||||
- 原因:角色动画生成请求曾把角色图片 `src` 原样作为 `sourceImageSrc` 放进 JSON;角色图如果是较大的 Data URL,会超过 Axum 默认 `2MB` body limit,在 `Json` 提取器阶段被拦截。
|
||||
- 处理:前端在角色图已持久化时优先提交 `objectKey`,只把 Data URL 作为未持久化本地临时图兜底;后端 `/api/editor/character-animations/generations` 单独配置 `12MB` body limit 兼容旧请求,但新链路不应依赖传大图 JSON。
|
||||
- 验证:`npm run test -- src/components/image-editor/ImageCanvasEditorView.test.tsx -t "only exposes character animation"`;`cargo test -p api-server editor_character_animation_accepts_character_image_body_above_default_limit --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联:`src/components/image-editor/ImageCanvasEditorView.tsx`、`server-rs/crates/api-server/src/modules/play_flow.rs`、`server-rs/crates/api-server/src/app.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
|
||||
- 现象:图片编辑器里对尚未上传的角色图点击 `生成动画` 后,前端或后端返回 `sourceImageSrc 必须先上传 OSS`。
|
||||
- 原因:角色动画会进入外部生成队列,浏览器 Data URL / Blob URL 既不适合持久任务,也会放大 JSON 请求体。
|
||||
- 处理:前端统一通过 `resolveEditorGenerationMediaReference(...)` 取得 `objectKey` 或画板资源引用;本地临时角色图必须先上传 OSS。后端在入队前同步拒绝内联媒体,不再通过放宽 body limit 兼容 Data URL。
|
||||
- 验证:`npm run test -- src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx`;`cargo test -p api-server editor_character_animation --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联:`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`server-rs/crates/api-server/src/app.rs`、`docs/【编辑器】画板角色形象生成入口设计-2026-06-15.md`。
|
||||
|
||||
## 图片编辑器角色动画抽帧不要采到视频尾点
|
||||
|
||||
@@ -328,11 +328,11 @@
|
||||
- 验证:`npm run test -- src/services/assetReadUrlService.test.ts --reporter verbose` 应覆盖大量不同 objectKey 同时换签时首批限量放行、后续按间隔派发;发布现场同类页面刷新时,Nginx `GET /api/assets/read-url` 429 应从 `upstream_status=-` / `genarrative_api_rps` 收敛。
|
||||
- 关联:`src/services/assetReadUrlService.ts`、`src/hooks/useResolvedAssetReadUrl.ts`、`src/components/ResolvedAssetImage.tsx`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。
|
||||
|
||||
## 图片编辑器 Seedance 2.0 参考媒体不要提交视频 Data URL
|
||||
## 图片编辑器 Seedance 2.0 参考媒体只提交稳定引用
|
||||
|
||||
- 现象:画板生成视频选择 Seedance 2.0 并上传参考视频后,请求体暴涨、可能返回 `413` 或上游拒绝 `video_url.url`;文档示例或测试如果写 `data:video/mp4;base64,...`,后续实现很容易照抄。
|
||||
- 原因:火山 Seedance 2.0 参考视频只支持公网 URL 或 `asset://` 素材 ID,项目内画板资源应以 `objectKey` 由后端换签;视频不支持 Base64 / `data:video`,且 50MB 视频转 Base64 后会逼近或超过 64MB 请求体上限。参考音频虽然支持 Base64,但也不能单独输入,且大文件同样不应塞进 JSON。
|
||||
- 处理:参考视频 / 音频上传先走 `/api/assets/direct-upload-tickets` 直传 OSS,再 `/api/assets/objects/confirm` 确认;前端保留 signed URL 做预览,提交生成时优先使用 `objectKey`。后端归一化必须拒绝 `data:video/*`,非 Seedance 模型携带参考字段也必须拒绝;Ark body 显式带 `generate_audio:false`。
|
||||
- 原因:画板生成会进入持久队列,Base64 / Data URL / Blob URL 会放大请求和任务 JSON;直接签名客户端给出的 objectKey 又会绕过跨账号素材归属校验。项目资源 ID、素材 ID 和 objectKey 必须先解析到当前 owner 的正式对象,公网 URL 与 `asset://` 才能按供应商契约直接透传。
|
||||
- 处理:本地参考图片 / 视频 / 音频先走 `/api/assets/direct-upload-tickets` 直传 OSS,再 `/api/assets/objects/confirm` 确认;前端保留 signed URL 做预览,提交生成时使用 `objectKey`、项目资源 ID 或素材 ID。后端归一化拒绝全部 `data:*` / `blob:*`,签名 OSS URL 前统一校验 owner;稳定引用字段总长度限制为 `256KB`,非 Seedance 模型携带参考字段也必须拒绝;Ark body 显式带 `generate_audio:false`。
|
||||
- 验证:`npx vitest run src/components/image-editor/useImageCanvasUploadWorkflow.test.tsx src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/services/image-editor/editorReferenceUploadClient.test.ts --reporter verbose`;`cargo test -p api-server editor_video --manifest-path server-rs/Cargo.toml`;`cargo test -p shared-contracts editor_video_request_supports_seedance_multimodal_references --manifest-path server-rs/Cargo.toml`。
|
||||
- 关联:`src/services/image-editor/editorReferenceUploadClient.ts`、`src/components/image-editor/useImageCanvasUploadWorkflow.ts`、`src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts`、`server-rs/crates/api-server/src/character_animation_assets.rs`、`docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`。
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -160,11 +160,11 @@
|
||||
|
||||
- 参考媒体只对 `seedance2.0-fast` / `seedance2.0` 开放;切换到 Kling 等非 Seedance 模型时,前端不提交 `referenceImageSrcs` / `referenceVideoSrcs` / `referenceAudioSrcs`,后端收到非 Seedance 参考字段必须拒绝。
|
||||
- 多模态参考按火山 Seedance 2.0 文档限制:参考图片 0~9 张,参考视频 0~3 个,参考音频 0~3 段;音频不可单独输入,必须至少搭配 1 张参考图片或 1 个参考视频。
|
||||
- 参考图片支持 URL / Base64 / `asset://` / 画板资源路径;格式为 jpeg、png、webp、bmp、tiff、gif、heic、heif,单张小于 30MB,请求体总大小不超过 64MB。
|
||||
- 参考视频只支持 URL / `asset://` / 画板资源路径,不支持 Base64 / `data:video/*`;格式为 mp4、mov,单个文件不超过 50MB,单个时长 [2, 15] 秒,最多 3 个且总时长不超过 15 秒。
|
||||
- 参考音频支持 URL / Base64 / `asset://` / 画板资源路径;格式为 wav、mp3,单个文件不超过 15MB,单个时长 [2, 15] 秒,最多 3 段且总时长不超过 15 秒。
|
||||
- 前端上传参考视频 / 参考音频必须走 `/api/assets/direct-upload-tickets` 直传 OSS,再 `/api/assets/objects/confirm` 入库;前端状态保存 signed URL 供预览,同时保存 `objectKey` / `assetObjectId`,提交生成时优先使用 `objectKey`。
|
||||
- 后端接收画板资源 `objectKey` 后统一重新签名为 Ark 可读 URL,再按文档构造 `content`:`image_url` + `role=reference_image`、`video_url` + `role=reference_video`、`audio_url` + `role=reference_audio`。
|
||||
- 参考图片支持公网 URL、`asset://`、当前账号的 `objectKey`、项目资源 ID 或素材 ID;格式为 jpeg、png、webp、bmp、tiff、gif、heic、heif,单张小于 30MB,不接受 Base64 / Data URL / Blob URL。
|
||||
- 参考视频支持公网 URL、`asset://`、当前账号的 `objectKey`、项目资源 ID 或素材 ID;格式为 mp4、mov,单个文件不超过 50MB,单个时长 [2, 15] 秒,最多 3 个且总时长不超过 15 秒,不接受 Base64 / Data URL / Blob URL。
|
||||
- 参考音频支持公网 URL、`asset://`、当前账号的 `objectKey`、项目资源 ID 或素材 ID;格式为 wav、mp3,单个文件不超过 15MB,单个时长 [2, 15] 秒,最多 3 段且总时长不超过 15 秒,不接受 Base64 / Data URL / Blob URL。三类参考媒体最终提交的稳定引用字段总长度不得超过 `256KB`。
|
||||
- 前端本地参考媒体必须走 `/api/assets/direct-upload-tickets` 直传 OSS,再 `/api/assets/objects/confirm` 入库;前端状态保存 signed URL 供预览,同时保存 `objectKey` / `assetObjectId`,提交生成时使用稳定引用。
|
||||
- 后端接收 `objectKey`、项目资源 ID 或素材 ID 后,先解析并校验对象属于当前 owner,再重新签名为 Ark 可读 URL,并按文档构造 `content`:`image_url` + `role=reference_image`、`video_url` + `role=reference_video`、`audio_url` + `role=reference_audio`。
|
||||
- 画板生成视频按静音 toggle 映射 Ark `generate_audio`:`sound=on` 时传 `true`,`sound=off` 时传 `false`,不能只依赖上游默认值。
|
||||
- Seedance 2.0 文档提示不支持直接上传含真人人脸的参考图 / 视频;当前画板尚未做真人脸授权证明、来源声明或服务端人脸拦截,后续开放真人素材前必须补授权/来源确认链路。
|
||||
|
||||
|
||||
@@ -147,9 +147,9 @@
|
||||
### Prompt 与生成契约
|
||||
|
||||
- 前端提交到 `POST /api/editor/character-animations/generations`。
|
||||
- 请求必须带上角色图片来源、原始尺寸、动画描述、分辨率、画面比例、帧数和时长,并可通过 `assetFolderId` 指定中间视频素材落点。角色图片已经持久化到 OSS 时,`sourceImageSrc` 必须优先传 `objectKey`;只有未持久化的本地临时图片才允许传 Data URL。
|
||||
- 请求必须带上角色图片来源、原始尺寸、动画描述、分辨率、画面比例、帧数和时长,并可通过 `assetFolderId` 指定中间视频素材落点。`sourceImageSrc` 只允许传当前账号的 `objectKey`、项目资源 ID 或素材 ID;未持久化的本地临时图片必须先上传 OSS,不能提交 Data URL 或 Blob URL。后端解析稳定引用后必须校验对象属于当前 owner,再读取 OSS 内容。
|
||||
- 后端使用角色图片作为首帧和尾帧参考,模型固定映射到 `doubao-seedance-2-0-fast-260128`。
|
||||
- 后端路由兼容旧 Data URL 请求并单独放宽 JSON body limit 到 `12MB`,但该限额只作为兼容兜底,不作为新链路默认传大图的方式。
|
||||
- 后端在内联执行和队列入队前都会拒绝 Data URL / Blob URL;路由使用默认 JSON body limit,不再为内联图片请求单独放宽。
|
||||
- 后端 prompt 使用以下固定骨架,并把面板输入追加到 `动作描述:` 后:
|
||||
|
||||
```text
|
||||
|
||||
@@ -1725,19 +1725,18 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_character_animation_accepts_character_image_body_above_default_limit() {
|
||||
async fn editor_character_animation_rejects_inline_data_url_before_queueing() {
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
external_generation_mode: ExternalGenerationMode::Queue,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138190", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_character_body");
|
||||
let app = build_router(state);
|
||||
let source_image_src = format!("data:image/png;base64,{}", "A".repeat(3 * 1024 * 1024));
|
||||
let request_body = serde_json::json!({
|
||||
"sourceLayerId": "layer-character-large",
|
||||
"sourceImageSrc": source_image_src,
|
||||
"sourceImageSrc": "data:image/png;base64,AAAA",
|
||||
"sourceWidth": 1024,
|
||||
"sourceHeight": 1024,
|
||||
"promptText": "待机呼吸循环。",
|
||||
@@ -1748,7 +1747,6 @@ mod tests {
|
||||
"model": "seedance2.0-fast"
|
||||
})
|
||||
.to_string();
|
||||
assert!(request_body.len() > 2 * 1024 * 1024);
|
||||
|
||||
let response = app
|
||||
.oneshot(
|
||||
@@ -1763,7 +1761,7 @@ mod tests {
|
||||
.await
|
||||
.expect("request should succeed");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
let body = response
|
||||
.into_body()
|
||||
.collect()
|
||||
@@ -1772,10 +1770,9 @@ mod tests {
|
||||
.to_bytes();
|
||||
let body_text = String::from_utf8_lossy(&body);
|
||||
assert!(
|
||||
body_text.contains("ARK_CHARACTER_VIDEO_BASE_URL"),
|
||||
"handler should parse the oversized character source image before checking Ark config: {body_text}"
|
||||
body_text.contains("先上传 OSS"),
|
||||
"handler should reject inline character sources: {body_text}"
|
||||
);
|
||||
assert!(!body_text.contains("length limit exceeded"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -1811,23 +1808,21 @@ mod tests {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_ui_design_asset_extraction_accepts_image_body_above_default_limit() {
|
||||
async fn editor_ui_design_asset_extraction_rejects_inline_data_url_before_queueing() {
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
external_generation_mode: ExternalGenerationMode::Queue,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138188", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_ui_extract_body");
|
||||
let app = build_router(state);
|
||||
let source_image_src = format!("data:image/png;base64,{}", "A".repeat(3 * 1024 * 1024));
|
||||
let request_body = serde_json::json!({
|
||||
"sourceImageSrc": source_image_src,
|
||||
"sourceImageSrc": "data:image/png;base64,AAAA",
|
||||
"aspectRatio": "1:1",
|
||||
"imageSize": "1K",
|
||||
})
|
||||
.to_string();
|
||||
assert!(request_body.len() > 2 * 1024 * 1024);
|
||||
|
||||
let response = app
|
||||
.oneshot(
|
||||
@@ -1842,7 +1837,7 @@ mod tests {
|
||||
.await
|
||||
.expect("request should succeed");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
let body = response
|
||||
.into_body()
|
||||
.collect()
|
||||
@@ -1851,33 +1846,28 @@ mod tests {
|
||||
.to_bytes();
|
||||
let body_text = String::from_utf8_lossy(&body);
|
||||
assert!(
|
||||
body_text.contains("VECTOR_ENGINE")
|
||||
|| body_text.contains("OPENAI")
|
||||
|| body_text.contains("配置"),
|
||||
"handler should parse the oversized UI design image before checking image config: {body_text}"
|
||||
body_text.contains("先上传 OSS"),
|
||||
"handler should reject inline UI design sources: {body_text}"
|
||||
);
|
||||
assert!(!body_text.contains("length limit exceeded"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_image_generation_accepts_reference_body_above_default_limit() {
|
||||
async fn editor_image_generation_rejects_inline_data_url_before_queueing() {
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
external_generation_mode: ExternalGenerationMode::Queue,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138028", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_image_reference_body");
|
||||
let app = build_router(state);
|
||||
let reference_image_src = format!("data:image/png;base64,{}", "A".repeat(3 * 1024 * 1024));
|
||||
let request_body = serde_json::json!({
|
||||
"prompt": "生成宣发素材",
|
||||
"kind": "publication-material",
|
||||
"size": "720x540",
|
||||
"referenceImageSrcs": [reference_image_src],
|
||||
"referenceImageSrcs": ["data:image/png;base64,AAAA"],
|
||||
})
|
||||
.to_string();
|
||||
assert!(request_body.len() > 2 * 1024 * 1024);
|
||||
|
||||
let response = app
|
||||
.oneshot(
|
||||
@@ -1892,7 +1882,7 @@ mod tests {
|
||||
.await
|
||||
.expect("request should succeed");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
let body = response
|
||||
.into_body()
|
||||
.collect()
|
||||
@@ -1901,31 +1891,28 @@ mod tests {
|
||||
.to_bytes();
|
||||
let body_text = String::from_utf8_lossy(&body);
|
||||
assert!(
|
||||
body_text.contains("VECTOR_ENGINE_BASE_URL"),
|
||||
"handler should parse the oversized editor reference before checking VectorEngine config: {body_text}"
|
||||
body_text.contains("先上传 OSS"),
|
||||
"handler should reject inline editor references: {body_text}"
|
||||
);
|
||||
assert!(!body_text.contains("length limit exceeded"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_image_edit_accepts_reference_body_above_default_limit() {
|
||||
async fn editor_image_edit_rejects_inline_data_url_before_queueing() {
|
||||
let state = AppState::new(AppConfig {
|
||||
external_generation_mode: ExternalGenerationMode::Inline,
|
||||
external_generation_mode: ExternalGenerationMode::Queue,
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build");
|
||||
let seed_user = seed_phone_user_with_password(&state, "13800138029", TEST_PASSWORD).await;
|
||||
let token = sign_test_user_token(&state, &seed_user, "sess_editor_image_edit_body");
|
||||
let app = build_router(state);
|
||||
let source_image_src = format!("data:image/png;base64,{}", "A".repeat(3 * 1024 * 1024));
|
||||
let request_body = serde_json::json!({
|
||||
"prompt": "快速编辑图片",
|
||||
"sourceImageSrc": source_image_src,
|
||||
"sourceImageSrc": "data:image/png;base64,AAAA",
|
||||
"size": "1024x1024",
|
||||
"model": "gpt-image-2"
|
||||
})
|
||||
.to_string();
|
||||
assert!(request_body.len() > 2 * 1024 * 1024);
|
||||
|
||||
let response = app
|
||||
.oneshot(
|
||||
@@ -1940,7 +1927,7 @@ mod tests {
|
||||
.await
|
||||
.expect("request should succeed");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
let body = response
|
||||
.into_body()
|
||||
.collect()
|
||||
@@ -1949,10 +1936,9 @@ mod tests {
|
||||
.to_bytes();
|
||||
let body_text = String::from_utf8_lossy(&body);
|
||||
assert!(
|
||||
body_text.contains("VECTOR_ENGINE_BASE_URL"),
|
||||
"handler should parse the oversized editor edit image before checking VectorEngine config: {body_text}"
|
||||
body_text.contains("先上传 OSS"),
|
||||
"handler should reject inline editor edit sources: {body_text}"
|
||||
);
|
||||
assert!(!body_text.contains("length limit exceeded"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -64,13 +64,15 @@ use crate::{
|
||||
EditorScreenBackgroundColor, editor_green_screen_character_prompt_clause,
|
||||
},
|
||||
editor_project::{
|
||||
EDITOR_BGFILTER_CROSS_CHECK_ENABLED, EDITOR_BGFILTER_DEFAULT_SEG_MODEL,
|
||||
EDITOR_ASSET_ID_PREFIX, EDITOR_BGFILTER_CROSS_CHECK_ENABLED,
|
||||
EDITOR_BGFILTER_DEFAULT_SEG_MODEL, EDITOR_RESOURCE_ID_PREFIX,
|
||||
EditorCanvasGeneratedLayerInput, EditorGenerationPhaseReporter,
|
||||
PersistEditorGeneratedAssetRequest,
|
||||
apply_editor_screen_background_decision_to_generation_inputs,
|
||||
build_editor_canvas_generated_layer_item, complete_editor_canvas_generation_with_items,
|
||||
persist_editor_generated_media_asset,
|
||||
remove_editor_generated_screen_background_with_bgfilter_with_request_timeout,
|
||||
resolve_editor_reference_object_key_for_owner,
|
||||
},
|
||||
editor_screen_background_decision::{
|
||||
EditorScreenBackgroundDecisionInput, EditorScreenBackgroundDecisionKind,
|
||||
@@ -130,7 +132,7 @@ const EDITOR_VIDEO_MODEL_VEO_3_1_FAST: &str = "veo3.1-fast";
|
||||
const EDITOR_VIDEO_REFERENCE_IMAGE_LIMIT: usize = 9;
|
||||
const EDITOR_VIDEO_REFERENCE_VIDEO_LIMIT: usize = 3;
|
||||
const EDITOR_VIDEO_REFERENCE_AUDIO_LIMIT: usize = 3;
|
||||
const EDITOR_VIDEO_REFERENCE_REQUEST_LIMIT_BYTES: usize = 64 * 1024 * 1024;
|
||||
const EDITOR_VIDEO_REFERENCE_REQUEST_LIMIT_BYTES: usize = 256 * 1024;
|
||||
const EDITOR_VIDEO_REFERENCE_READ_EXPIRE_SECONDS: u64 = 60 * 60;
|
||||
|
||||
const BUILT_IN_MOTION_TEMPLATES: [MotionTemplate; 4] = [
|
||||
@@ -568,6 +570,14 @@ pub async fn generate_editor_character_animation(
|
||||
)
|
||||
})?;
|
||||
let owner_user_id = authenticated.claims().user_id().to_string();
|
||||
if matches_inline_media_source(payload.source_image_src.as_str()) {
|
||||
return Err(character_animation_error_response(
|
||||
&request_context,
|
||||
editor_character_animation_bad_request(
|
||||
"sourceImageSrc 必须先上传 OSS,并使用 objectKey 或画板资源引用。",
|
||||
),
|
||||
));
|
||||
}
|
||||
if !state.config.external_generation_mode.is_inline() {
|
||||
let pricing = state.editor_generation_pricing().await.map_err(|error| {
|
||||
character_animation_error_response(
|
||||
@@ -641,6 +651,14 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
|
||||
let asset_folder_id = payload.asset_folder_id.clone();
|
||||
let asset_label =
|
||||
resolve_editor_character_animation_asset_label(payload.asset_label.as_deref());
|
||||
if matches_inline_media_source(payload.source_image_src.as_str()) {
|
||||
return Err(character_animation_error_response(
|
||||
&request_context,
|
||||
editor_character_animation_bad_request(
|
||||
"sourceImageSrc 必须先上传 OSS,并使用 objectKey 或画板资源引用。",
|
||||
),
|
||||
));
|
||||
}
|
||||
|
||||
let pricing = state.editor_generation_pricing().await.map_err(|error| {
|
||||
character_animation_error_response(
|
||||
@@ -654,10 +672,17 @@ pub(crate) async fn generate_editor_character_animation_for_owner(
|
||||
// 先解析源角色图:图生视频的主体颜色完全由源图决定,背景色决策必须看到这张图。
|
||||
let source_resolve_client = build_upstream_http_client(SOURCE_IMAGE_RESOLVE_TIMEOUT_MS)
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let source_object_key = resolve_editor_reference_object_key_for_owner(
|
||||
&state,
|
||||
owner_user_id.as_str(),
|
||||
payload.source_image_src.trim(),
|
||||
)
|
||||
.await
|
||||
.map_err(|error| character_animation_error_response(&request_context, error))?;
|
||||
let source_data_url = resolve_media_source_as_data_url(
|
||||
&state,
|
||||
&source_resolve_client,
|
||||
payload.source_image_src.trim(),
|
||||
source_object_key.as_str(),
|
||||
"sourceImageSrc",
|
||||
)
|
||||
.await
|
||||
@@ -2014,7 +2039,8 @@ async fn request_editor_video_preview(
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<GeneratedAnimationPreview, AppError> {
|
||||
let upstream_task_id =
|
||||
create_editor_text_to_video_task(state, http_client, settings, request).await?;
|
||||
create_editor_text_to_video_task(state, http_client, settings, owner_user_id, request)
|
||||
.await?;
|
||||
let video_url =
|
||||
wait_for_ark_content_generation_task(http_client, &settings.ark, upstream_task_id.as_str())
|
||||
.await?;
|
||||
@@ -2038,9 +2064,10 @@ async fn create_editor_text_to_video_task(
|
||||
state: &AppState,
|
||||
http_client: &reqwest::Client,
|
||||
settings: &EditorVideoSettings,
|
||||
owner_user_id: &str,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<String, AppError> {
|
||||
let request_body = build_editor_video_ark_request_body(state, request)?;
|
||||
let request_body = build_editor_video_ark_request_body(state, owner_user_id, request).await?;
|
||||
let response = http_client
|
||||
.post(format!(
|
||||
"{}/contents/generations/tasks",
|
||||
@@ -3402,11 +3429,12 @@ fn normalize_editor_video_request_with_pricing(
|
||||
})
|
||||
}
|
||||
|
||||
fn build_editor_video_ark_request_body(
|
||||
async fn build_editor_video_ark_request_body(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<Value, AppError> {
|
||||
let content = build_editor_video_ark_content(state, request)?;
|
||||
let content = build_editor_video_ark_content(state, owner_user_id, request).await?;
|
||||
Ok(json!({
|
||||
"model": request.provider_model,
|
||||
"content": content,
|
||||
@@ -3418,8 +3446,9 @@ fn build_editor_video_ark_request_body(
|
||||
}))
|
||||
}
|
||||
|
||||
fn build_editor_video_ark_content(
|
||||
async fn build_editor_video_ark_content(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
request: &NormalizedEditorVideoRequest,
|
||||
) -> Result<Vec<Value>, AppError> {
|
||||
let mut content = Vec::new();
|
||||
@@ -3431,22 +3460,25 @@ fn build_editor_video_ark_content(
|
||||
}
|
||||
let reference_image_urls = resolve_editor_video_reference_urls(
|
||||
state,
|
||||
owner_user_id,
|
||||
request.reference_image_srcs.as_slice(),
|
||||
"参考图片",
|
||||
true,
|
||||
)?;
|
||||
)
|
||||
.await?;
|
||||
let reference_video_urls = resolve_editor_video_reference_urls(
|
||||
state,
|
||||
owner_user_id,
|
||||
request.reference_video_srcs.as_slice(),
|
||||
"参考视频",
|
||||
false,
|
||||
)?;
|
||||
)
|
||||
.await?;
|
||||
let reference_audio_urls = resolve_editor_video_reference_urls(
|
||||
state,
|
||||
owner_user_id,
|
||||
request.reference_audio_srcs.as_slice(),
|
||||
"参考音频",
|
||||
true,
|
||||
)?;
|
||||
)
|
||||
.await?;
|
||||
content.extend(reference_image_urls.iter().map(|url| {
|
||||
json!({
|
||||
"type": "image_url",
|
||||
@@ -3471,31 +3503,30 @@ fn build_editor_video_ark_content(
|
||||
Ok(content)
|
||||
}
|
||||
|
||||
fn resolve_editor_video_reference_urls(
|
||||
async fn resolve_editor_video_reference_urls(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
sources: &[String],
|
||||
label: &str,
|
||||
allow_data_url: bool,
|
||||
) -> Result<Vec<String>, AppError> {
|
||||
sources
|
||||
.iter()
|
||||
.map(|source| {
|
||||
resolve_editor_video_reference_url(state, source.as_str(), label, allow_data_url)
|
||||
})
|
||||
.collect()
|
||||
let mut resolved = Vec::with_capacity(sources.len());
|
||||
for source in sources {
|
||||
resolved.push(
|
||||
resolve_editor_video_reference_url(state, owner_user_id, source.as_str(), label)
|
||||
.await?,
|
||||
);
|
||||
}
|
||||
Ok(resolved)
|
||||
}
|
||||
|
||||
fn resolve_editor_video_reference_url(
|
||||
async fn resolve_editor_video_reference_url(
|
||||
state: &AppState,
|
||||
owner_user_id: &str,
|
||||
source: &str,
|
||||
label: &str,
|
||||
allow_data_url: bool,
|
||||
) -> Result<String, AppError> {
|
||||
let value = source.trim();
|
||||
if value.starts_with("data:") {
|
||||
if allow_data_url {
|
||||
return Ok(value.to_string());
|
||||
}
|
||||
return Err(editor_video_bad_request(format!(
|
||||
"{label}格式不支持,请使用官方支持的 URL、素材 ID 或画板资源路径。"
|
||||
)));
|
||||
@@ -3507,17 +3538,24 @@ fn resolve_editor_video_reference_url(
|
||||
return Ok(value.to_string());
|
||||
}
|
||||
|
||||
let object_key = value.trim_start_matches('/');
|
||||
if LegacyAssetPrefix::from_object_key(object_key).is_none() {
|
||||
let format_hint = if allow_data_url {
|
||||
"公网 URL、素材 ID、data URL 或画板资源路径"
|
||||
} else {
|
||||
"公网 URL、素材 ID 或画板资源路径"
|
||||
};
|
||||
return Err(editor_video_bad_request(format!(
|
||||
"{label}必须是官方支持的{format_hint}。"
|
||||
)));
|
||||
}
|
||||
let object_key = resolve_editor_reference_object_key_for_owner(state, owner_user_id, value)
|
||||
.await
|
||||
.map_err(|error| {
|
||||
error.with_details(json!({
|
||||
"provider": "editor-video",
|
||||
"field": label,
|
||||
"message": format!(
|
||||
"{label}必须是当前账号的 objectKey、项目资源 ID、素材 ID,或官方支持的公网 URL。"
|
||||
),
|
||||
}))
|
||||
})?;
|
||||
sign_editor_video_object_key_url(state, object_key.as_str())
|
||||
}
|
||||
|
||||
fn sign_editor_video_object_key_url(
|
||||
state: &AppState,
|
||||
object_key: &str,
|
||||
) -> Result<String, AppError> {
|
||||
let oss_client = state.oss_client().ok_or_else(|| {
|
||||
AppError::from_status(StatusCode::SERVICE_UNAVAILABLE).with_details(json!({
|
||||
"provider": "aliyun-oss",
|
||||
@@ -3582,9 +3620,9 @@ fn normalize_editor_video_references(
|
||||
validate_editor_video_reference_src(
|
||||
src,
|
||||
"参考图片",
|
||||
&["data:image/", "http://", "https://", "asset://"],
|
||||
&["http://", "https://", "asset://"],
|
||||
true,
|
||||
"对应媒体 data URL",
|
||||
"URL、素材 ID 或画板资源路径",
|
||||
)?;
|
||||
}
|
||||
for src in &video_srcs {
|
||||
@@ -3600,9 +3638,9 @@ fn normalize_editor_video_references(
|
||||
validate_editor_video_reference_src(
|
||||
src,
|
||||
"参考音频",
|
||||
&["data:audio/", "http://", "https://", "asset://"],
|
||||
&["http://", "https://", "asset://"],
|
||||
true,
|
||||
"对应媒体 data URL",
|
||||
"URL、素材 ID 或画板资源路径",
|
||||
)?;
|
||||
}
|
||||
|
||||
@@ -3613,7 +3651,9 @@ fn normalize_editor_video_references(
|
||||
.map(String::len)
|
||||
.sum::<usize>();
|
||||
if request_bytes > EDITOR_VIDEO_REFERENCE_REQUEST_LIMIT_BYTES {
|
||||
return Err(editor_video_bad_request("参考文件总请求体不能超过 64MB。"));
|
||||
return Err(editor_video_bad_request(
|
||||
"参考素材引用字段总长度不能超过 256KB。",
|
||||
));
|
||||
}
|
||||
|
||||
Ok(NormalizedEditorVideoReferences {
|
||||
@@ -3631,6 +3671,11 @@ fn normalize_reference_srcs(values: Vec<String>) -> Vec<String> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn matches_inline_media_source(value: &str) -> bool {
|
||||
let value = value.trim_start().to_ascii_lowercase();
|
||||
value.starts_with("data:") || value.starts_with("blob:")
|
||||
}
|
||||
|
||||
fn validate_editor_video_reference_src(
|
||||
value: &str,
|
||||
label: &str,
|
||||
@@ -3649,6 +3694,9 @@ fn validate_editor_video_reference_src(
|
||||
{
|
||||
return Ok(());
|
||||
}
|
||||
if value.starts_with(EDITOR_RESOURCE_ID_PREFIX) || value.starts_with(EDITOR_ASSET_ID_PREFIX) {
|
||||
return Ok(());
|
||||
}
|
||||
Err(editor_video_bad_request(format!(
|
||||
"{label}格式不支持,请使用官方支持的{format_hint}。"
|
||||
)))
|
||||
@@ -6072,6 +6120,34 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_rejects_inline_media_before_queueing() {
|
||||
let source = include_str!("character_animation_assets.rs");
|
||||
assert_function_contains_in_order(
|
||||
source,
|
||||
"pub async fn generate_editor_character_animation",
|
||||
"pub(crate) async fn generate_editor_character_animation_for_owner",
|
||||
&[
|
||||
"matches_inline_media_source",
|
||||
"enqueue_editor_generation_job",
|
||||
],
|
||||
);
|
||||
assert_function_contains_in_order(
|
||||
source,
|
||||
"pub(crate) async fn generate_editor_character_animation_for_owner",
|
||||
"pub async fn generate_editor_video",
|
||||
&[
|
||||
"resolve_editor_reference_object_key_for_owner",
|
||||
"resolve_media_source_as_data_url",
|
||||
],
|
||||
);
|
||||
assert!(matches_inline_media_source("data:image/png;base64,AAAA"));
|
||||
assert!(matches_inline_media_source(" BLOB:local-preview"));
|
||||
assert!(!matches_inline_media_source(
|
||||
"/generated-images/editor/character.png"
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_character_animation_reports_processing_after_video_generation() {
|
||||
let source = include_str!("character_animation_assets.rs");
|
||||
@@ -6532,11 +6608,15 @@ mod tests {
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
web_search_enabled: true,
|
||||
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
|
||||
reference_image_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/image/demo.png".to_string(),
|
||||
],
|
||||
reference_video_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
reference_audio_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/audio/demo.mp3".to_string(),
|
||||
],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
@@ -6549,7 +6629,7 @@ mod tests {
|
||||
|
||||
assert_eq!(
|
||||
normalized.reference_image_srcs,
|
||||
vec!["data:image/png;base64,image"]
|
||||
vec!["generated-character-drafts/editor/seedance-references/image/demo.png"]
|
||||
);
|
||||
assert_eq!(
|
||||
normalized.reference_video_srcs,
|
||||
@@ -6557,7 +6637,7 @@ mod tests {
|
||||
);
|
||||
assert_eq!(
|
||||
normalized.reference_audio_srcs,
|
||||
vec!["data:audio/mpeg;base64,audio"]
|
||||
vec!["generated-character-drafts/editor/seedance-references/audio/demo.mp3"]
|
||||
);
|
||||
}
|
||||
|
||||
@@ -6589,7 +6669,46 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_ark_body_sets_generate_audio_false_without_mode() {
|
||||
fn editor_video_rejects_seedance_image_and_audio_data_urls() {
|
||||
for (image_srcs, audio_srcs, expected_message) in [
|
||||
(
|
||||
vec!["data:image/png;base64,image".to_string()],
|
||||
Vec::new(),
|
||||
"参考图片格式不支持",
|
||||
),
|
||||
(
|
||||
vec!["generated-character-drafts/editor/reference.png".to_string()],
|
||||
vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
"参考音频格式不支持",
|
||||
),
|
||||
] {
|
||||
let error = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: "参考素材生成视频".to_string(),
|
||||
model: EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST.to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
duration_seconds: 4,
|
||||
resolution: "480p".to_string(),
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
web_search_enabled: true,
|
||||
reference_image_srcs: image_srcs,
|
||||
reference_video_srcs: Vec::new(),
|
||||
reference_audio_srcs: audio_srcs,
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
source_resource_id: None,
|
||||
asset_kind: None,
|
||||
asset_folder_id: None,
|
||||
asset_label: None,
|
||||
})
|
||||
.expect_err("Seedance references must not accept data URLs");
|
||||
assert!(error.body_text().contains(expected_message));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn editor_video_ark_body_sets_generate_audio_false_without_mode() {
|
||||
let state = AppState::new(AppConfig::default()).expect("state should build");
|
||||
let request = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: "参考素材生成视频".to_string(),
|
||||
@@ -6600,9 +6719,9 @@ mod tests {
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
web_search_enabled: true,
|
||||
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
|
||||
reference_image_srcs: vec!["https://assets.example.com/reference.png".to_string()],
|
||||
reference_video_srcs: vec!["https://assets.example.com/reference.mp4".to_string()],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
reference_audio_srcs: vec!["https://assets.example.com/reference.mp3".to_string()],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
@@ -6613,8 +6732,9 @@ mod tests {
|
||||
})
|
||||
.expect("seedance multimodal request should normalize");
|
||||
|
||||
let body =
|
||||
build_editor_video_ark_request_body(&state, &request).expect("ark body should build");
|
||||
let body = build_editor_video_ark_request_body(&state, "video-owner", &request)
|
||||
.await
|
||||
.expect("ark body should build");
|
||||
|
||||
assert_eq!(body["generate_audio"], json!(false));
|
||||
assert!(body.get("mode").is_none());
|
||||
@@ -6628,7 +6748,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_ark_content_signs_generated_object_keys() {
|
||||
fn editor_video_reference_resolution_checks_owner_before_signing() {
|
||||
let state = AppState::new(AppConfig {
|
||||
oss_bucket: Some("genarrative-assets".to_string()),
|
||||
oss_endpoint: Some("oss-cn-shanghai.aliyuncs.com".to_string()),
|
||||
@@ -6637,7 +6757,14 @@ mod tests {
|
||||
..AppConfig::default()
|
||||
})
|
||||
.expect("state should build with OSS");
|
||||
let request = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
let object_key = "generated-character-drafts/editor/seedance-references/video/demo.mp4";
|
||||
let signed_url = sign_editor_video_object_key_url(&state, object_key)
|
||||
.expect("validated object keys should be signable");
|
||||
assert!(signed_url.starts_with("https://genarrative-assets.oss-cn-shanghai.aliyuncs.com/"));
|
||||
assert!(signed_url.contains(object_key));
|
||||
assert!(signed_url.contains("x-oss-signature="));
|
||||
|
||||
let registered_reference = normalize_editor_video_request(EditorVideoGenerateRequest {
|
||||
prompt: "参考视频生成视频".to_string(),
|
||||
model: EDITOR_VIDEO_MODEL_SEEDANCE_2_FAST.to_string(),
|
||||
aspect_ratio: "16:9".to_string(),
|
||||
@@ -6647,9 +6774,7 @@ mod tests {
|
||||
sound: "off".to_string(),
|
||||
web_search_enabled: true,
|
||||
reference_image_srcs: Vec::new(),
|
||||
reference_video_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_video_srcs: vec!["editor-resource-video-reference".to_string()],
|
||||
reference_audio_srcs: Vec::new(),
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
@@ -6659,23 +6784,21 @@ mod tests {
|
||||
asset_folder_id: None,
|
||||
asset_label: None,
|
||||
})
|
||||
.expect("seedance objectKey request should normalize");
|
||||
|
||||
let content = build_editor_video_ark_content(&state, &request)
|
||||
.expect("ark content should sign object keys");
|
||||
let signed_url = content[1]["video_url"]["url"]
|
||||
.as_str()
|
||||
.expect("video url should be string");
|
||||
|
||||
assert!(signed_url.starts_with("https://genarrative-assets.oss-cn-shanghai.aliyuncs.com/"));
|
||||
assert!(
|
||||
signed_url
|
||||
.contains("generated-character-drafts/editor/seedance-references/video/demo.mp4")
|
||||
.expect("registered editor resource IDs should normalize");
|
||||
assert_eq!(
|
||||
registered_reference.reference_video_srcs,
|
||||
vec!["editor-resource-video-reference"]
|
||||
);
|
||||
assert!(signed_url.contains("x-oss-signature="));
|
||||
assert_ne!(
|
||||
signed_url,
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4"
|
||||
|
||||
let source = include_str!("character_animation_assets.rs");
|
||||
assert_function_contains_in_order(
|
||||
source,
|
||||
"async fn resolve_editor_video_reference_url",
|
||||
"fn sign_editor_video_object_key_url",
|
||||
&[
|
||||
"resolve_editor_reference_object_key_for_owner",
|
||||
"sign_editor_video_object_key_url",
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -27,7 +27,6 @@ use crate::{
|
||||
state::AppState,
|
||||
};
|
||||
|
||||
const EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES: usize = 12 * 1024 * 1024;
|
||||
const EDITOR_AGENT_MESSAGE_BODY_LIMIT_BYTES: usize = 2 * 1024 * 1024;
|
||||
|
||||
pub fn router(state: AppState) -> Router<AppState> {
|
||||
@@ -168,49 +167,31 @@ pub fn router(state: AppState) -> Router<AppState> {
|
||||
)
|
||||
.route(
|
||||
"/api/editor/images/generations",
|
||||
post(generate_editor_image)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
// 中文注释:图片画布生图仍兼容参考图 Data URL 入参,宣发素材提交前会压缩,后端保留有限兜底。
|
||||
EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(generate_editor_image).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/images/edits",
|
||||
post(edit_editor_image)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(edit_editor_image).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/images/background-removals",
|
||||
post(remove_editor_image_background)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(remove_editor_image_background).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/icon-spritesheets/generations",
|
||||
post(generate_editor_icon_spritesheet)
|
||||
// 中文注释:图标规范参考图会以 Data URL 提交,需避免默认 2MB body limit 在 handler 前拦截。
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(generate_editor_icon_spritesheet).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/icon-spritesheets/slices",
|
||||
@@ -221,14 +202,9 @@ pub fn router(state: AppState) -> Router<AppState> {
|
||||
)
|
||||
.route(
|
||||
"/api/editor/ui-designs/assets/extractions",
|
||||
post(extract_editor_ui_design_assets)
|
||||
// 中文注释:UI 设计图提取素材复用图片 Data URL 入参,需兼容较大的画板生成图。
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(extract_editor_ui_design_assets).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
use axum::{
|
||||
Router,
|
||||
extract::DefaultBodyLimit,
|
||||
middleware,
|
||||
Router, middleware,
|
||||
routing::{get, patch, post},
|
||||
};
|
||||
|
||||
@@ -28,9 +26,6 @@ use crate::{
|
||||
state::AppState,
|
||||
};
|
||||
|
||||
const EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES: usize = 12 * 1024 * 1024;
|
||||
const EXTERNAL_EDITOR_VIDEO_BODY_LIMIT_BYTES: usize = 64 * 1024 * 1024;
|
||||
|
||||
pub fn router(state: AppState) -> Router<AppState> {
|
||||
Router::new()
|
||||
.route("/api/external/v1/openapi.json", get(openapi_json))
|
||||
@@ -141,69 +136,42 @@ pub fn router(state: AppState) -> Router<AppState> {
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/images/generations",
|
||||
post(generate_external_editor_image)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(generate_external_editor_image).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/images/edits",
|
||||
post(edit_external_editor_image)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(edit_external_editor_image).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/icon-spritesheets/generations",
|
||||
post(generate_external_editor_icon_spritesheet)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(generate_external_editor_icon_spritesheet).route_layer(
|
||||
middleware::from_fn_with_state(state.clone(), require_external_api_key),
|
||||
),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/ui-designs/assets/extractions",
|
||||
post(extract_external_editor_ui_design_assets)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(extract_external_editor_ui_design_assets).route_layer(
|
||||
middleware::from_fn_with_state(state.clone(), require_external_api_key),
|
||||
),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/character-animations/generations",
|
||||
post(generate_external_editor_character_animation)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_IMAGE_REFERENCE_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(generate_external_editor_character_animation).route_layer(
|
||||
middleware::from_fn_with_state(state.clone(), require_external_api_key),
|
||||
),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/videos/generations",
|
||||
post(generate_external_editor_video)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
EXTERNAL_EDITOR_VIDEO_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
post(generate_external_editor_video).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_external_api_key,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/external/v1/editor/audios/sound-effects/generations",
|
||||
|
||||
@@ -52,8 +52,6 @@ use crate::{
|
||||
};
|
||||
|
||||
const HYPER3D_IMAGE_TO_MODEL_BODY_LIMIT_BYTES: usize = 56 * 1024 * 1024;
|
||||
const EDITOR_CHARACTER_ANIMATION_BODY_LIMIT_BYTES: usize = 12 * 1024 * 1024;
|
||||
const EDITOR_VIDEO_BODY_LIMIT_BYTES: usize = 64 * 1024 * 1024;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub(crate) struct PlayFlowDomainAdapter {
|
||||
@@ -460,24 +458,17 @@ fn play_flow_support_router(state: AppState) -> Router<AppState> {
|
||||
)
|
||||
.route(
|
||||
"/api/editor/character-animations/generations",
|
||||
post(generate_editor_character_animation)
|
||||
.layer(DefaultBodyLimit::max(
|
||||
// 中文注释:画板角色动画首版仍兼容角色图 Data URL 入参,避免大于 Axum 默认 2MB 的角色图在 handler 前被拦截。
|
||||
EDITOR_CHARACTER_ANIMATION_BODY_LIMIT_BYTES,
|
||||
))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(generate_editor_character_animation).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/videos/generations",
|
||||
post(generate_editor_video)
|
||||
.layer(DefaultBodyLimit::max(EDITOR_VIDEO_BODY_LIMIT_BYTES))
|
||||
.route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
post(generate_editor_video).route_layer(middleware::from_fn_with_state(
|
||||
state.clone(),
|
||||
require_bearer_auth,
|
||||
)),
|
||||
)
|
||||
.route(
|
||||
"/api/editor/audios/sound-effects/generations",
|
||||
|
||||
@@ -1201,7 +1201,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn editor_video_request_supports_seedance_multimodal_references() {
|
||||
fn editor_video_request_supports_stable_seedance_multimodal_references() {
|
||||
let payload = serde_json::to_value(EditorVideoGenerateRequest {
|
||||
prompt: "参考素材生成视频".to_string(),
|
||||
model: "seedance2.0-fast".to_string(),
|
||||
@@ -1211,11 +1211,15 @@ mod tests {
|
||||
mode: "std".to_string(),
|
||||
sound: "off".to_string(),
|
||||
web_search_enabled: true,
|
||||
reference_image_srcs: vec!["data:image/png;base64,image".to_string()],
|
||||
reference_image_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/image/demo.png".to_string(),
|
||||
],
|
||||
reference_video_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/video/demo.mp4".to_string(),
|
||||
],
|
||||
reference_audio_srcs: vec!["data:audio/mpeg;base64,audio".to_string()],
|
||||
reference_audio_srcs: vec![
|
||||
"generated-character-drafts/editor/seedance-references/audio/demo.mp3".to_string(),
|
||||
],
|
||||
project_id: None,
|
||||
canvas_completion: None,
|
||||
generation_inputs: None,
|
||||
@@ -1228,7 +1232,7 @@ mod tests {
|
||||
|
||||
assert_eq!(
|
||||
payload["referenceImageSrcs"],
|
||||
json!(["data:image/png;base64,image"])
|
||||
json!(["generated-character-drafts/editor/seedance-references/image/demo.png"])
|
||||
);
|
||||
assert_eq!(
|
||||
payload["referenceVideoSrcs"],
|
||||
@@ -1236,7 +1240,7 @@ mod tests {
|
||||
);
|
||||
assert_eq!(
|
||||
payload["referenceAudioSrcs"],
|
||||
json!(["data:audio/mpeg;base64,audio"])
|
||||
json!(["generated-character-drafts/editor/seedance-references/audio/demo.mp3"])
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1288,8 +1292,8 @@ mod tests {
|
||||
#[test]
|
||||
fn editor_icon_spritesheet_request_omits_price_and_response_includes_price() {
|
||||
let request_payload = serde_json::to_value(EditorIconSpritesheetGenerateRequest {
|
||||
reference_image_src: "data:image/png;base64,spec".to_string(),
|
||||
reference_image_srcs: vec!["data:image/png;base64,style".to_string()],
|
||||
reference_image_src: "generated-images/editor/spec.png".to_string(),
|
||||
reference_image_srcs: vec!["generated-images/editor/style.png".to_string()],
|
||||
icon_descriptions: vec!["返回按钮".to_string(), "设置按钮".to_string()],
|
||||
model: Some("gemini-3.1-flash-image-preview".to_string()),
|
||||
aspect_ratio: Some("1:1".to_string()),
|
||||
@@ -1299,16 +1303,16 @@ mod tests {
|
||||
|
||||
assert_eq!(
|
||||
request_payload["referenceImageSrc"],
|
||||
json!("data:image/png;base64,spec")
|
||||
json!("generated-images/editor/spec.png")
|
||||
);
|
||||
assert_eq!(
|
||||
request_payload["referenceImageSrcs"],
|
||||
json!(["data:image/png;base64,style"])
|
||||
json!(["generated-images/editor/style.png"])
|
||||
);
|
||||
assert!(request_payload.get("priceMudPoints").is_none());
|
||||
|
||||
let response_payload = serde_json::to_value(EditorIconSpritesheetGenerateResponse {
|
||||
spritesheet_image_src: "data:image/png;base64,sheet".to_string(),
|
||||
spritesheet_image_src: "/generated-images/editor/sheet.png".to_string(),
|
||||
spritesheet_width: 512,
|
||||
spritesheet_height: 512,
|
||||
icon_image_srcs: Vec::new(),
|
||||
|
||||
@@ -411,7 +411,6 @@ export const SEEDANCE_VIDEO_REFERENCE_LIMITS = {
|
||||
image: 9,
|
||||
video: 3,
|
||||
audio: 3,
|
||||
requestBytes: 64 * 1024 * 1024,
|
||||
} as const;
|
||||
export const QUICK_EDIT_REFERENCE_LIMIT = 8;
|
||||
export const CHARACTER_ANIMATION_ACTION_PROMPTS = [
|
||||
|
||||
@@ -91,11 +91,6 @@ export function resolveImageReferenceSubmissionSource(reference: {
|
||||
return reference.objectKey?.trim() || reference.src;
|
||||
}
|
||||
|
||||
function estimateReferenceRequestBytes(references: Array<{ src: string }>) {
|
||||
// 中文注释:JSON 包装字段还有少量额外字节;这里按 src 字符串长度估算,前后端都会再兜底校验 64MB。
|
||||
return references.reduce((sum, reference) => sum + reference.src.length, 0);
|
||||
}
|
||||
|
||||
function buildSeedanceVideoReferenceInput(
|
||||
references: NonNullable<GenerateDialogState['generationReferences']>,
|
||||
) {
|
||||
@@ -129,13 +124,6 @@ function buildSeedanceVideoReferenceInput(
|
||||
) {
|
||||
throw new Error('参考音频必须搭配参考图片或参考视频');
|
||||
}
|
||||
if (
|
||||
estimateReferenceRequestBytes(references) >
|
||||
SEEDANCE_VIDEO_REFERENCE_LIMITS.requestBytes
|
||||
) {
|
||||
throw new Error('参考文件总请求体不能超过 64MB');
|
||||
}
|
||||
|
||||
return {
|
||||
referenceImageSrcs: imageReferences.map(resolveReferenceSource),
|
||||
referenceVideoSrcs: videoReferences.map(resolveReferenceSource),
|
||||
|
||||
@@ -537,6 +537,31 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
|
||||
expect(firstOptions.pathSegments).not.toEqual(secondOptions.pathSegments);
|
||||
});
|
||||
|
||||
it('uploads image references without an object reference before generation', async () => {
|
||||
resolveEditorImageReferenceDataUrlMock.mockResolvedValueOnce(
|
||||
'data:image/png;base64,ZXhhbXBsZQ==',
|
||||
);
|
||||
const result = await resolveEditorGenerationMediaReference(
|
||||
{ src: '/creation-type-references/example.webp' },
|
||||
'image',
|
||||
'project-1',
|
||||
);
|
||||
|
||||
expect(resolveEditorImageReferenceDataUrlMock).toHaveBeenCalledWith(
|
||||
'/creation-type-references/example.webp',
|
||||
);
|
||||
expect(uploadEditorMediaAssetFileMock).toHaveBeenCalledWith(
|
||||
expect.any(File),
|
||||
'image',
|
||||
expect.objectContaining({
|
||||
assetKind: 'editor_generation_reference_image',
|
||||
}),
|
||||
);
|
||||
expect(result).toBe(
|
||||
'generated-character-drafts/editor/generation-references/reference.png',
|
||||
);
|
||||
});
|
||||
|
||||
it('submits quick edits and updates the source layer directly', async () => {
|
||||
editEditorImageMock.mockResolvedValueOnce(
|
||||
createGenerated({
|
||||
|
||||
@@ -218,20 +218,27 @@ export async function resolveEditorGenerationMediaReference(
|
||||
projectId?: string | null,
|
||||
options: EditorGenerationMediaReferenceOptions = {},
|
||||
) {
|
||||
const resourceId = reference.resourceId?.trim();
|
||||
const hasRegisteredReference =
|
||||
options.allowRegisteredIds !== false &&
|
||||
((resourceId &&
|
||||
!resourceId.startsWith('local-resource-') &&
|
||||
!resourceId.startsWith('generation-dialog:')) ||
|
||||
reference.sourceAssetId?.trim());
|
||||
const source = resolveEditorGenerationMediaReferenceSource(
|
||||
reference,
|
||||
options.allowRegisteredIds !== false,
|
||||
);
|
||||
const inlineSource = /^(?:data|blob):/iu.test(source);
|
||||
const remoteImageSource =
|
||||
options.requireImageObjectReference === true &&
|
||||
const imageSourceRequiresUpload =
|
||||
options.requireImageObjectReference !== false &&
|
||||
mediaType === 'image' &&
|
||||
!reference.objectKey?.trim() &&
|
||||
/^(?:https?:|\/api\/)/iu.test(source);
|
||||
if (!inlineSource && !remoteImageSource) {
|
||||
!hasRegisteredReference;
|
||||
if (!inlineSource && !imageSourceRequiresUpload) {
|
||||
return source;
|
||||
}
|
||||
const uploadSource = remoteImageSource
|
||||
const uploadSource = imageSourceRequiresUpload && !inlineSource
|
||||
? await resolveEditorImageReferenceDataUrl(source)
|
||||
: source;
|
||||
return uploadEditorGenerationInlineMediaSource(
|
||||
|
||||
@@ -28,9 +28,8 @@ function encodeBytesAsBase64(bytes: Uint8Array) {
|
||||
}
|
||||
|
||||
/**
|
||||
* 图片画布的生成编辑接口只接收图片 Data URL。
|
||||
* 画布里既有上传 / 生成图,也有内置 public 参考图和历史 generated 路径;
|
||||
* 提交给后端前统一读取成 Data URL,避免后端误收到普通 URL 后进入 Data URL 校验错误。
|
||||
* 将画布图片读取为浏览器临时 Data URL,供压缩、标注等本地处理使用。
|
||||
* 生成请求提交前必须再上传 OSS,并把 objectKey 传给后端;Data URL 不属于接口契约。
|
||||
*/
|
||||
export async function resolveEditorImageReferenceDataUrl(
|
||||
source: string,
|
||||
@@ -121,7 +120,7 @@ export async function compressEditorImageReferenceDataUrlForGeneration(
|
||||
return dataUrl;
|
||||
}
|
||||
|
||||
// 中文注释:参考图只用于模型理解视觉方向,提交前压缩可避免原图 Data URL 撑爆 JSON 请求体。
|
||||
// 中文注释:参考图只用于模型理解视觉方向,上传 OSS 前压缩可减少传输与存储体积。
|
||||
if (mimeType === 'image/jpeg') {
|
||||
context.fillStyle = '#fff';
|
||||
context.fillRect(0, 0, targetWidth, targetHeight);
|
||||
|
||||
@@ -18,8 +18,8 @@ import {
|
||||
generateEditorVideo,
|
||||
listEditorProjects,
|
||||
listPublicEditorProjectResources,
|
||||
loadEditorGenerationPricing,
|
||||
loadEditorAssetLibrary,
|
||||
loadEditorGenerationPricing,
|
||||
loadEditorProject,
|
||||
loadOrCreateRecentEditorProject,
|
||||
removeEditorImageBackground,
|
||||
@@ -774,6 +774,47 @@ describe('editorProjectClient', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects inline media before submitting editor generation requests', async () => {
|
||||
await expect(
|
||||
generateEditorImage({
|
||||
prompt: '一张画布图片',
|
||||
referenceImageSrcs: ['data:image/png;base64,inline'],
|
||||
}),
|
||||
).rejects.toThrow('必须先上传 OSS');
|
||||
|
||||
await expect(
|
||||
generateEditorVideo({
|
||||
prompt: '生成视频',
|
||||
model: 'seedance2.0-fast',
|
||||
aspectRatio: '16:9',
|
||||
durationSeconds: 5,
|
||||
resolution: '720p',
|
||||
mode: 'std',
|
||||
sound: 'off',
|
||||
referenceVideoSrcs: ['blob:local-video'],
|
||||
}),
|
||||
).rejects.toThrow('必须先上传 OSS');
|
||||
|
||||
expect(requestJsonMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects oversized stable video reference fields before submission', async () => {
|
||||
await expect(
|
||||
generateEditorVideo({
|
||||
prompt: '生成视频',
|
||||
model: 'seedance2.0-fast',
|
||||
aspectRatio: '16:9',
|
||||
durationSeconds: 5,
|
||||
resolution: '720p',
|
||||
mode: 'std',
|
||||
sound: 'off',
|
||||
referenceVideoSrcs: [`https://assets.example.test/${'a'.repeat(256 * 1024)}`],
|
||||
}),
|
||||
).rejects.toThrow('稳定引用字段总长度不能超过 256KB');
|
||||
|
||||
expect(requestJsonMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('passes image model options to editor image generation', async () => {
|
||||
requestJsonMock.mockResolvedValueOnce({
|
||||
imageSrc: 'data:image/png;base64,character',
|
||||
@@ -919,8 +960,8 @@ describe('editorProjectClient', () => {
|
||||
});
|
||||
|
||||
const result = await generateEditorIconSpritesheet({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
referenceImageSrcs: ['data:image/png;base64,icon-ref'],
|
||||
referenceImageSrc: '/generated-images/editor/spec.png',
|
||||
referenceImageSrcs: ['/generated-images/editor/icon-ref.png'],
|
||||
iconDescriptions: ['返回按钮', '设置按钮'],
|
||||
assetLabel: '冒险游戏图标',
|
||||
});
|
||||
@@ -936,8 +977,8 @@ describe('editorProjectClient', () => {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
referenceImageSrcs: ['data:image/png;base64,icon-ref'],
|
||||
referenceImageSrc: '/generated-images/editor/spec.png',
|
||||
referenceImageSrcs: ['/generated-images/editor/icon-ref.png'],
|
||||
iconDescriptions: ['返回按钮', '设置按钮'],
|
||||
model: 'gemini-3.1-flash-image-preview',
|
||||
assetLabel: '冒险游戏图标',
|
||||
@@ -965,7 +1006,7 @@ describe('editorProjectClient', () => {
|
||||
});
|
||||
|
||||
await generateEditorIconSpritesheet({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
referenceImageSrc: '/generated-images/editor/spec.png',
|
||||
iconDescriptions: ['返回按钮'],
|
||||
model: 'gpt-image-2',
|
||||
screenColor: '#E6D8FF',
|
||||
@@ -986,7 +1027,7 @@ describe('editorProjectClient', () => {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
referenceImageSrc: 'data:image/png;base64,spec',
|
||||
referenceImageSrc: '/generated-images/editor/spec.png',
|
||||
iconDescriptions: ['返回按钮'],
|
||||
model: 'gpt-image-2',
|
||||
screenColor: '#E6D8FF',
|
||||
@@ -1105,11 +1146,11 @@ describe('editorProjectClient', () => {
|
||||
});
|
||||
|
||||
const result = await extractEditorUiDesignAssets({
|
||||
sourceImageSrc: 'data:image/png;base64,ui-design',
|
||||
sourceImageSrc: '/generated-images/editor/ui-design.png',
|
||||
model: 'gpt-image-2',
|
||||
screenColor: '#7FB3FF',
|
||||
segModel: 'anime-seg',
|
||||
referenceImageSrcs: ['data:image/png;base64,ui-ref'],
|
||||
referenceImageSrcs: ['/generated-images/editor/ui-ref.png'],
|
||||
projectId: 'editor-project-1',
|
||||
assetFolderId: 'project',
|
||||
spritesheetLabel: '战斗UI设计图 素材图集',
|
||||
@@ -1128,11 +1169,11 @@ describe('editorProjectClient', () => {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourceImageSrc: 'data:image/png;base64,ui-design',
|
||||
sourceImageSrc: '/generated-images/editor/ui-design.png',
|
||||
model: 'gpt-image-2',
|
||||
screenColor: '#7FB3FF',
|
||||
segModel: 'anime-seg',
|
||||
referenceImageSrcs: ['data:image/png;base64,ui-ref'],
|
||||
referenceImageSrcs: ['/generated-images/editor/ui-ref.png'],
|
||||
projectId: 'editor-project-1',
|
||||
generationInputs: {
|
||||
fields: [{ title: '来源', value: '战斗UI设计图' }],
|
||||
@@ -1212,7 +1253,7 @@ describe('editorProjectClient', () => {
|
||||
kind: 'publication-material',
|
||||
aspectRatio: '9:16',
|
||||
imageSize: '1K',
|
||||
referenceImageSrcs: ['data:image/png;base64,ref'],
|
||||
referenceImageSrcs: ['/generated-images/editor/publication-ref.png'],
|
||||
});
|
||||
|
||||
expect(result.height).toBe(1280);
|
||||
@@ -1227,7 +1268,7 @@ describe('editorProjectClient', () => {
|
||||
kind: 'publication-material',
|
||||
aspectRatio: '9:16',
|
||||
imageSize: '1K',
|
||||
referenceImageSrcs: ['data:image/png;base64,ref'],
|
||||
referenceImageSrcs: ['/generated-images/editor/publication-ref.png'],
|
||||
}),
|
||||
}),
|
||||
'生成图片失败',
|
||||
@@ -1259,7 +1300,7 @@ describe('editorProjectClient', () => {
|
||||
|
||||
const result = await generateEditorCharacterAnimation({
|
||||
sourceLayerId: 'layer-character',
|
||||
sourceImageSrc: 'data:image/png;base64,character',
|
||||
sourceImageSrc: '/generated-images/editor/character.png',
|
||||
sourceWidth: 1024,
|
||||
sourceHeight: 1024,
|
||||
promptText: '待机',
|
||||
@@ -1279,7 +1320,7 @@ describe('editorProjectClient', () => {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourceLayerId: 'layer-character',
|
||||
sourceImageSrc: 'data:image/png;base64,character',
|
||||
sourceImageSrc: '/generated-images/editor/character.png',
|
||||
sourceWidth: 1024,
|
||||
sourceHeight: 1024,
|
||||
promptText: '待机',
|
||||
@@ -1395,7 +1436,7 @@ describe('editorProjectClient', () => {
|
||||
resolution: '720p',
|
||||
mode: 'std',
|
||||
sound: 'off',
|
||||
referenceImageSrcs: ['data:image/png;base64,image'],
|
||||
referenceImageSrcs: ['/generated-images/editor/video-reference.png'],
|
||||
referenceVideoSrcs: [
|
||||
'generated-character-drafts/editor/seedance-references/video/video.mp4',
|
||||
],
|
||||
@@ -1418,7 +1459,7 @@ describe('editorProjectClient', () => {
|
||||
mode: 'std',
|
||||
sound: 'off',
|
||||
webSearchEnabled: true,
|
||||
referenceImageSrcs: ['data:image/png;base64,image'],
|
||||
referenceImageSrcs: ['/generated-images/editor/video-reference.png'],
|
||||
referenceVideoSrcs: [
|
||||
'generated-character-drafts/editor/seedance-references/video/video.mp4',
|
||||
],
|
||||
@@ -1578,12 +1619,12 @@ describe('editorProjectClient', () => {
|
||||
|
||||
const result = await editEditorImage({
|
||||
prompt: '把画面改成黄昏光线',
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
sourceImageSrc: '/generated-images/editor/source.png',
|
||||
size: '1024x1024',
|
||||
model: 'gpt-image-2',
|
||||
aspectRatio: '1:1',
|
||||
imageSize: '1K',
|
||||
referenceImageSrcs: ['data:image/png;base64,style'],
|
||||
referenceImageSrcs: ['/generated-images/editor/style.png'],
|
||||
projectId: 'editor-project-1',
|
||||
assetKind: 'character',
|
||||
assetFolderId: 'project',
|
||||
@@ -1604,12 +1645,12 @@ describe('editorProjectClient', () => {
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
prompt: '把画面改成黄昏光线',
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
sourceImageSrc: '/generated-images/editor/source.png',
|
||||
size: '1024x1024',
|
||||
model: 'gpt-image-2',
|
||||
aspectRatio: '1:1',
|
||||
imageSize: '1K',
|
||||
referenceImageSrcs: ['data:image/png;base64,style'],
|
||||
referenceImageSrcs: ['/generated-images/editor/style.png'],
|
||||
projectId: 'editor-project-1',
|
||||
assetKind: 'character',
|
||||
generationInputs: {
|
||||
@@ -1643,7 +1684,7 @@ describe('editorProjectClient', () => {
|
||||
});
|
||||
|
||||
const response = await removeEditorImageBackground({
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
sourceImageSrc: '/generated-images/editor/source.png',
|
||||
projectId: 'editor-project-1',
|
||||
targetLayerId: 'layer-source',
|
||||
canvasCompletion: {
|
||||
@@ -1668,7 +1709,7 @@ describe('editorProjectClient', () => {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
sourceImageSrc: 'data:image/png;base64,source',
|
||||
sourceImageSrc: '/generated-images/editor/source.png',
|
||||
projectId: 'editor-project-1',
|
||||
targetLayerId: 'layer-source',
|
||||
canvasCompletion: {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { requestJson } from '../apiClient';
|
||||
import type { ExternalGenerationJobStatusRecord } from '../../../packages/shared/src/contracts/externalGeneration';
|
||||
import { requestJson } from '../apiClient';
|
||||
import { EDITOR_REQUEST_RETRY_OPTIONS } from './editorRetryOptions';
|
||||
|
||||
const EDITOR_PROJECT_API_BASE = '/api/editor/projects';
|
||||
@@ -25,8 +25,38 @@ const EDITOR_BACKGROUND_MUSIC_GENERATION_API =
|
||||
'/api/editor/audios/background-music/generations';
|
||||
const EDITOR_GENERATION_PRICING_API = '/api/editor/generation-pricing';
|
||||
const EDITOR_IMAGE_MODEL_NANOBANANA2 = 'gemini-3.1-flash-image-preview';
|
||||
const EDITOR_VIDEO_REFERENCE_REQUEST_LIMIT_BYTES = 256 * 1024;
|
||||
const DEFAULT_PROJECT_TITLE = '未命名画布';
|
||||
|
||||
function assertStableEditorMediaReference(value: string, fieldLabel: string) {
|
||||
const normalized = value.trimStart().toLowerCase();
|
||||
if (normalized.startsWith('data:') || normalized.startsWith('blob:')) {
|
||||
throw new Error(
|
||||
`${fieldLabel}必须先上传 OSS,并使用 objectKey、项目资源、素材 ID 或支持的 URL 引用。`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function assertStableEditorMediaReferences(
|
||||
values: readonly string[] | undefined,
|
||||
fieldLabel: string,
|
||||
) {
|
||||
values?.forEach((value) => assertStableEditorMediaReference(value, fieldLabel));
|
||||
}
|
||||
|
||||
function assertStableEditorVideoReferences(input: EditorVideoGenerationInput) {
|
||||
const references = [
|
||||
...(input.referenceImageSrcs ?? []),
|
||||
...(input.referenceVideoSrcs ?? []),
|
||||
...(input.referenceAudioSrcs ?? []),
|
||||
];
|
||||
assertStableEditorMediaReferences(references, '视频参考素材');
|
||||
const requestBytes = new TextEncoder().encode(references.join('')).byteLength;
|
||||
if (requestBytes > EDITOR_VIDEO_REFERENCE_REQUEST_LIMIT_BYTES) {
|
||||
throw new Error('视频参考素材稳定引用字段总长度不能超过 256KB');
|
||||
}
|
||||
}
|
||||
|
||||
export type EditorCanvasViewport = {
|
||||
x: number;
|
||||
y: number;
|
||||
@@ -874,6 +904,7 @@ export async function toggleEditorShowcaseAssetLike(
|
||||
}
|
||||
|
||||
export async function generateEditorImage(input: EditorImageGenerationInput) {
|
||||
assertStableEditorMediaReferences(input.referenceImageSrcs, '生成参考图');
|
||||
return requestJson<EditorImageGenerationResponse>(
|
||||
EDITOR_IMAGE_GENERATION_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -913,6 +944,11 @@ export async function generateEditorImage(input: EditorImageGenerationInput) {
|
||||
export async function generateEditorIconSpritesheet(
|
||||
input: EditorIconSpritesheetGenerationInput,
|
||||
) {
|
||||
assertStableEditorMediaReference(input.referenceImageSrc, '图标素材规范');
|
||||
assertStableEditorMediaReferences(
|
||||
input.referenceImageSrcs,
|
||||
'图标素材参考图',
|
||||
);
|
||||
return requestJson<EditorIconSpritesheetGenerationResponse>(
|
||||
EDITOR_ICON_SPRITESHEET_GENERATION_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -962,6 +998,8 @@ export async function splitEditorIconSpritesheet(
|
||||
export async function extractEditorUiDesignAssets(
|
||||
input: EditorUiDesignAssetExtractionInput,
|
||||
) {
|
||||
assertStableEditorMediaReference(input.sourceImageSrc, 'UI设计图');
|
||||
assertStableEditorMediaReferences(input.referenceImageSrcs, 'UI素材参考图');
|
||||
return requestJson<EditorIconSpritesheetGenerationResponse>(
|
||||
EDITOR_UI_DESIGN_ASSET_EXTRACTION_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -995,6 +1033,8 @@ export async function extractEditorUiDesignAssets(
|
||||
}
|
||||
|
||||
export async function editEditorImage(input: EditorImageEditInput) {
|
||||
assertStableEditorMediaReference(input.sourceImageSrc, '待修改图片');
|
||||
assertStableEditorMediaReferences(input.referenceImageSrcs, '修改参考图');
|
||||
return requestJson<EditorImageGenerationResponse>(
|
||||
EDITOR_IMAGE_EDIT_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -1033,6 +1073,7 @@ export async function editEditorImage(input: EditorImageEditInput) {
|
||||
export async function removeEditorImageBackground(
|
||||
input: EditorBackgroundRemovalInput,
|
||||
) {
|
||||
assertStableEditorMediaReference(input.sourceImageSrc, '待去除背景图片');
|
||||
return requestJson<EditorBackgroundRemovalResponse>(
|
||||
EDITOR_BACKGROUND_REMOVAL_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -1064,6 +1105,7 @@ export async function removeEditorImageBackground(
|
||||
export async function generateEditorCharacterAnimation(
|
||||
input: EditorCharacterAnimationGenerationInput,
|
||||
) {
|
||||
assertStableEditorMediaReference(input.sourceImageSrc, '角色动画源图');
|
||||
return requestJson<EditorCharacterAnimationGenerationResult>(
|
||||
EDITOR_CHARACTER_ANIMATION_GENERATION_API,
|
||||
jsonRequest('POST', {
|
||||
@@ -1088,6 +1130,7 @@ export async function generateEditorCharacterAnimation(
|
||||
}
|
||||
|
||||
export async function generateEditorVideo(input: EditorVideoGenerationInput) {
|
||||
assertStableEditorVideoReferences(input);
|
||||
return requestJson<EditorVideoGenerationResponse>(
|
||||
EDITOR_VIDEO_GENERATION_API,
|
||||
jsonRequest('POST', {
|
||||
|
||||
Reference in New Issue
Block a user