GenarrativeAI/Genarrative
10
0
Fork 0
Code Issues 24 Pull Requests 9 Actions Packages Projects 1 Releases Wiki Activity

修复画板音视频素材保存

Browse Source 
生成视频持久化到 OSS 并回传 objectKey 与 assetObjectId

生成音频透传 OSS 对象身份并支持保存到素材库

普通素材上传支持 MP3 和 MP4 直传 OSS 后确认资产对象

素材库和画布恢复音频视频媒体类型与图层播放

更新画板音视频素材保存相关文档与测试
This commit is contained in:
段舒康
2026-06-19 15:11:42 +08:00
parent fdcccec385
commit 870d723eaf
27 changed files with 853 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md
+1 -1
View File
@@ -109,7 +109,7 @@
- 生成占位图和生成器对话框不是临时浮层，必须作为画布布局数据保存。
- 保存时在现有画布布局数组中追加 `itemType: "generation-dialog"` 项，记录生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和 `generatedLayerId`。
- 生成成功后仍保留生成器快照；画布渲染优先用 `generatedLayerId` 锚定到成品图层，不再重复显示灰色占位框。
- 图片类生成结果还要写入账号级素材库；视频结果先只作为画布资源和视频图层保存。
- 图片、视频和音频生成结果都要写入账号级素材库；视频 / 音频结果由后端持久化到 OSS 并回传 `objectKey` / `assetObjectId`，前端保存素材库时一并记录，后续预览和再次加入画布走统一换签链路。
- 刷新项目后，画布需要同时恢复图层、生成器快照和生成输入框跟随关系。
## 第一版计费配置
docs/【编辑器】画板音乐生成入口设计-2026-06-18.md
+7 -1
View File
@@ -42,7 +42,8 @@
- 新增素材类型：
- `assetKind="sound-effect"`
- `assetKind="background-music"`
- 音频结果以小型音频卡加入画布，卡片底部使用 `<audio controls>` 播放；音频结果当前只保存为画布资源，不进入图片素材库。
- 音频结果以小型音频卡加入画布，卡片底部使用 `<audio controls>` 播放；生成成功后同时保存为 OSS 私有对象、画布资源和账号级素材库素材，响应携带 `objectKey` / `assetObjectId` 供后续换签和复用。
- 普通素材上传入口首版支持图片、MP3 和 MP4；MP3 / MP4 先走 OSS 直传和 asset object confirm，再以素材库素材保存，其他音视频格式暂不开放。
- 音频结果卡片底部显示用户提示词与播放控件，不在卡片左下角或悬停左上角展示时长。
- 音频卡片中央播放区按状态切换：未悬停且未播放时显示音效 / 背景音乐图标，悬停且未播放时显示播放按钮，播放中始终显示暂停按钮。
- generated 私有音频资源播放前必须通过 `/api/assets/read-url` 换签；画布卡片不得直接把 `/generated-*` 或 generated OSS 私有地址交给 `<audio>` 裸请求。
@@ -80,6 +81,8 @@ POST /api/editor/audios/background-music/generations
{
ok: true,
audioSrc: string,
objectKey?: string | null,
assetObjectId?: string | null,
width: 420,
height: 120,
sourceType: "generated",
@@ -108,6 +111,7 @@ POST /api/editor/audios/background-music/generations
- `/api/editor/audios/sound-effects/generations`
- `/api/editor/audios/background-music/generations`
- BFF 复用现有 `vector_engine_audio_generation` 的任务轮询、下载、OSS 持久化和计费包装；音效 10 泥点，背景音乐 5 泥点。
- 生成音频持久化后返回 OSS `objectKey` 与 `assetObjectId`；前端保存素材库时继续使用 `audioSrc` 作为兼容路径，并把 OSS 身份写入素材记录。
## 验收
@@ -119,6 +123,8 @@ POST /api/editor/audios/background-music/generations
- 点击 `生成游戏背景音乐` 后出现背景音乐面板，字段为 `gpt_description_prompt`，不展示 `make_instrumental`。
- 音效提交到 `/api/editor/audios/sound-effects/generations`，背景音乐提交到 `/api/editor/audios/background-music/generations`。
- 成功后画布新增音频卡，能通过 `<audio controls>` 播放。
- 成功后音频素材自动出现在账号级素材库；从素材库再次添加到画布时仍恢复为 `mediaType="audio"` 音频图层。
- 普通素材上传 MP3 / MP4 会写入 OSS 并进入素材库；MP3 添加到画布为音频图层，MP4 添加到画布为视频图层。
- 成功后的音频卡展示提示词；卡片中央按未悬停图标、悬停播放、播放中暂停切换，播放控件固定在卡片底部。
- 音频卡片、悬停角标和待生成占位不显示时长；信息弹窗显示时长。
- 音频素材浮动工具栏只显示 `改造` 与 `下载按钮`，`改造` 复用对应生成面板且没有参考图组件。
server-rs/crates/api-server/src/character_animation_assets.rs
+72 -4
View File
@@ -135,6 +135,12 @@ const BUILT_IN_MOTION_TEMPLATES: [MotionTemplate; 4] = [
},
];
struct PersistedEditorVideo {
legacy_public_path: String,
object_key: String,
asset_object_id: String,
}
pub async fn list_character_animation_templates(
Extension(request_context): Extension<RequestContext>,
) -> Json<Value> {
@@ -630,6 +636,8 @@ pub async fn generate_editor_video(
EditorVideoGenerateResponse {
ok: true,
video_src: generated.preview_video_path,
object_key: generated.object_key,
asset_object_id: generated.asset_object_id,
width: normalized.width,
height: normalized.height,
source_type: "generated".to_string(),
@@ -1146,6 +1154,8 @@ async fn request_image_to_video_preview(
Ok(GeneratedAnimationPreview {
preview_video_path,
object_key: None,
asset_object_id: None,
upstream_task_id,
submitted_prompt,
moderation_fallback_applied,
@@ -1438,6 +1448,8 @@ async fn request_editor_character_animation_preview(
Ok(GeneratedAnimationPreview {
preview_video_path,
object_key: None,
asset_object_id: None,
upstream_task_id,
submitted_prompt: prompt.to_string(),
moderation_fallback_applied: false,
@@ -1459,11 +1471,13 @@ async fn request_editor_video_preview(
.await?;
let preview_payload =
download_generated_video(http_client, video_url.as_str(), "下载画板生成视频失败。").await?;
let preview_video_path =
let persisted =
put_generated_editor_video(state, owner_user_id, task_id, request, preview_payload).await?;
Ok(GeneratedAnimationPreview {
preview_video_path,
preview_video_path: persisted.legacy_public_path,
object_key: Some(persisted.object_key),
asset_object_id: Some(persisted.asset_object_id),
upstream_task_id,
submitted_prompt: request.prompt.clone(),
moderation_fallback_applied: false,
@@ -1519,7 +1533,8 @@ async fn put_generated_editor_video(
task_id: &str,
request: &NormalizedEditorVideoRequest,
preview_payload: MediaPayload,
) -> Result<String, AppError> {
) -> Result<PersistedEditorVideo, AppError> {
let content_type = preview_payload.mime_type.clone();
let put_result = put_character_animation_object(
state,
LegacyAssetPrefix::CharacterDrafts,
@@ -1541,7 +1556,19 @@ async fn put_generated_editor_video(
),
)
.await?;
Ok(put_result.legacy_public_path)
let confirmed = confirm_editor_video_asset_object(
state,
owner_user_id,
task_id,
put_result.object_key.clone(),
content_type,
)
.await?;
Ok(PersistedEditorVideo {
legacy_public_path: put_result.legacy_public_path,
object_key: put_result.object_key,
asset_object_id: confirmed.record.asset_object_id,
})
}
async fn create_editor_ark_image_to_video_task(
@@ -1978,6 +2005,45 @@ async fn confirm_character_animation_asset_object(
Ok(module_assets::ConfirmAssetObjectResult { record })
}
async fn confirm_editor_video_asset_object(
state: &AppState,
owner_user_id: &str,
task_id: &str,
object_key: String,
content_type: String,
) -> Result<module_assets::ConfirmAssetObjectResult, AppError> {
let oss_client = require_oss_client(state)?;
let head = oss_client
.head_object(&reqwest::Client::new(), OssHeadObjectRequest { object_key })
.await
.map_err(map_character_animation_oss_error)?;
let now_micros = current_utc_micros();
let record = state
.spacetime_client()
.confirm_asset_object(
build_asset_object_upsert_input(
generate_asset_object_id(now_micros),
head.bucket,
head.object_key,
AssetObjectAccessPolicy::Private,
head.content_type.or(Some(content_type)),
head.content_length,
head.etag,
EDITOR_VIDEO_ASSET_KIND.to_string(),
Some(task_id.to_string()),
Some(owner_user_id.to_string()),
None,
Some(task_id.to_string()),
now_micros,
)
.map_err(map_asset_object_prepare_error)?,
)
.await
.map_err(map_character_animation_spacetime_error)?;
Ok(module_assets::ConfirmAssetObjectResult { record })
}
async fn bind_character_animation_asset(
state: &AppState,
owner_user_id: &str,
@@ -4477,6 +4543,8 @@ struct NormalizedEditorVideoRequest {
struct GeneratedAnimationPreview {
preview_video_path: String,
object_key: Option<String>,
asset_object_id: Option<String>,
upstream_task_id: String,
submitted_prompt: String,
moderation_fallback_applied: bool,
server-rs/crates/api-server/src/vector_engine_audio_generation/generation.rs
+4
View File
@@ -163,6 +163,8 @@ pub async fn generate_editor_sound_effect(
assets::EditorAudioGenerateResponse {
ok: true,
audio_src,
object_key: generated.object_key,
asset_object_id: generated.asset_object_id,
width: EDITOR_AUDIO_WIDTH,
height: EDITOR_AUDIO_HEIGHT,
source_type: "generated".to_string(),
@@ -222,6 +224,8 @@ pub async fn generate_editor_background_music(
assets::EditorAudioGenerateResponse {
ok: true,
audio_src,
object_key: generated.object_key,
asset_object_id: generated.asset_object_id,
width: EDITOR_AUDIO_WIDTH,
height: EDITOR_AUDIO_HEIGHT,
source_type: "generated".to_string(),
server-rs/crates/api-server/src/vector_engine_audio_generation/persist.rs
+2
View File
@@ -17,6 +17,7 @@ use super::{
#[derive(Clone, Debug)]
pub(super) struct PersistedAudioAsset {
pub(super) asset_object_id: String,
pub(super) object_key: String,
pub(super) audio_src: String,
}
@@ -111,6 +112,7 @@ pub(super) async fn persist_generated_audio_asset(
Ok(PersistedAudioAsset {
asset_object_id: asset_object.asset_object_id,
object_key: put_result.object_key,
audio_src: put_result.legacy_public_path,
})
}
server-rs/crates/api-server/src/vector_engine_audio_generation/publish.rs
+2
View File
@@ -65,6 +65,7 @@ pub(super) async fn publish_generated_audio_asset_with_task_kind(
provider: task_kind.provider().to_string(),
status: status.clone(),
asset_object_id: None,
object_key: None,
asset_kind: None,
audio_src: None,
});
@@ -118,6 +119,7 @@ pub(super) async fn publish_generated_audio_asset_with_task_kind(
provider: task_kind.provider().to_string(),
status: "completed".to_string(),
asset_object_id: Some(persisted.asset_object_id),
object_key: Some(persisted.object_key),
asset_kind: Some(target.asset_kind),
audio_src: Some(persisted.audio_src),
})
server-rs/crates/shared-contracts/src/assets.rs
+22
View File
@@ -370,6 +370,10 @@ pub struct EditorVideoGenerateRequest {
pub struct EditorVideoGenerateResponse {
pub ok: bool,
pub video_src: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub object_key: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub asset_object_id: Option<String>,
pub width: u32,
pub height: u32,
pub source_type: String,
@@ -446,6 +450,10 @@ pub struct EditorBackgroundMusicGenerateRequest {
pub struct EditorAudioGenerateResponse {
pub ok: bool,
pub audio_src: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub object_key: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub asset_object_id: Option<String>,
pub width: u32,
pub height: u32,
pub source_type: String,
@@ -1091,6 +1099,8 @@ mod tests {
let payload = serde_json::to_value(EditorVideoGenerateResponse {
ok: true,
video_src: "/generated-editor-videos/task-1/preview.mp4".to_string(),
object_key: Some("generated-character-drafts/editor-videos/task-1/preview.mp4".to_string()),
asset_object_id: Some("assetobj_video_1".to_string()),
width: 1280,
height: 720,
source_type: "generated".to_string(),
@@ -1109,6 +1119,11 @@ mod tests {
payload["videoSrc"],
json!("/generated-editor-videos/task-1/preview.mp4")
);
assert_eq!(
payload["objectKey"],
json!("generated-character-drafts/editor-videos/task-1/preview.mp4")
);
assert_eq!(payload["assetObjectId"], json!("assetobj_video_1"));
assert_eq!(payload["sourceType"], json!("generated"));
assert_eq!(payload["durationSeconds"], json!(5));
}
@@ -1205,6 +1220,8 @@ mod tests {
let response_payload = serde_json::to_value(EditorAudioGenerateResponse {
ok: true,
audio_src: "/generated-character-drafts/editor-audios/sfx.mp3".to_string(),
object_key: Some("generated-character-drafts/editor-audios/sfx.mp3".to_string()),
asset_object_id: Some("assetobj_audio_1".to_string()),
width: 420,
height: 120,
source_type: "generated".to_string(),
@@ -1221,6 +1238,11 @@ mod tests {
response_payload["audioSrc"],
json!("/generated-character-drafts/editor-audios/sfx.mp3")
);
assert_eq!(
response_payload["objectKey"],
json!("generated-character-drafts/editor-audios/sfx.mp3")
);
assert_eq!(response_payload["assetObjectId"], json!("assetobj_audio_1"));
assert_eq!(response_payload["audioKind"], json!("sound-effect"));
}
server-rs/crates/shared-contracts/src/creation_audio.rs
+2
View File
@@ -88,6 +88,8 @@ pub struct GeneratedAudioAssetResponse {
#[serde(default)]
pub asset_object_id: Option<String>,
#[serde(default)]
pub object_key: Option<String>,
#[serde(default)]
pub asset_kind: Option<String>,
#[serde(default)]
pub audio_src: Option<String>,
src/components/image-editor/ImageCanvasAssetRowView.test.tsx
+16
View File
@@ -128,4 +128,20 @@ describe('ImageCanvasAssetRowView', () => {
screen.getByLabelText('素材账号素材A上传进度').getAttribute('value'),
).toBe('42');
});
it('uses icon thumbnails for audio assets instead of rendering media as images', () => {
renderAssetRow({
asset: createAsset({
label: '胜利音效.mp3',
src: '/generated-character-drafts/editor/asset-library/audio/胜利音效.mp3',
mediaType: 'audio',
}),
});
expect(screen.getByText('音频')).toBeTruthy();
expect(screen.queryByRole('img', { name: '素材：胜利音效.mp3' })).toBeNull();
expect(
document.querySelector('.image-canvas-editor__asset-media-overlay'),
).toBeTruthy();
});
});
src/components/image-editor/ImageCanvasAssetRowView.tsx
+22 -4
View File
@@ -1,4 +1,4 @@
import { Check, Pencil, Trash2, X } from 'lucide-react';
import { Check, Music2, Pencil, Trash2, Video, X } from 'lucide-react';
import type { Dispatch, SetStateAction } from 'react';
import { PlatformTextField } from '../common/PlatformTextField';
@@ -65,6 +65,22 @@ export function ImageCanvasAssetRowView({
const isUploadingAsset = asset.uploadStatus === 'uploading';
const isFailedUpload = asset.uploadStatus === 'failed';
const uploadProgress = clamp(asset.uploadProgress ?? 0, 0, 100);
const mediaDetail =
asset.mediaType === 'audio'
? '音频'
: asset.mediaType === 'video'
? '视频'
: `${asset.width} x ${asset.height}`;
const mediaOverlay =
asset.mediaType === 'audio' ? (
<div className="image-canvas-editor__asset-media-overlay">
<Music2 className="h-5 w-5" />
</div>
) : asset.mediaType === 'video' ? (
<div className="image-canvas-editor__asset-media-overlay">
<Video className="h-5 w-5" />
</div>
) : undefined;
const titleNode = isRenaming ? (
<PlatformTextField
aria-label={`重命名素材${asset.label}`}
@@ -134,8 +150,8 @@ export function ImageCanvasAssetRowView({
<div data-asset-id={asset.id}>
<SidebarMediaItem
title={asset.label}
detail={`${asset.width} x ${asset.height}`}
imageSrc={asset.src}
detail={mediaDetail}
imageSrc={asset.mediaType === 'image' ? asset.src : ''}
imageAlt={`素材：${asset.label}`}
primaryLabel={
isUploadingAsset
@@ -182,7 +198,9 @@ export function ImageCanvasAssetRowView({
<div className="image-canvas-editor__asset-upload-overlay">
<span>上传中</span>
</div>
) : undefined
) : (
mediaOverlay
)
}
footerNode={
isUploadingAsset || isFailedUpload ? (
src/components/image-editor/ImageCanvasEditorAssetsIntegration.test.tsx
+95 -6
View File
@@ -9,7 +9,7 @@ import {
within,
} from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { describe, expect, it, vi } from 'vitest';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import {
ApiClientError,
@@ -38,6 +38,7 @@ const loadEditorProjectMock = vi.hoisted(() => vi.fn());
const loadOrCreateRecentEditorProjectMock = vi.hoisted(() => vi.fn());
const renameEditorProjectMock = vi.hoisted(() => vi.fn());
const saveEditorProjectLayoutMock = vi.hoisted(() => vi.fn());
const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn());
vi.mock('../../services/image-editor/editorProjectClient', async () => {
const actual = await vi.importActual<
@@ -64,6 +65,10 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => {
};
});
vi.mock('../../services/image-editor/editorMediaAssetUploadClient', () => ({
uploadEditorMediaAssetFile: uploadEditorMediaAssetFileMock,
}));
describe('ImageCanvasEditorView asset library integration', () => {
setupImageCanvasEditorViewTestLifecycle({
generateEditorImageMock,
@@ -84,6 +89,17 @@ describe('ImageCanvasEditorView asset library integration', () => {
saveEditorProjectLayoutMock,
});
beforeEach(() => {
uploadEditorMediaAssetFileMock.mockImplementation(
async (file: File, mediaType: 'video' | 'audio') => ({
src: `https://signed.example.com/editor/${mediaType}/${file.name}`,
legacyPublicPath: `/generated-character-drafts/editor/asset-library/${mediaType}/${file.name}`,
objectKey: `generated-character-drafts/editor/asset-library/${mediaType}/${file.name}`,
assetObjectId: `assetobj-editor-${mediaType}`,
}),
);
});
it('keeps only one default asset folder when the persisted library returns duplicated defaults', async () => {
loadEditorAssetLibraryMock.mockResolvedValueOnce({
folders: [
@@ -202,7 +218,7 @@ describe('ImageCanvasEditorView asset library integration', () => {
await user.type(folderNameInput, '角色上传');
await user.click(screen.getByRole('button', { name: '保存素材文件夹' }));
const uploadInput = screen.getByLabelText('上传图片文件');
const uploadInput = screen.getByLabelText('上传媒体文件');
await user.click(screen.getByRole('button', { name: '上传到角色上传' }));
await userEvent.upload(
uploadInput,
@@ -346,7 +362,7 @@ describe('ImageCanvasEditorView asset library integration', () => {
it('uploads multiple files as account-level assets without adding canvas layers', async () => {
render(<ImageCanvasEditorView />);
await userEvent.upload(screen.getByLabelText('上传图片文件'), [
await userEvent.upload(screen.getByLabelText('上传媒体文件'), [
new File(['image-a'], '第一张.png', { type: 'image/png' }),
new File(['image-b'], '第二张.png', { type: 'image/png' }),
]);
@@ -364,6 +380,79 @@ describe('ImageCanvasEditorView asset library integration', () => {
expect(screen.queryByAltText('画布图片：第二张.png')).toBeNull();
});
it('uploads MP3 and MP4 files through OSS before adding them to the asset library', async () => {
const createdMediaElements: HTMLMediaElement[] = [];
const originalCreateElement = document.createElement.bind(document);
vi.spyOn(document, 'createElement').mockImplementation((tagName, options) => {
const element = originalCreateElement(tagName, options);
if (tagName === 'video' || tagName === 'audio') {
createdMediaElements.push(element as HTMLMediaElement);
}
return element;
});
Object.defineProperty(URL, 'createObjectURL', {
configurable: true,
value: vi.fn(() => 'blob:editor-media'),
});
Object.defineProperty(URL, 'revokeObjectURL', {
configurable: true,
value: vi.fn(),
});
Object.defineProperty(HTMLMediaElement.prototype, 'duration', {
configurable: true,
value: 6,
});
vi.spyOn(HTMLMediaElement.prototype, 'load').mockImplementation(
() => undefined,
);
render(<ImageCanvasEditorView />);
await userEvent.upload(screen.getByLabelText('上传媒体文件'), [
new File(['audio'], '胜利音效.mp3', { type: 'audio/mpeg' }),
new File(['video'], '开场动画.mp4', { type: 'video/mp4' }),
]);
act(() => {
createdMediaElements.forEach((element) =>
element.onloadedmetadata?.(new Event('loadedmetadata')),
);
});
await waitFor(() => {
expect(uploadEditorMediaAssetFileMock).toHaveBeenCalledWith(
expect.objectContaining({ name: '胜利音效.mp3' }),
'audio',
);
expect(uploadEditorMediaAssetFileMock).toHaveBeenCalledWith(
expect.objectContaining({ name: '开场动画.mp4' }),
'video',
);
});
await waitFor(() => {
expect(screen.getByRole('button', { name: '添加胜利音效.mp3' })).toBeTruthy();
expect(screen.getByRole('button', { name: '添加开场动画.mp4' })).toBeTruthy();
});
expect(createEditorAssetMock).toHaveBeenCalledWith(
expect.objectContaining({
label: '胜利音效.mp3',
imageSrc:
'/generated-character-drafts/editor/asset-library/audio/胜利音效.mp3',
objectKey:
'generated-character-drafts/editor/asset-library/audio/胜利音效.mp3',
assetObjectId: 'assetobj-editor-audio',
}),
);
expect(createEditorAssetMock).toHaveBeenCalledWith(
expect.objectContaining({
label: '开场动画.mp4',
imageSrc:
'/generated-character-drafts/editor/asset-library/video/开场动画.mp4',
objectKey:
'generated-character-drafts/editor/asset-library/video/开场动画.mp4',
assetObjectId: 'assetobj-editor-video',
}),
);
});
it('opens login before uploading assets while logged out and resumes after login', async () => {
const openLoginModal = vi.fn();
const authValue = createAuthValue({ openLoginModal });
@@ -374,7 +463,7 @@ describe('ImageCanvasEditorView asset library integration', () => {
</AuthUiContext.Provider>,
);
await userEvent.upload(screen.getByLabelText('上传图片文件'), [
await userEvent.upload(screen.getByLabelText('上传媒体文件'), [
new File(['image'], '登录后上传.png', { type: 'image/png' }),
]);
@@ -429,7 +518,7 @@ describe('ImageCanvasEditorView asset library integration', () => {
createEditorAssetMock.mockReturnValueOnce(deferredAsset.promise);
render(<ImageCanvasEditorView />);
await userEvent.upload(screen.getByLabelText('上传图片文件'), [
await userEvent.upload(screen.getByLabelText('上传媒体文件'), [
new File(['image'], '素材上传进度.png', { type: 'image/png' }),
]);
@@ -489,7 +578,7 @@ describe('ImageCanvasEditorView asset library integration', () => {
</AuthUiContext.Provider>,
);
await userEvent.upload(screen.getByLabelText('上传图片文件'), [
await userEvent.upload(screen.getByLabelText('上传媒体文件'), [
new File(['image'], '过期登录.png', { type: 'image/png' }),
]);
src/components/image-editor/ImageCanvasEditorModel.test.ts
+72 -20
View File
@@ -50,6 +50,58 @@ describe('ImageCanvasEditorModel', () => {
expect(library.folders[0]?.id).toBe('project');
});
it('infers MP3 and MP4 asset media types from persisted paths', () => {
const library = normalizeAssetLibrary({
folders: [
{
folderId: 'project',
label: '项目素材',
sortOrder: 0,
collapsed: false,
systemDefault: true,
},
],
assets: [
{
assetId: 'asset-audio',
folderId: 'project',
label: '胜利音效.mp3',
imageSrc: '/generated-character-drafts/editor/asset-library/audio/胜利音效.mp3',
width: 420,
height: 120,
sourceType: 'uploaded',
objectKey:
'generated-character-drafts/editor/asset-library/audio/胜利音效.mp3',
},
{
assetId: 'asset-video',
folderId: 'project',
label: '开场动画.mp4',
imageSrc: '/generated-character-drafts/editor/asset-library/video/开场动画.mp4',
width: 560,
height: 315,
sourceType: 'uploaded',
objectKey:
'generated-character-drafts/editor/asset-library/video/开场动画.mp4',
},
],
});
expect(library.assets[0]).toMatchObject({ mediaType: 'audio' });
expect(library.assets[1]).toMatchObject({ mediaType: 'video' });
expect(
createLayerFromAsset(
library.assets[0]!,
1,
{ x: 0, y: 0, scale: 1 },
{ x: 300, y: 200 },
),
).toMatchObject({
mediaType: 'audio',
assetKind: 'sound-effect',
});
});
it('creates a layer from an account asset at the requested screen point', () => {
const asset: EditorAsset = {
id: 'asset-1',
@@ -141,30 +193,30 @@ describe('ImageCanvasEditorModel', () => {
originalWidth: 420,
originalHeight: 120,
zIndex: 5,
sourceType: 'generated',
mediaType: 'audio',
assetKind: 'sound-effect',
durationSeconds: 2.4,
},
new Map([
[
'resource-audio',
{
imageSrc: '/generated-character-drafts/editor-audios/sfx.mp3',
durationSeconds: 2.4,
},
],
]),
);
sourceType: 'generated',
mediaType: 'audio',
assetKind: 'sound-effect',
durationSeconds: 2.4,
},
new Map([
[
'resource-audio',
{
imageSrc: '/generated-character-drafts/editor-audios/sfx.mp3',
durationSeconds: 2.4,
},
],
]),
);
expect(hydrated).toMatchObject({
id: 'layer-audio',
src: '/generated-character-drafts/editor-audios/sfx.mp3',
mediaType: 'audio',
assetKind: 'sound-effect',
durationSeconds: 2.4,
src: '/generated-character-drafts/editor-audios/sfx.mp3',
mediaType: 'audio',
assetKind: 'sound-effect',
durationSeconds: 2.4,
});
});
});
it('restores audio generation dialogs from saved layout', () => {
const dialog: CanvasGenerationDialogState = {
src/components/image-editor/ImageCanvasEditorModel.ts
+55 -19
View File
@@ -114,12 +114,19 @@ export function createLayerFromAsset(
const worldCenterX = (safeScreenCenter.x - viewport.x) / safeScale;
const worldCenterY = (safeScreenCenter.y - viewport.y) / safeScale;
const offset = index * 34;
const assetKind: CanvasAssetKind | undefined =
asset.mediaType === 'video'
? 'video'
: asset.mediaType === 'audio'
? inferAudioAssetKindFromLabel(asset.label)
: undefined;
return {
id: `layer-${asset.id}-${index}`,
resourceId: `local-resource-${asset.id}-${index}`,
title: asset.label,
src: asset.src,
mediaType: asset.mediaType,
x: worldCenterX - width / 2 + offset,
y: worldCenterY - height / 2 + offset,
width,
@@ -137,6 +144,7 @@ export function createLayerFromAsset(
assetObjectId: asset.assetObjectId,
durationSeconds: asset.durationSeconds,
sourceAssetId: asset.id,
assetKind,
} satisfies CanvasLayer;
}
@@ -436,28 +444,56 @@ export function mapAssetLibrarySnapshot(
systemDefault: folder.systemDefault,
persisted: true,
})),
assets: library.assets.map((asset) => ({
id: asset.assetId,
label: asset.label,
src: asset.imageSrc,
width: asset.width,
height: asset.height,
folderId: asset.folderId,
sourceKind: 'uploaded',
sourceType: asset.sourceType,
persisted: true,
prompt: asset.prompt ?? undefined,
actualPrompt: asset.actualPrompt ?? undefined,
model: asset.model ?? undefined,
provider: asset.provider ?? undefined,
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
durationSeconds: asset.durationSeconds ?? undefined,
})),
assets: library.assets.map((asset) => {
const mediaType = inferEditorAssetMediaType(
asset.imageSrc,
asset.objectKey ?? undefined,
);
return {
id: asset.assetId,
label: asset.label,
src: asset.imageSrc,
mediaType,
width: asset.width,
height: asset.height,
folderId: asset.folderId,
sourceKind: 'uploaded',
sourceType: asset.sourceType,
persisted: true,
prompt: asset.prompt ?? undefined,
actualPrompt: asset.actualPrompt ?? undefined,
model: asset.model ?? undefined,
provider: asset.provider ?? undefined,
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
durationSeconds: asset.durationSeconds ?? undefined,
};
}),
};
}
export function inferEditorAssetMediaType(
imageSrc: string,
objectKey?: string | null,
): CanvasMediaType {
const source = `${objectKey ?? ''} ${imageSrc}`.toLowerCase();
const sourceWithoutQuery = source.split(/[?#]/u)[0] ?? source;
if (sourceWithoutQuery.includes('.mp4')) {
return 'video';
}
if (sourceWithoutQuery.includes('.mp3')) {
return 'audio';
}
return 'image';
}
function inferAudioAssetKindFromLabel(label: string): CanvasAssetKind {
return label.includes('背景音乐') || label.toLowerCase().includes('music')
? 'background-music'
: 'sound-effect';
}
export function normalizeAssetLibrary(library: EditorAssetLibrarySnapshot) {
const mapped = mapAssetLibrarySnapshot(library);
let hasDefaultFolder = false;
src/components/image-editor/ImageCanvasEditorView.test-utils.ts
+3
View File
@@ -261,9 +261,12 @@ export function setupImageCanvasEditorViewTestLifecycle({
folderId: input.folderId,
label: input.label,
imageSrc: input.imageSrc,
objectKey: input.objectKey,
assetObjectId: input.assetObjectId,
width: input.width,
height: input.height,
sourceType: input.sourceType,
durationSeconds: input.durationSeconds,
}));
createEditorAssetFolderMock.mockResolvedValue({
folderId: 'folder-role-persisted',
src/components/image-editor/ImageCanvasEditorView.tsx
+3
View File
@@ -340,6 +340,7 @@ export function ImageCanvasEditorView() {
model: layer.model,
provider: layer.provider,
taskId: layer.taskId,
durationSeconds: layer.durationSeconds,
})
.then((asset) => {
setAssets((currentAssets) => [
@@ -350,6 +351,7 @@ export function ImageCanvasEditorView() {
id: asset.assetId,
label: asset.label,
src: asset.imageSrc,
mediaType: layer.mediaType,
width: asset.width,
height: asset.height,
folderId: asset.folderId,
@@ -363,6 +365,7 @@ export function ImageCanvasEditorView() {
taskId: asset.taskId ?? undefined,
objectKey: asset.objectKey ?? undefined,
assetObjectId: asset.assetObjectId ?? undefined,
durationSeconds: asset.durationSeconds ?? layer.durationSeconds,
},
]);
setAssetFolders((currentFolders) =>
src/components/image-editor/ImageCanvasFileModel.ts
+34 -2
View File
@@ -2,6 +2,9 @@ export function isImageFile(file: File) {
return file.type.startsWith('image/');
}
export const EDITOR_ASSET_UPLOAD_ACCEPT =
'image/*,.mp3,.mp4,audio/mpeg,audio/mp3,video/mp4';
export const SEEDANCE_REFERENCE_ACCEPT = {
image:
'.jpg,.jpeg,.png,.webp,.bmp,.tiff,.tif,.gif,.heic,.heif,image/jpeg,image/png,image/webp,image/bmp,image/tiff,image/gif,image/heic,image/heif',
@@ -19,7 +22,7 @@ export function getEditorUploadAccept(target: string) {
if (target === 'video-reference-image') {
return SEEDANCE_REFERENCE_ACCEPT.image;
}
return 'image/*';
return EDITOR_ASSET_UPLOAD_ACCEPT;
}
export const SEEDANCE_REFERENCE_FILE_LIMITS = {
@@ -45,6 +48,8 @@ const SEEDANCE_IMAGE_EXTENSIONS = new Set([
]);
const SEEDANCE_VIDEO_EXTENSIONS = new Set(['mp4', 'mov']);
const SEEDANCE_AUDIO_EXTENSIONS = new Set(['wav', 'mp3']);
const EDITOR_ASSET_VIDEO_EXTENSIONS = new Set(['mp4']);
const EDITOR_ASSET_AUDIO_EXTENSIONS = new Set(['mp3']);
function getFileExtension(file: File) {
return file.name.split('.').pop()?.toLowerCase() ?? '';
@@ -77,10 +82,37 @@ export function isSeedanceReferenceAudioFile(file: File) {
file.type === 'audio/mpeg' ||
file.type === 'audio/mp3' ||
file.type === '') &&
hasAllowedExtension(file, SEEDANCE_AUDIO_EXTENSIONS)
hasAllowedExtension(file, SEEDANCE_AUDIO_EXTENSIONS)
);
}
export function isEditorAssetVideoFile(file: File) {
return (
(file.type === 'video/mp4' || file.type === '') &&
hasAllowedExtension(file, EDITOR_ASSET_VIDEO_EXTENSIONS)
);
}
export function isEditorAssetAudioFile(file: File) {
return (
(file.type === 'audio/mpeg' || file.type === 'audio/mp3' || file.type === '') &&
hasAllowedExtension(file, EDITOR_ASSET_AUDIO_EXTENSIONS)
);
}
export function resolveEditorAssetUploadMediaType(file: File) {
if (isEditorAssetVideoFile(file)) {
return 'video' as const;
}
if (isEditorAssetAudioFile(file)) {
return 'audio' as const;
}
if (isImageFile(file)) {
return 'image' as const;
}
return null;
}
export function readFileAsDataUrl(file: File) {
return new Promise<string>((resolve, reject) => {
const reader = new FileReader();
src/components/image-editor/ImageCanvasGenerationLayerModel.test.ts
+38
View File
@@ -10,6 +10,7 @@ import {
createGeneratedResultLayer,
createIconSpritesheetResultLayers,
createQuickEditResultLayer,
createVideoResultLayer,
} from './ImageCanvasGenerationLayerModel';
function createGenerated(overrides = {}) {
@@ -267,6 +268,8 @@ describe('ImageCanvasGenerationLayerModel', () => {
const layer = createAudioResultLayer({
generated: {
audioSrc: '/generated-character-drafts/editor-audios/sfx.mp3',
objectKey: 'generated-character-drafts/editor-audios/sfx.mp3',
assetObjectId: 'assetobj-audio-1',
width: 420,
height: 120,
sourceType: 'generated',
@@ -315,10 +318,45 @@ describe('ImageCanvasGenerationLayerModel', () => {
model: 'audio1.0',
provider: 'VectorEngine',
taskId: 'sound-task-1',
objectKey: 'generated-character-drafts/editor-audios/sfx.mp3',
assetObjectId: 'assetobj-audio-1',
durationSeconds: 75,
});
});
it('creates video result layers with OSS asset identity', () => {
const layer = createVideoResultLayer({
generated: {
videoSrc: '/generated-character-drafts/editor-videos/video.mp4',
objectKey: 'generated-character-drafts/editor-videos/video.mp4',
assetObjectId: 'assetobj-video-1',
width: 1280,
height: 720,
sourceType: 'generated',
prompt: '开场动画',
actualPrompt: '开场动画',
model: 'seedance2.0-fast',
provider: 'VectorEngine',
taskId: 'video-task-1',
priceMudPoints: 40,
},
generatedIndex: 15,
title: '生成视频 15',
canvasSize: { width: 900, height: 640 },
viewport: { x: 10, y: 20, scale: 2 },
generationInputs: createGenerationInputs(),
});
expect(layer).toMatchObject({
id: 'layer-video-15',
src: '/generated-character-drafts/editor-videos/video.mp4',
mediaType: 'video',
assetKind: 'video',
objectKey: 'generated-character-drafts/editor-videos/video.mp4',
assetObjectId: 'assetobj-video-1',
});
});
it('creates character animation result layers with action metadata', () => {
const layer = createCharacterAnimationResultLayer({
generated: {
src/components/image-editor/ImageCanvasGenerationLayerModel.ts
+4
View File
@@ -365,6 +365,8 @@ export function createVideoResultLayer({
model: generated.model,
provider: generated.provider,
taskId: generated.taskId,
objectKey: generated.objectKey,
assetObjectId: generated.assetObjectId,
generationInputs,
};
}
@@ -415,6 +417,8 @@ export function createAudioResultLayer({
model: generated.model,
provider: generated.provider,
taskId: generated.taskId,
objectKey: generated.objectKey,
assetObjectId: generated.assetObjectId,
durationSeconds: generated.durationSeconds ?? undefined,
generationInputs,
};
src/components/image-editor/ImageCanvasUploadModel.ts
+56 -11
View File
@@ -18,10 +18,12 @@ type PersistedUploadAsset = {
folderId: string;
label: string;
imageSrc: string;
mediaType?: CanvasMediaType;
width: number;
height: number;
objectKey?: string | null;
assetObjectId?: string | null;
durationSeconds?: number | null;
};
type GenerationReferenceUploadTarget =
| 'character-reference'
@@ -46,6 +48,24 @@ export const UPLOAD_LAYER_FALLBACK_SIZE = {
width: 420,
height: 315,
} as const;
export const UPLOAD_VIDEO_FALLBACK_SIZE = {
width: 560,
height: 315,
} as const;
export const UPLOAD_AUDIO_FALLBACK_SIZE = {
width: 420,
height: 120,
} as const;
export function resolveUploadFallbackSize(mediaType: CanvasMediaType) {
if (mediaType === 'video') {
return UPLOAD_VIDEO_FALLBACK_SIZE;
}
if (mediaType === 'audio') {
return UPLOAD_AUDIO_FALLBACK_SIZE;
}
return UPLOAD_LAYER_FALLBACK_SIZE;
}
export function resolveUploadFolderId({
assetFolders,
@@ -259,21 +279,28 @@ export function createUploadingAssetPlaceholder({
uploadIndex,
fileName,
folderId,
mediaType = 'image',
durationSeconds,
}: {
uploadIndex: number;
fileName?: string;
folderId: string;
mediaType?: CanvasMediaType;
durationSeconds?: number;
}): EditorAsset {
const fallbackSize = resolveUploadFallbackSize(mediaType);
return {
id: `upload-${uploadIndex}`,
label: fileName || '上传图片',
label: fileName || (mediaType === 'audio' ? '上传音频' : mediaType === 'video' ? '上传视频' : '上传图片'),
src: '',
width: UPLOAD_LAYER_FALLBACK_SIZE.width,
height: UPLOAD_LAYER_FALLBACK_SIZE.height,
mediaType,
width: fallbackSize.width,
height: fallbackSize.height,
folderId,
sourceKind: 'uploaded',
sourceType: 'uploaded',
persisted: false,
durationSeconds,
uploadStatus: 'uploading',
uploadProgress: 8,
uploadMessage: '准备上传',
@@ -312,7 +339,7 @@ export function applyUploadAssetReadSuccess({
...asset,
src: imageSrc,
uploadProgress: 42,
uploadMessage: '读取图片',
uploadMessage: '读取素材',
}
: asset,
);
@@ -372,10 +399,12 @@ export function applyPersistedUploadAsset({
folderId: persistedAsset.folderId,
label: persistedAsset.label,
src: persistedAsset.imageSrc,
mediaType: persistedAsset.mediaType ?? asset.mediaType,
width: persistedAsset.width,
height: persistedAsset.height,
objectKey: persistedAsset.objectKey ?? undefined,
assetObjectId: persistedAsset.assetObjectId ?? undefined,
durationSeconds: persistedAsset.durationSeconds ?? asset.durationSeconds,
persisted: true,
uploadStatus: undefined,
uploadProgress: undefined,
@@ -442,8 +471,11 @@ export function bindUploadLayerToPersistedAsset({
? {
...layer,
sourceAssetId: persistedAsset.assetId,
mediaType: persistedAsset.mediaType ?? layer.mediaType,
objectKey: persistedAsset.objectKey ?? layer.objectKey,
assetObjectId: persistedAsset.assetObjectId ?? layer.assetObjectId,
durationSeconds:
persistedAsset.durationSeconds ?? layer.durationSeconds,
}
: layer,
);
@@ -471,6 +503,8 @@ export function createUploadCanvasLayer({
uploadIndex,
fileName,
imageSrc,
mediaType = 'image',
durationSeconds,
canvasPoint,
canvasSize,
viewport,
@@ -478,10 +512,13 @@ export function createUploadCanvasLayer({
uploadIndex: number;
fileName?: string;
imageSrc: string;
mediaType?: CanvasMediaType;
durationSeconds?: number;
canvasPoint?: CanvasPoint;
canvasSize: CanvasSize;
viewport: CanvasViewport;
}): CanvasLayer {
const fallbackSize = resolveUploadFallbackSize(mediaType);
const screenPoint = normalizeUploadScreenPoint({ canvasPoint, canvasSize });
const safeScale = viewport.scale > 0 ? viewport.scale : 1;
const worldCenterX = (screenPoint.x - viewport.x) / safeScale;
@@ -489,17 +526,25 @@ export function createUploadCanvasLayer({
return {
id: `layer-upload-${uploadIndex}`,
resourceId: `local-resource-upload-${uploadIndex}`,
title: fileName || '上传图片',
title: fileName || (mediaType === 'audio' ? '上传音频' : mediaType === 'video' ? '上传视频' : '上传图片'),
src: imageSrc,
x: worldCenterX - UPLOAD_LAYER_FALLBACK_SIZE.width / 2,
y: worldCenterY - UPLOAD_LAYER_FALLBACK_SIZE.height / 2,
width: UPLOAD_LAYER_FALLBACK_SIZE.width,
height: UPLOAD_LAYER_FALLBACK_SIZE.height,
originalWidth: UPLOAD_LAYER_FALLBACK_SIZE.width,
originalHeight: UPLOAD_LAYER_FALLBACK_SIZE.height,
mediaType,
assetKind:
mediaType === 'video'
? 'video'
: mediaType === 'audio'
? 'sound-effect'
: undefined,
x: worldCenterX - fallbackSize.width / 2,
y: worldCenterY - fallbackSize.height / 2,
width: fallbackSize.width,
height: fallbackSize.height,
originalWidth: fallbackSize.width,
originalHeight: fallbackSize.height,
zIndex: uploadIndex + 10,
sourceType: 'uploaded',
sourceAssetId: `upload-${uploadIndex}`,
durationSeconds,
};
}
src/components/image-editor/ImageCanvasWorldView.tsx
+29 -10
View File
@@ -314,6 +314,32 @@ function ImageCanvasAudioLayerCard({
);
}
function ImageCanvasVideoLayer({
layer,
mediaTransform,
}: {
layer: CanvasLayer;
mediaTransform?: string;
}) {
const { resolvedUrl } = useResolvedAssetReadUrl(layer.src, {
refreshKey: layer.taskId ?? layer.resourceId,
});
return (
<video
className="image-canvas-editor__layer-video"
src={resolvedUrl ?? layer.src}
controls
playsInline
preload="metadata"
aria-label={`画布视频：${layer.title}`}
style={{ transform: mediaTransform }}
onClick={(event) => event.stopPropagation()}
onPointerDown={(event) => event.stopPropagation()}
/>
);
}
export function ImageCanvasWorldView({
viewport,
snapGuide,
@@ -415,16 +441,9 @@ export function ImageCanvasWorldView({
isHovered={isHovered}
/>
) : layer.mediaType === 'video' ? (
<video
className="image-canvas-editor__layer-video"
src={layer.src}
controls
playsInline
preload="metadata"
aria-label={`画布视频：${layer.title}`}
style={{ transform: mediaTransform }}
onClick={(event) => event.stopPropagation()}
onPointerDown={(event) => event.stopPropagation()}
<ImageCanvasVideoLayer
layer={layer}
mediaTransform={mediaTransform}
/>
) : (
<img
src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts
+1 -5
View File
@@ -147,11 +147,7 @@ export function useImageCanvasGenerationSubmissionWorkflow({
const addGeneratedLayersToCanvas = useCallback(
(nextLayers: CanvasLayer[]) => {
appendCanvasLayersWithResources(nextLayers);
nextLayers
.filter(
(layer) => layer.mediaType !== 'video' && layer.mediaType !== 'audio',
)
.forEach((layer) => persistGeneratedAsset?.(layer));
nextLayers.forEach((layer) => persistGeneratedAsset?.(layer));
},
[appendCanvasLayersWithResources, persistGeneratedAsset],
);
src/components/image-editor/useImageCanvasUploadWorkflow.test.tsx
+13
View File
@@ -24,6 +24,7 @@ import { useImageCanvasUploadWorkflow } from './useImageCanvasUploadWorkflow';
const createEditorAssetMock = vi.hoisted(() => vi.fn());
const uploadEditorSeedanceReferenceFileMock = vi.hoisted(() => vi.fn());
const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn());
let createdMediaElements: HTMLMediaElement[] = [];
vi.mock('../../services/image-editor/editorProjectClient', async () => {
@@ -40,6 +41,10 @@ vi.mock('../../services/image-editor/editorReferenceUploadClient', () => ({
uploadEditorSeedanceReferenceFile: uploadEditorSeedanceReferenceFileMock,
}));
vi.mock('../../services/image-editor/editorMediaAssetUploadClient', () => ({
uploadEditorMediaAssetFile: uploadEditorMediaAssetFileMock,
}));
function createDefaultFolder(): EditorAssetFolder {
return {
id: 'project',
@@ -282,6 +287,14 @@ describe('useImageCanvasUploadWorkflow', () => {
assetObjectId: `assetobj-${mediaType}`,
}),
);
uploadEditorMediaAssetFileMock.mockImplementation(
async (file: File, mediaType: 'video' | 'audio') => ({
src: `https://signed.example.com/editor/${mediaType}/${file.name}`,
legacyPublicPath: `/generated-character-drafts/editor/asset-library/${mediaType}/${file.name}`,
objectKey: `generated-character-drafts/editor/asset-library/${mediaType}/${file.name}`,
assetObjectId: `assetobj-editor-${mediaType}`,
}),
);
Object.defineProperty(HTMLMediaElement.prototype, 'duration', {
configurable: true,
value: 4,
src/components/image-editor/useImageCanvasUploadWorkflow.ts
+27 -4
View File
@@ -8,6 +8,7 @@ import {
} from 'react';
import { ApiClientError } from '../../services/apiClient';
import { uploadEditorMediaAssetFile } from '../../services/image-editor/editorMediaAssetUploadClient';
import { createEditorAsset } from '../../services/image-editor/editorProjectClient';
import { uploadEditorSeedanceReferenceFile } from '../../services/image-editor/editorReferenceUploadClient';
import type {
@@ -23,6 +24,7 @@ import type {
import {
SEEDANCE_REFERENCE_FILE_LIMITS,
isImageFile,
resolveEditorAssetUploadMediaType,
isSeedanceReferenceAudioFile,
isSeedanceReferenceImageFile,
isSeedanceReferenceVideoFile,
@@ -612,8 +614,9 @@ export function useImageCanvasUploadWorkflow({
const uploadAssetFile = useCallback(
async (file: File, options: UploadAssetFileOptions) => {
if (!isImageFile(file)) {
window.alert('请选择图片文件');
const mediaType = resolveEditorAssetUploadMediaType(file);
if (!mediaType) {
window.alert('请选择图片、MP3 或 MP4 文件');
return;
}
@@ -627,6 +630,7 @@ export function useImageCanvasUploadWorkflow({
uploadIndex,
fileName: file.name,
folderId: uploadFolderId,
mediaType,
});
setAssets((currentAssets) => [...currentAssets, uploadedAsset]);
setAssetFolders((currentFolders) =>
@@ -637,8 +641,22 @@ export function useImageCanvasUploadWorkflow({
);
let imageSrc = '';
let objectKey: string | null = null;
let assetObjectId: string | null = null;
let durationSeconds: number | undefined;
try {
imageSrc = await readImageFileAsDataUrl(file);
if (mediaType === 'image') {
imageSrc = await readImageFileAsDataUrl(file);
} else {
durationSeconds = await probeMediaDurationSeconds(
file,
mediaType,
).catch(() => undefined);
const uploadedMedia = await uploadEditorMediaAssetFile(file, mediaType);
imageSrc = uploadedMedia.legacyPublicPath;
objectKey = uploadedMedia.objectKey;
assetObjectId = uploadedMedia.assetObjectId;
}
setAssets((currentAssets) =>
applyUploadAssetReadSuccess({
assets: currentAssets,
@@ -660,6 +678,8 @@ export function useImageCanvasUploadWorkflow({
uploadIndex,
fileName: file.name,
imageSrc,
mediaType,
durationSeconds,
canvasPoint: options.canvasPoint,
canvasSize,
viewport,
@@ -680,9 +700,12 @@ export function useImageCanvasUploadWorkflow({
folderId: uploadFolderId,
label: uploadedAsset.label,
imageSrc,
...(objectKey ? { objectKey } : {}),
...(assetObjectId ? { assetObjectId } : {}),
width: uploadedAsset.width,
height: uploadedAsset.height,
sourceType: 'uploaded',
...(durationSeconds ? { durationSeconds } : {}),
})
.then((asset) => {
setAssets((currentAssets) =>
@@ -716,7 +739,7 @@ export function useImageCanvasUploadWorkflow({
);
});
if (imageSrc) {
if (mediaType === 'image' && imageSrc) {
const uploadedImage = new Image();
uploadedImage.onload = () => {
const originalWidth =
src/index.css
+10
View File
@@ -4047,6 +4047,16 @@ html[data-mobile-keyboard-open='true'] .platform-mobile-bottom-dock {
background-size: 1rem 1rem;
}
.image-canvas-editor__asset-media-overlay {
display: flex;
align-items: center;
justify-content: center;
width: 100%;
height: 100%;
color: #f8fafc;
background: linear-gradient(135deg, rgba(15, 23, 42, 0.72), rgba(20, 83, 45, 0.52));
}
.image-canvas-editor__layer img,
.image-canvas-editor__layer video {
width: 100%;
src/services/image-editor/editorMediaAssetUploadClient.test.ts
+120
View File
@@ -0,0 +1,120 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { requestJson } from '../apiClient';
import { getSignedAssetReadUrl } from '../assetReadUrlService';
import { uploadEditorMediaAssetFile } from './editorMediaAssetUploadClient';
vi.mock('../apiClient', () => ({
requestJson: vi.fn(),
}));
vi.mock('../assetReadUrlService', () => ({
getSignedAssetReadUrl: vi.fn(),
}));
const requestJsonMock = vi.mocked(requestJson);
const getSignedAssetReadUrlMock = vi.mocked(getSignedAssetReadUrl);
describe('editorMediaAssetUploadClient', () => {
beforeEach(() => {
vi.clearAllMocks();
vi.spyOn(Date, 'now').mockReturnValue(1771400000000);
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(null, { status: 200 })),
);
});
afterEach(() => {
vi.restoreAllMocks();
vi.unstubAllGlobals();
});
it('uploads editor MP4 assets through direct OSS upload and confirms object metadata', async () => {
requestJsonMock
.mockResolvedValueOnce({
upload: {
bucket: 'bucket',
host: 'https://oss.example.com',
objectKey:
'generated-character-drafts/editor/asset-library/video/demo.mp4',
legacyPublicPath:
'/generated-character-drafts/editor/asset-library/video/demo.mp4',
formFields: {
key: 'generated-character-drafts/editor/asset-library/video/demo.mp4',
policy: 'policy',
'x-oss-signature': 'signature',
},
},
})
.mockResolvedValueOnce({
assetObject: {
assetObjectId: 'assetobj-video',
objectKey:
'generated-character-drafts/editor/asset-library/video/demo.mp4',
assetKind: 'editor_uploaded_video',
},
});
getSignedAssetReadUrlMock.mockResolvedValueOnce(
'https://signed.example.com/demo.mp4',
);
const result = await uploadEditorMediaAssetFile(
new File(['video'], 'demo.mp4', { type: 'video/mp4' }),
'video',
);
const ticketBody = JSON.parse(
(requestJsonMock.mock.calls[0]?.[1] as RequestInit).body as string,
);
expect(ticketBody).toMatchObject({
legacyPrefix: 'generated-character-drafts',
pathSegments: ['editor', 'asset-library', 'video', '1771400000000'],
fileName: 'demo.mp4',
contentType: 'video/mp4',
access: 'private',
maxSizeBytes: 5,
successActionStatus: 200,
metadata: {
asset_kind: 'editor_uploaded_video',
editor_media_type: 'video',
},
});
expect(fetch).toHaveBeenCalledWith(
'https://oss.example.com',
expect.objectContaining({
method: 'POST',
body: expect.any(FormData),
}),
);
const confirmBody = JSON.parse(
(requestJsonMock.mock.calls[1]?.[1] as RequestInit).body as string,
);
expect(confirmBody).toMatchObject({
bucket: 'bucket',
objectKey: 'generated-character-drafts/editor/asset-library/video/demo.mp4',
contentType: 'video/mp4',
contentLength: 5,
assetKind: 'editor_uploaded_video',
accessPolicy: 'private',
entityId: 'editor-asset-library-video',
});
expect(getSignedAssetReadUrlMock).toHaveBeenCalledWith(
{
objectKey:
'generated-character-drafts/editor/asset-library/video/demo.mp4',
expireSeconds: 3600,
},
undefined,
{ bypassCache: true },
);
expect(result).toEqual({
src: 'https://signed.example.com/demo.mp4',
objectKey:
'generated-character-drafts/editor/asset-library/video/demo.mp4',
assetObjectId: 'assetobj-video',
legacyPublicPath:
'/generated-character-drafts/editor/asset-library/video/demo.mp4',
});
});
});

Some files were not shown because too many files have changed in this diff Show More