diff --git a/.codex/skills/genarrative-external-editor-api/SKILL.md b/.codex/skills/genarrative-external-editor-api/SKILL.md index 6a99b11c1..d8aa83ba8 100644 --- a/.codex/skills/genarrative-external-editor-api/SKILL.md +++ b/.codex/skills/genarrative-external-editor-api/SKILL.md @@ -32,7 +32,7 @@ Prefer `scripts/genarrative_external_api.py` for runnable REST calls. It uses on - Use stable references such as `objectKey`, project resource ID, or asset ID in generation requests. Use `/assets/read-url` only for temporary preview/download access. - Preserve both warning channels after completion. A general `warning` can coexist with `sliceWarning`; do not discard either. - Do not invent missing derivatives. A source-preserved warning means the main source remains usable but requested post-processing failed. A slice warning means the complete transparent sheet is usable but individual slices are absent. -- Keep generated artifacts in the canvas and asset library together. Character animation may need a post-completion library fallback from the first returned frame when no direct asset is present; the helper implements it. +- Keep generated artifacts in the canvas and asset library together. Character animation accepts `assetFolderId` and `assetLabel`; its completed result directly returns the final `assetKind="character-animation"` resource and asset with formal sequence fields. Do not create a duplicate first-frame record. ## Documentation Navigation @@ -107,6 +107,8 @@ client.generate_image( Helper convenience methods wait locally, but the server still uses short asynchronous submit/status requests. For durable caller-controlled orchestration, call `submit_generation`, persist its `operationId` and idempotency key, then call `get_generation` or `wait_for_generation`. +For character animation, pass the canvas session and asset label to `animate_character`. The helper submits asynchronously and returns the completed compact result containing the authoritative formal `resource` and `asset`; do not synthesize a library asset from the first frame. + ## Guardrails - Do not change the fixed production base URL in generated examples. diff --git a/.codex/skills/genarrative-external-editor-api/references/api-operations.md b/.codex/skills/genarrative-external-editor-api/references/api-operations.md index 06fbf30a1..1597a2a12 100644 --- a/.codex/skills/genarrative-external-editor-api/references/api-operations.md +++ b/.codex/skills/genarrative-external-editor-api/references/api-operations.md @@ -46,7 +46,7 @@ Every generation row requires a stable `Idempotency-Key` header and returns HTTP | Image edit/redraw | `/api/external/v1/editor/images/edits` | `prompt`, `sourceImageSrc` | `referenceImageSrcs`, `model`, `size`, `projectId`, `assetFolderId`, `assetLabel`, `sourceResourceId`, `targetLayerId`, `canvasCompletion` | | Icon spritesheet | `/api/external/v1/editor/icon-spritesheets/generations` | `referenceImageSrc`, `iconDescriptions` | `style`, `referenceImageSrcs`, `screenColor`, `model`, `aspectRatio`, `imageSize`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | | UI asset extraction | `/api/external/v1/editor/ui-designs/assets/extractions` | `sourceImageSrc`, `aspectRatio`, `imageSize` | `screenColor`, `model`, `referenceImageSrcs`, `projectId`, `assetFolderId`, `spritesheetLabel`, `canvasCompletion` | -| Character animation | `/api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `canvasCompletion` | +| Character animation | `/api/external/v1/editor/character-animations/generations` | `sourceLayerId`, `sourceImageSrc`, `sourceWidth`, `sourceHeight`, `promptText`, `resolution`, `ratio`, `frameCount`, `durationSeconds`, `model` | `projectId`, `sourceResourceId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | | Video generation | `/api/external/v1/editor/videos/generations` | `prompt`, `model`, `aspectRatio`, `durationSeconds`, `resolution`, `mode`, `sound` | `referenceImageSrcs`, `referenceVideoSrcs`, `referenceAudioSrcs`, `webSearchEnabled`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion` | | Sound effect | `/api/external/v1/editor/audios/sound-effects/generations` | `prompt`, `duration` | `model`, `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | | Background music | `/api/external/v1/editor/audios/background-music/generations` | `gptDescriptionPrompt`, `makeInstrumental` | `projectId`, `assetFolderId`, `assetLabel`, `canvasCompletion`, `generationInputs` | @@ -64,7 +64,8 @@ Supply the `operationId` returned by submission. Poll no faster than `pollAfterM - Pass `projectId` and `canvasCompletion` to write generated output into the canvas. - Pass `assetFolderId` plus `assetLabel` for image, edit, icon spritesheet, video, sound effect, and BGM operations when supported. - UI extraction uses `assetFolderId` and `spritesheetLabel`. -- Character animation does not accept the same library fields. If its completed compact result lacks a direct `asset`, create a library record from the first returned frame; do not duplicate one when an asset already exists. +- Character animation accepts `assetFolderId` and `assetLabel`. Its completed compact result directly returns the final `assetKind="character-animation"` resource and asset with `imageSequenceFrames` and `imageSequenceDurationMs`; never create a duplicate first-frame resource or asset. +- If a caller must manually create a `character-animation` resource or asset, put the authoritative frames and total sequence duration in `imageSequenceFrames` and `imageSequenceDurationMs`. Keep `generationInputs` replayable: it must not contain legacy runtime fields such as `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, or `durationSeconds`. - Reload project/library state after completion when full current state is required. ## Reference Field Mapping diff --git a/.codex/skills/genarrative-external-editor-api/references/capability-routing.md b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md index 6b7091c3c..82e70f48d 100644 --- a/.codex/skills/genarrative-external-editor-api/references/capability-routing.md +++ b/.codex/skills/genarrative-external-editor-api/references/capability-routing.md @@ -20,7 +20,7 @@ Before the first generation in a new conversation, obtain a canvas name unless t 2. Read the asset library. Reuse a folder with the same label or create one with the canvas name. 3. Retain `canvasName`, `projectId`, `assetFolderId`, and the current art spec in conversation state. -Generated artifacts must enter both the current canvas and its same-name library folder whenever the endpoint supports that invariant. Pass `projectId`, `assetFolderId`, the endpoint's label field, and `canvasCompletion`. Character animation may return no direct library asset; after completion, create one from the first returned frame only when the compact result still lacks an asset. +Generated artifacts must enter both the current canvas and its same-name library folder whenever the endpoint supports that invariant. Pass `projectId`, `assetFolderId`, the endpoint's label field, and `canvasCompletion`. Character animation returns the final formal resource and asset directly; use those records and never create a duplicate from the first frame. ## Art Spec Routing diff --git a/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md index fe62ee1ee..1f3d04e3c 100644 --- a/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md +++ b/.codex/skills/genarrative-external-editor-api/references/requests-and-outputs.md @@ -98,7 +98,9 @@ A minimal `canvasCompletion` is: `dialogId` is optional. Do not reconstruct canvas state from completion results. Reload the project and asset library when complete authoritative snapshots are needed. -Character animation may complete without a direct `asset` field. To preserve the canvas/library invariant, create a library asset from the first returned frame only if the compact result lacks one. Prefer `client.animate_character(..., canvasSession=session, canvasTitle="...")`, which implements this fallback. +Character animation accepts `assetFolderId` and `assetLabel` and persists the final transparent sequence directly. Its completed compact result includes the authoritative `assetKind="character-animation"` resource and asset with `imageSequenceFrames` and `imageSequenceDurationMs`. Use those records directly and never synthesize a duplicate asset from the first frame. + +For the lower-level asset/resource creation endpoints, `generationInputs` is replayable request context rather than a media-runtime container. When `assetKind` is `character-animation`, the server rejects legacy runtime keys including `characterAnimation`, `frames`, `previewVideoPath`, `frameCount`, `fps`, and `durationSeconds`; send the formal sequence through `imageSequenceFrames` and `imageSequenceDurationMs`. Internal processing audit keys such as `screenColorHex`, `mattingProvider`, and `mattingModel` are removed before persistence. ## Art Spec and Image Request @@ -178,7 +180,9 @@ For character animation from a local-only source, use actual dimensions and a st "ratio": "9:16", "frameCount": 40, "durationSeconds": 5, - "model": "seedance2.0-fast" + "model": "seedance2.0-fast", + "assetFolderId": "", + "assetLabel": "角色呼吸动画" } ``` diff --git a/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py index 0e45ccf6d..bce78e20e 100644 --- a/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py +++ b/.codex/skills/genarrative-external-editor-api/scripts/genarrative_external_api.py @@ -600,16 +600,14 @@ class GenarrativeExternalClient: source_layer_id: str, **fields: Any, ) -> Any: - session, asset_label = self._apply_canvas_session_fields( + self._apply_canvas_session_fields( fields, fields.get("canvasTitle", "角色动画"), source_width, source_height, - asset_label_field=None, + asset_label_field="assetLabel", ) prompt_text = self._apply_art_spec(fields, prompt_text) - fields.pop("assetFolderId", None) - fields.pop("assetLabel", None) idempotency_key = fields.pop("idempotencyKey", None) body = { "sourceLayerId": source_layer_id, @@ -624,34 +622,11 @@ class GenarrativeExternalClient: **fields, "model": "seedance2.0-fast", } - result = self.submit_and_wait_generation( + return self.submit_and_wait_generation( "/api/external/v1/editor/character-animations/generations", body, idempotency_key=idempotency_key, ) - if isinstance(session, dict) and isinstance(result, dict) and not result.get("asset"): - frames = result.get("frames") - first_frame = frames[0] if isinstance(frames, list) and frames else None - folder_id = normalize_optional_text(session.get("assetFolderId")) - if isinstance(first_frame, dict) and folder_id: - asset = self.create_asset( - folder_id, - asset_label, - first_frame["imageSrc"], - int(first_frame["width"]), - int(first_frame["height"]), - prompt=result.get("prompt"), - model=result.get("model"), - provider="ark", - taskId=result.get("taskId"), - assetKind="character-animation", - generationInputs={ - "frames": frames, - "previewVideoPath": result.get("previewVideoPath"), - }, - ) - result["asset"] = unwrap_envelope(asset).get("asset") - return result def generate_video(self, prompt: str, **fields: Any) -> Any: fields.pop("mode", None) @@ -732,14 +707,31 @@ def _self_test() -> None: "timeout": timeout, "headers": headers, }) - if path == "/api/external/v1/editor/assets": - return {"asset": {"assetId": "editor-asset-demo"}} generated = { "taskId": "task-demo", "model": "seedance2.0-fast", "prompt": "角色呼吸", "previewVideoPath": "/generated/preview.mp4", - "frames": [{"frameIndex": 1, "imageSrc": "/generated/frame01.png", "width": 512, "height": 768}], + "frames": [{"imageSrc": "/generated/frame01.png", "width": 512, "height": 768}], + "resource": { + "resourceId": "editor-resource-demo", + "assetKind": "character-animation", + "sourceResourceId": "editor-resource-preview-demo", + "imageSequenceFrames": [ + {"imageSrc": "/generated/frame01.png", "width": 512, "height": 768}, + {"imageSrc": "/generated/frame02.png", "width": 512, "height": 768}, + ], + "imageSequenceDurationMs": 4000, + }, + "asset": { + "assetId": "editor-asset-demo", + "assetKind": "character-animation", + "imageSequenceFrames": [ + {"imageSrc": "/generated/frame01.png", "width": 512, "height": 768}, + {"imageSrc": "/generated/frame02.png", "width": 512, "height": 768}, + ], + "imageSequenceDurationMs": 4000, + }, } if method == "POST": return {"operationId": "task-operation-demo", "status": "queued", "pollAfterMs": 1} @@ -760,10 +752,14 @@ def _self_test() -> None: assert calls[0]["timeout"] == DEFAULT_REQUEST_TIMEOUT_SECONDS assert calls[0]["headers"]["Idempotency-Key"] assert calls[0]["body"]["projectId"] == "proj-demo" + assert calls[0]["body"]["assetFolderId"] == "editor-asset-folder-demo" + assert calls[0]["body"]["assetLabel"] == "角色呼吸动画" assert calls[0]["body"]["canvasCompletion"]["title"] == "角色呼吸动画" assert calls[1]["path"] == "/api/external/v1/generations/task-operation-demo" - assert calls[2]["path"] == "/api/external/v1/editor/assets" assert result["asset"]["assetId"] == "editor-asset-demo" + assert result["asset"]["assetKind"] == "character-animation" + assert len(result["asset"]["imageSequenceFrames"]) == 2 + assert result["asset"]["imageSequenceDurationMs"] == 4000 calls.clear() client.generate_icon_spritesheet( "editor-resource-spec", diff --git a/apps/admin-web/src/api/adminApiTypes.ts b/apps/admin-web/src/api/adminApiTypes.ts index 43030fd35..38c4a8e10 100644 --- a/apps/admin-web/src/api/adminApiTypes.ts +++ b/apps/admin-web/src/api/adminApiTypes.ts @@ -332,6 +332,14 @@ export interface AdminEditorAssetListQuery { limit?: number | null; } +export interface AdminEditorImageSequenceFramePayload { + imageSrc: string; + objectKey?: string | null; + assetObjectId?: string | null; + width: number; + height: number; +} + export interface AdminEditorAssetPayload { assetId: string; ownerUserId: string; @@ -362,6 +370,8 @@ export interface AdminEditorAssetPayload { taskGenerator: string; taskCostMudPoints: number; children: AdminEditorAssetPayload[]; + imageSequenceFrames?: AdminEditorImageSequenceFramePayload[] | null; + imageSequenceDurationMs?: number | null; } export interface AdminEditorAssetListResponse { @@ -413,6 +423,8 @@ export interface AdminEditorShowcaseAssetPayload { rejectedAt?: string | null; updatedAt: string; showcaseCategory?: string | null; + imageSequenceFrames?: AdminEditorImageSequenceFramePayload[] | null; + imageSequenceDurationMs?: number | null; } export interface AdminEditorShowcaseListResponse { diff --git a/apps/admin-web/src/components/AdminEditorAssetMedia.test.tsx b/apps/admin-web/src/components/AdminEditorAssetMedia.test.tsx new file mode 100644 index 000000000..de3169194 --- /dev/null +++ b/apps/admin-web/src/components/AdminEditorAssetMedia.test.tsx @@ -0,0 +1,354 @@ +/* @vitest-environment jsdom */ + +import { act, fireEvent, render, screen, within } from '@testing-library/react'; +import { afterEach, expect, test, vi } from 'vitest'; + +import { getAdminAssetReadUrl } from '../api/adminApiClient'; +import { AdminEditorAssetPreviewDialog } from './AdminEditorAssetMedia'; + +vi.mock('../api/adminApiClient', () => ({ + getAdminAssetReadUrl: vi.fn(), + isAdminApiError: vi.fn( + (error: unknown) => + typeof error === 'object' && + error !== null && + 'status' in error && + typeof error.status === 'number', + ), +})); + +afterEach(() => { + vi.clearAllMocks(); + vi.useRealTimers(); +}); + +function createDeferred() { + let resolve!: (value: T | PromiseLike) => void; + let reject!: (reason?: unknown) => void; + const promise = new Promise((resolvePromise, rejectPromise) => { + resolve = resolvePromise; + reject = rejectPromise; + }); + return { promise, resolve, reject }; +} + +test('角色动作预览跨窗口回播时复用父级换签缓存并等待目标帧就绪', async () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2026-07-04T10:50:00Z')); + const frameObjectKeys = [ + 'generated-animations/editor/task-cache/frame00.png', + 'generated-animations/editor/task-cache/frame01.png', + 'generated-animations/editor/task-cache/frame02.png', + 'generated-animations/editor/task-cache/frame03.png', + 'generated-animations/editor/task-cache/frame04.png', + ] as const; + const delayedFrame = createDeferred<{ + read: { + objectKey: string; + signedUrl: string; + expiresAt: string; + }; + }>(); + vi.mocked(getAdminAssetReadUrl).mockImplementation((_token, request) => { + const objectKey = request.objectKey ?? ''; + if (objectKey === frameObjectKeys[3]) { + return delayedFrame.promise; + } + return Promise.resolve({ + read: { + objectKey, + signedUrl: `https://signed.example.com/${objectKey}`, + expiresAt: '2026-07-04T11:00:00Z', + }, + }); + }); + + render( + ({ + imageSrc: `/${objectKey}`, + objectKey, + width: 192, + height: 256, + })), + imageSequenceDurationMs: 500, + }} + token="admin-token" + onClose={vi.fn()} + />, + ); + + const dialog = screen.getByRole('dialog', { name: '素材预览' }); + fireEvent.click(within(dialog).getByRole('button', { name: '暂停角色动作' })); + await act(async () => { + await vi.advanceTimersByTimeAsync(120); + }); + const initialFrames = Array.from( + dialog.querySelectorAll( + '.admin-asset-query-sequence-frame', + ), + ); + expect(initialFrames).toHaveLength(3); + initialFrames.forEach((frame) => fireEvent.load(frame)); + + fireEvent.click(within(dialog).getByRole('button', { name: '播放角色动作' })); + for (const elapsedMs of [100, 40, 60, 40, 60]) { + await act(async () => { + await vi.advanceTimersByTimeAsync(elapsedMs); + }); + } + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: frameObjectKeys[3], + expireSeconds: 300, + }); + expect(resolveVisibleFrameSrc(dialog)).toBe( + `https://signed.example.com/${frameObjectKeys[2]}`, + ); + + await act(async () => { + delayedFrame.resolve({ + read: { + objectKey: frameObjectKeys[3], + signedUrl: `https://signed.example.com/${frameObjectKeys[3]}`, + expiresAt: '2026-07-04T11:00:00Z', + }, + }); + await Promise.resolve(); + }); + const fourthFrame = resolveFrameByObjectKey(dialog, frameObjectKeys[3]); + expect(fourthFrame?.style.opacity).toBe('0'); + expect(resolveVisibleFrameSrc(dialog)).toBe( + `https://signed.example.com/${frameObjectKeys[2]}`, + ); + fireEvent.load(fourthFrame!); + fireEvent.load(resolveFrameByObjectKey(dialog, frameObjectKeys[4])!); + + for (let index = 0; index < 3; index += 1) { + await act(async () => { + await vi.advanceTimersByTimeAsync(100); + }); + } + expect(within(dialog).getByText('1/5')).toBeTruthy(); + expect(resolveVisibleFrameSrc(dialog)).toBe( + `https://signed.example.com/${frameObjectKeys[0]}`, + ); + expect( + vi + .mocked(getAdminAssetReadUrl) + .mock.calls.filter( + ([, request]) => request.objectKey === frameObjectKeys[0], + ), + ).toHaveLength(1); + expect( + dialog.querySelectorAll('.admin-asset-query-sequence-frame').length, + ).toBeLessThanOrEqual(3); +}); + +test('角色动作预览打开超过五分钟后回绕播放会在过期窗口内自动换签', async () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2099-01-01T00:00:00Z')); + const frameObjectKeys = Array.from( + { length: 5 }, + (_, index) => `generated-animations/editor/task-expiry/frame0${index}.png`, + ); + const requestCounts = new Map(); + vi.mocked(getAdminAssetReadUrl).mockImplementation((_token, request) => { + const objectKey = request.objectKey ?? ''; + const requestCount = (requestCounts.get(objectKey) ?? 0) + 1; + requestCounts.set(objectKey, requestCount); + return Promise.resolve({ + read: { + objectKey, + signedUrl: `https://signed.example.com/v${requestCount}/${objectKey}`, + expiresAt: + requestCount === 1 ? '2099-01-01T00:05:00Z' : '2099-01-01T00:10:00Z', + }, + }); + }); + + render( + ({ + imageSrc: `/${objectKey}`, + objectKey, + width: 192, + height: 256, + })), + imageSequenceDurationMs: 5_000, + }} + token="admin-token" + onClose={vi.fn()} + />, + ); + + const dialog = screen.getByRole('dialog', { name: '素材预览' }); + fireEvent.click(within(dialog).getByRole('button', { name: '暂停角色动作' })); + await act(async () => { + await vi.advanceTimersByTimeAsync(120); + }); + loadMountedFrames(dialog); + + for (let index = 0; index < frameObjectKeys.length; index += 1) { + fireEvent.click( + within(dialog).getByRole('button', { name: '播放角色动作' }), + ); + await act(async () => { + await vi.advanceTimersByTimeAsync(1_000); + }); + fireEvent.click( + within(dialog).getByRole('button', { name: '暂停角色动作' }), + ); + await act(async () => { + await vi.advanceTimersByTimeAsync(120); + }); + loadMountedFrames(dialog); + } + expect(within(dialog).getByText('1/5')).toBeTruthy(); + frameObjectKeys.forEach((objectKey) => { + expect(requestCounts.get(objectKey)).toBe(1); + }); + + await act(async () => { + await vi.advanceTimersByTimeAsync(300_001); + }); + loadMountedFrames(dialog); + expect(resolveVisibleFrameSrc(dialog)).toContain('/v2/'); + + for (let index = 0; index < frameObjectKeys.length; index += 1) { + fireEvent.click( + within(dialog).getByRole('button', { name: '播放角色动作' }), + ); + await act(async () => { + await vi.advanceTimersByTimeAsync(1_000); + }); + fireEvent.click( + within(dialog).getByRole('button', { name: '暂停角色动作' }), + ); + await act(async () => { + await vi.advanceTimersByTimeAsync(120); + }); + loadMountedFrames(dialog); + } + + expect(within(dialog).getByText('1/5')).toBeTruthy(); + expect(resolveVisibleFrameSrc(dialog)).toBe( + `https://signed.example.com/v2/${frameObjectKeys[0]}`, + ); + frameObjectKeys.forEach((objectKey) => { + expect(requestCounts.get(objectKey)).toBe(2); + }); + expect(within(dialog).queryByText(/帧加载失败/u)).toBeNull(); +}); + +test('角色动作预览提前换签失败时保留已就绪帧并有限重试', async () => { + vi.useFakeTimers(); + vi.setSystemTime(new Date('2099-01-01T00:00:00Z')); + const frameObjectKeys = [ + 'generated-animations/editor/task-refresh-failure/frame00.png', + 'generated-animations/editor/task-refresh-failure/frame01.png', + ] as const; + const requestCounts = new Map(); + vi.mocked(getAdminAssetReadUrl).mockImplementation((_token, request) => { + const objectKey = request.objectKey ?? ''; + const requestCount = (requestCounts.get(objectKey) ?? 0) + 1; + requestCounts.set(objectKey, requestCount); + if (objectKey === frameObjectKeys[0] && requestCount > 1) { + return Promise.reject(new Error('refresh unavailable')); + } + return Promise.resolve({ + read: { + objectKey, + signedUrl: `https://signed.example.com/v${requestCount}/${objectKey}`, + expiresAt: + requestCount === 1 ? '2099-01-01T00:05:00Z' : '2099-01-01T00:10:00Z', + }, + }); + }); + + render( + ({ + imageSrc: `/${objectKey}`, + objectKey, + width: 192, + height: 256, + })), + imageSequenceDurationMs: 2_000, + }} + token="admin-token" + onClose={vi.fn()} + />, + ); + + const dialog = screen.getByRole('dialog', { name: '素材预览' }); + fireEvent.click(within(dialog).getByRole('button', { name: '暂停角色动作' })); + await act(async () => { + await vi.advanceTimersByTimeAsync(120); + }); + loadMountedFrames(dialog); + const readyFrameSrc = resolveVisibleFrameSrc(dialog); + expect(readyFrameSrc).toBe( + `https://signed.example.com/v1/${frameObjectKeys[0]}`, + ); + + await act(async () => { + await vi.advanceTimersByTimeAsync(270_100); + }); + expect(resolveVisibleFrameSrc(dialog)).toBe(readyFrameSrc); + expect(within(dialog).queryByText(/帧加载失败/u)).toBeNull(); + + for (const retryWindowMs of [450, 1_250, 3_050]) { + await act(async () => { + await vi.advanceTimersByTimeAsync(retryWindowMs); + }); + expect(resolveVisibleFrameSrc(dialog)).toBe(readyFrameSrc); + } + expect(requestCounts.get(frameObjectKeys[0])).toBe(5); + expect(within(dialog).queryByText(/帧加载失败/u)).toBeNull(); + + await act(async () => { + await vi.advanceTimersByTimeAsync(20_000); + }); + expect(requestCounts.get(frameObjectKeys[0])).toBe(5); + expect(resolveVisibleFrameSrc(dialog)).toBe(readyFrameSrc); +}); + +function loadMountedFrames(dialog: HTMLElement) { + dialog + .querySelectorAll('.admin-asset-query-sequence-frame') + .forEach((frame) => fireEvent.load(frame)); +} + +function resolveFrameByObjectKey(dialog: HTMLElement, objectKey: string) { + return Array.from( + dialog.querySelectorAll( + '.admin-asset-query-sequence-frame', + ), + ).find((frame) => frame.src.endsWith(objectKey)); +} + +function resolveVisibleFrameSrc(dialog: HTMLElement) { + return Array.from( + dialog.querySelectorAll( + '.admin-asset-query-sequence-frame', + ), + ).find((frame) => frame.style.opacity === '1')?.src; +} diff --git a/apps/admin-web/src/components/AdminEditorAssetMedia.tsx b/apps/admin-web/src/components/AdminEditorAssetMedia.tsx index 9e93be572..c2f6fba40 100644 --- a/apps/admin-web/src/components/AdminEditorAssetMedia.tsx +++ b/apps/admin-web/src/components/AdminEditorAssetMedia.tsx @@ -1,8 +1,13 @@ -import { X } from 'lucide-react'; -import { useCallback, useEffect, useState } from 'react'; +import { Pause, Play, X } from 'lucide-react'; +import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { + parseSignedReadUrlExpiresAtMs, + SIGNED_READ_URL_CACHE_SAFETY_WINDOW_MS, +} from '../../../../packages/shared/src/utils/signedReadUrlCache'; import type { AdminAssetReadUrlResponse } from '../api/adminApiClient'; import { getAdminAssetReadUrl, isAdminApiError } from '../api/adminApiClient'; +import type { AdminEditorImageSequenceFramePayload } from '../api/adminApiTypes'; const ADMIN_ASSET_READ_EXPIRE_SECONDS = 300; const ADMIN_ASSET_READ_DISPATCH_SPACING_MS = 40; @@ -18,6 +23,8 @@ export interface AdminPreviewableEditorAsset { objectKey?: string | null; assetKind?: string | null; thumbnailSrc?: string | null; + imageSequenceFrames?: AdminEditorImageSequenceFramePayload[] | null; + imageSequenceDurationMs?: number | null; } export function AdminEditorAssetThumbnail({ @@ -31,7 +38,7 @@ export function AdminEditorAssetThumbnail({ }) { const thumbnailSource = resolveAdminAssetThumbnailSource(entry); const { observeElement, shouldLoad } = useAdminAssetThumbnailVisibility(); - const imageSrc = useAdminResolvedAssetUrl( + const { resolvedUrl: imageSrc } = useAdminResolvedAssetUrl( token, thumbnailSource.src, thumbnailSource.objectKey, @@ -98,14 +105,53 @@ function AdminEditorAssetPreviewMedia({ token: string; }) { const mediaKind = resolveAdminAssetMediaKind(entry); + const label = entry.label || entry.assetId; + + if (mediaKind === 'image-sequence') { + const sequence = normalizeAdminImageSequence(entry); + return sequence ? ( + + ) : ( +
+ 序列数据损坏，无法预览 +
+ ); + } + + return ( + + ); +} + +function AdminEditorStandardAssetPreviewMedia({ + entry, + mediaKind, + token, +}: { + entry: AdminPreviewableEditorAsset; + mediaKind: Exclude; + token: string; +}) { const isAudio = mediaKind === 'audio'; const isVideo = mediaKind === 'video'; - const mediaSrc = useAdminResolvedAssetUrl( + const { resolvedUrl: mediaSrc } = useAdminResolvedAssetUrl( token, entry.imageSrc, entry.objectKey, ); - const posterSrc = useAdminResolvedAssetUrl( + const { resolvedUrl: posterSrc } = useAdminResolvedAssetUrl( token, isVideo ? (entry.thumbnailSrc ?? '') : '', null, @@ -162,8 +208,542 @@ function AdminEditorAssetPreviewMedia({ ); } +function AdminEditorImageSequenceFrame({ + entry, + frameKey, + resolvedUrl, + visible, + ready, + onReady, + onFailed, +}: { + entry: AdminPreviewableEditorAsset; + frameKey: string; + resolvedUrl: string; + visible: boolean; + ready: boolean; + onReady: (frameKey: string) => void; + onFailed: (frameKey: string) => void; +}) { + const failureReportedRef = useRef(false); + const reportFailure = useCallback(() => { + if (failureReportedRef.current) { + return; + } + failureReportedRef.current = true; + onFailed(frameKey); + }, [frameKey, onFailed]); + + if (!resolvedUrl) { + return null; + } + return ( + {visible onReady(frameKey)} + onError={reportFailure} + /> + ); +} + +function AdminEditorImageSequencePreview({ + entry, + frames, + durationMs, + token, +}: { + entry: AdminPreviewableEditorAsset; + frames: AdminEditorImageSequenceFramePayload[]; + durationMs: number; + token: string; +}) { + const frameItems = useMemo( + () => + frames.map((frame, index) => ({ + frame, + index, + key: [entry.assetId, frame.objectKey ?? '', frame.imageSrc, index].join( + ':', + ), + })), + [entry.assetId, frames], + ); + const sequenceKey = frameItems.map((item) => item.key).join('|'); + const frameCacheOwnerKey = `${token}\u0000${sequenceKey}`; + const firstFrameKey = frameItems[0]?.key ?? ''; + const [frameIndex, setFrameIndex] = useState(0); + const [isPlaying, setIsPlaying] = useState(true); + const { + frameCache, + resolveFrames, + markFrameReady, + markFrameFailed, + retryFailedFrames: clearFailedFrameCache, + } = useAdminImageSequenceFrameCache( + token, + frameCacheOwnerKey, + frameItems.length, + ); + const [visibleFrameKey, setVisibleFrameKey] = useState(firstFrameKey); + const failedFrameKeys = useMemo( + () => + new Set( + Array.from(frameCache.entries()).flatMap(([frameKey, cached]) => + cached.status === 'failed' ? [frameKey] : [], + ), + ), + [frameCache], + ); + const currentFrameItem = + frameItems[Math.min(frameIndex, frameItems.length - 1)]; + const currentFrameKey = currentFrameItem?.key ?? ''; + const mountedFrameKeys = useMemo(() => { + const nextMountedFrameKeys = new Set(); + if (visibleFrameKey && !failedFrameKeys.has(visibleFrameKey)) { + nextMountedFrameKeys.add(visibleFrameKey); + } + for ( + let offset = 0; + offset < frameItems.length && nextMountedFrameKeys.size < 3; + offset += 1 + ) { + const item = frameItems[(frameIndex + offset) % frameItems.length]; + if (item && !failedFrameKeys.has(item.key)) { + nextMountedFrameKeys.add(item.key); + } + } + return nextMountedFrameKeys; + }, [failedFrameKeys, frameIndex, frameItems, visibleFrameKey]); + const mountedFrameItems = useMemo( + () => frameItems.filter((item) => mountedFrameKeys.has(item.key)), + [frameItems, mountedFrameKeys], + ); + const availableFrameCount = frameItems.length - failedFrameKeys.size; + + const retryFailedFrames = useCallback(() => { + setFrameIndex(0); + setIsPlaying(true); + clearFailedFrameCache(); + }, [clearFailedFrameCache]); + + useEffect(() => { + setFrameIndex(0); + setIsPlaying(true); + setVisibleFrameKey(firstFrameKey); + }, [firstFrameKey, sequenceKey]); + + useEffect(() => { + resolveFrames(mountedFrameItems); + }, [mountedFrameItems, resolveFrames]); + + useEffect(() => { + const refreshAtMs = mountedFrameItems.reduce( + (earliest, item) => { + const cached = frameCache.get(item.key); + if ( + !cached || + cached.expiresAtMs === null || + cached.status === 'resolving' || + cached.status === 'refreshing' || + cached.status === 'failed' + ) { + return earliest; + } + const candidate = cached.nextRefreshAtMs; + if (candidate === null) { + return earliest; + } + return earliest === null || candidate < earliest ? candidate : earliest; + }, + null as number | null, + ); + if (refreshAtMs === null) { + return undefined; + } + const timer = window.setTimeout( + () => resolveFrames(mountedFrameItems), + Math.min(2_147_483_647, Math.max(0, refreshAtMs - Date.now())), + ); + return () => window.clearTimeout(timer); + }, [frameCache, mountedFrameItems, resolveFrames]); + + useEffect(() => { + if (!currentFrameKey || !failedFrameKeys.has(currentFrameKey)) { + return; + } + const nextIndex = findNextAvailableFrameIndex( + frameItems, + failedFrameKeys, + frameIndex, + ); + if (nextIndex === null) { + setVisibleFrameKey(''); + setIsPlaying(false); + return; + } + setFrameIndex(nextIndex); + }, [currentFrameKey, failedFrameKeys, frameIndex, frameItems]); + + useEffect(() => { + if (!currentFrameKey) { + setVisibleFrameKey(''); + return; + } + if ( + !isAdminImageSequenceFrameReady(frameCache.get(currentFrameKey)) && + visibleFrameKey + ) { + return; + } + setVisibleFrameKey(currentFrameKey); + }, [currentFrameKey, frameCache, visibleFrameKey]); + + useEffect(() => { + if (!isPlaying || availableFrameCount < 2) { + return undefined; + } + const frameIntervalMs = Math.min(2_147_483_647, durationMs / frames.length); + const timer = window.setInterval(() => { + setFrameIndex( + (currentIndex) => + findNextAvailableFrameIndex( + frameItems, + failedFrameKeys, + currentIndex, + ) ?? currentIndex, + ); + }, frameIntervalMs); + return () => window.clearInterval(timer); + }, [ + availableFrameCount, + durationMs, + failedFrameKeys, + frameItems, + frames.length, + isPlaying, + ]); + + return ( +
+ {mountedFrameItems.map((item) => { + const cachedFrame = frameCache.get(item.key); + return ( + + ); + })} +
+ + {`${currentFrameItem ? currentFrameItem.index + 1 : 0}/${frames.length}`} + {failedFrameKeys.size ? ( + <> + {`${failedFrameKeys.size} 帧加载失败`} + + + ) : null} +
+
+ ); +} + +type AdminImageSequenceFrameItem = { + frame: AdminEditorImageSequenceFramePayload; + index: number; + key: string; +}; + +type AdminImageSequenceFrameCacheEntry = { + expiresAtMs: number | null; + nextRefreshAtMs: number | null; + refreshRetryIndex: number; + resolvedUrl: string; + status: 'resolving' | 'refreshing' | 'loading' | 'ready' | 'failed'; +}; + +type AdminImageSequenceFrameCacheState = { + ownerKey: string; + entries: ReadonlyMap; +}; + +const EMPTY_ADMIN_IMAGE_SEQUENCE_FRAME_CACHE = new Map< + string, + AdminImageSequenceFrameCacheEntry +>(); + +function useAdminImageSequenceFrameCache( + token: string, + ownerKey: string, + maximumEntryCount: number, +) { + const [cacheState, setCacheState] = + useState(() => ({ + ownerKey, + entries: new Map(), + })); + const resolutionControllersRef = useRef(new Map()); + const frameCache = + cacheState.ownerKey === ownerKey + ? cacheState.entries + : EMPTY_ADMIN_IMAGE_SEQUENCE_FRAME_CACHE; + + useEffect(() => { + const resolutionControllers = resolutionControllersRef.current; + resolutionControllers.forEach((controller) => controller.abort()); + resolutionControllers.clear(); + setCacheState({ ownerKey, entries: new Map() }); + return () => { + resolutionControllers.forEach((controller) => controller.abort()); + resolutionControllers.clear(); + }; + }, [ownerKey]); + + const updateFrameCacheEntry = useCallback( + ( + frameKey: string, + update: ( + current: AdminImageSequenceFrameCacheEntry | undefined, + ) => AdminImageSequenceFrameCacheEntry | null, + ) => { + setCacheState((currentState) => { + const currentEntries = + currentState.ownerKey === ownerKey + ? currentState.entries + : new Map(); + const nextEntry = update(currentEntries.get(frameKey)); + if (nextEntry === currentEntries.get(frameKey)) { + return currentState; + } + const nextEntries = new Map(currentEntries); + if (nextEntry) { + if ( + !nextEntries.has(frameKey) && + nextEntries.size >= maximumEntryCount + ) { + return currentState; + } + nextEntries.set(frameKey, nextEntry); + } else { + nextEntries.delete(frameKey); + } + return { ownerKey, entries: nextEntries }; + }); + }, + [maximumEntryCount, ownerKey], + ); + + const resolveFrames = useCallback( + (frameItemsToResolve: ReadonlyArray) => { + for (const item of frameItemsToResolve) { + const cached = frameCache.get(item.key); + if (resolutionControllersRef.current.has(item.key)) { + continue; + } + if (cached && shouldReuseAdminImageSequenceFrameCacheEntry(cached)) { + continue; + } + const refreshReadyFrame = isAdminImageSequenceFrameReady(cached); + const controller = new AbortController(); + resolutionControllersRef.current.set(item.key, controller); + updateFrameCacheEntry(item.key, (current) => + refreshReadyFrame && current?.resolvedUrl + ? { ...current, status: 'refreshing' } + : { + expiresAtMs: null, + nextRefreshAtMs: null, + refreshRetryIndex: 0, + resolvedUrl: '', + status: 'resolving', + }, + ); + void resolveAdminAssetUrl( + token, + item.frame.imageSrc, + item.frame.objectKey, + controller.signal, + ) + .then((resolved) => { + updateFrameCacheEntry(item.key, () => ({ + ...resolved, + nextRefreshAtMs: + resolved.expiresAtMs === null + ? null + : resolved.expiresAtMs - + SIGNED_READ_URL_CACHE_SAFETY_WINDOW_MS, + refreshRetryIndex: 0, + status: refreshReadyFrame ? 'refreshing' : 'loading', + })); + }) + .catch((error: unknown) => { + if (!controller.signal.aborted) { + updateFrameCacheEntry(item.key, (current) => { + if (!isAdminImageSequenceFrameUrlUsable(current)) { + return { + expiresAtMs: null, + nextRefreshAtMs: null, + refreshRetryIndex: 0, + resolvedUrl: '', + status: 'failed', + }; + } + const retryDelay = + isAdminApiError(error) && error.status === 429 + ? undefined + : ADMIN_ASSET_READ_RETRY_DELAYS_MS[ + current.refreshRetryIndex + ]; + return { + ...current, + nextRefreshAtMs: + typeof retryDelay === 'number' + ? Math.min( + current.expiresAtMs ?? Number.POSITIVE_INFINITY, + Date.now() + retryDelay, + ) + : current.expiresAtMs, + refreshRetryIndex: + typeof retryDelay === 'number' + ? current.refreshRetryIndex + 1 + : current.refreshRetryIndex, + status: 'ready', + }; + }); + } + }) + .finally(() => { + if (resolutionControllersRef.current.get(item.key) === controller) { + resolutionControllersRef.current.delete(item.key); + } + }); + } + }, + [frameCache, token, updateFrameCacheEntry], + ); + + const markFrameReady = useCallback( + (frameKey: string) => { + updateFrameCacheEntry(frameKey, (current) => + current?.resolvedUrl + ? { ...current, status: 'ready' } + : (current ?? null), + ); + }, + [updateFrameCacheEntry], + ); + + const markFrameFailed = useCallback( + (frameKey: string) => { + resolutionControllersRef.current.get(frameKey)?.abort(); + resolutionControllersRef.current.delete(frameKey); + updateFrameCacheEntry(frameKey, () => ({ + expiresAtMs: null, + nextRefreshAtMs: null, + refreshRetryIndex: 0, + resolvedUrl: '', + status: 'failed', + })); + }, + [updateFrameCacheEntry], + ); + + const retryFailedFrames = useCallback(() => { + setCacheState((currentState) => { + if (currentState.ownerKey !== ownerKey) { + return { ownerKey, entries: new Map() }; + } + const nextEntries = new Map( + Array.from(currentState.entries).filter( + ([, cached]) => cached.status !== 'failed', + ), + ); + return { ownerKey, entries: nextEntries }; + }); + }, [ownerKey]); + + return { + frameCache, + resolveFrames, + markFrameReady, + markFrameFailed, + retryFailedFrames, + }; +} + +function isAdminImageSequenceFrameReady( + cached: AdminImageSequenceFrameCacheEntry | undefined, +) { + return cached?.status === 'ready' || cached?.status === 'refreshing'; +} + +function shouldReuseAdminImageSequenceFrameCacheEntry( + cached: AdminImageSequenceFrameCacheEntry, +) { + if (cached.status === 'failed') { + return false; + } + return cached.nextRefreshAtMs === null || cached.nextRefreshAtMs > Date.now(); +} + +function isAdminImageSequenceFrameUrlUsable( + cached: AdminImageSequenceFrameCacheEntry | undefined, +): cached is AdminImageSequenceFrameCacheEntry { + return Boolean( + cached?.resolvedUrl && + (cached.expiresAtMs === null || cached.expiresAtMs > Date.now()), + ); +} + +function findNextAvailableFrameIndex( + frameItems: ReadonlyArray<{ key: string }>, + failedFrameKeys: ReadonlySet, + currentIndex: number, +) { + for (let offset = 1; offset <= frameItems.length; offset += 1) { + const nextIndex = (currentIndex + offset) % frameItems.length; + const item = frameItems[nextIndex]; + if (item && !failedFrameKeys.has(item.key)) { + return nextIndex; + } + } + return null; +} + function resolveAdminAssetThumbnailSource(entry: AdminPreviewableEditorAsset) { const mediaKind = resolveAdminAssetMediaKind(entry); + if (mediaKind === 'image-sequence') { + const sequence = normalizeAdminImageSequence(entry); + const firstFrame = sequence?.frames[0]; + return firstFrame + ? { src: firstFrame.imageSrc, objectKey: firstFrame.objectKey ?? null } + : { src: '', objectKey: null }; + } if (mediaKind === 'audio') { return { src: AUDIO_ASSET_COVER_SRC, objectKey: null }; } @@ -216,19 +796,58 @@ function useAdminAssetThumbnailVisibility() { return { observeElement, shouldLoad }; } -type AdminAssetMediaKind = 'image' | 'audio' | 'video'; +type AdminAssetMediaKind = 'image' | 'audio' | 'video' | 'image-sequence'; + +function normalizeAdminImageSequence(entry: AdminPreviewableEditorAsset) { + const frames = entry.imageSequenceFrames; + const durationMs = entry.imageSequenceDurationMs; + if ( + !Array.isArray(frames) || + frames.length < 2 || + typeof durationMs !== 'number' || + !Number.isFinite(durationMs) || + durationMs <= 0 + ) { + return null; + } + const normalizedFrames = frames.flatMap((frame) => { + const imageSrc = frame.imageSrc?.trim() ?? ''; + if ( + !imageSrc || + !Number.isFinite(frame.width) || + frame.width <= 0 || + !Number.isFinite(frame.height) || + frame.height <= 0 + ) { + return []; + } + return [ + { + ...frame, + imageSrc, + objectKey: frame.objectKey?.trim() || null, + assetObjectId: frame.assetObjectId?.trim() || null, + }, + ]; + }); + return normalizedFrames.length === frames.length + ? { frames: normalizedFrames, durationMs } + : null; +} function resolveAdminAssetMediaKind( entry: AdminPreviewableEditorAsset, ): AdminAssetMediaKind { + const assetKind = entry.assetKind?.trim() ?? ''; + if (assetKind === 'character-animation') { + return 'image-sequence'; + } const pathMediaKind = resolveAdminAssetMediaKindFromPath(entry.imageSrc) ?? resolveAdminAssetMediaKindFromPath(entry.objectKey ?? ''); if (pathMediaKind) { return pathMediaKind; } - - const assetKind = entry.assetKind?.trim() ?? ''; if ( assetKind === 'sound-effect' || assetKind === 'background-music' || @@ -279,6 +898,7 @@ function useAdminResolvedAssetUrl( imageSrc: string | null | undefined, objectKey: string | null | undefined, enabled = true, + refreshKey: string | number | null = null, ) { const normalizedImageSrc = imageSrc?.trim() ?? ''; const normalizedObjectKey = normalizeAdminObjectKey(objectKey); @@ -290,18 +910,22 @@ function useAdminResolvedAssetUrl( const [resolvedImageSrc, setResolvedImageSrc] = useState( shouldResolve ? '' : normalizedImageSrc, ); + const [hasError, setHasError] = useState(false); useEffect(() => { if (!normalizedImageSrc && !normalizedObjectKey) { setResolvedImageSrc(''); + setHasError(false); return; } if (!shouldResolve) { setResolvedImageSrc(normalizedImageSrc); + setHasError(false); return; } if (!enabled) { setResolvedImageSrc(''); + setHasError(false); return; } @@ -310,6 +934,7 @@ function useAdminResolvedAssetUrl( let retryIndex = 0; const dispatchController = new AbortController(); setResolvedImageSrc(''); + setHasError(false); const resolveReadUrl = async () => { try { @@ -330,7 +955,9 @@ function useAdminResolvedAssetUrl( }, ); if (!cancelled) { - setResolvedImageSrc(resolveAdminAssetReadSignedUrl(response)); + const nextUrl = resolveAdminAssetReadSignedUrl(response); + setResolvedImageSrc(nextUrl); + setHasError(!nextUrl); } } catch (error: unknown) { if (cancelled) { @@ -347,6 +974,7 @@ function useAdminResolvedAssetUrl( return; } setResolvedImageSrc(''); + setHasError(true); } }; @@ -364,11 +992,84 @@ function useAdminResolvedAssetUrl( normalizedImageSrc, normalizedLegacyPublicPath, normalizedObjectKey, + refreshKey, shouldResolve, token, ]); - return resolvedImageSrc; + return { + resolvedUrl: resolvedImageSrc, + hasError, + }; +} + +async function resolveAdminAssetUrl( + token: string, + imageSrc: string | null | undefined, + objectKey: string | null | undefined, + signal: AbortSignal, +) { + const normalizedImageSrc = imageSrc?.trim() ?? ''; + const normalizedObjectKey = normalizeAdminObjectKey(objectKey); + const normalizedLegacyPublicPath = isGeneratedLegacyPath(normalizedImageSrc) + ? normalizedImageSrc + : resolveAdminGeneratedLegacyPathFromUrl(normalizedImageSrc); + if (!normalizedObjectKey && !normalizedLegacyPublicPath) { + return { + expiresAtMs: null, + resolvedUrl: normalizedImageSrc, + }; + } + + let retryIndex = 0; + while (!signal.aborted) { + try { + await waitForAdminAssetReadDispatch(signal); + const response = await getAdminAssetReadUrl( + token, + normalizedObjectKey + ? { + objectKey: normalizedObjectKey, + expireSeconds: ADMIN_ASSET_READ_EXPIRE_SECONDS, + } + : { + legacyPublicPath: normalizedLegacyPublicPath, + expireSeconds: ADMIN_ASSET_READ_EXPIRE_SECONDS, + }, + ); + if (signal.aborted) { + throw new DOMException('The operation was aborted.', 'AbortError'); + } + const read = response.read ?? response; + const resolvedUrl = resolveAdminAssetReadSignedUrl(response); + const expiresAt = + typeof read.expiresAt === 'string' ? read.expiresAt.trim() : ''; + const expiresAtMs = parseSignedReadUrlExpiresAtMs(expiresAt); + if (!resolvedUrl) { + throw new Error('素材读取地址为空'); + } + if (expiresAtMs <= Date.now()) { + throw new Error('素材读取地址过期时间无效'); + } + return { expiresAtMs, resolvedUrl }; + } catch (error: unknown) { + if (signal.aborted) { + throw error; + } + const retryDelay = ADMIN_ASSET_READ_RETRY_DELAYS_MS[retryIndex]; + if ( + isAdminApiError(error) && + error.status === 429 && + typeof retryDelay === 'number' + ) { + retryIndex += 1; + await waitForAdminAssetReadDelay(retryDelay, signal); + continue; + } + throw error; + } + } + throw new DOMException('The operation was aborted.', 'AbortError'); } async function waitForAdminAssetReadDispatch(signal: AbortSignal) { @@ -381,6 +1082,16 @@ async function waitForAdminAssetReadDispatch(signal: AbortSignal) { } async function waitForAdminAssetReadDispatchSpacing(signal: AbortSignal) { + await waitForAdminAssetReadDelay( + ADMIN_ASSET_READ_DISPATCH_SPACING_MS, + signal, + ); +} + +async function waitForAdminAssetReadDelay( + delayMs: number, + signal: AbortSignal, +) { if (signal.aborted) { throw new DOMException('The operation was aborted.', 'AbortError'); } @@ -388,7 +1099,7 @@ async function waitForAdminAssetReadDispatchSpacing(signal: AbortSignal) { const timer = setTimeout(() => { signal.removeEventListener('abort', handleAbort); resolve(); - }, ADMIN_ASSET_READ_DISPATCH_SPACING_MS); + }, delayMs); function handleAbort() { clearTimeout(timer); diff --git a/apps/admin-web/src/pages/AdminEditorAssetQueryPage.test.tsx b/apps/admin-web/src/pages/AdminEditorAssetQueryPage.test.tsx index 5bd550cd3..bedb0adb1 100644 --- a/apps/admin-web/src/pages/AdminEditorAssetQueryPage.test.tsx +++ b/apps/admin-web/src/pages/AdminEditorAssetQueryPage.test.tsx @@ -1068,7 +1068,7 @@ test('后台素材查询点击图片缩略图可打开放大预览', async () => }); }); -test('后台素材查询将角色动画首帧 PNG 作为图片预览', async () => { +test('后台素材查询在现有预览弹窗播放完整角色动作序列', async () => { const user = userEvent.setup(); vi.mocked(listAdminEditorAssets).mockResolvedValueOnce({ entries: [ @@ -1081,17 +1081,65 @@ test('后台素材查询将角色动画首帧 PNG 作为图片预览', async () assetKind: 'character-animation', thumbnailSrc: '/generated-animations/editor/source-1/task-1/frame00.png', + imageSequenceFrames: [ + { + imageSrc: + '/generated-animations/editor/source-1/task-1/frame00.png', + objectKey: + 'generated-animations/editor/source-1/task-1/frame00.png', + width: 192, + height: 256, + }, + { + imageSrc: + '/generated-animations/editor/source-1/task-1/frame01.png', + objectKey: + 'generated-animations/editor/source-1/task-1/frame01.png', + width: 192, + height: 256, + }, + { + imageSrc: + '/generated-animations/editor/source-1/task-1/frame02.png', + objectKey: + 'generated-animations/editor/source-1/task-1/frame02.png', + width: 192, + height: 256, + }, + { + imageSrc: + '/generated-animations/editor/source-1/task-1/frame03.png', + objectKey: + 'generated-animations/editor/source-1/task-1/frame03.png', + width: 192, + height: 256, + }, + { + imageSrc: + '/generated-animations/editor/source-1/task-1/frame04.png', + objectKey: + 'generated-animations/editor/source-1/task-1/frame04.png', + width: 192, + height: 256, + }, + ], + imageSequenceDurationMs: 250, }, ], nextCursor: null, }); - vi.mocked(getAdminAssetReadUrl).mockResolvedValue({ - read: { - objectKey: 'generated-animations/editor/source-1/task-1/frame00.png', - signedUrl: 'https://signed.example.com/character-animation-frame00.png', - expiresAt: '2026-07-04T11:00:00Z', + vi.mocked(getAdminAssetReadUrl).mockImplementation( + async (_token, request) => { + const objectKey = request.objectKey ?? undefined; + return { + read: { + objectKey, + signedUrl: `https://signed.example.com/${objectKey ?? ''}`, + expiresAt: '2099-01-01T00:05:00Z', + }, + }; }, - }); + ); render( , @@ -1102,15 +1150,56 @@ test('后台素材查询将角色动画首帧 PNG 作为图片预览', async () ); const dialog = await screen.findByRole('dialog', { name: '素材预览' }); - const image = await within(dialog).findByRole('img', { - name: '图片预览：角色动作首帧', - }); + await user.click( + await within(dialog).findByRole('button', { + name: '暂停角色动作', + }), + ); await waitFor(() => { - expect(image.getAttribute('src')).toBe( - 'https://signed.example.com/character-animation-frame00.png', + expect( + dialog.querySelectorAll('.admin-asset-query-sequence-frame'), + ).toHaveLength(3); + }); + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: 'generated-animations/editor/source-1/task-1/frame00.png', + expireSeconds: 300, + }); + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: 'generated-animations/editor/source-1/task-1/frame01.png', + expireSeconds: 300, + }); + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: 'generated-animations/editor/source-1/task-1/frame02.png', + expireSeconds: 300, + }); + expect(getAdminAssetReadUrl).not.toHaveBeenCalledWith('admin-token', { + objectKey: 'generated-animations/editor/source-1/task-1/frame03.png', + expireSeconds: 300, + }); + + const failedFrame = dialog.querySelector( + '.admin-asset-query-sequence-frame', + ) as HTMLImageElement; + fireEvent.error(failedFrame); + expect(await within(dialog).findByText('1 帧加载失败')).toBeTruthy(); + await waitFor(() => { + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: 'generated-animations/editor/source-1/task-1/frame03.png', + expireSeconds: 300, + }); + }); + const callsBeforeRetry = vi.mocked(getAdminAssetReadUrl).mock.calls.length; + await user.click( + within(dialog).getByRole('button', { name: '重试失败帧' }), + ); + await waitFor(() => { + expect(vi.mocked(getAdminAssetReadUrl).mock.calls.length).toBeGreaterThan( + callsBeforeRetry, ); }); - expect(within(dialog).queryByLabelText('视频预览：角色动作首帧')).toBeNull(); + expect( + within(dialog).getByRole('button', { name: '暂停角色动作' }), + ).toBeTruthy(); }); test('后台素材查询音频素材使用统一封面缩略图', async () => { diff --git a/apps/admin-web/src/pages/AdminEditorShowcaseReviewPage.test.tsx b/apps/admin-web/src/pages/AdminEditorShowcaseReviewPage.test.tsx index f207ba285..c6bef5f79 100644 --- a/apps/admin-web/src/pages/AdminEditorShowcaseReviewPage.test.tsx +++ b/apps/admin-web/src/pages/AdminEditorShowcaseReviewPage.test.tsx @@ -318,6 +318,101 @@ test('后台精选审核缩略图进入视口后换签并可打开图片预览', expect(screen.queryByRole('dialog', { name: '精选素材详情' })).toBeNull(); }); +test('后台精选审核在共用预览弹窗播放完整角色动作', async () => { + vi.mocked(listAdminEditorShowcaseAssets).mockResolvedValueOnce({ + entries: [ + { + ...pendingShowcaseAsset, + label: '待机动作', + assetKind: 'character-animation', + imageSrc: '/generated/action/frame-01.png', + objectKey: 'generated/action/frame-01.png', + imageSequenceFrames: [ + { + imageSrc: '/generated/action/frame-01.png', + objectKey: 'generated/action/frame-01.png', + width: 192, + height: 256, + }, + { + imageSrc: '/generated/action/frame-02.png', + objectKey: 'generated/action/frame-02.png', + width: 192, + height: 256, + }, + ], + imageSequenceDurationMs: 250, + }, + ], + nextCursor: null, + }); + vi.mocked(getAdminAssetReadUrl).mockImplementation( + async (_token, request) => { + const objectKey = request.objectKey ?? undefined; + return { + read: { + objectKey, + signedUrl: `https://signed.example.com/${objectKey ?? ''}`, + expiresAt: '2099-01-01T00:05:00Z', + }, + }; + }, + ); + + render( + , + ); + fireEvent.click(await screen.findByTitle('预览素材')); + + const dialog = await screen.findByRole('dialog', { name: '素材预览' }); + expect( + await within(dialog).findByRole('button', { name: '暂停角色动作' }), + ).toBeTruthy(); + await waitFor(() => { + expect( + dialog.querySelectorAll('.admin-asset-query-sequence-frame'), + ).toHaveLength(2); + }); + expect(getAdminAssetReadUrl).toHaveBeenCalledWith('admin-token', { + objectKey: 'generated/action/frame-02.png', + expireSeconds: 300, + }); +}); + +test('后台精选审核对损坏角色动作显示错误而不回退首帧', async () => { + vi.mocked(listAdminEditorShowcaseAssets).mockResolvedValueOnce({ + entries: [ + { + ...pendingShowcaseAsset, + label: '损坏动作', + assetKind: 'character-animation', + imageSequenceFrames: null, + imageSequenceDurationMs: null, + }, + ], + nextCursor: null, + }); + + render( + , + ); + fireEvent.click(await screen.findByTitle('预览素材')); + + const dialog = await screen.findByRole('dialog', { name: '素材预览' }); + expect( + within(dialog).getByLabelText('角色动作序列损坏：损坏动作'), + ).toBeTruthy(); + expect( + within(dialog).queryByRole('img', { name: '图片预览：损坏动作' }), + ).toBeNull(); +}); + test('后台精选审核将无 objectKey 的绝对 OSS 图片地址换签后预览', async () => { vi.mocked(listAdminEditorShowcaseAssets).mockResolvedValueOnce({ entries: [ diff --git a/apps/admin-web/src/styles/admin.css b/apps/admin-web/src/styles/admin.css index 191fbd840..e7610f4f0 100644 --- a/apps/admin-web/src/styles/admin.css +++ b/apps/admin-web/src/styles/admin.css @@ -1703,6 +1703,58 @@ button:disabled { object-fit: contain; } +.admin-asset-query-sequence-preview { + position: relative; + width: min(100%, 720px); + min-height: min(64dvh, 560px); + overflow: hidden; +} + +.admin-asset-query-sequence-frame { + position: absolute; + inset: 0; + display: block; + width: 100%; + height: 100%; + object-fit: contain; + object-position: center; +} + +.admin-asset-query-sequence-controls { + position: absolute; + right: 12px; + bottom: 12px; + z-index: 2; + display: flex; + max-width: calc(100% - 24px); + flex-wrap: wrap; + align-items: center; + gap: 8px; + border-radius: 999px; + background: rgba(43, 31, 22, 0.82); + color: #fff; + padding: 6px 10px; + font-size: 12px; +} + +.admin-asset-query-sequence-controls button { + display: grid; + min-width: 44px; + min-height: 44px; + border: 0; + background: transparent; + color: inherit; + cursor: pointer; + padding: 0; + place-items: center; +} + +.admin-asset-query-sequence-error { + display: grid; + color: #9d3127; + place-items: center; +} + .admin-asset-query-preview-audio { display: grid; justify-items: center; diff --git a/docs/openapi/genarrative-external-v1.openapi.json b/docs/openapi/genarrative-external-v1.openapi.json index 1ddc9543d..659c13a6f 100644 --- a/docs/openapi/genarrative-external-v1.openapi.json +++ b/docs/openapi/genarrative-external-v1.openapi.json @@ -2051,6 +2051,47 @@ }, "ExternalEditorAssetCreateRequest": { "type": "object", + "allOf": [ + { + "if": { + "properties": { + "assetKind": { + "const": "character-animation" + } + }, + "required": [ + "assetKind" + ] + }, + "then": { + "required": [ + "imageSequenceFrames", + "imageSequenceDurationMs" + ], + "properties": { + "generationInputs": { + "$ref": "#/components/schemas/ExternalCharacterAnimationGenerationInputs" + } + } + }, + "else": { + "not": { + "anyOf": [ + { + "required": [ + "imageSequenceFrames" + ] + }, + { + "required": [ + "imageSequenceDurationMs" + ] + } + ] + } + } + } + ], "required": [ "folderId", "label", @@ -2132,16 +2173,71 @@ "type": [ "string", "null" - ] + ], + "description": "权威媒体类别。character-animation 渲染为序列帧，video 渲染为视频，audio/sound-effect/background-music 渲染为音频，其余渲染为图片。" + }, + "imageSequenceFrames": { + "type": "array", + "minItems": 2, + "items": { + "$ref": "#/components/schemas/EditorImageSequenceFrame" + } + }, + "imageSequenceDurationMs": { + "type": "integer", + "minimum": 1, + "description": "assetKind=character-animation 时完整图片序列播放一次的毫秒时长；与音频、视频生成参数 durationSeconds 无关。" }, "generationInputs": { - "$ref": "#/components/schemas/JsonValue" + "$ref": "#/components/schemas/JsonValue", + "description": "可重放的生成输入。assetKind=character-animation 时不得包含 characterAnimation、frames、previewVideoPath、frameCount、fps 或 durationSeconds 等旧运行字段；正式媒体数据必须写入 imageSequenceFrames 和 imageSequenceDurationMs。服务端会移除 screenColorHex、mattingProvider、mattingModel 等内部处理审计字段。" } }, "additionalProperties": false }, "ExternalEditorProjectResourceCreateRequest": { "type": "object", + "allOf": [ + { + "if": { + "properties": { + "assetKind": { + "const": "character-animation" + } + }, + "required": [ + "assetKind" + ] + }, + "then": { + "required": [ + "imageSequenceFrames", + "imageSequenceDurationMs" + ], + "properties": { + "generationInputs": { + "$ref": "#/components/schemas/ExternalCharacterAnimationGenerationInputs" + } + } + }, + "else": { + "not": { + "anyOf": [ + { + "required": [ + "imageSequenceFrames" + ] + }, + { + "required": [ + "imageSequenceDurationMs" + ] + } + ] + } + } + } + ], "required": [ "imageSrc", "width", @@ -2215,14 +2311,71 @@ "type": [ "string", "null" - ] + ], + "description": "权威媒体类别。character-animation 渲染为序列帧，video 渲染为视频，audio/sound-effect/background-music 渲染为音频，其余渲染为图片。" + }, + "imageSequenceFrames": { + "type": "array", + "minItems": 2, + "items": { + "$ref": "#/components/schemas/EditorImageSequenceFrame" + } + }, + "imageSequenceDurationMs": { + "type": "integer", + "minimum": 1, + "description": "assetKind=character-animation 时完整图片序列播放一次的毫秒时长；与音频、视频生成参数 durationSeconds 无关。" }, "generationInputs": { - "$ref": "#/components/schemas/JsonValue" + "$ref": "#/components/schemas/JsonValue", + "description": "可重放的生成输入。assetKind=character-animation 时不得包含 characterAnimation、frames、previewVideoPath、frameCount、fps 或 durationSeconds 等旧运行字段；正式媒体数据必须写入 imageSequenceFrames 和 imageSequenceDurationMs。服务端会移除 screenColorHex、mattingProvider、mattingModel 等内部处理审计字段。" } }, "additionalProperties": false }, + "ExternalCharacterAnimationGenerationInputs": { + "not": { + "anyOf": [ + { + "type": "object", + "required": [ + "characterAnimation" + ] + }, + { + "type": "object", + "required": [ + "frames" + ] + }, + { + "type": "object", + "required": [ + "previewVideoPath" + ] + }, + { + "type": "object", + "required": [ + "frameCount" + ] + }, + { + "type": "object", + "required": [ + "fps" + ] + }, + { + "type": "object", + "required": [ + "durationSeconds" + ] + } + ] + }, + "description": "assetKind=character-animation 时 generationInputs 不得包含旧运行字段；正式媒体数据使用 imageSequenceFrames 和 imageSequenceDurationMs。" + }, "ExternalEditorAssetUpdateRequest": { "type": "object", "properties": { @@ -2545,7 +2698,19 @@ "type": [ "string", "null" - ] + ], + "description": "权威媒体类别。character-animation 渲染为序列帧，video 渲染为视频，audio/sound-effect/background-music 渲染为音频，其余渲染为图片。" + }, + "imageSequenceFrames": { + "type": "array", + "items": { + "$ref": "#/components/schemas/EditorImageSequenceFrame" + } + }, + "imageSequenceDurationMs": { + "type": "integer", + "minimum": 1, + "description": "assetKind=character-animation 时完整图片序列播放一次的毫秒时长；与音频、视频生成参数 durationSeconds 无关。" }, "generationInputs": { "$ref": "#/components/schemas/JsonValue" @@ -2688,6 +2853,18 @@ "sourceType": { "type": "string" }, + "imageSequenceFrames": { + "type": "array", + "items": { + "$ref": "#/components/schemas/EditorImageSequenceFrame" + }, + "description": "assetKind=character-animation 时的完整可播放帧集合。" + }, + "imageSequenceDurationMs": { + "type": "integer", + "minimum": 1, + "description": "assetKind=character-animation 时完整图片序列播放一次的毫秒时长；与音频、视频生成参数 durationSeconds 无关。" + }, "prompt": { "type": [ "string", @@ -2716,7 +2893,8 @@ "type": [ "string", "null" - ] + ], + "description": "权威媒体类别。character-animation 渲染为序列帧，video 渲染为视频，audio/sound-effect/background-music 渲染为音频，其余渲染为图片。" }, "generationInputs": { "$ref": "#/components/schemas/JsonValue" @@ -3852,7 +4030,7 @@ "string", "null" ], - "description": "角色动作绿幕预览视频写入账号素材库的文件夹；省略时写入默认项目素材文件夹。" + "description": "角色动作预览视频和最终透明序列帧素材写入账号素材库的文件夹；省略时写入默认项目素材文件夹。" }, "assetLabel": { "type": [ @@ -3865,22 +4043,29 @@ }, "additionalProperties": false }, - "EditorCharacterAnimationFrame": { + "EditorImageSequenceFrame": { "type": "object", "required": [ - "frameIndex", "imageSrc", + "objectKey", + "assetObjectId", "width", "height" ], "properties": { - "frameIndex": { - "type": "integer", - "minimum": 0 - }, "imageSrc": { "type": "string" }, + "objectKey": { + "type": "string", + "minLength": 1, + "description": "帧对应的稳定 OSS Object Key；服务端据此重建持久 imageSrc，不能只提交临时签名 URL。" + }, + "assetObjectId": { + "type": "string", + "minLength": 1, + "description": "帧对应的稳定资产对象 ID。" + }, "width": { "type": "integer", "minimum": 1 @@ -3889,7 +4074,9 @@ "type": "integer", "minimum": 1 } - } + }, + "additionalProperties": false, + "description": "图片序列帧。数组位置是唯一播放顺序，不携带额外序号字段；每帧必须同时携带 objectKey 与 assetObjectId，imageSrc 按 objectKey 规范化为持久站内路径。" }, "EditorCharacterAnimationGenerationResponse": { "type": "object", @@ -3926,7 +4113,7 @@ "frames": { "type": "array", "items": { - "$ref": "#/components/schemas/EditorCharacterAnimationFrame" + "$ref": "#/components/schemas/EditorImageSequenceFrame" } }, "frameCount": { @@ -3964,6 +4151,28 @@ ], "description": "当请求携带 canvasCompletion 且服务端成功写入画布布局时返回最新项目快照。" }, + "resource": { + "anyOf": [ + { + "$ref": "#/components/schemas/EditorProjectResource" + }, + { + "type": "null" + } + ], + "description": "最终透明角色动作对应的项目资源。携带 projectId 时，画布图层必须直接引用其 resourceId，不得再次创建重复资源。" + }, + "asset": { + "anyOf": [ + { + "$ref": "#/components/schemas/EditorAsset" + }, + { + "type": "null" + } + ], + "description": "最终透明角色动作素材。assetKind 为 character-animation，并直接包含序列帧字段。" + }, "queueState": { "anyOf": [ { diff --git a/docs/project-memory/shared-memory/decision-log.md b/docs/project-memory/shared-memory/decision-log.md index f8f2adead..3e8f563c2 100644 --- a/docs/project-memory/shared-memory/decision-log.md +++ b/docs/project-memory/shared-memory/decision-log.md @@ -5946,6 +5946,21 @@ - 微信边界：小程序客户端仍只上传 `wechatPhoneCode`；`platform-auth` 必须要求微信成功响应中的 `phoneNumber`、`countryCode` 与 `purePhoneNumber` 均存在且非空，但只使用后两项执行国家码校验和 E.164 构造。腾讯官方仅说明境外 `phoneNumber` 会带区号，并未承诺 E.164 格式，中国号码示例中它与纯号码相同，因此不得校验 `phoneNumber == +{countryCode}{purePhoneNumber}`。微信字段缺失时失败关闭，不能使用普通请求的 `86` 默认值。 - 数据边界：认证投影与 SpacetimeDB 的 `phone_number_e164` 保持不变，不新增国家码或纯号码列，也不需要 schema 迁移或 bindings 生成。 +## 2026-07-28 编辑器媒体类型统一由 assetKind 判定 + +- 决策：`assetKind` 是编辑器资源和素材唯一权威媒体类别；不新增或返回 `mediaType`。`character-animation` 渲染为序列帧，`video` 渲染为视频，`audio/sound-effect/background-music` 渲染为音频，其余类别渲染为图片。前端内部可保留派生的 `CanvasMediaType` 选择渲染器，但不能把它作为后端事实。 +- 持久化边界：`editor_project_resource` 与 `editor_asset` 表尾只保存 `image_sequence_frames_json` 与 `image_sequence_duration_ms`，默认均为 `None`；`editor_showcase_asset` 作为提交时冻结的审核与公开快照，同样在表尾保存这两个字段并从账号素材逐字段复制，旧行默认均为 `None`。正式帧对象和角色动作生成响应都不保存或返回 `frameIndex`，数组位置是唯一播放顺序；帧数取数组长度，FPS 由帧数和毫秒时长即时推导，不持久化 `frame_count`、`fps` 或通用 `duration_seconds`。后端处理抽帧和逐帧去背时仍保留内部 `frame_index`，仅用于乱序并发收口、OSS 命名、日志和错误定位；仓库内旧 helper 同步升级，不为它保留外部兼容字段。 +- 时长口径：`image_sequence_duration_ms` 只表示角色图片序列完整播放一次的毫秒时长，与音频 / 视频生成请求中的 `durationSeconds` 完全分离。角色动作与视频生成响应仍可携带各自既有的请求 / 结果级 `frameCount/fps/durationSeconds`；通用音视频秒数不进入资源 / 素材正式列、`EditorAsset`、`CanvasLayer` 或画布 layout，只允许把上传探测值或生成请求值格式化为用户可见字符串后写入 `generation_inputs_json.fields[]` 供素材详情展示。素材详情和画布 ZIP 用户可见元数据只透传实际存在的 `fields[]` 时长项；缺少时直接省略，不生成 `--:--` 占位。素材放置与工程恢复不做媒体探测，也不从 layout / resource 回退；音频播放只使用媒体 `loadedmetadata.duration`。不得为音频生成响应新增 `durationSeconds`，也不得把音视频秒数写入图片序列字段。前端角色动作播放器使用 `imageSequenceDurationMs / imageSequenceFrames.length`，Spine 导出时才换算秒数并推导 FPS。 +- 写入与复用边界：`assetKind=character-animation` 必须在 SpacetimeDB storage/procedure 边界同时提供至少两帧有效数组与大于 0 的图片序列毫秒时长；其他类别不得携带图片序列字段。同项目同源同媒体资源只允许 `None → Some` 单调回填，非空冲突失败关闭，延迟重试的 `updated_at` 取请求时间与既有时间的较大值。 +- 生成关联边界：角色动作生成响应同时返回已经持久化的最终 `resource` / `asset`；带项目上下文时前端结果图层必须直接使用 `resource.resourceId` 及其预览视频来源血缘，缺少 resource 直接失败。无项目放置也必须绑定响应中的正式账号素材，不能从响应 `frames` 构造无资产真相的本地动作结果。“动作（原始视频）”只是 `asset_kind = video` 的 provider 中间产物，其 `editor_project_resource` 与 `editor_asset` 两行都固定 `generation_inputs_json = NULL`，不得携带引用或开放参数复用；完整用户生成配方只保存在最终 `asset_kind = character-animation` 的序列资源 / 素材。角色动作规范化迁移对权威证明的 preview video 同样清空整份生成输入，但不改写其 `source_resource_id`。 +- repair 幂等边界：legacy 音频 repair 不再派生或回填任何资源级通用时长，因此新列上线前已完成 repair 的重放继续按原字段精确匹配；图片序列字段在音频行上均为 `None`。 +- 布局边界：`editor_project_resource` 的正式字段是角色动作唯一媒体真相；动作 layout 只保存资源引用和 placement，不保存帧、时长、预览、生成输入、资源元数据或顶层 `mediaType`。前端仍可从 `assetKind` 派生内部 `CanvasMediaType`，但后端响应清洗不得把普通 layer 的旧 `mediaType` 传回前端，也不得递归删除 `generationInputs.references[*].mediaType`。 +- 迁移边界（2026-08-04 收口）：存量 `generation_inputs_json.characterAnimation`、已证明动作行的 helper 顶层 `frames/previewVideoPath/frameCount/fps/durationSeconds`、`screenColorHex`、正式帧 `frameIndex`、误标预览 MP4 和动作 layout 副本，由 `normalize_editor_character_animation_metadata_and_return` 按 `asset → project-resource → showcase → canvas` 一次性规范化。顶层字段只有先证明动作身份后才解释；普通图片 / 视频任意 JSON 中的同名字段不动。canvas dry-run 可消费前置 scope 的计划态结果，但 apply 仍要求前置 scope 已物理完成；同 task 候选先按权威对象规划分类并排除预览视频，只有唯一最终图片序列可补建资源。正式序列每帧必须按首帧 bucket 的稳定路径精确匹配同 owner / task 的图片 `editor_character_animation` 对象并补齐 `objectKey/assetObjectId`。迁移永不验证 layout 复制的 `sourceResourceId`，补建资源采用最终素材的 DB 血缘；新生成直接来源链仍严格校验。正式/旧版冲突、最终候选为零或多个、对象不匹配形成 blocker；apply 必须绑定同批 dry-run SHA-256，结束后全量复核零匹配、零 blocker。迁移完成后删除 api-server、admin、Web 与 helper 的 action fallback。 +- 新写入边界：`assetKind=character-animation` 的 `generationInputs` 若含旧运行字段或 `screenColorHex`，正式帧若含 `frameIndex`，api-server 和 SpacetimeDB storage 均失败关闭；门禁只对角色动作生效，不误伤其它素材的任意 generation input JSON。 +- 交互边界：角色动作素材下载导出完整序列 ZIP；点击、HTML5 拖放和指针拖放创建可移动、循环播放且可保存恢复的序列图层。`assetKind=character-animation` 但缺少有效帧时按损坏素材失败关闭，不回退首帧 PNG。 +- 精选展示边界：`/creation` 精选卡片和预览弹窗继续扩展各自现有媒体 renderer，不复用或重构画布图层组件；卡片静止时只读取首帧，hover / focus 后才加载并播放完整序列，预览弹窗提供播放暂停。后台素材查询与精选审核继续共用 `AdminEditorAssetMedia`，列表只读首帧，预览弹窗才逐帧使用管理员换签。两端均按 `imageSequenceDurationMs / imageSequenceFrames.length` 切帧，下一帧未就绪时保留上一帧且禁止淡入；损坏动作不退回普通图片。 +- 精选帧授权边界：公开换签只对当前已通过、已展示、返还完成且未删除的精选动作，按同 owner 的冻结帧 `assetObjectId` / `objectKey` 形成 exact grant；不从 `imageSrc` 或前缀推导。隐藏、拒绝、删除或快照损坏后逐帧授权随当前事务真相撤销，`read-url` 与 `read-bytes` 继续共用该判断。 + ## 2026-07-24 后台用户详情展示历史花费泥点 - 口径：`historicalConsumedPoints` 表示用户历史总消费，只累计 `profile_wallet_ledger.source_type = asset_operation_consume` 且 `amount_delta < 0` 的绝对值；`asset_operation_refund` 不冲减，充值退款追回、余额重置、赠送和退款 hold 均不计入。 @@ -6450,6 +6465,8 @@ - local 状态：`local-*` 只是 ID 形状，不能直接解释为“素材仍在保存”。新上传 / 新生成素材是否 pending 取资源登记在途状态；严格满足兼容谓词的历史自包含本地角色动作序列是持久化终态，不得误报等待。若当前版本尚不能复制这类序列，以准确原因失败关闭；既非 pending 又不满足历史谓词的 unresolved local 图层也失败关闭，但不得承诺稍后一定自动恢复。layout PATCH pending 不参与资源登记判断，系统剪贴板图片导入不受影响。 - schema 与迁移：在现有 `EditorCanvasLayer` 结构体末尾追加 `#[default(None::)] asset_kind_override: Option`，不删除、改名、重排或改类型。legacy 图层类型与资源默认相同则迁移为 `None`，不同则迁移为 override；资源无默认值时只有全部引用图层显式同值才补资源默认，否则保留各自 override；自包含历史序列的显式类型迁入 override，不伪造资源。同步 `migration.rs`、表目录 / 数据契约、生成 bindings、HTTP DTO 与结构化 canonical hash，并运行 `npm run spacetime:generate`、`npm run check:spacetime-schema`。 - 并发边界：未登记图层被禁止复制后，不再按临时资源 ID 合并项目资源创建请求，也不再用一次响应批量改写共享临时 ID。每个合法新增图层保留自己的响应快照与回调；layout PATCH 的串行 latest-wins 队列、共享资源的多布局引用和 session 资源快照按 `resourceId` 去重继续保留，它们与资源创建 single-flight 是不同机制。 +- 媒体兼容边界：override 只允许在资源默认类型的同一媒体族内变化。动作、视频各自独立成族，`audio/sound-effect/background-music` 同属音频族，其余类型与空默认值同属图片族。前端菜单禁用跨族标签且更新入口重复校验；后端对每个结构化图层按资源完整校验，跨族值清空后回退资源默认类型，不因该兼容错误拒绝整个保存。 +- 历史恢复：客户端读取到已持久化的跨族 override 时必须保留图层、回退资源默认类型、显示明确提示并自动提交清理后的布局；不得再用 `hydrateLayer() -> null -> filter(Boolean)` 静默隐藏持久层仍存在的图层。修复保存只替换命中图层的规范化布局项，其他尚不能 hydrate 的历史项原样保留，避免修复一个标签时顺带删除无关数据。 ## 2026-08-03 Agent Runtime 原生工具合同本地失败关闭 diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index a86b96227..3574369ec 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -4033,6 +4033,34 @@ - 验证：`adminRoutes` 必须包含 `gray-release`，admin-web TypeScript/ESLint/Vitest 不得排除灰度页；页面测试必须断言只请求 feature-gates，并继续覆盖现役固定 target、直接 Gate Key 保存与新 target 状态重置。 - 关联：`apps/admin-web/src/pages/AdminGrayReleaseConfigPage.tsx`、`apps/admin-web/src/app/adminRoutes.ts`、`server-rs/crates/api-server/src/modules/admin.rs`、`docs/technical/【架构下线】旧创作模板业务退役方案-2026-07-17.md`。 +## 角色动作不能靠素材主图或通用生成输入恢复 + +- 现象：角色动作在整画布导出时正常，但从素材库单项下载只得到第一帧 PNG，拖回画布也成为普通静态图片。 +- 原因：`editor_asset.image_src` 只指向首帧；若 worker 把完整帧集塞进 `generation_inputs_json`，素材 DTO、用户输入清洗或画布布局任一层丢字段，就会退化成 PNG。再增加一个 `mediaType` 只能掩盖结果字段没有落到正式资源的问题。 +- 处理：worker 只把完整帧集与图片序列毫秒时长写入 `editor_project_resource` / `editor_asset` 的 `image_sequence_frames_json`、`image_sequence_duration_ms`；数组位置是唯一帧序，不保存 `frameIndex`，帧数和 FPS 均按需派生。`assetKind=character-animation` 决定序列渲染。素材映射、单项下载和拖回画布只读取这两个正式字段，项目 resource 在保存 / 刷新后继续作为主真相；动作 layout 只保留资源引用和 placement，不再复制正式媒体结果。账号素材提交精选审核时，`editor_showcase_asset` 必须冻结复制相同字段，公开 read model 只返回正式字段。外部 helper 只调用一次动作生成接口并直接使用响应 `resource` / `asset`。 +- 画布回填：角色动作会形成“原角色资源 → 预览视频资源 → 最终序列资源”的血缘链。生成响应必须返回已经持久化的最终 resource，前端图层直接使用其 `resourceId`；不能继续构造 `local-resource-character-animation-*`，否则 `appendCanvasLayersWithResources` 会再次创建重复资源。新图层的 `sourceResourceId` 同时使用最终 resource 的直接来源（预览视频 resource），不能继续沿用请求中的原角色 resource；否则结构化保存会在已生成并计费后因血缘不一致而拒绝。修复时只替换资源关联与血缘字段，不要顺带把动作图层显示尺寸从生成占位尺寸改成原始帧分辨率。 +- 历史处理：不要再在 read mapper 增加 `generationInputs` / layout fallback。使用 migration operator procedure 按 `asset → project-resource → showcase → canvas` 迁移；只有动作身份已由 `assetKind`、正式字段、嵌套 `characterAnimation` 或权威对象证明后，才解释顶层 `frames/durationSeconds`，否则会把无关任意 JSON 误分类。同一 task 可能同时存在误标为动作的预览 MP4 和最终首帧 PNG，候选查找必须先按权威对象类型做计划态分类，排除视频并要求唯一正式图片序列，不能按原始 `assetKind` 计数。账号素材仍有旧帧、但后来拖入画布的 project-resource 只剩清洗后 `fields/references` 时，project-resource dry-run 必须按同 owner / task / 首帧对象精确消费 asset 计划态结果；apply 仍要求前置 asset scope 已物理完成。canvas 判断已有 resource 是否为动作时也必须消费 project-resource 的计划态类型：旧库误标为动作、但权威对象证明为 preview MP4 且 layout 本身是 video 的图层直接跳过动作清理；layout 明确为 `image-sequence` 却指向该视频时继续形成 blocker，资源规划本身有 blocker 时也不得静默跳过。正式序列还要逐帧用稳定对象路径匹配同 owner / task 的已登记图片对象并补齐 `objectKey/assetObjectId`。迁移不得验证 layout 复制的 `sourceResourceId`：历史 layer 可能仍指向原角色，而最终素材已指向预览资源；清理副本后采用最终素材的 DB 血缘即可，新生成链路仍保持严格校验。正式与旧版结果冲突、候选为零或多个均形成 blocker；脚本诊断应直接打印 scope、ID、原因、owner/project/task、对象身份和来源资源，不能只报 blocker ID。普通 layer 顶层 `mediaType` 在迁移和响应清洗时删除，但嵌套生成参考的 `mediaType` 保留。 +- 新写入与验证：动作 `generationInputs` 出现 `characterAnimation/frames/previewVideoPath/frameCount/fps/durationSeconds/screenColorHex`，或正式帧出现 `frameIndex`，HTTP 与 storage 双层拒绝；其它 asset kind 的任意 JSON 不受该动作门禁影响。测试覆盖两种历史 JSON、无关顶层同名字段、正式/旧版相等与冲突、可选帧引用合并、screen color 和 frameIndex 清理、预览 MP4 重分类、幂等、blocker/hash apply、画布 placement 清理/资源补建、正式字段缺失失败关闭和 helper 单请求。 + +## 图片序列时长不要复用通用媒体秒数 + +- 现象：把角色动作、视频、音频和上传媒体都写进通用 `duration_seconds`，随后又尝试用持久化 `frame_count/fps/duration_seconds` 互相校验，造成取整口径、生成参数和实际播放时长彼此污染。 +- 原因：角色动作需要的是一组图片完整播放一次的精确时长；视频 / 音频的 `durationSeconds` 是生成请求或临时运行态参数。帧数已经由数组长度唯一确定，FPS 也可按需要推导，无需维护三份可冲突真相。 +- 处理：资源 / 素材只保存 `image_sequence_frames_json` 与 `image_sequence_duration_ms`，精选审核快照只冻结复制这两个正式字段。角色动作要求至少两帧且毫秒时长大于 0；播放器按 `时长毫秒 / 数组长度` 计算间隔，Spine 导出时再换算秒数并推导 FPS。音频 / 视频 `durationSeconds` 不映射到这两个字段。 +- 关联：`server-rs/crates/spacetime-module/src/editor_project_storage.rs`、`src/components/image-editor/ImageCanvasWorldView.tsx`、`src/components/image-editor/ImageCanvasExportModel.ts`。 + +## 精选角色动作显示首帧还要检查前端 renderer 与逐帧授权 + +- 现象：精选接口已经返回 `imageSequenceFrames` 和正确的 5 / 6 秒成本，但创作主页或后台审核仍只显示首帧；接入播放器后又可能只有第一帧成功、后续帧换签返回 404。 +- 原因：快照字段、展示 renderer 和私有对象授权是三道独立边界。公开 `imageSrc/objectKey` 只代表首帧，不能让前端自动获得完整帧集；顶层精选 exact grant 也不会自动覆盖其它帧对象。 +- 处理：公开精选模型必须把 `assetKind=character-animation` 映射到序列 renderer，并携带完整帧与毫秒时长；后台素材查询和精选审核共同透传同一字段并复用 `AdminEditorAssetMedia`。生成端不能在 `ProcessedEditorCharacterAnimationFrame → EditorCharacterAnimationFramePayload` 收口时丢弃逐帧 `assetObjectId/objectKey`，正式序列 JSON 必须保留已确认对象的稳定引用。公开授权在 SpacetimeDB 同一事务快照中只按有效精选动作的同 owner 逐帧 `assetObjectId/objectKey` 匹配，不能放宽 generated 前缀。列表未交互时只读首帧，打开或激活动作预览后也只挂载当前帧和有界预读窗口，避免再次制造换签突发；单帧换签或解码失败时跳过该帧、暂停全帧失败的序列并提供显式重试，不能长期显示空白或旧帧。卡片 hover 与 focus 分别跟踪，只要任一状态仍成立就继续播放，系统请求 `prefers-reduced-motion` 时卡片和弹窗默认暂停，用户仍可在弹窗中手动播放。 +- 验证：模型 / 组件测试覆盖 4 / 5 / 6 秒动作、损坏序列不回退 PNG、后台两页共用播放器和未激活列表不逐帧请求；SpacetimeDB 测试覆盖主对象、每帧对象、无关对象、跨 owner 与取消展示后的授权撤销。真实浏览器和端到端验收由人工单独执行，不把 unit / component 结果写成 E2E PASS。 + +## 可复用资源回填必须保持时间戳单调 + +- 现象：延迟重试携带比既有行更旧的调用方时间，回填图片序列字段时若无条件写入，会使 `updated_at` 倒退，导致基于时间戳的同步看不到更新或排序错误。 +- 处理：同源图片序列字段只允许 `None → Some`，非空冲突失败关闭；发生回填时 `updated_at = max(existing.updated_at, request_timestamp)`。legacy 音频 repair 不派生资源级图片序列或通用时长，重放继续精确匹配。 + ## 历史钱包消费不能从最近流水或通用订单快照推算 - 现象：后台用户详情要展示累计花费时，直接复用只返回最近 50 条的 `list_profile_wallet_ledger`，或在充值订单每行使用的通用钱包快照里扫描该用户全部流水。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index e5fdfaf35..a1f56d560 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -14,7 +14,7 @@ - 中央画布支持背景拖拽平移、滚轮二维平移、`Ctrl / Cmd + 滚轮` 缩放、缩放百分比菜单、显示所有元素和固定比例缩放。 - 画布左下角提供 Lovart 式状态控件：背景色圆点、素材 / 图层入口、小地图开关；小地图显示图层缩略分布和当前视口框，点击小地图执行显示所有元素。 - 画布 chrome 的边框、hover / 选中态、吸附 / 框选参考线、生成类按钮和通用 active 控件使用陶泥儿暖色主题（以 `--platform-accent`、陶土橙主按钮和深棕文字为基准），不得回退为黑色或蓝色主题；元素类型自身的识别色可继续保留。 -- 画布中的图片可展示、悬浮显示图片 Resolution 尺寸与边框，点击后在图片上方显示浮动工具栏；浮动工具栏只保留当前可执行的编辑动作，不放调整 / 复制 / 删除 / 查看信息占位按钮。图片右上角素材类型标签、图片信息角标和悬浮尺寸标签在画布缩小时必须按 viewport 反向缩放，保持屏幕可读尺寸。素材类型采用双层语义：`editor_project_resource.asset_kind` 是跨布局共享的资源默认类型，`editor_canvas_layer.asset_kind_override` 是当前布局实例可空的覆盖值，显示与功能判断统一使用 `effectiveAssetKind = assetKindOverride ?? resource.assetKind`；两层都为空时显示 `未知`。点击标签只修改或清除当前图层的 `assetKindOverride`，不能创建新资源、替换 `resourceId` 或修改其他引用同一资源的图层；清除覆盖后立即恢复继承资源默认类型，也不能触发图层选择 / 拖拽事件。图片信息角标使用圆形 `i` 图标，不使用中括号或花括号样式。图片不再维护独立展示 `Size` 字段，画布显示宽高统一取 `originalWidth/originalHeight`（图片信息中的 `Resolution`）。 +- 画布中的图片可展示、悬浮显示图片 Resolution 尺寸与边框，点击后在图片上方显示浮动工具栏；浮动工具栏只保留当前可执行的编辑动作，不放调整 / 复制 / 删除 / 查看信息占位按钮。图片右上角素材类型标签、图片信息角标和悬浮尺寸标签在画布缩小时必须按 viewport 反向缩放，保持屏幕可读尺寸。素材类型采用双层语义：`editor_project_resource.asset_kind` 是跨布局共享的资源默认类型，`editor_canvas_layer.asset_kind_override` 是当前布局实例可空的覆盖值，显示与功能判断统一使用 `effectiveAssetKind = assetKindOverride ?? resource.assetKind`；两层都为空时显示 `未知`。点击标签只修改或清除当前图层的 `assetKindOverride`，不能创建新资源、替换 `resourceId` 或修改其他引用同一资源的图层；清除覆盖后立即恢复继承资源默认类型，也不能触发图层选择 / 拖拽事件。标签覆盖必须与资源默认类型属于同一媒体族：`character-animation` 独立为动作族，`video` 独立为视频族，`audio/sound-effect/background-music` 属于音频族，其余类型和空默认值属于图片族；标签菜单禁用跨族选项，更新回调再次校验。加载到历史跨族 override 时不得静默过滤图层，必须清除内存 override、按资源默认类型恢复图层、向用户提示并排队修复保存。图片信息角标使用圆形 `i` 图标，不使用中括号或花括号样式。图片不再维护独立展示 `Size` 字段，画布显示宽高统一取 `originalWidth/originalHeight`（图片信息中的 `Resolution`）。 - 默认工具为选择模式；底部工具栏采用 AI 画布工作流工具组：选择、抓手、上传、生成图片、生成视频、生成音乐、生成规范、生成角色形象、生成图标素材、生成 UI 设计图。底部栏不再展示文字工具、形状标注工具和导出工具；上传与生成图片之间、生成音乐与生成规范之间各有一个半图标高度分割线。 - 鼠标中键拖拽始终平移画布；长按 Space 临时进入抓手模式，松开后恢复原工具。 - 图片拖拽时显示水平 / 垂直吸附参考线，吸附到其它图层、生成占位框或画板的边缘与中心线；当移动元素接近两个同轴元素形成的等距位置时，支持横向或纵向等距吸附。 @@ -197,7 +197,8 @@ - 上传按钮和拖拽上传都支持多文件；底部工具栏的上传入口选择文件后直接进入“上传素材”并在当前画布视口中心创建画布图层，素材栏文件夹内的上传入口只写入对应素材文件夹、不自动入画布；拖到文件夹或该文件夹内素材时进入目标文件夹；拖到画布时进入“上传素材”并在投放点创建画布图层。上传图片必须在创建占位素材、画布图层和账号级素材记录前先读取原图 Resolution，图层宽高、`originalWidth/originalHeight` 和素材库 `width/height` 都使用图片本身尺寸；上传视频同样在创建素材和图层前读取视频 metadata 宽高，保证单层下载或 ZIP 导出的真实视频文件重新导入后仍按文件自身尺寸入画布；仅在无法解析尺寸时才使用对应媒体兜底尺寸。OSS 直传确认成功后，即时占位素材、即时画布图层和补充图片尺寸探测必须使用本次换签得到的 signed URL，不得把私有对象的裸 `/generated-*` 兼容路径交给浏览器读取；账号素材持久化仍只写 `legacyPublicPath`、`objectKey` 和 `assetObjectId`，禁止把会过期的 signed URL 写入资源或素材记录。 - 音频 / 视频素材卡和画布媒体图层必须提供稳定的非文字视觉预览：优先使用 `thumbnailSrc` / 视频 `poster`，没有真实首帧或音频封面时使用由媒体类型、素材名和地址派生的确定性视觉底图。视频图层使用原生 `