diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index 5e0c54a2d..42506a86f 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -247,6 +247,14 @@ - 验证：Network 中 `/api/editor/projects*`、`PATCH /api/editor/projects/{id}`、素材库接口不应出现 `data:image` / `data:video` / `data:audio`；素材库和图层面板缩略图都能换签显示；`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/components/image-editor/ImageCanvasAssetRowView.test.tsx src/components/common/PlatformMediaFrame.test.tsx src/services/assetReadUrlService.test.ts src/services/image-editor/editorProjectClient.test.ts`，后端跑 `cargo test -p api-server editor_project --manifest-path server-rs/Cargo.toml`。 - 关联：`server-rs/crates/api-server/src/editor_project.rs`、`src/components/image-editor/ImageCanvasEditorModel.ts`、`src/components/image-editor/useImageCanvasProjectPersistence.ts`、`src/components/common/PlatformMediaFrame.tsx`、`src/services/assetReadUrlService.ts`。 +## 图片画布裁扩后刷新或去背景丢图先查项目资源化 + +- 现象：从规范图裁切 / 裁扩出新图层后立即执行去除背景，去背景完成时原裁扩图从画布消失；刷新后裁扩图仍不在，但去背景占位可能变成结果图。 +- 原因：裁扩结果由浏览器 canvas 本地渲染为 `data:image/png`。如果项目态先把 `local-resource-*` 图层加入画布，`serializeLayer` 不会保存 `src`，`resolveProjectResourceCreateImageSrc` 又会跳过内联 Data URL，后续应用后端 project snapshot 或刷新 hydrate 时找不到对应 `editor_project_resource`，该源图层就会被过滤。去背景带 `canvasCompletion` 时后端只负责把结果写入生成占位，不会恢复这个未资源化的裁扩源层。 +- 处理：项目上下文中的裁扩结果必须在加入画布前先上传 OSS / asset object，再创建 `editor_project_resource`，并用服务端返回的 `resourceId/objectKey/assetObjectId` 创建裁扩图层；随后去背景的 `sourceResourceId` 和图片读取都指向正式资源。queue 模式下去背景完成后，如果首次读取的项目快照中对应 `generation-dialog` 仍是 `generating` 或缺少 `generatedLayerId`，前端短暂等待后再读取一次项目快照。 +- 验证：`npm run test -- src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx -- --runInBand` 应覆盖裁扩先上传再创建项目资源，以及去背景队列完成后对未完成占位进行二次项目读取。 +- 关联：`src/components/image-editor/useImageCanvasGenerationWorkflow.ts`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`。 + ## 图片画布项目封面上传失败要有本地展示兜底 - 现象：画布项目已反复打开、保存或操作，但 `/project` 列表卡片仍只显示“项目”占位，没有封面图。 diff --git a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md index 1b1ac5a2d..0e2fe57f6 100644 --- a/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md +++ b/docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md @@ -52,7 +52,7 @@ - `editor_project_resource` 表保存工程画布引用过的资源快照：`resourceId`、`projectId`、`ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind`、`generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource，图层只引用 resourceId；图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource，并把 `resourceId` 随生成响应返回给前端。图片生成请求如果同时携带 `canvasCompletion`（生成器 `dialogId`、标题和占位框，或无 dialog 的右侧完成占位），BFF / worker 在生成成功后必须直接读取当前项目布局，优先使用最新 `generation-dialog` 占位框位置；只有当前布局仍存在对应 `generation-dialog` 时才插入轻量结果图层、把生成器标记为 `idle` 并写入 `generatedLayerId`，沿用后端当前 viewport 保存布局，再返回或刷新最新项目快照；前端只应用该快照刷新显示，不把生成完成态作为本地业务真相，也不在项目加载时根据资源行推断完成态。有项目上下文但后端没有返回项目快照时，前端不得本地补结果图层，只保留当前生成器交互状态等待下一次项目刷新。 - 项目封面图是画布当前视口栅格化后的静态快照资源，不在项目列表页临时重放 `layers + viewport`。前端在项目加载后和防抖保存 layout 时生成 320x240 PNG，走私有 OSS / asset object 上传，再创建 `editor_project_resource`，其中 `assetKind="project-cover-snapshot"`、`sourceType="uploaded"`。`/project` 与 `/creation` 最近项目卡只读取最新封面快照资源渲染；没有封面快照时显示普通项目占位，不回退为实时画布组合。 - 图片、音频、视频和角色动画帧文件本体继续走 OSS / asset object；浏览器读取私有 generated 对象统一经 `/api/assets/read-url` 换签，签名 URL 可在 session 内复用，但不得作为持久化真相。`/api/assets/read-url` 属于页面展示层高频后台请求，前端统一在 `assetReadUrlService` 内做同 key pending 去重、session 缓存和跨组件节流；UI 设计切片、角色动画帧或大量素材恢复时不得绕过该服务并发换签，否则单页可在同一秒内打满发布入口 `genarrative_api_rps` burst。 -- 登录态上传和生成结果必须先落 OSS / asset object，再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/"`、`objectKey` 和 `assetObjectId`；未登录演示态可以在内存里使用 Data URL 预览，但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*`、`data:video/*`、`data:audio/*` 或 `blob:`。旧数据读取时如果已有 `objectKey`，`imageSrc` 归一成 `/`；没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行；没有当前工程 ID 时才创建账号级 `editor_asset` 行，随后把对应 `resourceId` 或 `assetId` 写入参考图临时状态，生成请求仍使用临时状态中的图片源或 `objectKey`。 +- 登录态上传和生成结果必须先落 OSS / asset object，再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/"`、`objectKey` 和 `assetObjectId`；未登录演示态可以在内存里使用 Data URL 预览，但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*`、`data:video/*`、`data:audio/*` 或 `blob:`。旧数据读取时如果已有 `objectKey`，`imageSrc` 归一成 `/`；没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。裁扩在项目上下文中虽然由前端 canvas 本地渲染 PNG，也必须先上传 OSS / asset object 并创建 `editor_project_resource`，再把带正式 `resourceId/objectKey/assetObjectId` 的裁扩图层加入画布；不能先把 `local-resource-*` + Data URL 图层交给项目保存或后续去背景。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行；没有当前工程 ID 时才创建账号级 `editor_asset` 行，随后把对应 `resourceId` 或 `assetId` 写入参考图临时状态，生成请求仍使用临时状态中的图片源或 `objectKey`。 - 资源表保存资源和素材级元数据；图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组：普通图层按 `layerId/resourceId` 保存，生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存，不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON；刷新时优先从 `editor_project_resource` 恢复，旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`；角色、图标等纯色抠图生成器的前端用户路径不保存或恢复 `screenColor` / `segModel`，同源重绘也不再从 `generationInputs.fields` 恢复 `抠图背景色` 或 `抠图模型`；宣发素材生成器还必须保存并恢复 `publicationWorkflowId`、`publicationGameInfo` 和 `publicationReferences`，避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label，不保存图片 Data URL、signed URL 或 `objectKey`；刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照，只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引，只允许保存 `{ title, label, refType, refId }`；生成接口所需的图片 Data URL、signed URL 或 `objectKey` 只存在于提交前的临时参考图状态和请求体字段，不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存，刷新与新建图层均按 `Resolution`（`originalWidth/originalHeight`）原分辨率显示。图层组第一版是画布内布局语义，不单独建表。 - 图片类、生成视频和音频结果除作为 `editor_project_resource` 和画布图层保存外，还要写入账号级 `editor_asset` 素材库；该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下，画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`，前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照；`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底，不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后，右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额，不做本地乐观扣减。生成视频会单独抽取首帧封面并写入 `thumbnailSrc`，素材栏和拖回画布时沿用该封面作为 poster。 - 画布 Agent 会话按“SpacetimeDB 元数据 + OSS 消息正文”存储：`editor_agent_conversation` 只保存 `conversationId/projectId/ownerUserId/title/messagesObjectKey/deleted/createdAt/updatedAt` 等会话元数据；消息正文整体保存为私有 OSS JSON 文档 `editor-agent/{conversationId}.json`。消息文档单对象上限为 2 MiB，同一会话的消息追加和 SSE 最终写回由 api-server 按 `conversationId` 串行化，避免“读 OSS → 改消息 → 写 OSS”并发覆盖。前端只通过 api-server BFF 读取和发送会话，不直接读写 SpacetimeDB，也不直接读写 OSS。 diff --git a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx index a119c7c7e..af406df39 100644 --- a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx +++ b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx @@ -23,7 +23,10 @@ import type { import { createQuickEditPanelDraft } from './ImageCanvasGenerationDialogModel'; import type { UiAssetExtractionState } from './ImageCanvasUiAssetExtractionModel'; import { useCanvasGenerationDialogs } from './useCanvasGenerationDialogs'; -import { useImageCanvasGenerationSubmissionWorkflow } from './useImageCanvasGenerationSubmissionWorkflow'; +import { + applyQueuedEditorGenerationProject, + useImageCanvasGenerationSubmissionWorkflow, +} from './useImageCanvasGenerationSubmissionWorkflow'; const resolveEditorImageReferenceDataUrlMock = vi.hoisted(() => vi.fn()); const resolveEditorImageReferenceDataUrlForGenerationMock = vi.hoisted(() => @@ -1455,6 +1458,82 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => { ); }); + it('reloads a queued project snapshot again when the completion dialog is still unresolved', async () => { + vi.useFakeTimers(); + try { + const applyProjectSnapshot = vi.fn(); + const refreshTaskList = vi.fn(); + const refreshWalletBalance = vi.fn(); + const unresolvedProject = { + projectId: 'editor-project-1', + title: '队列项目', + viewport: { x: 0, y: 0, scale: 1 }, + layers: [ + { + layerId: 'generation-dialog-background-removal', + resourceId: 'generation-dialog-background-removal', + itemType: 'generation-dialog', + dialog: { + id: 'dialog-background-removal', + status: 'generating', + }, + }, + ], + resources: [], + updatedAt: '2026-06-23T08:00:00.000Z', + }; + const completedProject = { + ...unresolvedProject, + layers: [ + { + layerId: 'generation-dialog-background-removal', + resourceId: 'generation-dialog-background-removal', + itemType: 'generation-dialog', + dialog: { + id: 'dialog-background-removal', + status: 'idle', + generatedLayerId: 'layer-background-removal-result', + }, + }, + ], + updatedAt: '2026-06-23T08:00:01.000Z', + }; + loadEditorProjectMock + .mockResolvedValueOnce(unresolvedProject) + .mockResolvedValueOnce(completedProject); + + const applyPromise = applyQueuedEditorGenerationProject( + { queueState: createQueueState() }, + 'editor-project-1', + applyProjectSnapshot, + refreshTaskList, + refreshWalletBalance, + 'dialog-background-removal', + ); + await Promise.resolve(); + await Promise.resolve(); + + expect(refreshTaskList).toHaveBeenCalledTimes(1); + expect(refreshWalletBalance).toHaveBeenCalledTimes(1); + expect(loadEditorProjectMock).toHaveBeenCalledTimes(1); + + await vi.advanceTimersByTimeAsync(650); + await applyPromise; + + expect(loadEditorProjectMock).toHaveBeenCalledTimes(2); + expect(applyProjectSnapshot).toHaveBeenNthCalledWith( + 1, + unresolvedProject, + ); + expect(applyProjectSnapshot).toHaveBeenNthCalledWith( + 2, + completedProject, + ); + } finally { + vi.useRealTimers(); + } + }); + it('keeps a queued generation pending when the frontend queue wait expires', async () => { vi.useFakeTimers(); try { diff --git a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts index 8f7783988..72b263c94 100644 --- a/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts @@ -101,6 +101,7 @@ type UiDesignAssetExtractionOptions = { const EDITOR_GENERATION_QUEUE_POLL_INTERVAL_MS = 1600; const EDITOR_GENERATION_QUEUE_TIMEOUT_MS = 20 * 60 * 1000; +const EDITOR_GENERATION_QUEUE_PROJECT_REFRESH_RETRY_MS = 650; type GenerationSubmissionWorkflowOptions = { layers: CanvasLayer[]; @@ -176,6 +177,31 @@ function queuedStateFromResponse( return response?.queueState ?? null; } +function projectHasUnresolvedGenerationDialog( + project: EditorProjectSnapshot, + dialogId: string | null | undefined, +) { + const normalizedDialogId = dialogId?.trim(); + if (!normalizedDialogId) { + return false; + } + return project.layers.some((item) => { + if (item.itemType !== 'generation-dialog') { + return false; + } + const dialog = + item.dialog && typeof item.dialog === 'object' + ? (item.dialog as Record) + : null; + return ( + dialog?.id === normalizedDialogId && + (dialog.status === 'generating' || + typeof dialog.generatedLayerId !== 'string' || + dialog.generatedLayerId.trim() === '') + ); + }); +} + function notifyWalletBalanceMayHaveChanged(callback?: () => void) { callback?.(); } @@ -224,6 +250,8 @@ export async function applyQueuedEditorGenerationProject( applyProjectSnapshot: ((project: EditorProjectSnapshot) => void) | undefined, onQueuedGenerationTask?: () => void, onWalletBalanceMayHaveChanged?: () => void, + completionDialogId?: string | null, + transformProjectSnapshot?: (project: EditorProjectSnapshot) => EditorProjectSnapshot, ) { const queueState = queuedStateFromResponse(response); if (!queueState) { @@ -240,7 +268,15 @@ export async function applyQueuedEditorGenerationProject( return true; } if (projectId && applyProjectSnapshot) { - applyProjectSnapshot(await loadEditorProject(projectId)); + const applyLoadedProject = (project: EditorProjectSnapshot) => { + applyProjectSnapshot(transformProjectSnapshot?.(project) ?? project); + }; + const project = await loadEditorProject(projectId); + applyLoadedProject(project); + if (projectHasUnresolvedGenerationDialog(project, completionDialogId)) { + await delay(EDITOR_GENERATION_QUEUE_PROJECT_REFRESH_RETRY_MS); + applyLoadedProject(await loadEditorProject(projectId)); + } } return true; } diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx index 992f48830..37a37385d 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx @@ -22,6 +22,8 @@ const generateEditorIconSpritesheetMock = vi.hoisted(() => vi.fn()); const generateEditorSoundEffectMock = vi.hoisted(() => vi.fn()); const generateEditorBackgroundMusicMock = vi.hoisted(() => vi.fn()); const editEditorImageMock = vi.hoisted(() => vi.fn()); +const createEditorProjectResourceMock = vi.hoisted(() => vi.fn()); +const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn()); const renderCropExpandImageMock = vi.hoisted(() => vi.fn()); const removeImageBackgroundMock = vi.hoisted(() => vi.fn()); const resolveEditorImageReferenceDataUrlMock = vi.hoisted(() => vi.fn()); @@ -46,6 +48,7 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => { return { ...actual, editEditorImage: editEditorImageMock, + createEditorProjectResource: createEditorProjectResourceMock, generateEditorBackgroundMusic: generateEditorBackgroundMusicMock, generateEditorCharacterAnimation: generateEditorCharacterAnimationMock, generateEditorIconSpritesheet: generateEditorIconSpritesheetMock, @@ -54,6 +57,10 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => { }; }); +vi.mock('../../services/image-editor/editorMediaAssetUploadClient', () => ({ + uploadEditorMediaAssetFile: uploadEditorMediaAssetFileMock, +})); + vi.mock('./ImageCanvasRasterEditModel', async () => { const actual = await vi.importActual< typeof import('./ImageCanvasRasterEditModel') @@ -190,6 +197,14 @@ function GenerationWorkflowHarness({ ) .join('|')} + + {layers + .map( + (layer) => + `${layer.id}:${layer.resourceId}:${layer.objectKey ?? '-'}:${layer.assetObjectId ?? '-'}`, + ) + .join('|')} + {activeDialog ? `${activeDialog.mode}:${activeDialog.status}:${activeDialog.composerOpen !== false ? 'open' : 'closed'}:${activeDialog.generatedLayerId ?? '-'}:${activeDialog.placeholder ? 'placeholder' : '-'}` @@ -823,6 +838,27 @@ describe('useImageCanvasGenerationWorkflow', () => { resolveEditorImageReferenceDataUrlMock.mockImplementation( async (src: string) => src, ); + uploadEditorMediaAssetFileMock.mockResolvedValue({ + src: 'https://signed.example.test/crop-expand.png', + objectKey: + 'generated-character-drafts/editor/crop-expand/project-1/result.png', + assetObjectId: 'asset-object-crop-expand', + legacyPublicPath: + '/generated-character-drafts/editor/crop-expand/project-1/result.png', + }); + createEditorProjectResourceMock.mockResolvedValue({ + resourceId: 'resource-crop-expand', + projectId: 'project-1', + imageSrc: + '/generated-character-drafts/editor/crop-expand/project-1/result.png', + objectKey: + 'generated-character-drafts/editor/crop-expand/project-1/result.png', + assetObjectId: 'asset-object-crop-expand', + width: 380, + height: 280, + sourceType: 'generated', + sourceResourceId: 'resource-source', + }); window.localStorage.clear(); }); @@ -1662,6 +1698,63 @@ describe('useImageCanvasGenerationWorkflow', () => { expect(screen.getByTestId('fit-count').textContent).toBe('1'); }); + it('uploads a project crop-expand result before adding it to the canvas', async () => { + renderCropExpandImageMock.mockResolvedValueOnce({ + imageSrc: 'data:image/png;base64,Y3JvcC1leHBhbmRlZA==', + width: 380, + height: 280, + }); + render(); + + fireEvent.click(screen.getByRole('button', { name: '打开裁扩' })); + fireEvent.click(screen.getByRole('button', { name: '拖拽裁扩边界' })); + fireEvent.click(screen.getByRole('button', { name: '完成裁扩' })); + + await waitFor(() => { + expect(uploadEditorMediaAssetFileMock).toHaveBeenCalledWith( + expect.any(File), + 'image', + expect.objectContaining({ + assetKind: 'editor_crop_expand_image', + entityId: 'project-1', + metadata: { + editor_project_id: 'project-1', + source_resource_id: 'resource-source', + }, + }), + ); + }); + const uploadCallOrder = + uploadEditorMediaAssetFileMock.mock.invocationCallOrder[0]; + const resourceCreateCallOrder = + createEditorProjectResourceMock.mock.invocationCallOrder[0]; + expect(uploadCallOrder).toBeDefined(); + expect(resourceCreateCallOrder).toBeDefined(); + expect(uploadCallOrder!).toBeLessThan(resourceCreateCallOrder!); + expect(createEditorProjectResourceMock).toHaveBeenCalledWith( + 'project-1', + expect.objectContaining({ + imageSrc: + '/generated-character-drafts/editor/crop-expand/project-1/result.png', + objectKey: + 'generated-character-drafts/editor/crop-expand/project-1/result.png', + assetObjectId: 'asset-object-crop-expand', + width: 380, + height: 280, + sourceType: 'generated', + sourceResourceId: 'resource-source', + }), + ); + await waitFor(() => { + expect(screen.getByTestId('layer-resources').textContent).toContain( + 'layer-crop-expand-1:resource-crop-expand:generated-character-drafts/editor/crop-expand/project-1/result.png:asset-object-crop-expand', + ); + }); + expect(screen.getByTestId('layers').textContent).not.toContain( + 'layer-crop-expand-1:源图 裁扩:resource-source:-:data:image', + ); + }); + it('resolves private character image before removing background', async () => { resolveEditorImageReferenceDataUrlMock.mockResolvedValueOnce( 'data:image/png;base64,resolved-character', @@ -1775,6 +1868,115 @@ describe('useImageCanvasGenerationWorkflow', () => { ); }); + it('preserves a just-created crop layer when applying a background removal project snapshot', async () => { + const applyProjectSnapshot = vi.fn(); + let resolveBackgroundRemoval: ((value: unknown) => void) | null = null; + removeImageBackgroundMock.mockImplementationOnce( + () => + new Promise((resolve) => { + resolveBackgroundRemoval = resolve; + }), + ); + render( + , + ); + + fireEvent.click(screen.getByRole('button', { name: '去除背景' })); + + await waitFor(() => expect(removeImageBackgroundMock).toHaveBeenCalled()); + await act(async () => { + resolveBackgroundRemoval?.({ + imageSrc: + '/generated-character-drafts/editor/background-removal/project-result.png', + objectKey: + 'generated-character-drafts/editor/background-removal/project-result.png', + assetObjectId: 'asset-object-background-removal', + width: 512, + height: 768, + taskId: 'background-removal-project-task', + elapsedMs: 1234, + provider: 'BiRefNet', + project: { + projectId: 'project-1', + title: '队列项目', + viewport: { x: 0, y: 0, scale: 1 }, + layers: [ + { + layerId: 'layer-background-removal-result', + resourceId: 'resource-background-removal', + title: '源图 裁扩 去背景', + x: 480, + y: 140, + width: 512, + height: 768, + originalWidth: 512, + originalHeight: 768, + zIndex: 9, + sourceType: 'generated', + }, + ], + resources: [ + { + resourceId: 'resource-background-removal', + projectId: 'project-1', + imageSrc: + '/generated-character-drafts/editor/background-removal/project-result.png', + objectKey: + 'generated-character-drafts/editor/background-removal/project-result.png', + assetObjectId: 'asset-object-background-removal', + width: 512, + height: 768, + sourceType: 'generated', + sourceResourceId: 'resource-crop-expand', + }, + ], + updatedAt: '2026-06-23T08:00:01.000Z', + }, + }); + }); + + const appliedProject = applyProjectSnapshot.mock.calls[0]?.[0]; + expect(appliedProject?.layers).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + layerId: 'layer-crop-expand-1', + resourceId: 'resource-crop-expand', + title: '源图 裁扩', + }), + expect.objectContaining({ + layerId: 'layer-background-removal-result', + resourceId: 'resource-background-removal', + }), + ]), + ); + expect(appliedProject?.resources).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + resourceId: 'resource-crop-expand', + objectKey: + 'generated-character-drafts/editor/crop-expand/project-1/result.png', + assetObjectId: 'asset-object-crop-expand', + }), + ]), + ); + }); + it('opens UI design extraction as a mark selection state before submitting', () => { render(); diff --git a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts index 5d2bd23d9..51a7a208c 100644 --- a/src/components/image-editor/useImageCanvasGenerationWorkflow.ts +++ b/src/components/image-editor/useImageCanvasGenerationWorkflow.ts @@ -11,11 +11,14 @@ import { } from 'react'; import { resolveEditorImageReferenceDataUrl } from '../../services/image-editor/editorImageReference'; -import type { - EditorAssetSnapshot, - EditorProjectSnapshot, +import { uploadEditorMediaAssetFile } from '../../services/image-editor/editorMediaAssetUploadClient'; +import { + createEditorProjectResource, + type EditorAssetSnapshot, + type EditorProjectLayerSnapshot, + type EditorProjectResourceSnapshot, + type EditorProjectSnapshot, } from '../../services/image-editor/editorProjectClient'; -import { resizeCropExpandFrame } from './ImageCanvasCropExpandModel'; import type { CanvasGenerationDialogState, CanvasLayer, @@ -33,6 +36,7 @@ import type { SpecFormValues, SpecGenerationType, } from './ImageCanvasEditorTypes'; +import { resizeCropExpandFrame } from './ImageCanvasCropExpandModel'; import { appendCharacterReference, appendGenerationReference, @@ -125,6 +129,103 @@ type RememberedImageGenerationOptions = { imageSize: string; }; +function dataUrlToImageFile(dataUrl: string, fileName: string) { + const [header = '', payload = ''] = dataUrl.split(','); + const mimeMatch = /^data:([^;]+)(;base64)?$/iu.exec(header); + const type = mimeMatch?.[1] ?? 'image/png'; + const isBase64 = Boolean(mimeMatch?.[2]); + const binary = isBase64 + ? typeof atob === 'function' + ? atob(payload) + : Buffer.from(payload, 'base64').toString('binary') + : decodeURIComponent(payload); + const bytes = new Uint8Array(binary.length); + for (let index = 0; index < binary.length; index += 1) { + bytes[index] = binary.charCodeAt(index); + } + return new File([bytes], fileName, { type }); +} + +function createProjectResourceSnapshotFromLayer( + projectId: string, + layer: CanvasLayer, +): EditorProjectResourceSnapshot | null { + const imageSrc = layer.objectKey?.trim() + ? `/${layer.objectKey.trim().replace(/^\/+/u, '')}` + : layer.src.trim(); + if (!imageSrc) { + return null; + } + return { + resourceId: layer.resourceId, + projectId, + label: layer.title, + imageSrc, + objectKey: layer.objectKey, + assetObjectId: layer.assetObjectId, + width: layer.originalWidth, + height: layer.originalHeight, + sourceType: layer.sourceType, + prompt: layer.prompt, + actualPrompt: layer.actualPrompt, + model: layer.model, + provider: layer.provider, + taskId: layer.taskId, + durationSeconds: layer.durationSeconds, + sourceResourceId: layer.sourceResourceId, + assetKind: layer.assetKind, + generationInputs: layer.generationInputs, + }; +} + +function createProjectLayerSnapshotFromLayer( + layer: CanvasLayer, +): EditorProjectLayerSnapshot { + return { + layerId: layer.id, + resourceId: layer.resourceId, + title: layer.title, + x: layer.x, + y: layer.y, + width: layer.width, + height: layer.height, + originalWidth: layer.originalWidth, + originalHeight: layer.originalHeight, + zIndex: layer.zIndex, + sourceType: layer.sourceType, + sourceResourceId: layer.sourceResourceId, + assetKind: layer.assetKind, + mediaType: layer.mediaType, + objectKey: layer.objectKey, + assetObjectId: layer.assetObjectId, + durationSeconds: layer.durationSeconds, + }; +} + +function preserveSourceLayerInProjectSnapshot( + project: EditorProjectSnapshot, + sourceLayer: CanvasLayer, +): EditorProjectSnapshot { + if ( + project.layers.some((layer) => layer.layerId === sourceLayer.id) || + sourceLayer.resourceId.startsWith('generation-dialog:') + ) { + return project; + } + const sourceResource = project.resources.some( + (resource) => resource.resourceId === sourceLayer.resourceId, + ) + ? null + : createProjectResourceSnapshotFromLayer(project.projectId, sourceLayer); + return { + ...project, + resources: sourceResource + ? [...project.resources, sourceResource] + : project.resources, + layers: [createProjectLayerSnapshotFromLayer(sourceLayer), ...project.layers], + }; +} + function findSourceGenerationDialog( dialogs: CanvasGenerationDialogState[], sourceLayer: CanvasLayer, @@ -1323,12 +1424,77 @@ export function useImageCanvasGenerationWorkflow({ }); layerCounterRef.current += 1; const cropExpandIndex = layerCounterRef.current; + const cropExpandTitle = `${cropExpandSourceLayer.title} 裁扩`; + const cropExpandProjectId = projectId?.trim(); + let cropExpandResourceId = `local-resource-crop-expand-${cropExpandIndex}`; + let cropExpandImageSrc = result.imageSrc; + let cropExpandObjectKey: string | null = null; + let cropExpandAssetObjectId: string | null = null; + let cropExpandSourceResourceId: string | null = + cropExpandSourceLayer.resourceId; + let cropExpandAssetKind = cropExpandSourceLayer.assetKind; + if (cropExpandProjectId) { + const cropExpandFile = dataUrlToImageFile( + result.imageSrc, + `crop-expand-${cropExpandIndex}.png`, + ); + const uploadedCropExpand = await uploadEditorMediaAssetFile( + cropExpandFile, + 'image', + { + assetKind: 'editor_crop_expand_image', + pathSegments: [ + 'editor', + 'crop-expand', + cropExpandProjectId, + `${Date.now()}`, + ], + entityId: cropExpandProjectId, + metadata: { + editor_project_id: cropExpandProjectId, + source_resource_id: cropExpandSourceLayer.resourceId, + }, + }, + ); + const cropExpandResource = await createEditorProjectResource( + cropExpandProjectId, + { + imageSrc: uploadedCropExpand.legacyPublicPath, + objectKey: uploadedCropExpand.objectKey, + assetObjectId: uploadedCropExpand.assetObjectId, + width: result.width, + height: result.height, + sourceType: 'generated', + prompt: cropExpandSourceLayer.prompt, + actualPrompt: cropExpandSourceLayer.actualPrompt, + model: cropExpandSourceLayer.model, + provider: cropExpandSourceLayer.provider, + taskId: cropExpandSourceLayer.taskId, + sourceResourceId: cropExpandSourceLayer.resourceId, + assetKind: cropExpandSourceLayer.assetKind, + generationInputs: cropExpandSourceLayer.generationInputs, + }, + ); + cropExpandResourceId = cropExpandResource.resourceId; + cropExpandImageSrc = cropExpandResource.imageSrc; + cropExpandObjectKey = + cropExpandResource.objectKey ?? uploadedCropExpand.objectKey; + cropExpandAssetObjectId = + cropExpandResource.assetObjectId ?? + uploadedCropExpand.assetObjectId; + cropExpandSourceResourceId = + cropExpandResource.sourceResourceId ?? + cropExpandSourceLayer.resourceId; + cropExpandAssetKind = + (cropExpandResource.assetKind as CanvasLayer['assetKind']) ?? + cropExpandSourceLayer.assetKind; + } const nextLayer: CanvasLayer = { ...cropExpandSourceLayer, id: `layer-crop-expand-${cropExpandIndex}`, - resourceId: `local-resource-crop-expand-${cropExpandIndex}`, - title: `${cropExpandSourceLayer.title} 裁扩`, - src: result.imageSrc, + resourceId: cropExpandResourceId, + title: cropExpandTitle, + src: cropExpandImageSrc, x: cropExpandSourceLayer.x + cropExpandSourceLayer.width + 32, y: cropExpandSourceLayer.y, width: result.width, @@ -1337,10 +1503,11 @@ export function useImageCanvasGenerationWorkflow({ originalHeight: result.height, zIndex: cropExpandIndex + 10, sourceType: 'generated', - sourceResourceId: cropExpandSourceLayer.resourceId, - objectKey: null, - assetObjectId: null, + sourceResourceId: cropExpandSourceResourceId, + objectKey: cropExpandObjectKey, + assetObjectId: cropExpandAssetObjectId, sourceAssetId: null, + assetKind: cropExpandAssetKind, }; captureCanvasHistory(); appendCanvasLayersWithResources([nextLayer]); @@ -1367,6 +1534,7 @@ export function useImageCanvasGenerationWorkflow({ fitLayers, layerCounterRef, persistGeneratedAsset, + projectId, selectSingleLayer, setActiveSidebarPanel, ]); @@ -1397,6 +1565,14 @@ export function useImageCanvasGenerationWorkflow({ }) : undefined; const backgroundRemovalDialogId = backgroundRemovalPlacement?.dialogId; + const applyBackgroundRemovalProjectSnapshot = + projectId && applyProjectSnapshot + ? (project: EditorProjectSnapshot) => { + applyProjectSnapshot( + preserveSourceLayerInProjectSnapshot(project, sourceLayer), + ); + } + : undefined; try { const sourceObjectKey = sourceLayer.objectKey?.trim(); const sourceImageSrc = sourceObjectKey @@ -1428,12 +1604,17 @@ export function useImageCanvasGenerationWorkflow({ applyProjectSnapshot, refreshTaskListForQueuedGeneration, onWalletBalanceMayHaveChanged, + backgroundRemovalDialogId, + applyBackgroundRemovalProjectSnapshot + ? (project) => + preserveSourceLayerInProjectSnapshot(project, sourceLayer) + : undefined, ) ) { return; } - if (result.project && applyProjectSnapshot) { - applyProjectSnapshot(result.project); + if (result.project && applyBackgroundRemovalProjectSnapshot) { + applyBackgroundRemovalProjectSnapshot(result.project); return; } if (backgroundRemovalPlacement?.placeholder && applyProjectSnapshot) { diff --git a/src/components/image-editor/useImageCanvasStageInteractions.test.tsx b/src/components/image-editor/useImageCanvasStageInteractions.test.tsx index 84389c114..33f12ad48 100644 --- a/src/components/image-editor/useImageCanvasStageInteractions.test.tsx +++ b/src/components/image-editor/useImageCanvasStageInteractions.test.tsx @@ -421,6 +421,29 @@ function StageInteractionsHarness({ > 直接移动图层 + +