修复创作页滚动与画布生成状态同步

为创作页恢复可见滚动条并补充样式测试
普通点击画布对象时收敛多选状态,避免批量拖动
裁扩结果先资源化,并在去背完成时保留源图层
队列完成后重读未决项目快照并补充回归测试
同步图片画布持久化文档与长期排障记录
This commit is contained in:
2026-07-10 08:46:08 +08:00
parent 2de9b79e69
commit e464e62697
11 changed files with 729 additions and 19 deletions
@@ -247,6 +247,14 @@
- 验证:Network 中 `/api/editor/projects*``PATCH /api/editor/projects/{id}`、素材库接口不应出现 `data:image` / `data:video` / `data:audio`;素材库和图层面板缩略图都能换签显示;`npm run test -- src/components/image-editor/ImageCanvasEditorModel.test.ts src/components/image-editor/useImageCanvasProjectPersistence.test.tsx src/components/image-editor/ImageCanvasAssetRowView.test.tsx src/components/common/PlatformMediaFrame.test.tsx src/services/assetReadUrlService.test.ts src/services/image-editor/editorProjectClient.test.ts`,后端跑 `cargo test -p api-server editor_project --manifest-path server-rs/Cargo.toml`
- 关联:`server-rs/crates/api-server/src/editor_project.rs``src/components/image-editor/ImageCanvasEditorModel.ts``src/components/image-editor/useImageCanvasProjectPersistence.ts``src/components/common/PlatformMediaFrame.tsx``src/services/assetReadUrlService.ts`
## 图片画布裁扩后刷新或去背景丢图先查项目资源化
- 现象:从规范图裁切 / 裁扩出新图层后立即执行去除背景,去背景完成时原裁扩图从画布消失;刷新后裁扩图仍不在,但去背景占位可能变成结果图。
- 原因:裁扩结果由浏览器 canvas 本地渲染为 `data:image/png`。如果项目态先把 `local-resource-*` 图层加入画布,`serializeLayer` 不会保存 `src``resolveProjectResourceCreateImageSrc` 又会跳过内联 Data URL,后续应用后端 project snapshot 或刷新 hydrate 时找不到对应 `editor_project_resource`,该源图层就会被过滤。去背景带 `canvasCompletion` 时后端只负责把结果写入生成占位,不会恢复这个未资源化的裁扩源层。
- 处理:项目上下文中的裁扩结果必须在加入画布前先上传 OSS / asset object,再创建 `editor_project_resource`,并用服务端返回的 `resourceId/objectKey/assetObjectId` 创建裁扩图层;随后去背景的 `sourceResourceId` 和图片读取都指向正式资源。queue 模式下去背景完成后,如果首次读取的项目快照中对应 `generation-dialog` 仍是 `generating` 或缺少 `generatedLayerId`,前端短暂等待后再读取一次项目快照。
- 验证:`npm run test -- src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx -- --runInBand` 应覆盖裁扩先上传再创建项目资源,以及去背景队列完成后对未完成占位进行二次项目读取。
- 关联:`src/components/image-editor/useImageCanvasGenerationWorkflow.ts``src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts``docs/technical/【前端架构】图片画布编辑器MVP接入方案-2026-06-11.md`
## 图片画布项目封面上传失败要有本地展示兜底
- 现象:画布项目已反复打开、保存或操作,但 `/project` 列表卡片仍只显示“项目”占位,没有封面图。
@@ -52,7 +52,7 @@
- `editor_project_resource` 表保存工程画布引用过的资源快照:`resourceId``projectId``ownerUserId`、OSS / asset object 引用、图片尺寸、来源类型、prompt、actualPrompt、model、provider、taskId、sourceResourceId、`assetKind``generationInputs`、创建时间和更新时间。上传素材被拖入画布时会复制为 project resource,图层只引用 resourceId;图片、图标和 UI 素材生成 BFF 在请求携带 `projectId` 时由后端直接创建新 resource,并把 `resourceId` 随生成响应返回给前端。图片生成请求如果同时携带 `canvasCompletion`(生成器 `dialogId`、标题和占位框,或无 dialog 的右侧完成占位),BFF / worker 在生成成功后必须直接读取当前项目布局,优先使用最新 `generation-dialog` 占位框位置;只有当前布局仍存在对应 `generation-dialog` 时才插入轻量结果图层、把生成器标记为 `idle` 并写入 `generatedLayerId`,沿用后端当前 viewport 保存布局,再返回或刷新最新项目快照;前端只应用该快照刷新显示,不把生成完成态作为本地业务真相,也不在项目加载时根据资源行推断完成态。有项目上下文但后端没有返回项目快照时,前端不得本地补结果图层,只保留当前生成器交互状态等待下一次项目刷新。
- 项目封面图是画布当前视口栅格化后的静态快照资源,不在项目列表页临时重放 `layers + viewport`。前端在项目加载后和防抖保存 layout 时生成 320x240 PNG,走私有 OSS / asset object 上传,再创建 `editor_project_resource`,其中 `assetKind="project-cover-snapshot"``sourceType="uploaded"``/project``/creation` 最近项目卡只读取最新封面快照资源渲染;没有封面快照时显示普通项目占位,不回退为实时画布组合。
- 图片、音频、视频和角色动画帧文件本体继续走 OSS / asset object;浏览器读取私有 generated 对象统一经 `/api/assets/read-url` 换签,签名 URL 可在 session 内复用,但不得作为持久化真相。`/api/assets/read-url` 属于页面展示层高频后台请求,前端统一在 `assetReadUrlService` 内做同 key pending 去重、session 缓存和跨组件节流;UI 设计切片、角色动画帧或大量素材恢复时不得绕过该服务并发换签,否则单页可在同一秒内打满发布入口 `genarrative_api_rps` burst。
- 登录态上传和生成结果必须先落 OSS / asset object,再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/<objectKey>"``objectKey``assetObjectId`;未登录演示态可以在内存里使用 Data URL 预览,但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*``data:video/*``data:audio/*``blob:`。旧数据读取时如果已有 `objectKey``imageSrc` 归一成 `/<objectKey>`;没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行;没有当前工程 ID 时才创建账号级 `editor_asset` 行,随后把对应 `resourceId``assetId` 写入参考图临时状态,生成请求仍使用临时状态中的图片源或 `objectKey`
- 登录态上传和生成结果必须先落 OSS / asset object,再向 `editor_project_resource` / `editor_asset` 写入轻量 `imageSrc: "/<objectKey>"``objectKey``assetObjectId`;未登录演示态可以在内存里使用 Data URL 预览,但项目、素材库、项目资源和 `editor_canvas.layers_json` 不得写入 `data:image/*``data:video/*``data:audio/*``blob:`。旧数据读取时如果已有 `objectKey``imageSrc` 归一成 `/<objectKey>`;没有 `objectKey` 的旧 Data URL 需要走修复上传并回写轻量引用。裁扩在项目上下文中虽然由前端 canvas 本地渲染 PNG,也必须先上传 OSS / asset object 并创建 `editor_project_resource`,再把带正式 `resourceId/objectKey/assetObjectId` 的裁扩图层加入画布;不能先把 `local-resource-*` + Data URL 图层交给项目保存或后续去背景。上传到生成面板参考图槽位的图片必须先创建 `editor_project_resource` 行;没有当前工程 ID 时才创建账号级 `editor_asset` 行,随后把对应 `resourceId``assetId` 写入参考图临时状态,生成请求仍使用临时状态中的图片源或 `objectKey`
- 资源表保存资源和素材级元数据;图层位置、层级、分组选中所需 ID 和 groupId 保存在 `editor_canvas` 的布局 JSON。布局 JSON 是混合数组:普通图层按 `layerId/resourceId` 保存,生成器占位和生成器对话框按 `itemType: "generation-dialog"` 保存,不新增单独表。普通图层的新保存不再把 `assetKind/generationInputs` 写入布局 JSON;刷新时优先从 `editor_project_resource` 恢复,旧布局中的同名字段只作为兼容兜底。生成器快照必须包含生成器 ID、模式、提示词、参数、参考图、状态、占位框位置和可选 `generatedLayerId`;角色、图标等纯色抠图生成器的前端用户路径不保存或恢复 `screenColor` / `segModel`,同源重绘也不再从 `generationInputs.fields` 恢复 `抠图背景色``抠图模型`;宣发素材生成器还必须保存并恢复 `publicationWorkflowId``publicationGameInfo``publicationReferences`,避免刷新后生成卡片字段或参考图丢失。生成器快照中的参考图同样只保存 `resourceId/sourceAssetId` 行引用和展示所需 label,不保存图片 Data URL、signed URL 或 `objectKey`;刷新时用 `editor_project_resource` / `editor_asset` 行恢复临时生成请求所需图片源。生成成功后仍保存该快照,只是渲染时由 `generatedLayerId` 锚定到成品图层而不重复显示灰色占位框。`generationInputs.references` 是用户可见输入快照中的行级索引,只允许保存 `{ title, label, refType, refId }`;生成接口所需的图片 Data URL、signed URL 或 `objectKey` 只存在于提交前的临时参考图状态和请求体字段,不进入资源 / 素材元数据。图层展示尺寸不再作为独立 `Size` 真相保存,刷新与新建图层均按 `Resolution``originalWidth/originalHeight`)原分辨率显示。图层组第一版是画布内布局语义,不单独建表。
- 图片类、生成视频和音频结果除作为 `editor_project_resource` 和画布图层保存外,还要写入账号级 `editor_asset` 素材库;该写入由生成 BFF 在请求携带 `assetFolderId` 时完成。`GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 下,画布图片、改图、图标素材、UI 素材提取、角色动作、视频、音效和背景音乐生成都先返回 `queueState`,前端轮询 `/api/runtime/external-generation/jobs/{jobId}` 到完成后重新读取项目快照;`inline` 或无项目上下文时才使用响应中的 resource / asset 快照做本地落画布兜底,不再把同一生成结果二次调用素材创建接口。生成请求失败、inline 完成或 queue 任务终态完成 / 失败后,右上角泥点 chip 必须通过 `/profile/dashboard` 回读余额,不做本地乐观扣减。生成视频会单独抽取首帧封面并写入 `thumbnailSrc`,素材栏和拖回画布时沿用该封面作为 poster。
- 画布 Agent 会话按“SpacetimeDB 元数据 + OSS 消息正文”存储:`editor_agent_conversation` 只保存 `conversationId/projectId/ownerUserId/title/messagesObjectKey/deleted/createdAt/updatedAt` 等会话元数据;消息正文整体保存为私有 OSS JSON 文档 `editor-agent/{conversationId}.json`。消息文档单对象上限为 2 MiB,同一会话的消息追加和 SSE 最终写回由 api-server 按 `conversationId` 串行化,避免“读 OSS → 改消息 → 写 OSS”并发覆盖。前端只通过 api-server BFF 读取和发送会话,不直接读写 SpacetimeDB,也不直接读写 OSS。
@@ -23,7 +23,10 @@ import type {
import { createQuickEditPanelDraft } from './ImageCanvasGenerationDialogModel';
import type { UiAssetExtractionState } from './ImageCanvasUiAssetExtractionModel';
import { useCanvasGenerationDialogs } from './useCanvasGenerationDialogs';
import { useImageCanvasGenerationSubmissionWorkflow } from './useImageCanvasGenerationSubmissionWorkflow';
import {
applyQueuedEditorGenerationProject,
useImageCanvasGenerationSubmissionWorkflow,
} from './useImageCanvasGenerationSubmissionWorkflow';
const resolveEditorImageReferenceDataUrlMock = vi.hoisted(() => vi.fn());
const resolveEditorImageReferenceDataUrlForGenerationMock = vi.hoisted(() =>
@@ -1455,6 +1458,82 @@ describe('useImageCanvasGenerationSubmissionWorkflow', () => {
);
});
it('reloads a queued project snapshot again when the completion dialog is still unresolved', async () => {
vi.useFakeTimers();
try {
const applyProjectSnapshot = vi.fn();
const refreshTaskList = vi.fn();
const refreshWalletBalance = vi.fn();
const unresolvedProject = {
projectId: 'editor-project-1',
title: '队列项目',
viewport: { x: 0, y: 0, scale: 1 },
layers: [
{
layerId: 'generation-dialog-background-removal',
resourceId: 'generation-dialog-background-removal',
itemType: 'generation-dialog',
dialog: {
id: 'dialog-background-removal',
status: 'generating',
},
},
],
resources: [],
updatedAt: '2026-06-23T08:00:00.000Z',
};
const completedProject = {
...unresolvedProject,
layers: [
{
layerId: 'generation-dialog-background-removal',
resourceId: 'generation-dialog-background-removal',
itemType: 'generation-dialog',
dialog: {
id: 'dialog-background-removal',
status: 'idle',
generatedLayerId: 'layer-background-removal-result',
},
},
],
updatedAt: '2026-06-23T08:00:01.000Z',
};
loadEditorProjectMock
.mockResolvedValueOnce(unresolvedProject)
.mockResolvedValueOnce(completedProject);
const applyPromise = applyQueuedEditorGenerationProject(
{ queueState: createQueueState() },
'editor-project-1',
applyProjectSnapshot,
refreshTaskList,
refreshWalletBalance,
'dialog-background-removal',
);
await Promise.resolve();
await Promise.resolve();
expect(refreshTaskList).toHaveBeenCalledTimes(1);
expect(refreshWalletBalance).toHaveBeenCalledTimes(1);
expect(loadEditorProjectMock).toHaveBeenCalledTimes(1);
await vi.advanceTimersByTimeAsync(650);
await applyPromise;
expect(loadEditorProjectMock).toHaveBeenCalledTimes(2);
expect(applyProjectSnapshot).toHaveBeenNthCalledWith(
1,
unresolvedProject,
);
expect(applyProjectSnapshot).toHaveBeenNthCalledWith(
2,
completedProject,
);
} finally {
vi.useRealTimers();
}
});
it('keeps a queued generation pending when the frontend queue wait expires', async () => {
vi.useFakeTimers();
try {
@@ -101,6 +101,7 @@ type UiDesignAssetExtractionOptions = {
const EDITOR_GENERATION_QUEUE_POLL_INTERVAL_MS = 1600;
const EDITOR_GENERATION_QUEUE_TIMEOUT_MS = 20 * 60 * 1000;
const EDITOR_GENERATION_QUEUE_PROJECT_REFRESH_RETRY_MS = 650;
type GenerationSubmissionWorkflowOptions = {
layers: CanvasLayer[];
@@ -176,6 +177,31 @@ function queuedStateFromResponse(
return response?.queueState ?? null;
}
function projectHasUnresolvedGenerationDialog(
project: EditorProjectSnapshot,
dialogId: string | null | undefined,
) {
const normalizedDialogId = dialogId?.trim();
if (!normalizedDialogId) {
return false;
}
return project.layers.some((item) => {
if (item.itemType !== 'generation-dialog') {
return false;
}
const dialog =
item.dialog && typeof item.dialog === 'object'
? (item.dialog as Record<string, unknown>)
: null;
return (
dialog?.id === normalizedDialogId &&
(dialog.status === 'generating' ||
typeof dialog.generatedLayerId !== 'string' ||
dialog.generatedLayerId.trim() === '')
);
});
}
function notifyWalletBalanceMayHaveChanged(callback?: () => void) {
callback?.();
}
@@ -224,6 +250,8 @@ export async function applyQueuedEditorGenerationProject(
applyProjectSnapshot: ((project: EditorProjectSnapshot) => void) | undefined,
onQueuedGenerationTask?: () => void,
onWalletBalanceMayHaveChanged?: () => void,
completionDialogId?: string | null,
transformProjectSnapshot?: (project: EditorProjectSnapshot) => EditorProjectSnapshot,
) {
const queueState = queuedStateFromResponse(response);
if (!queueState) {
@@ -240,7 +268,15 @@ export async function applyQueuedEditorGenerationProject(
return true;
}
if (projectId && applyProjectSnapshot) {
applyProjectSnapshot(await loadEditorProject(projectId));
const applyLoadedProject = (project: EditorProjectSnapshot) => {
applyProjectSnapshot(transformProjectSnapshot?.(project) ?? project);
};
const project = await loadEditorProject(projectId);
applyLoadedProject(project);
if (projectHasUnresolvedGenerationDialog(project, completionDialogId)) {
await delay(EDITOR_GENERATION_QUEUE_PROJECT_REFRESH_RETRY_MS);
applyLoadedProject(await loadEditorProject(projectId));
}
}
return true;
}
@@ -22,6 +22,8 @@ const generateEditorIconSpritesheetMock = vi.hoisted(() => vi.fn());
const generateEditorSoundEffectMock = vi.hoisted(() => vi.fn());
const generateEditorBackgroundMusicMock = vi.hoisted(() => vi.fn());
const editEditorImageMock = vi.hoisted(() => vi.fn());
const createEditorProjectResourceMock = vi.hoisted(() => vi.fn());
const uploadEditorMediaAssetFileMock = vi.hoisted(() => vi.fn());
const renderCropExpandImageMock = vi.hoisted(() => vi.fn());
const removeImageBackgroundMock = vi.hoisted(() => vi.fn());
const resolveEditorImageReferenceDataUrlMock = vi.hoisted(() => vi.fn());
@@ -46,6 +48,7 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => {
return {
...actual,
editEditorImage: editEditorImageMock,
createEditorProjectResource: createEditorProjectResourceMock,
generateEditorBackgroundMusic: generateEditorBackgroundMusicMock,
generateEditorCharacterAnimation: generateEditorCharacterAnimationMock,
generateEditorIconSpritesheet: generateEditorIconSpritesheetMock,
@@ -54,6 +57,10 @@ vi.mock('../../services/image-editor/editorProjectClient', async () => {
};
});
vi.mock('../../services/image-editor/editorMediaAssetUploadClient', () => ({
uploadEditorMediaAssetFile: uploadEditorMediaAssetFileMock,
}));
vi.mock('./ImageCanvasRasterEditModel', async () => {
const actual = await vi.importActual<
typeof import('./ImageCanvasRasterEditModel')
@@ -190,6 +197,14 @@ function GenerationWorkflowHarness({
)
.join('|')}
</span>
<span data-testid="layer-resources">
{layers
.map(
(layer) =>
`${layer.id}:${layer.resourceId}:${layer.objectKey ?? '-'}:${layer.assetObjectId ?? '-'}`,
)
.join('|')}
</span>
<span data-testid="dialog">
{activeDialog
? `${activeDialog.mode}:${activeDialog.status}:${activeDialog.composerOpen !== false ? 'open' : 'closed'}:${activeDialog.generatedLayerId ?? '-'}:${activeDialog.placeholder ? 'placeholder' : '-'}`
@@ -823,6 +838,27 @@ describe('useImageCanvasGenerationWorkflow', () => {
resolveEditorImageReferenceDataUrlMock.mockImplementation(
async (src: string) => src,
);
uploadEditorMediaAssetFileMock.mockResolvedValue({
src: 'https://signed.example.test/crop-expand.png',
objectKey:
'generated-character-drafts/editor/crop-expand/project-1/result.png',
assetObjectId: 'asset-object-crop-expand',
legacyPublicPath:
'/generated-character-drafts/editor/crop-expand/project-1/result.png',
});
createEditorProjectResourceMock.mockResolvedValue({
resourceId: 'resource-crop-expand',
projectId: 'project-1',
imageSrc:
'/generated-character-drafts/editor/crop-expand/project-1/result.png',
objectKey:
'generated-character-drafts/editor/crop-expand/project-1/result.png',
assetObjectId: 'asset-object-crop-expand',
width: 380,
height: 280,
sourceType: 'generated',
sourceResourceId: 'resource-source',
});
window.localStorage.clear();
});
@@ -1662,6 +1698,63 @@ describe('useImageCanvasGenerationWorkflow', () => {
expect(screen.getByTestId('fit-count').textContent).toBe('1');
});
it('uploads a project crop-expand result before adding it to the canvas', async () => {
renderCropExpandImageMock.mockResolvedValueOnce({
imageSrc: 'data:image/png;base64,Y3JvcC1leHBhbmRlZA==',
width: 380,
height: 280,
});
render(<GenerationWorkflowHarness projectId="project-1" />);
fireEvent.click(screen.getByRole('button', { name: '打开裁扩' }));
fireEvent.click(screen.getByRole('button', { name: '拖拽裁扩边界' }));
fireEvent.click(screen.getByRole('button', { name: '完成裁扩' }));
await waitFor(() => {
expect(uploadEditorMediaAssetFileMock).toHaveBeenCalledWith(
expect.any(File),
'image',
expect.objectContaining({
assetKind: 'editor_crop_expand_image',
entityId: 'project-1',
metadata: {
editor_project_id: 'project-1',
source_resource_id: 'resource-source',
},
}),
);
});
const uploadCallOrder =
uploadEditorMediaAssetFileMock.mock.invocationCallOrder[0];
const resourceCreateCallOrder =
createEditorProjectResourceMock.mock.invocationCallOrder[0];
expect(uploadCallOrder).toBeDefined();
expect(resourceCreateCallOrder).toBeDefined();
expect(uploadCallOrder!).toBeLessThan(resourceCreateCallOrder!);
expect(createEditorProjectResourceMock).toHaveBeenCalledWith(
'project-1',
expect.objectContaining({
imageSrc:
'/generated-character-drafts/editor/crop-expand/project-1/result.png',
objectKey:
'generated-character-drafts/editor/crop-expand/project-1/result.png',
assetObjectId: 'asset-object-crop-expand',
width: 380,
height: 280,
sourceType: 'generated',
sourceResourceId: 'resource-source',
}),
);
await waitFor(() => {
expect(screen.getByTestId('layer-resources').textContent).toContain(
'layer-crop-expand-1:resource-crop-expand:generated-character-drafts/editor/crop-expand/project-1/result.png:asset-object-crop-expand',
);
});
expect(screen.getByTestId('layers').textContent).not.toContain(
'layer-crop-expand-1:源图 裁扩:resource-source:-:data:image',
);
});
it('resolves private character image before removing background', async () => {
resolveEditorImageReferenceDataUrlMock.mockResolvedValueOnce(
'data:image/png;base64,resolved-character',
@@ -1775,6 +1868,115 @@ describe('useImageCanvasGenerationWorkflow', () => {
);
});
it('preserves a just-created crop layer when applying a background removal project snapshot', async () => {
const applyProjectSnapshot = vi.fn();
let resolveBackgroundRemoval: ((value: unknown) => void) | null = null;
removeImageBackgroundMock.mockImplementationOnce(
() =>
new Promise((resolve) => {
resolveBackgroundRemoval = resolve;
}),
);
render(
<GenerationWorkflowHarness
projectId="project-1"
applyProjectSnapshot={applyProjectSnapshot}
initialLayers={[
createLayer({
id: 'layer-crop-expand-1',
resourceId: 'resource-crop-expand',
title: '源图 裁扩',
src: '/generated-character-drafts/editor/crop-expand/project-1/result.png',
objectKey:
'generated-character-drafts/editor/crop-expand/project-1/result.png',
assetObjectId: 'asset-object-crop-expand',
sourceType: 'generated',
sourceResourceId: 'resource-source',
}),
]}
/>,
);
fireEvent.click(screen.getByRole('button', { name: '去除背景' }));
await waitFor(() => expect(removeImageBackgroundMock).toHaveBeenCalled());
await act(async () => {
resolveBackgroundRemoval?.({
imageSrc:
'/generated-character-drafts/editor/background-removal/project-result.png',
objectKey:
'generated-character-drafts/editor/background-removal/project-result.png',
assetObjectId: 'asset-object-background-removal',
width: 512,
height: 768,
taskId: 'background-removal-project-task',
elapsedMs: 1234,
provider: 'BiRefNet',
project: {
projectId: 'project-1',
title: '队列项目',
viewport: { x: 0, y: 0, scale: 1 },
layers: [
{
layerId: 'layer-background-removal-result',
resourceId: 'resource-background-removal',
title: '源图 裁扩 去背景',
x: 480,
y: 140,
width: 512,
height: 768,
originalWidth: 512,
originalHeight: 768,
zIndex: 9,
sourceType: 'generated',
},
],
resources: [
{
resourceId: 'resource-background-removal',
projectId: 'project-1',
imageSrc:
'/generated-character-drafts/editor/background-removal/project-result.png',
objectKey:
'generated-character-drafts/editor/background-removal/project-result.png',
assetObjectId: 'asset-object-background-removal',
width: 512,
height: 768,
sourceType: 'generated',
sourceResourceId: 'resource-crop-expand',
},
],
updatedAt: '2026-06-23T08:00:01.000Z',
},
});
});
const appliedProject = applyProjectSnapshot.mock.calls[0]?.[0];
expect(appliedProject?.layers).toEqual(
expect.arrayContaining([
expect.objectContaining({
layerId: 'layer-crop-expand-1',
resourceId: 'resource-crop-expand',
title: '源图 裁扩',
}),
expect.objectContaining({
layerId: 'layer-background-removal-result',
resourceId: 'resource-background-removal',
}),
]),
);
expect(appliedProject?.resources).toEqual(
expect.arrayContaining([
expect.objectContaining({
resourceId: 'resource-crop-expand',
objectKey:
'generated-character-drafts/editor/crop-expand/project-1/result.png',
assetObjectId: 'asset-object-crop-expand',
}),
]),
);
});
it('opens UI design extraction as a mark selection state before submitting', () => {
render(<GenerationWorkflowHarness />);
@@ -11,11 +11,14 @@ import {
} from 'react';
import { resolveEditorImageReferenceDataUrl } from '../../services/image-editor/editorImageReference';
import type {
EditorAssetSnapshot,
EditorProjectSnapshot,
import { uploadEditorMediaAssetFile } from '../../services/image-editor/editorMediaAssetUploadClient';
import {
createEditorProjectResource,
type EditorAssetSnapshot,
type EditorProjectLayerSnapshot,
type EditorProjectResourceSnapshot,
type EditorProjectSnapshot,
} from '../../services/image-editor/editorProjectClient';
import { resizeCropExpandFrame } from './ImageCanvasCropExpandModel';
import type {
CanvasGenerationDialogState,
CanvasLayer,
@@ -33,6 +36,7 @@ import type {
SpecFormValues,
SpecGenerationType,
} from './ImageCanvasEditorTypes';
import { resizeCropExpandFrame } from './ImageCanvasCropExpandModel';
import {
appendCharacterReference,
appendGenerationReference,
@@ -125,6 +129,103 @@ type RememberedImageGenerationOptions = {
imageSize: string;
};
function dataUrlToImageFile(dataUrl: string, fileName: string) {
const [header = '', payload = ''] = dataUrl.split(',');
const mimeMatch = /^data:([^;]+)(;base64)?$/iu.exec(header);
const type = mimeMatch?.[1] ?? 'image/png';
const isBase64 = Boolean(mimeMatch?.[2]);
const binary = isBase64
? typeof atob === 'function'
? atob(payload)
: Buffer.from(payload, 'base64').toString('binary')
: decodeURIComponent(payload);
const bytes = new Uint8Array(binary.length);
for (let index = 0; index < binary.length; index += 1) {
bytes[index] = binary.charCodeAt(index);
}
return new File([bytes], fileName, { type });
}
function createProjectResourceSnapshotFromLayer(
projectId: string,
layer: CanvasLayer,
): EditorProjectResourceSnapshot | null {
const imageSrc = layer.objectKey?.trim()
? `/${layer.objectKey.trim().replace(/^\/+/u, '')}`
: layer.src.trim();
if (!imageSrc) {
return null;
}
return {
resourceId: layer.resourceId,
projectId,
label: layer.title,
imageSrc,
objectKey: layer.objectKey,
assetObjectId: layer.assetObjectId,
width: layer.originalWidth,
height: layer.originalHeight,
sourceType: layer.sourceType,
prompt: layer.prompt,
actualPrompt: layer.actualPrompt,
model: layer.model,
provider: layer.provider,
taskId: layer.taskId,
durationSeconds: layer.durationSeconds,
sourceResourceId: layer.sourceResourceId,
assetKind: layer.assetKind,
generationInputs: layer.generationInputs,
};
}
function createProjectLayerSnapshotFromLayer(
layer: CanvasLayer,
): EditorProjectLayerSnapshot {
return {
layerId: layer.id,
resourceId: layer.resourceId,
title: layer.title,
x: layer.x,
y: layer.y,
width: layer.width,
height: layer.height,
originalWidth: layer.originalWidth,
originalHeight: layer.originalHeight,
zIndex: layer.zIndex,
sourceType: layer.sourceType,
sourceResourceId: layer.sourceResourceId,
assetKind: layer.assetKind,
mediaType: layer.mediaType,
objectKey: layer.objectKey,
assetObjectId: layer.assetObjectId,
durationSeconds: layer.durationSeconds,
};
}
function preserveSourceLayerInProjectSnapshot(
project: EditorProjectSnapshot,
sourceLayer: CanvasLayer,
): EditorProjectSnapshot {
if (
project.layers.some((layer) => layer.layerId === sourceLayer.id) ||
sourceLayer.resourceId.startsWith('generation-dialog:')
) {
return project;
}
const sourceResource = project.resources.some(
(resource) => resource.resourceId === sourceLayer.resourceId,
)
? null
: createProjectResourceSnapshotFromLayer(project.projectId, sourceLayer);
return {
...project,
resources: sourceResource
? [...project.resources, sourceResource]
: project.resources,
layers: [createProjectLayerSnapshotFromLayer(sourceLayer), ...project.layers],
};
}
function findSourceGenerationDialog(
dialogs: CanvasGenerationDialogState[],
sourceLayer: CanvasLayer,
@@ -1323,12 +1424,77 @@ export function useImageCanvasGenerationWorkflow({
});
layerCounterRef.current += 1;
const cropExpandIndex = layerCounterRef.current;
const cropExpandTitle = `${cropExpandSourceLayer.title} `;
const cropExpandProjectId = projectId?.trim();
let cropExpandResourceId = `local-resource-crop-expand-${cropExpandIndex}`;
let cropExpandImageSrc = result.imageSrc;
let cropExpandObjectKey: string | null = null;
let cropExpandAssetObjectId: string | null = null;
let cropExpandSourceResourceId: string | null =
cropExpandSourceLayer.resourceId;
let cropExpandAssetKind = cropExpandSourceLayer.assetKind;
if (cropExpandProjectId) {
const cropExpandFile = dataUrlToImageFile(
result.imageSrc,
`crop-expand-${cropExpandIndex}.png`,
);
const uploadedCropExpand = await uploadEditorMediaAssetFile(
cropExpandFile,
'image',
{
assetKind: 'editor_crop_expand_image',
pathSegments: [
'editor',
'crop-expand',
cropExpandProjectId,
`${Date.now()}`,
],
entityId: cropExpandProjectId,
metadata: {
editor_project_id: cropExpandProjectId,
source_resource_id: cropExpandSourceLayer.resourceId,
},
},
);
const cropExpandResource = await createEditorProjectResource(
cropExpandProjectId,
{
imageSrc: uploadedCropExpand.legacyPublicPath,
objectKey: uploadedCropExpand.objectKey,
assetObjectId: uploadedCropExpand.assetObjectId,
width: result.width,
height: result.height,
sourceType: 'generated',
prompt: cropExpandSourceLayer.prompt,
actualPrompt: cropExpandSourceLayer.actualPrompt,
model: cropExpandSourceLayer.model,
provider: cropExpandSourceLayer.provider,
taskId: cropExpandSourceLayer.taskId,
sourceResourceId: cropExpandSourceLayer.resourceId,
assetKind: cropExpandSourceLayer.assetKind,
generationInputs: cropExpandSourceLayer.generationInputs,
},
);
cropExpandResourceId = cropExpandResource.resourceId;
cropExpandImageSrc = cropExpandResource.imageSrc;
cropExpandObjectKey =
cropExpandResource.objectKey ?? uploadedCropExpand.objectKey;
cropExpandAssetObjectId =
cropExpandResource.assetObjectId ??
uploadedCropExpand.assetObjectId;
cropExpandSourceResourceId =
cropExpandResource.sourceResourceId ??
cropExpandSourceLayer.resourceId;
cropExpandAssetKind =
(cropExpandResource.assetKind as CanvasLayer['assetKind']) ??
cropExpandSourceLayer.assetKind;
}
const nextLayer: CanvasLayer = {
...cropExpandSourceLayer,
id: `layer-crop-expand-${cropExpandIndex}`,
resourceId: `local-resource-crop-expand-${cropExpandIndex}`,
title: `${cropExpandSourceLayer.title} 裁扩`,
src: result.imageSrc,
resourceId: cropExpandResourceId,
title: cropExpandTitle,
src: cropExpandImageSrc,
x: cropExpandSourceLayer.x + cropExpandSourceLayer.width + 32,
y: cropExpandSourceLayer.y,
width: result.width,
@@ -1337,10 +1503,11 @@ export function useImageCanvasGenerationWorkflow({
originalHeight: result.height,
zIndex: cropExpandIndex + 10,
sourceType: 'generated',
sourceResourceId: cropExpandSourceLayer.resourceId,
objectKey: null,
assetObjectId: null,
sourceResourceId: cropExpandSourceResourceId,
objectKey: cropExpandObjectKey,
assetObjectId: cropExpandAssetObjectId,
sourceAssetId: null,
assetKind: cropExpandAssetKind,
};
captureCanvasHistory();
appendCanvasLayersWithResources([nextLayer]);
@@ -1367,6 +1534,7 @@ export function useImageCanvasGenerationWorkflow({
fitLayers,
layerCounterRef,
persistGeneratedAsset,
projectId,
selectSingleLayer,
setActiveSidebarPanel,
]);
@@ -1397,6 +1565,14 @@ export function useImageCanvasGenerationWorkflow({
})
: undefined;
const backgroundRemovalDialogId = backgroundRemovalPlacement?.dialogId;
const applyBackgroundRemovalProjectSnapshot =
projectId && applyProjectSnapshot
? (project: EditorProjectSnapshot) => {
applyProjectSnapshot(
preserveSourceLayerInProjectSnapshot(project, sourceLayer),
);
}
: undefined;
try {
const sourceObjectKey = sourceLayer.objectKey?.trim();
const sourceImageSrc = sourceObjectKey
@@ -1428,12 +1604,17 @@ export function useImageCanvasGenerationWorkflow({
applyProjectSnapshot,
refreshTaskListForQueuedGeneration,
onWalletBalanceMayHaveChanged,
backgroundRemovalDialogId,
applyBackgroundRemovalProjectSnapshot
? (project) =>
preserveSourceLayerInProjectSnapshot(project, sourceLayer)
: undefined,
)
) {
return;
}
if (result.project && applyProjectSnapshot) {
applyProjectSnapshot(result.project);
if (result.project && applyBackgroundRemovalProjectSnapshot) {
applyBackgroundRemovalProjectSnapshot(result.project);
return;
}
if (backgroundRemovalPlacement?.placeholder && applyProjectSnapshot) {
@@ -421,6 +421,29 @@ function StageInteractionsHarness({
>
</button>
<button
type="button"
onClick={(event) => {
interaction.setShiftPressed(false);
interaction.handleLayerPointerDown(
createPointerEvent(event.currentTarget, {
pointerId: 10,
clientX: 220,
clientY: 60,
}),
secondLayer,
);
interaction.finishDrag(
createPointerEvent(getViewportElement(), {
pointerId: 10,
clientX: 220,
clientY: 60,
}),
);
}}
>
</button>
<button
type="button"
onClick={(event) => {
@@ -508,7 +531,7 @@ function StageInteractionsHarness({
</button>
<button
type="button"
onClick={(event) => {
onClick={() => {
const frameElement = document.createElement('div');
const dialog = asCanvasGenerationDialog(generateDialog);
if (!dialog) {
@@ -581,6 +604,34 @@ function StageInteractionsHarness({
>
</button>
<button
type="button"
onClick={(event) => {
const frameElement = document.createElement('div');
const dialog = asCanvasGenerationDialog(generateDialog);
if (!dialog) {
return;
}
interaction.setShiftPressed(false);
interaction.handleGenerationFramePointerDown(
createPointerEvent(frameElement, {
pointerId: 11,
clientX: 300,
clientY: 200,
}),
dialog,
);
interaction.finishDrag(
createPointerEvent(getViewportElement(), {
pointerId: 11,
clientX: 300,
clientY: 200,
}),
);
}}
>
</button>
<button
type="button"
onClick={(event) => {
@@ -674,6 +725,26 @@ describe('useImageCanvasStageInteractions', () => {
expect(screen.getByTestId('snap').textContent).toBe('-');
});
it('collapses an unmoved layer pointer gesture to a single selection', () => {
render(<StageInteractionsHarness />);
act(() => {
screen.getByRole('button', { name: '直接选第一层' }).click();
});
act(() => {
screen.getByRole('button', { name: '直接追加第二层' }).click();
});
expect(screen.getByTestId('selection').textContent).toBe(
'second:first,second',
);
act(() => {
screen.getByRole('button', { name: '直接点击第二层' }).click();
});
expect(screen.getByTestId('selection').textContent).toBe('second:second');
});
it('handles marquee, temporary hand mode, and generation frame dragging', () => {
const activateCanvasGenerationDialog = vi.fn();
render(
@@ -764,6 +835,30 @@ describe('useImageCanvasStageInteractions', () => {
);
});
it('collapses an unmoved generation frame pointer gesture to that frame', () => {
render(<StageInteractionsHarness />);
act(() => {
screen.getByRole('button', { name: '直接选第一层' }).click();
});
act(() => {
screen.getByRole('button', { name: '直接追加生成器' }).click();
});
const generationSelectionId =
screen.getByTestId('dialog-selection-id').textContent;
expect(screen.getByTestId('selection').textContent).toBe(
`first:first,${generationSelectionId}`,
);
act(() => {
screen.getByRole('button', { name: '直接点击生成器' }).click();
});
expect(screen.getByTestId('selection').textContent).toBe(
`-:${generationSelectionId}`,
);
});
it('distinguishes minimap click and drag movement', () => {
const moveViewportFromMinimapPointer = vi.fn();
const updateViewportFromMinimapDrag = vi.fn();
@@ -17,6 +17,7 @@ import {
moveViewportFromPan,
selectCanvasObjectsInsideMarquee,
} from './ImageCanvasInteractionModel';
import { getCanvasGenerationSelectionId } from './ImageCanvasSelectionModel';
import {
createCanvasMarqueeState,
createGenerationFrameSelectionStart,
@@ -42,6 +43,26 @@ import type {
SnapGuide,
} from './ImageCanvasEditorTypes';
const CLICK_COLLAPSE_MOVEMENT_THRESHOLD_PX = 3;
type PendingClickCollapse =
| {
kind: 'layer';
pointerId: number;
targetId: string;
startClientX: number;
startClientY: number;
isMultiSelectGesture: boolean;
}
| {
kind: 'generation-frame';
pointerId: number;
targetId: string;
startClientX: number;
startClientY: number;
isMultiSelectGesture: boolean;
};
type UseImageCanvasStageInteractionsOptions = {
canvasViewportRef: RefObject<HTMLDivElement | null>;
activeTool: CanvasTool;
@@ -97,6 +118,21 @@ function focusCanvasInteractionTarget(target: HTMLElement) {
target.focus({ preventScroll: true });
}
function shouldCollapsePointerSelection(
pending: PendingClickCollapse,
pointer: { x: number; y: number },
) {
if (pending.isMultiSelectGesture) {
return false;
}
return (
Math.hypot(
pointer.x - pending.startClientX,
pointer.y - pending.startClientY,
) < CLICK_COLLAPSE_MOVEMENT_THRESHOLD_PX
);
}
export function useImageCanvasStageInteractions({
canvasViewportRef,
activeTool,
@@ -137,6 +173,7 @@ export function useImageCanvasStageInteractions({
onCloseImageContextMenu,
}: UseImageCanvasStageInteractionsOptions) {
const dragStateRef = useRef<DragState | null>(null);
const pendingClickCollapseRef = useRef<PendingClickCollapse | null>(null);
const isShiftPressedRef = useRef(false);
const suppressNextLayerClickRef = useRef(false);
const isViewportInteractionActiveRef = useRef(false);
@@ -170,6 +207,7 @@ export function useImageCanvasStageInteractions({
flushMinimapViewportDrag();
}
dragStateRef.current = null;
pendingClickCollapseRef.current = null;
setCanvasMarquee(null);
setIsPanning(false);
setSnapGuide(null);
@@ -342,6 +380,14 @@ export function useImageCanvasStageInteractions({
updateGenerateDialogForLayerPointerDown(currentDialog, layer.id),
);
dragStateRef.current = layerDragStart.dragState;
pendingClickCollapseRef.current = {
kind: 'layer',
pointerId: getPointerId(event),
targetId: layer.id,
startClientX: pointer.x,
startClientY: pointer.y,
isMultiSelectGesture,
};
},
[
canvasViewportRef,
@@ -465,6 +511,14 @@ export function useImageCanvasStageInteractions({
setSelectedLayerId(frameDragStart.selectedLayerId);
setSelectedLayerIds(frameDragStart.selectedLayerIds);
dragStateRef.current = frameDragStart.dragState;
pendingClickCollapseRef.current = {
kind: 'generation-frame',
pointerId: getPointerId(event),
targetId: dialog.id,
startClientX: pointer.x,
startClientY: pointer.y,
isMultiSelectGesture,
};
},
[
activateCanvasGenerationDialog,
@@ -650,6 +704,26 @@ export function useImageCanvasStageInteractions({
const pointer = getPointerClient(event);
moveViewportFromMinimapPointer(pointer.x, pointer.y);
}
const pendingClickCollapse = pendingClickCollapseRef.current;
const pointer = getPointerClient(event);
if (
pendingClickCollapse &&
(pendingClickCollapse.pointerId < 0 ||
pointerId < 0 ||
pendingClickCollapse.pointerId === pointerId) &&
shouldCollapsePointerSelection(pendingClickCollapse, pointer)
) {
if (pendingClickCollapse.kind === 'layer') {
setSelectedLayerId(pendingClickCollapse.targetId);
setSelectedLayerIds([pendingClickCollapse.targetId]);
} else {
setSelectedLayerId(null);
setSelectedLayerIds([
getCanvasGenerationSelectionId(pendingClickCollapse.targetId),
]);
}
}
pendingClickCollapseRef.current = null;
if (dragState.kind === 'minimap') {
flushMinimapViewportDrag();
}
@@ -670,6 +744,8 @@ export function useImageCanvasStageInteractions({
finishViewportInteraction,
flushMinimapViewportDrag,
moveViewportFromMinimapPointer,
setSelectedLayerId,
setSelectedLayerIds,
],
);
@@ -15734,7 +15734,7 @@ export function PlatformEntryFlowShellImpl({
initial={{ opacity: 0, y: 12 }}
animate={{ opacity: 1, y: 0 }}
exit={{ opacity: 0, y: -12 }}
className="creation-home-stage flex h-full min-h-0 min-w-0 flex-col overflow-hidden"
className="creation-home-stage creation-home-stage--landing flex h-full min-h-0 min-w-0 flex-col overflow-hidden"
>
{platformHomeView}
</motion.div>
+28
View File
@@ -13022,6 +13022,34 @@ button.image-canvas-editor__reference-chip:disabled {
background: transparent;
}
.creation-home-stage--landing
.platform-desktop-shell--workbench
#platform-tab-panel-create {
-ms-overflow-style: auto;
scrollbar-width: thin;
}
.creation-home-stage--landing
.platform-desktop-shell--workbench
#platform-tab-panel-create::-webkit-scrollbar {
display: block;
width: 0.45rem;
height: 0.45rem;
}
.creation-home-stage--landing
.platform-desktop-shell--workbench
#platform-tab-panel-create::-webkit-scrollbar-thumb {
border-radius: 999px;
background: rgba(145, 67, 21, 0.34);
}
.creation-home-stage--landing
.platform-desktop-shell--workbench
#platform-tab-panel-create::-webkit-scrollbar-track {
background: rgba(255, 248, 240, 0.72);
}
.platform-desktop-trending-item {
position: relative;
overflow: hidden;
+7 -2
View File
@@ -128,10 +128,10 @@ describe('index stylesheet unread dots', () => {
'.platform-desktop-shell--workbench .platform-desktop-rail__button--primary .platform-desktop-rail__label,\n.platform-desktop-shell--workbench .platform-desktop-rail__button--active .platform-desktop-rail__label',
);
expect(css).toContain(
'.platform-desktop-shell--workbench .platform-desktop-rail__button--active .platform-desktop-rail__icon-shell',
'.platform-desktop-rail__button--active .platform-desktop-rail__icon-shell',
);
expect(css).toContain(
'.platform-desktop-shell--workbench .platform-desktop-rail__button--active .platform-desktop-rail__label',
'.platform-desktop-rail__button--active .platform-desktop-rail__label',
);
});
@@ -158,6 +158,11 @@ describe('index stylesheet unread dots', () => {
expect(css).toContain('.platform-page-stage::-webkit-scrollbar');
expect(css).toContain('.unified-creation-page::-webkit-scrollbar');
expect(css).toContain('.platform-work-detail__scroll::-webkit-scrollbar');
expect(css).toContain('.creation-home-stage--landing');
expect(css).toContain('scrollbar-width: thin;');
expect(css).toContain(
'.creation-home-stage--landing\n .platform-desktop-shell--workbench\n #platform-tab-panel-create::-webkit-scrollbar',
);
expect(css).toContain('.scrollbar-hide');
expect(css).toContain('::-webkit-scrollbar-thumb');