完善画板生成器与素材导出体验

图片改造改为独立 quick-edit 生成器占位并补齐提交链路
生成完成应用项目快照后同步刷新素材库
序列帧导出补充 Spine JSON 与独立验证器
序列帧播放预加载帧资源以减少加载闪烁
本地 dev 默认启用 all 进程角色并更新文档
This commit is contained in:
2026-06-25 00:12:01 +08:00
parent 6f846c72d7
commit 6bf8b0051b
40 changed files with 3762 additions and 135 deletions
@@ -527,7 +527,7 @@
- 背景:外部图片生成已从 HTTP 长请求迁到 `external_generation_job` 队列;跳一跳、拼消消和敲木鱼继续扩展时需要统一 job 粒度、前端等待展示和本地 / 生产验证口径。
- 决策:队列 BFF 暴露用户可见队列概览 `GET /api/runtime/external-generation/queue-overview` 和单 job 状态 `GET /api/runtime/external-generation/jobs/{jobId}`;首版固定“单动作单 job”,不拆提示词 / 生图 / 切图 / 持久化等阶段 job。进入队列的范围为跳一跳 `compile-draft` / `regenerate-tiles`、拼消消 `compile-draft` / `regenerate-atlas`、敲木鱼 `compile-draft` / `regenerate-hit-object` 图片资产动作;非外部图片生成动作继续 inline。
- 影响范围:外部生成 worker Module、api-server BFF、生成页等待展示、跳一跳 / 拼消消 / 敲木鱼创作与结果页生成动作、本地和生产验证文档。
- 验证方式:本地 `npm run dev` 默认保留 inline 开发体验;验证 worker 队列、等待展示、lease 或扩缩容时显式使用 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 并启动 worker,或运行 `npm run container:worker-smoke -- smoke`。部署后确认 `/healthz``/readyz`、队列概览 BFF、单 job 状态和对应玩法 session/detail 状态都能收敛。
- 验证方式:本地 `npm run dev` `npm run dev:api-server` 默认注入 `GENARRATIVE_PROCESS_ROLE=all`,同一 Rust 进程监听 HTTP 并消费外部生成队列;验证生产式拆分角色、lease 或扩缩容时分别启动 `api``external-generation-worker``external-generation-controller`,或运行 `npm run container:worker-smoke -- smoke`。部署后确认 `/healthz``/readyz`、队列概览 BFF、单 job 状态和对应玩法 session/detail 状态都能收敛。
- 关联文档:`docs/technical/【后端架构】外部生成Worker化方案-2026-06-03.md``docs/【开发运维】本地开发验证与生产运维-2026-05-15.md`
## 2026-06-11 本地服务器管理入口采用 SSH alias + egui 桌面面板
@@ -247,6 +247,22 @@
- 验证:`npm run test -- src/components/image-editor/ImageCanvasGenerationPlacementModel.test.ts src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx`
- 关联:`src/components/image-editor/ImageCanvasGenerationPlacementModel.ts``src/components/image-editor/useImageCanvasGenerationWorkflow.ts``docs/【编辑器】生成类面板Lovart统一改造方案-2026-06-17.md`
## 图片画布图片改造也必须创建独立生成器占位
- 现象:点击图片图层的“改造”后,输入框直接挂在原图上,提交时既不像其它生成入口一样有独立占位,也容易让用户误以为会覆盖源图。
- 原因:图片改造复用了旧 `QuickEditPanelState` / redraw 面板路径,只把源图选中并在原图附近打开快速编辑框,没有进入统一的 `CanvasGenerationDialogState` 占位链路。
- 处理:图片和用户快照图层的“改造”统一创建 `mode="quick-edit"` 的 generation dialog,占位仍走 `ImageCanvasGenerationPlacementModel`;源图作为隐式最后一张参考图提交,并把“当前图”提示词归一到对应参考图编号。音频改造继续走音频生成器路径,非图片 fallback 才保留旧面板。参考图 Data URL 提交前可压缩,但浏览器图片解码卡住时必须超时透传原图,不能阻塞生成请求。
- 验证:`npm run test -- src/components/image-editor/useImageCanvasGenerationWorkflow.test.tsx src/components/image-editor/ImageCanvasGenerationSubmissionModel.test.ts src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.test.tsx src/services/image-editor/editorImageReference.test.ts -- --runInBand`,以及 `npm run test -- src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx -t "hides quick edit and redraw panels|opens generated image info|shows the quick edit generator" -- --runInBand`
- 关联:`src/components/image-editor/useImageCanvasGenerationWorkflow.ts``src/components/image-editor/ImageCanvasGenerationSubmissionModel.ts``src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts``src/services/image-editor/editorImageReference.ts`
## 图片画布生成完成应用项目快照后也要刷新素材库
- 现象:部分素材生成成功后画布上已经出现结果,但左侧素材库没有立刻出现新素材,刷新页面后才显示。
- 原因:生成接口带 `project` 快照时,前端只调用 `applyProjectSnapshot(...)` 刷新画布布局;左侧素材库状态仍停留在首次 `loadEditorAssetLibrary()` 的结果。只有少数图片分支手动 `upsertGeneratedAsset`,图标素材图集、视频、音频、排队完成后重新 `loadEditorProject` 等分支不会统一更新素材库。
- 处理:素材库 hook 必须提供显式 `refreshAssetLibrary()`;传给生成工作流的项目快照应用函数应包装为“先应用项目快照,再刷新素材库”。新增生成分支不要在各自分支散落刷新逻辑,除非是无项目快照的本地图层回填,才继续使用 `generatedAssetSnapshot` / `upsertGeneratedAsset`
- 验证:`npm run test -- src/components/image-editor/ImageCanvasEditorGenerationIntegration.test.tsx -t "refreshes the asset library after an icon generation project snapshot is applied" -- --runInBand``npm run test -- src/components/image-editor/useImageCanvasAssetLibrary.test.tsx -- --runInBand`
- 关联:`src/components/image-editor/useImageCanvasAssetLibrary.ts``src/components/image-editor/ImageCanvasEditorView.tsx``src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`
## Windows 本地 dev 不要把 RUSTC_WRAPPER 绕过写成 rustc
- 现象:Windows 上执行 `npm run dev:api-server` 时,api-server 在 Cargo 启动阶段失败,日志出现 `error: multiple input filenames provided (first two filenames are ... rustc.exe and -)``/healthz` 无法访问。
@@ -56,12 +56,29 @@
│ ├─ frames/
│ │ ├─ frame-01.png
│ │ └─ frame-02.png
│ ├─ skeleton.json
│ ├─ README.md
│ ├─ metadata.json
│ └─ manifest.txt
├─ metadata.json
└─ manifest.txt
```
角色动作单图层导出时,直接生成独立 ZIP:
```text
角色动作-SpineJSON.zip
├─ frames/
│ ├─ frame-01.png
│ └─ frame-02.png
├─ skeleton.json
├─ README.md
├─ metadata.json
└─ manifest.txt
```
`skeleton.json` 使用 Spine JSON 4.2 结构表达序列帧:一个 `root` bone、一个 `character` slot、每帧一个 region attachment`animations.animation.slots.character.attachment` 按时间切换 attachment。它不是 `.spine` 编辑器工程,也不包含 mesh、权重、IK 或拆件绑定。
## 元数据结构
`metadata.json` 记录项目、导出时间、图层和图片文件映射:
@@ -134,6 +151,7 @@ assetObjectId > objectKey > sourceAssetId > src
- 同源或可访问 URL 使用 `fetch` 拉取 Blob。
- 私有 generated / OSS 素材必须走同源 `/api/assets/read-bytes` 读取字节;不要在导出流程里直接 `fetch` OSS 签名 URL,避免浏览器 CORS 拦截。
- `mediaType="image-sequence"` 逐帧读取 `imageSequenceFrames`,写入 `sequences/<编号-标题>/frames/`
- `mediaType="image-sequence"` 同步写入 `skeleton.json`,供 Spine Editor 或 runtime 以 slot attachment timeline 方式播放序列帧。
- 拉取失败时记录失败项,不中断整个导出。
5. 使用 `JSZip` 写入 `images/``metadata.json``manifest.txt`
6. `zip.generateAsync({ type: 'blob' })` 生成文件。
@@ -158,6 +176,32 @@ assetObjectId > objectKey > sourceAssetId > src
- 图层的锁定、翻转、分组状态写入元数据。
- `data:image` 图片不经过网络请求即可导出。
- URL 图片 fetch 失败时不中断其他素材导出。
- 序列帧导出的 `skeleton.json` 可被独立验证器解析并预览。
## Spine JSON 验证器
导出产物验证项目位于 `tools/spine-json-export-validator/`,独立于主站构建。它用 `@esotericsoftware/spine-webgl``SkeletonJson`、自定义 `AttachmentLoader``GLTexture``SceneRenderer` 直接验证导出的 `skeleton.json + frames/`,不要求 atlas。
首次使用安装验证器自己的依赖:
```bash
npm --prefix tools/spine-json-export-validator install
```
本地启动:
```bash
npm run spine-export-validator:dev
```
检查命令:
```bash
npm run spine-export-validator:typecheck
npm run spine-export-validator:build
```
验证器支持根目录 `skeleton.json`,也支持画布素材 ZIP 内 `sequences/<编号-标题>/skeleton.json`。当前只验证 region attachment;如果未来导出 mesh、sequence attachment 或多动画命名,需要同步扩展验证器的 loader 和 UI 摘要。
## 后续扩展
@@ -208,11 +208,11 @@ npm run test -- src/components/rpg-entry/RpgEntryFlowShell.agent.interaction.tes
本地 smoke
```bash
GENARRATIVE_PROCESS_ROLE=all npm run dev
npm run dev
curl -f http://127.0.0.1:<api-port>/healthz
```
本地 `npm run dev` 默认保持 `inline` 开发体验:未显式配置 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue` 时,普通本地联调可以同步确认 provider、OSS 和 SpacetimeDB 写回链路本身是否可行。需要验证 worker 队列、BFF 队列状态、lease 重领或扩缩容时,必须显式使用 `queue`,并启动 worker 角色;可以用 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue GENARRATIVE_PROCESS_ROLE=all npm run dev:api-server` 做临时单进程 smoke,也可以使用隔离容器 smoke。
本地 `npm run dev` `npm run dev:api-server` 默认注入 `GENARRATIVE_PROCESS_ROLE=all`,同一 Rust 进程同时监听 HTTP 并消费外部生成队列;显式设置 `GENARRATIVE_PROCESS_ROLE` 时保留显式值。需要验证生产式拆分角色、lease 重领或扩缩容时,再分别启动 `api``external-generation-worker``external-generation-controller`,也可以使用隔离容器 smoke。
生产 smoke 需要保持 `GENARRATIVE_EXTERNAL_GENERATION_MODE=queue`,并至少启动一个 `api` 角色、一个 `external-generation-worker` 角色和一个 `external-generation-controller` 角色;发布脚本会在默认 worker pattern 下自动启用并启动 `genarrative-external-generation-worker@1.service`,重启并验活 `genarrative-external-generation-controller.service`。若 worker 数量归零,生成任务会保持 `queued/running`,不会由 HTTP 进程偷偷执行。部署验证除 `/healthz` / `/readyz` 外,还要确认任务列表 BFF 可读、未确认终态任务会弹出提示、提示展示后后台 acknowledge 且刷新后不再弹出,单 job 状态能从 `queued/running` 收敛到业务 session/detail 的 ready 或 failed。
File diff suppressed because one or more lines are too long
+3
View File
@@ -66,6 +66,9 @@
"deploy:pingora-direct-rollback": "bash scripts/deploy/pingora-direct-rollback.sh",
"assets:child-motion-demo": "node scripts/generate-child-motion-demo-assets.mjs",
"assets:match3d-style-references": "node scripts/generate-match3d-style-references.mjs",
"spine-export-validator:dev": "npm --prefix tools/spine-json-export-validator run dev --",
"spine-export-validator:typecheck": "npm --prefix tools/spine-json-export-validator run typecheck",
"spine-export-validator:build": "npm --prefix tools/spine-json-export-validator run build",
"check:visual-novel-vn11": "node scripts/check-visual-novel-vn11-negative-scan.mjs",
"check:visual-novel-vn12": "node scripts/check-visual-novel-vn12-acceptance.mjs",
"check:wechat-miniprogram-auth": "node scripts/check-wechat-miniprogram-auth-smoke.mjs",
+1
View File
@@ -2253,6 +2253,7 @@ function buildApiServerProcessEnv({
...baseEnv,
// 本地 dev 允许密码入口直接创建账号,生产默认仍由 api-server 配置保持关闭。
GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED: 'true',
GENARRATIVE_PROCESS_ROLE: baseEnv.GENARRATIVE_PROCESS_ROLE || 'all',
GENARRATIVE_API_HOST: options.apiHost,
GENARRATIVE_API_PORT: String(options.apiPort),
GENARRATIVE_API_LOG: options.apiLog,
+12
View File
@@ -211,10 +211,22 @@ describe('dev scheduler api-server env', () => {
});
expect(env.GENARRATIVE_DEV_PASSWORD_ENTRY_AUTO_REGISTER_ENABLED).toBe('true');
expect(env.GENARRATIVE_PROCESS_ROLE).toBe('all');
expect(env.GENARRATIVE_API_PORT).toBe('9091');
expect(env.GENARRATIVE_SPACETIME_SERVER_URL).toBe('http://127.0.0.1:3199');
});
test('dev 脚本保留显式 api-server 进程角色', () => {
const {options} = parseArgs(['api-server'], {});
const env = buildApiServerProcessEnv({
baseEnv: {GENARRATIVE_PROCESS_ROLE: 'api'},
options,
state: {spacetimeServer: 'http://127.0.0.1:3199'},
});
expect(env.GENARRATIVE_PROCESS_ROLE).toBe('api');
});
test('Windows 本地 dev 自动注入已安装的 FFmpeg 路径', () => {
const tempDir = mkdtempSync(join(tmpdir(), 'genarrative-ffmpeg-'));
try {
@@ -91,6 +91,7 @@ export function ImageCanvasBasicGenerationComposerView({
onSubmit,
}: ImageCanvasBasicGenerationComposerViewProps) {
const references = dialog.generationReferences ?? [];
const isQuickEdit = dialog.mode === 'quick-edit';
useImageCanvasFloatingOptionDismiss({
isOpen: isGenerationReferenceMenuOpen,
boundaryRefs: [generationReferenceButtonRef],
@@ -103,7 +104,7 @@ export function ImageCanvasBasicGenerationComposerView({
className="image-canvas-editor__generation-composer image-canvas-editor__generation-composer--image"
style={style}
role="dialog"
aria-label="生成图片"
aria-label={isQuickEdit ? '改造图片' : '生成图片'}
onPointerDown={(event) => event.stopPropagation()}
onSubmit={(event) => {
event.preventDefault();
@@ -145,10 +146,10 @@ export function ImageCanvasBasicGenerationComposerView({
</div>
<PlatformTextField
variant="textarea"
aria-label="生成提示词"
aria-label={isQuickEdit ? '改造提示词' : '生成提示词'}
value={dialog.prompt}
disabled={dialog.status === 'generating'}
placeholder="今天想生成什么画面?"
placeholder={isQuickEdit ? '想怎么改造这张图?' : '今天想生成什么画面?'}
size="sm"
density="compact"
className="image-canvas-editor__generation-prompt"
@@ -169,7 +170,7 @@ export function ImageCanvasBasicGenerationComposerView({
setGenerateDialog={setGenerateDialog}
includeDimensions
onRememberImageModel={onRememberImageModel}
optionLabelPrefix="生成图片"
optionLabelPrefix={isQuickEdit ? '改造图片' : '生成图片'}
cost={calculateEditorImageGenerationPrice({
model: dialog.imageModel,
imageSize: dialog.imageSize,
@@ -115,9 +115,24 @@ describe('ImageCanvasContextMenusView', () => {
expect(props.onToggleContextLayerLock).toHaveBeenCalledTimes(1);
expect(props.onFlipContextLayers).toHaveBeenCalledWith('x');
expect(props.onOpenCharacterAnimationPanel).toHaveBeenCalledWith(layer);
expect(props.onCloseContextMenu).toHaveBeenCalledTimes(1);
expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(1);
expect(props.onDeleteContextLayers).toHaveBeenCalledTimes(1);
});
it('closes the standalone image menu after opening character animation', () => {
const layer = createLayer({ assetKind: 'character' });
const props = renderContextMenus({
imageContextMenu: { layerId: layer.id, x: 20, y: 22 },
imageContextMenuLayer: layer,
});
fireEvent.click(screen.getByRole('menuitem', { name: '生成动画' }));
expect(props.onOpenCharacterAnimationPanel).toHaveBeenCalledWith(layer);
expect(props.onCloseImageContextMenu).toHaveBeenCalledTimes(1);
});
it('renders the image context menu when no canvas context menu is open', () => {
const layer = createLayer({ assetKind: 'character' });
const props = renderContextMenus({
@@ -249,9 +249,11 @@ export function ImageCanvasContextMenusView({
<button
type="button"
role="menuitem"
onClick={() =>
onOpenCharacterAnimationPanel(imageContextMenuLayer)
}
onClick={() => {
onOpenCharacterAnimationPanel(imageContextMenuLayer);
onCloseContextMenu();
onCloseImageContextMenu();
}}
>
</button>
@@ -300,7 +302,10 @@ export function ImageCanvasContextMenusView({
{imageContextMenuLayer.assetKind === 'character' ? (
<PlatformFloatingMenuItem
className="image-canvas-editor__context-menu-item"
onClick={() => onOpenCharacterAnimationPanel(imageContextMenuLayer)}
onClick={() => {
onOpenCharacterAnimationPanel(imageContextMenuLayer);
onCloseImageContextMenu();
}}
>
</PlatformFloatingMenuItem>
@@ -985,8 +985,8 @@ describe('ImageCanvasEditorView generation integration', () => {
clientY: 120,
});
fireEvent.click(screen.getByRole('button', { name: '改造' }));
expect(screen.getByRole('dialog', { name: '重绘图片' })).toBeTruthy();
expect(screen.queryByRole('button', { name: '关闭重绘图片' })).toBeNull();
expect(screen.getByRole('dialog', { name: '改造图片' })).toBeTruthy();
expect(screen.getByLabelText('快速编辑生成占位图')).toBeTruthy();
fireEvent.pointerDown(viewport, {
button: 0,
@@ -995,7 +995,7 @@ describe('ImageCanvasEditorView generation integration', () => {
clientY: 190,
});
expect(screen.queryByRole('dialog', { name: '重绘图片' })).toBeNull();
expect(screen.queryByRole('dialog', { name: '改造图片' })).toBeNull();
});
it('does not render a standalone close button on the generation composer', () => {
@@ -2413,6 +2413,148 @@ describe('ImageCanvasEditorView generation integration', () => {
expect(within(iconInfoPanel).getByText('清爽按钮图标规范')).toBeTruthy();
});
it('refreshes the asset library after an icon generation project snapshot is applied', async () => {
loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({
projectId: 'editor-project-icon-library-refresh',
title: '图标素材刷新画布',
viewport: { x: 0, y: 0, scale: 1 },
layers: [
{
layerId: 'layer-icon-spec-refresh',
resourceId: 'resource-icon-spec-refresh',
title: '清爽按钮图标规范',
src: 'data:image/png;base64,icon-spec-refresh',
x: 240,
y: 80,
width: 160,
height: 120,
originalWidth: 2048,
originalHeight: 1152,
zIndex: 11,
sourceType: 'generated',
assetKind: 'icon-spec',
},
],
resources: [],
updatedAt: '2026-06-15T00:00:00.000Z',
});
loadEditorAssetLibraryMock
.mockResolvedValueOnce({
folders: [
{
folderId: 'project',
label: '项目素材',
sortOrder: 0,
collapsed: false,
systemDefault: true,
},
],
assets: [],
})
.mockResolvedValueOnce({
folders: [
{
folderId: 'project',
label: '项目素材',
sortOrder: 0,
collapsed: false,
systemDefault: true,
},
],
assets: [
createGeneratedAsset({
assetId: 'asset-icon-spritesheet-refresh',
folderId: 'project',
label: '清爽按钮素材图集',
imageSrc: 'data:image/png;base64,icon-sheet-refresh',
width: 512,
height: 512,
assetKind: 'icon-spritesheet',
}),
],
});
const spritesheetResource = createGeneratedResource({
resourceId: 'resource-icon-spritesheet-refresh',
projectId: 'editor-project-icon-library-refresh',
imageSrc: 'data:image/png;base64,icon-sheet-refresh',
width: 512,
height: 512,
assetKind: 'icon-spritesheet',
});
generateEditorIconSpritesheetMock.mockResolvedValueOnce({
spritesheetImageSrc: 'data:image/png;base64,icon-sheet-refresh',
spritesheetWidth: 512,
spritesheetHeight: 512,
iconImageSrcs: [],
prompt: '图标 prompt',
actualPrompt: '图标 prompt',
model: 'gemini-3.1-flash-image-preview',
provider: 'VectorEngine',
taskId: 'icon-library-refresh-task',
priceMudPoints: 3,
spritesheetResource,
spritesheetAsset: createGeneratedAsset({
assetId: 'asset-icon-spritesheet-refresh',
folderId: 'project',
label: '清爽按钮素材图集',
imageSrc: 'data:image/png;base64,icon-sheet-refresh',
width: 512,
height: 512,
assetKind: 'icon-spritesheet',
}),
project: {
projectId: 'editor-project-icon-library-refresh',
title: '图标素材刷新画布',
viewport: { x: 0, y: 0, scale: 1 },
layers: [],
resources: [spritesheetResource],
updatedAt: '2026-06-15T00:01:00.000Z',
},
});
render(<ImageCanvasEditorView />);
await screen.findByAltText('画布图片:清爽按钮图标规范');
await waitFor(() => {
expect(loadEditorAssetLibraryMock).toHaveBeenCalledTimes(1);
});
expect(
screen.queryByRole('button', { name: '添加清爽按钮素材图集' }),
).toBeNull();
fireEvent.click(screen.getByRole('button', { name: '生成图标素材' }));
const iconPanel = screen.getByRole('dialog', { name: '生成图标素材' });
fireEvent.click(
within(iconPanel).getByRole('button', { name: '图标规范' }),
);
fireEvent.click(screen.getByRole('menuitem', { name: '从画布中选择' }));
fireEvent.pointerDown(
screen.getByAltText('画布图片:清爽按钮图标规范').closest('button')!,
{
button: 0,
pointerId: 182,
clientX: 260,
clientY: 100,
},
);
fireEvent.change(
within(iconPanel).getByRole('textbox', { name: '素材描述' }),
{
target: { value: '返回按钮' },
},
);
fireEvent.click(within(iconPanel).getByRole('button', { name: '生成' }));
await waitFor(() => {
expect(generateEditorIconSpritesheetMock).toHaveBeenCalled();
});
await waitFor(() => {
expect(loadEditorAssetLibraryMock).toHaveBeenCalledTimes(2);
});
expect(
await screen.findByRole('button', { name: '添加清爽按钮素材图集' }),
).toBeTruthy();
});
it('allows icon asset generation to reference icon specs created from the renamed spec entry', async () => {
generateEditorImageMock.mockImplementationOnce(async (input) =>
createGeneratedImageResponse(input, {
@@ -3457,12 +3599,12 @@ describe('ImageCanvasEditorView generation integration', () => {
expect(within(metadataDialog).getByText('1024 x 1024 px')).toBeTruthy();
fireEvent.click(screen.getByRole('button', { name: '改造' }));
const editDialog = screen.getByRole('dialog', { name: '重绘图片' });
const editDialog = screen.getByRole('dialog', { name: '改造图片' });
expect(editDialog).toBeTruthy();
const editPrompt = within(editDialog).getByLabelText('重绘提示词');
const editPrompt = within(editDialog).getByLabelText('改造提示词');
expect(editPrompt.className).toContain('platform-text-field');
expect(editPrompt.className).toContain(
'image-canvas-editor__quick-edit-prompt',
'image-canvas-editor__generation-prompt',
);
fireEvent.change(editPrompt, {
target: { value: '把画面改成黄昏光线' },
@@ -3484,32 +3626,28 @@ describe('ImageCanvasEditorView generation integration', () => {
});
await waitFor(() => {
expect(screen.getByRole('dialog', { name: '重绘图片' })).toBeTruthy();
expect(screen.queryByRole('dialog', { name: '改造图片' })).toBeNull();
});
expect(screen.getByAltText(/画布图片:生成图片 .* 重绘/)).toBeTruthy();
expect(screen.getByAltText(/画布图片:生成图片 .* 快速编辑/)).toBeTruthy();
fireEvent.click(
screen.getAllByRole('button', {
name: / .* /u,
name: / .* /u,
})[0]!,
);
const editedMetadataDialog = screen.getByRole('dialog', {
name: '图片信息',
});
expect(within(editedMetadataDialog).queryByText('Prompt')).toBeNull();
expect(within(editedMetadataDialog).getByText('重绘提示词')).toBeTruthy();
expect(within(editedMetadataDialog).getByText('快速编辑提示词')).toBeTruthy();
expect(
within(editedMetadataDialog).getByText('把画面改成黄昏光线'),
).toBeTruthy();
expect(within(editedMetadataDialog).getByText('参考图')).toBeTruthy();
expect(
within(editedMetadataDialog).getByText(/^ \d+$/u),
).toBeTruthy();
expect(
screen.getByRole('button', { name: / \d+%/u }),
).toBeTruthy();
});
it('keeps the redraw panel open after generation starts while keeping the source preview visible', async () => {
it('shows the quick edit generator after redraw starts while keeping the source preview visible', async () => {
loadOrCreateRecentEditorProjectMock.mockResolvedValueOnce({
projectId: 'editor-project-edit-generating',
title: '重绘生成中画布',
@@ -3555,20 +3693,18 @@ describe('ImageCanvasEditorView generation integration', () => {
clientY: 180,
});
fireEvent.click(screen.getByRole('button', { name: '改造' }));
const editDialog = screen.getByRole('dialog', { name: '重绘图片' });
fireEvent.change(within(editDialog).getByLabelText('重绘提示词'), {
const editDialog = screen.getByRole('dialog', { name: '改造图片' });
fireEvent.change(within(editDialog).getByLabelText('改造提示词'), {
target: { value: '改成雨夜灯光' },
});
fireEvent.click(within(editDialog).getByRole('button', { name: '生成' }));
expect(screen.getByRole('dialog', { name: '重绘图片' })).toBeTruthy();
expect(screen.queryByRole('dialog', { name: '改造图片' })).toBeNull();
expect(screen.getByLabelText('快速编辑生成占位图')).toBeTruthy();
expect(screen.getByAltText('画布图片:待重绘')).toBeTruthy();
expect(sourceLayer.className).not.toContain(
'image-canvas-editor__layer--generating',
);
expect(
within(editDialog).getByRole('button', { name: '生成中' }),
).toBeTruthy();
});
it('hides the quick edit panel after generation starts while keeping the source preview visible', async () => {
@@ -13,6 +13,7 @@ import {
createEditorAsset,
createEditorProjectResource,
type EditorAssetSnapshot,
type EditorProjectSnapshot,
loadEditorGenerationPricing,
} from '../../services/image-editor/editorProjectClient';
import { shouldShowRechargeEntry } from '../../services/payment/paymentPlatform';
@@ -394,6 +395,7 @@ export function ImageCanvasEditorView() {
handleAssetMarqueePointerDown,
handleAssetMarqueePointerMove,
handleAssetMarqueePointerUp,
refreshAssetLibrary,
} = useImageCanvasAssetLibrary({
assetListRef,
canAccessProtectedData: authUi ? authUi.canAccessProtectedData : true,
@@ -670,6 +672,13 @@ export function ImageCanvasEditorView() {
canAccessProtectedData: authUi ? authUi.canAccessProtectedData : true,
openEditorLoginModal,
});
const applyGeneratedProjectSnapshot = useCallback(
(project: EditorProjectSnapshot) => {
applyProjectSnapshot(project);
void refreshAssetLibrary();
},
[applyProjectSnapshot, refreshAssetLibrary],
);
const persistUpdatedLayerResource = useCallback(
(layer: CanvasLayer) => {
if (!projectId) {
@@ -788,7 +797,7 @@ export function ImageCanvasEditorView() {
projectId,
assetFolderId: activeUploadFolderId,
upsertGeneratedAsset,
applyProjectSnapshot,
applyProjectSnapshot: applyGeneratedProjectSnapshot,
onWalletBalanceMayHaveChanged: refreshEditorWalletBalance,
});
const {
@@ -1,7 +1,10 @@
import { describe, expect, it, vi } from 'vitest';
import type { CanvasLayer } from './ImageCanvasEditorTypes';
import {
buildLayerExportMetadata,
buildSpineImageSequenceJson,
buildSpineImageSequenceReadme,
dataUrlToBlob,
formatExportDate,
getImageExtensionFromTypeOrSrc,
@@ -11,7 +14,6 @@ import {
readLayerImageSequenceFrameBlob,
sanitizeExportFilePart,
} from './ImageCanvasExportModel';
import type { CanvasLayer } from './ImageCanvasEditorTypes';
import {
DEFAULT_ICON_DESCRIPTIONS,
UI_DESIGN_ASSET_EXTRACTION_PROMPT,
@@ -169,6 +171,56 @@ describe('ImageCanvasExportModel', () => {
});
});
it('builds a Spine JSON attachment timeline from image sequence frames', () => {
const layer = buildLayer({
mediaType: 'image-sequence',
originalWidth: 192,
originalHeight: 256,
durationSeconds: 0.5,
});
const spineJson = buildSpineImageSequenceJson({
layer,
frames: [
{ frameIndex: 1, fileName: 'frame-01.png', width: 192, height: 256 },
{ frameIndex: 2, fileName: 'frame-02.png', width: 192, height: 256 },
{ frameIndex: 3, fileName: 'frame-03.png', width: 192, height: 256 },
{ frameIndex: 4, fileName: 'frame-04.png', width: 192, height: 256 },
],
});
expect(spineJson.skeleton).toMatchObject({
images: './frames/',
fps: 8,
width: 192,
height: 256,
});
expect(spineJson.bones).toEqual([{ name: 'root' }]);
expect(spineJson.slots[0]).toEqual({
name: 'character',
bone: 'root',
attachment: 'animation_0001',
});
expect(
spineJson.skins[0]!.attachments.character.animation_0001,
).toMatchObject({
type: 'region',
path: 'frame-01',
y: 128,
width: 192,
height: 256,
});
expect(
spineJson.animations.animation.slots.character.attachment,
).toEqual([
{ time: 0, name: 'animation_0001' },
{ time: 0.125, name: 'animation_0002' },
{ time: 0.25, name: 'animation_0003' },
{ time: 0.375, name: 'animation_0004' },
{ time: 0.5, name: 'animation_0001' },
]);
expect(buildSpineImageSequenceReadme(layer)).toContain('skeleton.json');
});
it('reads private object-key assets through same-origin bytes before exporting', async () => {
const originalFetch = globalThis.fetch;
const fetchMock = vi.fn(async (url: string) => {
@@ -1,9 +1,11 @@
import { readAssetBytes } from '../../services/assetReadUrlService';
import {
readAssetBytes,
} from '../../services/assetReadUrlService';
import type {
CanvasAssetExportMetadata,
CanvasGenerationInputField,
CanvasMediaType,
CanvasLayer,
CanvasMediaType,
} from './ImageCanvasEditorTypes';
import {
DEFAULT_ICON_DESCRIPTIONS,
@@ -210,6 +212,119 @@ export function getImageSequenceFrameFileName(
return `frame-${frameNumber}.${extension}`;
}
export type ExportedImageSequenceFrame = {
frameIndex: number;
fileName: string;
width: number;
height: number;
};
const DEFAULT_SPINE_VERSION = '4.2.00';
const SPINE_SEQUENCE_BONE_NAME = 'root';
const SPINE_SEQUENCE_SLOT_NAME = 'character';
const SPINE_SEQUENCE_ANIMATION_NAME = 'animation';
function stripFileExtension(fileName: string) {
return fileName.replace(/\.[^.]+$/u, '');
}
function roundSpineTime(value: number) {
return Number(value.toFixed(6));
}
export function buildSpineImageSequenceJson({
layer,
frames,
}: {
layer: CanvasLayer;
frames: ExportedImageSequenceFrame[];
}) {
if (!frames.length) {
throw new Error('序列帧为空');
}
const durationSeconds =
typeof layer.durationSeconds === 'number' && layer.durationSeconds > 0
? layer.durationSeconds
: frames.length;
const frameDuration = durationSeconds / frames.length;
const skeletonWidth = frames[0]?.width || layer.originalWidth;
const skeletonHeight = frames[0]?.height || layer.originalHeight;
const attachments = Object.fromEntries(
frames.map((frame, index) => {
const attachmentName = `${SPINE_SEQUENCE_ANIMATION_NAME}_${String(
index + 1,
).padStart(4, '0')}`;
return [
attachmentName,
{
type: 'region',
path: stripFileExtension(frame.fileName),
x: 0,
y: roundSpineTime(frame.height / 2),
width: frame.width,
height: frame.height,
},
];
}),
);
const timeline = frames.map((_frame, index) => ({
time: roundSpineTime(index * frameDuration),
name: `${SPINE_SEQUENCE_ANIMATION_NAME}_${String(index + 1).padStart(
4,
'0',
)}`,
}));
timeline.push({
time: roundSpineTime(durationSeconds),
name: `${SPINE_SEQUENCE_ANIMATION_NAME}_0001`,
});
return {
skeleton: {
spine: DEFAULT_SPINE_VERSION,
images: './frames/',
fps: Math.max(1, Math.round(frames.length / durationSeconds)),
width: skeletonWidth,
height: skeletonHeight,
},
bones: [{ name: SPINE_SEQUENCE_BONE_NAME }],
slots: [
{
name: SPINE_SEQUENCE_SLOT_NAME,
bone: SPINE_SEQUENCE_BONE_NAME,
attachment: `${SPINE_SEQUENCE_ANIMATION_NAME}_0001`,
},
],
skins: [
{
name: 'default',
attachments: {
[SPINE_SEQUENCE_SLOT_NAME]: attachments,
},
},
],
animations: {
[SPINE_SEQUENCE_ANIMATION_NAME]: {
slots: {
[SPINE_SEQUENCE_SLOT_NAME]: {
attachment: timeline,
},
},
},
},
};
}
export function buildSpineImageSequenceReadme(layer: CanvasLayer) {
return [
`素材:${layer.title}`,
'内容:Spine JSON 序列帧导入包',
'导入:在 Spine Editor 中导入 skeleton.json,并保持 frames/ 目录与 skeleton.json 同级。',
'说明:该 JSON 使用单 slot attachment timeline 播放序列帧,不包含拆件、骨骼绑定或 mesh 权重。',
].join('\n');
}
export async function blobToUint8Array(blob: Blob) {
if (typeof blob.arrayBuffer === 'function') {
return new Uint8Array(await blob.arrayBuffer());
@@ -985,7 +985,8 @@ export function ImageCanvasGenerationComposerView({
)
: null}
{generateDialog?.mode === 'generate' &&
{(generateDialog?.mode === 'generate' ||
generateDialog?.mode === 'quick-edit') &&
generateDialog.composerOpen !== false &&
generationComposerStyle ? (
<ImageCanvasBasicGenerationComposerView
@@ -449,6 +449,25 @@ describe('ImageCanvasGenerationModel', () => {
}),
),
).toBe('请先登录后再生成图片');
expect(
resolveImageGenerationErrorMessage(
new ApiClientError({
message: '请求冲突(requestId: wallet-one',
status: 409,
code: 'CONFLICT',
details: { provider: 'profile-wallet' },
}),
),
).toBe('泥点余额不足,请充值后再生成');
expect(
resolveImageGenerationErrorMessage(
new ApiClientError({
message: '请求冲突(requestId: conflict-one',
status: 409,
code: 'CONFLICT',
}),
),
).toBe('生成任务未完成,请稍后重试');
});
});
@@ -1304,7 +1304,10 @@ export function getGenerationFrameLabel(dialog: CanvasGenerationDialogState) {
return 'Image Generator';
}
export function resolveImageGenerationErrorMessage(error: unknown) {
export function resolveImageGenerationErrorMessage(
error: unknown,
fallbackMessage = '生成图片失败',
) {
if (
error instanceof ApiClientError &&
(error.status === 401 || error.status === 403)
@@ -1312,7 +1315,22 @@ export function resolveImageGenerationErrorMessage(error: unknown) {
return '请先登录后再生成图片';
}
if (error instanceof ApiClientError && error.status === 409) {
const message = error.message.trim();
if (message && !message.startsWith('请求冲突')) {
return message;
}
const provider =
typeof error.details?.provider === 'string'
? error.details.provider.trim()
: '';
if (provider === 'profile-wallet') {
return '泥点余额不足,请充值后再生成';
}
return '生成任务未完成,请稍后重试';
}
return error instanceof Error && error.message.trim()
? error.message
: '生成图片失败';
: fallbackMessage;
}
@@ -118,6 +118,61 @@ describe('ImageCanvasGenerationSubmissionModel', () => {
});
});
it('builds quick-edit canvas generator plans with the source layer as the final reference', () => {
const sourceLayer = createLayer({
objectKey: 'generated-character-drafts/editor/source.png',
assetKind: 'character',
});
const plan = buildImageGenerationSubmissionPlan({
dialog: {
mode: 'quick-edit',
prompt: '把当前图改成雨天',
status: 'idle',
sourceLayerId: sourceLayer.id,
imageModel: IMAGE_MODEL_GPT_IMAGE_2,
aspectRatio: '4:3',
imageSize: '1K',
generationReferences: [
{
id: 'style-reference',
label: '风格参考',
src: 'data:image/png;base64,style',
},
],
},
layers: [sourceLayer],
nextGeneratedIndex: 4,
});
expect(plan).toMatchObject({
kind: 'quick-edit',
normalizedPrompt: '把图2改成雨天',
sourceLayer,
input: {
prompt: '把图2改成雨天',
size: '1024x768',
kind: 'character',
model: IMAGE_MODEL_GPT_IMAGE_2,
aspectRatio: '4:3',
imageSize: '1K',
referenceImageSrcs: [
'data:image/png;base64,style',
'generated-character-drafts/editor/source.png',
],
},
result: {
title: '源图 快速编辑',
assetKind: 'character',
generationInputs: {
fields: [{ title: '快速编辑提示词', value: '把图2改成雨天' }],
references: [],
},
},
rememberImageModel: IMAGE_MODEL_GPT_IMAGE_2,
});
});
it('builds spec generation plans with reference prompt semantics', () => {
const plan = buildImageGenerationSubmissionPlan({
dialog: {
@@ -21,6 +21,7 @@ import {
buildPublicationMaterialsGenerationInputs,
buildPublicationMaterialsGenerationPrompt,
buildPublicationMaterialsPrompt,
buildQuickEditGenerationInputs,
buildSoundEffectGenerationInputs,
buildSpecGenerationInputs,
buildSpecPrompt,
@@ -38,8 +39,12 @@ import {
DEFAULT_VIDEO_WEB_SEARCH_ENABLED,
ICON_DESCRIPTION_LIMIT,
IMAGE_MODEL_GPT_IMAGE_2,
inferEditorImageAspectRatio,
inferEditorImageSizeLabel,
normalizeEditorImageModel,
normalizeQuickEditPromptSourceReference,
resolveCharacterAnimationSourceImageSrc,
resolveEditorImageGenerationPixelSize,
SEEDANCE_VIDEO_REFERENCE_LIMITS,
SPEC_GENERATION_ASPECT_RATIO,
SPEC_GENERATION_IMAGE_SIZE,
@@ -157,6 +162,18 @@ export type ImageGenerationSubmissionPlan =
};
rememberImageModel?: string;
}
| {
kind: 'quick-edit';
normalizedPrompt: string;
sourceLayer: CanvasLayer;
input: EditorImageGenerationInput;
result: {
title: string;
assetKind?: CanvasLayer['assetKind'];
generationInputs: CanvasGenerationInputs;
};
rememberImageModel?: string;
}
| {
kind: 'video';
normalizedPrompt: string;
@@ -232,6 +249,70 @@ export function buildImageGenerationSubmissionPlan({
};
}
if (dialog.mode === 'quick-edit' && dialog.sourceLayerId) {
const sourceLayer = layers.find(
(layer) => layer.id === dialog.sourceLayerId,
);
if (!sourceLayer) {
throw new Error('未找到要改造的图片');
}
const quickEditReferences = dialog.generationReferences ?? [];
const sourceReferenceIndex = quickEditReferences.length + 1;
const basePrompt = dialog.prompt.trim() || '快速编辑图片';
const normalizedQuickEditPrompt = normalizeQuickEditPromptSourceReference(
basePrompt,
sourceReferenceIndex,
);
const imageModel = normalizeEditorImageModel(dialog.imageModel);
const aspectRatio =
dialog.aspectRatio ??
inferEditorImageAspectRatio(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const imageSize =
dialog.imageSize ??
inferEditorImageSizeLabel(
sourceLayer.originalWidth,
sourceLayer.originalHeight,
);
const outputSize = resolveEditorImageGenerationPixelSize({
model: imageModel,
aspectRatio,
imageSize,
});
return {
kind: 'quick-edit',
normalizedPrompt: normalizedQuickEditPrompt,
sourceLayer,
input: {
prompt: normalizedQuickEditPrompt,
size: `${outputSize.width}x${outputSize.height}`,
kind: sourceLayer.assetKind === 'character' ? 'character' : 'quick-edit',
model: imageModel,
aspectRatio,
imageSize,
referenceImageSrcs: [
...quickEditReferences.map((reference) =>
resolveImageReferenceSubmissionSource(reference),
),
sourceLayer.objectKey?.trim() || sourceLayer.src,
],
},
result: {
title: `${sourceLayer.title} 快速编辑`,
assetKind: sourceLayer.assetKind,
generationInputs: buildQuickEditGenerationInputs(
'快速编辑提示词',
normalizedQuickEditPrompt,
sourceLayer,
quickEditReferences,
),
},
rememberImageModel: imageModel,
};
}
if (dialog.mode === 'spec') {
const specType = dialog.specType ?? 'custom';
const specValues = dialog.specValues ?? DEFAULT_SPEC_FORM_VALUES[specType];
@@ -1,6 +1,13 @@
/* @vitest-environment jsdom */
import { fireEvent, render, screen, waitFor, within } from '@testing-library/react';
import {
act,
fireEvent,
render,
screen,
waitFor,
within,
} from '@testing-library/react';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type {
@@ -1091,6 +1098,81 @@ describe('ImageCanvasWorldView', () => {
).toBeTruthy();
});
it('keeps the last loaded sequence frame visible while the next frame is resolving', () => {
vi.useFakeTimers();
try {
useResolvedAssetReadUrlMock.mockImplementation(
(source: string) => ({
resolvedUrl:
source === '/generated-character-drafts/editor/frame01.png'
? 'https://oss.example.com/frame01.png?signature=1'
: '',
isResolving:
source === '/generated-character-drafts/editor/frame02.png',
shouldResolve: true,
}),
);
const layer = createLayer({
title: '角色动作',
src: '/generated-character-drafts/editor/frame01.png',
mediaType: 'image-sequence',
assetKind: 'character-animation',
thumbnailSrc: '/generated-character-drafts/editor/frame01.png',
imageSequenceFrames: [
{
frameIndex: 1,
imageSrc: '/generated-character-drafts/editor/frame01.png',
objectKey: 'generated-character-drafts/editor/frame01.png',
width: 1024,
height: 1024,
},
{
frameIndex: 2,
imageSrc: '/generated-character-drafts/editor/frame02.png',
objectKey: 'generated-character-drafts/editor/frame02.png',
width: 1024,
height: 1024,
},
],
durationSeconds: 0.1,
});
renderWorldView({ layers: [layer] });
const layerButton = screen.getByRole('button', { name: '选择角色动作' });
const firstFrame = within(layerButton).getByAltText(
'画布序列帧:角色动作',
);
fireEvent.load(firstFrame);
expect(
layerButton.querySelector('.image-canvas-editor__layer-image-loading'),
).toBeNull();
act(() => {
vi.advanceTimersByTime(60);
});
expect(within(layerButton).getByText('2/2')).toBeTruthy();
expect(
within(layerButton)
.getByAltText('画布序列帧:角色动作')
.getAttribute('src'),
).toBe('https://oss.example.com/frame01.png?signature=1');
expect(
layerButton.querySelector('.image-canvas-editor__layer-image-loading'),
).toBeNull();
expect(useResolvedAssetReadUrlMock).toHaveBeenCalledWith(
'/generated-character-drafts/editor/frame02.png',
expect.objectContaining({
objectKey: 'generated-character-drafts/editor/frame02.png',
}),
);
} finally {
vi.useRealTimers();
}
});
it('keeps saved generator placeholders hidden after a result layer exists', () => {
renderWorldView({
layers: [createLayer({ id: 'layer-generated', title: '生成图片' })],
@@ -15,7 +15,7 @@ import {
Volume2,
VolumeX,
} from 'lucide-react';
import { useEffect, useMemo, useRef, useState } from 'react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import type {
CSSProperties,
KeyboardEvent as ReactKeyboardEvent,
@@ -700,6 +700,78 @@ function getLayerImageSequenceFrames(layer: CanvasLayer) {
: [];
}
function getImageSequenceFrameKey(
layer: CanvasLayer,
frame: ReturnType<typeof getLayerImageSequenceFrames>[number],
index: number,
) {
return [
layer.id,
layer.taskId ?? layer.resourceId ?? '',
frame.frameIndex,
frame.objectKey ?? '',
frame.imageSrc,
index,
].join(':');
}
function ImageCanvasImageSequenceFrame({
layer,
frame,
frameKey,
isVisible,
isLoaded,
mediaTransform,
onFrameReady,
}: {
layer: CanvasLayer;
frame: ReturnType<typeof getLayerImageSequenceFrames>[number];
frameKey: string;
isVisible: boolean;
isLoaded: boolean;
mediaTransform?: string;
onFrameReady: (frameKey: string) => void;
}) {
const { resolvedUrl } = useResolvedAssetReadUrl(frame.imageSrc, {
objectKey: frame.objectKey,
refreshKey: `${layer.taskId ?? layer.resourceId}:${frame.frameIndex}`,
});
const frameImageRef = useRef<HTMLImageElement | null>(null);
useEffect(() => {
if (resolvedUrl && isImageAlreadyDecoded(frameImageRef.current)) {
onFrameReady(frameKey);
}
}, [frameKey, onFrameReady, resolvedUrl]);
if (!resolvedUrl) {
return null;
}
return (
<img
ref={frameImageRef}
src={resolvedUrl}
alt={isVisible ? `画布序列帧:${layer.title}` : ''}
aria-hidden={isVisible ? undefined : true}
decoding="async"
className={`image-canvas-editor__layer-image-sequence-frame ${
isVisible && isLoaded
? 'image-canvas-editor__layer-image--loaded'
: 'image-canvas-editor__layer-image--loading'
}`}
style={{
inset: 0,
opacity: isVisible && isLoaded ? 1 : 0,
position: 'absolute',
transform: mediaTransform,
}}
onLoad={() => onFrameReady(frameKey)}
onError={() => onFrameReady(frameKey)}
/>
);
}
function ImageCanvasImageSequenceLayer({
layer,
mediaTransform,
@@ -708,26 +780,57 @@ function ImageCanvasImageSequenceLayer({
mediaTransform?: string;
}) {
const frames = getLayerImageSequenceFrames(layer);
const frameItems = frames.map((frame, index) => ({
frame,
index,
key: getImageSequenceFrameKey(layer, frame, index),
}));
const frameSequenceKey = frameItems.map((item) => item.key).join('|');
const firstFrameKey = frameItems[0]?.key ?? '';
const [frameIndex, setFrameIndex] = useState(0);
const [isPlaying, setIsPlaying] = useState(frames.length > 1);
const currentFrame = frames[Math.min(frameIndex, frames.length - 1)];
const hasFrameSource = Boolean(currentFrame?.imageSrc?.trim());
const { resolvedUrl } = useResolvedAssetReadUrl(currentFrame?.imageSrc, {
refreshKey: `${layer.taskId ?? layer.resourceId}:${
currentFrame?.frameIndex ?? frameIndex + 1
}`,
});
const frameImageRef = useRef<HTMLImageElement | null>(null);
const [isFrameLoaded, setIsFrameLoaded] = useState(false);
const currentFrameItem =
frameItems[Math.min(frameIndex, frameItems.length - 1)];
const currentFrameKey = currentFrameItem?.key ?? '';
const [loadedFrameKeys, setLoadedFrameKeys] = useState<ReadonlySet<string>>(
() => new Set(),
);
const [visibleFrameKey, setVisibleFrameKey] = useState(firstFrameKey);
const visibleFrameLoaded = Boolean(
visibleFrameKey && loadedFrameKeys.has(visibleFrameKey),
);
const hasFrameSource = frameItems.some((item) => item.frame.imageSrc.trim());
const handleFrameReady = useCallback((readyFrameKey: string) => {
setLoadedFrameKeys((currentKeys) => {
if (currentKeys.has(readyFrameKey)) {
return currentKeys;
}
const nextKeys = new Set(currentKeys);
nextKeys.add(readyFrameKey);
return nextKeys;
});
}, []);
useEffect(() => {
setFrameIndex(0);
setIsPlaying(frames.length > 1);
}, [layer.taskId, layer.resourceId, frames.length]);
setLoadedFrameKeys(new Set());
setVisibleFrameKey(firstFrameKey);
}, [firstFrameKey, frameSequenceKey, frames.length]);
useEffect(() => {
setIsFrameLoaded(isImageAlreadyDecoded(frameImageRef.current));
}, [resolvedUrl]);
if (!currentFrameKey) {
setVisibleFrameKey('');
return;
}
if (!loadedFrameKeys.has(currentFrameKey) && visibleFrameKey) {
return;
}
setVisibleFrameKey((currentKey) =>
currentKey === currentFrameKey ? currentKey : currentFrameKey,
);
}, [currentFrameKey, loadedFrameKeys, visibleFrameKey]);
useEffect(() => {
if (!isPlaying || frames.length <= 1) {
@@ -749,28 +852,24 @@ function ImageCanvasImageSequenceLayer({
return (
<>
{hasFrameSource && (!resolvedUrl || !isFrameLoaded) ? (
{hasFrameSource && !visibleFrameLoaded ? (
<span
className="image-canvas-editor__layer-image-loading"
aria-hidden="true"
/>
) : null}
{resolvedUrl ? (
<img
ref={frameImageRef}
src={resolvedUrl}
alt={`画布序列帧:${layer.title}`}
decoding="async"
className={`image-canvas-editor__layer-image-sequence-frame ${
isFrameLoaded
? 'image-canvas-editor__layer-image--loaded'
: 'image-canvas-editor__layer-image--loading'
}`}
style={{ transform: mediaTransform }}
onLoad={() => setIsFrameLoaded(true)}
onError={() => setIsFrameLoaded(true)}
{frameItems.map((item) => (
<ImageCanvasImageSequenceFrame
key={item.key}
layer={layer}
frame={item.frame}
frameKey={item.key}
isVisible={item.key === visibleFrameKey}
isLoaded={loadedFrameKeys.has(item.key)}
mediaTransform={mediaTransform}
onFrameReady={handleFrameReady}
/>
) : null}
))}
<div
className="image-canvas-editor__image-sequence-controls"
onPointerDown={(event) => event.stopPropagation()}
@@ -794,7 +893,9 @@ function ImageCanvasImageSequenceLayer({
)}
</button>
<span className="image-canvas-editor__image-sequence-count">
{frames.length ? frameIndex + 1 : 0}/{frames.length || 0}
{`${currentFrameItem ? currentFrameItem.index + 1 : 0}/${
frames.length || 0
}`}
</span>
</div>
</>
@@ -224,6 +224,15 @@ describe('useImageCanvasAssetExportWorkflow', () => {
expect(
zip.file('导出项目-画布素材/sequences/005-角色动作/manifest.txt'),
).toBeTruthy();
expect(
zip.file('导出项目-画布素材/sequences/005-角色动作/skeleton.json'),
).toBeTruthy();
expect(
await readZipText(
zip,
'导出项目-画布素材/sequences/005-角色动作/README.md',
),
).toContain('Spine JSON');
expect(zip.file('导出项目-画布素材/images/006-失败图.png')).toBeNull();
const metadata = JSON.parse(
@@ -277,6 +286,23 @@ describe('useImageCanvasAssetExportWorkflow', () => {
expect(sequenceMetadata.taskId).toBeUndefined();
expect(sequenceMetadata.previewVideoPath).toBeUndefined();
expect(sequenceMetadata.failedFrames).toEqual([]);
const sequenceSpineJson = JSON.parse(
await readZipText(
zip,
'导出项目-画布素材/sequences/005-角色动作/skeleton.json',
),
);
expect(sequenceSpineJson.skeleton.images).toBe('./frames/');
expect(
sequenceSpineJson.skins[0].attachments.character.animation_0001.path,
).toBe('frame-01');
expect(
sequenceSpineJson.animations.animation.slots.character.attachment,
).toEqual([
{ time: 0, name: 'animation_0001' },
{ time: 2, name: 'animation_0002' },
{ time: 4, name: 'animation_0001' },
]);
expect(metadata.layers[6].file).toBeNull();
expect(metadata.layers[6].exportError).toContain('404');
expect(metadata.failedImages).toHaveLength(1);
@@ -455,11 +481,28 @@ describe('useImageCanvasAssetExportWorkflow', () => {
await waitFor(() => {
expect(exportedBlob).toBeTruthy();
});
expect(downloadName).toBe('角色动作 跑步-序列帧.zip');
expect(downloadName).toBe('角色动作 跑步-SpineJSON.zip');
const zip = await JSZip.loadAsync(exportedBlob!);
expect(zip.file('frames/frame-01.png')).toBeTruthy();
expect(zip.file('frames/frame-02.png')).toBeTruthy();
expect(await readZipText(zip, 'manifest.txt')).toContain('类型:序列帧');
expect(await readZipText(zip, 'manifest.txt')).toContain(
'Spine JSONskeleton.json',
);
expect(await readZipText(zip, 'README.md')).toContain('Spine JSON');
const spineJson = JSON.parse(await readZipText(zip, 'skeleton.json'));
expect(spineJson.skeleton).toMatchObject({
images: './frames/',
width: 1024,
height: 1024,
});
expect(spineJson.skins[0].attachments.character.animation_0001).toEqual({
type: 'region',
path: 'frame-01',
x: 0,
y: 512,
width: 1024,
height: 1024,
});
const metadata = JSON.parse(await readZipText(zip, 'metadata.json'));
expect(metadata.title).toBe('角色动作/跑步');
expect(metadata.visible.type).toBe('角色动作序列帧');
@@ -10,6 +10,9 @@ import {
blobToUint8Array,
buildLayerExportMetadata,
buildLayerVisibleExportMetadata,
buildSpineImageSequenceJson,
buildSpineImageSequenceReadme,
type ExportedImageSequenceFrame,
formatExportDate,
getImageSequenceFrameFileName,
getLayerAssetExtensionFromTypeOrSrc,
@@ -60,15 +63,20 @@ async function buildImageSequenceZip(layer: CanvasLayer) {
frameIndex: number;
error: string;
}> = [];
const exportedFrames: ExportedImageSequenceFrame[] = [];
let successCount = 0;
for (const [index, frame] of frames.entries()) {
try {
const blob = await readLayerImageSequenceFrameBlob(layer, frame);
framesFolder.file(
getImageSequenceFrameFileName(frame, index, blob.type),
await blobToUint8Array(blob),
);
const fileName = getImageSequenceFrameFileName(frame, index, blob.type);
framesFolder.file(fileName, await blobToUint8Array(blob));
exportedFrames.push({
frameIndex: frame.frameIndex || index + 1,
fileName,
width: frame.width,
height: frame.height,
});
successCount += 1;
} catch (error) {
failedFrames.push({
@@ -96,12 +104,22 @@ async function buildImageSequenceZip(layer: CanvasLayer) {
2,
),
);
zip.file(
'skeleton.json',
JSON.stringify(
buildSpineImageSequenceJson({ layer, frames: exportedFrames }),
null,
2,
),
);
zip.file('README.md', buildSpineImageSequenceReadme(layer));
zip.file(
'manifest.txt',
[
`素材:${layer.title}`,
`类型:序列帧`,
`帧数:${successCount}/${frames.length}`,
'Spine JSONskeleton.json',
layer.durationSeconds ? `时长:${layer.durationSeconds}s` : null,
failedFrames.length ? `失败帧数:${failedFrames.length}` : null,
]
@@ -129,7 +147,7 @@ export function useImageCanvasAssetExportWorkflow({
.then((zipBlob) => {
const downloaded = triggerBrowserDownload(
zipBlob,
`${sanitizeExportFilePart(layer.title, '角色动作')}-序列帧.zip`,
`${sanitizeExportFilePart(layer.title, '角色动作')}-SpineJSON.zip`,
);
if (!downloaded) {
setAssetExportStatus({
@@ -231,6 +249,7 @@ export function useImageCanvasAssetExportWorkflow({
frameIndex: number;
error: string;
}> = [];
const exportedFrames: ExportedImageSequenceFrame[] = [];
let successCount = 0;
for (const [frameArrayIndex, frame] of frames.entries()) {
@@ -239,14 +258,18 @@ export function useImageCanvasAssetExportWorkflow({
layer,
frame,
);
framesFolder.file(
getImageSequenceFrameFileName(
frame,
frameArrayIndex,
frameBlob.type,
),
await blobToUint8Array(frameBlob),
const fileName = getImageSequenceFrameFileName(
frame,
frameArrayIndex,
frameBlob.type,
);
framesFolder.file(fileName, await blobToUint8Array(frameBlob));
exportedFrames.push({
frameIndex: frame.frameIndex || frameArrayIndex + 1,
fileName,
width: frame.width,
height: frame.height,
});
successCount += 1;
} catch (error) {
failedFrames.push({
@@ -275,12 +298,25 @@ export function useImageCanvasAssetExportWorkflow({
2,
),
);
sequenceFolder.file(
'skeleton.json',
JSON.stringify(
buildSpineImageSequenceJson({ layer, frames: exportedFrames }),
null,
2,
),
);
sequenceFolder.file(
'README.md',
buildSpineImageSequenceReadme(layer),
);
sequenceFolder.file(
'manifest.txt',
[
`素材:${layer.title}`,
`类型:序列帧`,
`帧数:${successCount}/${frames.length}`,
'Spine JSONskeleton.json',
layer.durationSeconds
? `时长:${layer.durationSeconds}s`
: null,
@@ -154,6 +154,47 @@ export function useImageCanvasAssetLibrary({
selectedAssetIds,
);
const applyAssetLibrarySnapshot = useCallback(
(
library: Awaited<ReturnType<typeof loadEditorAssetLibrary>>,
options: { preferDefaultUploadFolder?: boolean } = {},
) => {
const nextLibrary = normalizeAssetLibrary(library);
setAssetFolders(nextLibrary.folders);
setAssets(nextLibrary.assets);
const defaultFolder =
nextLibrary.folders.find((folder) => folder.systemDefault) ??
nextLibrary.folders[0];
setActiveUploadFolderId((currentFolderId) => {
if (
!options.preferDefaultUploadFolder &&
nextLibrary.folders.some((folder) => folder.id === currentFolderId)
) {
return currentFolderId;
}
return defaultFolder?.id ?? 'project';
});
},
[],
);
const refreshAssetLibrary = useCallback(async () => {
if (!canAccessProtectedData) {
return;
}
try {
applyAssetLibrarySnapshot(await loadEditorAssetLibrary());
} catch (error: unknown) {
if (isEditorAuthError(error)) {
openEditorLoginModal();
}
}
}, [
applyAssetLibrarySnapshot,
canAccessProtectedData,
openEditorLoginModal,
]);
useEffect(() => {
if (!canAccessProtectedData) {
return undefined;
@@ -164,15 +205,7 @@ export function useImageCanvasAssetLibrary({
if (cancelled) {
return;
}
const nextLibrary = normalizeAssetLibrary(library);
setAssetFolders(nextLibrary.folders);
setAssets(nextLibrary.assets);
const defaultFolder = nextLibrary.folders.find(
(folder) => folder.systemDefault,
);
setActiveUploadFolderId(
defaultFolder?.id ?? nextLibrary.folders[0]?.id ?? 'project',
);
applyAssetLibrarySnapshot(library, { preferDefaultUploadFolder: true });
})
.catch((error: unknown) => {
if (!cancelled && isEditorAuthError(error)) {
@@ -182,7 +215,7 @@ export function useImageCanvasAssetLibrary({
return () => {
cancelled = true;
};
}, [canAccessProtectedData, openEditorLoginModal]);
}, [applyAssetLibrarySnapshot, canAccessProtectedData, openEditorLoginModal]);
const resolveAssetFolderId = useCallback(
(clientX: number, clientY: number) => {
@@ -603,6 +636,7 @@ export function useImageCanvasAssetLibrary({
handleAssetMarqueePointerDown,
handleAssetMarqueePointerMove,
handleAssetMarqueePointerUp,
refreshAssetLibrary,
};
}

Some files were not shown because too many files have changed in this diff Show More