diff --git a/docs/project-memory/shared-memory/pitfalls.md b/docs/project-memory/shared-memory/pitfalls.md index cb4dbfd78..52a05a1f9 100644 --- a/docs/project-memory/shared-memory/pitfalls.md +++ b/docs/project-memory/shared-memory/pitfalls.md @@ -50,7 +50,7 @@ - 现象：工程、素材、图层和元数据都已禁止 Data URL 后，服务器仍在生成高峰出现 SpacetimeDB / api-server 内存急剧膨胀甚至 OOM；读取少量正式生成任务也会造成远大于响应体的瞬时内存增长。 - 原因：同步接口 worker 化时把原请求整体序列化到 `external_generation_job.request_payload_json`，而前端又把已有 `objectKey` 下载成 Data URL 提交。任务表也是正式持久化边界；列表 procedure 若先收集完整任务行再截断，还会把 request/result 大字段在 SpacetimeDB、SDK mapper 和 BFF 多次持有。 -- 处理：先在事故涉及的编辑器持久任务 JSON 上由 api-server 与 SpacetimeDB 两层递归拒绝 `data:` / `blob:` 并限制字节数；已有媒体传 `objectKey` / `resourceId` / `assetId`，本地派生图先用强唯一 key 上传。其它玩法若仍以 Data URL 作为正式请求契约，必须先资源化，不能直接扩大门禁造成玩法回归。列表、详情和 acknowledge 只走无 payload 的摘要投影，ack 不能为了同步旧字段重写大任务行；摘要错误文本也必须清除内联媒体并设硬上限，列表只能维护有界 top-N，不能先收集 owner 全量历史再截断。历史只通过迁移操作员的 dry-run + B-tree cursor 分批 procedure 压缩 `editor-canvas` 终态任务，cursor 选择读取量必须受 limit 约束，绝不全表扫描、绝不处理 pending / running；dry-run 后 apply 同一批时保持输入 cursor 不变，最后一批即使 `has_more=false` 只要仍有命中也必须 apply，只有 apply 成功后才推进到返回 cursor。SpacetimeDB CLI 2.5 的 `Option` 非空参数必须使用 SATS sum 编码；维护脚本要统一编码 `cursor_job_id`、`owner_user_id` 和 `completed_before_micros`，否则首批空 cursor 可运行，但第二批或带截止时间的调用会在写入前被拒绝。 +- 处理：先在事故涉及的编辑器持久任务 JSON 上由 api-server 与 SpacetimeDB 两层递归拒绝 `data:` / `blob:` 并限制字节数；已有媒体传 `objectKey` / `resourceId` / `assetId`，本地派生图先用强唯一 key 上传。其它玩法若仍以 Data URL 作为正式请求契约，必须先资源化，不能直接扩大门禁造成玩法回归。列表、详情和 acknowledge 只走无 payload 的摘要投影，ack 不能为了同步旧字段重写大任务行；摘要错误文本也必须清除内联媒体并设硬上限，列表只能维护有界 top-N，不能先收集 owner 全量历史再截断。历史只通过迁移操作员的 dry-run + B-tree cursor 分批 procedure 压缩 `editor-canvas` 终态任务，cursor 选择读取量必须受 limit 约束，绝不全表扫描、绝不处理 pending / running；dry-run 后 apply 同一批时保持输入 cursor 不变，最后一批即使 `has_more=false` 只要仍有命中也必须 apply，只有 apply 成功后才推进到返回 cursor。SpacetimeDB CLI 2.5 的 `Option` 非空参数必须使用 SATS sum 编码；维护脚本要统一编码 `cursor_job_id`、`owner_user_id` 和 `completed_before_micros`，否则首批空 cursor 可运行，但第二批或带截止时间的调用会在写入前被拒绝。运维 plan 可以保留人工契约字段，但组装 CLI procedure input 时必须按生成 schema 的 wire 名称映射；例如 Rust 字段 `expected_canvas_layout_sha256` 的 CLI wire 名为 `expected_canvas_layout_sha_256`，不能直接透传 plan key。 - 发布门禁：生产发布入口必须固定 `--delete-data=never` 与 scoped `--yes=migrate,break-clients`，普通 Jenkins 参数不得暴露清库开关；需要删数据的 schema 冲突必须直接阻断并重新检查 artifact/schema，不能靠裸 `--yes` 放行。 - 验证：构造嵌套 Data URL、Blob URL 和超限 JSON 确认入队失败；检查正式 UI procedure / client record 不含 request/result payload；用 dry-run 和 apply 测试确认活动任务不变、终态普通提示词保留且内联媒体被替换；至少带一次非空 `--cursor-job-id` 与 `--completed-before-micros` 验证 CLI Option 编码，而不是只测首批空 cursor。 - 关联：`server-rs/crates/api-server/src/editor_generation_queue.rs`、`server-rs/crates/spacetime-module/src/external_generation.rs`、`server-rs/crates/api-server/src/external_generation.rs`、`src/components/image-editor/useImageCanvasGenerationSubmissionWorkflow.ts`、`docs/【后端架构】server-rs与SpacetimeDB数据契约-2026-05-15.md`。 diff --git a/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md b/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md index 8639b6569..5b96a70a9 100644 --- a/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md +++ b/docs/【编辑器】图片画布结构化持久化与迁移回滚方案-2026-07-19.md @@ -73,7 +73,7 @@ worker 完成生成任务时，本次先用读取时 revision 调用 CAS 保存 若全量 backfill 审计发现普通图层缺少 `editor_project_resource`，先使用定向资源修复入口：`npm run spacetime:editor-canvas-resources:repair -- --database --server --plan-file `。plan 必须是仓库外、当前用户持有、权限严格为 `0600` 的普通文件；每个 canvas 绑定 owner / project、expected revision、`editor_canvas.layers_json` 与 legacy `editor_project.layers_json` 两份原始字节 SHA-256，以及精确 layer/resource/sourceResourceId 或 asset_object 修复动作。默认逐 canvas dry-run；apply 必须追加 `--apply --confirm-plan-sha256 `，成功后脚本自动以同一 plan 再 dry-run，并要求全部返回 `already_repaired`。 -`repair_editor_canvas_resources_and_return` 只允许 migration operator 调用，并且只修尚无迁移记录的 legacy canvas。图片动作仅在缺失旧 resourceId、精确旧 sourceResourceId、同工程唯一现存资源、layout 资源元数据与 private asset_object owner/objectKey/task 谱系全部一致时，替换 `resourceId` 并删除重复 `sourceResourceId`。音频动作仅在原 resourceId 全局不存在、图层与 private asset_object 的 owner/objectKey/source job/entity/content type/长度全部匹配时，恢复 `420x120` 的 `sound-effect` / `background-music` 项目资源行；资源 `asset_kind` 分别映射 asset_object 的 `editor_sound_effect` / `editor_background_music`。apply 在一个事务内插入资源、同步两份 legacy layout、递增一次 canvas revision；任一 guard 失败整画布回滚。plan、脚本输出、测试和文档均不得包含生产真实 ID。 +`repair_editor_canvas_resources_and_return` 只允许 migration operator 调用，并且只修尚无迁移记录的 legacy canvas。图片动作仅在缺失旧 resourceId、精确旧 sourceResourceId、同工程唯一现存资源、layout 资源元数据与 private asset_object owner/objectKey/task 谱系全部一致时，替换 `resourceId` 并删除重复 `sourceResourceId`；plan 中 `expected_layout_source_resource_id` 为字符串时必须与 layout 原值完全一致，显式为 `null` 时则要求 layout 中该字段明确存在且值为 JSON `null`，不得把字段缺失与 `null` 混同。音频动作仅在原 resourceId 全局不存在、图层与 private asset_object 的 owner/objectKey/source job/entity/content type/长度全部匹配时，恢复 `420x120` 的 `sound-effect` / `background-music` 项目资源行；资源 `asset_kind` 分别映射 asset_object 的 `editor_sound_effect` / `editor_background_music`。apply 在一个事务内插入资源、同步两份 legacy layout、递增一次 canvas revision；任一 guard 失败整画布回滚。plan、脚本输出、测试和文档均不得包含生产真实 ID。 ## 5. 兼容与回滚 diff --git a/scripts/spacetime-repair-editor-canvas-resources.mjs b/scripts/spacetime-repair-editor-canvas-resources.mjs index a8967d9c7..ffaf90274 100644 --- a/scripts/spacetime-repair-editor-canvas-resources.mjs +++ b/scripts/spacetime-repair-editor-canvas-resources.mjs @@ -7,6 +7,7 @@ import { fileURLToPath } from 'node:url'; import { callSpacetimeProcedureViaCli, + encodeSpacetimeCliOption, } from './spacetime-migration-common.mjs'; const PROCEDURE_NAME = 'repair_editor_canvas_resources_and_return'; @@ -304,11 +305,6 @@ function validateRepairAction(action, label) { `${label}.replacement_resource_id 为复用资源动作的必填项。`, ); } - if (!normalized.expected_layout_source_resource_id) { - throw new Error( - `${label}.expected_layout_source_resource_id 为复用资源动作的必填项。`, - ); - } if ( normalized.replacement_resource_id === normalized.expected_resource_id ) { @@ -327,7 +323,7 @@ function validateRepairAction(action, label) { `${label} 的补资源动作不能携带 replacement_resource_id。`, ); } - if (normalized.expected_layout_source_resource_id) { + if (normalized.expected_layout_source_resource_id !== null) { throw new Error( `${label} 的补资源动作不能携带 expected_layout_source_resource_id。`, ); @@ -415,16 +411,17 @@ export function buildProcedureInput(canvas, updatedAtMicros, dryRun) { project_id: canvas.project_id, owner_user_id: canvas.owner_user_id, expected_revision: canvas.expected_revision, - expected_canvas_layout_sha256: canvas.expected_canvas_layout_sha256, - expected_project_layout_sha256: canvas.expected_project_layout_sha256, + expected_canvas_layout_sha_256: canvas.expected_canvas_layout_sha256, + expected_project_layout_sha_256: canvas.expected_project_layout_sha256, remaps: canvas.actions .filter((action) => action.action_kind === 'reuse_project_resource') .map((action) => ({ layer_id: action.layer_id, expected_missing_resource_id: action.expected_resource_id, replacement_resource_id: action.replacement_resource_id, - expected_layout_source_resource_id: + expected_layout_source_resource_id: encodeSpacetimeCliOption( action.expected_layout_source_resource_id, + ), })), restores: canvas.actions .filter((action) => action.action_kind === 'restore_project_resource') diff --git a/scripts/spacetime-repair-editor-canvas-resources.test.ts b/scripts/spacetime-repair-editor-canvas-resources.test.ts index ecb13b527..7d2e221e9 100644 --- a/scripts/spacetime-repair-editor-canvas-resources.test.ts +++ b/scripts/spacetime-repair-editor-canvas-resources.test.ts @@ -166,6 +166,39 @@ describe('spacetime editor canvas resource repair plan', () => { ]); }); + it('accepts an explicit null layout source only for reuse actions', () => { + const plan = validateRepairPlan( + planValue([ + canvas({ + actions: [ + action({ expected_layout_source_resource_id: null }), + ], + }), + ]), + ); + + expect( + plan.canvases[0].actions[0].expected_layout_source_resource_id, + ).toBeNull(); + expect(() => + validateRepairPlan( + planValue([ + canvas({ + actions: [ + action({ + action_kind: 'restore_project_resource', + replacement_resource_id: null, + expected_layout_source_resource_id: 'unexpected-source', + asset_object_id: 'private-asset-object-id', + asset_kind: 'sound-effect', + }), + ], + }), + ]), + ), + ).toThrow('不能携带 expected_layout_source_resource_id'); + }); + it('rejects count mismatches, duplicate layers, and invalid kind-specific fields', () => { expect(() => validateRepairPlan({ ...planValue(), expected_action_count: 2 }), @@ -263,7 +296,10 @@ describe('spacetime editor canvas resource repair execution', () => { layer_id: 'private-layer-id', expected_missing_resource_id: 'missing-private-resource-id', replacement_resource_id: 'replacement-private-resource-id', - expected_layout_source_resource_id: 'source-private-resource-id', + expected_layout_source_resource_id: [ + 0, + 'source-private-resource-id', + ], }, ]); expect(input.restores).toEqual([ @@ -275,10 +311,37 @@ describe('spacetime editor canvas resource repair execution', () => { }, ]); expect(input.canvas_id).toBe('private-canvas-id'); + expect(input.expected_canvas_layout_sha_256).toBe(SHA_A); + expect(input.expected_project_layout_sha_256).toBe(SHA_B); + expect(input).not.toHaveProperty('expected_canvas_layout_sha256'); + expect(input).not.toHaveProperty('expected_project_layout_sha256'); expect(input.updated_at_micros).toBe(123); expect(input.dry_run).toBe(true); }); + it('encodes an explicit null expected layout source as a CLI Option null', () => { + const plan = validateRepairPlan( + planValue([ + canvas({ + actions: [ + action({ expected_layout_source_resource_id: null }), + ], + }), + ]), + ); + + const input = buildProcedureInput(plan.canvases[0], 123, true); + + expect(input.remaps).toEqual([ + { + layer_id: 'private-layer-id', + expected_missing_resource_id: 'missing-private-resource-id', + replacement_resource_id: 'replacement-private-resource-id', + expected_layout_source_resource_id: null, + }, + ]); + }); + it('calls one procedure per canvas and emits only anonymous aggregate output', async () => { const plan = validateRepairPlan( planValue([ diff --git a/server-rs/crates/spacetime-client/src/module_bindings/editor_canvas_resource_remap_input_type.rs b/server-rs/crates/spacetime-client/src/module_bindings/editor_canvas_resource_remap_input_type.rs index 4519f05c0..b4b021631 100644 --- a/server-rs/crates/spacetime-client/src/module_bindings/editor_canvas_resource_remap_input_type.rs +++ b/server-rs/crates/spacetime-client/src/module_bindings/editor_canvas_resource_remap_input_type.rs @@ -10,7 +10,7 @@ pub struct EditorCanvasResourceRemapInput { pub layer_id: String, pub expected_missing_resource_id: String, pub replacement_resource_id: String, - pub expected_layout_source_resource_id: String, + pub expected_layout_source_resource_id: Option, } impl __sdk::InModule for EditorCanvasResourceRemapInput { diff --git a/server-rs/crates/spacetime-module/src/editor_project_storage.rs b/server-rs/crates/spacetime-module/src/editor_project_storage.rs index ccd6e6f0d..15de6dca4 100644 --- a/server-rs/crates/spacetime-module/src/editor_project_storage.rs +++ b/server-rs/crates/spacetime-module/src/editor_project_storage.rs @@ -514,7 +514,7 @@ pub struct EditorCanvasResourceRemapInput { pub layer_id: String, pub expected_missing_resource_id: String, pub replacement_resource_id: String, - pub expected_layout_source_resource_id: String, + pub expected_layout_source_resource_id: Option, } #[derive(Clone, Debug, PartialEq, Eq, SpacetimeType)] @@ -6669,10 +6669,13 @@ fn validate_editor_canvas_resource_repair_actions( &action.replacement_resource_id, "remap.replacement_resource_id", )?; - normalize_required( - &action.expected_layout_source_resource_id, - "remap.expected_layout_source_resource_id", - )?; + if let Some(expected_layout_source_resource_id) = &action.expected_layout_source_resource_id + { + normalize_required( + expected_layout_source_resource_id, + "remap.expected_layout_source_resource_id", + )?; + } if missing == replacement { return Err("资源 remap 的缺失 ID 与替换 ID 不能相同".to_string()); } @@ -6761,10 +6764,21 @@ fn patch_editor_canvas_resource_layout( if current_resource_id != action.expected_missing_resource_id { return Err("图片图层的原 resourceId 与修复计划不一致".to_string()); } - let source_resource_id = - json_required_string(object.get("sourceResourceId"), "sourceResourceId")?; - if source_resource_id != action.expected_layout_source_resource_id { - return Err("图片图层的 sourceResourceId 与修复计划不一致".to_string()); + match &action.expected_layout_source_resource_id { + Some(expected_source_resource_id) => { + let source_resource_id = + json_required_string(object.get("sourceResourceId"), "sourceResourceId")?; + if source_resource_id != expected_source_resource_id.as_str() { + return Err("图片图层的 sourceResourceId 与修复计划不一致".to_string()); + } + } + None => { + if object.get("sourceResourceId") != Some(&JsonValue::Null) { + return Err( + "图片图层的 sourceResourceId 必须明确为 null 且与修复计划一致".to_string(), + ); + } + } } object.insert( "resourceId".to_string(), @@ -8040,7 +8054,7 @@ mod tests { layer_id: "layer-target".to_string(), expected_missing_resource_id: "resource-missing".to_string(), replacement_resource_id: "resource-existing".to_string(), - expected_layout_source_resource_id: "resource-source-old".to_string(), + expected_layout_source_resource_id: Some("resource-source-old".to_string()), }]; let patch = patch_editor_canvas_resource_layout( @@ -8074,7 +8088,7 @@ mod tests { layer_id: "layer-target".to_string(), expected_missing_resource_id: "resource-missing".to_string(), replacement_resource_id: "resource-existing".to_string(), - expected_layout_source_resource_id: "expected-source".to_string(), + expected_layout_source_resource_id: Some("expected-source".to_string()), }]; let error = patch_editor_canvas_resource_layout( @@ -8088,6 +8102,93 @@ mod tests { assert!(error.contains("sourceResourceId")); } + #[test] + fn editor_canvas_resource_repair_accepts_explicit_null_source_resource_id() { + let layout = json!([{ + "layerId": "layer-target", + "resourceId": "resource-missing", + "sourceResourceId": null, + "prompt": "保留提示词" + }]); + let remaps = vec![EditorCanvasResourceRemapInput { + layer_id: "layer-target".to_string(), + expected_missing_resource_id: "resource-missing".to_string(), + replacement_resource_id: "resource-existing".to_string(), + expected_layout_source_resource_id: None, + }]; + + let patch = patch_editor_canvas_resource_layout( + layout.to_string().as_str(), + remaps.as_slice(), + &[], + false, + ) + .expect("explicit null source resource should match null repair plan"); + let repaired: JsonValue = + serde_json::from_str(&patch.layers_json).expect("repaired layout json"); + + assert_eq!(repaired[0]["resourceId"], json!("resource-existing")); + assert!(repaired[0].get("sourceResourceId").is_none()); + assert_eq!(repaired[0]["prompt"], json!("保留提示词")); + assert_eq!(patch.remapped_layer_count, 1); + assert_eq!(patch.removed_source_resource_id_count, 1); + } + + #[test] + fn editor_canvas_resource_repair_null_plan_rejects_missing_or_string_source_resource_id() { + let remaps = vec![EditorCanvasResourceRemapInput { + layer_id: "layer-target".to_string(), + expected_missing_resource_id: "resource-missing".to_string(), + replacement_resource_id: "resource-existing".to_string(), + expected_layout_source_resource_id: None, + }]; + let missing_field = json!([{ + "layerId": "layer-target", + "resourceId": "resource-missing" + }]); + let string_field = json!([{ + "layerId": "layer-target", + "resourceId": "resource-missing", + "sourceResourceId": "source-old" + }]); + + for layout in [missing_field, string_field] { + let error = patch_editor_canvas_resource_layout( + layout.to_string().as_str(), + remaps.as_slice(), + &[], + false, + ) + .expect_err("null plan must only match an explicitly null sourceResourceId"); + assert!(error.contains("明确为 null")); + } + } + + #[test] + fn editor_canvas_resource_repair_string_plan_rejects_null_source_resource_id() { + let layout = json!([{ + "layerId": "layer-target", + "resourceId": "resource-missing", + "sourceResourceId": null + }]); + let remaps = vec![EditorCanvasResourceRemapInput { + layer_id: "layer-target".to_string(), + expected_missing_resource_id: "resource-missing".to_string(), + replacement_resource_id: "resource-existing".to_string(), + expected_layout_source_resource_id: Some("source-old".to_string()), + }]; + + let error = patch_editor_canvas_resource_layout( + layout.to_string().as_str(), + remaps.as_slice(), + &[], + false, + ) + .expect_err("string plan must reject a null sourceResourceId"); + + assert!(error.contains("sourceResourceId")); + } + #[test] fn editor_canvas_resource_repair_recognizes_only_exact_applied_state() { let repaired = json!([{ @@ -8099,7 +8200,7 @@ mod tests { layer_id: "layer-target".to_string(), expected_missing_resource_id: "resource-missing".to_string(), replacement_resource_id: "resource-existing".to_string(), - expected_layout_source_resource_id: "resource-source-old".to_string(), + expected_layout_source_resource_id: Some("resource-source-old".to_string()), }]; let patch = patch_editor_canvas_resource_layout( @@ -8126,6 +8227,21 @@ mod tests { ) .is_err() ); + + let unsafe_null_state = json!([{ + "layerId": "layer-target", + "resourceId": "resource-existing", + "sourceResourceId": null + }]); + assert!( + patch_editor_canvas_resource_layout( + unsafe_null_state.to_string().as_str(), + remaps.as_slice(), + &[], + true, + ) + .is_err() + ); } #[test] @@ -8169,7 +8285,7 @@ mod tests { layer_id: "layer-target".to_string(), expected_missing_resource_id: "resource-missing".to_string(), replacement_resource_id: "resource-existing".to_string(), - expected_layout_source_resource_id: "source-old".to_string(), + expected_layout_source_resource_id: Some("source-old".to_string()), }]; let error = patch_editor_canvas_resource_layout(