feat(edutainment): refresh baby object match flow
This commit is contained in:
@@ -172,7 +172,7 @@
|
||||
## 2026-05-12 宝贝识物创作同时生成玩法视觉主题包
|
||||
|
||||
- 背景：`宝贝识物` 创作原本只根据两个关键词生成物品透明图，运行态背景、UI、礼物盒和篮子仍使用固定 CSS 绘本风，无法根据“小猪佩琪 / 奥特曼”或“苹果 / 橘子”等创作者提示词做主题化包装。
|
||||
- 决策：`POST /api/creation/edutainment/baby-object-match/assets` 同一次 image-2 / VectorEngine 调用链返回两个物品图和 `visualPackage`。视觉包包含 `background`、`ui-frame`、`gift-box`、`basket`、`smoke-puff` 五类资源；总风格保持寓教于乐明亮卡通绘本插画风，主题按两个物品关键词匹配，水果偏果园自然，动漫角色 / 玩具偏动漫玩具。物品图和礼物盒 / 篮子 / UI / 烟雾特效资源走透明 PNG 后处理，背景为清爽不遮挡玩法区的环境图；运行态中礼物盒按约 2 倍视觉尺寸展示、篮子按约 1.5 倍展示，礼物盒打开时使用 `smoke-puff` 弹出中央物品并移除礼盒。前端草稿保存该包，运行态消费该包；旧草稿以 `visualPackage = null` 继续使用 CSS 兜底。
|
||||
- 决策：`POST /api/creation/edutainment/baby-object-match/assets` 同一次 image-2 / VectorEngine 调用链返回两个物品图和 `visualPackage`。为降低调用成本，新链路只生成一张 `1024x1024` 的 `2x2` 素材 sheet 和一张 `1536x1024` 场景背景图；`2x2` sheet 固定左上物品 A、右上物品 B、左下篮子、右下礼物盒，服务端按格切图并把物品、篮子和礼物盒转透明 PNG。视觉包必需资源为 `background`、`gift-box`、`basket`；总风格保持寓教于乐明亮卡通绘本插画风，主题按两个物品关键词匹配。左右手位置指示器是运行态默认静态素材，使用项目内置第一人称半抓握手，不再随每次创作生成。运行态中礼物盒按约 2 倍视觉尺寸展示、篮子按约 1.5 倍展示，中央物品 UI 与篮子物品图标使用固定正方形槽位并等比 `contain` 缩放，礼物盒打开烟雾特效由 CSS 兜底；历史草稿中的 `ui-frame` / `smoke-puff` / `left-hand` / `right-hand` 仅兼容读取或忽略。前端草稿保存该包，运行态消费该包；旧草稿以 `visualPackage = null` 继续使用 CSS 兜底。
|
||||
- 影响范围：`packages/shared/src/contracts/edutainmentBabyObject.ts`、`server-rs/crates/api-server/src/edutainment_baby_object.rs`、`src/services/edutainment-baby-object/babyObjectMatchClient.ts`、`src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.tsx`、`src/index.css`、宝贝识物 PRD 与技术方案。
|
||||
- 验证方式：执行宝贝识物 service / runtime 定向测试、`cargo test -p api-server edutainment_baby_object --manifest-path server-rs/Cargo.toml`、相关 ESLint 与编码检查；真实生图需配置 `VECTOR_ENGINE_BASE_URL` 与 `VECTOR_ENGINE_API_KEY`。
|
||||
- 关联文档：`docs/prd/BABY_OBJECT_MATCH_EDUTAINMENT_TEMPLATE_PRD_2026-05-11.md`、`docs/technical/BABY_OBJECT_MATCH_CREATION_PUBLISH_IMPLEMENTATION_2026-05-11.md`。
|
||||
@@ -197,7 +197,7 @@
|
||||
## 2026-05-10 儿童动作 Demo 视觉资产统一为绘本草地舞台
|
||||
|
||||
- 背景：儿童动作 Demo 需要从暗色科技风切换到更适合儿童互动的卡通绘本草地风格，并且要让背景、地面、UI、地面指示环和用户轮廓使用同一套 image-2 资源口径。
|
||||
- 决策：热身舞台及后续儿童动作 Demo 场景、物品、UI 资源统一采用明亮卡通绘本草地视觉语言。真实资源默认输出到 `public/child-motion-demo/`。背景沿用 `picture-book-grass-stage.png`；地面、指示环、角色轮廓和 UI 已拆分为 v2 用途专属资源：`picture-book-foreground-grass-v2.png`、`picture-book-ground-ring-v2.png`、`picture-book-character-outline-v2.png`、`picture-book-hud-strip-v2.png`、`picture-book-calibration-strip-v2.png`、`picture-book-start-panel-v2.png` 和 `picture-book-ui-button-v2.png`。生成脚本固定为 `scripts/generate-child-motion-demo-assets.mjs`，并通过 `npm run assets:child-motion-demo` 调用 VectorEngine `gpt-image-2-all`；透明资源使用品红底生成后本地去背，中间源图仅保存在 `tmp/child-motion-demo-assets/`。在缺少 `VECTOR_ENGINE_BASE_URL` 或 `VECTOR_ENGINE_API_KEY` 时，只允许 dry-run 和 CSS 兜底，不伪造 live 生图结果。
|
||||
- 决策：热身舞台及后续儿童动作 Demo 场景、物品、UI 资源统一采用明亮卡通绘本草地视觉语言。真实资源默认输出到 `public/child-motion-demo/`。背景沿用 `picture-book-grass-stage.png`；地面、指示环、角色指示器和 UI 已拆分为用途专属资源：`picture-book-foreground-grass-v2.png`、`picture-book-ground-ring-v3.png`、`picture-book-character-outline-v4.png`、`picture-book-hud-strip-v2.png`、`picture-book-calibration-strip-v2.png`、`picture-book-start-panel-v2.png` 和 `picture-book-ui-button-v2.png`。其中角色指示器 v4 基于 v2 本地后处理为更细的白色描边样式，内部透明，耳朵、手指、脚趾等细节已弱化，页面显示尺寸相对上一版放大 50%。生成脚本固定为 `scripts/generate-child-motion-demo-assets.mjs`，并通过 `npm run assets:child-motion-demo` 调用 VectorEngine `gpt-image-2-all`；透明资源使用品红底生成后本地去背，中间源图仅保存在 `tmp/child-motion-demo-assets/`。在缺少 `VECTOR_ENGINE_BASE_URL` 或 `VECTOR_ENGINE_API_KEY` 时，只允许 dry-run 和 CSS 兜底，不伪造 live 生图结果。
|
||||
- 影响范围：`src/index.css`、`src/components/child-motion-demo/ChildMotionWarmupDemo.tsx` 的舞台视觉层、儿童动作 Demo 技术文档、后续 image-2 资产生成流程。
|
||||
- 验证方式：检查 `/child-motion-demo` 舞台是否在未生成资产时仍有可用草地绘本兜底；补齐 VectorEngine 私密配置后运行 `npm run assets:child-motion-demo -- --live` 或 `--live --only <asset-id>` 应能写出对应 PNG，并确认页面静态资源返回 `image/png`。若只调整透明去背、裁切或品红边缘，可运行 `npm run assets:child-motion-demo -- --live --postprocess-only --force --only <asset-id>` 复用源图后处理。页面接入时必须按资源原始比例等比使用，不得把方形软纸面板拉伸成 HUD、状态条或底部草坪。
|
||||
- 关联文档：`docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md`、`docs/technical/VECTOR_ENGINE_GPT_IMAGE_2_GENERATION_2026-05-09.md`。
|
||||
|
||||
@@ -83,6 +83,14 @@
|
||||
- 验证：运行仓库已有编码检查；人工抽查修改文件中的中文内容。
|
||||
- 关联：`AGENTS.md`、`npm run check:encoding`。
|
||||
|
||||
## 陶泥儿 logo 生图慢请求先缩短 prompt 并单张串行
|
||||
|
||||
- 现象：使用 VectorEngine `gpt-image-2-all` 生成陶泥儿 logo 概念图时，部分 prompt 会超过 10 分钟仍无响应，或返回 `429` / `当前分组上游负载已饱和`；同一批次里后续图片会被前面的慢请求拖住。
|
||||
- 原因：复杂抽象 logo prompt 同时包含品牌解释、禁用元素、中文结构和多重隐喻时，上游排队与生成时长不稳定；并发或批量运行会放大单条慢请求的影响。
|
||||
- 处理：先 `--dry-run` 看请求体；真实生成时优先短 prompt、单一造型、单张串行或小批量。失败后不要反复重试同一长 prompt，先压缩到“一个主体 + 一个负形 + 颜色 + 禁用文字/播放键/聊天气泡”再跑。联系表中的中文标签不要通过 PowerShell 管道内联 Python 写入，容易因编码链路显示为问号，可改用英文标签或脚本文件方式。
|
||||
- 验证：生成文件落在 `public/branding/taonier-logo-*/`，用 Pillow 检查图片尺寸和非空；执行 `node --check scripts/generate-taonier-logo-concepts.mjs`、`npm run check:encoding`、`git diff --check`。
|
||||
- 关联：`scripts/generate-taonier-logo-concepts.mjs`、`docs/design/TAONIER_BRAND_LOGO_CONCEPTS_2026-05-13.md`。
|
||||
|
||||
## 忘记密码后仍提示手机号或密码错误先查认证快照同步
|
||||
|
||||
- 现象：用户通过“忘记密码”重设密码后，接口返回成功或页面进入登录态，但再次使用新密码登录仍提示“手机号或密码错误”；重启后还可能出现 `Bearer JWT 版本已失效`，日志里的 token version 与本地快照不一致。
|
||||
@@ -144,10 +152,10 @@
|
||||
## 宝贝识物选篮误触发先查多套判定和残余轨迹
|
||||
|
||||
- 现象：`宝贝识物` 运行态打开礼物盒或反馈结束后，当前物品被连续送入左侧或右侧篮子，或硬件动作名偶发命中导致未做明确横移动作也触发选篮。
|
||||
- 原因：选篮如果同时消费 `wave_left_hand` / `wave_right_hand` / `wave` 动作名和手部轨迹，或在 `correct` / `wrong` 反馈阶段继续累计手部路径，会把抓握、反馈期间残留移动或未知侧别手部误算成下一次选篮。
|
||||
- 处理：宝贝识物选篮只使用明确 `leftHand` / `rightHand` 的连续横向轨迹阈值；侧别为 `unknown` 的手部轨迹不参与选篮；反馈阶段清空轨迹，不在非 `active` 阶段累计路径。进入关卡和每次正确反馈结束后自动弹出物品，不再用 `open_palm -> grab` 抓握序列激活礼物盒。
|
||||
- 补充：当前本地 mocap 的 handedness 是摄像头视角，宝贝识物选篮前需要换算为用户身体视角；`rightHand` 轨迹代表玩家左手并进入左篮，`leftHand` 轨迹代表玩家右手并进入右篮。键鼠调试不走该换算，仍保持鼠标左键=左篮、右键=右篮。
|
||||
- 验证：运行 `npm run test -- src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.test.tsx src/services/useMocapInput.test.ts`，确认动作名负向测试、未知侧别负向测试和左右手横向轨迹测试通过。
|
||||
- 原因：选篮如果同时消费 `wave_left_hand` / `wave_right_hand` / `wave` 动作名、连续横向轨迹和左右手固定篮子规则，或在 `correct` / `wrong` 反馈阶段继续累计手部状态，会把反馈期间残留移动或未知侧别手部误算成下一次选篮。
|
||||
- 处理：宝贝识物当前选篮只允许“手先触碰中央物品 UI，物品绑定到该手，随后拖入左侧或右侧篮子区域”这一套路径；侧别为 `unknown` 的手部不参与抓取或选篮；反馈阶段清空持有状态，不在非 `active` 阶段累计输入。进入关卡和每次正确反馈结束后自动弹出物品，不再用 `open_palm -> grab` 抓握序列激活礼物盒。
|
||||
- 补充：当前本地 mocap 的 handedness 是摄像头视角，宝贝识物仍需换算为用户身体视角以展示左右手：`rightHand` 坐标代表玩家左手，`leftHand` 坐标代表玩家右手。换算不再决定只能选择哪侧篮子；任意一只手都可以拖物品到任意篮子。键鼠调试保持鼠标左键=左手位置、右键=右手位置，也必须先触碰中央物品再拖入篮子。
|
||||
- 验证：运行 `npm run test -- src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.test.tsx src/services/useMocapInput.test.ts`，确认动作名负向测试、未知侧别负向测试、触碰前不能选篮和任意手拖入任意篮子用例通过。
|
||||
- 关联：`src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.tsx`、`docs/technical/BABY_OBJECT_MATCH_CREATION_PUBLISH_IMPLEMENTATION_2026-05-11.md`。
|
||||
|
||||
## 宝贝爱画左右手反了先查 mocap 摄像头视角换算
|
||||
@@ -161,11 +169,27 @@
|
||||
## 宝贝识物创作卡在准备结果页先查长耗时 image-2 请求
|
||||
|
||||
- 现象：`/creation/baby-object-match` 创作生成停在“准备结果页”，约 3 分钟后显示“生成失败 / 请求超时”；后端日志可能出现同一路由 `status=502 latency_ms=231291`，或前端已失败但后端稍后返回 200。
|
||||
- 原因：宝贝识物一次创作会生成 2 张物品图和 `background`、`ui-frame`、`gift-box`、`basket`、`smoke-puff` 5 张视觉包装图。旧前端只等待 180 秒并对长耗时 POST 自动重试，容易在 VectorEngine 仍在生成时先 abort，再重复发起第二次生成；上游某张图超过后端 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 或返回 5xx 时会表现为 502。
|
||||
- 处理：`babyObjectMatchClient` 对 `/api/creation/edutainment/baby-object-match/assets` 使用 10 分钟超时并取消自动重试；后端并发启动物品图和视觉主题包生成，并把该路由的 VectorEngine 单图请求等待预算提升到至少 8 分钟，按资源类别输出开始、完成和耗时日志。
|
||||
- 验证：运行 `npm run test -- src/services/edutainment-baby-object/babyObjectMatchClient.test.ts src/services/miniGameDraftGenerationProgress.test.ts`、`cargo test -p api-server edutainment_baby_object --manifest-path server-rs/Cargo.toml` 和编码检查；真实联调时查看 `宝贝识物 image-2 资源生成完成` 耗时是否小于前端超时，若仍 502 再看 `VectorEngine 图片生成上游错误` 的 `upstreamStatus/raw_excerpt`。
|
||||
- 原因：宝贝识物创作属于长耗时 image-2 链路。旧前端只等待 180 秒并对长耗时 POST 自动重试，容易在 VectorEngine 仍在生成时先 abort，再重复发起第二次生成；上游某张图超过后端 `VECTOR_ENGINE_IMAGE_REQUEST_TIMEOUT_MS` 或返回 5xx 时会表现为 502。2026-05-14 后，新链路已从“2 张物品图 + 5 张视觉包装图”收敛为“1 张 `2x2` 素材 sheet + 1 张场景背景图”，左右手位置指示器改为运行态默认静态素材，不再每次创作生成，但仍需要按长耗时链路排查。
|
||||
- 处理：`babyObjectMatchClient` 对 `/api/creation/edutainment/baby-object-match/assets` 使用 10 分钟超时并取消自动重试；后端并发启动 `2x2` 素材 sheet 和场景背景生成，并把该路由的 VectorEngine 单图请求等待预算提升到至少 8 分钟，按资源类别输出开始、完成和耗时日志。`2x2` sheet 固定包含物品 A、物品 B、篮子和礼物盒，服务端按格切图并转透明 PNG；`ui-frame` / `smoke-puff` / `left-hand` / `right-hand` 不再作为新生成必需资源。
|
||||
- 验证：运行 `npm run test -- src/services/edutainment-baby-object/babyObjectMatchClient.test.ts src/services/miniGameDraftGenerationProgress.test.ts`、`cargo test -p api-server edutainment_baby_object --manifest-path server-rs/Cargo.toml` 和编码检查；真实联调时查看 `宝贝识物 image-2 2x2 素材 sheet 生成完成`、`宝贝识物 image-2 场景资源生成完成` 和整体 `宝贝识物 image-2 资源生成完成` 耗时是否小于前端超时，若仍 502 再看 `VectorEngine 图片生成上游错误` 的 `upstreamStatus/raw_excerpt`。
|
||||
- 关联：`src/services/edutainment-baby-object/babyObjectMatchClient.ts`、`src/services/miniGameDraftGenerationProgress.ts`、`server-rs/crates/api-server/src/edutainment_baby_object.rs`、`docs/technical/BABY_OBJECT_MATCH_CREATION_PUBLISH_IMPLEMENTATION_2026-05-11.md`。
|
||||
|
||||
## 宝贝识物篮子手柄白底先查 sheet 切图后处理
|
||||
|
||||
- 现象：`宝贝识物` 新生成的主题篮子在左右手柄、篮口镂空或边缘处仍出现白底块或白色毛边，尤其是 2x2 sheet 背景被抠透明后，封闭镂空区域可能没有被通用边缘连通抠图清理掉。
|
||||
- 原因：宝贝识物为了降低 image-2 成本，把物品 A、物品 B、篮子和礼物盒放在同一张 `2x2` sheet。通用背景透明处理主要从单格边缘连通背景开始，封闭在篮子手柄内部的近白区域不一定与边缘连通，因此会残留；如果把强力近白清理应用到物品格，又可能误伤白色物品主体。
|
||||
- 处理：后端 `slice_baby_object_match_sheet` 只在 `BabyObjectMatchSheetSlot::Basket` 编码前执行近白、低饱和 matte 清理；物品格和礼物盒格继续只走通用背景透明处理。sheet prompt 同步要求篮子手柄和篮口镂空处不要留下白底描边或毛边。运行态左右篮子的物品图标和名称 UI 以篮子中心线对齐，避免素材放大后看起来偏移。
|
||||
- 验证：运行 `cargo test -p api-server edutainment_baby_object --manifest-path server-rs/Cargo.toml` 与 `npm run test -- src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.test.tsx`；真实联调需要重新生成宝贝识物资源，旧草稿中已保存的 base64 篮子图不会自动被新后处理改写。
|
||||
- 关联：`server-rs/crates/api-server/src/edutainment_baby_object.rs`、`src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.tsx`、`src/index.css`、`docs/technical/BABY_OBJECT_MATCH_CREATION_PUBLISH_IMPLEMENTATION_2026-05-11.md`。
|
||||
|
||||
## 宝贝识物物品框被长条素材拉伸先查固定槽位
|
||||
|
||||
- 现象：用户用手机、筷子等长条关键词生成素材后，中央物品 UI 或篮子上方物品图标看起来被拉成长框，圆形 UI 失去固定比例。
|
||||
- 原因：运行态如果让图片固有宽高或外层自适应内容，就会把长条透明 PNG 的主体比例传导到 UI 容器。
|
||||
- 处理：中央物品 UI 和篮子物品图标都必须使用固定正方形槽位，外层尺寸由 CSS 变量控制；生成素材图片只在槽位内 `object-fit: contain` 等比缩放，不改变外层圆形 UI 框尺寸。
|
||||
- 验证：用长条物品草稿进入宝贝识物运行态，中央物品框和篮子图标框仍为正圆，长条主体在框内缩小显示。
|
||||
- 关联：`src/index.css`、`src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.tsx`、`docs/technical/BABY_OBJECT_MATCH_CREATION_PUBLISH_IMPLEMENTATION_2026-05-11.md`。
|
||||
|
||||
## 寓教于乐作品和宝贝识物模板同时消失先查入口种子
|
||||
|
||||
- 现象：发现页“寓教于乐”分类下已发布的宝贝识物作品突然消失，同时创作界面模板选项中也看不到或无法正常展示 `宝贝识物`。
|
||||
@@ -186,10 +210,18 @@
|
||||
|
||||
- 现象：`/child-motion-demo` 背景风格正确，但底部草坪被拉成厚色块、顶部 HUD 或右下状态条像方形面板被横向拉伸，或旧 `picture-book-ui-panel.png` 与新资源叠在一起。
|
||||
- 原因：早期资源中 `picture-book-ui-panel.png` 是接近方形画布，`picture-book-grass-floor.png` 也含大量透明边界；若 CSS 用 `background-size: 100% 100%` 把同一资源强行铺成 HUD、状态条、开始面板或底部地板，就会出现变形和层叠观感。
|
||||
- 处理：使用 v2 用途专属资源：`picture-book-foreground-grass-v2.png`、`picture-book-ground-ring-v2.png`、`picture-book-character-outline-v2.png`、`picture-book-hud-strip-v2.png`、`picture-book-calibration-strip-v2.png`、`picture-book-start-panel-v2.png`、`picture-book-ui-button-v2.png`；CSS 按资源比例等比缩放，底部草坪只覆盖下沿，HUD / 状态条 / 开始托盘分别引用各自资源。若只需修透明裁切或品红边，运行 `npm run assets:child-motion-demo -- --live --postprocess-only --force --only <asset-id>`，不重新请求 image-2。
|
||||
- 处理：使用用途专属资源：`picture-book-foreground-grass-v2.png`、`picture-book-ground-ring-v3.png`、`picture-book-character-outline-v4.png`、`picture-book-hud-strip-v2.png`、`picture-book-calibration-strip-v2.png`、`picture-book-start-panel-v2.png`、`picture-book-ui-button-v2.png`；CSS 按资源比例等比缩放，底部草坪只覆盖下沿，HUD / 状态条 / 开始托盘分别引用各自资源。角色指示器使用 v4 更细白色描边资源，内部透明且显示尺寸相对上一版放大 50%；若只需修透明裁切、品红边或纯描边后处理，运行 `npm run assets:child-motion-demo -- --live --postprocess-only --force --only <asset-id>`，不重新请求 image-2。
|
||||
- 验证：用横屏截图检查没有新旧资源叠加、没有方形面板拉成长条、角色和地面指示环不被前景草坪埋住；同时运行 `npm run check:encoding`。
|
||||
- 关联：`scripts/generate-child-motion-demo-assets.mjs`、`src/index.css`、`public/child-motion-demo/`、`docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md`。
|
||||
|
||||
## 儿童动作 Demo 猫咪挥手拆件错位先查动画父级和肩部挂点
|
||||
|
||||
- 现象：`/child-motion-demo` 打个招呼阶段的猫咪图和风格正确，但挥手时左右手臂像漂浮在身体旁边，视频里能看到肢体没有稳定接在肩膀上。
|
||||
- 原因：猫咪身体和手臂如果分别做上下浮动，或手臂使用透明方形画布的默认中心/底部旋转轴，就会在摆动极值时放大肩点偏差；镜像左臂还需要把资源内部连接点换算到镜像后的坐标。
|
||||
- 处理：`.child-motion-gesture-guide__wave-cat` 父级统一承接 bob 动画，身体层保持静态贴底且层级低于手臂；左右手臂作为同一父级下的兄弟层，只做旋转动画并显示在身体前方。身体使用去掉左右小圆点的 `picture-book-wave-cat-body-guide-v7.png`；手臂 v7 资源当前按身体外缘摆放，圆猫爪掌面朝向玩家；左右侧距为 `12%`，左臂使用原图层与 `60% 78%` 旋转轴，右臂使用镜像图层与 `40% 78%` 旋转轴，动画周期为 `0.47s`，左右手臂不设置错峰延迟；不要把 `scaleX(...)` 和 rotate 放在同一个手臂 wrapper 上。
|
||||
- 验证：用用户录屏关键帧或离线合成预览检查摆动两端的手臂根部仍贴住肩点；再运行儿童动作 Demo 定向组件测试、ESLint 和 `npm run check:encoding`。
|
||||
- 关联：`src/index.css`、`public/child-motion-demo/picture-book-wave-cat-body-guide-v7.png`、`public/child-motion-demo/picture-book-wave-cat-arm-guide-v7.png`、`docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md`。
|
||||
|
||||
## GPT-image-2 不再读 APIMart 图片配置
|
||||
|
||||
- 现象：配置了 `APIMART_BASE_URL` / `APIMART_API_KEY` 后，RPG、拼图或方洞的 GPT-image-2 生图仍返回缺配置，或请求体里还出现 `official_fallback` / `image_urls`。
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,121 @@
|
||||
# 宝贝识物寓教于乐模板 PRD 2026-05-11
|
||||
|
||||
## 1. 目标
|
||||
|
||||
新增寓教于乐内容线的创作模板：
|
||||
|
||||
```text
|
||||
宝贝识物
|
||||
```
|
||||
|
||||
创作者必须通过该模板创作并发布作品后，用户才能在寓教于乐板块体验对应关卡。
|
||||
|
||||
本模板只服务儿童动作 Demo 内容线，不把普通教育题材作品自动归入寓教于乐。
|
||||
|
||||
## 2. 创作输入
|
||||
|
||||
创作者必须填写两个物品名称：
|
||||
|
||||
1. 物品 A 名称；
|
||||
2. 物品 B 名称。
|
||||
|
||||
两个名称都必须去除首尾空白后非空。当前阶段不新增题材、难度、计时、失败次数、分数、体力或递增规则。
|
||||
|
||||
## 3. 生成规则
|
||||
|
||||
提交后生成一份宝贝识物草稿，草稿包含：
|
||||
|
||||
1. 模板 ID：`baby-object-match`；
|
||||
2. 模板名称：`宝贝识物`；
|
||||
3. 两个物品；
|
||||
4. 两个物品图；
|
||||
5. 游戏视觉主题包；
|
||||
6. 作品标签。
|
||||
|
||||
素材使用 VectorEngine `gpt-image-2-all` / image-2 生成。图片生成只能走后端接口，前端不得读取、拼接或暴露 `VECTOR_ENGINE_API_KEY`。
|
||||
|
||||
为降低生成成本，创作提交后只生成两张原始图片：一张 `2x2` 素材 sheet 和一张单独场景背景图。`2x2` 素材 sheet 固定包含左上物品 A、右上物品 B、左下篮子、右下礼物盒。服务端必须按固定格切图，并把物品、篮子和礼物盒转成透明 PNG。只有透明抠图后的两个物品素材才允许写入草稿 `itemAssets` 并进入游戏运行态。左右手位置指示器属于运行态默认规则，使用项目内置静态素材，不在每次创作时生成。
|
||||
|
||||
同一次创作还必须生成游戏视觉主题包，必需资源为背景环境、礼物盒、篮子。主题包必须继续保持寓教于乐插画风，并根据用户填写的两个物品关键词匹配主题：例如关键词偏动漫角色或玩具时，背景环境和元素可使用动漫、玩具主题；关键词偏水果时，背景环境和元素可匹配果园、自然主题；其它关键词按其语义匹配合适主题。主题包不得改变关卡玩法规则，不新增文字说明、额外按钮或额外判定规则。
|
||||
|
||||
视觉主题包的资源边界：
|
||||
|
||||
1. 背景环境图不做透明抠图，但必须保证屏幕中间、中下方和底部左右篮子区域清爽，不遮挡放大后的物品、礼物盒和篮子；
|
||||
2. 礼物盒资源从 `2x2` 素材 sheet 右下格切出，输出为透明 PNG，运行态按当前礼盒视觉的 2 倍尺寸展示，素材主体必须饱满清晰；
|
||||
3. 篮子资源从 `2x2` 素材 sheet 左下格切出，输出为透明 PNG，运行态按当前篮子视觉的 1.5 倍尺寸展示，左右篮子仍固定为两个物品对应选项，篮子造型资源可以复用同一张主题篮子图；篮子切图不得保留手柄、篮口或边缘处的白底描边和抠图毛边；
|
||||
4. 运行态左右手位置指示器使用内置默认静态素材，姿势为用户第一人称看到的半抓握手，不随创作关键词重新生成；
|
||||
5. 礼物盒打开时的烟雾弹出特效由运行态 CSS 动效兜底；历史草稿如果已有 `smoke-puff` 资源可继续兼容读取，但新生成链路不再单独生成该资源。
|
||||
|
||||
当前本地 Demo 阶段已接入真实 image-2 资源链路。创作提交必须成功获得 `generationProvider = "vector-engine-gpt-image-2"` 的两个物品透明 PNG、背景环境图、礼物盒和篮子后，才能进入结果页、试玩或发布；若后端接口、登录态、VectorEngine 配置或上游生成失败，前端必须停留在生成失败状态并展示错误，不得静默回退为占位图。历史草稿中若仍存在 `generationProvider = "placeholder"` 的占位资源，结果页必须提示重新生成，试玩和发布前必须先补齐 image-2 资源。
|
||||
|
||||
## 4. 标签规则
|
||||
|
||||
发布作品必须携带精确标签：
|
||||
|
||||
```text
|
||||
寓教于乐
|
||||
```
|
||||
|
||||
标签识别只接受精确等于 `寓教于乐`。不接受 `儿童教育`、`动作教育`、`寓教于乐 ` 等近似标签。
|
||||
|
||||
宝贝识物草稿与发布 payload 中都必须保留该标签。发布后的公开展示、搜索、深链和入口开关继续遵循 `CHILD_MOTION_EDUTAINMENT_DISCOVER_ENTRY_2026-05-09.md`。
|
||||
|
||||
## 5. 结果页能力
|
||||
|
||||
结果页展示：
|
||||
|
||||
1. 作品名称；
|
||||
2. 两个物品名称；
|
||||
3. 两个物品图；
|
||||
4. 标签；
|
||||
5. 保存草稿；
|
||||
6. 发布；
|
||||
7. 试玩。
|
||||
|
||||
结果页不展示长规则说明文案。试玩按钮直接进入宝贝识物首关本地运行态。
|
||||
|
||||
试玩按钮进入宝贝识物首关运行态，运行态消费当前草稿中的两个物品名称和两张物品图，不重新生成或改写物品内容。
|
||||
|
||||
若草稿包含视觉主题包，运行态还必须消费该主题包中的背景环境、礼物盒和篮子资源；左右手位置指示器始终使用内置默认静态素材。旧草稿或接口失败时允许回退到当前 CSS 绘本风兜底。历史草稿中若已有 UI 装饰、左右手或烟雾弹出特效资源，运行态仅做兼容读取或忽略，不作为新链路必需资源。
|
||||
|
||||
## 6. 发布后体验
|
||||
|
||||
发布完成后作品应进入寓教于乐内容线，并在寓教于乐入口开启时可被板块消费。
|
||||
|
||||
入口关闭时，发布作品完全不可见，不能通过推荐、发现普通频道、搜索、作品号、公开详情深链或浏览历史访问。
|
||||
|
||||
## 7. 与运行时线程的边界
|
||||
|
||||
本 PRD 同步约束首关运行态，已确认规则包括：
|
||||
|
||||
1. 进入关卡后先展示两个目标物品：物品 A 居中展示 2 秒，名称 UI 与字体约为默认大小的 2 倍，随后物品和名称飞入左侧篮子预设位置，并在飞行过程中恢复为默认大小；左侧就绪后等待 1 秒，再展示物品 B 并飞入右侧篮子预设位置；全部就绪后等待 1 秒再进入礼物盒入场。
|
||||
2. 目标展示完成后，首次礼物盒自动打开并弹出首个随机物品；后续每次正确反馈完全结束后重新进入礼物盒入场。
|
||||
3. 每轮仅中间礼物盒跳出的物品随机；左右两侧篮子固定为当前草稿两个物品的顺序；
|
||||
4. 下一关按钮当前占位；
|
||||
5. 不新增用户未确认的计时、失败次数、分数、体力或难度递增。
|
||||
6. 屏幕中上方字幕固定为“将物品放入对应的篮子里”。
|
||||
7. 礼物盒位于屏幕中下方并按当前视觉放大一倍，首次进入关卡和每次正确反馈结束后的新轮次都从上方落下后自动打开。
|
||||
8. 屏幕下方左侧和右侧分别展示两个固定篮子，左侧固定使用草稿第一个物品图，右侧固定使用草稿第二个物品图。
|
||||
9. 左右篮子按当前视觉放大 50%，物品图标与篮子中心尽量对齐，物品图标下方展示对应物品名称 UI。
|
||||
10. 礼物盒打开时播放烟雾特效，中央物品从烟雾特效中弹出；物品弹出后礼物盒从舞台移除。
|
||||
11. 中央物品 UI 和左右篮子上方物品图标都使用固定正方形槽位，生成素材只在槽位内等比缩放；长条形物品不得拉伸外层 UI 框。
|
||||
12. 运行态实时展示用户左右手位置；任意一只手先接触中央物品 UI 后，中央物品绑定并跟随该手移动，手带物品进入左侧或右侧篮子区域时代表选择对应篮子；选篮不使用动作名判定，也不再使用左手固定选左篮、右手固定选右篮的规则。
|
||||
13. 正确时展示“真棒”字幕和正确特效；错误时展示“再想一想吧”字幕和错误特效，物品回到中央。
|
||||
14. 成功 20 次后展示“恭喜你！小朋友！”字幕和特效，并展示“再来一次”和“下一关”按钮。
|
||||
15. 当前本地 Demo 阶段音效与语音播报接口只预留调用点，不在前端写死外部硬件或服务接口。
|
||||
|
||||
## 8. 验收
|
||||
|
||||
1. 创作入口显示 `宝贝识物` 并可进入模板表单。
|
||||
2. 未填写任一物品名称时不能生成草稿。
|
||||
3. 生成草稿后进入结果页，展示两个物品名称和物品图。
|
||||
4. 生成草稿后包含视觉主题包，主题包含背景环境、礼物盒、篮子三类必需资源。
|
||||
5. 草稿标签中始终包含精确 `寓教于乐`。
|
||||
6. 发布 payload 始终包含精确 `寓教于乐`。
|
||||
7. 发布完成后出现分享弹窗或发布完成状态。
|
||||
8. 前端不读取或暴露 VectorEngine 密钥。
|
||||
9. 结果页试玩进入宝贝识物运行态，不再显示“试玩关卡正在接入中”。
|
||||
10. 运行态通过鼠标左键映射左手位置、鼠标右键映射右手位置；调试输入也必须先触碰中央物品，再拖入任一篮子完成选择。
|
||||
11. 成功 20 次后出现“再来一次”和“下一关”按钮。
|
||||
12. 使用长条形物品素材时，中央物品 UI 和篮子物品图标仍保持固定正方形槽位，只缩放物品本体。
|
||||
13. 运行态开局先完成两个目标物品的居中展示和飞入篮子动画，之后才出现礼物盒并进入首轮随机物品。
|
||||
@@ -0,0 +1,274 @@
|
||||
# 宝贝识物创作发布实现方案 2026-05-11
|
||||
|
||||
## 1. 范围
|
||||
|
||||
本方案对应第 2 线程：创作发布线程。
|
||||
|
||||
本线程落地：
|
||||
|
||||
1. 创作入口配置；
|
||||
2. 模板表单；
|
||||
3. 本地草稿生成 service；
|
||||
4. 结果页；
|
||||
5. 发布 payload 约束；
|
||||
6. 本地 Demo 运行态；
|
||||
7. 后端 image-2 / 作品持久化 / 运行态接口预留形状。
|
||||
|
||||
本阶段运行态先做浏览器本地 Demo，并消费现有本地 mocap 动作数据源；正式硬件接口和摄像头调教在后续接口稳定后继续接入。
|
||||
|
||||
## 2. 前端接入点
|
||||
|
||||
新增玩法 ID：
|
||||
|
||||
```text
|
||||
baby-object-match
|
||||
```
|
||||
|
||||
用户展示名：
|
||||
|
||||
```text
|
||||
宝贝识物
|
||||
```
|
||||
|
||||
工程接入文件：
|
||||
|
||||
1. `server-rs/crates/spacetime-module/src/runtime/creation_entry_config.rs`
|
||||
2. `src/components/platform-entry/platformEntryCreationTypes.ts`
|
||||
3. `src/components/platform-entry/PlatformEntryCreationTypeModal.tsx`
|
||||
4. `src/components/platform-entry/PlatformEntryFlowShellImpl.tsx`
|
||||
|
||||
`src/config/newWorkEntryConfig.ts` 已迁移删除，不再作为入口事实源。`baby-object-match` 必须存在于 SpacetimeDB `creation_entry_type_config` 默认种子中，默认展示名为 `宝贝识物`、`visible=true`、`open=true`、`sortOrder=90`；前端只通过 `GET /api/creation-entry/config` 读取后端配置并在 `platformEntryCreationTypes.ts` 做展示派生。
|
||||
|
||||
`baby-object-match` 必须复用 `VITE_ENABLE_EDUTAINMENT_ENTRY` 开关；开关关闭时，创作类型弹层不展示 `宝贝识物`，创作页作品架不展示本地宝贝识物草稿或已发布作品卡，公开发现、搜索、详情、作品号和浏览历史也继续完全不可见。
|
||||
|
||||
新增阶段：
|
||||
|
||||
```text
|
||||
baby-object-match-workspace
|
||||
baby-object-match-generating
|
||||
baby-object-match-result
|
||||
baby-object-match-runtime
|
||||
```
|
||||
|
||||
## 3. 契约
|
||||
|
||||
前端共享契约放在：
|
||||
|
||||
```text
|
||||
packages/shared/src/contracts/edutainmentBabyObject.ts
|
||||
```
|
||||
|
||||
核心字段：
|
||||
|
||||
1. `BabyObjectMatchDraft.templateId = "baby-object-match"`；
|
||||
2. `BabyObjectMatchDraft.templateName = "宝贝识物"`；
|
||||
3. `BabyObjectMatchDraft.themeTags` 必须包含精确 `寓教于乐`；
|
||||
4. `BabyObjectMatchItemAsset.generationProvider` 首版允许为 `vector-engine-gpt-image-2` 或 `placeholder`；
|
||||
5. `BabyObjectMatchDraft.visualPackage` 可选承载背景环境、礼物盒和篮子三类必需视觉资源；历史草稿中的 `ui-frame`、`smoke-puff`、`left-hand` 与 `right-hand` 仅保留运行态兼容读取或忽略；
|
||||
6. `BabyObjectMatchPublishRequest.draft.themeTags` 发布前必须归一化补齐 `寓教于乐`。
|
||||
|
||||
## 4. Service 边界
|
||||
|
||||
前端 service 放在：
|
||||
|
||||
```text
|
||||
src/services/edutainment-baby-object/babyObjectMatchClient.ts
|
||||
```
|
||||
|
||||
首版提供：
|
||||
|
||||
1. `createBabyObjectMatchDraft(payload)`；
|
||||
2. `saveBabyObjectMatchDraft(draft)`；
|
||||
3. `publishBabyObjectMatchWork(payload)`；
|
||||
4. `deleteLocalBabyObjectMatchDraft(profileId)`；
|
||||
5. `regenerateBabyObjectMatchDraftAssets(draft)`；
|
||||
6. `hasBabyObjectMatchPlaceholderAssets(draft)`。
|
||||
|
||||
当前后端正式作品持久化接口未在本线程扩表落地，因此 service 仍使用本地 Demo 存储草稿和发布状态。由于 image-2 会返回多张 base64 PNG 大图，本地 Demo 草稿必须优先写入 IndexedDB `genarrative-edutainment-baby-object-drafts/drafts`，不得把完整草稿 JSON 写入 `localStorage`；`localStorage` 仅作为旧版小草稿迁移读取来源，读取后迁移到 IndexedDB 并清理旧 key，避免触发浏览器 `Storage` 配额错误。
|
||||
|
||||
物品图片生成已接入后端 image-2 接口：
|
||||
|
||||
```text
|
||||
POST /api/creation/edutainment/baby-object-match/assets
|
||||
```
|
||||
|
||||
请求体：
|
||||
|
||||
```json
|
||||
{
|
||||
"itemNames": ["苹果", "香蕉"]
|
||||
}
|
||||
```
|
||||
|
||||
响应体：
|
||||
|
||||
```json
|
||||
{
|
||||
"assets": [
|
||||
{
|
||||
"itemId": "baby-object-item-1",
|
||||
"itemName": "苹果",
|
||||
"imageSrc": "data:image/png;base64,...",
|
||||
"assetObjectId": null,
|
||||
"generationProvider": "vector-engine-gpt-image-2",
|
||||
"prompt": "..."
|
||||
}
|
||||
],
|
||||
"visualPackage": {
|
||||
"themePrompt": "...",
|
||||
"assets": [
|
||||
{
|
||||
"assetId": "baby-object-visual-background",
|
||||
"assetKind": "background",
|
||||
"imageSrc": "data:image/png;base64,...",
|
||||
"assetObjectId": null,
|
||||
"generationProvider": "vector-engine-gpt-image-2",
|
||||
"prompt": "..."
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
该接口返回从同一张 `2x2` 素材 sheet 切出的两个物品透明 PNG、礼物盒透明 PNG、篮子透明 PNG，以及单独生成的一张背景环境图。本地 Demo 阶段暂不写入 OSS 或 SpacetimeDB `asset_object`。当前创作链路必须真实拿到 `generationProvider = "vector-engine-gpt-image-2"` 的两个物品图和必需视觉主题包后才允许进入结果页；若本地未配置 VectorEngine、登录态失效、接口返回 401/5xx、上游生成失败或响应缺少任一必需资源，前端 service 必须抛出错误并停留在生成失败状态，不得静默回退到占位图。左右手位置指示器是运行态默认静态素材，不在该接口中生成。
|
||||
|
||||
为了降低 image-2 调用成本，一次创作只发起两次图片生成：一次 `1024x1024` 的 `2x2` 素材 sheet，固定包含左上物品 A、右上物品 B、左下篮子、右下礼物盒；一次 `1536x1024` 的场景背景图。前端 `babyObjectMatchClient` 对该 POST 使用 10 分钟请求超时，且不做自动重试，避免第一次生成仍在后端执行时又发起第二次重复生成。后端并发启动两张图生成，并把该路由的 VectorEngine 单图请求等待预算提升到至少 8 分钟，避免某张图 3 分钟附近仍在生成时被后端提前断开。后端日志记录资源开始、完成和耗时，排查时优先按同一次 HTTP 请求查看 `宝贝识物 image-2 2x2 素材 sheet 生成完成`、`宝贝识物 image-2 场景资源生成完成` 与 `VectorEngine 图片生成上游错误`。
|
||||
|
||||
历史本地草稿中若已保存 `generationProvider = "placeholder"` 的旧占位资源，结果页必须提示“重新生成 image-2 资源”，并禁用试玩和发布。用户点击重新生成、发布或试玩前，前端统一调用 `regenerateBabyObjectMatchDraftAssets(draft)` 补齐资源；补齐失败时保留在结果页并展示错误。
|
||||
|
||||
后续正式作品持久化接入时，应补齐：
|
||||
|
||||
```text
|
||||
POST /api/creation/edutainment/baby-object-match/drafts
|
||||
PUT /api/creation/edutainment/baby-object-match/drafts/{draftId}
|
||||
POST /api/creation/edutainment/baby-object-match/drafts/{draftId}/publish
|
||||
```
|
||||
|
||||
图片生成必须在后端调用 VectorEngine `gpt-image-2-all`，不得从前端直接调用外部图片接口。
|
||||
|
||||
后端 `2x2` 素材 sheet prompt 约束：
|
||||
|
||||
1. 锁定寓教于乐板块统一的明亮卡通绘本插画风；
|
||||
2. 固定四格布局：左上格物品 A，右上格物品 B，左下格篮子，右下格礼物盒；
|
||||
3. 两个物品格只能围绕对应关键词生成单一主体，不生成背景、场景、人物、手、篮子、礼物盒、文字、水印或 UI；
|
||||
4. 篮子格只生成一个主体饱满、开口清晰的大号篮子，不放入待分类物品，手柄和篮口镂空处不得留下白底描边或毛边；
|
||||
5. 礼物盒格只生成一个主体饱满、中心构图的大号礼物盒；
|
||||
6. 每格使用纯白或接近纯白背景，不绘制网格线、标签、按钮或边框；
|
||||
7. 服务端按 `2x2` 固定格切图，并按单格边缘采样背景色转透明 PNG，返回的物品、篮子和礼物盒素材必须已完成透明背景后处理；
|
||||
8. 篮子切图在通用透明背景处理后，还必须额外清理近白、低饱和的白底毛边，优先覆盖手柄镂空、篮口镂空和边缘残留白底；该处理仅应用于篮子格，不应用于两个物品格，避免误伤白色物品主体。
|
||||
|
||||
后端场景背景 prompt 约束：
|
||||
|
||||
1. 背景图单独生成，总风格继续锁定寓教于乐明亮卡通绘本插画风；
|
||||
2. 若关键词偏动漫角色、玩具或公仔，背景环境匹配动漫、玩具主题；若关键词偏水果，匹配果园、自然主题；其它关键词按语义匹配合适主题；
|
||||
3. 背景环境图使用非透明横向图，但必须保证中间、中下方和底部左右篮子区域清爽，给放大后的礼物盒、中央物品和左右篮子预留空间；
|
||||
4. 背景图不画入礼物盒、篮子、物品、人物、文字或操作 UI；
|
||||
5. 左右篮子的固定选项规则不受主题包影响，运行态只把 `basket` 作为篮子造型包装复用。
|
||||
|
||||
运行态左右手位置指示器不随创作生成。默认素材保存在 `public/edutainment-baby-object/image2-picture-book-hands/baby-object-left-hand-v8-transparent.png` 与 `public/edutainment-baby-object/image2-picture-book-hands/baby-object-right-hand-v8-transparent.png`，姿势沿用图1的圆形手与斜向手臂结构，并按寓教于乐明亮绘本插画风完成 image2 填色和风格化处理。后续若要替换默认手型，应更新这两个静态资源和运行态 CSS 默认变量，而不是恢复每次创作的左右手 image-2 生成。
|
||||
|
||||
## 5. UI 边界
|
||||
|
||||
工作台只展示两个必填输入和生成按钮。
|
||||
|
||||
结果页只展示草稿核心信息、两个物品、保存草稿、发布、试玩。不在 UI 内写玩法说明长文案。
|
||||
|
||||
移动端优先：表单和结果页使用单列布局，桌面端自然扩展为双列。
|
||||
|
||||
## 6. 运行态边界
|
||||
|
||||
前端运行态放在：
|
||||
|
||||
```text
|
||||
src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.tsx
|
||||
```
|
||||
|
||||
运行态直接消费 `BabyObjectMatchDraft`，必须使用草稿中的两个物品名称和物品图。
|
||||
每轮只随机当前从礼物盒跳出的物品；左右篮子不随机交换，左侧固定为草稿 `itemAssets[0]`，右侧固定为草稿 `itemAssets[1]`。
|
||||
|
||||
若草稿包含 `visualPackage`，运行态通过背景图片层、CSS 变量和图片节点消费：
|
||||
|
||||
1. `background`：作为舞台最底层 `ResolvedAssetImage` 背景图；存在该资源时必须关闭默认草地兜底层，避免生成场景被 CSS 草地遮住或弱化；
|
||||
2. `gift-box`：替换 CSS 礼物盒主体，按旧视觉约 2 倍尺寸展示，只在礼盒入场和打开阶段存在；
|
||||
3. `basket`：替换篮子主体造型，按旧视觉约 1.5 倍尺寸展示，左右两侧复用同一张主题篮子图；
|
||||
4. 左右手位置指示器：始终使用运行态默认静态素材；历史草稿中若带有 `left-hand` / `right-hand` 资源，不再作为视觉包完整性或运行时皮肤来源。
|
||||
|
||||
左右篮子的选项 UI 必须以篮子中心线为基准居中展示：物品图标位于篮子上方，图标下方展示对应物品名称短标签，左侧固定展示草稿第一个物品，右侧固定展示草稿第二个物品。该名称标签是运行态 UI 的一部分，用于后续只看图案或只看名称的玩法变体预留，但当前不新增额外规则。
|
||||
|
||||
历史草稿若包含 `ui-frame` 或 `smoke-puff`，运行态继续兼容读取；新生成链路不再把这两类资源作为必需 image-2 产物。礼物盒打开烟雾特效优先使用 CSS 动效兜底，避免为了单个特效额外增加生图调用。
|
||||
|
||||
旧草稿或接口失败时 `visualPackage = null`，运行态继续使用现有 CSS 绘本风兜底。
|
||||
|
||||
中央物品 UI 与左右篮子上方物品图标必须使用固定正方形槽位，不允许因为生成物品是手机、长条玩具等窄长形状而拉伸外层 UI 框。素材图片在槽位内使用等比 `contain` 缩放，长条形状只缩小主体，不改变圆形 UI 框尺寸。
|
||||
|
||||
首关状态机：
|
||||
|
||||
1. `intro-left-showing`：物品 A 居中展示 2 秒，名称 UI 和字体约为默认大小的 2 倍，不接受动作判定；
|
||||
2. `intro-left-flying`：物品 A 和名称飞入左侧篮子预设位置，飞行过程中名称 UI 和字体恢复为默认大小，不接受动作判定；
|
||||
3. `intro-left-ready`：左侧目标就绪后等待 1 秒，不接受动作判定；
|
||||
4. `intro-right-showing`：物品 B 居中展示 2 秒，名称 UI 和字体约为默认大小的 2 倍，不接受动作判定；
|
||||
5. `intro-right-flying`：物品 B 和名称飞入右侧篮子预设位置，飞行过程中名称 UI 和字体恢复为默认大小，不接受动作判定；
|
||||
6. `intro-right-ready`：右侧目标就绪后等待 1 秒，不接受动作判定；
|
||||
7. `gift-entering`：礼物盒从上方落下入场动画阶段，不接受动作判定；首次进入该状态必须发生在两个目标展示完成后，后续正确反馈结束后直接进入该状态；
|
||||
8. `gift-opening`：礼物盒打开并播放烟雾特效阶段，不接受动作判定；
|
||||
9. `item-appearing`：礼物盒从舞台移除，当前物品从烟雾中出现并停稳，不接受动作判定；
|
||||
10. `active`：物品彻底出现后才开放选篮判定；
|
||||
11. `correct`：展示“真棒”反馈，对应篮筐播放正确特效并停顿，成功次数加 1；特效完全结束后重新进入 `gift-entering`，下一轮礼物盒从上方落下，不重复目标展示；
|
||||
12. `wrong`：展示“再想一想吧”反馈，物品弹回中央；反馈结束后回到 `active`，不重新随机物品；
|
||||
13. `complete`：成功次数达到 20，展示“恭喜你！小朋友！”和按钮。
|
||||
|
||||
动作输入：
|
||||
|
||||
1. 运行态实时展示左右手位置，手部位置来自 `useMocapInput` 的明确左/右手坐标；
|
||||
2. 任意一只手先接触中央物品 UI 后，当前物品绑定到该手并跟随移动；
|
||||
3. 绑定手带物品进入左侧篮子区域时选择左篮，进入右侧篮子区域时选择右篮；
|
||||
4. 正确时沿用“真棒”反馈和对应篮筐特效，错误时物品弹回中央并回到可再次抓取状态；
|
||||
5. 物品被某只手持有时，手部指示器不再压在物品图标中心；左手吸附到当前物品图标左下角，右手吸附到当前物品图标右下角，保持图案主体可读；
|
||||
6. 不再使用“左手固定选左篮、右手固定选右篮”的规则，也不再使用连续横向轨迹阈值直接选篮。
|
||||
|
||||
运行态直接通过 `useMocapInput` 消费本地 mocap WebSocket `/stream`。宝贝识物优先使用 `general.limb_nodes` / `limb_nodes` 里的骨架手腕节点作为左右手指示器、抓取和选篮坐标；若当前帧没有骨架手腕，再回退到每只手的 `wrist` 挂点，最后才回退到 `hand.x / hand.y`。该策略只让 `useMocapInput` 额外暴露 `bodyJoints.leftWrist/rightWrist`，不修改全局掌心派生点规则，避免影响拼图、热身关和其它运行态。选篮不再通过 `wave_left_hand`、`wave_right_hand`、`wave` 等动作名触发；侧别为 `unknown` 的手部轨迹不参与抓取或选篮。动作判定只在 `active` 阶段开放，礼盒入场、礼盒打开、物品出现、正确反馈和错误反馈阶段收到的动作包必须清空持有状态并忽略，不允许跨阶段补判定。当前本地 mocap 输出的 handedness 按摄像头视角标记，宝贝识物运行态必须先换算为用户身体视角：骨架 `rightWrist` / `rightHand.wrist` / `rightHand` 坐标映射玩家左手，骨架 `leftWrist` / `leftHand.wrist` / `leftHand` 坐标映射玩家右手；换算只用于展示和抓取手身份，不再决定只能选择哪一侧篮子。草稿试玩、发布后正式体验和热身关后的本地 Demo 都复用同一个运行态，因此三条入口都必须具备同一套动作控制能力。
|
||||
|
||||
开发者调试输入：
|
||||
|
||||
1. 鼠标左键按下并拖动：映射左手位置；
|
||||
2. 鼠标右键按下并拖动：映射右手位置；
|
||||
3. 调试输入同样必须先触碰中央物品，物品绑定到目标手后，再拖入左侧或右侧篮子完成选择。
|
||||
|
||||
运行态控制按钮不参与调试输入和选篮判定。左上角返回按钮、完成弹层按钮以及后续新增的运行态控制元素，其 `pointerdown` 不得被舞台拖拽逻辑 `preventDefault` 或指针捕获吞掉，保证游戏进行中仍可直接点击返回。
|
||||
|
||||
当前篮子判定仍只认篮子主体附近区域，但在上一版核心区基础上扩大约 50%；命中阈值为左篮 `x <= 0.36 && y >= 0.62`、右篮 `x >= 0.64 && y >= 0.62`，既避免物品尚未贴近篮子主体就提前判定，也避免贴到篮子边缘后仍难以命中。
|
||||
|
||||
运行态不得新增计时、失败次数、分数、体力或难度递增规则。
|
||||
|
||||
音效和语音播报当前只保留接口预留边界，正式语音接口后续接入。
|
||||
|
||||
## 7. 发布约束
|
||||
|
||||
发布前必须执行：
|
||||
|
||||
1. 两个物品名非空；
|
||||
2. 两个物品名对应的 asset 存在；
|
||||
3. 标签补齐精确 `寓教于乐`；
|
||||
4. `publicationStatus` 从 `draft` 变为 `published`。
|
||||
|
||||
发布后首版本地响应返回 `publicWorkCode`，用于分享弹窗；正式后端接入时 public code 生成规则需要纳入统一作品号服务。
|
||||
|
||||
## 8. 热身关衔接
|
||||
|
||||
`/child-motion-demo` 热身完成后的“开始游戏”按钮进入同一个 `BabyObjectMatchRuntimeShell`。
|
||||
|
||||
热身关独立 Demo 没有创作者草稿上下文，因此使用固定本地 Demo 草稿承载两个物品，仅用于热身关后验证首关体验；正式平台体验仍必须从 `宝贝识物` 模板创作发布后进入寓教于乐板块。
|
||||
|
||||
## 9. 验收命令
|
||||
|
||||
```bash
|
||||
npm run test -- src/components/platform-entry/platformEntryCreationTypes.test.ts src/components/edutainment-creation/BabyObjectMatchWorkspace.test.tsx src/components/edutainment-result/BabyObjectMatchResultView.test.tsx src/components/edutainment-runtime/BabyObjectMatchRuntimeShell.test.tsx src/components/child-motion-demo/ChildMotionWarmupDemo.test.tsx src/services/edutainment-baby-object/babyObjectMatchClient.test.ts
|
||||
cargo test -p api-server edutainment_baby_object --manifest-path server-rs/Cargo.toml
|
||||
npx vitest run src/components/platform-entry/platformEdutainmentVisibility.test.ts src/components/platform-entry/PlatformWorkDetailView.test.tsx src/components/custom-world-home/creationWorkShelf.test.ts src/services/useMocapInput.test.ts src/services/child-motion-demo/childMotionDebugInput.test.ts src/routing/appRoutes.test.ts
|
||||
npx eslint src/components/platform-entry/platformEntryCreationTypes.ts src/components/platform-entry/platformEntryCreationTypes.test.ts src/components/platform-entry/PlatformEntryFlowShellImpl.tsx --ext .ts,.tsx --max-warnings 0
|
||||
npm run check:encoding
|
||||
npm run typecheck
|
||||
npm run build:raw
|
||||
```
|
||||
|
||||
若后续接入真实 Rust API 和 SpacetimeDB 表，再补充 `npm run api-server`、`/healthz`、Rust contract / api-server / spacetime-client 定向测试和 migration 表目录更新。
|
||||
File diff suppressed because it is too large
Load Diff
@@ -24,7 +24,9 @@ export type BabyObjectMatchVisualAssetKind =
|
||||
| 'ui-frame'
|
||||
| 'gift-box'
|
||||
| 'basket'
|
||||
| 'smoke-puff';
|
||||
| 'smoke-puff'
|
||||
| 'left-hand'
|
||||
| 'right-hand';
|
||||
|
||||
export type BabyObjectMatchVisualAsset = {
|
||||
assetId: string;
|
||||
|
||||
@@ -158,6 +158,26 @@ const assetDefinitions = [
|
||||
chromaKeyNote,
|
||||
].join(''),
|
||||
},
|
||||
{
|
||||
id: 'character-outline-only-v3',
|
||||
output: 'picture-book-character-outline-v3.png',
|
||||
sourceOutput: 'picture-book-character-outline-v2.png',
|
||||
sourceDirectory: 'asset',
|
||||
transparent: true,
|
||||
localPostprocess: 'character-outline-only',
|
||||
prompt:
|
||||
'本地后处理资源：基于 character-outline-v2 提取用户角色外轮廓，只保留浅青白描边，中间完全透明，不保留原有半透明材质、填充和明暗变化。',
|
||||
},
|
||||
{
|
||||
id: 'character-outline-white-v4',
|
||||
output: 'picture-book-character-outline-v4.png',
|
||||
sourceOutput: 'picture-book-character-outline-v2.png',
|
||||
sourceDirectory: 'asset',
|
||||
transparent: true,
|
||||
localPostprocess: 'character-outline-white-thin',
|
||||
prompt:
|
||||
'本地后处理资源：基于 character-outline-v2 提取用户角色外轮廓，先弱化耳朵、手指、脚趾等细碎凸起，再输出更细的白色描边，中间完全透明。',
|
||||
},
|
||||
{
|
||||
id: 'hud-strip',
|
||||
output: 'picture-book-hud-strip-v2.png',
|
||||
@@ -601,6 +621,16 @@ const assetDefinitions = [
|
||||
chromaKeyNote,
|
||||
].join(''),
|
||||
},
|
||||
{
|
||||
id: 'wave-cat-body-guide-v7',
|
||||
output: 'picture-book-wave-cat-body-guide-v7.png',
|
||||
sourceOutput: 'picture-book-wave-cat-body-guide-v6.png',
|
||||
sourceDirectory: 'asset',
|
||||
transparent: true,
|
||||
localPostprocess: 'remove-cat-body-shoulder-dots',
|
||||
prompt:
|
||||
'本地后处理资源：基于 wave-cat-body-guide-v6 去除身体左右两侧不协调的小圆点，保留猫头、身体、透明边界和整体水彩风格。',
|
||||
},
|
||||
{
|
||||
id: 'wave-cat-arm-guide-v6',
|
||||
output: 'picture-book-wave-cat-arm-guide-v6.png',
|
||||
@@ -632,6 +662,37 @@ const assetDefinitions = [
|
||||
chromaKeyNote,
|
||||
].join(''),
|
||||
},
|
||||
{
|
||||
id: 'wave-cat-arm-guide-v7',
|
||||
output: 'picture-book-wave-cat-arm-guide-v7.png',
|
||||
sourceOutput: 'picture-book-wave-cat-arm-guide-v7-source.png',
|
||||
size: '1024x1024',
|
||||
transparent: true,
|
||||
transparencyCleanup: 'cat-guide',
|
||||
useWaveCatHeadReference: true,
|
||||
useWaveCatArmReference: true,
|
||||
layoutNormalization: {
|
||||
canvasWidth: 1024,
|
||||
canvasHeight: 1024,
|
||||
fit: 'contain',
|
||||
fillWidth: 0.74,
|
||||
fillHeight: 0.88,
|
||||
anchorY: 'bottom',
|
||||
padding: 20,
|
||||
},
|
||||
prompt: [
|
||||
'请在参考手臂资源的基础上重新绘制儿童动作互动游戏猫咪挥手动画用的单侧手臂资源，严格作为可动纸偶拆件。只画一条橘白猫手臂：底部是肩膀连接端，向左上方弯曲，末端是一只简化圆猫手。',
|
||||
'关键修改：末端圆猫爪必须正面对镜头，像在对观众挥手。圆爪正面轮廓要清楚可见，不要转成侧面，不要转向画面内侧或角色中心，不要画成握拳或背面。可以用浅奶油白圆形爪面、柔和高光和非常淡的短弧线表现正面对镜头。',
|
||||
'猫手必须像多啦A梦式圆手或软玩具圆爪：一个完整圆润圆爪，不画分开的手指，不画尖爪，不画黑色或深色爪垫。若需要爪面细节，只允许非常浅的桃色小圆面或柔和弧线，不能变成真实动物爪垫。',
|
||||
'手臂短而厚实，像小猫上肢，不要成人类长手臂。资源必须适合网页左右镜像复用和围绕肩部连接点旋转：肩膀连接端在画面底部偏内侧，圆手在画面上方，四周留透明空白。',
|
||||
'颜色参考输入猫猫头和参考手臂：浅奶油白和淡橘色为主体，少量浅橘斑纹，柔和暖棕或浅橘棕描边；不要纯黑粗描边。',
|
||||
'请避免粉色大背景、避免主体外侧彩色光晕，主体贴纸外轮廓之外必须直接是纯色背景。资源自身保持清晰不透明，半透明效果由网页 CSS 控制。',
|
||||
'不要画猫头、躯干、另一只手臂、完整动物、腿、脚、文字、数字、按钮、面板、水印、真实照片质感、黑色、灰色、黑白毛、黑灰重阴影或深色大面积毛。',
|
||||
styleReferenceNote,
|
||||
noStretchNote,
|
||||
chromaKeyNote,
|
||||
].join(''),
|
||||
},
|
||||
];
|
||||
|
||||
const args = new Map();
|
||||
@@ -811,6 +872,12 @@ function buildRequestBody(asset, size) {
|
||||
path.join(assetDir, 'picture-book-wave-cat-head-guide-v2.png'),
|
||||
);
|
||||
}
|
||||
if (asset.useWaveCatArmReference) {
|
||||
pushReferenceImage(
|
||||
body,
|
||||
path.join(assetDir, 'picture-book-wave-cat-arm-guide-v6.png'),
|
||||
);
|
||||
}
|
||||
return body;
|
||||
}
|
||||
|
||||
@@ -864,6 +931,9 @@ function outputPathFor(asset) {
|
||||
}
|
||||
|
||||
function sourceOutputPathFor(asset) {
|
||||
if (asset.sourceDirectory === 'asset') {
|
||||
return path.join(assetDir, asset.sourceOutput || asset.output);
|
||||
}
|
||||
return path.join(intermediateDir, asset.sourceOutput || asset.output);
|
||||
}
|
||||
|
||||
@@ -1038,6 +1108,92 @@ function removeCharacterOutlineChromaKey(sourcePath, finalPath) {
|
||||
}
|
||||
}
|
||||
|
||||
function createCharacterOutlineOnlyIndicator(sourcePath, finalPath) {
|
||||
const script = [
|
||||
'from PIL import Image, ImageChops, ImageFilter',
|
||||
'import sys',
|
||||
'source, out = sys.argv[1], sys.argv[2]',
|
||||
'im = Image.open(source).convert("RGBA")',
|
||||
'alpha = im.getchannel("A")',
|
||||
'mask = alpha.point(lambda v: 255 if v > 24 else 0)',
|
||||
'mask = mask.filter(ImageFilter.MaxFilter(5)).filter(ImageFilter.MinFilter(5))',
|
||||
'outer = mask.filter(ImageFilter.MaxFilter(47))',
|
||||
'inner = mask.filter(ImageFilter.MinFilter(47))',
|
||||
'stroke = ImageChops.subtract(outer, inner)',
|
||||
'stroke = stroke.filter(ImageFilter.GaussianBlur(0.45))',
|
||||
'glow = stroke.filter(ImageFilter.GaussianBlur(3.0)).point(lambda v: int(v * 0.34))',
|
||||
'result = Image.new("RGBA", im.size, (0, 0, 0, 0))',
|
||||
'glow_layer = Image.new("RGBA", im.size, (91, 205, 197, 0))',
|
||||
'glow_layer.putalpha(glow)',
|
||||
'line_layer = Image.new("RGBA", im.size, (224, 255, 247, 0))',
|
||||
'line_layer.putalpha(stroke.point(lambda v: min(235, int(v * 0.92))))',
|
||||
'result.alpha_composite(glow_layer)',
|
||||
'result.alpha_composite(line_layer)',
|
||||
'result.save(out)',
|
||||
].join('\n');
|
||||
|
||||
const result = spawnSync('python', ['-c', script, sourcePath, finalPath], {
|
||||
cwd: repoRoot,
|
||||
encoding: 'utf8',
|
||||
});
|
||||
|
||||
if (result.status !== 0) {
|
||||
throw new Error(
|
||||
`Failed to create outline-only character indicator: ${(result.stderr || result.stdout).trim()}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function createWhiteCharacterOutlineIndicator(sourcePath, finalPath) {
|
||||
const script = [
|
||||
'from pathlib import Path',
|
||||
'import cv2',
|
||||
'import numpy as np',
|
||||
'from PIL import Image',
|
||||
'import sys',
|
||||
'source, out = Path(sys.argv[1]), Path(sys.argv[2])',
|
||||
'rgba = np.array(Image.open(source).convert("RGBA"))',
|
||||
'alpha = rgba[:, :, 3]',
|
||||
'mask = (alpha > 24).astype(np.uint8) * 255',
|
||||
'contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)',
|
||||
'body = np.zeros_like(mask)',
|
||||
'if contours:',
|
||||
' largest = max(contours, key=cv2.contourArea)',
|
||||
' cv2.drawContours(body, [largest], -1, 255, thickness=cv2.FILLED)',
|
||||
'open_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (25, 25))',
|
||||
'close_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (35, 35))',
|
||||
'body = cv2.morphologyEx(body, cv2.MORPH_OPEN, open_kernel, iterations=1)',
|
||||
'body = cv2.morphologyEx(body, cv2.MORPH_CLOSE, close_kernel, iterations=1)',
|
||||
'body = cv2.GaussianBlur(body, (0, 0), 7.0)',
|
||||
'_, body = cv2.threshold(body, 92, 255, cv2.THRESH_BINARY)',
|
||||
'body = cv2.GaussianBlur(body, (0, 0), 1.4)',
|
||||
'_, body = cv2.threshold(body, 64, 255, cv2.THRESH_BINARY)',
|
||||
'line_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (15, 15))',
|
||||
'outer = cv2.dilate(body, line_kernel, iterations=1)',
|
||||
'inner = cv2.erode(body, line_kernel, iterations=1)',
|
||||
'stroke = cv2.subtract(outer, inner)',
|
||||
'stroke = cv2.GaussianBlur(stroke, (0, 0), 0.55)',
|
||||
'glow = cv2.GaussianBlur(stroke, (0, 0), 2.2)',
|
||||
'result = np.zeros((mask.shape[0], mask.shape[1], 4), dtype=np.uint8)',
|
||||
'glow_alpha = np.clip(glow.astype(np.float32) * 0.22, 0, 70).astype(np.uint8)',
|
||||
'line_alpha = np.clip(stroke.astype(np.float32) * 0.78, 0, 205).astype(np.uint8)',
|
||||
'result[:, :, 0:3] = 255',
|
||||
'result[:, :, 3] = np.maximum(glow_alpha, line_alpha)',
|
||||
'Image.fromarray(result, "RGBA").save(out)',
|
||||
].join('\n');
|
||||
|
||||
const result = spawnSync('python', ['-c', script, sourcePath, finalPath], {
|
||||
cwd: repoRoot,
|
||||
encoding: 'utf8',
|
||||
});
|
||||
|
||||
if (result.status !== 0) {
|
||||
throw new Error(
|
||||
`Failed to create thin white character indicator: ${(result.stderr || result.stdout).trim()}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function removeCatGuideChromaKey(sourcePath, finalPath) {
|
||||
const script = [
|
||||
'from collections import deque',
|
||||
@@ -1253,6 +1409,50 @@ function scrubChromaFringe(finalPath) {
|
||||
}
|
||||
}
|
||||
|
||||
function removeCatBodyShoulderDots(sourcePath, finalPath) {
|
||||
const script = [
|
||||
'from pathlib import Path',
|
||||
'import cv2',
|
||||
'import numpy as np',
|
||||
'from PIL import Image',
|
||||
'source, out = Path(__import__("sys").argv[1]), Path(__import__("sys").argv[2])',
|
||||
'rgba = np.array(Image.open(source).convert("RGBA"))',
|
||||
'rgb = rgba[:, :, :3].copy()',
|
||||
'alpha = rgba[:, :, 3]',
|
||||
'opaque = alpha > 10',
|
||||
'known = opaque.astype(np.uint8)',
|
||||
'unknown = (1 - known).astype(np.uint8)',
|
||||
'_, labels = cv2.distanceTransformWithLabels(unknown, cv2.DIST_L2, 5, labelType=cv2.DIST_LABEL_PIXEL)',
|
||||
'flat_known_indices = np.flatnonzero(known.reshape(-1))',
|
||||
'filled_rgb = rgb.copy().reshape(-1, 3)',
|
||||
'labels_flat = labels.reshape(-1)',
|
||||
'unknown_flat = unknown.reshape(-1).astype(bool)',
|
||||
'if flat_known_indices.size > 0 and unknown_flat.any():',
|
||||
' nearest_known_flat_index = flat_known_indices[np.maximum(labels_flat[unknown_flat] - 1, 0)]',
|
||||
' filled_rgb[unknown_flat] = filled_rgb[nearest_known_flat_index]',
|
||||
'filled_rgb = filled_rgb.reshape(rgb.shape)',
|
||||
'bgr = cv2.cvtColor(filled_rgb, cv2.COLOR_RGB2BGR)',
|
||||
'mask = np.zeros(alpha.shape, dtype=np.uint8)',
|
||||
'cv2.ellipse(mask, (383, 763), (23, 26), 0, 0, 360, 255, -1)',
|
||||
'cv2.ellipse(mask, (648, 762), (23, 26), 0, 0, 360, 255, -1)',
|
||||
'mask = cv2.bitwise_and(mask, opaque.astype(np.uint8) * 255)',
|
||||
'repaired = cv2.inpaint(bgr, mask, 7, cv2.INPAINT_TELEA)',
|
||||
'repaired_rgb = cv2.cvtColor(repaired, cv2.COLOR_BGR2RGB)',
|
||||
'Image.fromarray(np.dstack([repaired_rgb, alpha]), "RGBA").save(out)',
|
||||
].join('\n');
|
||||
|
||||
const result = spawnSync('python', ['-c', script, sourcePath, finalPath], {
|
||||
cwd: repoRoot,
|
||||
encoding: 'utf8',
|
||||
});
|
||||
|
||||
if (result.status !== 0) {
|
||||
throw new Error(
|
||||
`Failed to remove cat body shoulder dots: ${(result.stderr || result.stdout).trim()}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function writeOpaquePng(sourcePath, outputPath) {
|
||||
const result = spawnSync(
|
||||
'python',
|
||||
@@ -1291,6 +1491,54 @@ async function generateAsset(asset, env, size, force) {
|
||||
}
|
||||
|
||||
if (args.has('--postprocess-only')) {
|
||||
if (asset.localPostprocess === 'character-outline-white-thin') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for postprocess-only: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
createWhiteCharacterOutlineIndicator(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (asset.localPostprocess === 'character-outline-only') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for postprocess-only: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
createCharacterOutlineOnlyIndicator(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (asset.localPostprocess === 'remove-cat-body-shoulder-dots') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for postprocess-only: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
removeCatBodyShoulderDots(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (!asset.transparent) {
|
||||
return {
|
||||
id: asset.id,
|
||||
@@ -1328,6 +1576,54 @@ async function generateAsset(asset, env, size, force) {
|
||||
};
|
||||
}
|
||||
|
||||
if (asset.localPostprocess === 'character-outline-white-thin') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for local postprocess: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
createWhiteCharacterOutlineIndicator(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (asset.localPostprocess === 'character-outline-only') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for local postprocess: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
createCharacterOutlineOnlyIndicator(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
if (asset.localPostprocess === 'remove-cat-body-shoulder-dots') {
|
||||
const sourcePath = sourceOutputPathFor(asset);
|
||||
if (!existsSync(sourcePath)) {
|
||||
throw new Error(`Missing source image for local postprocess: ${sourcePath}`);
|
||||
}
|
||||
mkdirSync(assetDir, { recursive: true });
|
||||
removeCatBodyShoulderDots(sourcePath, finalPath);
|
||||
return {
|
||||
id: asset.id,
|
||||
ok: true,
|
||||
file: finalPath,
|
||||
sourceFile: sourcePath,
|
||||
postprocessedOnly: true,
|
||||
};
|
||||
}
|
||||
|
||||
const requestBody = buildRequestBody(asset, size);
|
||||
const payloadText = await fetchWithTimeout(
|
||||
buildVectorEngineImagesGenerationUrl(env.baseUrl),
|
||||
@@ -1427,6 +1723,7 @@ function dryRun(selectedAssets, size) {
|
||||
? sourceOutputPathFor(asset)
|
||||
: undefined,
|
||||
transparent: asset.transparent,
|
||||
localPostprocess: asset.localPostprocess,
|
||||
body: {
|
||||
...body,
|
||||
image: body.image ? ['<local style reference image>'] : undefined,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -48,6 +48,8 @@ const mocapMock = vi.hoisted(() => ({
|
||||
rightShoulder?: { x: number; y: number } | null;
|
||||
leftElbow?: { x: number; y: number } | null;
|
||||
rightElbow?: { x: number; y: number } | null;
|
||||
leftWrist?: { x: number; y: number } | null;
|
||||
rightWrist?: { x: number; y: number } | null;
|
||||
};
|
||||
},
|
||||
receivedAtMs: 1,
|
||||
@@ -242,16 +244,18 @@ test('renders the warmup stage and starts with the center ring step', () => {
|
||||
render(<ChildMotionWarmupDemo />);
|
||||
|
||||
expect(screen.getByTestId('child-motion-demo')).toBeTruthy();
|
||||
expect(screen.getByText('来到圆圈这里')).toBeTruthy();
|
||||
expect(screen.queryByRole('heading', { name: '来到圆圈这里' })).toBeNull();
|
||||
expect(screen.getByText('欢迎你，小朋友，见到你真开心')).toBeTruthy();
|
||||
expect(screen.queryByLabelText('绿色圆环')).toBeNull();
|
||||
expect(screen.getByText('请横屏体验')).toBeTruthy();
|
||||
});
|
||||
|
||||
test('shows narration first before revealing the step cue', async () => {
|
||||
test('shows the first subtitle before revealing the step cue', async () => {
|
||||
vi.useFakeTimers();
|
||||
render(<ChildMotionWarmupDemo />);
|
||||
|
||||
expect(screen.getByText('来到圆圈这里')).toBeTruthy();
|
||||
expect(screen.getByText('欢迎你，小朋友，见到你真开心')).toBeTruthy();
|
||||
expect(screen.queryByText('来圆圈这里和我打个招呼吧')).toBeNull();
|
||||
expect(screen.queryByLabelText('绿色圆环')).toBeNull();
|
||||
expect(screen.getByTestId('child-motion-stage').dataset.stepPhase).toBe('intro');
|
||||
|
||||
@@ -261,6 +265,25 @@ test('shows narration first before revealing the step cue', async () => {
|
||||
expect(screen.getByTestId('child-motion-stage').dataset.stepPhase).toBe('active');
|
||||
});
|
||||
|
||||
test('switches the center step subtitle to the second line after a two second pause', async () => {
|
||||
vi.useFakeTimers();
|
||||
render(<ChildMotionWarmupDemo />);
|
||||
|
||||
expect(screen.queryByRole('heading', { name: '来到圆圈这里' })).toBeNull();
|
||||
expect(screen.getByText('欢迎你，小朋友，见到你真开心')).toBeTruthy();
|
||||
expect(screen.queryByText('来圆圈这里和我打个招呼吧')).toBeNull();
|
||||
|
||||
await advanceWarmupTime(1999);
|
||||
|
||||
expect(screen.getByText('欢迎你，小朋友，见到你真开心')).toBeTruthy();
|
||||
expect(screen.queryByText('来圆圈这里和我打个招呼吧')).toBeNull();
|
||||
|
||||
await advanceWarmupTime(1);
|
||||
|
||||
expect(screen.queryByText('欢迎你，小朋友，见到你真开心')).toBeNull();
|
||||
expect(screen.getByText('来圆圈这里和我打个招呼吧')).toBeTruthy();
|
||||
});
|
||||
|
||||
test('re-entering within the same runtime session opens the start button', () => {
|
||||
markChildMotionWarmupCompletedInRuntime();
|
||||
|
||||
@@ -299,6 +322,50 @@ test('developer keyboard input moves the avatar and triggers jump state', () =>
|
||||
expect(avatar.className).toContain('child-motion-avatar--jumping');
|
||||
});
|
||||
|
||||
test('developer pointer input renders baby object hand indicators in warmup', async () => {
|
||||
vi.useFakeTimers();
|
||||
render(<ChildMotionWarmupDemo />);
|
||||
|
||||
const stage = screen.getByTestId('child-motion-stage');
|
||||
vi.spyOn(stage, 'getBoundingClientRect').mockReturnValue({
|
||||
x: 0,
|
||||
y: 0,
|
||||
width: 1000,
|
||||
height: 500,
|
||||
top: 0,
|
||||
right: 1000,
|
||||
bottom: 500,
|
||||
left: 0,
|
||||
toJSON: () => ({}),
|
||||
});
|
||||
|
||||
await revealCurrentStepCue();
|
||||
const pointerDownEvent = new Event('pointerdown', {
|
||||
bubbles: true,
|
||||
cancelable: true,
|
||||
});
|
||||
Object.defineProperties(pointerDownEvent, {
|
||||
button: { value: 0 },
|
||||
buttons: { value: 1 },
|
||||
clientX: { value: 250 },
|
||||
clientY: { value: 150 },
|
||||
pointerId: { value: 1 },
|
||||
});
|
||||
await act(async () => {
|
||||
stage.dispatchEvent(pointerDownEvent);
|
||||
});
|
||||
|
||||
const leftHand = screen.getByTestId('child-motion-left-hand-indicator');
|
||||
expect(leftHand.className).toContain('baby-object-runtime__hand--left');
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-x: 25%',
|
||||
);
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-y: 30%',
|
||||
);
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
test('mocap body center dampens small jitter before moving the avatar', async () => {
|
||||
setMocapBodyCenter(0.5);
|
||||
const { rerender } = render(<ChildMotionWarmupDemo />);
|
||||
@@ -325,6 +392,68 @@ test('mocap body center dampens small jitter before moving the avatar', async ()
|
||||
expect(style).not.toContain('left: 34%');
|
||||
});
|
||||
|
||||
test('mocap hand positions render with baby object hand indicators in body-side mapping', async () => {
|
||||
setMocapCameraHandTrackPoint({ cameraSide: 'right', x: 0.24, y: 0.36 });
|
||||
const { rerender } = render(<ChildMotionWarmupDemo />);
|
||||
|
||||
await act(async () => {
|
||||
rerender(<ChildMotionWarmupDemo />);
|
||||
});
|
||||
|
||||
const leftHand = await screen.findByTestId(
|
||||
'child-motion-left-hand-indicator',
|
||||
);
|
||||
expect(leftHand.className).toContain('baby-object-runtime__hand--left');
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-x: 24%',
|
||||
);
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-y: 36%',
|
||||
);
|
||||
expect(screen.queryByTestId('child-motion-right-hand-indicator')).toBeNull();
|
||||
});
|
||||
|
||||
test('mocap hand indicators prefer skeleton wrist nodes in warmup', async () => {
|
||||
const cameraRightHand = {
|
||||
x: 0.24,
|
||||
y: 0.36,
|
||||
state: 'unknown',
|
||||
side: 'right',
|
||||
wrist: { x: 0.27, y: 0.39 },
|
||||
};
|
||||
mocapMock.command = {
|
||||
actions: [],
|
||||
bodyCenter: { x: 0.5, y: 0.7 },
|
||||
bodyJoints: {
|
||||
leftShoulder: { x: 0.62, y: 0.48 },
|
||||
leftElbow: { x: 0.7, y: 0.5 },
|
||||
rightShoulder: { x: 0.38, y: 0.48 },
|
||||
rightElbow: { x: 0.3, y: 0.5 },
|
||||
rightWrist: { x: 0.64, y: 0.25 },
|
||||
},
|
||||
hands: [cameraRightHand],
|
||||
primaryHand: cameraRightHand,
|
||||
leftHand: null,
|
||||
rightHand: cameraRightHand,
|
||||
};
|
||||
mocapMock.receivedAtMs += 1;
|
||||
const { rerender } = render(<ChildMotionWarmupDemo />);
|
||||
|
||||
await act(async () => {
|
||||
rerender(<ChildMotionWarmupDemo />);
|
||||
});
|
||||
|
||||
const leftHand = await screen.findByTestId(
|
||||
'child-motion-left-hand-indicator',
|
||||
);
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-x: 64%',
|
||||
);
|
||||
expect(leftHand.getAttribute('style')).toContain(
|
||||
'--baby-object-hand-y: 25%',
|
||||
);
|
||||
});
|
||||
|
||||
test('mocap body center keeps the warmup flow on the motion data source', async () => {
|
||||
vi.useFakeTimers();
|
||||
setMocapBodyCenter(0.5);
|
||||
@@ -440,6 +569,33 @@ test('mocap greeting requires a real horizontal wave track', async () => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
test('greeting completion goes to warmup intro without praise float text', async () => {
|
||||
vi.useFakeTimers();
|
||||
const { rerender, unmount } = render(<ChildMotionWarmupDemo />);
|
||||
|
||||
await revealCurrentStepCue();
|
||||
await completeCurrentPositionStepByHold();
|
||||
await vi.waitFor(() => {
|
||||
expect(screen.getByText('打个招呼')).toBeTruthy();
|
||||
});
|
||||
|
||||
await revealCurrentStepCue();
|
||||
await completeGreetingByWaveTrack(rerender);
|
||||
|
||||
expect(screen.queryByText('真棒')).toBeNull();
|
||||
|
||||
await advanceWarmupTime(900);
|
||||
await vi.waitFor(() => {
|
||||
expect(screen.getByText('准备热身')).toBeTruthy();
|
||||
});
|
||||
expect(screen.queryByText('真棒')).toBeNull();
|
||||
|
||||
await act(async () => {
|
||||
unmount();
|
||||
});
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
test('mocap arm swing steps require body-side mapping and vertical open arm motion', async () => {
|
||||
vi.useFakeTimers();
|
||||
const { rerender, unmount } = render(<ChildMotionWarmupDemo />);
|
||||
@@ -477,6 +633,14 @@ test('mocap arm swing steps require body-side mapping and vertical open arm moti
|
||||
});
|
||||
|
||||
await revealCurrentStepCue();
|
||||
expect(screen.getByTestId('child-motion-arm-swing-guide-left')).toBeTruthy();
|
||||
expect(screen.queryByTestId('child-motion-arm-swing-guide-right')).toBeNull();
|
||||
expect(
|
||||
screen
|
||||
.getByTestId('child-motion-arm-swing-guide-left')
|
||||
.querySelector('.child-motion-gesture-guide__arm-swing-paw-asset'),
|
||||
).toBeTruthy();
|
||||
|
||||
await sendMocapCameraHandTrack(rerender, 'left', [
|
||||
{ x: 0.78, y: 0.5 },
|
||||
{ x: 0.86, y: 0.5 },
|
||||
@@ -505,6 +669,14 @@ test('mocap arm swing steps require body-side mapping and vertical open arm moti
|
||||
});
|
||||
|
||||
await revealCurrentStepCue();
|
||||
expect(screen.getByTestId('child-motion-arm-swing-guide-right')).toBeTruthy();
|
||||
expect(screen.queryByTestId('child-motion-arm-swing-guide-left')).toBeNull();
|
||||
expect(
|
||||
screen
|
||||
.getByTestId('child-motion-arm-swing-guide-right')
|
||||
.querySelector('.child-motion-gesture-guide__arm-swing-paw-asset'),
|
||||
).toBeTruthy();
|
||||
|
||||
await sendMocapCameraHandTrack(rerender, 'right', [
|
||||
{ x: 0.2, y: 0.5 },
|
||||
{ x: 0.16, y: 0.42 },
|
||||
@@ -519,8 +691,9 @@ test('mocap arm swing steps require body-side mapping and vertical open arm moti
|
||||
|
||||
await advanceWarmupTime(900);
|
||||
await vi.waitFor(() => {
|
||||
expect(screen.getByRole('heading', { name: '原地跳一下' })).toBeTruthy();
|
||||
expect(screen.getByRole('heading', { name: '热身完成' })).toBeTruthy();
|
||||
});
|
||||
expect(screen.queryByRole('heading', { name: '原地跳一下' })).toBeNull();
|
||||
await advanceWarmupTime(720);
|
||||
await act(async () => {
|
||||
unmount();
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { CSSProperties, PointerEvent as ReactPointerEvent } from 'react';
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
|
||||
import type { BabyObjectMatchDraft } from '../../../packages/shared/src/contracts/edutainmentBabyObject';
|
||||
import {
|
||||
@@ -40,9 +40,9 @@ type WarmupStepPhase = 'intro' | 'active' | 'complete';
|
||||
type WarmupMocapGestureIntent =
|
||||
| 'greeting'
|
||||
| 'left-hand'
|
||||
| 'right-hand'
|
||||
| 'jump';
|
||||
| 'right-hand';
|
||||
type WarmupBodyHandSide = 'left' | 'right';
|
||||
type WarmupHandIndicators = Record<DragHand, ChildMotionPoint | null>;
|
||||
|
||||
const CHILD_MOTION_BABY_OBJECT_DEMO_DRAFT: BabyObjectMatchDraft = {
|
||||
draftId: 'child-motion-demo-baby-object-draft',
|
||||
@@ -94,7 +94,9 @@ const WARMUP_GREETING_WAVE_DIRECTION_EPSILON = 0.008;
|
||||
const WARMUP_GREETING_WRIST_ABOVE_ELBOW_MARGIN = 0.04;
|
||||
const WARMUP_GREETING_WRIST_ABOVE_SHOULDER_MARGIN = 0.08;
|
||||
const WARMUP_STEP_INTRO_DELAY_MS = 1000;
|
||||
const WARMUP_SUBTITLE_LINE_DELAY_MS = 2000;
|
||||
const WARMUP_STEP_COMPLETE_PAUSE_MS = 820;
|
||||
const WARMUP_TOTAL_STEPS = 11;
|
||||
const AVATAR_MOCAP_DEAD_ZONE = 0.012;
|
||||
const AVATAR_MOCAP_SMOOTHING = 0.28;
|
||||
const AVATAR_MOCAP_MAX_STEP = 0.035;
|
||||
@@ -128,6 +130,13 @@ function formatAvatarLeftPercent(value: number) {
|
||||
return `${Math.round(clampMotionUnit(value) * 1000) / 10}%`;
|
||||
}
|
||||
|
||||
function createEmptyWarmupHandIndicators(): WarmupHandIndicators {
|
||||
return {
|
||||
left: null,
|
||||
right: null,
|
||||
};
|
||||
}
|
||||
|
||||
function resolveMocapHandWithBodySide(
|
||||
command: MocapInputCommand,
|
||||
side: WarmupBodyHandSide,
|
||||
@@ -136,6 +145,29 @@ function resolveMocapHandWithBodySide(
|
||||
return side === 'left' ? command.rightHand : command.leftHand;
|
||||
}
|
||||
|
||||
function resolveMocapSkeletonWristWithBodySide(
|
||||
command: MocapInputCommand,
|
||||
side: WarmupBodyHandSide,
|
||||
) {
|
||||
const joints = command.bodyJoints;
|
||||
return side === 'left' ? joints?.rightWrist : joints?.leftWrist;
|
||||
}
|
||||
|
||||
function resolveWarmupHandIndicatorsFromMocap(
|
||||
command: MocapInputCommand,
|
||||
): WarmupHandIndicators {
|
||||
return {
|
||||
left: mocapHandToWarmupIndicatorPoint(
|
||||
resolveMocapHandWithBodySide(command, 'left'),
|
||||
resolveMocapSkeletonWristWithBodySide(command, 'left'),
|
||||
),
|
||||
right: mocapHandToWarmupIndicatorPoint(
|
||||
resolveMocapHandWithBodySide(command, 'right'),
|
||||
resolveMocapSkeletonWristWithBodySide(command, 'right'),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
function resolveMocapJointWithBodySide(
|
||||
command: MocapInputCommand,
|
||||
side: WarmupBodyHandSide,
|
||||
@@ -175,6 +207,22 @@ function mocapHandToChildMotionPoint(
|
||||
};
|
||||
}
|
||||
|
||||
function mocapHandToWarmupIndicatorPoint(
|
||||
hand: MocapHandInput | null | undefined,
|
||||
skeletonWrist: MocapPointInput | null | undefined,
|
||||
): ChildMotionPoint | null {
|
||||
// 骨架手腕节点比手掌识别结果更稳定；热身指示器优先跟随骨架手腕。
|
||||
const point = skeletonWrist ?? hand?.wrist ?? hand;
|
||||
if (!point) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
x: clampMotionUnit(point.x),
|
||||
y: clampMotionUnit(point.y),
|
||||
};
|
||||
}
|
||||
|
||||
function appendWarmupMocapPoint(
|
||||
points: ChildMotionPoint[],
|
||||
point: ChildMotionPoint,
|
||||
@@ -218,13 +266,6 @@ function getMotionSourceText(state: MotionSourceState) {
|
||||
return '正在连接动作数据';
|
||||
}
|
||||
|
||||
function hasWarmupMocapAction(
|
||||
command: MocapInputCommand,
|
||||
expectedActions: string[],
|
||||
) {
|
||||
return command.actions.some((action) => expectedActions.includes(action));
|
||||
}
|
||||
|
||||
function countWarmupVerticalDirectionChanges(points: ChildMotionPoint[]) {
|
||||
let previousDirection = 0;
|
||||
let directionChanges = 0;
|
||||
@@ -403,7 +444,6 @@ function resolveDampedAvatarX(current: number, target: number) {
|
||||
|
||||
function resolveWarmupMocapGestureIntent(
|
||||
stepId: ChildMotionWarmupStepId,
|
||||
command: MocapInputCommand,
|
||||
paths: {
|
||||
leftHandPath: ChildMotionPoint[];
|
||||
rightHandPath: ChildMotionPoint[];
|
||||
@@ -434,19 +474,6 @@ function resolveWarmupMocapGestureIntent(
|
||||
return 'right-hand';
|
||||
}
|
||||
|
||||
if (
|
||||
stepId === 'jump_once' &&
|
||||
hasWarmupMocapAction(command, [
|
||||
'jump',
|
||||
'jump_once',
|
||||
'hop',
|
||||
'跳跃',
|
||||
'原地跳',
|
||||
])
|
||||
) {
|
||||
return 'jump';
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -475,7 +502,6 @@ function getStepIndex(stepId: ChildMotionWarmupStepId) {
|
||||
'return_center_2',
|
||||
'wave_left_hand',
|
||||
'wave_right_hand',
|
||||
'jump_once',
|
||||
'warmup_finish',
|
||||
'level_select',
|
||||
];
|
||||
@@ -546,11 +572,15 @@ function ChildMotionGestureGuide({
|
||||
const isLeft = stepId === 'wave_left_hand';
|
||||
const isRight = stepId === 'wave_right_hand';
|
||||
const isGreeting = stepId === 'wave_greeting';
|
||||
const isJump = stepId === 'jump_once';
|
||||
const activePath = isLeft ? leftHandPath : isRight ? rightHandPath : [];
|
||||
|
||||
return (
|
||||
<div className="child-motion-gesture-guide" aria-hidden="true">
|
||||
<div
|
||||
className={`child-motion-gesture-guide ${
|
||||
isGreeting ? 'child-motion-gesture-guide--greeting' : ''
|
||||
}`}
|
||||
aria-hidden="true"
|
||||
>
|
||||
{isGreeting ? (
|
||||
<span className="child-motion-gesture-guide__wave-cat">
|
||||
<span className="child-motion-gesture-guide__wave-cat-body" />
|
||||
@@ -561,8 +591,14 @@ function ChildMotionGestureGuide({
|
||||
{isLeft || isRight ? (
|
||||
<>
|
||||
<span
|
||||
className={`child-motion-gesture-guide__arm child-motion-gesture-guide__arm--${isLeft ? 'left' : 'right'}`}
|
||||
/>
|
||||
className={`child-motion-gesture-guide__arm-swing child-motion-gesture-guide__arm-swing--${isLeft ? 'left' : 'right'}`}
|
||||
data-testid={`child-motion-arm-swing-guide-${isLeft ? 'left' : 'right'}`}
|
||||
>
|
||||
<span className="child-motion-gesture-guide__arm-swing-track" />
|
||||
<span className="child-motion-gesture-guide__arm-swing-paw">
|
||||
<span className="child-motion-gesture-guide__arm-swing-paw-asset" />
|
||||
</span>
|
||||
</span>
|
||||
{activePath.map((point, index) => (
|
||||
<span
|
||||
key={`${isLeft ? 'left' : 'right'}-${index}`}
|
||||
@@ -576,9 +612,40 @@ function ChildMotionGestureGuide({
|
||||
))}
|
||||
</>
|
||||
) : null}
|
||||
{isJump ? (
|
||||
<span className="child-motion-gesture-guide__jump">跳</span>
|
||||
) : null}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ChildMotionHandIndicators({
|
||||
hands,
|
||||
}: {
|
||||
hands: WarmupHandIndicators;
|
||||
}) {
|
||||
return (
|
||||
<div
|
||||
className="baby-object-runtime__hands child-motion-hand-indicators"
|
||||
aria-hidden="true"
|
||||
>
|
||||
{(['left', 'right'] as const).map((hand) => {
|
||||
const point = hands[hand];
|
||||
if (!point) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
key={hand}
|
||||
className={`baby-object-runtime__hand baby-object-runtime__hand--${hand} child-motion-hand-indicator`}
|
||||
data-testid={`child-motion-${hand}-hand-indicator`}
|
||||
style={
|
||||
{
|
||||
'--baby-object-hand-x': `${point.x * 100}%`,
|
||||
'--baby-object-hand-y': `${point.y * 100}%`,
|
||||
} as CSSProperties
|
||||
}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -606,10 +673,6 @@ function ChildMotionCalibrationPanel({
|
||||
<span>右手</span>
|
||||
<strong>{calibration.rightHandPath.length}</strong>
|
||||
</div>
|
||||
<div>
|
||||
<span>跳跃</span>
|
||||
<strong>{formatPercent(calibration.jumpSpace)}</strong>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -630,11 +693,15 @@ export function ChildMotionWarmupDemo() {
|
||||
const [nowMs, setNowMs] = useState(() => Date.now());
|
||||
const [leftHandPath, setLeftHandPath] = useState<ChildMotionPoint[]>([]);
|
||||
const [rightHandPath, setRightHandPath] = useState<ChildMotionPoint[]>([]);
|
||||
const [handIndicators, setHandIndicators] = useState(
|
||||
createEmptyWarmupHandIndicators,
|
||||
);
|
||||
const [activeHand, setActiveHand] = useState<DragHand | null>(null);
|
||||
const [isJumping, setIsJumping] = useState(false);
|
||||
const [justCompletedText, setJustCompletedText] = useState<string | null>(
|
||||
null,
|
||||
);
|
||||
const [subtitleLineIndex, setSubtitleLineIndex] = useState(0);
|
||||
const [cameraAccessState, setCameraAccessState] = useState<CameraAccessState>(
|
||||
() =>
|
||||
typeof navigator === 'undefined' || !navigator.mediaDevices?.getUserMedia
|
||||
@@ -657,7 +724,9 @@ export function ChildMotionWarmupDemo() {
|
||||
step.kind === 'finish',
|
||||
});
|
||||
const stepIndex = getStepIndex(stepId);
|
||||
const progressPercent = Math.round((stepIndex / 12) * 100);
|
||||
const progressPercent = Math.round(
|
||||
(stepIndex / (WARMUP_TOTAL_STEPS - 1)) * 100,
|
||||
);
|
||||
const holdProgress = getHoldProgress(stepId, avatarX, holdStartedAt, nowMs);
|
||||
const isStepActive = stepPhase === 'active';
|
||||
const shouldShowStepCues = stepPhase !== 'intro';
|
||||
@@ -681,12 +750,14 @@ export function ChildMotionWarmupDemo() {
|
||||
);
|
||||
|
||||
const nextStep = resolveNextChildMotionWarmupStep(stepId);
|
||||
if (stepId === 'jump_once') {
|
||||
if (stepId === 'warmup_finish') {
|
||||
markChildMotionWarmupCompletedInRuntime();
|
||||
}
|
||||
|
||||
const completionText =
|
||||
stepId === 'warmup_finish' || stepId === 'jump_once' ? null : '真棒';
|
||||
stepId === 'wave_greeting' || stepId === 'warmup_finish'
|
||||
? null
|
||||
: '真棒';
|
||||
setJustCompletedText(completionText);
|
||||
setStepPhase('complete');
|
||||
setHoldStartedAt(null);
|
||||
@@ -803,6 +874,7 @@ export function ChildMotionWarmupDemo() {
|
||||
setHoldStartedAt(null);
|
||||
setLeftHandPath([]);
|
||||
setRightHandPath([]);
|
||||
setSubtitleLineIndex(0);
|
||||
handledMocapPacketKeyRef.current = null;
|
||||
|
||||
if (step.kind === 'levelSelect') {
|
||||
@@ -819,6 +891,20 @@ export function ChildMotionWarmupDemo() {
|
||||
return () => window.clearTimeout(timeout);
|
||||
}, [step.kind, stepId]);
|
||||
|
||||
useEffect(() => {
|
||||
if (step.spokenLines.length <= 1) {
|
||||
return;
|
||||
}
|
||||
|
||||
setSubtitleLineIndex(0);
|
||||
const timeout = window.setTimeout(() => {
|
||||
setSubtitleLineIndex((current) =>
|
||||
Math.min(current + 1, step.spokenLines.length - 1),
|
||||
);
|
||||
}, WARMUP_SUBTITLE_LINE_DELAY_MS);
|
||||
return () => window.clearTimeout(timeout);
|
||||
}, [step.spokenLines, stepId]);
|
||||
|
||||
useEffect(() => {
|
||||
if (step.kind !== 'position' || !isStepActive) {
|
||||
return;
|
||||
@@ -925,7 +1011,7 @@ export function ChildMotionWarmupDemo() {
|
||||
setRightHandPath(nextRightHandPath);
|
||||
}
|
||||
|
||||
const intent = resolveWarmupMocapGestureIntent(stepId, command, {
|
||||
const intent = resolveWarmupMocapGestureIntent(stepId, {
|
||||
leftHandPath: nextLeftHandPath,
|
||||
rightHandPath: nextRightHandPath,
|
||||
primaryHandPath: nextPrimaryHandPath,
|
||||
@@ -934,13 +1020,6 @@ export function ChildMotionWarmupDemo() {
|
||||
return;
|
||||
}
|
||||
|
||||
if (intent === 'jump') {
|
||||
setIsJumping(true);
|
||||
window.setTimeout(() => setIsJumping(false), 360);
|
||||
completeStep({ type: 'jump', jumpSpace: 0.14 });
|
||||
return;
|
||||
}
|
||||
|
||||
if (intent === 'right-hand') {
|
||||
const path = [...nextRightHandPath, rightPoint].filter(
|
||||
(point): point is ChildMotionPoint => Boolean(point),
|
||||
@@ -965,6 +1044,21 @@ export function ChildMotionWarmupDemo() {
|
||||
stepId,
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
if (stepPhase === 'complete' || !mocapInput.latestCommand) {
|
||||
return;
|
||||
}
|
||||
|
||||
setHandIndicators(
|
||||
resolveWarmupHandIndicatorsFromMocap(mocapInput.latestCommand),
|
||||
);
|
||||
}, [
|
||||
mocapInput.latestCommand,
|
||||
mocapInput.rawPacketPreview?.receivedAtMs,
|
||||
mocapInput.rawPacketPreview?.text,
|
||||
stepPhase,
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
if (stepPhase === 'complete' || !mocapInput.latestCommand) {
|
||||
return;
|
||||
@@ -1008,15 +1102,12 @@ export function ChildMotionWarmupDemo() {
|
||||
event.preventDefault();
|
||||
setIsJumping(true);
|
||||
window.setTimeout(() => setIsJumping(false), 360);
|
||||
if (stepId === 'jump_once' && isStepActive) {
|
||||
completeStep({ type: 'jump', jumpSpace: 0.14 });
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
window.addEventListener('keydown', handleKeyDown);
|
||||
return () => window.removeEventListener('keydown', handleKeyDown);
|
||||
}, [completeStep, isStepActive, stepId, stepPhase]);
|
||||
}, [stepPhase]);
|
||||
|
||||
useEffect(() => {
|
||||
const handleKeyUp = (event: KeyboardEvent) => {
|
||||
@@ -1040,20 +1131,29 @@ export function ChildMotionWarmupDemo() {
|
||||
return;
|
||||
}
|
||||
|
||||
if (event.button !== 0 && event.button !== 2) {
|
||||
if (
|
||||
event.button !== 0 &&
|
||||
event.button !== 2 &&
|
||||
event.buttons !== 1 &&
|
||||
event.buttons !== 2
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
event.preventDefault();
|
||||
const nextHand: DragHand = event.button === 2 ? 'right' : 'left';
|
||||
const nextHand: DragHand =
|
||||
event.button === 2 || event.buttons === 2 ? 'right' : 'left';
|
||||
setActiveHand(nextHand);
|
||||
const point = normalizePointerPoint(event, event.currentTarget);
|
||||
setHandIndicators((current) => ({ ...current, [nextHand]: point }));
|
||||
if (nextHand === 'left') {
|
||||
setLeftHandPath([point]);
|
||||
} else {
|
||||
setRightHandPath([point]);
|
||||
}
|
||||
event.currentTarget.setPointerCapture(event.pointerId);
|
||||
if (typeof event.currentTarget.setPointerCapture === 'function') {
|
||||
event.currentTarget.setPointerCapture(event.pointerId);
|
||||
}
|
||||
};
|
||||
|
||||
const handleStagePointerMove = (event: ReactPointerEvent<HTMLElement>) => {
|
||||
@@ -1062,6 +1162,7 @@ export function ChildMotionWarmupDemo() {
|
||||
}
|
||||
|
||||
const point = normalizePointerPoint(event, event.currentTarget);
|
||||
setHandIndicators((current) => ({ ...current, [activeHand]: point }));
|
||||
const appendPoint = (points: ChildMotionPoint[]) =>
|
||||
[...points, point].slice(-16);
|
||||
if (activeHand === 'left') {
|
||||
@@ -1076,7 +1177,10 @@ export function ChildMotionWarmupDemo() {
|
||||
return;
|
||||
}
|
||||
|
||||
if (event.currentTarget.hasPointerCapture(event.pointerId)) {
|
||||
if (
|
||||
typeof event.currentTarget.hasPointerCapture === 'function' &&
|
||||
event.currentTarget.hasPointerCapture(event.pointerId)
|
||||
) {
|
||||
event.currentTarget.releasePointerCapture(event.pointerId);
|
||||
}
|
||||
const hand = activeHand;
|
||||
@@ -1110,10 +1214,8 @@ export function ChildMotionWarmupDemo() {
|
||||
setIsBabyObjectRuntimeOpen(true);
|
||||
};
|
||||
|
||||
const lineText = useMemo(
|
||||
() => step.spokenLines.join('，'),
|
||||
[step.spokenLines],
|
||||
);
|
||||
const shouldHideStepTitle = stepId === 'center_arrive';
|
||||
const subtitleText = step.spokenLines[subtitleLineIndex] ?? step.spokenLines[0];
|
||||
|
||||
if (isBabyObjectRuntimeOpen) {
|
||||
return (
|
||||
@@ -1170,6 +1272,7 @@ export function ChildMotionWarmupDemo() {
|
||||
rightHandPath={rightHandPath}
|
||||
/>
|
||||
) : null}
|
||||
<ChildMotionHandIndicators hands={handIndicators} />
|
||||
<ChildMotionAvatar avatarX={avatarX} isJumping={isJumping} />
|
||||
{justCompletedText ? (
|
||||
<div className="child-motion-floating-reward">
|
||||
@@ -1178,10 +1281,16 @@ export function ChildMotionWarmupDemo() {
|
||||
) : null}
|
||||
|
||||
<div className="child-motion-hud child-motion-hud--top">
|
||||
<span className="child-motion-step-count">{`${Math.min(stepIndex + 1, 12)}/12`}</span>
|
||||
<div>
|
||||
<h1>{step.title}</h1>
|
||||
<p>{lineText}</p>
|
||||
<span className="child-motion-step-count">{`${Math.min(stepIndex + 1, WARMUP_TOTAL_STEPS)}/${WARMUP_TOTAL_STEPS}`}</span>
|
||||
<div
|
||||
className={
|
||||
shouldHideStepTitle
|
||||
? 'child-motion-hud__copy child-motion-hud__copy--subtitle-only'
|
||||
: 'child-motion-hud__copy'
|
||||
}
|
||||
>
|
||||
{shouldHideStepTitle ? null : <h1>{step.title}</h1>}
|
||||
<p>{subtitleText}</p>
|
||||
</div>
|
||||
<span className="child-motion-progress">{progressPercent}%</span>
|
||||
</div>
|
||||
|
||||
@@ -22,7 +22,6 @@ describe('childMotionWarmupModel', () => {
|
||||
'return_center_2',
|
||||
'wave_left_hand',
|
||||
'wave_right_hand',
|
||||
'jump_once',
|
||||
'warmup_finish',
|
||||
'level_select',
|
||||
]);
|
||||
@@ -76,19 +75,11 @@ describe('childMotionWarmupModel', () => {
|
||||
],
|
||||
},
|
||||
);
|
||||
const completed = applyChildMotionWarmupCompletion(
|
||||
'jump_once',
|
||||
withRightHand,
|
||||
{
|
||||
type: 'jump',
|
||||
jumpSpace: 0.14,
|
||||
},
|
||||
);
|
||||
|
||||
expect(completed.leftBoundary).toBeCloseTo(0.16);
|
||||
expect(completed.rightBoundary).toBeCloseTo(0.16);
|
||||
expect(completed.leftHandPath).toHaveLength(2);
|
||||
expect(completed.leftHandSpace).toEqual({
|
||||
expect(withRightHand.leftBoundary).toBeCloseTo(0.16);
|
||||
expect(withRightHand.rightBoundary).toBeCloseTo(0.16);
|
||||
expect(withRightHand.leftHandPath).toHaveLength(2);
|
||||
expect(withRightHand.leftHandSpace).toEqual({
|
||||
minX: 0.3,
|
||||
maxX: 0.34,
|
||||
minY: 0.32,
|
||||
@@ -97,7 +88,6 @@ describe('childMotionWarmupModel', () => {
|
||||
maxAngleDeg: 44,
|
||||
maxReach: 0.28,
|
||||
});
|
||||
expect(completed.rightHandSpace?.maxReach).toBe(0.31);
|
||||
expect(completed.jumpSpace).toBe(0.14);
|
||||
expect(withRightHand.rightHandSpace?.maxReach).toBe(0.31);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,7 +8,6 @@ export type ChildMotionWarmupStepId =
|
||||
| 'return_center_2'
|
||||
| 'wave_left_hand'
|
||||
| 'wave_right_hand'
|
||||
| 'jump_once'
|
||||
| 'warmup_finish'
|
||||
| 'level_select';
|
||||
|
||||
@@ -55,7 +54,6 @@ export type ChildMotionWarmupCalibration = {
|
||||
rightHandPath: ChildMotionPoint[];
|
||||
leftHandSpace: ChildMotionHandSpace | null;
|
||||
rightHandSpace: ChildMotionHandSpace | null;
|
||||
jumpSpace: number | null;
|
||||
};
|
||||
|
||||
export type ChildMotionWarmupCompletion =
|
||||
@@ -71,10 +69,6 @@ export type ChildMotionWarmupCompletion =
|
||||
type: 'right-hand';
|
||||
path: ChildMotionPoint[];
|
||||
}
|
||||
| {
|
||||
type: 'jump';
|
||||
jumpSpace: number;
|
||||
}
|
||||
| {
|
||||
type: 'narration';
|
||||
};
|
||||
@@ -92,14 +86,14 @@ export const CHILD_MOTION_WARMUP_STEPS: ChildMotionWarmupStep[] = [
|
||||
id: 'center_arrive',
|
||||
kind: 'position',
|
||||
title: '来到圆圈这里',
|
||||
spokenLines: ['欢迎你，小朋友，见到你真开心', '请你来到圆圈这里和我打个招呼吧'],
|
||||
spokenLines: ['欢迎你，小朋友，见到你真开心', '来圆圈这里和我打个招呼吧'],
|
||||
target: 'center',
|
||||
},
|
||||
{
|
||||
id: 'wave_greeting',
|
||||
kind: 'gesture',
|
||||
title: '打个招呼',
|
||||
spokenLines: ['请你来到圆圈这里和我打个招呼吧'],
|
||||
spokenLines: ['来圆圈这里和我打个招呼吧'],
|
||||
},
|
||||
{
|
||||
id: 'warmup_intro',
|
||||
@@ -147,12 +141,6 @@ export const CHILD_MOTION_WARMUP_STEPS: ChildMotionWarmupStep[] = [
|
||||
title: '挥动右手',
|
||||
spokenLines: ['挥动右手'],
|
||||
},
|
||||
{
|
||||
id: 'jump_once',
|
||||
kind: 'gesture',
|
||||
title: '原地跳一下',
|
||||
spokenLines: ['原地跳一下'],
|
||||
},
|
||||
{
|
||||
id: 'warmup_finish',
|
||||
kind: 'finish',
|
||||
@@ -224,7 +212,6 @@ export function createEmptyChildMotionCalibration(): ChildMotionWarmupCalibratio
|
||||
rightHandPath: [],
|
||||
leftHandSpace: null,
|
||||
rightHandSpace: null,
|
||||
jumpSpace: null,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -290,13 +277,6 @@ export function applyChildMotionWarmupCompletion(
|
||||
};
|
||||
}
|
||||
|
||||
if (stepId === 'jump_once' && completion.type === 'jump') {
|
||||
return {
|
||||
...calibration,
|
||||
jumpSpace: completion.jumpSpace,
|
||||
};
|
||||
}
|
||||
|
||||
return calibration;
|
||||
}
|
||||
|
||||
|
||||
@@ -94,14 +94,6 @@ function createGeneratedDraft() {
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'background',
|
||||
},
|
||||
{
|
||||
assetId: 'baby-object-visual-ui-frame',
|
||||
assetKind: 'ui-frame',
|
||||
imageSrc: 'data:image/png;base64,ui',
|
||||
assetObjectId: null,
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'ui',
|
||||
},
|
||||
{
|
||||
assetId: 'baby-object-visual-gift-box',
|
||||
assetKind: 'gift-box',
|
||||
@@ -118,14 +110,6 @@ function createGeneratedDraft() {
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'basket',
|
||||
},
|
||||
{
|
||||
assetId: 'baby-object-visual-smoke-puff',
|
||||
assetKind: 'smoke-puff',
|
||||
imageSrc: 'data:image/png;base64,smoke',
|
||||
assetObjectId: null,
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'smoke',
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
@@ -38,10 +38,8 @@ function normalizeDraftForAction(draft: BabyObjectMatchDraft) {
|
||||
|
||||
const REQUIRED_VISUAL_ASSET_KINDS = [
|
||||
'background',
|
||||
'ui-frame',
|
||||
'gift-box',
|
||||
'basket',
|
||||
'smoke-puff',
|
||||
] as const;
|
||||
|
||||
export function BabyObjectMatchResultView({
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+404
-75
File diff suppressed because it is too large
Load Diff
@@ -69,14 +69,6 @@ describe('babyObjectMatchClient', () => {
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'background prompt',
|
||||
},
|
||||
{
|
||||
assetId: 'server-ui',
|
||||
assetKind: 'ui-frame',
|
||||
imageSrc: 'data:image/png;base64,ui',
|
||||
assetObjectId: null,
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'ui prompt',
|
||||
},
|
||||
{
|
||||
assetId: 'server-gift',
|
||||
assetKind: 'gift-box',
|
||||
@@ -93,14 +85,6 @@ describe('babyObjectMatchClient', () => {
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'basket prompt',
|
||||
},
|
||||
{
|
||||
assetId: 'server-smoke',
|
||||
assetKind: 'smoke-puff',
|
||||
imageSrc: 'data:image/png;base64,smoke',
|
||||
assetObjectId: null,
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
prompt: 'smoke prompt',
|
||||
},
|
||||
],
|
||||
},
|
||||
}),
|
||||
@@ -127,7 +111,7 @@ describe('babyObjectMatchClient', () => {
|
||||
expect(response.draft.itemAssets[0]?.generationProvider).toBe(
|
||||
'vector-engine-gpt-image-2',
|
||||
);
|
||||
expect(response.draft.visualPackage?.assets).toHaveLength(5);
|
||||
expect(response.draft.visualPackage?.assets).toHaveLength(3);
|
||||
expect(response.draft.visualPackage?.assets[0]?.generationProvider).toBe(
|
||||
'vector-engine-gpt-image-2',
|
||||
);
|
||||
@@ -169,7 +153,7 @@ describe('babyObjectMatchClient', () => {
|
||||
expect(response.draft.visualPackage?.themePrompt).toBe('果园主题视觉包装');
|
||||
expect(
|
||||
response.draft.visualPackage?.assets.map((asset) => asset.assetKind),
|
||||
).toEqual(['background', 'ui-frame', 'gift-box', 'basket', 'smoke-puff']);
|
||||
).toEqual(['background', 'gift-box', 'basket']);
|
||||
expect(response.draft.visualPackage?.assets[0]).toMatchObject({
|
||||
assetId: 'baby-object-visual-background',
|
||||
generationProvider: 'vector-engine-gpt-image-2',
|
||||
|
||||
@@ -28,7 +28,7 @@ const BABY_OBJECT_MATCH_ASSET_REQUEST_RETRY: ApiRetryOptions = {
|
||||
maxRetries: 0,
|
||||
};
|
||||
const BABY_OBJECT_MATCH_REQUIRED_VISUAL_KINDS: BabyObjectMatchVisualAssetKind[] =
|
||||
['background', 'ui-frame', 'gift-box', 'basket', 'smoke-puff'];
|
||||
['background', 'gift-box', 'basket'];
|
||||
const DRAFT_DB_NAME = 'genarrative-edutainment-baby-object-drafts';
|
||||
const DRAFT_DB_VERSION = 1;
|
||||
const DRAFT_STORE_NAME = 'drafts';
|
||||
|
||||
@@ -90,8 +90,10 @@ describe('parseMocapPacket', () => {
|
||||
limb_nodes: [
|
||||
{ name: 'left_shoulder', x: 0.28, y: 0.42 },
|
||||
{ name: 'left_elbow', x: 0.24, y: 0.5 },
|
||||
{ name: 'left_wrist', x: 0.2, y: 0.57 },
|
||||
{ name: 'right_shoulder', x: 0.72, y: 0.42 },
|
||||
{ name: 'right_elbow', x: 0.76, y: 0.5 },
|
||||
{ name: 'right_wrist', x: 0.8, y: 0.57 },
|
||||
],
|
||||
},
|
||||
actions: [{ gesture: 'wave-left-hand' }],
|
||||
@@ -120,8 +122,10 @@ describe('parseMocapPacket', () => {
|
||||
expect(command.bodyJoints).toEqual({
|
||||
leftShoulder: {x: 0.28, y: 0.42},
|
||||
leftElbow: {x: 0.24, y: 0.5},
|
||||
leftWrist: {x: 0.2, y: 0.57},
|
||||
rightShoulder: {x: 0.72, y: 0.42},
|
||||
rightElbow: {x: 0.76, y: 0.5},
|
||||
rightWrist: {x: 0.8, y: 0.57},
|
||||
});
|
||||
expect(command.actions).toEqual(
|
||||
expect.arrayContaining(['wave_left_hand', 'open_palm']),
|
||||
|
||||
@@ -27,6 +27,8 @@ export type MocapBodyJointsInput = {
|
||||
rightShoulder?: MocapPointInput | null;
|
||||
leftElbow?: MocapPointInput | null;
|
||||
rightElbow?: MocapPointInput | null;
|
||||
leftWrist?: MocapPointInput | null;
|
||||
rightWrist?: MocapPointInput | null;
|
||||
};
|
||||
|
||||
export type MocapInputCommand = {
|
||||
@@ -289,6 +291,14 @@ function normalizeBodyJointName(name: unknown) {
|
||||
return 'rightElbow' as const;
|
||||
}
|
||||
|
||||
if (normalized === 'left_wrist' || normalized === 'leftwrist') {
|
||||
return 'leftWrist' as const;
|
||||
}
|
||||
|
||||
if (normalized === 'right_wrist' || normalized === 'rightwrist') {
|
||||
return 'rightWrist' as const;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user