GenarrativeAI/Genarrative
5
0
Fork 0
Code Issues Pull Requests 1 Actions 1 Packages Projects Releases Wiki Activity

master #14

Merged
kdletters merged 226 commits from master into release 2026-05-13 13:23:09 +08:00
Conversation 0 Commits 226 Files Changed 1950 +5894 -341
46 changed files with 5894 additions and 341 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit d0a9348e72 - Show all commits

16
.hermes/shared-memory/decision-log.md
View File

@@ -16,6 +16,22 @@
---
## 2026-05-10 运行态输入设备抽象层全项目通用化
- 背景：拼图运行态接入 mocap 后，鼠标/触控和 mocap 各自维护输入逻辑会导致合并大块、拖拽语义和取消会话行为不一致；后续其他玩法也需要复用体感、摇杆、键盘等设备输入。
- 决策：前端运行态输入统一通过 `src/services/input-devices/` 承接，设备适配层只输出 `press / move / release / tap / drop` 等通用语义和通用坐标；玩法组件自己解释目标对象、落点和业务动作，输入层不得写拼图等玩法专用规则。
- 影响范围：拼图运行态鼠标/触控/mocap 输入、后续运行态设备接入、运行态输入技术文档与相关前端回归测试。
- 验证方式：执行 `npm run test -- src\services\input-devices\runtimeDragInputController.test.ts`、`npm run test -- src\components\puzzle-runtime\PuzzleRuntimeShell.test.tsx`、`npm run typecheck` 和编码检查。
- 关联文档：`docs/technical/RUNTIME_INPUT_DEVICE_ABSTRACTION_2026-05-10.md`、`docs/technical/PUZZLE_RUNTIME_FRONTEND_LOGIC_REHOME_2026-05-02.md`。
## 2026-05-10 儿童动作热身关直接消费 mocap 手势流
- 背景：儿童动作 Demo 的挥手、左右手挥动和跳跃阶段不能只依赖键鼠调试输入，否则真实硬件接入后会出现“能看到画面但动作不推进”的卡点。
- 决策：热身关在 gesture 阶段直接接入 `useMocapInput`，通过本地 mocap WebSocket `/stream` 消费 `actions/action/gesture/gestures/event/name/type` 动作名，以及 `hands[]`、`leftHand/rightHand`、`left_hand/right_hand` 手部坐标；`wave_greeting`、`wave_left_hand`、`wave_right_hand` 和 `jump_once` 都可以由 mocap 包推进，同时保留键鼠作为本地调试兜底。
- 影响范围：`src/services/useMocapInput.ts`、`src/components/child-motion-demo/ChildMotionWarmupDemo.tsx`、对应单测与热身关技术文档。
- 验证方式：执行 `npx vitest run src/components/child-motion-demo/ChildMotionWarmupDemo.test.tsx src/components/child-motion-demo/childMotionWarmupModel.test.ts src/services/child-motion-demo/childMotionDebugInput.test.ts src/routing/appRoutes.test.ts`、`npx eslint ...`、`npm run typecheck`、`npm run check:encoding`，并确认 `http://127.0.0.1:3000/child-motion-demo` 与 `http://127.0.0.1:3100/healthz` 可访问。
- 关联文档：`docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md`。
## 2026-05-09 GPT-image-2 图片生成统一迁移到 VectorEngine
- 背景：仓库内 RPG、拼图、方洞和本地模板脚本的 GPT-image-2 生图此前依赖 APIMart 图片网关；团队要求参考 VectorEngine Apifox `api-448710071`，后续不再使用 APIMart 执行 GPT-image-2 图片生成。

20
.hermes/shared-memory/pitfalls.md
View File

@@ -43,6 +43,14 @@
- 验证：提交前检查 `git diff -- .hermes`，确认没有密钥、会话记录或个人路径敏感信息。
- 关联：`.hermes/README.md`。
## 儿童动作 Demo 挥手阶段不推进先查 mocap 消费链路
- 现象：`/child-motion-demo` 能打开摄像头画面，但到“打个招呼”或左右手挥动阶段时，真实硬件动作无法检测通过，只能用鼠标拖拽或键盘调试继续。
- 原因：摄像头视频流只是舞台背景；如果热身关没有消费 `useMocapInput` 的动作名和手部坐标，就不会把硬件动作转换成热身状态机完成事件。
- 处理：确认 `src/components/child-motion-demo/ChildMotionWarmupDemo.tsx` 在 `step.kind === 'gesture'` 时启用 `useMocapInput`；确认 `src/services/useMocapInput.ts` 能解析 `/stream` 包里的 `actions/action/gesture/gestures/event/name/type`、`hands[]`、`leftHand/rightHand`、`left_hand/right_hand`、左右手标记和 `open_palm/grab` 状态。热身关应由 mocap 推进，键鼠只作为本地调试兜底。
- 验证：运行 `npx vitest run src\components\child-motion-demo\ChildMotionWarmupDemo.test.tsx`，并在本地硬件服务启动后进入 `/child-motion-demo` 实测招手、左右手挥动和跳跃阶段。
- 关联：`src/services/useMocapInput.ts`、`src/components/child-motion-demo/ChildMotionWarmupDemo.tsx`、`docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md`。
## GPT-image-2 不再读 APIMart 图片配置
- 现象：配置了 `APIMART_BASE_URL` / `APIMART_API_KEY` 后，RPG、拼图或方洞的 GPT-image-2 生图仍返回缺配置，或请求体里还出现 `official_fallback` / `image_urls`。
@@ -119,8 +127,8 @@
- 现象：本地 `npm run dev` 因 `3101` 已占用、重复发布 SpacetimeDB wasm 编译太慢，或只想检查 `spacetime-module` 语法而被完整联调链路拖慢。
- 原因：`npm run dev` 默认同时启动 SpacetimeDB standalone、发布 `server-rs/crates/spacetime-module`、启动 Rust `api-server`、主站 Vite 与后台 Vite；并非每个阶段都需要完整重启和重新发布。
- 处理：`npm run dev` 启动后会把实际 SpacetimeDB URL 记录到 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`。下次启动即使没有传 `--skip-spacetime`，脚本也会先检查 `spacetime.pid` 对应进程和该 URL 是否在线；在线则直接复用现有宿主。确认需要新启动 SpacetimeDB 时，脚本先检测 `3101`，被占用则输出占用进程并选择最近可用端口，保证 publish 与 `api-server` 都连接同一个实际 SpacetimeDB URL。`api-server` 启动前也会检测 `8082` 并选择最近可用端口。Windows / Git Bash 下不要用 `tr/head/xargs` 管道读取 `spacetime.pid` 或 URL 记录，脚本应使用 Node 读取并短重试，避免 `tr: read error: Device or resource busy`；未修改 `spacetime-module` 时使用 `npm run dev -- --skip-publish`；只查模块语法时执行 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml`。
- 验证：`--skip-spacetime` 后脚本复用现有 `http://127.0.0.1:3101`；`3101` 或 `8082` 被其他进程占用时，脚本输出占用进程并使用最近可用端口；`--skip-publish` 后不再进入 publish 阶段；`cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml` 能完成 Rust 语法和类型检查。
- 处理：`npm run dev` 启动后会把实际 SpacetimeDB URL 记录到 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`。下次启动即使没有传 `--skip-spacetime`，脚本也会先检查 `spacetime.pid` 对应进程和该 URL 是否在线；在线则直接复用现有宿主。确认需要新启动 SpacetimeDB 时，脚本先检测 `3101`，被占用则输出占用进程并选择最近可用端口，保证 publish 与 `api-server` 都连接同一个实际 SpacetimeDB URL。`api-server` 启动前也会检测 `8082` 并选择最近可用端口。Windows / Git Bash 下不要用 `tr/head/xargs` 管道读取 `spacetime.pid` 或 URL 记录，脚本应使用 Node 读取并短重试，避免 `tr: read error: Device or resource busy`；未修改 `spacetime-module` 时使用 `npm run dev -- --skip-publish`；只查模块语法时执行 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml`。`npm run dev` 会在启动前检查 SpacetimeDB、api-server、主站 Vite、后台 Vite 端口，不可用时自动寻找后续可用端口，并把实际端口传给 publish、后端环境变量和前端代理目标。
- 验证：`--skip-spacetime` 后脚本复用现有 `http://127.0.0.1:3101`；`3101` 或 `8082` 被其他进程占用时，脚本输出占用进程并使用最近可用端口；`--skip-publish` 后不再进入 publish 阶段；`cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml` 能完成 Rust 语法和类型检查。端口漂移时控制台会打印 `[dev:ports] ... 不可用，改用 ...`，后续 `[dev:rust] web/admin web/rust api/spacetime` 地址应与实际端口一致。
- 关联：`docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md`、`scripts/dev-rust-stack.sh`。
## 本地 SpacetimeDB publish 401 可清本地库重发
@@ -191,6 +199,14 @@
- 验证：`http://127.0.0.1:3000/api/auth/login-options` 返回至少 `{"availableLoginMethods":["phone","password"]}` 后，登录弹窗会恢复短信登录页签和“获取验证码”按钮。
- 关联：`scripts/api-server-dev.mjs`、`scripts/api-server-maincloud.mjs`、`scripts/dev-rust-stack.sh`、`scripts/dev-web-rust.mjs`、`docs/technical/AUTH_LOGIN_OPTIONS_DESIGN_2026-04-21.md`。
## 本地短信收不到验证码先查 provider
- 现象：登录弹窗可以进入短信页签，但点击“获取验证码”后，手机没有收到短信。
- 原因：本地 `.env.local` 里如果是 `SMS_AUTH_PROVIDER="mock"`，后端不会发真实短信，只会返回固定 mock 验证码；另外 `npm run api-server` 过去曾让 `.env` 覆盖 `.env.local`，导致本地真实短信配置被错误压回默认值。
- 处理：真实短信联调时把 `.env.local` 的 `SMS_AUTH_PROVIDER` 显式设为 `aliyun`，然后重启 `api-server`；如果只想验证 UI 和账号链路，则保留 `mock` 并使用 `SMS_AUTH_MOCK_VERIFY_CODE`。
- 验证：`GET /api/auth/login-options` 返回 `["phone","password"]`，`api-server` 日志里 `provider=aliyun` 才说明真实短信链路已生效。
- 关联：`scripts/api-server-dev.mjs`、`docs/technical/AUTH_LOGIN_OPTIONS_DESIGN_2026-04-21.md`、`docs/technical/PHONE_SMS_REAL_PROVIDER_MANUAL_VERIFICATION_RUNBOOK_2026-04-23.md`。
## 手机验证码登录 500 先查短信 provider 语义
- 现象：登录弹窗手机号验证码登录失败，浏览器看到 `POST /api/auth/phone/login 500`，后端日志里同时出现阿里云短信 `UNKNOWN`、`biz.FREQUENCY` 或 `check frequency failed`。

127
.hermes/skills/genarrative-dev-stack-port-routing/SKILL.md Normal file
View File

@@ -0,0 +1,127 @@
---
name: genarrative-dev-stack-port-routing
short_description: 修改 Genarrative 本地 dev 启动端口、代理目标、端口冲突处理时使用。
description: 在 Genarrative 中修改 npm run dev / npm run dev:rust / npm run dev:web 的本地启动端口、端口可用性探测、端口漂移、SpacetimeDB publish server、api-server 环境变量、Vite 代理目标和后台 admin-web 启动串联时使用。
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
hermes:
tags: [Genarrative, dev-stack, 端口探测, Vite, api-server, SpacetimeDB, npm-run-dev]
related_skills: [genarrative-admin-backoffice]
---
# Genarrative 本地 dev 启动端口与代理目标串联流程
用于维护 Genarrative 本地开发栈启动脚本，重点覆盖 `npm run dev` / `npm run dev:rust` / `npm run dev:web` 的端口检查、端口漂移和后续流程目标传递。
## 适用场景
- 修改 `scripts/dev-rust-stack.sh`、`scripts/dev-web-rust.mjs`、`scripts/dev-stack-port-utils.mjs`。
- 处理 `3000`、`3101`、`3102`、`8082` 等端口被占用导致本地开发栈启动失败。
- 排查 Vite 代理仍指向旧 api-server 端口、前端打开了旧 dev server、后台代理错配。
- 调整 SpacetimeDB standalone、publish、Rust `api-server`、主站 Vite、后台 Vite 的启动顺序。
- 修改本地联调文档或 `.hermes/shared-memory/pitfalls.md` 中的 dev 启动口径。
## 当前端口职责
默认优先端口：
1. 主站 Vite：`3000`，对浏览器通常展示为 `http://127.0.0.1:<web-port>/`。
2. Rust `api-server`：`8082`，健康检查为 `http://127.0.0.1:<api-port>/healthz`。
3. SpacetimeDB standalone：`3101`，健康检查为 `http://127.0.0.1:<spacetime-port>/v1/ping`。
4. 后台 Vite：`3102`，后台地址为 `http://127.0.0.1:<admin-web-port>/admin/`。
端口不可用时，脚本会从优先端口开始向后寻找可用端口。后续流程必须以解析后的实际端口为准，不能继续使用默认端口。
## 实现入口
- `package.json`
- `dev` 和 `dev:rust`：执行 `node scripts/run-bash-script.mjs scripts/dev-rust-stack.sh`。
- `dev:web`：执行 `node scripts/dev-web-rust.mjs`。
- `scripts/dev-stack-port-utils.mjs`
- `isPortAvailable(...)`：探测端口是否可监听。
- `findAvailablePort(...)`：从优先端口向后寻找可用端口，`0` 表示申请临时端口。
- `resolveDevStackPorts(...)`：一次性解析 SpacetimeDB、api-server、主站 Vite、后台 Vite 端口，并避免本次解析结果互相冲突。
- CLI 模式：`node scripts/dev-stack-port-utils.mjs resolve-dev-stack spacetime:127.0.0.1:3101 api:127.0.0.1:8082 web:0.0.0.0:3000 adminWeb:127.0.0.1:3102`。
- `scripts/dev-rust-stack.sh`
- 解析 CLI 参数后，先计算 `API_TARGET_HOST` 与 `ADMIN_WEB_TARGET_HOST`。
- 调用 `resolve_dev_stack_ports` 覆盖 `SPACETIME_PORT`、`API_PORT`、`WEB_PORT`、`ADMIN_WEB_PORT`。
- 再构造 `SPACETIME_SERVER` 和 `RUST_SERVER_TARGET`。
- `scripts/dev-web-rust.mjs`
- 单独启动主站前端时，也先用 `findAvailablePort` 检查 `WEB_PORT` / 默认 `3000`。
## 必须保持的传递链路
`npm run dev` / `npm run dev:rust` 中端口解析后，必须同步到以下位置：
1. SpacetimeDB 启动：`spacetime start --listen-addr "${SPACETIME_HOST}:${SPACETIME_PORT}"`。
2. SpacetimeDB 发布：`spacetime publish ... --server "${SPACETIME_SERVER}"`。
3. Rust api-server：`GENARRATIVE_API_HOST`、`GENARRATIVE_API_PORT`、`GENARRATIVE_SPACETIME_SERVER_URL`、`GENARRATIVE_SPACETIME_DATABASE`。
4. api-server 健康检查：`wait_for_api_server "${RUST_SERVER_TARGET}/healthz" ...`。
5. 主站 Vite：`RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET`、`ADMIN_WEB_TARGET`、`ADMIN_WEB_PORT`、`--port=${WEB_PORT}`、`--host=${WEB_HOST}`。
6. 后台 Vite：`ADMIN_API_TARGET`、`GENARRATIVE_API_TARGET`、`GENARRATIVE_API_PORT`、`--port=${ADMIN_WEB_PORT}`。
7. 控制台日志：`[dev:ports]` 和 `[dev:rust] web/admin web/rust api/spacetime` 必须显示最终实际地址。
如果只改了其中一段，通常会出现：浏览器打开的前端可用，但 `/api/*` 代理到旧端口；后台页面可用但后台 API 失败；SpacetimeDB 启动在新端口但 publish 仍发往旧端口。
## 修改流程
1. 先读当前脚本和文档：
- `scripts/dev-stack-port-utils.mjs`
- `scripts/dev-rust-stack.sh`
- `scripts/dev-web-rust.mjs`
- `docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md`
- `.hermes/shared-memory/pitfalls.md`
2. 优先改公共端口工具，不要把端口探测逻辑复制到多个脚本。
3. 对 Bash 脚本只做局部补丁，避免整文件重写导致中文注释或换行大面积变化。
4. 修改 `dev-rust-stack.sh` 时确认变量顺序：
- 先有 `REPO_ROOT`。
- 再计算 `API_TARGET_HOST` / `ADMIN_WEB_TARGET_HOST`。
- 再调用端口解析工具。
- 再构造 `SPACETIME_SERVER` / `RUST_SERVER_TARGET`。
5. 修改 `dev:web` 时不要自动改后端目标策略；`dev:web` 只负责主站 Vite 端口可用性与已有后端目标选择。
6. 同步更新技术文档和团队共享记忆。
## 测试与验证
最小验证：
```bash
bash -n scripts/dev-rust-stack.sh
npm run test -- scripts/dev-stack-port-utils.test.ts
npm run check:encoding
node scripts/dev-stack-port-utils.mjs resolve-dev-stack spacetime:127.0.0.1:0 api:127.0.0.1:0 web:0.0.0.0:0 adminWeb:127.0.0.1:0
```
端口冲突回归测试建议：
1. 用测试或临时 Node server 占用某个优先端口。
2. 调用 `findAvailablePort`，断言结果大于被占用端口。
3. 调用 `resolveDevStackPorts`，断言四个结果互不相同。
4. 如果实际启动完整栈，观察控制台：
- `[dev:ports] ... 不可用，改用 ...`
- `[dev:rust] rust api: http://...:<actual-api-port>`
- `[dev:rust] spacetime: http://...:<actual-spacetime-port>`
- 主站和后台 Vite 启动端口与日志一致。
完整启动属于长驻进程。需要 smoke 时用 background 方式启动，并另开命令检查 `/healthz`、`/v1/ping` 和页面端口；不要等待 `npm run dev` 自然退出。
## 常见坑
1. **只让 Vite 自己漂移端口。** 这样终端可能出现可访问前端，但脚本和文档仍认为是 `3000`，后台目标或日志会错。
2. **只改 SpacetimeDB start，不改 publish。** standalone 可能监听新端口，但 publish 仍连旧 `3101`。
3. **只改 `GENARRATIVE_API_PORT`，不改 `RUST_SERVER_TARGET`。** api-server 已在新端口监听，但 Vite 代理仍打旧端口。
4. **使用 `0.0.0.0` 作为浏览器访问地址。** 监听可以是 `0.0.0.0`，展示给用户和健康检查通常用 `127.0.0.1`。
5. **端口探测和实际启动之间存在竞态。** 已经探测可用的端口仍可能被外部进程抢占；SpacetimeDB 启动后仍要解析实际监听地址，api-server 和 Vite 失败时要打印清晰日志。
6. **运行全仓库 lint 误判。** 当前仓库可能有既有 lint 问题。验证本功能时优先运行定向测试、Bash 语法检查、编码检查，并在最终说明中区分既有 lint 失败与本次改动。
## 验收清单
- [ ] 端口工具有测试覆盖端口被占用和多端口互斥解析。
- [ ] `dev-rust-stack.sh` 通过 `bash -n`。
- [ ] `npm run dev` / `npm run dev:rust` 的 SpacetimeDB、publish、api-server、主站 Vite、后台 Vite 都使用实际端口。
- [ ] `npm run dev:web` 在主站端口不可用时能切换到可用端口。
- [ ] 文档同步更新 `docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md`。
- [ ] 长期踩坑同步更新 `.hermes/shared-memory/pitfalls.md`。
- [ ] 修改中文文件后运行 `npm run check:encoding`。

1
AGENTS.md
View File

@@ -46,6 +46,7 @@
- [$spacetimedb-concepts](.codex\\skills\\spacetimedb-concepts\\SKILL.md)
- [$spacetimedb-typescript](.codex\\skills\\spacetimedb-typescript\\SKILL.md)
- 涉及 `spacetime` CLI、发布、绑定生成、本地联调时，按 `spacetimedb-cli` 执行。
- 涉及 `npm run dev` / `npm run dev:rust` / `npm run dev:web` 的端口探测、端口漂移、SpacetimeDB publish server、api-server 环境变量、Vite 代理目标或后台 dev 端口时，按 [`.hermes/skills/genarrative-dev-stack-port-routing/SKILL.md`](.hermes/skills/genarrative-dev-stack-port-routing/SKILL.md) 执行。
- 涉及 `crates/spacetime-module` 的表、reducer、view、Rust API 使用时，按 `spacetimedb-rust` 与 `spacetimedb-concepts` 执行。
- 涉及前端或 Node 侧的 SpacetimeDB TypeScript SDK、订阅、绑定使用时，按 `spacetimedb-typescript` 与 `spacetimedb-concepts` 执行。
- 若仓库内旧实现或旧文档与这些 skill 冲突，先修正文档和方案，再继续编码。

42
apps/admin-web/src/config/trackingEventDefinitions.test.ts Normal file
View File

@@ -0,0 +1,42 @@
import {describe, expect, test} from 'vitest';
import {
adminProfileTaskTrackingEventDefinitions,
adminTrackingEventDefinitions,
filterAdminProfileTaskTrackingEventDefinitions,
filterAdminTrackingEventDefinitions,
findAdminTrackingEventDefinition,
} from './trackingEventDefinitions';
describe('admin tracking event definitions', () => {
test('后台埋点筛选候选包含后端通用埋点清单', () => {
const keys = adminTrackingEventDefinitions.map((definition) => definition.key);
expect(keys.length).toBeGreaterThan(40);
expect(keys).toContain('daily_login');
expect(keys).toContain('auth_login_options_view');
expect(keys).toContain('task_center_view');
expect(keys).toContain('asset_upload_ticket_create');
expect(keys).toContain('creative_agent_route_success');
expect(keys).toContain('work_play_start');
});
test('任务配置候选只开放适合个人任务的事件', () => {
expect(adminProfileTaskTrackingEventDefinitions.map(({key}) => key)).toEqual([
'daily_login',
]);
expect(filterAdminProfileTaskTrackingEventDefinitions('').map(({key}) => key)).toEqual([
'daily_login',
]);
});
test('后台埋点筛选支持按中文名称和 key 搜索', () => {
expect(filterAdminTrackingEventDefinitions('上传票据').map(({key}) => key)).toEqual([
'asset_upload_ticket_create',
]);
expect(filterAdminTrackingEventDefinitions('work_play').map(({key}) => key)).toEqual([
'work_play_start',
]);
expect(findAdminTrackingEventDefinition(' daily_login ')?.title).toBe('每日登录');
});
});

403
apps/admin-web/src/config/trackingEventDefinitions.ts
View File

@@ -5,17 +5,400 @@ export interface AdminTrackingEventDefinition {
title: string;
scopeKind: TrackingScopeKind;
remark: string;
taskConfigEligible?: boolean;
}
export const adminTrackingEventDefinitions: AdminTrackingEventDefinition[] = [
{
key: 'auth_login_options_view',
title: '登录方式查看',
scopeKind: 'site',
remark: '读取当前可用登录方式时记录，用于观察登录入口曝光。',
},
{
key: 'auth_phone_code_send',
title: '发送手机验证码',
scopeKind: 'site',
remark: '提交手机验证码发送请求成功后记录。',
},
{
key: 'daily_login',
title: '每日登录',
scopeKind: 'user',
remark: '用户打开任务中心时由后端幂等记录，用于每日登录任务进度校验。',
remark: '认证成功或 refresh 续期后由后端幂等记录，用于每日登录任务进度校验。',
taskConfigEligible: true,
},
{
key: 'auth_phone_login_success',
title: '手机号登录成功',
scopeKind: 'user',
remark: '手机号验证码登录成功后记录。',
},
{
key: 'auth_me_view',
title: '当前账号查看',
scopeKind: 'user',
remark: '读取当前登录账号信息成功后记录。',
},
{
key: 'auth_sessions_view',
title: '登录会话查看',
scopeKind: 'user',
remark: '读取当前账号登录会话列表成功后记录。',
},
{
key: 'auth_refresh_success',
title: '登录续期成功',
scopeKind: 'site',
remark: 'refresh cookie 续期成功后记录。',
},
{
key: 'auth_logout',
title: '退出登录',
scopeKind: 'user',
remark: '退出当前登录会话成功后记录。',
},
{
key: 'auth_logout_all',
title: '退出全部会话',
scopeKind: 'user',
remark: '退出全部登录会话成功后记录。',
},
{
key: 'auth_wechat_bind_phone_success',
title: '微信绑定手机成功',
scopeKind: 'user',
remark: '微信账号绑定手机号成功后记录。',
},
{
key: 'profile_identity_update',
title: '资料更新',
scopeKind: 'user',
remark: '用户资料更新成功后记录。',
},
{
key: 'profile_dashboard_view',
title: '个人看板查看',
scopeKind: 'user',
remark: '读取个人看板成功后记录。',
},
{
key: 'wallet_ledger_view',
title: '钱包流水查看',
scopeKind: 'user',
remark: '读取光点钱包流水成功后记录。',
},
{
key: 'recharge_center_view',
title: '充值中心查看',
scopeKind: 'user',
remark: '读取充值中心信息成功后记录。',
},
{
key: 'recharge_order_create',
title: '充值订单创建',
scopeKind: 'user',
remark: '创建充值订单成功后记录。',
},
{
key: 'feedback_submit',
title: '反馈提交',
scopeKind: 'user',
remark: '资料页反馈提交成功后记录。',
},
{
key: 'invite_center_view',
title: '邀请中心查看',
scopeKind: 'user',
remark: '读取邀请中心成功后记录。',
},
{
key: 'referral_invite_code_redeem',
title: '邀请码绑定',
scopeKind: 'user',
remark: '绑定推荐邀请码成功后记录。',
},
{
key: 'redeem_code_submit',
title: '兑换码提交',
scopeKind: 'user',
remark: '提交运营兑换码成功后记录。',
},
{
key: 'task_center_view',
title: '任务中心查看',
scopeKind: 'user',
remark: '读取个人任务中心成功后记录。',
},
{
key: 'task_reward_claim',
title: '任务奖励领取',
scopeKind: 'user',
remark: '领取个人任务奖励成功后记录。',
},
{
key: 'save_archive_list_view',
title: '存档列表查看',
scopeKind: 'user',
remark: '读取个人存档列表成功后记录。',
},
{
key: 'save_archive_detail_view',
title: '存档详情查看',
scopeKind: 'user',
remark: '读取个人存档详情成功后记录。',
},
{
key: 'browse_history_view',
title: '浏览历史查看',
scopeKind: 'user',
remark: '读取浏览历史成功后记录。',
},
{
key: 'browse_history_record',
title: '浏览历史写入',
scopeKind: 'user',
remark: '记录浏览历史成功后记录。',
},
{
key: 'browse_history_clear',
title: '浏览历史清空',
scopeKind: 'user',
remark: '清空浏览历史成功后记录。',
},
{
key: 'play_stats_view',
title: '游玩统计查看',
scopeKind: 'user',
remark: '读取个人游玩统计成功后记录。',
},
{
key: 'profile_analytics_metric_view',
title: '个人指标查看',
scopeKind: 'user',
remark: '读取个人埋点指标成功后记录。',
},
{
key: 'ai_task_create',
title: 'AI 任务创建',
scopeKind: 'user',
remark: '创建 AI 任务成功后记录。',
},
{
key: 'ai_task_start',
title: 'AI 任务启动',
scopeKind: 'user',
remark: '启动 AI 任务成功后记录。',
},
{
key: 'ai_task_stage_start',
title: 'AI 阶段启动',
scopeKind: 'user',
remark: '启动 AI 任务阶段成功后记录。',
},
{
key: 'ai_task_chunk_append',
title: 'AI 分片追加',
scopeKind: 'user',
remark: '追加 AI 文本分片成功后记录。',
},
{
key: 'ai_task_stage_complete',
title: 'AI 阶段完成',
scopeKind: 'user',
remark: '完成 AI 任务阶段成功后记录。',
},
{
key: 'ai_task_reference_attach',
title: 'AI 结果引用绑定',
scopeKind: 'user',
remark: '绑定 AI 结果引用成功后记录。',
},
{
key: 'ai_task_complete',
title: 'AI 任务完成',
scopeKind: 'user',
remark: '完成 AI 任务成功后记录。',
},
{
key: 'ai_task_fail',
title: 'AI 任务失败标记',
scopeKind: 'user',
remark: '标记 AI 任务失败成功后记录。',
},
{
key: 'ai_task_cancel',
title: 'AI 任务取消',
scopeKind: 'user',
remark: '取消 AI 任务成功后记录。',
},
{
key: 'asset_upload_ticket_create',
title: '资产上传票据创建',
scopeKind: 'user',
remark: '创建直传票据成功后记录，metadata 包含低敏资产定位字段。',
},
{
key: 'asset_sts_credentials_create',
title: '资产 STS 凭证创建',
scopeKind: 'user',
remark: '创建临时上传凭证成功后记录。',
},
{
key: 'asset_upload_confirm',
title: '资产上传确认',
scopeKind: 'user',
remark: '确认 OSS 对象为平台资产成功后记录。',
},
{
key: 'asset_bind',
title: '资产绑定',
scopeKind: 'user',
remark: '把平台资产绑定到业务实体成功后记录。',
},
{
key: 'asset_character_visual_generate',
title: '角色形象生成',
scopeKind: 'user',
remark: '角色主图生成请求成功后记录。',
},
{
key: 'asset_character_visual_publish',
title: '角色形象发布',
scopeKind: 'user',
remark: '角色主图发布成功后记录。',
},
{
key: 'asset_character_animation_generate',
title: '角色动画生成',
scopeKind: 'user',
remark: '角色动画生成请求成功后记录。',
},
{
key: 'asset_character_animation_publish',
title: '角色动画发布',
scopeKind: 'user',
remark: '角色动画发布成功后记录。',
},
{
key: 'asset_character_animation_import',
title: '角色动画视频导入',
scopeKind: 'user',
remark: '角色动画导入视频成功后记录。',
},
{
key: 'asset_character_workflow_cache_save',
title: '角色工作流缓存保存',
scopeKind: 'user',
remark: '保存角色工作流缓存成功后记录。',
},
{
key: 'asset_history_view',
title: '资产历史查看',
scopeKind: 'user',
remark: '读取资产历史成功后记录。',
},
{
key: 'llm_request',
title: 'LLM 请求',
scopeKind: 'user',
remark: '调用后端 LLM 门面成功后记录。',
},
{
key: 'speech_config_view',
title: '语音配置查看',
scopeKind: 'user',
remark: '读取语音配置成功后记录。',
},
{
key: 'asr_stream_start',
title: 'ASR 流启动',
scopeKind: 'user',
remark: '启动语音识别流成功后记录。',
},
{
key: 'tts_bidirection_start',
title: 'TTS 双向流启动',
scopeKind: 'user',
remark: '启动双向语音合成流成功后记录。',
},
{
key: 'tts_sse_start',
title: 'TTS SSE 启动',
scopeKind: 'user',
remark: '启动 SSE 语音合成成功后记录。',
},
{
key: 'runtime_settings_view',
title: '运行设置查看',
scopeKind: 'user',
remark: '读取运行设置成功后记录。',
},
{
key: 'runtime_settings_update',
title: '运行设置更新',
scopeKind: 'user',
remark: '更新运行设置成功后记录。',
},
{
key: 'runtime_snapshot_view',
title: '运行快照查看',
scopeKind: 'user',
remark: '读取运行快照成功后记录。',
},
{
key: 'runtime_snapshot_save',
title: '运行快照保存',
scopeKind: 'user',
remark: '保存运行快照成功后记录。',
},
{
key: 'runtime_snapshot_delete',
title: '运行快照删除',
scopeKind: 'user',
remark: '删除运行快照成功后记录。',
},
{
key: 'puzzle_route_success',
title: '拼图路由成功',
scopeKind: 'user',
remark: '拼图运行或创作接口成功响应后兜底记录；GET 入口可能按 site 统计。',
},
{
key: 'match3d_route_success',
title: '抓大鹅路由成功',
scopeKind: 'user',
remark: '抓大鹅创作或运行接口成功响应后兜底记录；GET 入口可能按 site 统计。',
},
{
key: 'square_hole_route_success',
title: '方洞路由成功',
scopeKind: 'user',
remark: '方洞创作或运行接口成功响应后兜底记录；GET 入口可能按 site 统计。',
},
{
key: 'custom_world_route_success',
title: '自定义世界路由成功',
scopeKind: 'user',
remark: '自定义世界运行接口成功响应后兜底记录；GET 入口可能按 site 统计。',
},
{
key: 'creative_agent_route_success',
title: '创意 Agent 路由成功',
scopeKind: 'user',
remark: '创意 Agent 接口成功响应后兜底记录；GET 入口可能按 site 统计。',
},
{
key: 'work_play_start',
title: '作品开始游玩',
scopeKind: 'work',
remark: '拼图、抓大鹅、方洞、自定义世界、大鱼吃小鱼、Visual Novel 正式开始游玩时记录。',
},
];
export const adminProfileTaskTrackingEventDefinitions =
adminTrackingEventDefinitions.filter((definition) => definition.taskConfigEligible);
export function findAdminTrackingEventDefinition(eventKey: string) {
const normalizedEventKey = eventKey.trim();
return (
@@ -26,12 +409,26 @@ export function findAdminTrackingEventDefinition(eventKey: string) {
}
export function filterAdminTrackingEventDefinitions(query: string) {
return filterTrackingEventDefinitions(adminTrackingEventDefinitions, query);
}
export function filterAdminProfileTaskTrackingEventDefinitions(query: string) {
return filterTrackingEventDefinitions(
adminProfileTaskTrackingEventDefinitions,
query,
);
}
function filterTrackingEventDefinitions(
definitions: AdminTrackingEventDefinition[],
query: string,
) {
const normalizedQuery = query.trim().toLowerCase();
if (!normalizedQuery) {
return adminTrackingEventDefinitions;
return definitions;
}
return adminTrackingEventDefinitions.filter((definition) => {
return definitions.filter((definition) => {
const haystack = [
definition.key,
definition.title,

4
apps/admin-web/src/pages/AdminTaskConfigPage.tsx
View File

@@ -13,7 +13,7 @@ import type {
} from '../api/adminApiTypes';
import {useAdminWriteConfirm} from '../components/useAdminWriteConfirm';
import {
filterAdminTrackingEventDefinitions,
filterAdminProfileTaskTrackingEventDefinitions,
findAdminTrackingEventDefinition,
} from '../config/trackingEventDefinitions';
import {handlePageError} from './pageUtils';
@@ -68,7 +68,7 @@ export function AdminTaskConfigPage({
[eventKey],
);
const filteredEventDefinitions = useMemo(
() => filterAdminTrackingEventDefinitions(eventKeySearch),
() => filterAdminProfileTaskTrackingEventDefinitions(eventKeySearch),
[eventKeySearch],
);

32
docs/design/CHILD_MOTION_EDUTAINMENT_DISCOVER_ENTRY_2026-05-09.md
View File

@@ -16,6 +16,8 @@
后续生产的该内容线模板和游戏关卡，都放置在“寓教于乐”独立标签下。
该内容线当前只覆盖儿童动作识别 Demo 内容。后续创作环节需要继续对该板块内容做区分和独立管理，不把普通公开作品仅凭近似教育题材自动归入本板块。
## 2. 展示边界
寓教于乐内容不直接展示在以下位置：
@@ -29,6 +31,8 @@
寓教于乐内容只在“发现 / 寓教于乐”标签下展示。
“寓教于乐”标签在发现页频道列表中放在最后，桌面端和移动端都显示。移动端访问该内容线的动作识别 Demo 时，需要提示横屏体验。
## 3. 开关规则
该入口需要支持灵活开关。
@@ -43,13 +47,14 @@
1. 发现页隐藏“寓教于乐”标签；
2. 隐藏“寓教于乐”标签下内容；
3. 该内容线内容不进入推荐、今日、分类、排行和搜索结果。
3. 该内容线内容不进入推荐、今日、分类、排行和搜索结果；
4. 该内容线内容完全不可见，公开作品搜索、作品号搜索直达、公开详情深链、浏览历史入口等平台公开入口都不能打开该内容。
## 4. 内容识别规则
临时阶段使用作品标签识别寓教于乐内容。
当公开作品标签中包含：
当公开作品标签中存在一个精确等于以下文本的标签：
```text
寓教于乐
@@ -57,6 +62,10 @@
则该作品归入寓教于乐内容线。
识别规则为精确匹配，不做包含匹配，不兼容空格、大小写变体或同义标签，例如“教育”“儿童教育”“动作教育”都不视为寓教于乐内容。
关闭开关时，即使作品具备精确的“寓教于乐”标签，也不允许通过任何平台公开展示入口或搜索入口访问。
## 5. 技术落地边界
本次只做前端入口和前端展示过滤，不新增后端接口。
@@ -87,3 +96,22 @@ no
3. 带有“寓教于乐”标签的公开作品不进入推荐页。
4. 带有“寓教于乐”标签的公开作品不进入发现页推荐、今日、分类、排行和搜索结果。
5. 带有“寓教于乐”标签的公开作品只在“发现 / 寓教于乐”标签下展示。
6. “寓教于乐”标签位于发现页频道列表最后，桌面端和移动端均可见。
7. 开关关闭后，带有“寓教于乐”标签的公开作品不可通过作品号搜索、公开详情深链或浏览历史入口打开。
8. 标签识别只接受精确等于“寓教于乐”的作品标签，近似标签不归入该内容线。
## 7. 待补充事项
“寓教于乐”标签下暂无内容时的空状态文案待定。落地时可先复用平台现有空状态组件，但不新增功能说明类长文案。
## 8. 第 1-2 项工程落地状态
第 1 项“发现页入口与过滤”和第 2 项“搜索 / 深链 / 历史入口拦截”已进入前端落地阶段，当前实现口径如下：
1. 入口开关由 `VITE_ENABLE_EDUTAINMENT_ENTRY` 控制，默认开启，显式配置 `false`、`0`、`off`、`no` 时关闭。
2. 内容识别集中在 `src/components/platform-entry/platformEdutainmentVisibility.ts`，只读取公开作品原始 `themeTags`，且只接受精确等于“寓教于乐”的标签。
3. `src/components/rpg-entry/RpgEntryHomeView.tsx` 已在发现页频道末尾追加“寓教于乐”频道，并将该类作品从推荐、今日、分类、排行、搜索、本地搜索兜底和桌面推荐模块中过滤。
4. `src/components/platform-entry/PlatformEntryFlowShellImpl.tsx` 已复用同一过滤 helper，避免推荐运行态自动启动寓教于乐作品，并在公开详情、作品号直达和公开详情深链等公开入口保留不可见保护。
5. 浏览历史入口会优先按当前公开作品集合匹配作品标签；匹配到“寓教于乐”作品且开关关闭时不再展示历史入口。
6. `/child-motion-demo` 本地动作 Demo 直达路由也复用同一开关；开关关闭时不匹配独立 Demo 应用，回落到主站入口。
7. 定向回归覆盖在 `src/components/rpg-entry/RpgEntryHomeView.recharge.test.tsx`、`src/components/rpg-entry/RpgEntryFlowShell.agent.interaction.test.tsx`、`src/components/platform-entry/platformEdutainmentVisibility.test.ts` 和 `src/routing/appRoutes.test.ts`，包含频道顺序、开关关闭、普通列表过滤、搜索过滤、作品号直达拦截、Demo 直达路由拦截和精确标签识别。

2
docs/technical/ADMIN_TRACKING_EVENT_DETAIL_EXPORT_2026-05-07.md
View File

@@ -72,7 +72,7 @@ GET /admin/api/tracking/events?eventKey=&userId=&scopeKind=&scopeId=&limit=
页面能力：
1. 顶部筛选区：Event Key、用户 ID、Scope Kind、Scope ID、刷新、导出 Excel。
1. 顶部筛选区：Event Key、用户 ID、Scope Kind、Scope ID、刷新、导出 Excel。Event Key 候选来自后台前端的埋点定义注册表，需覆盖 `BACKEND_TRACKING_EVENT_COVERAGE_2026-05-09.md` 中已接入的通用埋点事件，不能只保留 `daily_login`。
2. 列表区：移动端可横向滚动，桌面端表格展示。
3. 详情区：每行有“详情”按钮，弹出独立面板展示完整字段与格式化后的 metadata JSON。
4. 导出：导出当前页面已加载结果，文件名形如 `tracking-events-2026-05-07.xls`。

16
docs/technical/AUTH_LOGIN_OPTIONS_DESIGN_2026-04-21.md
View File

@@ -137,3 +137,19 @@
4. `npm run dev` / `npm run dev:rust` 完整栈默认由脚本计算 API 端口；加载 `.env.local` 给后端使用后，脚本必须重新固定 `RUST_SERVER_TARGET`，避免 `.env.local` 中的旧代理目标覆盖本次启动的实际 API 端口。
5. `npm run dev:web` 只启动前端，不会自动拉起 Rust API；如果 `.env.local` / 当前环境已经显式声明 `GENARRATIVE_RUNTIME_SERVER_TARGET`、`RUST_SERVER_TARGET`、`GENARRATIVE_API_TARGET` 或 `GENARRATIVE_API_PORT`，脚本必须固定使用该目标。目标当下不可用时只打印警告，不自动切到另一个端口，避免前端进程长时间绑定到随后会停掉的临时 API。
6. 如果 `3000` 仍然返回 `500`，先确认浏览器是不是还开着旧的前端进程。当前脚本如果因为端口占用漂移到 `3001` / `3002`，应直接关掉旧进程后重启，而不是继续用旧的 3000 页面判断登录入口状态。
## 11. 2026-05-10 `npm run api-server` 环境加载与短信 provider 排查记录
本地单独启动 `api-server` 时，环境变量合并顺序固定为：
```text
外层 shell > .env > .env.local > .env.secrets.local
```
这保证 `.env.local` 能覆盖 `.env.example` 派生出的默认值，`.env.secrets.local` 能继续覆盖本地私密密钥配置。`scripts/api-server-dev.mjs` 不得让 `.env` 后加载并覆盖 `.env.local`，否则 `SMS_AUTH_ENABLED` 或 `SMS_AUTH_PROVIDER` 可能被压回错误值。
排查“点击获取验证码但手机收不到短信”时，除了确认 `availableLoginMethods` 包含 `phone`，还必须确认当前进程实际使用的 provider：
1. `SMS_AUTH_PROVIDER="mock"` 只用于本地 UI / 账号链路联调，不会向手机发送真实短信；此时应使用 `SMS_AUTH_MOCK_VERIFY_CODE`，默认 `123456`。
2. 真实短信链路必须使用 `SMS_AUTH_PROVIDER="aliyun"`，并在修改 `.env.local` 后重启 `api-server`，运行中的进程不会自动切换 provider。
3. 真实 provider 是否被使用，以 `api-server` 日志中的 `provider=aliyun`、`provider_request_id` 和 `provider_out_id` 为准。

89
docs/technical/CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md
View File

@@ -21,6 +21,8 @@
9. 后续关卡使用热身记录的边界进行安全提醒和暂停恢复。
10. 热身结束后进入关卡选择。
当前阶段先落浏览器本地 Demo。浏览器摄像头视频流已接入舞台背景；热身动作阶段已接入本地 mocap 动作检测接口，通过 `useMocapInput` 连接 `http://127.0.0.1:8876/stream` 对应的 WebSocket 流，消费手势、左右手坐标和跳跃事件推进招手、左右手挥动与原地跳跃步骤。正式语音播报接口继续预留适配层，不阻塞前端热身流程、调试输入和页面表现骨架落地。
## 2. 非目标范围
热身关当前不包含以下内容：
@@ -33,6 +35,7 @@
6. 不做特定用户识别。
7. 不跨会话保存左右空间边界、手臂挥动空间和跳跃空间。
8. 不对手部细节进行识别，只对肢体进行区分。
9. 本阶段不处理无硬件、拒绝摄像头、多人入镜、识别丢失等异常流程；这些问题记录为待决策事项，后续硬件与摄像头方案稳定后再重新设计。
## 3. 运行入口与流向
@@ -44,6 +47,8 @@
用户完成热身关所有步骤后，进入关卡选择。
当前后续游戏仍在设计中。热身结束后可先展示“开始游戏”按钮作为关卡选择占位，用户点击后进入下一关占位界面。
### 3.3 固定流程顺序
热身关必须按照以下顺序执行：
@@ -166,6 +171,20 @@
4. 动作类状态没有最长等待时间。
5. 动作类状态等待 3 秒后可以播放对应引导动画。
### 6.3 开发者调试输入
本地 Demo 需要支持开发者调试模式，用于无摄像头和自动化验证场景。
调试映射如下：
1. `A` 键映射用户向左移动。
2. `D` 键映射用户向右移动。
3. 鼠标左键按下并拖动映射左手轨迹。
4. 鼠标右键按下并拖动映射右手轨迹。
5. 空格键映射原地跳跃。
调试输入只作为本地 Demo 与测试辅助，不代表正式动作识别硬件口径。正式摄像头接入后，位置、手势和跳跃判断需要按摄像头硬件调教结果重新校准。
## 7. 分步骤开发规格
### 7.1 进入热身关
@@ -438,6 +457,12 @@
4. 右手挥动空间。
5. 跳跃空间。
当前 Demo 体验会话数据需要满足：
1. 用户刷新产品或退出产品后失效。
2. 用户只关闭当前游戏关卡并重新进入时，可以直接来到开始游戏界面，不强制重复热身。
3. 首版可使用前端运行时内存或同等生命周期容器保存；不得跨产品刷新持久化保存。
### 8.2 当前 Demo 体验会话定义
“当前 Demo 体验会话”指用户本次打开并体验 Demo 的过程。
@@ -523,10 +548,14 @@
18. 关卡暂停时屏幕中央地面绿色圆圈。
19. 关卡暂停提示文案。
角色剪影、绿色圆环、虚影提醒、圆圈消失特效、手势引导动画和热身结束特效的正式视觉资源将通过 gpt-image-2 设计和生成。本地 Demo 阶段可以先使用 CSS、Canvas 或临时占位资源实现相同交互位置与状态，不把占位资源写死为正式资产。
## 12. 固定文案与语音清单
以下文案需要作为屏幕中上方浮现文字，并同步语音播报。
正式语音播报后续接入语音播报功能接口。本地 Demo 阶段保留播报适配层与调用点，可先只展示文字，不强制生成或播放正式语音资产。
```text
欢迎你，小朋友，见到你真开心
请你来到圆圈这里和我打个招呼吧
@@ -593,10 +622,58 @@
当前需求已明确本文所需的热身关开发规格。
以下内容未在当前文档中强行定义，后续如进入工程实现阶段，可再补充对应技术细节：
以下内容作为待决策事项保留，后续硬件、摄像头和正式关卡设计稳定后再补充：
1. 具体接入的动作识别 SDK 或硬件接口。
2. 角色剪影、圆环、虚影提醒、特效、手势引导动画的具体资源文件命名。
3. 当前 Demo 体验会话数据在前端状态、运行时上下文或其他容器中的具体存放位置。
4. 绿色圆环、角色剪影、安全边界在线性空间或屏幕坐标中的具体计算公式。
5. 关卡选择页的具体页面结构。
1. 具体接入的动作识别 SDK、硬件接口和摄像头接口。
2. 无硬件、摄像头拒绝授权、多人入镜、识别不到用户、跟踪丢失等异常流程。
3. 角色剪影、圆环、虚影提醒、特效、手势引导动画的正式资源文件命名。
4. 绿色圆环、角色剪影、安全边界在线性空间或屏幕坐标中的正式计算公式。
5. 正式关卡选择页与后续游戏关卡的具体页面结构。
## 15. 第 3 项本地 Demo 落地记录
本地浏览器 Demo 入口已落在：
```text
/child-motion-demo
```
当前实现范围：
1. `src/ChildMotionDemoApp.tsx` 挂载独立 Demo 应用壳。
2. `src/components/child-motion-demo/childMotionWarmupModel.ts` 维护热身步骤、圆环目标、2 秒保持判定、热身校准记录和当前运行时会话完成标记。
3. `src/components/child-motion-demo/ChildMotionWarmupDemo.tsx` 实现横屏舞台、背景虚化占位层、角色剪影、绿色圆环、手势引导、热身记录面板、热身完成后的“开始游戏”按钮和下一关占位界面。
4. `src/services/child-motion-demo/childMotionDebugInput.ts` 保留开发者调试输入适配层，后续可被正式动作识别 SDK 适配层替换或并行接入。
5. `src/routing/appRoutes.tsx` 新增 `/child-motion-demo` 独立路由，并复用 `VITE_ENABLE_EDUTAINMENT_ENTRY` 开关；开关关闭时不允许通过该直达路径进入 Demo。
当前调试输入：
1. `A` 键映射用户向左移动，松开后回到中心。
2. `D` 键映射用户向右移动，松开后回到中心。
3. 鼠标左键按下并拖动映射左手轨迹。
4. 鼠标右键按下并拖动映射右手轨迹。
5. 空格键映射原地跳跃。
当前硬件和动作检测接口接入：
1. 浏览器摄像头视频流已接入舞台背景。
2. 热身关手势阶段已通过 `src/services/useMocapInput.ts` 接入本地 mocap WebSocket `/stream`。
3. mocap 包支持从 `actions/action/gesture/gestures/event/name/type` 读取动作名，并支持 `hands[]`、`leftHand/rightHand`、`left_hand/right_hand` 读取左右手坐标。
4. `wave_greeting` 可由 `wave/wave_greeting/hand_wave/open_palm` 等动作或 open palm 手势完成。
5. `wave_left_hand` 和 `wave_right_hand` 优先消费对应左右手动作名；当硬件只持续输出手部坐标时，也可以根据连续手部横向轨迹完成挥手检测。
6. `jump_once` 消费 `jump/jump_once/hop` 等跳跃动作事件完成。
7. 键盘 `A/D/Space` 与鼠标左右键拖拽仍保留为本地 Demo 调试兜底，不代表正式硬件口径。
当前未接入但已保留边界：
1. 正式语音播报接口暂不接入，当前先展示热身文案。
2. 正式 gpt-image-2 视觉资源暂不接入，当前使用 CSS 占位表达相同位置和状态。
3. 后续关卡安全边界暂停逻辑暂未落地，当前只完成热身记录和下一关按钮占位。
已执行的定向验证命令：
```bash
npx eslint src/components/child-motion-demo/ChildMotionWarmupDemo.tsx src/components/child-motion-demo/childMotionWarmupModel.ts src/components/child-motion-demo/ChildMotionWarmupDemo.test.tsx src/components/child-motion-demo/childMotionWarmupModel.test.ts src/services/child-motion-demo/childMotionDebugInput.ts src/services/child-motion-demo/childMotionDebugInput.test.ts src/services/child-motion-demo/index.ts src/ChildMotionDemoApp.tsx src/routing/appRoutes.tsx src/routing/appRoutes.test.ts --ext .ts,.tsx --max-warnings 0
npx vitest run src/components/child-motion-demo/childMotionWarmupModel.test.ts src/components/child-motion-demo/ChildMotionWarmupDemo.test.tsx src/services/child-motion-demo/childMotionDebugInput.test.ts src/routing/appRoutes.test.ts
npm run check:encoding
```

2
docs/technical/PROFILE_TASK_AND_TRACKING_SYSTEM_2026-05-03.md
View File

@@ -46,7 +46,7 @@
用户成功登录时，认证链路会通过统一后端埋点 helper 幂等记录当日 `daily_login` 并刷新任务进度；用户打开任务中心只记录 `task_center_view` 浏览事件，不再承担每日登录事实写入。用户点击领取时，后端校验当日进度、领奖记录和配置状态，然后同事务写入领奖记录与钱包流水。
后台任务配置页的 `Event Key` 使用可搜索下拉控件，选项来自前端后台的埋点定义注册表。当前注册表默认包含 `daily_login`，展示中文名称和备注；后续新增任务依赖的埋点时，应先补充注册表，再开放运营配置。
后台任务配置页的 `Event Key` 使用可搜索下拉控件，选项来自前端后台的埋点定义注册表。后台全量埋点筛选候选应对齐 `BACKEND_TRACKING_EVENT_COVERAGE_2026-05-09.md` 的事件清单；任务配置页只展示标记为个人任务可用的事件，当前仅开放 `daily_login`，展示中文名称和备注。后续新增任务依赖的埋点时，应先补充注册表并显式标记任务可用，再开放运营配置。
## 6. 接口

12
docs/technical/PUZZLE_RUNTIME_FRONTEND_LOGIC_REHOME_2026-05-02.md
View File

@@ -23,6 +23,9 @@
7. 当前作品没有下一关时，通关弹窗展示后端 handoff 返回的相似作品；用户点击具体候选作品时直接 `startPuzzleRun(profileId, null)`，从目标作品第 `1` 关重新开始。
8. 失败状态点击“重新开始”时，正式 run 使用当前关 `levelId` 重新 `startPuzzleRun`，草稿/本地 run 使用本地重建，二者都保留当前失败关卡。
9. 结果页草稿试玩没有正式后端 run 时，继续使用本地 run、local leaderboard 和本地下一关兜底。
10. 运行态输入采用全项目通用的 `src/services/input-devices/` 抽象层承接，指针、触控、mocap 等设备都先归一为 `press / move / release / tap / drop` 拖拽语义，再由拼图运行态解析具体拼块和落点。
11. mocap `grab` 不是点击选中语义，而是持续拖拽语义；松手时按当前棋盘归一坐标提交 drop。合并大块只需要提交其中任一成员拼块 `pieceId`，本地拼图运行时会按 `mergedGroupId` 解析整组平移。
12. 拼图作品详情或开局遇到后端 `404 / NOT_FOUND / 资源不存在` 时，平台入口不再停留在空详情或运行态错误页，而是清理当前拼图详情/run 状态并返回首页。
## 工程落点
@@ -38,6 +41,15 @@
3. `src/components/rpg-entry/RpgEntryFlowShell.agent.interaction.test.tsx`
- 公开拼图玩法交互测试断言前端本地交换函数被调用。
- 同时断言后端 `swap / drag` 不参与棋盘交互，后端 `leaderboard / next-level` 继续参与非即时链路。
4. `src/services/input-devices/`
- `runtimeDragInputController` 提供设备无关的拖拽会话状态机。
- `runtimeInputGeometry` 提供屏幕坐标、归一坐标和网格命中的通用转换能力。
- 玩法组件只传入“这个点对应哪个目标”和“drop 到哪个目标”的玩法解释，不在输入层写拼图专用规则。
5. `src/components/puzzle-runtime/PuzzleRuntimeShell.tsx`
- 鼠标/触控与 mocap 共用同一个 runtime drag controller。
- 合并块成员不再被 mocap 路径过滤；mocap 可从合并块任一占用格抓取，并复用本地运行时的大块拖拽规则。
6. `src/components/platform-entry/PlatformEntryFlowShellImpl.tsx`
- `openPuzzleDetail`、`openPuzzlePublicWorkDetail`、`startPuzzleRunFromProfile` 对拼图作品缺失统一回首页。
## 边界

1
docs/technical/README.md
View File

@@ -5,6 +5,7 @@
## 文档列表
- [CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md](./CHILD_MOTION_DEMO_WARMUP_IMPLEMENTATION_SPEC_2026-05-09.md)：冻结儿童动作识别互动玩法 Demo 固定热身关的开发落地规格，覆盖横屏展示、摄像头背景虚化、角色剪影、绿色圆环 2 秒保持、动作教学、当前会话内空间边界记录和后续关卡安全暂停规则。
- [RUNTIME_INPUT_DEVICE_ABSTRACTION_2026-05-10.md](./RUNTIME_INPUT_DEVICE_ABSTRACTION_2026-05-10.md)：记录运行态输入设备抽象层，明确鼠标、触控、mocap 等设备统一归一为通用拖拽语义，玩法组件只负责解释目标和落点。
- [RUST_WORKSPACE_DEPENDENCY_CONSOLIDATION_2026-05-07.md](./RUST_WORKSPACE_DEPENDENCY_CONSOLIDATION_2026-05-07.md)：记录 `server-rs` Cargo 依赖集中配置口径，第三方版本和 workspace 内部 crate path 统一维护在根 `server-rs/Cargo.toml`，成员 crate 只保留 feature/optional 差异。
- [DEV_RUST_STACK_PORT_CONFLICT_PRECHECK_2026-05-09.md](./DEV_RUST_STACK_PORT_CONFLICT_PRECHECK_2026-05-09.md)：记录本地完整 Rust 栈启动时 `api-server`、主站 Vite 和后台 Vite 端口占用的误判根因、脚本预检策略和 Windows 排障命令。
- [VECTOR_ENGINE_GPT_IMAGE_2_GENERATION_2026-05-09.md](./VECTOR_ENGINE_GPT_IMAGE_2_GENERATION_2026-05-09.md)：记录 GPT-image-2 图片生成从 APIMart 迁移到 VectorEngine `gpt-image-2-all` 的接口、环境变量、尺寸映射、错误口径和验收命令。

57
docs/technical/RUNTIME_INPUT_DEVICE_ABSTRACTION_2026-05-10.md Normal file
View File

@@ -0,0 +1,57 @@
# 运行态输入设备抽象层 2026-05-10
## 背景
拼图运行态接入 mocap 后，鼠标/触控和 mocap 曾各自维护一套选择、坐标换算和拖拽提交逻辑。这样会让新设备只能在单个玩法里打补丁，也容易出现同一动作在不同设备下语义不一致的问题：例如 mocap `grab` 只触发选中，而不是像鼠标按住一样持续拖拽。
后续运行态还会接摇杆、键盘、体感、摄像头手势等输入来源，因此输入设备接入必须收口到全项目通用层。
## 决策
新增 `src/services/input-devices/` 作为前端运行态通用输入设备抽象层：
1. `runtimeDragInputController` 只维护设备无关的拖拽会话状态机。
2. `runtimeInputGeometry` 只处理 client 坐标、归一坐标、元素边界和网格命中换算。
3. 设备适配层把鼠标、触控、mocap 等输入归一为 `press / move / release / tap / drop`。
4. 玩法组件负责把通用输入点解释成自己的目标对象和落点，不把拼图、方洞或大鱼等玩法规则写进输入层。
## 当前接入
`useMocapInput` 解析 mocap `hands[].landmarks` 时应优先用 MediaPipe 21 点里的 `wrist / index_mcp / middle_mcp / ring_mcp / pinky_mcp` 加权计算掌心派生点；少于 3 个掌心关键点时才回退到 `wrist` 或直出 `hand.x/y`。这样运行态光标不会直接贴在腕部或指尖。
拼图运行态已接入该层：
- 鼠标/触控 `pointerdown / pointermove / pointerup` 进入同一个 drag controller。
- mocap `grab` 进入同一个 drag controller，并强制使用持续拖拽语义。
- mocap 光标按 60Hz 插值更新 UI 位置，并在拖拽中用插值后的当前点持续驱动输入层，避免输入包帧率低或抖动时出现明显跳变。
- 合并大块由拼图运行态把手部坐标命中到任一成员拼块；本地拼图运行时再按 `mergedGroupId` 执行整组平移。
## 接入规则
新玩法或新设备接入时遵循以下边界：
1. 输入层可以知道设备类型和几何换算，但不能知道玩法业务规则。
2. 设备适配层只负责把原始输入转换成通用输入事件。
3. 玩法壳层负责从通用输入点解析本玩法目标，例如拼块、洞口、角色或实体。
4. 玩法壳层负责决定 drop 后调用哪个本地运行态函数或后端接口。
5. 需要取消输入时优先按 `inputId` 取消，避免 mocap 丢帧误伤正在进行的鼠标/触控会话。
## 验证
基础抽象层验证：
```bash
npm run test -- src\services\input-devices\runtimeDragInputController.test.ts src\services\useMocapInput.test.ts
```
拼图接入验证：
```bash
npm run test -- src\components\puzzle-runtime\PuzzleRuntimeShell.test.tsx
```
跨平台入口缺失作品兜底验证：
```bash
npm run test -- src\components\rpg-entry\RpgEntryFlowShell.agent.interaction.test.tsx -t "missing puzzle public detail returns to platform home"
```

37
docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md
View File

@@ -25,24 +25,33 @@ npm run dev:rust
默认端口：
1. Web 前端：`http://127.0.0.1:3000`
2. Rust `api-server`：`http://127.0.0.1:8082`
3. SpacetimeDB standalone：`http://127.0.0.1:3101`
4. SpacetimeDB database：优先读取仓库根目录 `spacetime.local.json` 的 `database` 字段；没有该字段时才回退到 `genarrative-dev`
5. SpacetimeDB 本地数据与日志目录：`server-rs/.spacetimedb/local`
1. Web 前端：优先 `http://127.0.0.1:3000`
2. Rust `api-server`：优先 `http://127.0.0.1:8082`
3. SpacetimeDB standalone：优先 `http://127.0.0.1:3101`
4. 后台 Web 前端：优先 `http://127.0.0.1:3102`
5. SpacetimeDB database：优先读取仓库根目录 `spacetime.local.json` 的 `database` 字段；没有该字段时才回退到 `genarrative-dev`
6. SpacetimeDB 本地数据与日志目录：`server-rs/.spacetimedb/local`
启动前端口处理：
1. `npm run dev` / `npm run dev:rust` 会先检查 SpacetimeDB、Rust `api-server`、主站 Vite、后台 Vite 需要使用的端口。
2. 如果优先端口不可用，脚本会从该端口开始向后寻找可用端口，并将解析后的端口覆盖到后续 `spacetime start`、`spacetime publish --server`、`GENARRATIVE_API_PORT`、`RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET`、`ADMIN_API_TARGET` 与 Vite 启动参数。
3. 控制台会打印 `[dev:ports] ... 可用` 或 `[dev:ports] ... 不可用，改用 ...`，排查代理错配时以该日志和后续 `[dev:rust] web/admin web/rust api/spacetime` 实际地址为准。
4. 单独 `npm run dev:web` 也会检查主站 Vite 端口；`WEB_PORT` 或默认 `3000` 不可用时，会自动切到后续可用端口并继续严格端口启动。
默认流程：
1. 检查 `cargo`、`node` 与 `spacetime` CLI。
2. Windows Git Bash 下如 `server-rs/.spacetimedb/local/bin/current/spacetimedb-cli.exe` 不存在，先把本机 `spacetime` 所在安装目录的 `bin/` 与 `spacetime.exe` 同步到 `server-rs/.spacetimedb/local/`。
3. 启动 SpacetimeDB 前先检查 `server-rs/.spacetimedb/local/data/spacetime.pid`：如果 pid 对应进程仍存在，且同目录 `dev-rust-spacetime-url` 中记录的 URL 可被 `spacetime server ping` 判定在线，则直接复用该宿主；如果 URL 记录缺失，会依次尝试从 `logs/dev-rust-spacetime-start.log` 和 `logs/spacetime-standalone.log` 中解析最近一次监听地址兜底。否则按正常流程重新启动。
4. 如果确认需要新启动 SpacetimeDB，脚本会先检测 `127.0.0.1:3101` 是否可监听；若已占用，输出占用进程并选择从 `3101` 起向后的最近可用端口，再执行 `spacetime start --data-dir server-rs/.spacetimedb/local/data --listen-addr <实际地址>`。启动成功后把实际 URL 写入 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`，后续 publish 与 `api-server` 都使用同一个实际 URL。
5. 等待 SpacetimeDB 就绪：优先接受 `spacetime server ping http://127.0.0.1:<spacetime-port>` 输出中的 `Server is online:`；如果 Windows 下 SpacetimeDB CLI `2.1.0` 对已经监听的 standalone 仍打印 `502 Bad Gateway`，脚本会兜底请求 `http://127.0.0.1:<spacetime-port>/v1/ping`，只有该健康端点返回 `2xx` 时才放行。不能只依赖 CLI 退出码，因为 CLI 在 `502 Bad Gateway` 时也可能返回退出码 `0`。
6. 执行 `spacetime publish <本地数据库名> --server <实际 SpacetimeDB URL> --module-path server-rs/crates/spacetime-module --build-options="--debug" -c=on-conflict --yes`，确保 publish 仍由 SpacetimeDB CLI 负责构建和发布模块，同时使用 debug 构建参数降低本地开发等待时间；当前开发阶段允许新版模块表结构变化且发生 schema 冲突时清除旧模块数据。
7. 启动 `api-server` 前先检测默认 API 端口 `8082` 是否可监听；若已占用，输出占用进程并选择从 `8082` 起向后的最近可用端口。随后注入 `GENARRATIVE_API_*` 与 `GENARRATIVE_SPACETIME_*`，启动默认 debug profile 的 `cargo run -p api-server`；直接运行 `api-server` 时，如未显式设置 `GENARRATIVE_SPACETIME_DATABASE`，服务端也会向上查找 `spacetime.local.json` 作为本地默认库名。
8. 等待 `http://127.0.0.1:<api-port>/healthz` 返回 HTTP 响应后再启动 Vite，避免前端初始化请求早于 Rust `api-server` 监听完成并在终端刷出 `ECONNREFUSED 127.0.0.1:<api-port>`。
9. 注入 `RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET` 后启动 Vite。
10. 任一子进程退出时，脚本回收其余子进程。
2. 检查并解析本次联调需要使用的端口；端口不可用时先寻找可用端口，再把实际端口传给后续流程。
3. Windows Git Bash 下如 `server-rs/.spacetimedb/local/bin/current/spacetimedb-cli.exe` 不存在，先把本机 `spacetime` 所在安装目录的 `bin/` 与 `spacetime.exe` 同步到 `server-rs/.spacetimedb/local/`。
4. 启动 SpacetimeDB 前先检查 `server-rs/.spacetimedb/local/data/spacetime.pid`：如果 pid 对应进程仍存在，且同目录 `dev-rust-spacetime-url` 中记录的 URL 可被 `spacetime server ping` 判定在线，则直接复用该宿主；如果 URL 记录缺失，会依次尝试从 `logs/dev-rust-spacetime-start.log` 和 `logs/spacetime-standalone.log` 中解析最近一次监听地址兜底。否则按正常流程重新启动。
5. 如果确认需要新启动 SpacetimeDB，脚本会先检测 `127.0.0.1:3101` 是否可监听；若已占用，输出占用进程并选择从 `3101` 起向后的最近可用端口，再执行 `spacetime start --data-dir server-rs/.spacetimedb/local/data --listen-addr <实际地址>`。启动成功后把实际 URL 写入 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`，后续 publish 与 `api-server` 都使用同一个实际 URL。
6. 等待 SpacetimeDB 就绪：优先接受 `spacetime server ping http://127.0.0.1:<spacetime-port>` 输出中的 `Server is online:`；如果 Windows 下 SpacetimeDB CLI `2.1.0` 对已经监听的 standalone 仍打印 `502 Bad Gateway`，脚本会兜底请求 `http://127.0.0.1:<spacetime-port>/v1/ping`，只有该健康端点返回 `2xx` 时才放行。不能只依赖 CLI 退出码，因为 CLI 在 `502 Bad Gateway` 时也可能返回退出码 `0`。
7. 执行 `spacetime publish <本地数据库名> --server <实际 SpacetimeDB URL> --module-path server-rs/crates/spacetime-module --build-options="--debug" -c=on-conflict --yes`，确保 publish 仍由 SpacetimeDB CLI 负责构建和发布模块，同时使用 debug 构建参数降低本地开发等待时间；当前开发阶段允许新版模块表结构变化且发生 schema 冲突时清除旧模块数据。
8. 启动 `api-server` 前先检测默认 API 端口 `8082` 是否可监听；若已占用，输出占用进程并选择从 `8082` 起向后的最近可用端口。随后注入 `GENARRATIVE_API_*` 与 `GENARRATIVE_SPACETIME_*`，启动默认 debug profile 的 `cargo run -p api-server`；直接运行 `api-server` 时，如未显式设置 `GENARRATIVE_SPACETIME_DATABASE`，服务端也会向上查找 `spacetime.local.json` 作为本地默认库名。
9. 等待 `http://127.0.0.1:<api-port>/healthz` 返回 HTTP 响应后再启动 Vite，避免前端初始化请求早于 Rust `api-server` 监听完成并在终端刷出 `ECONNREFUSED 127.0.0.1:<api-port>`。
10. 注入 `RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET` 后启动 Vite。
11. 任一子进程退出时，脚本回收其余子进程。
Vite 代理覆盖范围：

2
scripts/api-server-dev.mjs
View File

@@ -45,6 +45,8 @@ export function loadApiServerEnv(
target,
protectedKeys = shellEnvKeys,
) {
// 保持与 dev-web-rust.mjs / dev-rust-stack.sh 一致：
// shell > .env > .env.local > .env.secrets.local。
for (const fileName of LOCAL_ENV_FILES) {
loadEnvFile(resolve(repoRootPath, fileName), target, protectedKeys);
}

59
scripts/dev-rust-stack.sh
View File

@@ -17,7 +17,7 @@ usage() {
说明:
1. 默认同时启动 SpacetimeDB standalone、Rust api-server、主站 Vite 与后台 Vite。
2. 当前开发阶段默认 publish server-rs/crates/spacetime-module 时追加 -c=on-conflict 在结构冲突时清理旧模块数据。
2. 当前开发阶段默认 publish server-rs/crates/spacetime-module 时追加 -c=on-conflict，在结构冲突时清理旧模块数据。
3. 只有显式传入 --preserve-database 时，才会跳过 -c=on-conflict。
4. SpacetimeDB 默认使用 server-rs/.spacetimedb/local/data 作为本地数据目录；需要新启动时会先检测端口并选择最近可用端口。
5. 默认在发布模块前随机生成迁移引导密钥，注入 GENARRATIVE_SPACETIME_MIGRATION_BOOTSTRAP_SECRET 并显示在控制台。
@@ -99,6 +99,25 @@ NODE
)
}
resolve_dev_stack_ports() {
local key
local value
while IFS='=' read -r key value; do
case "${key}" in
SPACETIME_PORT|API_PORT|WEB_PORT|ADMIN_WEB_PORT)
export "${key}=${value}"
;;
esac
done < <(
node "${REPO_ROOT}/scripts/dev-stack-port-utils.mjs" resolve-dev-stack \
"spacetime:${SPACETIME_HOST}:${SPACETIME_PORT}" \
"api:${API_TARGET_HOST}:${API_PORT}" \
"web:${WEB_HOST}:${WEB_PORT}" \
"adminWeb:${ADMIN_WEB_TARGET_HOST}:${ADMIN_WEB_PORT}"
)
}
cleanup() {
local index
@@ -876,8 +895,10 @@ fi
SPACETIME_SERVER="http://${SPACETIME_HOST}:${SPACETIME_PORT}"
API_TARGET_HOST="$(resolve_client_host "${API_HOST}")"
RUST_SERVER_TARGET="http://${API_TARGET_HOST}:${API_PORT}"
ADMIN_WEB_TARGET_HOST="$(resolve_client_host "${ADMIN_WEB_HOST}")"
resolve_dev_stack_ports
SPACETIME_SERVER="http://${SPACETIME_HOST}:${SPACETIME_PORT}"
RUST_SERVER_TARGET="http://${API_TARGET_HOST}:${API_PORT}"
trap cleanup EXIT INT TERM
@@ -906,24 +927,22 @@ if [[ "${SKIP_SPACETIME}" -ne 1 ]]; then
exit 1
fi
SPACETIME_PORT="$(find_nearest_available_port "${SPACETIME_HOST}" "${SPACETIME_PORT}" "SpacetimeDB")"
SPACETIME_SERVER="http://${SPACETIME_HOST}:${SPACETIME_PORT}"
SPACETIME_START_LOG="$(spacetime_start_log_path "${SPACETIME_DATA_DIR}")"
mkdir -p "$(dirname -- "${SPACETIME_START_LOG}")"
: >"${SPACETIME_START_LOG}"
echo "[dev:rust] 启动 spacetimedb"
(
cd "${SERVER_RS_DIR}"
# 启动前已经由脚本选定端口，避免 api-server 和 SpacetimeDB 对数据库地址认知不一致。
spacetime \
start \
--data-dir "${SPACETIME_DATA_DIR}" \
--listen-addr "${SPACETIME_HOST}:${SPACETIME_PORT}" \
--non-interactive
) 2>&1 | tee "${SPACETIME_START_LOG}" &
PIDS+=("$!")
NAMES+=("spacetimedb")
SPACETIME_START_LOG="${SPACETIME_DATA_DIR}/logs/dev-rust-spacetime-start.log"
mkdir -p "$(dirname -- "${SPACETIME_START_LOG}")"
: >"${SPACETIME_START_LOG}"
echo "[dev:rust] 启动 spacetimedb"
(
cd "${SERVER_RS_DIR}"
# 当目标端口被占用时，SpacetimeDB 会询问是否使用最近的可用端口；
# 这里直接发送回车接受默认建议，再从启动日志解析实际监听端口。
printf '\n' | spacetime \
start \
--data-dir "${SPACETIME_DATA_DIR}" \
--listen-addr "${SPACETIME_HOST}:${SPACETIME_PORT}" \
--non-interactive
) 2>&1 | tee "${SPACETIME_START_LOG}" &
PIDS+=("$!")
NAMES+=("spacetimedb")
SPACETIME_LISTEN_ADDR="$(wait_for_spacetime_listen_addr "${SPACETIME_START_LOG}" "${SPACETIME_TIMEOUT_SECONDS}" "${PIDS[0]:-}")"
SPACETIME_PORT="$(port_from_listen_addr "${SPACETIME_LISTEN_ADDR}")"

164
scripts/dev-stack-port-utils.mjs Normal file
View File

@@ -0,0 +1,164 @@
import {createServer} from 'node:net';
function toListenHosts(host) {
if (host === '0.0.0.0') {
return ['0.0.0.0'];
}
if (host === '::') {
return ['::'];
}
return [host];
}
export function normalizePort(value, fallback) {
const port = Number.parseInt(String(value ?? ''), 10);
if (!Number.isInteger(port) || port < 0 || port > 65535) {
return fallback;
}
return port;
}
export async function isPortAvailable({host, port}) {
if (port === 0) {
return true;
}
const listenHosts = toListenHosts(host);
for (const listenHost of listenHosts) {
const available = await new Promise((resolve) => {
const server = createServer();
server.unref();
server.once('error', () => resolve(false));
server.listen({host: listenHost, port}, () => {
server.close(() => resolve(true));
});
});
if (!available) {
return false;
}
}
return true;
}
export async function findAvailablePort({host, preferredPort, reservedPorts = new Set(), maxAttempts = 200}) {
const startPort = normalizePort(preferredPort, 0);
if (startPort === 0) {
return await reserveEphemeralPort(host, reservedPorts);
}
for (let offset = 0; offset <= maxAttempts; offset += 1) {
const candidate = startPort + offset;
if (candidate > 65535) {
break;
}
if (reservedPorts.has(candidate)) {
continue;
}
if (await isPortAvailable({host, port: candidate})) {
return candidate;
}
}
throw new Error(`无法从 ${host}:${startPort} 开始找到可用端口`);
}
async function reserveEphemeralPort(host, reservedPorts) {
for (let attempt = 0; attempt < 20; attempt += 1) {
const port = await new Promise((resolve, reject) => {
const server = createServer();
server.unref();
server.once('error', reject);
server.listen({host, port: 0}, () => {
const address = server.address();
const resolvedPort = typeof address === 'object' && address ? address.port : 0;
server.close(() => resolve(resolvedPort));
});
});
if (typeof port === 'number' && port > 0 && !reservedPorts.has(port)) {
return port;
}
}
throw new Error(`无法为 ${host} 分配临时可用端口`);
}
export async function resolveDevStackPorts(config) {
const reservedPorts = new Set();
const entries = [
['spacetime', config.spacetime],
['api', config.api],
['web', config.web],
['adminWeb', config.adminWeb],
].filter(([, portConfig]) => Boolean(portConfig));
const result = {};
for (const [name, portConfig] of entries) {
const resolvedPort = await findAvailablePort({
host: portConfig.host,
preferredPort: portConfig.preferredPort,
reservedPorts,
});
reservedPorts.add(resolvedPort);
result[name] = resolvedPort;
}
return result;
}
export function formatPortDecision({name, host, preferredPort, resolvedPort}) {
if (preferredPort === resolvedPort && preferredPort !== 0) {
return `[dev:ports] ${name}: ${host}:${resolvedPort} 可用`;
}
return `[dev:ports] ${name}: ${host}:${preferredPort} 不可用，改用 ${host}:${resolvedPort}`;
}
function parseCliPortConfig(rawArgs) {
const config = {};
for (const rawArg of rawArgs) {
const [name, host, rawPreferredPort] = rawArg.split(':');
if (!name || !host || rawPreferredPort == null) {
throw new Error(`端口配置参数格式错误: ${rawArg}`);
}
config[name] = {
host,
preferredPort: normalizePort(rawPreferredPort, 0),
};
}
return config;
}
function envKeyForPortName(name) {
return `${name.replace(/[A-Z]/gu, (letter) => `_${letter}`).toUpperCase()}_PORT`;
}
if (process.argv[2] === 'resolve-dev-stack') {
const config = parseCliPortConfig(process.argv.slice(3));
const resolvedPorts = await resolveDevStackPorts(config);
for (const [name, resolvedPort] of Object.entries(resolvedPorts)) {
const portConfig = config[name];
console.error(
formatPortDecision({
name,
host: portConfig.host,
preferredPort: portConfig.preferredPort,
resolvedPort,
}),
);
console.log(`${envKeyForPortName(name)}=${resolvedPort}`);
}
}

51
scripts/dev-stack-port-utils.test.ts Normal file
View File

@@ -0,0 +1,51 @@
import {createServer} from 'node:net';
import {describe, expect, it} from 'vitest';
import {
findAvailablePort,
resolveDevStackPorts,
} from './dev-stack-port-utils.mjs';
function reservePort(port) {
return new Promise((resolve, reject) => {
const server = createServer();
server.once('error', reject);
server.listen(port, '127.0.0.1', () => {
server.off('error', reject);
resolve(server);
});
});
}
describe('dev stack port utils', () => {
it('使用端口可用性检查为被占用端口寻找后续可用端口', async () => {
const firstServer = await reservePort(0);
const firstPort = firstServer.address().port;
const secondServer = await reservePort(firstPort + 1);
try {
const availablePort = await findAvailablePort({
host: '127.0.0.1',
preferredPort: firstPort,
});
expect(availablePort).toBeGreaterThan(firstPort + 1);
} finally {
await Promise.all([
new Promise((resolve) => firstServer.close(resolve)),
new Promise((resolve) => secondServer.close(resolve)),
]);
}
});
it('为 npm run dev 的所有后续流程解析互不冲突的端口', async () => {
const resolvedPorts = await resolveDevStackPorts({
spacetime: {host: '127.0.0.1', preferredPort: 0},
api: {host: '127.0.0.1', preferredPort: 0},
web: {host: '127.0.0.1', preferredPort: 0},
adminWeb: {host: '127.0.0.1', preferredPort: 0},
});
expect(new Set(Object.values(resolvedPorts)).size).toBe(4);
});
});

22
scripts/dev-web-rust.mjs
View File

@@ -1,6 +1,11 @@
import {spawn} from 'node:child_process';
import {existsSync, readFileSync} from 'node:fs';
import {resolve} from 'node:path';
import {
findAvailablePort,
formatPortDecision,
normalizePort,
} from './dev-stack-port-utils.mjs';
const repoRoot = process.cwd();
const shellEnvKeys = new Set(Object.keys(process.env));
@@ -121,9 +126,24 @@ const mergedEnv = {
console.log(`[dev:web] backend=rust target=${mergedEnv.GENARRATIVE_RUNTIME_SERVER_TARGET}`);
const webHost = '0.0.0.0';
const preferredWebPort = normalizePort(fileEnv.WEB_PORT, 3000);
const webPort = await findAvailablePort({
host: webHost,
preferredPort: preferredWebPort,
});
console.log(
formatPortDecision({
name: 'web',
host: webHost,
preferredPort: preferredWebPort,
resolvedPort: webPort,
}),
);
const child = spawn(
'node',
['scripts/vite-cli.mjs', '--port=3000', '--host=0.0.0.0', '--strictPort'],
['scripts/vite-cli.mjs', `--port=${webPort}`, `--host=${webHost}`, '--strictPort'],
{
cwd: process.cwd(),
env: mergedEnv,

5
src/ChildMotionDemoApp.tsx Normal file
View File

@@ -0,0 +1,5 @@
import { ChildMotionWarmupDemo } from './components/child-motion-demo/ChildMotionWarmupDemo';
export default function ChildMotionDemoApp() {
return <ChildMotionWarmupDemo />;
}

364
src/components/child-motion-demo/ChildMotionWarmupDemo.test.tsx Normal file
View File

@@ -0,0 +1,364 @@
/* @vitest-environment jsdom */
import { act, fireEvent, render, screen } from '@testing-library/react';
import { afterEach, beforeEach, expect, test, vi } from 'vitest';
import { ChildMotionWarmupDemo } from './ChildMotionWarmupDemo';
import {
markChildMotionWarmupCompletedInRuntime,
resetChildMotionWarmupRuntimeSession,
} from './childMotionWarmupModel';
const mocapMock = vi.hoisted(() => ({
status: 'connected' as 'idle' | 'connecting' | 'connected' | 'error',
command: null as null | {
actions: string[];
hands?: Array<{ x: number; y: number; state: string; side: string }>;
primaryHand?: { x: number; y: number; state: string; side: string } | null;
leftHand?: { x: number; y: number; state: string; side: string } | null;
rightHand?: { x: number; y: number; state: string; side: string } | null;
bodyCenter?: { x: number; y: number } | null;
},
receivedAtMs: 1,
}));
vi.mock('../../services/useMocapInput', () => ({
useMocapInput: ({ enabled }: { enabled: boolean }) => ({
status: enabled ? mocapMock.status : 'idle',
latestCommand: enabled ? mocapMock.command : null,
rawPacketPreview:
enabled && mocapMock.command
? {
text: JSON.stringify(mocapMock.command),
receivedAtMs: mocapMock.receivedAtMs,
}
: null,
error: null,
}),
}));
beforeEach(() => {
resetChildMotionWarmupRuntimeSession();
vi.restoreAllMocks();
mocapMock.status = 'connected';
mocapMock.command = null;
mocapMock.receivedAtMs = 1;
Object.defineProperty(navigator, 'mediaDevices', {
configurable: true,
value: undefined,
});
});
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});
test('renders the warmup stage and starts with the center ring step', () => {
render(<ChildMotionWarmupDemo />);
expect(screen.getByTestId('child-motion-demo')).toBeTruthy();
expect(screen.getByText('来到圆圈这里')).toBeTruthy();
expect(screen.getByLabelText('绿色圆环')).toBeTruthy();
expect(screen.getByText('请横屏体验')).toBeTruthy();
});
test('re-entering within the same runtime session opens the start button', () => {
markChildMotionWarmupCompletedInRuntime();
render(<ChildMotionWarmupDemo />);
expect(screen.getByRole('button', { name: '开始游戏' })).toBeTruthy();
});
test('developer keyboard input moves the avatar and triggers jump state', () => {
render(<ChildMotionWarmupDemo />);
const avatar = screen.getByTestId('child-motion-avatar');
fireEvent.keyDown(window, { key: 'a', code: 'KeyA' });
expect(avatar.getAttribute('style')).toContain('left: 34%');
fireEvent.keyDown(window, { key: 'd', code: 'KeyD' });
expect(avatar.getAttribute('style')).toContain('left: 66%');
fireEvent.keyUp(window, { key: 'd', code: 'KeyD' });
expect(avatar.getAttribute('style')).toContain('left: 50%');
fireEvent.keyDown(window, { key: ' ', code: 'Space' });
expect(avatar.className).toContain('child-motion-avatar--jumping');
});
test('mocap body center keeps the warmup flow on the motion data source', async () => {
vi.useFakeTimers();
mocapMock.command = {
actions: [],
bodyCenter: { x: 0.5, y: 0.6 },
hands: [],
primaryHand: null,
leftHand: null,
rightHand: null,
};
const { rerender, unmount } = render(<ChildMotionWarmupDemo />);
expect(screen.queryByText('摄像头暂不可用，已切换到本地演示')).toBeNull();
expect(screen.queryByText('动作数据已连接，等待识别')).toBeNull();
expect(screen.getByTestId('child-motion-avatar').getAttribute('style')).toContain(
'left: 50%',
);
await act(async () => {
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByText('打个招呼')).toBeTruthy();
});
mocapMock.command = {
actions: ['open_palm'],
bodyCenter: { x: 0.5, y: 0.6 },
hands: [{ x: 0.48, y: 0.34, state: 'open_palm', side: 'left' }],
primaryHand: { x: 0.48, y: 0.34, state: 'open_palm', side: 'left' },
leftHand: { x: 0.48, y: 0.34, state: 'open_palm', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
vi.advanceTimersByTime(1000);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByText('准备热身')).toBeTruthy();
});
await act(async () => {
vi.advanceTimersByTime(1000);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByRole('heading', { name: '向左一步' })).toBeTruthy();
});
mocapMock.command = {
actions: [],
bodyCenter: { x: 0.34, y: 0.6 },
hands: [],
primaryHand: null,
leftHand: null,
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
await vi.waitFor(() => {
expect(screen.getByTestId('child-motion-avatar').getAttribute('style')).toContain(
'left: 34%',
);
});
await act(async () => {
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByRole('heading', { name: '回到中间来' })).toBeTruthy();
});
await act(async () => {
unmount();
});
vi.useRealTimers();
});
test('mocap open palm completes the greeting wave step', async () => {
vi.useFakeTimers();
const { rerender, unmount } = render(<ChildMotionWarmupDemo />);
await act(async () => {
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByText('打个招呼')).toBeTruthy();
});
mocapMock.command = {
actions: ['open_palm'],
hands: [{ x: 0.46, y: 0.34, state: 'open_palm', side: 'left' }],
primaryHand: { x: 0.46, y: 0.34, state: 'open_palm', side: 'left' },
leftHand: { x: 0.46, y: 0.34, state: 'open_palm', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
await vi.waitFor(() => {
expect(screen.getByText('准备热身')).toBeTruthy();
});
await act(async () => {
unmount();
});
vi.useRealTimers();
});
test('mocap hand tracks complete left and right wave steps only after movement is visible', async () => {
vi.useFakeTimers();
const { rerender, unmount } = render(<ChildMotionWarmupDemo />);
const advancePositionStep = async (key: string, code: string) => {
await act(async () => {
fireEvent.keyDown(window, { key, code });
});
await act(async () => {
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await act(async () => {
fireEvent.keyUp(window, { key, code });
});
};
await act(async () => {
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByText('打个招呼')).toBeTruthy();
});
mocapMock.command = {
actions: ['open_palm'],
hands: [{ x: 0.48, y: 0.34, state: 'open_palm', side: 'left' }],
primaryHand: { x: 0.48, y: 0.34, state: 'open_palm', side: 'left' },
leftHand: { x: 0.48, y: 0.34, state: 'open_palm', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
await act(async () => {
vi.advanceTimersByTime(1000);
await vi.runOnlyPendingTimersAsync();
});
await advancePositionStep('a', 'KeyA');
await act(async () => {
vi.advanceTimersByTime(120);
await vi.runOnlyPendingTimersAsync();
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await advancePositionStep('d', 'KeyD');
await act(async () => {
vi.advanceTimersByTime(120);
await vi.runOnlyPendingTimersAsync();
vi.advanceTimersByTime(2100);
await vi.runOnlyPendingTimersAsync();
});
await vi.waitFor(() => {
expect(screen.getByRole('heading', { name: '挥动左手' })).toBeTruthy();
});
mocapMock.command = {
actions: [],
leftHand: { x: 0.3, y: 0.38, state: 'unknown', side: 'left' },
primaryHand: { x: 0.3, y: 0.38, state: 'unknown', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
mocapMock.command = {
actions: [],
leftHand: { x: 0.39, y: 0.36, state: 'unknown', side: 'left' },
primaryHand: { x: 0.39, y: 0.36, state: 'unknown', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
mocapMock.command = {
actions: [],
leftHand: { x: 0.31, y: 0.34, state: 'unknown', side: 'left' },
primaryHand: { x: 0.31, y: 0.34, state: 'unknown', side: 'left' },
rightHand: null,
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
await vi.waitFor(() => {
expect(screen.getByRole('heading', { name: '挥动右手' })).toBeTruthy();
});
mocapMock.command = {
actions: ['right_hand_wave'],
leftHand: null,
primaryHand: { x: 0.64, y: 0.35, state: 'unknown', side: 'right' },
rightHand: { x: 0.64, y: 0.35, state: 'unknown', side: 'right' },
};
mocapMock.receivedAtMs += 1;
await act(async () => {
rerender(<ChildMotionWarmupDemo />);
});
await vi.waitFor(() => {
expect(screen.getByRole('heading', { name: '原地跳一下' })).toBeTruthy();
});
await act(async () => {
vi.advanceTimersByTime(720);
await vi.runOnlyPendingTimersAsync();
unmount();
});
vi.useRealTimers();
});
test('connects camera stream and releases it on unmount', async () => {
const stopTrack = vi.fn();
const stream = {
getTracks: () => [
{
stop: stopTrack,
},
],
} as unknown as MediaStream;
const getUserMedia = vi.fn().mockResolvedValue(stream);
const play = vi
.spyOn(HTMLMediaElement.prototype, 'play')
.mockResolvedValue(undefined);
Object.defineProperty(navigator, 'mediaDevices', {
configurable: true,
value: {
getUserMedia,
},
});
const { unmount } = render(<ChildMotionWarmupDemo />);
expect(screen.queryByText('摄像头暂不可用，已切换到本地演示')).toBeNull();
expect(screen.getByText('动作数据已连接，等待识别')).toBeTruthy();
await vi.waitFor(() => {
expect(getUserMedia).toHaveBeenCalledWith({
audio: false,
video: {
facingMode: 'user',
},
});
expect(play).toHaveBeenCalled();
});
unmount();
expect(stopTrack).toHaveBeenCalled();
});

870
src/components/child-motion-demo/ChildMotionWarmupDemo.tsx Normal file
View File

@@ -0,0 +1,870 @@
import type {
CSSProperties,
PointerEvent as ReactPointerEvent,
} from 'react';
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import type {
MocapConnectionStatus,
MocapHandInput,
MocapInputCommand,
} from '../../services/useMocapInput';
import { useMocapInput } from '../../services/useMocapInput';
import {
applyChildMotionWarmupCompletion,
CHILD_MOTION_CENTER_X,
CHILD_MOTION_FINISH_DURATION_MS,
CHILD_MOTION_HOLD_DURATION_MS,
CHILD_MOTION_NARRATION_DURATION_MS,
type ChildMotionPoint,
type ChildMotionWarmupCalibration,
type ChildMotionWarmupStepId,
createEmptyChildMotionCalibration,
getChildMotionTargetX,
getChildMotionWarmupStep,
hasCompletedChildMotionWarmupInRuntime,
isAvatarOnWarmupTarget,
markChildMotionWarmupCompletedInRuntime,
resolveNextChildMotionWarmupStep,
} from './childMotionWarmupModel';
type DragHand = 'left' | 'right';
type CameraAccessState = 'idle' | 'requesting' | 'ready' | 'blocked';
type MotionSourceState = 'connecting' | 'ready' | 'waiting' | 'offline';
type WarmupMocapGestureIntent = 'greeting' | 'left-hand' | 'right-hand' | 'jump';
const WARMUP_MOCAP_WAVE_MIN_POINTS = 3;
const WARMUP_MOCAP_WAVE_MIN_X_RANGE = 0.055;
function clampMotionUnit(value: number) {
return Math.max(0, Math.min(1, value));
}
function normalizePointerPoint(
event: ReactPointerEvent<HTMLElement>,
element: HTMLElement,
): ChildMotionPoint {
const rect = element.getBoundingClientRect();
const width = rect.width || 1;
const height = rect.height || 1;
return {
x: clampMotionUnit((event.clientX - rect.left) / width),
y: clampMotionUnit((event.clientY - rect.top) / height),
};
}
function formatPercent(value: number | null) {
if (value === null) {
return '--';
}
return `${Math.round(value * 100)}%`;
}
function mocapHandToChildMotionPoint(
hand: MocapHandInput | null | undefined,
): ChildMotionPoint | null {
if (!hand) {
return null;
}
return {
x: clampMotionUnit(hand.x),
y: clampMotionUnit(hand.y),
};
}
function appendWarmupMocapPoint(
points: ChildMotionPoint[],
point: ChildMotionPoint,
) {
return [...points, point].slice(-16);
}
function getMotionSourceState(
mocapStatus: MocapConnectionStatus,
latestCommand: MocapInputCommand | null,
): MotionSourceState {
if (mocapStatus === 'connecting' || mocapStatus === 'idle') {
return 'connecting';
}
if (mocapStatus === 'connected') {
return latestCommand &&
(Boolean(latestCommand.bodyCenter) ||
Boolean(latestCommand.hands?.length) ||
latestCommand.actions.length > 0)
? 'ready'
: 'waiting';
}
return 'offline';
}
function getMotionSourceText(state: MotionSourceState) {
if (state === 'ready') {
return '动作数据已连接';
}
if (state === 'waiting') {
return '动作数据已连接，等待识别';
}
if (state === 'offline') {
return '动作数据暂不可用，已保留本地调试';
}
return '正在连接动作数据';
}
function hasWarmupMocapAction(
command: MocapInputCommand,
expectedActions: string[],
) {
return command.actions.some((action) => expectedActions.includes(action));
}
function hasWarmupMocapWavePath(points: ChildMotionPoint[]) {
if (points.length < WARMUP_MOCAP_WAVE_MIN_POINTS) {
return false;
}
const xValues = points.map((point) => point.x);
return (
Math.max(...xValues) - Math.min(...xValues) >=
WARMUP_MOCAP_WAVE_MIN_X_RANGE
);
}
function resolveAvatarXFromMocap(command: MocapInputCommand) {
return command.bodyCenter?.x ?? null;
}
function resolveWarmupMocapGestureIntent(
stepId: ChildMotionWarmupStepId,
command: MocapInputCommand,
paths: {
leftHandPath: ChildMotionPoint[];
rightHandPath: ChildMotionPoint[];
primaryHandPath: ChildMotionPoint[];
},
): WarmupMocapGestureIntent | null {
if (stepId === 'wave_greeting') {
if (
hasWarmupMocapAction(command, [
'wave',
'wave_greeting',
'hand_wave',
'hello',
'greeting',
'open_palm',
'handwave',
'wavehand',
'招手',
'挥手',
]) ||
command.hands?.some((hand) => hand.state === 'open_palm') ||
hasWarmupMocapWavePath(paths.leftHandPath) ||
hasWarmupMocapWavePath(paths.rightHandPath) ||
hasWarmupMocapWavePath(paths.primaryHandPath)
) {
return 'greeting';
}
}
if (
stepId === 'wave_left_hand' &&
(hasWarmupMocapAction(command, [
'left_wave',
'wave_left',
'left_hand_wave',
'wave_left_hand',
'left_handwave',
'lefthand_wave',
'lefthandwave',
'左手挥手',
'挥动左手',
]) ||
hasWarmupMocapWavePath(paths.leftHandPath))
) {
return 'left-hand';
}
if (
stepId === 'wave_right_hand' &&
(hasWarmupMocapAction(command, [
'right_wave',
'wave_right',
'right_hand_wave',
'wave_right_hand',
'right_handwave',
'righthand_wave',
'righthandwave',
'右手挥手',
'挥动右手',
]) ||
hasWarmupMocapWavePath(paths.rightHandPath))
) {
return 'right-hand';
}
if (
stepId === 'jump_once' &&
hasWarmupMocapAction(command, ['jump', 'jump_once', 'hop', '跳跃', '原地跳'])
) {
return 'jump';
}
return null;
}
function getHoldProgress(
stepId: ChildMotionWarmupStepId,
avatarX: number,
holdStartedAt: number | null,
nowMs: number,
) {
const step = getChildMotionWarmupStep(stepId);
if (!isAvatarOnWarmupTarget(step, avatarX) || holdStartedAt === null) {
return 0;
}
return Math.min(1, (nowMs - holdStartedAt) / CHILD_MOTION_HOLD_DURATION_MS);
}
function getStepIndex(stepId: ChildMotionWarmupStepId) {
const order: ChildMotionWarmupStepId[] = [
'center_arrive',
'wave_greeting',
'warmup_intro',
'move_left',
'return_center_1',
'move_right',
'return_center_2',
'wave_left_hand',
'wave_right_hand',
'jump_once',
'warmup_finish',
'level_select',
'play_placeholder',
];
return Math.max(0, order.indexOf(stepId));
}
function ChildMotionAvatar({
avatarX,
isJumping,
}: {
avatarX: number;
isJumping: boolean;
}) {
return (
<div
className={`child-motion-avatar ${isJumping ? 'child-motion-avatar--jumping' : ''}`}
data-testid="child-motion-avatar"
style={{
left: `${avatarX * 100}%`,
}}
aria-label="用户角色剪影"
>
<span className="child-motion-avatar__head" />
<span className="child-motion-avatar__body" />
<span className="child-motion-avatar__arm child-motion-avatar__arm--left" />
<span className="child-motion-avatar__arm child-motion-avatar__arm--right" />
<span className="child-motion-avatar__leg child-motion-avatar__leg--left" />
<span className="child-motion-avatar__leg child-motion-avatar__leg--right" />
</div>
);
}
function ChildMotionRing({
targetX,
progress,
}: {
targetX: number;
progress: number;
}) {
return (
<div
className={`child-motion-ring ${progress > 0 ? 'child-motion-ring--active' : ''}`}
data-testid="child-motion-ring"
style={{
left: `${targetX * 100}%`,
'--child-motion-ring-progress': `${Math.round(progress * 360)}deg`,
} as CSSProperties}
aria-label="绿色圆环"
>
<span className="child-motion-ring__core" />
</div>
);
}
function ChildMotionGestureGuide({
stepId,
leftHandPath,
rightHandPath,
}: {
stepId: ChildMotionWarmupStepId;
leftHandPath: ChildMotionPoint[];
rightHandPath: ChildMotionPoint[];
}) {
const isLeft = stepId === 'wave_left_hand';
const isRight = stepId === 'wave_right_hand';
const isGreeting = stepId === 'wave_greeting';
const isJump = stepId === 'jump_once';
const activePath = isLeft ? leftHandPath : isRight ? rightHandPath : [];
return (
<div className="child-motion-gesture-guide" aria-hidden="true">
{isGreeting ? (
<span className="child-motion-gesture-guide__wave">挥手</span>
) : null}
{isLeft || isRight ? (
<>
<span
className={`child-motion-gesture-guide__hand child-motion-gesture-guide__hand--${isLeft ? 'left' : 'right'}`}
/>
{activePath.map((point, index) => (
<span
key={`${isLeft ? 'left' : 'right'}-${index}`}
className="child-motion-gesture-guide__trail"
style={{
left: `${point.x * 100}%`,
top: `${point.y * 100}%`,
opacity: 0.22 + (index / Math.max(1, activePath.length)) * 0.58,
}}
/>
))}
</>
) : null}
{isJump ? <span className="child-motion-gesture-guide__jump">跳</span> : null}
</div>
);
}
function ChildMotionCalibrationPanel({
calibration,
}: {
calibration: ChildMotionWarmupCalibration;
}) {
return (
<div className="child-motion-calibration" aria-label="热身记录">
<div>
<span>左边界</span>
<strong>{formatPercent(calibration.leftBoundary)}</strong>
</div>
<div>
<span>右边界</span>
<strong>{formatPercent(calibration.rightBoundary)}</strong>
</div>
<div>
<span>左手</span>
<strong>{calibration.leftHandPath.length}</strong>
</div>
<div>
<span>右手</span>
<strong>{calibration.rightHandPath.length}</strong>
</div>
<div>
<span>跳跃</span>
<strong>{formatPercent(calibration.jumpSpace)}</strong>
</div>
</div>
);
}
export function ChildMotionWarmupDemo() {
const [stepId, setStepId] = useState<ChildMotionWarmupStepId>(() =>
hasCompletedChildMotionWarmupInRuntime() ? 'level_select' : 'center_arrive',
);
const [avatarX, setAvatarX] = useState(CHILD_MOTION_CENTER_X);
const [calibration, setCalibration] = useState(
createEmptyChildMotionCalibration,
);
const [holdStartedAt, setHoldStartedAt] = useState<number | null>(null);
const [nowMs, setNowMs] = useState(() => Date.now());
const [leftHandPath, setLeftHandPath] = useState<ChildMotionPoint[]>([]);
const [rightHandPath, setRightHandPath] = useState<ChildMotionPoint[]>([]);
const [activeHand, setActiveHand] = useState<DragHand | null>(null);
const [isJumping, setIsJumping] = useState(false);
const [justCompletedText, setJustCompletedText] = useState<string | null>(null);
const [cameraAccessState, setCameraAccessState] =
useState<CameraAccessState>(() =>
typeof navigator === 'undefined' ||
!navigator.mediaDevices?.getUserMedia
? 'blocked'
: 'idle',
);
const holdCompletionRef = useRef(false);
const cameraVideoRef = useRef<HTMLVideoElement | null>(null);
const cameraStreamRef = useRef<MediaStream | null>(null);
const handledMocapPacketKeyRef = useRef<string | null>(null);
const step = getChildMotionWarmupStep(stepId);
const mocapInput = useMocapInput({
enabled:
step.kind === 'position' ||
step.kind === 'gesture' ||
step.kind === 'narration' ||
step.kind === 'finish',
});
const stepIndex = getStepIndex(stepId);
const progressPercent = Math.round((stepIndex / 12) * 100);
const holdProgress = getHoldProgress(stepId, avatarX, holdStartedAt, nowMs);
const targetX = step.target ? getChildMotionTargetX(step.target) : null;
const motionSourceState = getMotionSourceState(
mocapInput.status,
mocapInput.latestCommand,
);
const motionSourceText = getMotionSourceText(motionSourceState);
const completeStep = useCallback(
(completion: Parameters<typeof applyChildMotionWarmupCompletion>[2]) => {
setCalibration((current) =>
applyChildMotionWarmupCompletion(stepId, current, completion),
);
const nextStep = resolveNextChildMotionWarmupStep(stepId);
if (stepId === 'jump_once') {
markChildMotionWarmupCompletedInRuntime();
}
setJustCompletedText(
stepId === 'warmup_finish' || stepId === 'jump_once' ? null : '真棒',
);
window.setTimeout(() => setJustCompletedText(null), 720);
setStepId(nextStep);
setHoldStartedAt(null);
holdCompletionRef.current = false;
},
[stepId],
);
useEffect(() => {
const timer = window.setInterval(() => setNowMs(Date.now()), 120);
return () => window.clearInterval(timer);
}, []);
useEffect(() => {
const videoElement = cameraVideoRef.current;
if (
typeof navigator === 'undefined' ||
!navigator.mediaDevices?.getUserMedia ||
!videoElement
) {
return;
}
let isMounted = true;
const startCamera = async () => {
if (!navigator.mediaDevices?.getUserMedia) {
if (isMounted) {
setCameraAccessState('blocked');
}
return;
}
try {
setCameraAccessState('requesting');
const stream = await navigator.mediaDevices.getUserMedia({
audio: false,
video: {
facingMode: 'user',
},
});
if (!isMounted) {
stream.getTracks().forEach((track) => track.stop());
return;
}
cameraStreamRef.current?.getTracks().forEach((track) => track.stop());
cameraStreamRef.current = stream;
videoElement.srcObject = stream;
await videoElement.play();
setCameraAccessState('ready');
} catch {
cameraStreamRef.current?.getTracks().forEach((track) => track.stop());
cameraStreamRef.current = null;
videoElement.srcObject = null;
if (isMounted) {
setCameraAccessState('blocked');
}
}
};
void startCamera();
return () => {
isMounted = false;
const stream = cameraStreamRef.current;
cameraStreamRef.current = null;
if (stream) {
stream.getTracks().forEach((track) => track.stop());
}
videoElement.srcObject = null;
};
}, []);
useEffect(() => {
const stream = cameraStreamRef.current;
const videoElement = cameraVideoRef.current;
if (stream && videoElement && videoElement.srcObject !== stream) {
videoElement.srcObject = stream;
}
}, [cameraAccessState]);
useEffect(() => {
holdCompletionRef.current = false;
setHoldStartedAt(null);
setLeftHandPath([]);
setRightHandPath([]);
}, [stepId]);
useEffect(() => {
if (step.kind !== 'position') {
return;
}
if (!isAvatarOnWarmupTarget(step, avatarX)) {
setHoldStartedAt(null);
holdCompletionRef.current = false;
return;
}
setHoldStartedAt((current) => current ?? Date.now());
}, [avatarX, step]);
useEffect(() => {
if (
step.kind !== 'position' ||
holdStartedAt === null ||
holdCompletionRef.current ||
nowMs - holdStartedAt < CHILD_MOTION_HOLD_DURATION_MS
) {
return;
}
holdCompletionRef.current = true;
completeStep({ type: 'position', avatarX });
}, [avatarX, completeStep, holdStartedAt, nowMs, step.kind]);
useEffect(() => {
if (step.kind !== 'narration' && step.kind !== 'finish') {
return;
}
const timeout = window.setTimeout(
() => completeStep({ type: 'narration' }),
step.kind === 'finish'
? CHILD_MOTION_FINISH_DURATION_MS
: CHILD_MOTION_NARRATION_DURATION_MS,
);
return () => window.clearTimeout(timeout);
}, [completeStep, step.kind]);
useEffect(() => {
if (step.kind !== 'gesture' || !mocapInput.latestCommand) {
return;
}
const command = mocapInput.latestCommand;
const packetKey =
mocapInput.rawPacketPreview?.receivedAtMs !== undefined
? `${mocapInput.rawPacketPreview.receivedAtMs}:${mocapInput.rawPacketPreview.text}`
: JSON.stringify(command);
if (handledMocapPacketKeyRef.current === packetKey) {
return;
}
const primaryPoint = mocapHandToChildMotionPoint(command.primaryHand);
const primaryHandSide = command.primaryHand?.side ?? 'unknown';
const fallbackPrimaryToLeft =
Boolean(primaryPoint) &&
!command.leftHand &&
(primaryHandSide === 'left' ||
primaryHandSide === 'unknown' ||
stepId === 'wave_left_hand' ||
stepId === 'wave_greeting');
const fallbackPrimaryToRight =
Boolean(primaryPoint) &&
!command.rightHand &&
(primaryHandSide === 'right' ||
stepId === 'wave_right_hand');
const leftPoint =
mocapHandToChildMotionPoint(command.leftHand) ??
(fallbackPrimaryToLeft ? primaryPoint : null);
const rightPoint =
mocapHandToChildMotionPoint(command.rightHand) ??
(fallbackPrimaryToRight ? primaryPoint : null);
const nextLeftHandPath = leftPoint
? appendWarmupMocapPoint(leftHandPath, leftPoint)
: leftHandPath;
const nextRightHandPath = rightPoint
? appendWarmupMocapPoint(rightHandPath, rightPoint)
: rightHandPath;
const nextPrimaryHandPath = primaryPoint
? command.primaryHand?.side === 'right'
? nextRightHandPath
: nextLeftHandPath
: [];
handledMocapPacketKeyRef.current = packetKey;
if (leftPoint) {
setLeftHandPath(nextLeftHandPath);
}
if (rightPoint) {
setRightHandPath(nextRightHandPath);
}
const intent = resolveWarmupMocapGestureIntent(stepId, command, {
leftHandPath: nextLeftHandPath,
rightHandPath: nextRightHandPath,
primaryHandPath: nextPrimaryHandPath,
});
if (!intent) {
return;
}
if (intent === 'jump') {
setIsJumping(true);
window.setTimeout(() => setIsJumping(false), 360);
completeStep({ type: 'jump', jumpSpace: 0.14 });
return;
}
if (intent === 'right-hand') {
const path = [...nextRightHandPath, rightPoint ?? primaryPoint].filter(
(point): point is ChildMotionPoint => Boolean(point),
);
completeStep({ type: 'right-hand', path: path.slice(-16) });
return;
}
const path = [...nextLeftHandPath, leftPoint ?? primaryPoint].filter(
(point): point is ChildMotionPoint => Boolean(point),
);
completeStep({ type: 'left-hand', path: path.slice(-16) });
}, [
completeStep,
leftHandPath,
mocapInput.latestCommand,
mocapInput.rawPacketPreview?.receivedAtMs,
mocapInput.rawPacketPreview?.text,
rightHandPath,
step.kind,
stepId,
]);
useEffect(() => {
if (!mocapInput.latestCommand) {
return;
}
const nextAvatarX = resolveAvatarXFromMocap(mocapInput.latestCommand);
if (nextAvatarX === null) {
return;
}
setAvatarX(nextAvatarX);
}, [
mocapInput.latestCommand,
mocapInput.rawPacketPreview?.receivedAtMs,
mocapInput.rawPacketPreview?.text,
]);
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
if (event.repeat) {
return;
}
const key = event.key.toLowerCase();
if (key === 'a') {
setAvatarX(0.34);
return;
}
if (key === 'd') {
setAvatarX(0.66);
return;
}
if (event.code === 'Space') {
event.preventDefault();
setIsJumping(true);
window.setTimeout(() => setIsJumping(false), 360);
if (stepId === 'jump_once') {
completeStep({ type: 'jump', jumpSpace: 0.14 });
}
}
};
window.addEventListener('keydown', handleKeyDown);
return () => window.removeEventListener('keydown', handleKeyDown);
}, [completeStep, stepId]);
useEffect(() => {
const handleKeyUp = (event: KeyboardEvent) => {
const key = event.key.toLowerCase();
if (key === 'a' || key === 'd' || event.code === 'KeyA' || event.code === 'KeyD') {
setAvatarX(CHILD_MOTION_CENTER_X);
}
};
window.addEventListener('keyup', handleKeyUp);
return () => window.removeEventListener('keyup', handleKeyUp);
}, []);
const handleStagePointerDown = (event: ReactPointerEvent<HTMLElement>) => {
if (event.button !== 0 && event.button !== 2) {
return;
}
event.preventDefault();
const nextHand: DragHand = event.button === 2 ? 'right' : 'left';
setActiveHand(nextHand);
const point = normalizePointerPoint(event, event.currentTarget);
if (nextHand === 'left') {
setLeftHandPath([point]);
} else {
setRightHandPath([point]);
}
event.currentTarget.setPointerCapture(event.pointerId);
};
const handleStagePointerMove = (event: ReactPointerEvent<HTMLElement>) => {
if (!activeHand) {
return;
}
const point = normalizePointerPoint(event, event.currentTarget);
const appendPoint = (points: ChildMotionPoint[]) =>
[...points, point].slice(-16);
if (activeHand === 'left') {
setLeftHandPath(appendPoint);
} else {
setRightHandPath(appendPoint);
}
};
const handleStagePointerUp = (event: ReactPointerEvent<HTMLElement>) => {
if (!activeHand) {
return;
}
if (event.currentTarget.hasPointerCapture(event.pointerId)) {
event.currentTarget.releasePointerCapture(event.pointerId);
}
const hand = activeHand;
const point = normalizePointerPoint(event, event.currentTarget);
const completedPath =
hand === 'left'
? [...leftHandPath, point].slice(-16)
: [...rightHandPath, point].slice(-16);
setActiveHand(null);
if (stepId === 'wave_greeting') {
completeStep({ type: 'left-hand', path: completedPath });
return;
}
if (stepId === 'wave_left_hand' && hand === 'left') {
completeStep({ type: 'left-hand', path: completedPath });
return;
}
if (stepId === 'wave_right_hand' && hand === 'right') {
completeStep({ type: 'right-hand', path: completedPath });
}
};
const handleStartPlaceholderLevel = () => {
setStepId('play_placeholder');
};
const handleReturnToStart = () => {
setStepId('level_select');
};
const lineText = useMemo(() => step.spokenLines.join('，'), [step.spokenLines]);
return (
<main className="child-motion-demo" data-testid="child-motion-demo">
<div className="child-motion-orientation-tip" role="status">
请横屏体验
</div>
<section
className="child-motion-stage"
data-testid="child-motion-stage"
onPointerDown={handleStagePointerDown}
onPointerMove={handleStagePointerMove}
onPointerUp={handleStagePointerUp}
onPointerCancel={handleStagePointerUp}
onContextMenu={(event) => event.preventDefault()}
>
<video
ref={cameraVideoRef}
className="child-motion-camera-layer"
aria-hidden="true"
autoPlay
muted
playsInline
/>
{motionSourceState !== 'ready' ? (
<div
className={`child-motion-camera-state child-motion-camera-state--${motionSourceState}`}
aria-live="polite"
>
{motionSourceText}
</div>
) : null}
<div className="child-motion-floor" aria-hidden="true" />
{targetX !== null && step.kind === 'position' ? (
<ChildMotionRing targetX={targetX} progress={holdProgress} />
) : null}
{step.kind === 'gesture' ? (
<ChildMotionGestureGuide
stepId={stepId}
leftHandPath={leftHandPath}
rightHandPath={rightHandPath}
/>
) : null}
<ChildMotionAvatar avatarX={avatarX} isJumping={isJumping} />
{justCompletedText ? (
<div className="child-motion-floating-reward">{justCompletedText}</div>
) : null}
<div className="child-motion-hud child-motion-hud--top">
<span className="child-motion-step-count">{`${Math.min(stepIndex + 1, 12)}/12`}</span>
<div>
<h1>{step.title}</h1>
<p>{lineText}</p>
</div>
<span className="child-motion-progress">{progressPercent}%</span>
</div>
{step.kind === 'levelSelect' ? (
<div className="child-motion-start-panel">
<button type="button" onClick={handleStartPlaceholderLevel}>
开始游戏
</button>
</div>
) : null}
{step.kind === 'placeholder' ? (
<div className="child-motion-start-panel">
<span>下一关正在设计中</span>
<button type="button" onClick={handleReturnToStart}>
回到开始
</button>
</div>
) : null}
<ChildMotionCalibrationPanel calibration={calibration} />
</section>
</main>
);
}
export default ChildMotionWarmupDemo;

85
src/components/child-motion-demo/childMotionWarmupModel.test.ts Normal file
View File

@@ -0,0 +1,85 @@
import { describe, expect, it } from 'vitest';
import {
applyChildMotionWarmupCompletion,
CHILD_MOTION_CENTER_X,
CHILD_MOTION_WARMUP_STEPS,
createEmptyChildMotionCalibration,
getChildMotionWarmupStep,
isAvatarOnWarmupTarget,
resolveNextChildMotionWarmupStep,
} from './childMotionWarmupModel';
describe('childMotionWarmupModel', () => {
it('keeps the confirmed warmup order as a strict state chain', () => {
expect(CHILD_MOTION_WARMUP_STEPS.map((step) => step.id)).toEqual([
'center_arrive',
'wave_greeting',
'warmup_intro',
'move_left',
'return_center_1',
'move_right',
'return_center_2',
'wave_left_hand',
'wave_right_hand',
'jump_once',
'warmup_finish',
'level_select',
'play_placeholder',
]);
expect(resolveNextChildMotionWarmupStep('center_arrive')).toBe(
'wave_greeting',
);
expect(resolveNextChildMotionWarmupStep('level_select')).toBe(
'play_placeholder',
);
});
it('checks position completion against the active green ring target', () => {
expect(
isAvatarOnWarmupTarget(
getChildMotionWarmupStep('center_arrive'),
CHILD_MOTION_CENTER_X,
),
).toBe(true);
expect(
isAvatarOnWarmupTarget(getChildMotionWarmupStep('move_left'), 0.66),
).toBe(false);
});
it('records session-only calibration values from completed steps', () => {
const empty = createEmptyChildMotionCalibration();
const withLeft = applyChildMotionWarmupCompletion('move_left', empty, {
type: 'position',
avatarX: 0.34,
});
const withRight = applyChildMotionWarmupCompletion('move_right', withLeft, {
type: 'position',
avatarX: 0.66,
});
const withLeftHand = applyChildMotionWarmupCompletion(
'wave_left_hand',
withRight,
{
type: 'left-hand',
path: [
{ x: 0.3, y: 0.4 },
{ x: 0.34, y: 0.32 },
],
},
);
const completed = applyChildMotionWarmupCompletion(
'jump_once',
withLeftHand,
{
type: 'jump',
jumpSpace: 0.14,
},
);
expect(completed.leftBoundary).toBeCloseTo(0.16);
expect(completed.rightBoundary).toBeCloseTo(0.16);
expect(completed.leftHandPath).toHaveLength(2);
expect(completed.jumpSpace).toBe(0.14);
});
});

274
src/components/child-motion-demo/childMotionWarmupModel.ts Normal file
View File

@@ -0,0 +1,274 @@
export type ChildMotionWarmupStepId =
| 'center_arrive'
| 'wave_greeting'
| 'warmup_intro'
| 'move_left'
| 'return_center_1'
| 'move_right'
| 'return_center_2'
| 'wave_left_hand'
| 'wave_right_hand'
| 'jump_once'
| 'warmup_finish'
| 'level_select'
| 'play_placeholder';
export type ChildMotionWarmupTarget = 'center' | 'left' | 'right';
export type ChildMotionWarmupStepKind =
| 'position'
| 'gesture'
| 'narration'
| 'finish'
| 'levelSelect'
| 'placeholder';
export type ChildMotionWarmupStep = {
id: ChildMotionWarmupStepId;
kind: ChildMotionWarmupStepKind;
title: string;
spokenLines: string[];
target?: ChildMotionWarmupTarget;
};
export type ChildMotionPoint = {
x: number;
y: number;
};
export type ChildMotionWarmupCalibration = {
leftBoundary: number | null;
rightBoundary: number | null;
leftHandPath: ChildMotionPoint[];
rightHandPath: ChildMotionPoint[];
jumpSpace: number | null;
};
export type ChildMotionWarmupCompletion =
| {
type: 'position';
avatarX: number;
}
| {
type: 'left-hand';
path: ChildMotionPoint[];
}
| {
type: 'right-hand';
path: ChildMotionPoint[];
}
| {
type: 'jump';
jumpSpace: number;
}
| {
type: 'narration';
};
export const CHILD_MOTION_CENTER_X = 0.5;
export const CHILD_MOTION_LEFT_X = 0.34;
export const CHILD_MOTION_RIGHT_X = 0.66;
export const CHILD_MOTION_POSITION_EPSILON = 0.045;
export const CHILD_MOTION_HOLD_DURATION_MS = 2000;
export const CHILD_MOTION_NARRATION_DURATION_MS = 900;
export const CHILD_MOTION_FINISH_DURATION_MS = 1200;
export const CHILD_MOTION_WARMUP_STEPS: ChildMotionWarmupStep[] = [
{
id: 'center_arrive',
kind: 'position',
title: '来到圆圈这里',
spokenLines: ['欢迎你，小朋友，见到你真开心', '请你来到圆圈这里和我打个招呼吧'],
target: 'center',
},
{
id: 'wave_greeting',
kind: 'gesture',
title: '打个招呼',
spokenLines: ['请你来到圆圈这里和我打个招呼吧'],
},
{
id: 'warmup_intro',
kind: 'narration',
title: '准备热身',
spokenLines: ['你好呀小朋友，为了你玩的安全和开心，先来和我一起热个身吧'],
},
{
id: 'move_left',
kind: 'position',
title: '向左一步',
spokenLines: ['向左一步'],
target: 'left',
},
{
id: 'return_center_1',
kind: 'position',
title: '回到中间来',
spokenLines: ['回到中间来'],
target: 'center',
},
{
id: 'move_right',
kind: 'position',
title: '向右一步',
spokenLines: ['向右一步'],
target: 'right',
},
{
id: 'return_center_2',
kind: 'position',
title: '回到中间来',
spokenLines: ['回到中间来'],
target: 'center',
},
{
id: 'wave_left_hand',
kind: 'gesture',
title: '挥动左手',
spokenLines: ['挥动左手'],
},
{
id: 'wave_right_hand',
kind: 'gesture',
title: '挥动右手',
spokenLines: ['挥动右手'],
},
{
id: 'jump_once',
kind: 'gesture',
title: '原地跳一下',
spokenLines: ['原地跳一下'],
},
{
id: 'warmup_finish',
kind: 'finish',
title: '热身完成',
spokenLines: ['真厉害，你是我见过最聪明的小朋友', '别走开，现在开始我们的游戏吧'],
},
{
id: 'level_select',
kind: 'levelSelect',
title: '准备开始',
spokenLines: ['现在开始我们的游戏吧'],
},
{
id: 'play_placeholder',
kind: 'placeholder',
title: '下一关',
spokenLines: ['游戏关卡正在准备中'],
},
];
const STEP_BY_ID = new Map(
CHILD_MOTION_WARMUP_STEPS.map((step) => [step.id, step]),
);
const NEXT_STEP_BY_ID = new Map<ChildMotionWarmupStepId, ChildMotionWarmupStepId>(
CHILD_MOTION_WARMUP_STEPS.slice(0, -1).map((step, index) => [
step.id,
CHILD_MOTION_WARMUP_STEPS[index + 1]!.id,
]),
);
let childMotionWarmupCompletedInRuntime = false;
export function getChildMotionWarmupStep(id: ChildMotionWarmupStepId) {
return STEP_BY_ID.get(id) ?? CHILD_MOTION_WARMUP_STEPS[0]!;
}
export function getChildMotionTargetX(target: ChildMotionWarmupTarget) {
if (target === 'left') {
return CHILD_MOTION_LEFT_X;
}
if (target === 'right') {
return CHILD_MOTION_RIGHT_X;
}
return CHILD_MOTION_CENTER_X;
}
export function isAvatarOnWarmupTarget(
step: ChildMotionWarmupStep,
avatarX: number,
) {
if (step.kind !== 'position' || !step.target) {
return false;
}
return (
Math.abs(avatarX - getChildMotionTargetX(step.target)) <=
CHILD_MOTION_POSITION_EPSILON
);
}
export function resolveNextChildMotionWarmupStep(
stepId: ChildMotionWarmupStepId,
) {
return NEXT_STEP_BY_ID.get(stepId) ?? stepId;
}
export function createEmptyChildMotionCalibration(): ChildMotionWarmupCalibration {
return {
leftBoundary: null,
rightBoundary: null,
leftHandPath: [],
rightHandPath: [],
jumpSpace: null,
};
}
export function applyChildMotionWarmupCompletion(
stepId: ChildMotionWarmupStepId,
calibration: ChildMotionWarmupCalibration,
completion: ChildMotionWarmupCompletion,
): ChildMotionWarmupCalibration {
if (stepId === 'move_left' && completion.type === 'position') {
return {
...calibration,
leftBoundary: Math.max(0, CHILD_MOTION_CENTER_X - completion.avatarX),
};
}
if (stepId === 'move_right' && completion.type === 'position') {
return {
...calibration,
rightBoundary: Math.max(0, completion.avatarX - CHILD_MOTION_CENTER_X),
};
}
if (stepId === 'wave_left_hand' && completion.type === 'left-hand') {
return {
...calibration,
leftHandPath: completion.path,
};
}
if (stepId === 'wave_right_hand' && completion.type === 'right-hand') {
return {
...calibration,
rightHandPath: completion.path,
};
}
if (stepId === 'jump_once' && completion.type === 'jump') {
return {
...calibration,
jumpSpace: completion.jumpSpace,
};
}
return calibration;
}
export function hasCompletedChildMotionWarmupInRuntime() {
return childMotionWarmupCompletedInRuntime;
}
export function markChildMotionWarmupCompletedInRuntime() {
childMotionWarmupCompletedInRuntime = true;
}
export function resetChildMotionWarmupRuntimeSession() {
childMotionWarmupCompletedInRuntime = false;
}

154
src/components/platform-entry/PlatformEntryFlowShellImpl.tsx
View File

@@ -291,6 +291,11 @@ import {
type VisualNovelEntryFormPayload,
} from '../visual-novel-creation/VisualNovelAgentWorkspace';
import { createMockVisualNovelRunFromDraft } from '../visual-novel-runtime/visualNovelMockData';
import {
canExposePublicWork,
EDUTAINMENT_HIDDEN_MESSAGE,
filterGeneralPublicWorks,
} from './platformEdutainmentVisibility';
import { PlatformEntryCreationTypeModal } from './PlatformEntryCreationTypeModal';
import type { PlatformCreationTypeId } from './platformEntryCreationTypes';
import {
@@ -865,6 +870,19 @@ function shouldUseLocalPuzzleOnboardingFallback(error: unknown) {
);
}
function isMissingPuzzleWorkError(error: unknown) {
return (
(error instanceof ApiClientError &&
error.status === 404 &&
(error.code === 'NOT_FOUND' ||
error.message.includes('资源不存在') ||
error.message.includes('未找到'))) ||
(error instanceof Error &&
(error.message.includes('资源不存在') ||
error.message.includes('未找到拼图作品')))
);
}
function hasSeenPuzzleOnboarding() {
if (typeof window === 'undefined') {
return true;
@@ -2165,7 +2183,10 @@ export function PlatformEntryFlowShellImpl({
const recommendRuntimeEntries = useMemo(
() => {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
[...featuredGalleryEntries, ...latestGalleryEntries].forEach((entry) => {
filterGeneralPublicWorks([
...featuredGalleryEntries,
...latestGalleryEntries,
]).forEach((entry) => {
entryMap.set(getPlatformPublicGalleryEntryKey(entry), entry);
});
return Array.from(entryMap.values());
@@ -4244,6 +4265,19 @@ export function PlatformEntryFlowShellImpl({
}
return true;
} catch (error) {
if (isMissingPuzzleWorkError(error)) {
setSelectedPuzzleDetail(null);
setPuzzleDetailReturnTarget(null);
setPuzzleRun(null);
setPuzzleRuntimeAuthMode('default');
setPuzzleError(null);
setPublicWorkDetailError(null);
setPlatformTab('home');
setSelectionStage('platform');
pushAppHistoryPath('/');
return false;
}
const message = resolvePuzzleErrorMessage(error, '启动拼图玩法失败。');
setPuzzleError(message);
if (mirrorErrorToPublicDetail) {
@@ -4259,8 +4293,8 @@ export function PlatformEntryFlowShellImpl({
resolvePuzzleErrorMessage,
setIsPuzzleBusy,
setPuzzleError,
setPlatformTab,
setSelectionStage,
startPuzzleRun,
],
);
@@ -5421,6 +5455,13 @@ export function PlatformEntryFlowShellImpl({
const openPublicWorkDetail = useCallback(
(entry: PlatformPublicGalleryCard) => {
if (!canExposePublicWork(entry)) {
setSelectedPublicWorkDetail(null);
setPublicWorkDetailError(EDUTAINMENT_HIDDEN_MESSAGE);
setSelectionStage('platform');
return;
}
setSelectedPublicWorkDetail(entry);
setPublicWorkDetailError(null);
setSelectionStage('work-detail');
@@ -5648,6 +5689,13 @@ export function PlatformEntryFlowShellImpl({
const openRpgPublicWorkDetail = useCallback(
async (entry: CustomWorldGalleryCard) => {
if (!canExposePublicWork(entry)) {
setSelectedPublicWorkDetail(null);
setPublicWorkDetailError(EDUTAINMENT_HIDDEN_MESSAGE);
setSelectionStage('platform');
return;
}
setIsPublicWorkDetailBusy(true);
setPublicWorkDetailError(null);
clearSelectedPublicWorkAuthor();
@@ -5659,6 +5707,14 @@ export function PlatformEntryFlowShellImpl({
await detailNavigation.loadGalleryDetailEntry(entry);
setSelectedDetailEntry(detailEntry);
const detailCard = mapRpgGalleryCardToPublicWorkDetail(detailEntry);
if (!canExposePublicWork(detailCard)) {
setSelectedDetailEntry(null);
setSelectedPublicWorkDetail(null);
setPublicWorkDetailError(EDUTAINMENT_HIDDEN_MESSAGE);
setSelectionStage('platform');
return;
}
setSelectedPublicWorkDetail(detailCard);
if (detailEntry.publicWorkCode?.trim()) {
pushAppHistoryPath(
@@ -5697,10 +5753,31 @@ export function PlatformEntryFlowShellImpl({
try {
const { item } = await getPuzzleGalleryDetail(profileId);
const detailEntry = mapPuzzleWorkToPublicWorkDetail(item);
if (!canExposePublicWork(detailEntry)) {
setSelectedPuzzleDetail(null);
setPublicWorkDetailError(EDUTAINMENT_HIDDEN_MESSAGE);
setSelectionStage('platform');
return;
}
setSelectedPuzzleDetail(item);
setPuzzleDetailReturnTarget(returnTarget);
openPublicWorkDetail(mapPuzzleWorkToPublicWorkDetail(item));
openPublicWorkDetail(detailEntry);
} catch (error) {
if (isMissingPuzzleWorkError(error)) {
setSelectedPuzzleDetail(null);
setPuzzleDetailReturnTarget(null);
setPuzzleRun(null);
setPuzzleRuntimeAuthMode('default');
setPuzzleError(null);
setPublicWorkDetailError(null);
setPlatformTab('home');
setSelectionStage('platform');
pushAppHistoryPath('/');
return;
}
setPublicWorkDetailError(
resolvePuzzleErrorMessage(error, '读取拼图详情失败。'),
);
@@ -5715,6 +5792,7 @@ export function PlatformEntryFlowShellImpl({
resolvePuzzleErrorMessage,
setIsPuzzleBusy,
setPuzzleError,
setPlatformTab,
setSelectionStage,
],
);
@@ -5906,6 +5984,19 @@ export function PlatformEntryFlowShellImpl({
),
);
} catch (error) {
if (isMissingPuzzleWorkError(error)) {
setSelectedPuzzleDetail(null);
setPuzzleDetailReturnTarget(null);
setPuzzleRun(null);
setPuzzleRuntimeAuthMode('default');
setPuzzleError(null);
setPublicWorkDetailError(null);
setPlatformTab('home');
setSelectionStage('platform');
pushAppHistoryPath('/');
return;
}
setPuzzleError(resolvePuzzleErrorMessage(error, '读取拼图详情失败。'));
} finally {
setIsPuzzleBusy(false);
@@ -5916,6 +6007,7 @@ export function PlatformEntryFlowShellImpl({
resolvePuzzleErrorMessage,
setIsPuzzleBusy,
setPuzzleError,
setPlatformTab,
setSelectionStage,
],
);
@@ -6710,11 +6802,14 @@ export function PlatformEntryFlowShellImpl({
match3dError,
match3dFlow,
match3dRun,
platformBootstrap.platformTab,
platformThemeClass,
puzzleError,
puzzleRun,
recommendRuntimeEntries,
remodelCurrentPuzzleRuntimeWork,
resolveMatch3DErrorMessage,
resolveSquareHoleErrorMessage,
reportBigFishObservedPlayTime,
restartBigFishRun,
selectedPuzzleDetail,
@@ -7063,6 +7158,10 @@ export function PlatformEntryFlowShellImpl({
remixCount: entry.remixCount ?? 0,
likeCount: entry.likeCount ?? 0,
} satisfies CustomWorldGalleryCard;
if (!canExposePublicWork(card)) {
throw new Error(EDUTAINMENT_HIDDEN_MESSAGE);
}
setSelectedDetailEntry(entry);
openPublicWorkDetail(card);
};
@@ -7071,9 +7170,12 @@ export function PlatformEntryFlowShellImpl({
puzzleGalleryEntries.length > 0
? puzzleGalleryEntries
: await refreshPuzzleGallery();
const matchedEntry = entries.find((entry) =>
isSamePuzzlePublicWorkCode(normalizedKeyword, entry.profileId),
);
const matchedEntry = entries
.map(mapPuzzleWorkToPublicWorkDetail)
.filter(canExposePublicWork)
.find((entry) =>
isSamePuzzlePublicWorkCode(normalizedKeyword, entry.profileId),
);
if (!matchedEntry) {
throw new Error('未找到拼图作品。');
@@ -7088,9 +7190,13 @@ export function PlatformEntryFlowShellImpl({
bigFishGalleryEntries.length > 0
? bigFishGalleryEntries
: await refreshBigFishGallery();
const matchedEntry = entries.find((entry) =>
isSameBigFishPublicWorkCode(normalizedKeyword, entry.sourceSessionId),
);
const matchedEntry = entries.find((entry) => {
const detailEntry = mapBigFishWorkToPublicWorkDetail(entry);
return (
canExposePublicWork(detailEntry) &&
isSameBigFishPublicWorkCode(normalizedKeyword, entry.sourceSessionId)
);
});
if (!matchedEntry) {
throw new Error('未找到大鱼吃小鱼作品。');
@@ -7103,9 +7209,13 @@ export function PlatformEntryFlowShellImpl({
match3dGalleryEntries.length > 0
? match3dGalleryEntries
: await refreshMatch3DGallery();
const matchedEntry = entries.find((entry) =>
isSameMatch3DPublicWorkCode(normalizedKeyword, entry.profileId),
);
const matchedEntry = entries.find((entry) => {
const detailEntry = mapMatch3DWorkToPublicWorkDetail(entry);
return (
canExposePublicWork(detailEntry) &&
isSameMatch3DPublicWorkCode(normalizedKeyword, entry.profileId)
);
});
if (!matchedEntry) {
throw new Error('未找到抓大鹅作品。');
@@ -7118,9 +7228,13 @@ export function PlatformEntryFlowShellImpl({
squareHoleGalleryEntries.length > 0
? squareHoleGalleryEntries
: await refreshSquareHoleGallery();
const matchedEntry = entries.find((entry) =>
isSameSquareHolePublicWorkCode(normalizedKeyword, entry.profileId),
);
const matchedEntry = entries.find((entry) => {
const detailEntry = mapSquareHoleWorkToPublicWorkDetail(entry);
return (
canExposePublicWork(detailEntry) &&
isSameSquareHolePublicWorkCode(normalizedKeyword, entry.profileId)
);
});
if (!matchedEntry) {
throw new Error('未找到方洞挑战作品。');
@@ -7133,9 +7247,13 @@ export function PlatformEntryFlowShellImpl({
visualNovelGalleryEntries.length > 0
? visualNovelGalleryEntries
: await refreshVisualNovelGallery();
const matchedEntry = entries.find((entry) =>
isSameVisualNovelPublicWorkCode(normalizedKeyword, entry.profileId),
);
const matchedEntry = entries.find((entry) => {
const detailEntry = mapVisualNovelWorkToPublicWorkDetail(entry);
return (
canExposePublicWork(detailEntry) &&
isSameVisualNovelPublicWorkCode(normalizedKeyword, entry.profileId)
);
});
if (!matchedEntry) {
throw new Error('未找到视觉小说作品。');

59
src/components/platform-entry/platformEdutainmentVisibility.test.ts Normal file
View File

@@ -0,0 +1,59 @@
import { afterEach, describe, expect, test, vi } from 'vitest';
import type { PlatformPublicGalleryCard } from '../rpg-entry/rpgEntryWorldPresentation';
import {
canExposePublicWork,
filterEdutainmentPublicWorks,
filterGeneralPublicWorks,
isEdutainmentEntryEnabled,
isEdutainmentPublicWork,
} from './platformEdutainmentVisibility';
function buildPuzzleCard(themeTags: string[]): PlatformPublicGalleryCard {
return {
sourceType: 'puzzle',
workId: 'puzzle-work-education-demo',
profileId: 'puzzle-profile-education-demo',
publicWorkCode: 'PZ-EDUDEMO',
ownerUserId: 'user-education',
authorDisplayName: '动作 Demo 作者',
worldName: '儿童动作热身 Demo',
subtitle: '拼图关卡',
summaryText: '本地动作 Demo。',
coverImageSrc: null,
themeTags,
visibility: 'published',
publishedAt: '2026-05-09T10:00:00.000Z',
updatedAt: '2026-05-09T10:00:00.000Z',
};
}
afterEach(() => {
vi.unstubAllEnvs();
});
describe('platformEdutainmentVisibility', () => {
test('matches only the exact edutainment tag from full work tags', () => {
const exact = buildPuzzleCard(['运动', '安全', '拼图', '寓教于乐']);
const fuzzy = buildPuzzleCard(['儿童教育', '寓教于乐 ']);
expect(isEdutainmentPublicWork(exact)).toBe(true);
expect(isEdutainmentPublicWork(fuzzy)).toBe(false);
expect(filterEdutainmentPublicWorks([exact, fuzzy])).toEqual([exact]);
expect(filterGeneralPublicWorks([exact, fuzzy])).toEqual([fuzzy]);
});
test('defaults to enabled and blocks exact edutainment works only when disabled', () => {
const exact = buildPuzzleCard(['寓教于乐']);
const general = buildPuzzleCard(['儿童教育']);
expect(isEdutainmentEntryEnabled()).toBe(true);
expect(canExposePublicWork(exact)).toBe(true);
vi.stubEnv('VITE_ENABLE_EDUTAINMENT_ENTRY', 'false');
expect(isEdutainmentEntryEnabled()).toBe(false);
expect(canExposePublicWork(exact)).toBe(false);
expect(canExposePublicWork(general)).toBe(true);
});
});

58
src/components/platform-entry/platformEdutainmentVisibility.ts Normal file
View File

@@ -0,0 +1,58 @@
import type { PlatformBrowseHistoryEntry } from '../../../packages/shared/src/contracts/runtime';
import type { PlatformPublicGalleryCard } from '../rpg-entry/rpgEntryWorldPresentation';
export const EDUTAINMENT_WORK_TAG = '寓教于乐';
export const EDUTAINMENT_HIDDEN_MESSAGE = '该内容暂不可见。';
const EDUTAINMENT_ENTRY_DISABLED_VALUES = new Set(['false', '0', 'off', 'no']);
// 中文注释：入口默认开启；只有明确写入关闭值时才完全隐藏寓教于乐内容。
export function isEdutainmentEntryEnabled(
rawValue = import.meta.env.VITE_ENABLE_EDUTAINMENT_ENTRY,
) {
const normalized = (rawValue ?? '').trim().toLowerCase();
return !EDUTAINMENT_ENTRY_DISABLED_VALUES.has(normalized);
}
function getPlatformPublicWorkTags(entry: PlatformPublicGalleryCard) {
if ('themeTags' in entry) {
return entry.themeTags;
}
return [];
}
export function isEdutainmentPublicWork(entry: PlatformPublicGalleryCard) {
return getPlatformPublicWorkTags(entry).some(
(tag) => tag === EDUTAINMENT_WORK_TAG,
);
}
export function canExposePublicWork(entry: PlatformPublicGalleryCard) {
return isEdutainmentEntryEnabled() || !isEdutainmentPublicWork(entry);
}
export function filterGeneralPublicWorks(entries: PlatformPublicGalleryCard[]) {
return entries.filter((entry) => !isEdutainmentPublicWork(entry));
}
export function filterEdutainmentPublicWorks(
entries: PlatformPublicGalleryCard[],
) {
return entries.filter(isEdutainmentPublicWork);
}
export function filterVisiblePublicWorks(entries: PlatformPublicGalleryCard[]) {
return entries.filter(canExposePublicWork);
}
export function findPublicWorkForHistoryEntry(
historyEntry: PlatformBrowseHistoryEntry,
entries: PlatformPublicGalleryCard[],
) {
return entries.find(
(entry) =>
entry.ownerUserId === historyEntry.ownerUserId &&
entry.profileId === historyEntry.profileId,
);
}

149
src/components/puzzle-runtime/PuzzleRuntimeShell.test.tsx
View File

@@ -36,7 +36,7 @@ vi.mock('../../services/useMocapInput', () => ({
status: 'connected',
latestCommand: {
actions: [mocapMock.state],
primaryHand: {x: mocapMock.x, y: mocapMock.y, state: mocapMock.state},
primaryHand: {x: mocapMock.x, y: mocapMock.y, state: mocapMock.state, source: 'palm_center'},
parseWarnings: [],
},
rawPacketPreview: {text: '{"hands":[{"state":"grab"}]}', receivedAtMs: 1},
@@ -207,9 +207,11 @@ test('拼图界面在 mocap open_palm 时显示体感光标', () => {
const cursor = screen.getByTestId('puzzle-mocap-cursor');
expect(cursor).toBeTruthy();
expect(cursor.style.left).toBe('42%');
expect(cursor.style.top).toBe('58%');
expect(Number.parseFloat(cursor.style.left)).toBeCloseTo(42);
expect(Number.parseFloat(cursor.style.top)).toBeCloseTo(58);
mocapMock.state = 'grab';
mocapMock.x = 0.42;
mocapMock.y = 0.58;
});
test('抓握时会触发拖拽提交并在松开时落子', () => {
@@ -302,6 +304,144 @@ test('抓握时会触发拖拽提交并在松开时落子', () => {
);
});
test('mocap 抓握合并大块时按大块锚点提交拖拽', () => {
const originalRequestAnimationFrame = window.requestAnimationFrame;
const originalCancelAnimationFrame = window.cancelAnimationFrame;
mocapMock.state = 'open_palm';
mocapMock.x = 0.2;
mocapMock.y = 0.2;
const onDragPiece = vi.fn();
const mergedRun: PuzzleRunSnapshot = {
...clearedRun,
currentLevel: {
...clearedRun.currentLevel!,
status: 'playing',
startedAtMs: Date.now(),
remainingMs: 300_000,
timeLimitMs: 300_000,
board: {
...clearedRun.currentLevel!.board,
allTilesResolved: false,
mergedGroups: [
{
groupId: 'group-large',
pieceIds: ['piece-0', 'piece-1', 'piece-3'],
occupiedCells: [
{ row: 0, col: 0 },
{ row: 0, col: 1 },
{ row: 1, col: 0 },
],
},
],
pieces: clearedRun.currentLevel!.board.pieces.map((piece) =>
['piece-0', 'piece-1', 'piece-3'].includes(piece.pieceId)
? { ...piece, mergedGroupId: 'group-large' }
: piece,
),
},
},
};
Object.defineProperty(window, 'requestAnimationFrame', {
configurable: true,
value: vi.fn(() => 1),
});
Object.defineProperty(window, 'cancelAnimationFrame', {
configurable: true,
value: vi.fn(),
});
const { container, rerender, unmount } = renderPuzzleRuntime(
<PuzzleRuntimeShell
run={mergedRun}
onBack={vi.fn()}
onSwapPieces={vi.fn()}
onDragPiece={onDragPiece}
onAdvanceNextLevel={vi.fn()}
/>,
);
const board = container.querySelector(
'[data-testid="puzzle-board"]',
) as HTMLElement | null;
if (!board) {
throw new Error('缺少测试棋盘');
}
board.getBoundingClientRect = () =>
({
x: 0,
y: 0,
left: 0,
top: 0,
right: 300,
bottom: 300,
width: 300,
height: 300,
toJSON: () => ({}),
}) as DOMRect;
mocapMock.state = 'grab';
mocapMock.x = 0.2;
mocapMock.y = 0.2;
rerender(
<AuthUiContext.Provider value={createAuthValue()}>
<PuzzleRuntimeShell
run={mergedRun}
onBack={vi.fn()}
onSwapPieces={vi.fn()}
onDragPiece={onDragPiece}
onAdvanceNextLevel={vi.fn()}
/>
</AuthUiContext.Provider>,
);
mocapMock.x = 0.7;
mocapMock.y = 0.7;
rerender(
<AuthUiContext.Provider value={createAuthValue()}>
<PuzzleRuntimeShell
run={mergedRun}
onBack={vi.fn()}
onSwapPieces={vi.fn()}
onDragPiece={onDragPiece}
onAdvanceNextLevel={vi.fn()}
/>
</AuthUiContext.Provider>,
);
mocapMock.state = 'open_palm';
rerender(
<AuthUiContext.Provider value={createAuthValue()}>
<PuzzleRuntimeShell
run={mergedRun}
onBack={vi.fn()}
onSwapPieces={vi.fn()}
onDragPiece={onDragPiece}
onAdvanceNextLevel={vi.fn()}
/>
</AuthUiContext.Provider>,
);
expect(onDragPiece).toHaveBeenCalledTimes(1);
expect(onDragPiece).toHaveBeenCalledWith({
pieceId: 'piece-0',
targetRow: 2,
targetCol: 2,
});
unmount();
Object.defineProperty(window, 'requestAnimationFrame', {
configurable: true,
value: originalRequestAnimationFrame,
});
Object.defineProperty(window, 'cancelAnimationFrame', {
configurable: true,
value: originalCancelAnimationFrame,
});
mocapMock.state = 'grab';
mocapMock.x = 0.42;
mocapMock.y = 0.58;
});
test('通关后显示结算弹窗、排行榜和下一关按钮', () => {
vi.useFakeTimers();
const onAdvanceNextLevel = vi.fn();
@@ -822,6 +962,9 @@ test('移动端点击拼图片时立即触发一次震动反馈', () => {
const originalRequestAnimationFrame = window.requestAnimationFrame;
const originalCancelAnimationFrame = window.cancelAnimationFrame;
const vibrate = vi.fn();
mocapMock.state = 'open_palm';
mocapMock.x = 0.42;
mocapMock.y = 0.58;
const playingRun: PuzzleRunSnapshot = {
...clearedRun,
currentLevel: {

543
src/components/puzzle-runtime/PuzzleRuntimeShell.tsx
View File

@@ -23,6 +23,15 @@ import type {
SwapPuzzlePiecesRequest,
} from '../../../packages/shared/src/contracts/puzzleRuntimeSession';
import { useResolvedAssetReadUrl } from '../../hooks/useResolvedAssetReadUrl';
import {
createRuntimeDragInputController,
createRuntimeInputPointFromClient,
createRuntimeInputPointFromNormalized,
readRuntimeInputElementBounds,
resolveRuntimeInputGridCell,
type RuntimeDragInputSession,
type RuntimeInputPoint,
} from '../../services/input-devices';
import { useMocapInput } from '../../services/useMocapInput';
import { CHROME_ICONS, getNineSliceStyle, UI_CHROME } from '../../uiAssets';
import { useAuthUi } from '../auth/AuthUiContext';
@@ -211,6 +220,8 @@ const PUZZLE_HINT_DEMO_DURATION_MS = 1_250;
const PUZZLE_PIECE_PRESS_HAPTIC_PATTERN_MS = 12;
const PUZZLE_EXIT_REMODEL_PROMPT_STORAGE_PREFIX =
'genarrative.puzzle-runtime.exit-remodel-prompt.v1';
const PUZZLE_MOCAP_DRAG_INPUT_ID = 'mocap:primary-hand';
const PUZZLE_MOCAP_CURSOR_FRAME_MS = 1000 / 60;
const shownExitRemodelPromptProfileIds = new Set<string>();
@@ -290,6 +301,15 @@ type PuzzleMocapCursorState = {
state: string;
};
type PuzzleMocapCursorSample = PuzzleMocapCursorState & {
receivedAtMs: number;
};
type PuzzleRuntimeDragTargetState = {
pieceId: string;
groupId: string | null;
};
function triggerPuzzlePiecePressHapticFeedback() {
if (typeof navigator === 'undefined') {
return;
@@ -328,6 +348,8 @@ export function PuzzleRuntimeShell({
const mergedGroupSvgIdPrefix = sanitizeSvgId(useId());
const authUi = useAuthUi();
const [selectedPieceId, setSelectedPieceId] = useState<string | null>(null);
const selectedPieceIdRef = useRef<string | null>(null);
const selectedPieceBeforeInputRef = useRef<string | null>(null);
const [isSettingsPanelOpen, setIsSettingsPanelOpen] = useState(false);
const [isExitRemodelPromptOpen, setIsExitRemodelPromptOpen] =
useState(false);
@@ -354,7 +376,7 @@ export function PuzzleRuntimeShell({
const timeExpiredSyncKeyRef = useRef<string | null>(null);
const dragSessionRef = useRef<{
pieceId: string;
pointerId: number;
inputId: string;
dragging: boolean;
startX: number;
startY: number;
@@ -377,7 +399,18 @@ export function PuzzleRuntimeShell({
const [mocapCursor, setMocapCursor] = useState<PuzzleMocapCursorState | null>(
null,
);
const mocapDragRef = useRef<{pieceId: string} | null>(null);
const mocapCursorPreviousSampleRef = useRef<PuzzleMocapCursorSample | null>(
null,
);
const mocapCursorTargetSampleRef = useRef<PuzzleMocapCursorSample | null>(null);
const mocapCursorIntervalRef = useRef<number | null>(null);
const updateMocapCursorSampleRef = useRef<(
nextSample: PuzzleMocapCursorSample,
) => void>(() => {});
const runtimeDragInputControllerRef = useRef(
createRuntimeDragInputController<string>(),
);
const draggingTargetRef = useRef<PuzzleRuntimeDragTargetState | null>(null);
const [dismissedClearKey, setDismissedClearKey] = useState<string | null>(
null,
);
@@ -400,6 +433,8 @@ export function PuzzleRuntimeShell({
? 'failed'
: currentLevel.status
: 'playing';
const isInteractionLocked =
isBusy || runtimeStatus !== 'playing' || Boolean(propDialog);
const clearResultKey = currentLevel
? `${run?.runId ?? 'run'}:${currentLevel.profileId}:${currentLevel.levelIndex}`
: null;
@@ -409,12 +444,19 @@ export function PuzzleRuntimeShell({
currentLevel?.coverImageSrc ?? null,
);
const mocapInput = useMocapInput({enabled: runtimeStatus === 'playing'});
const primaryMocapHand = mocapInput.latestCommand?.primaryHand;
const primaryMocapHandState = primaryMocapHand?.state;
const primaryMocapHandX = primaryMocapHand?.x;
const primaryMocapHandY = primaryMocapHand?.y;
const mocapActionsLabel =
mocapInput.latestCommand?.actions.length
? mocapInput.latestCommand.actions.join(', ')
: '无';
const mocapHandLabel = mocapInput.latestCommand?.primaryHand
? `${mocapInput.latestCommand.primaryHand.state} @ ${mocapInput.latestCommand.primaryHand.x.toFixed(2)}, ${mocapInput.latestCommand.primaryHand.y.toFixed(2)}`
const mocapHandLabel =
primaryMocapHandState &&
typeof primaryMocapHandX === 'number' &&
typeof primaryMocapHandY === 'number'
? `${primaryMocapHandState} @ ${primaryMocapHandX.toFixed(2)}, ${primaryMocapHandY.toFixed(2)}`
: '无';
const mocapParseWarningLabel = mocapInput.latestCommand?.parseWarnings?.length
? mocapInput.latestCommand.parseWarnings.join('；')
@@ -425,6 +467,11 @@ export function PuzzleRuntimeShell({
currentLevelRef.current = currentLevel;
}, [currentLevel]);
const commitSelectedPieceId = (pieceId: string | null) => {
selectedPieceIdRef.current = pieceId;
setSelectedPieceId(pieceId);
};
const pieces = useMemo<PuzzleBoardPieceViewModel[]>(() => {
if (!board) {
return [];
@@ -586,13 +633,18 @@ export function PuzzleRuntimeShell({
dragVisualFrameRef.current = null;
};
const resetDragInteraction = () => {
const resetDragInteractionState = () => {
cancelDragVisualFrame();
dragOffsetRef.current = null;
dragSessionRef.current = null;
draggingTargetRef.current = null;
resetDragVisualTarget();
};
const resetDragInteraction = () => {
runtimeDragInputControllerRef.current.cancel();
};
const flushDragVisual = () => {
dragVisualFrameRef.current = null;
const dragSession = dragSessionRef.current;
@@ -602,7 +654,8 @@ export function PuzzleRuntimeShell({
}
const piece = pieceById.get(dragSession.pieceId) ?? null;
const groupId = piece?.mergedGroupId ?? null;
const groupId =
draggingTargetRef.current?.groupId ?? piece?.mergedGroupId ?? null;
const nextTarget = {
pieceId: dragSession.pieceId,
groupId,
@@ -808,6 +861,293 @@ export function PuzzleRuntimeShell({
];
}, [clearResultKey, currentLevel, dismissedClearKey]);
const handlePieceTap = (
pieceId: string,
selectedPieceIdBeforeInput: string | null,
) => {
if (isInteractionLocked) {
return;
}
if (!selectedPieceIdBeforeInput) {
commitSelectedPieceId(pieceId);
return;
}
if (selectedPieceIdBeforeInput === pieceId) {
commitSelectedPieceId(null);
return;
}
onSwapPieces({
firstPieceId: selectedPieceIdBeforeInput,
secondPieceId: pieceId,
});
commitSelectedPieceId(null);
};
const resolvePuzzleRuntimeDragTarget = (
pieceId: string,
): PuzzleRuntimeDragTargetState | null => {
const sourcePiece = pieceById.get(pieceId) ?? null;
if (!sourcePiece) {
return null;
}
return {
pieceId: sourcePiece.pieceId,
groupId: sourcePiece.mergedGroupId ?? null,
};
};
const commitPuzzleRuntimeDrag = (
target: PuzzleRuntimeDragTargetState | null,
point: RuntimeInputPoint,
) => {
const dragSession = dragSessionRef.current;
if (!target || !dragSession) {
return;
}
const targetCell = board
? resolveRuntimeInputGridCell(point, board)
: null;
if (!targetCell) {
return;
}
onDragPiece({
pieceId: target.pieceId,
targetRow: targetCell.row,
targetCol: targetCell.col,
});
};
const resolveBoardInputPointFromClient = (
clientX: number,
clientY: number,
) =>
createRuntimeInputPointFromClient(
clientX,
clientY,
readRuntimeInputElementBounds(boardRef.current),
);
const resolveBoardInputPointFromNormalized = (
normalizedX: number,
normalizedY: number,
) =>
createRuntimeInputPointFromNormalized(
normalizedX,
normalizedY,
readRuntimeInputElementBounds(boardRef.current),
);
const resetMocapCursorInterpolation = () => {
mocapCursorPreviousSampleRef.current = null;
mocapCursorTargetSampleRef.current = null;
setMocapCursor(null);
};
updateMocapCursorSampleRef.current = (nextSample: PuzzleMocapCursorSample) => {
const previousTarget = mocapCursorTargetSampleRef.current;
mocapCursorPreviousSampleRef.current = previousTarget ?? nextSample;
mocapCursorTargetSampleRef.current = nextSample;
if (!previousTarget) {
setMocapCursor(nextSample);
}
};
const syncRuntimeDragFromController = (
session: RuntimeDragInputSession<string> | null,
) => {
if (!session) {
return;
}
draggingTargetRef.current = resolvePuzzleRuntimeDragTarget(session.targetId);
dragSessionRef.current = {
pieceId: session.targetId,
inputId: session.inputId,
dragging: session.dragging,
startX: session.startPoint.clientX,
startY: session.startPoint.clientY,
currentX: session.currentPoint.clientX,
currentY: session.currentPoint.clientY,
};
if (session.dragging) {
flushDragVisual();
scheduleDragVisual();
}
};
runtimeDragInputControllerRef.current.setOptions({
dragThresholdPx: 8,
onPress: (session) => {
draggingTargetRef.current = resolvePuzzleRuntimeDragTarget(session.targetId);
syncRuntimeDragFromController(session);
selectedPieceBeforeInputRef.current = selectedPieceIdRef.current;
commitSelectedPieceId(session.targetId);
triggerPuzzlePiecePressHapticFeedback();
},
onDragStart: (session) => {
draggingTargetRef.current = resolvePuzzleRuntimeDragTarget(session.targetId);
syncRuntimeDragFromController(session);
},
onDragMove: (session) => {
syncRuntimeDragFromController(session);
},
onDrop: (session) => {
draggingTargetRef.current = resolvePuzzleRuntimeDragTarget(session.targetId);
syncRuntimeDragFromController(session);
commitPuzzleRuntimeDrag(draggingTargetRef.current, session.currentPoint);
commitSelectedPieceId(null);
selectedPieceBeforeInputRef.current = null;
resetDragInteractionState();
},
onTap: (session) => {
handlePieceTap(session.targetId, selectedPieceBeforeInputRef.current);
selectedPieceBeforeInputRef.current = null;
resetDragInteractionState();
},
onCancel: () => {
commitSelectedPieceId(selectedPieceBeforeInputRef.current);
selectedPieceBeforeInputRef.current = null;
resetDragInteractionState();
},
});
useEffect(() => {
const activeSession = runtimeDragInputControllerRef.current.getSession();
if (!board || runtimeStatus !== 'playing' || isInteractionLocked) {
runtimeDragInputControllerRef.current.cancel();
resetMocapCursorInterpolation();
return;
}
if (
!primaryMocapHandState ||
typeof primaryMocapHandX !== 'number' ||
typeof primaryMocapHandY !== 'number'
) {
runtimeDragInputControllerRef.current.cancel(PUZZLE_MOCAP_DRAG_INPUT_ID);
resetMocapCursorInterpolation();
return;
}
const nextSample = {
x: primaryMocapHandX,
y: primaryMocapHandY,
state: primaryMocapHandState,
receivedAtMs: performance.now(),
};
updateMocapCursorSampleRef.current(nextSample);
const handPoint = resolveBoardInputPointFromNormalized(nextSample.x, nextSample.y);
if (primaryMocapHandState === 'grab') {
if (activeSession?.inputId !== PUZZLE_MOCAP_DRAG_INPUT_ID) {
const sourceCell = resolveRuntimeInputGridCell(handPoint, board);
const sourcePiece = sourceCell
? pieceByCell.get(`${sourceCell.row}:${sourceCell.col}`) ?? null
: null;
if (!sourcePiece) {
runtimeDragInputControllerRef.current.cancel(
PUZZLE_MOCAP_DRAG_INPUT_ID,
);
return;
}
runtimeDragInputControllerRef.current.press({
targetId: sourcePiece.pieceId,
inputId: PUZZLE_MOCAP_DRAG_INPUT_ID,
deviceKind: 'mocap',
point: handPoint,
});
return;
}
runtimeDragInputControllerRef.current.move({
inputId: PUZZLE_MOCAP_DRAG_INPUT_ID,
point: handPoint,
forceDragging: true,
});
return;
}
if (activeSession?.inputId === PUZZLE_MOCAP_DRAG_INPUT_ID) {
runtimeDragInputControllerRef.current.release({
inputId: PUZZLE_MOCAP_DRAG_INPUT_ID,
point: handPoint,
forceDrop: activeSession.deviceKind === 'mocap',
});
}
}, [
board,
isInteractionLocked,
pieceByCell,
primaryMocapHandState,
primaryMocapHandX,
primaryMocapHandY,
runtimeStatus,
]);
useEffect(() => {
if (!board || runtimeStatus !== 'playing') {
if (mocapCursorIntervalRef.current !== null) {
window.clearInterval(mocapCursorIntervalRef.current);
mocapCursorIntervalRef.current = null;
}
return;
}
const tickMocapCursor = () => {
const targetSample = mocapCursorTargetSampleRef.current;
if (!targetSample) {
return;
}
const previousSample = mocapCursorPreviousSampleRef.current ?? targetSample;
const durationMs = Math.max(
PUZZLE_MOCAP_CURSOR_FRAME_MS,
targetSample.receivedAtMs - previousSample.receivedAtMs,
);
const progress = targetSample.receivedAtMs === previousSample.receivedAtMs
? 1
: Math.min(
1,
Math.max(0, (performance.now() - targetSample.receivedAtMs) / durationMs),
);
const nextCursor = {
x: previousSample.x + (targetSample.x - previousSample.x) * progress,
y: previousSample.y + (targetSample.y - previousSample.y) * progress,
state: targetSample.state,
};
const nextPoint = resolveBoardInputPointFromNormalized(
nextCursor.x,
nextCursor.y,
);
setMocapCursor(nextCursor);
const activeSession = runtimeDragInputControllerRef.current.getSession();
if (activeSession?.inputId === PUZZLE_MOCAP_DRAG_INPUT_ID) {
runtimeDragInputControllerRef.current.move({
inputId: PUZZLE_MOCAP_DRAG_INPUT_ID,
point: nextPoint,
forceDragging: true,
});
}
};
tickMocapCursor();
mocapCursorIntervalRef.current = window.setInterval(
tickMocapCursor,
PUZZLE_MOCAP_CURSOR_FRAME_MS,
);
return () => {
if (mocapCursorIntervalRef.current !== null) {
window.clearInterval(mocapCursorIntervalRef.current);
mocapCursorIntervalRef.current = null;
}
};
}, [board, runtimeStatus]);
if (!run || !currentLevel || !board) {
return (
<div
@@ -821,131 +1161,12 @@ export function PuzzleRuntimeShell({
);
}
const handlePieceClick = (pieceId: string) => {
if (isInteractionLocked) {
return;
}
if (!selectedPieceId) {
setSelectedPieceId(pieceId);
return;
}
if (selectedPieceId === pieceId) {
setSelectedPieceId(null);
return;
}
onSwapPieces({
firstPieceId: selectedPieceId,
secondPieceId: pieceId,
});
setSelectedPieceId(null);
};
const resolveBoardCellFromPointer = (clientX: number, clientY: number) => {
const boardElement = boardRef.current;
if (!boardElement) {
return null;
}
const rect = boardElement.getBoundingClientRect();
if (
clientX < rect.left ||
clientX > rect.right ||
clientY < rect.top ||
clientY > rect.bottom
) {
return null;
}
const relativeX = clientX - rect.left;
const relativeY = clientY - rect.top;
const col = Math.min(
board.cols - 1,
Math.max(0, Math.floor((relativeX / rect.width) * board.cols)),
);
const row = Math.min(
board.rows - 1,
Math.max(0, Math.floor((relativeY / rect.height) * board.rows)),
);
return { row, col };
};
const resolveMocapTargetCell = (x: number, y: number) => ({
row: Math.min(board.rows - 1, Math.max(0, Math.floor(y * board.rows))),
col: Math.min(board.cols - 1, Math.max(0, Math.floor(x * board.cols))),
});
const handleMocapInputCommand = () => {
const hand = mocapInput.latestCommand?.primaryHand;
if (runtimeStatus !== 'playing' || isInteractionLocked || !hand) {
mocapDragRef.current = null;
setMocapCursor(null);
return;
}
setMocapCursor({x: hand.x, y: hand.y, state: hand.state});
if (hand.state === 'grab') {
if (mocapDragRef.current) {
return;
}
const sourceCell = resolveMocapTargetCell(hand.x, hand.y);
const sourcePiece = pieceByCell.get(`${sourceCell.row}:${sourceCell.col}`) ?? null;
if (!sourcePiece || sourcePiece.mergedGroupId) {
return;
}
mocapDragRef.current = {pieceId: sourcePiece.pieceId};
setSelectedPieceId(sourcePiece.pieceId);
triggerPuzzlePiecePressHapticFeedback();
return;
}
const draggingPiece = mocapDragRef.current;
if (!draggingPiece) {
return;
}
const targetCell = resolveMocapTargetCell(hand.x, hand.y);
mocapDragRef.current = null;
setSelectedPieceId(null);
onDragPiece({
pieceId: draggingPiece.pieceId,
targetRow: targetCell.row,
targetCol: targetCell.col,
});
};
const handlePiecePointerUp = (
pieceId: string,
event: React.PointerEvent<HTMLDivElement>,
) => {
const currentDragSession = dragSessionRef.current;
if (!currentDragSession || currentDragSession.pieceId !== pieceId) {
return;
}
const handlePiecePointerUp = (event: React.PointerEvent<HTMLDivElement>) => {
event.currentTarget.releasePointerCapture?.(event.pointerId);
if (currentDragSession.dragging) {
const targetCell = resolveBoardCellFromPointer(
event.clientX,
event.clientY,
);
resetDragInteraction();
if (targetCell) {
onDragPiece({
pieceId,
targetRow: targetCell.row,
targetCol: targetCell.col,
});
}
setSelectedPieceId(null);
return;
}
resetDragInteraction();
handlePieceClick(pieceId);
runtimeDragInputControllerRef.current.release({
inputId: `pointer:${event.pointerId}`,
point: resolveBoardInputPointFromClient(event.clientX, event.clientY),
});
};
const handlePiecePointerDown = (
@@ -958,46 +1179,20 @@ export function PuzzleRuntimeShell({
event.preventDefault();
resetDragInteraction();
event.currentTarget.setPointerCapture?.(event.pointerId);
// 按下可交互拼图片时立即给移动端短震反馈，点击选择与拖起都会有同一套手感。
triggerPuzzlePiecePressHapticFeedback();
dragSessionRef.current = {
pieceId,
pointerId: event.pointerId,
dragging: false,
startX: event.clientX,
startY: event.clientY,
currentX: event.clientX,
currentY: event.clientY,
};
runtimeDragInputControllerRef.current.press({
targetId: pieceId,
inputId: `pointer:${event.pointerId}`,
deviceKind: 'pointer',
point: resolveBoardInputPointFromClient(event.clientX, event.clientY),
});
};
const handlePiecePointerMove = (
pieceId: string,
event: React.PointerEvent<HTMLDivElement>,
) => {
const dragSession = dragSessionRef.current;
if (
!dragSession ||
dragSession.pieceId !== pieceId ||
dragSession.pointerId !== event.pointerId
) {
return;
}
const handlePiecePointerMove = (event: React.PointerEvent<HTMLDivElement>) => {
event.preventDefault();
const deltaX = event.clientX - dragSession.startX;
const deltaY = event.clientY - dragSession.startY;
const dragging = dragSession.dragging || Math.hypot(deltaX, deltaY) >= 8;
dragSession.dragging = dragging;
dragSession.currentX = event.clientX;
dragSession.currentY = event.clientY;
if (!dragging) {
return;
}
// 首帧拖拽反馈立即落到 DOM，确保层级提升不会滞后一帧；后续仍保留 raf 兜底连续刷新。
flushDragVisual();
scheduleDragVisual();
runtimeDragInputControllerRef.current.move({
inputId: `pointer:${event.pointerId}`,
point: resolveBoardInputPointFromClient(event.clientX, event.clientY),
});
};
const draggingPieceId = dragRenderTarget?.pieceId ?? null;
@@ -1037,8 +1232,6 @@ export function PuzzleRuntimeShell({
currentLevel.status === 'cleared' &&
dismissedClearKey !== clearResultKey &&
isClearResultReady;
const isInteractionLocked =
isBusy || runtimeStatus !== 'playing' || Boolean(propDialog);
const handleBackRequest = () => {
if (hideExitControls) {
return;
@@ -1150,10 +1343,6 @@ export function PuzzleRuntimeShell({
}
};
useEffect(() => {
handleMocapInputCommand();
}, [mocapInput.latestCommand?.primaryHand]);
return (
<div
className={`puzzle-runtime-shell ${embedded ? 'relative h-full min-h-0 w-full' : 'fixed inset-0 z-[100]'} flex justify-center`}
@@ -1311,11 +1500,11 @@ export function PuzzleRuntimeShell({
if (!piece || isMerged) {
return;
}
handlePiecePointerMove(piece.pieceId, event);
handlePiecePointerMove(event);
}}
onPointerUp={(event) => {
if (piece && !isMerged) {
handlePiecePointerUp(piece.pieceId, event);
handlePiecePointerUp(event);
}
}}
onPointerCancel={() => {
@@ -1460,10 +1649,10 @@ export function PuzzleRuntimeShell({
handlePiecePointerDown(piece.pieceId, event);
}}
onPointerMove={(event) => {
handlePiecePointerMove(piece.pieceId, event);
handlePiecePointerMove(event);
}}
onPointerUp={(event) => {
handlePiecePointerUp(piece.pieceId, event);
handlePiecePointerUp(event);
}}
onPointerCancel={() => {
resetDragInteraction();

99
src/components/rpg-entry/RpgEntryFlowShell.agent.interaction.test.tsx
View File

@@ -3,7 +3,7 @@
import { render, screen, waitFor, within } from '@testing-library/react';
import userEvent from '@testing-library/user-event';
import { useState } from 'react';
import { beforeEach, expect, test, vi } from 'vitest';
import { afterEach, beforeEach, expect, test, vi } from 'vitest';
import type { BigFishWorkSummary } from '../../../packages/shared/src/contracts/bigFishWorkSummary';
import type { CreativeAgentSessionSnapshot } from '../../../packages/shared/src/contracts/creativeAgent';
@@ -2378,6 +2378,10 @@ beforeEach(() => {
);
});
afterEach(() => {
vi.unstubAllEnvs();
});
test('create tab shows template tabs and embeds puzzle form by default', async () => {
const user = userEvent.setup();
@@ -2413,6 +2417,7 @@ test('create tab shows template tabs and embeds puzzle form by default', async (
expect(screen.queryByRole('button', { name: /智能创作/u })).toBeNull();
expect(screen.queryByPlaceholderText('问一问百梦')).toBeNull();
expect(screen.queryByRole('button', { name: /角色扮演/u })).toBeNull();
expect(screen.getByRole('tab', { name: /抓大鹅/u })).toBeTruthy();
expect(createRpgCreationSession).not.toHaveBeenCalled();
expect(match3dCreationClient.createSession).not.toHaveBeenCalled();
expect(createPuzzleAgentSession).not.toHaveBeenCalled();
@@ -3098,6 +3103,50 @@ test('logged out public detail gates big fish start before local runtime', async
expect(recordBigFishPlay).not.toHaveBeenCalled();
});
test('public code search blocks edutainment work when entry switch is disabled', async () => {
vi.stubEnv('VITE_ENABLE_EDUTAINMENT_ENTRY', 'false');
const user = userEvent.setup();
const edutainmentPuzzleWork: PuzzleWorkSummary = {
workId: 'puzzle-work-edutainment-1',
profileId: 'puzzle-profile-edutainment-1',
ownerUserId: 'user-2',
sourceSessionId: 'puzzle-session-edutainment-1',
authorDisplayName: '动作 Demo 作者',
levelName: '儿童动作热身 Demo',
summary: '寓教于乐专属动作 Demo。',
themeTags: ['运动', '安全', '拼图', '寓教于乐'],
coverImageSrc: null,
coverAssetId: null,
publicationStatus: 'published',
updatedAt: '2026-05-09T10:00:00.000Z',
publishedAt: '2026-05-09T10:00:00.000Z',
playCount: 3,
remixCount: 0,
likeCount: 0,
publishReady: true,
};
vi.mocked(listPuzzleGallery).mockResolvedValue({
items: [edutainmentPuzzleWork],
});
vi.mocked(getPuzzleGalleryDetail).mockResolvedValue({
item: edutainmentPuzzleWork,
});
render(<TestWrapper withAuth />);
await openDiscoverHub(user);
const searchInput = await screen.findByPlaceholderText(
'搜索作品号、名称、作者、描述',
);
await user.type(searchInput, 'PZ-TMENT1');
await user.click(screen.getByRole('button', { name: '搜索' }));
expect(await screen.findByText('未找到结果')).toBeTruthy();
expect(screen.queryByText('儿童动作热身 Demo')).toBeNull();
expect(getPuzzleGalleryDetail).not.toHaveBeenCalled();
});
test('creation hub clears all private work shelves immediately after logout state', async () => {
const user = userEvent.setup();
const loggedInAuth = createAuthValue();
@@ -4294,6 +4343,54 @@ test('public code search opens a published puzzle by PZ code', async () => {
expect(getRpgEntryWorldGalleryDetailByCode).not.toHaveBeenCalled();
});
test('missing puzzle public detail returns to platform home', async () => {
const user = userEvent.setup();
const missingPuzzleWork = {
workId: 'puzzle-work-missing-1',
profileId: 'puzzle-profile-missing-1',
ownerUserId: 'user-2',
sourceSessionId: null,
authorDisplayName: '拼图作者',
levelName: '失效拼图',
summary: '这个作品已经不可用。',
themeTags: ['失效'],
coverImageSrc: null,
coverAssetId: null,
publicationStatus: 'published',
updatedAt: '2026-04-25T09:00:00.000Z',
publishedAt: '2026-04-25T09:00:00.000Z',
playCount: 1,
remixCount: 0,
likeCount: 0,
publishReady: true,
} satisfies PuzzleWorkSummary;
vi.mocked(listPuzzleGallery).mockResolvedValue({
items: [missingPuzzleWork],
});
vi.mocked(getPuzzleGalleryDetail).mockRejectedValueOnce(
new ApiClientError({
message: '资源不存在',
status: 404,
code: 'NOT_FOUND',
}),
);
render(<TestWrapper />);
await openDiscoverHub(user);
const workCards = await screen.findAllByRole('button', { name: /失效拼图/u });
await user.click(workCards[0]!);
await waitFor(() => {
expect(window.location.pathname).toBe('/');
});
expect(getPlatformTabPanel('home').getAttribute('aria-hidden')).toBe('false');
expect(screen.queryByText('详情')).toBeNull();
expect(screen.queryByText('资源不存在')).toBeNull();
expect(startPuzzleRun).toHaveBeenCalledTimes(0);
});
test('public code search opens a published big fish work by BF code', async () => {
const user = userEvent.setup();
const bigFishWork: BigFishWorkSummary = {

125
src/components/rpg-entry/RpgEntryHomeView.recharge.test.tsx
View File

@@ -25,7 +25,10 @@ import {
RpgEntryHomeView,
type RpgEntryHomeViewProps,
} from './RpgEntryHomeView';
import type { PlatformPublicGalleryCard } from './rpgEntryWorldPresentation';
import type {
PlatformPublicGalleryCard,
PlatformPuzzleGalleryCard,
} from './rpgEntryWorldPresentation';
const {
mockBuildReferralCenter,
@@ -425,6 +428,23 @@ const longTextRankEntry = {
updatedAt: '2026-04-29T10:00:00.000Z',
} satisfies PlatformPublicGalleryCard;
function buildTaggedPuzzleEntry(
id: string,
worldName: string,
themeTags: string[],
overrides: Partial<PlatformPuzzleGalleryCard> = {},
) {
return {
...puzzlePublicEntry,
workId: `puzzle-work-${id}`,
profileId: `puzzle-profile-${id}`,
publicWorkCode: `PZ-${id.toUpperCase()}`,
worldName,
themeTags,
...overrides,
} satisfies PlatformPuzzleGalleryCard;
}
function mockDesktopLayout() {
Object.defineProperty(window, 'matchMedia', {
configurable: true,
@@ -688,6 +708,7 @@ function renderStatefulLoggedOutHomeView(
afterEach(() => {
vi.useRealTimers();
vi.clearAllMocks();
vi.unstubAllEnvs();
mockGetRpgProfileReferralInviteCenter.mockResolvedValue(
mockBuildReferralCenter(),
);
@@ -1097,6 +1118,108 @@ test('discover search fuzzy matches public work id, name, author and description
expect(onOpenGalleryDetail).toHaveBeenCalledWith(entries[1]);
});
test('mobile discover keeps edutainment works in the last dedicated channel only', async () => {
const user = userEvent.setup();
const onSearchPublicCode = vi.fn();
const generalEntry = buildTaggedPuzzleEntry('normal01', '普通拼图作品', [
'儿童教育',
]);
const edutainmentEntry = buildTaggedPuzzleEntry(
'edu001',
'儿童动作热身 Demo',
['运动', '安全', '拼图', '寓教于乐'],
{
playCount: 99,
remixCount: 30,
likeCount: 50,
recentPlayCount7d: 88,
publishedAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
summaryText: '寓教于乐专属内容',
},
);
renderStatefulLoggedOutHomeView({
latestEntries: [edutainmentEntry, generalEntry],
onSearchPublicCode,
});
await user.click(screen.getByRole('button', { name: '发现' }));
const discoverPanel = document.getElementById('platform-tab-panel-category');
if (!discoverPanel) {
throw new Error('缺少发现面板');
}
const channels = Array.from(
discoverPanel.querySelectorAll('.platform-mobile-home-channel'),
).map((button) => button.textContent);
expect(channels).toEqual(['推荐', '今日', '分类', '排行', '寓教于乐']);
expect(within(discoverPanel).getByText('普通拼图作品')).toBeTruthy();
expect(within(discoverPanel).queryByText('儿童动作热身 Demo')).toBeNull();
await user.click(screen.getByRole('button', { name: '今日' }));
expect(within(discoverPanel).queryByText('儿童动作热身 Demo')).toBeNull();
await user.click(screen.getByRole('button', { name: '分类' }));
expect(screen.getByRole('button', { name: '儿童教育' })).toBeTruthy();
expect(screen.queryByRole('button', { name: '寓教于乐' })).toBeTruthy();
expect(within(discoverPanel).queryByText('儿童动作热身 Demo')).toBeNull();
await user.click(screen.getByRole('button', { name: '排行' }));
expect(within(discoverPanel).queryByText('儿童动作热身 Demo')).toBeNull();
await user.click(screen.getByRole('button', { name: '寓教于乐' }));
expect(
within(discoverPanel).getByRole('button', {
name: /儿童动作热身 Demo/u,
}),
).toBeTruthy();
expect(within(discoverPanel).queryByText('普通拼图作品')).toBeNull();
const searchInput = screen.getByPlaceholderText('搜索作品号、名称、作者、描述');
await user.type(searchInput, '儿童动作热身{enter}');
expect(await within(discoverPanel).findByText('搜索结果')).toBeTruthy();
expect(within(discoverPanel).queryByText('儿童动作热身 Demo')).toBeNull();
expect(onSearchPublicCode).not.toHaveBeenCalled();
});
test('mobile discover hides edutainment channel and work when switch is disabled', async () => {
vi.stubEnv('VITE_ENABLE_EDUTAINMENT_ENTRY', 'false');
const user = userEvent.setup();
const onSearchPublicCode = vi.fn();
const edutainmentEntry = buildTaggedPuzzleEntry(
'eduoff1',
'关闭后隐藏的热身 Demo',
['寓教于乐'],
{
summaryText: '关闭后不可见',
publishedAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
},
);
renderStatefulLoggedOutHomeView({
latestEntries: [edutainmentEntry],
onSearchPublicCode,
});
await user.click(screen.getByRole('button', { name: '发现' }));
const discoverPanel = document.getElementById('platform-tab-panel-category');
if (!discoverPanel) {
throw new Error('缺少发现面板');
}
const channels = Array.from(
discoverPanel.querySelectorAll('.platform-mobile-home-channel'),
).map((button) => button.textContent);
expect(channels).toEqual(['推荐', '今日', '分类', '排行']);
expect(within(discoverPanel).queryByText('关闭后隐藏的热身 Demo')).toBeNull();
const searchInput = screen.getByPlaceholderText('搜索作品号、名称、作者、描述');
await user.type(searchInput, 'PZ-EDUOFF1{enter}');
expect(await within(discoverPanel).findByText('搜索结果')).toBeTruthy();
expect(within(discoverPanel).queryByText('关闭后隐藏的热身 Demo')).toBeNull();
expect(onSearchPublicCode).not.toHaveBeenCalled();
});
test('discover search keeps public code fallback when local works do not match', async () => {
const user = userEvent.setup();
const onSearchPublicCode = vi.fn();

287
src/components/rpg-entry/RpgEntryHomeView.tsx
View File

@@ -75,6 +75,14 @@ import {
} from '../../services/rpg-entry/rpgProfileClient';
import type { CustomWorldProfile } from '../../types';
import { useAuthUi } from '../auth/AuthUiContext';
import {
canExposePublicWork,
EDUTAINMENT_WORK_TAG,
filterEdutainmentPublicWorks,
filterGeneralPublicWorks,
findPublicWorkForHistoryEntry,
isEdutainmentEntryEnabled,
} from '../platform-entry/platformEdutainmentVisibility';
import { ResolvedAssetImage } from '../ResolvedAssetImage';
import { RpgEntryBrandLogo } from './RpgEntryBrandLogo';
import {
@@ -183,7 +191,12 @@ const RECOMMEND_ENTRY_COMMIT_ANIMATION_MS = 180;
const RECOMMEND_ENTRY_DRAG_LIMIT_PX = 160;
type ProfilePopupPanel = 'invite' | 'redeem' | 'community';
type DiscoverChannel = 'recommend' | 'today' | 'category' | 'ranking';
type DiscoverChannel =
| 'recommend'
| 'today'
| 'category'
| 'ranking'
| 'edutainment';
type PlatformRankingTab = 'hot' | 'remix' | 'new' | 'like';
const COMMUNITY_QR_CODES = [
@@ -208,6 +221,10 @@ const DISCOVER_CHANNELS: Array<{
{ id: 'category', label: '分类' },
{ id: 'ranking', label: '排行' },
];
const EDUTAINMENT_DISCOVER_CHANNEL = {
id: 'edutainment',
label: EDUTAINMENT_WORK_TAG,
} as const;
const PLATFORM_RANKING_TABS: Array<{
id: PlatformRankingTab;
@@ -1313,9 +1330,11 @@ function buildPublicCategoryGroups(
) {
const publicEntryMap = new Map<string, PlatformPublicGalleryCard>();
[...featuredEntries, ...latestEntries].forEach((entry) => {
publicEntryMap.set(buildPublicGalleryCardKey(entry), entry);
});
filterGeneralPublicWorks([...featuredEntries, ...latestEntries]).forEach(
(entry) => {
publicEntryMap.set(buildPublicGalleryCardKey(entry), entry);
},
);
const categoryMap = new Map<string, PlatformPublicGalleryCard[]>();
Array.from(publicEntryMap.values()).forEach((entry) => {
@@ -1346,6 +1365,21 @@ function getPlatformPublicEntries(
) {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
filterGeneralPublicWorks([...featuredEntries, ...latestEntries]).forEach(
(entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
},
);
return Array.from(entryMap.values());
}
function getAllPlatformPublicEntries(
featuredEntries: PlatformPublicGalleryCard[],
latestEntries: PlatformPublicGalleryCard[],
) {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
[...featuredEntries, ...latestEntries].forEach((entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
});
@@ -3148,21 +3182,62 @@ export function RpgEntryHomeView({
const [avatarError, setAvatarError] = useState<string | null>(null);
const [isSavingAvatar, setIsSavingAvatar] = useState(false);
const isAuthenticated = Boolean(authUi?.user);
const edutainmentEntryEnabled = isEdutainmentEntryEnabled();
const isDesktopLayout = usePlatformDesktopLayout();
const openRecommendGalleryDetail =
onOpenRecommendGalleryDetail ?? onOpenGalleryDetail;
const featuredShelf = useMemo(
() => featuredEntries.slice(0, 6),
const generalFeaturedEntries = useMemo(
() => filterGeneralPublicWorks(featuredEntries),
[featuredEntries],
);
const categoryGroups = useMemo(
() => buildPublicCategoryGroups(featuredEntries, latestEntries),
const featuredShelf = useMemo(
() => generalFeaturedEntries.slice(0, 6),
[generalFeaturedEntries],
);
const generalLatestEntries = useMemo(
() => filterGeneralPublicWorks(latestEntries),
[latestEntries],
);
const allEdutainmentEntries = useMemo(
() => filterEdutainmentPublicWorks([...featuredEntries, ...latestEntries]),
[featuredEntries, latestEntries],
);
const edutainmentEntries = useMemo(
() => (edutainmentEntryEnabled ? allEdutainmentEntries : []),
[allEdutainmentEntries, edutainmentEntryEnabled],
);
const visibleDiscoverChannels = useMemo(
() =>
edutainmentEntryEnabled
? [...DISCOVER_CHANNELS, EDUTAINMENT_DISCOVER_CHANNEL]
: DISCOVER_CHANNELS,
[edutainmentEntryEnabled],
);
const categoryGroups = useMemo(
() =>
buildPublicCategoryGroups(generalFeaturedEntries, generalLatestEntries),
[generalFeaturedEntries, generalLatestEntries],
);
const publicEntries = useMemo(
() => getPlatformPublicEntries(featuredEntries, latestEntries),
() =>
getPlatformPublicEntries(generalFeaturedEntries, generalLatestEntries),
[generalFeaturedEntries, generalLatestEntries],
);
const allPublicEntries = useMemo(
() => getAllPlatformPublicEntries(featuredEntries, latestEntries),
[featuredEntries, latestEntries],
);
const visibleHistoryEntries = useMemo(
() =>
historyEntries.filter((entry) => {
const matchedPublicWork = findPublicWorkForHistoryEntry(
entry,
allPublicEntries,
);
return !matchedPublicWork || canExposePublicWork(matchedPublicWork);
}),
[allPublicEntries, historyEntries],
);
const workSearchResults = useMemo(
() =>
filterPlatformWorkSearchResults(publicEntries, activeWorkSearchKeyword),
@@ -3257,6 +3332,12 @@ export function RpgEntryHomeView({
}
}, [activeTab, isAuthenticated, onTabChange, visibleTabs]);
useEffect(() => {
if (!visibleDiscoverChannels.some((channel) => channel.id === discoverChannel)) {
setDiscoverChannel('recommend');
}
}, [discoverChannel, visibleDiscoverChannels]);
useEffect(() => {
setVisitedTabs((currentTabs) => {
if (currentTabs.has(activeTab)) {
@@ -3739,6 +3820,10 @@ export function RpgEntryHomeView({
publicEntries,
trimmedKeyword,
);
const hiddenEdutainmentMatches = filterPlatformWorkSearchResults(
allEdutainmentEntries,
trimmedKeyword,
);
if (
matchedEntries.length > 0 &&
isExactPublicWorkCodeSearch(matchedEntries, trimmedKeyword) &&
@@ -3755,6 +3840,11 @@ export function RpgEntryHomeView({
return;
}
if (hiddenEdutainmentMatches.length > 0) {
setActiveWorkSearchKeyword(trimmedKeyword);
return;
}
setActiveWorkSearchKeyword('');
if (!onSearchPublicCode || isSearchingPublicCode) {
return;
@@ -3769,50 +3859,58 @@ export function RpgEntryHomeView({
submitWorkSearch(mobileSearchKeyword);
};
const desktopHeroEntry =
featuredShelf[0] ?? latestEntries[0] ?? myEntries[0] ?? null;
featuredShelf[0] ?? generalLatestEntries[0] ?? myEntries[0] ?? null;
const desktopHeroCover = desktopHeroEntry
? resolvePlatformWorldCoverImage(desktopHeroEntry)
: null;
const desktopHeroStripEntries = (
featuredShelf.length > 0 ? featuredShelf : latestEntries
featuredShelf.length > 0 ? featuredShelf : generalLatestEntries
).slice(0, 5);
// 网页端保留原有宽屏布局，只把模块数据同步到移动端首页频道语义。
const desktopRecommendEntries = useMemo(() => {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
[...featuredShelf, ...latestEntries].forEach((entry) => {
[...featuredShelf, ...generalLatestEntries].forEach((entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
});
return Array.from(entryMap.values());
}, [featuredShelf, latestEntries]);
}, [featuredShelf, generalLatestEntries]);
const desktopTodayEntries = useMemo(
() => filterTodayPublishedEntries(latestEntries),
[latestEntries],
() => filterTodayPublishedEntries(generalLatestEntries),
[generalLatestEntries],
);
const desktopFeaturedGrid = desktopRecommendEntries.slice(0, 4);
const desktopCategoryGrid = activeCategoryEntries.slice(0, 6);
const desktopLibraryPreview = myEntries.slice(0, 2);
const recommendedFeedEntries = useMemo(() => {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
[...featuredShelf, ...latestEntries].forEach((entry) => {
[...featuredShelf, ...generalLatestEntries].forEach((entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
});
return Array.from(entryMap.values());
}, [featuredShelf, latestEntries]);
}, [featuredShelf, generalLatestEntries]);
const discoverFeedEntries = useMemo(() => {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
const sourceEntries =
discoverChannel === 'recommend'
? recommendedFeedEntries
: filterTodayPublishedEntries(latestEntries);
: filterTodayPublishedEntries(generalLatestEntries);
sourceEntries.forEach((entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
});
return Array.from(entryMap.values());
}, [discoverChannel, latestEntries, recommendedFeedEntries]);
}, [discoverChannel, generalLatestEntries, recommendedFeedEntries]);
const edutainmentFeedEntries = useMemo(() => {
const entryMap = new Map<string, PlatformPublicGalleryCard>();
edutainmentEntries.forEach((entry) => {
entryMap.set(buildPublicGalleryCardKey(entry), entry);
});
return Array.from(entryMap.values());
}, [edutainmentEntries]);
const mobileFeedCarouselEnabled =
!isDesktopLayout &&
activeTab === 'category' &&
@@ -4125,7 +4223,7 @@ export function RpgEntryHomeView({
isAuthenticated,
openRecommendGalleryDetail,
]);
const leadPublicEntry = featuredShelf[0] ?? latestEntries[0] ?? null;
const leadPublicEntry = featuredShelf[0] ?? generalLatestEntries[0] ?? null;
const openLeadPublicEntry = () => {
if (leadPublicEntry) {
openRecommendGalleryDetail(leadPublicEntry);
@@ -4324,7 +4422,7 @@ export function RpgEntryHomeView({
) : (
<>
<div className="platform-mobile-home-channelbar flex min-w-0 gap-4 overflow-x-auto pb-1 scrollbar-hide">
{DISCOVER_CHANNELS.map((channel) => {
{visibleDiscoverChannels.map((channel) => {
const active = discoverChannel === channel.id;
return (
<button
@@ -4403,6 +4501,31 @@ export function RpgEntryHomeView({
<EmptyShelf text="公开广场暂时还没有可分类的作品。" />
)}
</section>
) : discoverChannel === 'edutainment' ? (
<section className="platform-mobile-home-feed">
{isLoadingPlatform ? (
<EmptyShelf text="正在读取公开作品..." />
) : edutainmentFeedEntries.length > 0 ? (
<div className="grid min-w-0 gap-3">
{edutainmentFeedEntries.map((entry) => {
const cardKey = buildPublicGalleryCardKey(entry);
return (
<WorldCard
key={`${cardKey}:mobile-edutainment`}
entry={entry}
onClick={() => onOpenGalleryDetail(entry)}
className="w-full"
authorAvatarUrl={getPublicEntryAuthorAvatarUrl(entry)}
feedCardKey={cardKey}
/>
);
})}
</div>
) : (
<EmptyShelf text="暂时还没有可展示的作品。" />
)}
</section>
) : (
<section
ref={mobileDiscoverFeedRef}
@@ -4439,8 +4562,122 @@ export function RpgEntryHomeView({
</div>
);
const desktopDiscoverContent: ReactNode = (
<div className={DESKTOP_PAGE_STAGE_CLASS}>
<div className="platform-mobile-home-channelbar flex min-w-0 gap-4 overflow-x-auto pb-1 scrollbar-hide">
{visibleDiscoverChannels.map((channel) => {
const active = discoverChannel === channel.id;
return (
<button
key={`desktop-${channel.id}`}
type="button"
onClick={() => setDiscoverChannel(channel.id)}
className={`platform-mobile-home-channel shrink-0 ${active ? 'platform-mobile-home-channel--active' : ''}`}
>
{channel.label}
</button>
);
})}
</div>
{platformError ? (
<div className="rounded-[1.5rem] border border-rose-400/20 bg-rose-500/10 px-4 py-3 text-sm leading-6 text-rose-700">
{platformError}
</div>
) : null}
{discoverChannel === 'ranking' ? (
mobileRankingPanel
) : discoverChannel === 'category' ? (
<section className="platform-desktop-panel px-5 py-5">
<SectionHeader title="作品分类" detail="GAME CATEGORY" />
{isLoadingPlatform ? (
<EmptyShelf text="正在读取作品分类..." />
) : activeCategoryGroup && desktopCategoryGrid.length > 0 ? (
<>
<div className="mb-4 flex min-w-0 items-center gap-2 overflow-x-auto pb-1 scrollbar-hide">
{categoryGroups.map((group) => {
const active = group.tag === activeCategoryGroup.tag;
return (
<button
key={`${group.tag}:desktop-discover-category`}
type="button"
onClick={() => setSelectedCategoryTag(group.tag)}
className={`platform-category-chip shrink-0 ${active ? 'platform-category-chip--active' : ''}`}
>
{group.tag}
</button>
);
})}
</div>
<div className="grid gap-4 xl:grid-cols-3">
{desktopCategoryGrid.map((entry) => (
<WorldCard
key={`${buildPublicGalleryCardKey(entry)}:desktop-discover-category:${activeCategoryGroup.tag}`}
entry={entry}
onClick={() => openRecommendGalleryDetail(entry)}
className="w-full min-w-0"
authorAvatarUrl={getPublicEntryAuthorAvatarUrl(entry)}
/>
))}
</div>
</>
) : (
<EmptyShelf text="暂时还没有可分类的作品。" />
)}
</section>
) : discoverChannel === 'edutainment' ? (
<section className="platform-desktop-panel px-5 py-5">
<SectionHeader title={EDUTAINMENT_WORK_TAG} detail="EDUTAINMENT" />
{isLoadingPlatform ? (
<EmptyShelf text="正在读取公开作品..." />
) : edutainmentFeedEntries.length > 0 ? (
<div className="grid gap-4 xl:grid-cols-3">
{edutainmentFeedEntries.map((entry) => (
<WorldCard
key={`${buildPublicGalleryCardKey(entry)}:desktop-edutainment`}
entry={entry}
onClick={() => openRecommendGalleryDetail(entry)}
className="w-full min-w-0"
authorAvatarUrl={getPublicEntryAuthorAvatarUrl(entry)}
/>
))}
</div>
) : (
<EmptyShelf text="暂时还没有可展示的作品。" />
)}
</section>
) : (
<section className="platform-desktop-panel px-5 py-5">
<SectionHeader
title={discoverChannel === 'today' ? '今日游戏' : '推荐'}
detail={discoverChannel === 'today' ? 'TODAY GAMES' : 'RECOMMENDED'}
/>
{isLoadingPlatform ? (
<EmptyShelf text="正在读取公开作品..." />
) : discoverFeedEntries.length > 0 ? (
<div className="grid gap-4 xl:grid-cols-3">
{discoverFeedEntries.map((entry) => (
<WorldCard
key={`${buildPublicGalleryCardKey(entry)}:desktop-discover-feed:${discoverChannel}`}
entry={entry}
onClick={() => openRecommendGalleryDetail(entry)}
className="w-full min-w-0"
authorAvatarUrl={getPublicEntryAuthorAvatarUrl(entry)}
/>
))}
</div>
) : (
<EmptyShelf text="公开广场暂时还没有可展示的作品。" />
)}
</section>
)}
</div>
);
const categoryContent: ReactNode = isDesktopLayout ? (
<div className={DESKTOP_PAGE_STAGE_CLASS}>{mobileRankingPanel}</div>
desktopDiscoverContent
) : (
mobileDiscoverContent
);
@@ -4880,7 +5117,7 @@ export function RpgEntryHomeView({
</div>
<div
className={`grid gap-5 ${desktopLibraryPreview.length > 0 || historyEntries.length > 0 ? '2xl:grid-cols-[minmax(0,1.2fr)_minmax(22rem,0.8fr)]' : ''}`}
className={`grid gap-5 ${desktopLibraryPreview.length > 0 || visibleHistoryEntries.length > 0 ? '2xl:grid-cols-[minmax(0,1.2fr)_minmax(22rem,0.8fr)]' : ''}`}
>
<section className="platform-desktop-panel px-5 py-5">
<SectionHeader title="推荐" detail="RECOMMENDED" />
@@ -4903,7 +5140,7 @@ export function RpgEntryHomeView({
)}
</section>
{desktopLibraryPreview.length > 0 || historyEntries.length > 0 ? (
{desktopLibraryPreview.length > 0 || visibleHistoryEntries.length > 0 ? (
<section className="platform-desktop-panel px-5 py-5">
<SectionHeader
title={desktopLibraryPreview.length > 0 ? '最近作品' : '最近浏览'}
@@ -4948,7 +5185,7 @@ export function RpgEntryHomeView({
</div>
) : (
<div className="mt-3 space-y-3">
{historyEntries.slice(0, 2).map((entry) => {
{visibleHistoryEntries.slice(0, 2).map((entry) => {
const displayName = formatPlatformWorkDisplayName(
entry.worldName,
);

467
src/index.css
View File

@@ -5695,6 +5695,473 @@ button {
color: rgba(255, 255, 255, 0.9) !important;
}
.child-motion-demo {
--child-motion-bg: #07151c;
--child-motion-panel: rgba(6, 24, 30, 0.64);
--child-motion-panel-border: rgba(178, 239, 220, 0.25);
--child-motion-text: #eefcf7;
--child-motion-soft: rgba(238, 252, 247, 0.7);
--child-motion-green: #5ff08f;
--child-motion-sky: #8fd8ff;
display: grid;
width: 100%;
min-width: 0;
height: 100vh;
min-height: 100vh;
place-items: center;
overflow: hidden;
background:
radial-gradient(circle at 18% 14%, rgba(143, 216, 255, 0.24), transparent 32%),
radial-gradient(circle at 82% 22%, rgba(95, 240, 143, 0.18), transparent 30%),
linear-gradient(180deg, #092433 0%, var(--child-motion-bg) 54%, #0a1f18 100%);
color: var(--child-motion-text);
font-family: Inter, ui-sans-serif, system-ui, sans-serif;
}
@supports (height: 100dvh) {
.child-motion-demo {
height: 100dvh;
min-height: 100dvh;
}
}
.child-motion-stage {
position: relative;
width: min(100vw, calc(100vh * 16 / 9));
height: min(100vh, calc(100vw * 9 / 16));
overflow: hidden;
background:
linear-gradient(180deg, rgba(16, 64, 86, 0.86), rgba(9, 42, 39, 0.9)),
var(--child-motion-bg);
box-shadow: 0 30px 100px rgba(0, 0, 0, 0.38);
touch-action: none;
user-select: none;
}
@supports (height: 100dvh) {
.child-motion-stage {
width: min(100vw, calc(100dvh * 16 / 9));
height: min(100dvh, calc(100vw * 9 / 16));
}
}
.child-motion-camera-layer {
position: absolute;
inset: 0;
width: 100%;
height: 100%;
object-fit: cover;
background:
radial-gradient(circle at 50% 33%, rgba(255, 255, 255, 0.12), transparent 28%),
linear-gradient(110deg, rgba(255, 255, 255, 0.06) 0 12%, transparent 12% 20%, rgba(255, 255, 255, 0.04) 20% 31%, transparent 31% 100%);
filter: blur(7px) saturate(0.8);
opacity: 0.62;
transform: scale(1.05);
}
.child-motion-camera-state {
position: absolute;
top: 18%;
left: 50%;
z-index: 7;
transform: translateX(-50%);
border: 1px solid rgba(238, 252, 247, 0.2);
border-radius: 999px;
background: rgba(6, 24, 30, 0.52);
color: rgba(238, 252, 247, 0.82);
padding: 0.45rem 0.9rem;
font-size: clamp(0.68rem, 1.35vw, 0.84rem);
font-weight: 800;
backdrop-filter: blur(12px);
}
.child-motion-floor {
position: absolute;
right: -8%;
bottom: -19%;
left: -8%;
height: 47%;
border-radius: 50% 50% 0 0;
background:
radial-gradient(ellipse at 50% 8%, rgba(190, 255, 220, 0.22), transparent 36%),
linear-gradient(180deg, rgba(24, 86, 67, 0.84), rgba(7, 43, 34, 0.96));
box-shadow: inset 0 22px 70px rgba(255, 255, 255, 0.07);
}
.child-motion-hud {
position: absolute;
z-index: 8;
display: flex;
align-items: center;
gap: clamp(0.6rem, 1.8vw, 1rem);
border: 1px solid var(--child-motion-panel-border);
border-radius: clamp(0.75rem, 2vw, 1.25rem);
background: var(--child-motion-panel);
box-shadow: 0 18px 48px rgba(0, 0, 0, 0.2);
backdrop-filter: blur(14px);
}
.child-motion-hud--top {
top: 4.2%;
left: 50%;
width: min(72%, 48rem);
min-height: clamp(4.2rem, 11vh, 6.25rem);
transform: translateX(-50%);
padding: clamp(0.65rem, 1.8vw, 1rem) clamp(0.8rem, 2.2vw, 1.25rem);
}
.child-motion-hud h1 {
margin: 0;
color: var(--child-motion-text);
font-size: clamp(1.2rem, 3.2vw, 2rem);
font-weight: 900;
line-height: 1.08;
}
.child-motion-hud p {
margin: 0.28rem 0 0;
color: var(--child-motion-soft);
font-size: clamp(0.72rem, 1.45vw, 0.98rem);
font-weight: 700;
line-height: 1.45;
}
.child-motion-step-count,
.child-motion-progress {
display: inline-flex;
width: clamp(2.7rem, 7vw, 4rem);
height: clamp(2.7rem, 7vw, 4rem);
flex: 0 0 auto;
align-items: center;
justify-content: center;
border: 1px solid rgba(238, 252, 247, 0.2);
border-radius: 999px;
background: rgba(255, 255, 255, 0.1);
color: var(--child-motion-text);
font-size: clamp(0.72rem, 1.45vw, 0.95rem);
font-weight: 900;
}
.child-motion-ring {
position: absolute;
bottom: 20.5%;
z-index: 3;
width: clamp(5.8rem, 13vw, 9rem);
aspect-ratio: 1;
transform: translateX(-50%) rotateX(62deg);
border-radius: 999px;
background:
conic-gradient(
from -90deg,
rgba(255, 255, 255, 0.95) 0 var(--child-motion-ring-progress),
rgba(95, 240, 143, 0.18) var(--child-motion-ring-progress) 360deg
);
box-shadow:
0 0 28px rgba(95, 240, 143, 0.42),
inset 0 0 26px rgba(255, 255, 255, 0.18);
}
.child-motion-ring::before {
position: absolute;
inset: 14%;
border-radius: inherit;
background: rgba(8, 44, 36, 0.94);
content: '';
}
.child-motion-ring__core {
position: absolute;
inset: 34%;
border-radius: 999px;
background: var(--child-motion-green);
opacity: 0.28;
}
.child-motion-ring--active {
animation: child-motion-ring-pulse 0.78s ease-in-out infinite alternate;
}
@keyframes child-motion-ring-pulse {
from {
filter: brightness(1);
}
to {
filter: brightness(1.25);
}
}
.child-motion-avatar {
position: absolute;
bottom: 24%;
z-index: 5;
width: clamp(3.4rem, 7vw, 5.6rem);
height: clamp(6rem, 13vw, 10rem);
transform: translateX(-50%);
transition: left 260ms ease, transform 220ms ease;
}
.child-motion-avatar--jumping {
transform: translate(-50%, -14%);
}
.child-motion-avatar__head,
.child-motion-avatar__body,
.child-motion-avatar__arm,
.child-motion-avatar__leg {
position: absolute;
display: block;
background: rgba(7, 18, 24, 0.82);
box-shadow: 0 0 24px rgba(143, 216, 255, 0.18);
}
.child-motion-avatar__head {
top: 0;
left: 50%;
width: 34%;
aspect-ratio: 1;
transform: translateX(-50%);
border-radius: 999px;
}
.child-motion-avatar__body {
top: 27%;
left: 50%;
width: 42%;
height: 36%;
transform: translateX(-50%);
border-radius: 999px 999px 45% 45%;
}
.child-motion-avatar__arm {
top: 33%;
width: 15%;
height: 34%;
border-radius: 999px;
}
.child-motion-avatar__arm--left {
left: 17%;
transform: rotate(18deg);
}
.child-motion-avatar__arm--right {
right: 17%;
transform: rotate(-18deg);
}
.child-motion-avatar__leg {
bottom: 0;
width: 15%;
height: 34%;
border-radius: 999px;
}
.child-motion-avatar__leg--left {
left: 36%;
transform: rotate(7deg);
}
.child-motion-avatar__leg--right {
right: 36%;
transform: rotate(-7deg);
}
.child-motion-gesture-guide {
position: absolute;
inset: 20% 22% 19%;
z-index: 4;
pointer-events: none;
}
.child-motion-gesture-guide__wave,
.child-motion-gesture-guide__jump {
position: absolute;
left: 50%;
top: 38%;
display: inline-flex;
width: clamp(4.5rem, 11vw, 8rem);
aspect-ratio: 1;
transform: translate(-50%, -50%);
align-items: center;
justify-content: center;
border: 2px solid rgba(95, 240, 143, 0.64);
border-radius: 999px;
background: rgba(95, 240, 143, 0.1);
color: var(--child-motion-text);
font-size: clamp(1rem, 2.4vw, 1.55rem);
font-weight: 900;
}
.child-motion-gesture-guide__hand {
position: absolute;
top: 28%;
width: clamp(4rem, 9vw, 7rem);
aspect-ratio: 1;
border: 2px dashed rgba(95, 240, 143, 0.58);
border-radius: 999px;
animation: child-motion-hand-guide 1.1s ease-in-out infinite alternate;
}
.child-motion-gesture-guide__hand--left {
left: 22%;
}
.child-motion-gesture-guide__hand--right {
right: 22%;
}
@keyframes child-motion-hand-guide {
from {
transform: translateY(0);
}
to {
transform: translateY(-10%);
}
}
.child-motion-gesture-guide__trail {
position: absolute;
width: 0.8rem;
height: 0.8rem;
transform: translate(-50%, -50%);
border-radius: 999px;
background: #b9ffd0;
box-shadow: 0 0 16px rgba(95, 240, 143, 0.56);
}
.child-motion-floating-reward {
position: absolute;
left: 50%;
top: 34%;
z-index: 9;
transform: translateX(-50%);
color: #ffffff;
font-size: clamp(1.4rem, 4vw, 2.4rem);
font-weight: 900;
text-shadow: 0 4px 26px rgba(0, 0, 0, 0.42);
animation: child-motion-reward-rise 0.72s ease-out forwards;
}
@keyframes child-motion-reward-rise {
from {
opacity: 0;
transform: translate(-50%, 22%);
}
to {
opacity: 1;
transform: translate(-50%, -18%);
}
}
.child-motion-calibration {
position: absolute;
right: 3.2%;
bottom: 4%;
z-index: 8;
display: grid;
grid-template-columns: repeat(5, minmax(0, auto));
gap: 0.45rem;
max-width: 82%;
border: 1px solid var(--child-motion-panel-border);
border-radius: 999px;
background: var(--child-motion-panel);
padding: 0.45rem;
backdrop-filter: blur(14px);
}
.child-motion-calibration div {
display: grid;
min-width: clamp(3.2rem, 7vw, 4.8rem);
gap: 0.08rem;
justify-items: center;
border-radius: 999px;
background: rgba(255, 255, 255, 0.08);
padding: 0.36rem 0.55rem;
}
.child-motion-calibration span {
color: var(--child-motion-soft);
font-size: clamp(0.55rem, 1.2vw, 0.72rem);
font-weight: 800;
}
.child-motion-calibration strong {
color: var(--child-motion-text);
font-size: clamp(0.72rem, 1.5vw, 0.95rem);
font-weight: 900;
}
.child-motion-start-panel {
position: absolute;
left: 50%;
top: 53%;
z-index: 10;
display: flex;
transform: translate(-50%, -50%);
align-items: center;
gap: 0.85rem;
border: 1px solid rgba(178, 239, 220, 0.32);
border-radius: 1.4rem;
background: rgba(6, 24, 30, 0.7);
padding: clamp(0.85rem, 2vw, 1.15rem);
box-shadow: 0 24px 70px rgba(0, 0, 0, 0.28);
backdrop-filter: blur(14px);
}
.child-motion-start-panel button {
min-width: clamp(8rem, 18vw, 12rem);
min-height: clamp(3rem, 7vw, 4.2rem);
border: 0;
border-radius: 999px;
background: linear-gradient(135deg, #5ff08f, #8fd8ff);
color: #062018;
font-size: clamp(1rem, 2.5vw, 1.4rem);
font-weight: 950;
cursor: pointer;
box-shadow: 0 16px 44px rgba(95, 240, 143, 0.28);
}
.child-motion-start-panel span {
color: var(--child-motion-text);
font-size: clamp(1rem, 2vw, 1.25rem);
font-weight: 900;
}
.child-motion-orientation-tip {
position: fixed;
inset: 0;
z-index: 30;
display: none;
place-items: center;
background: #07151c;
color: var(--child-motion-text);
font-size: 1.25rem;
font-weight: 900;
}
@media (orientation: portrait) and (max-width: 920px) {
.child-motion-orientation-tip {
display: grid;
}
}
@media (max-width: 760px) {
.child-motion-hud--top {
width: 88%;
}
.child-motion-calibration {
left: 50%;
right: auto;
grid-template-columns: repeat(5, minmax(0, 1fr));
width: min(92%, 35rem);
transform: translateX(-50%);
}
}
@media (min-width: 768px) {
.platform-work-detail {
border-radius: 1.2rem;

20
src/routing/appRoutes.test.ts
View File

@@ -1,7 +1,11 @@
import { describe, expect, it } from 'vitest';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { matchAppRoute } from './appRoutes';
afterEach(() => {
vi.unstubAllEnvs();
});
describe('matchAppRoute', () => {
it('routes the main app by default', () => {
expect(matchAppRoute('/')).toEqual({
@@ -27,6 +31,20 @@ describe('matchAppRoute', () => {
});
});
it('routes child motion demo path to the standalone warmup demo', () => {
expect(matchAppRoute('/CHILD-MOTION-DEMO/')).toEqual({
kind: 'child-motion-demo',
});
});
it('blocks direct child motion demo path when edutainment entry is disabled', () => {
vi.stubEnv('VITE_ENABLE_EDUTAINMENT_ENTRY', 'false');
expect(matchAppRoute('/child-motion-demo')).toEqual({
kind: 'game',
});
});
it('routes former standalone editor paths back to the main game', () => {
expect(matchAppRoute('/item-editor/tools')).toEqual({
kind: 'game',

23
src/routing/appRoutes.tsx
View File

@@ -2,6 +2,7 @@
import { type ComponentType, lazy, type LazyExoticComponent } from 'react';
import { isEdutainmentEntryEnabled } from '../components/platform-entry/platformEdutainmentVisibility';
import { normalizeAppPath } from './appPageRoutes';
type AppRouteComponent = LazyExoticComponent<
@@ -18,6 +19,9 @@ export type AppRouteMatch =
| {
kind: 'match3d-playground';
}
| {
kind: 'child-motion-demo';
}
| {
kind: 'game';
};
@@ -34,6 +38,7 @@ const GameApp = lazy(() => import('../AuthenticatedApp')) as AppRouteComponent;
const BigFishPlaygroundApp = lazy(() => import('../BigFishPlaygroundApp')) as AppRouteComponent;
const Match3DPlaygroundApp = lazy(() => import('../Match3DPlaygroundApp')) as AppRouteComponent;
const PuzzlePlaygroundApp = lazy(() => import('../PuzzlePlaygroundApp')) as AppRouteComponent;
const ChildMotionDemoApp = lazy(() => import('../ChildMotionDemoApp')) as AppRouteComponent;
function normalizeRoutePath(pathname: string) {
return normalizeAppPath(pathname);
@@ -60,6 +65,15 @@ export function matchAppRoute(pathname: string): AppRouteMatch {
};
}
if (
normalizedPath === '/child-motion-demo' &&
isEdutainmentEntryEnabled()
) {
return {
kind: 'child-motion-demo',
};
}
return {
kind: 'game',
};
@@ -95,6 +109,15 @@ export function resolveAppRoute(pathname: string): ResolvedAppRoute {
};
}
if (matchedRoute.kind === 'child-motion-demo') {
return {
kind: 'child-motion-demo',
loadingEyebrow: '正在载入热身关',
loadingText: '正在进入寓教于乐 Demo...',
Component: ChildMotionDemoApp,
};
}
return {
kind: 'game',
loadingEyebrow: '正在载入游戏',

263
src/services/child-motion-demo/childMotionDebugInput.test.ts Normal file
View File

@@ -0,0 +1,263 @@
// @vitest-environment jsdom
import { afterEach, describe, expect, test, vi } from 'vitest';
import {
type ChildMotionDebugAction,
createChildMotionDebugInputController,
resolveKeyboardDebugAction,
} from './childMotionDebugInput';
let mountedTargets: HTMLElement[] = [];
afterEach(() => {
mountedTargets.forEach((target) => target.remove());
mountedTargets = [];
});
function createTarget() {
const target = document.createElement('div');
document.body.appendChild(target);
mountedTargets.push(target);
return target;
}
function dispatchKeyboard(
target: HTMLElement,
options: { key: string; code?: string; repeat?: boolean },
) {
const event = new KeyboardEvent('keydown', {
bubbles: true,
cancelable: true,
key: options.key,
code: options.code ?? '',
repeat: options.repeat ?? false,
});
target.dispatchEvent(event);
return event;
}
function dispatchPointer(
target: HTMLElement,
type: string,
options: {
button?: number;
clientX: number;
clientY: number;
pointerId?: number;
},
) {
const event = new MouseEvent(type, {
bubbles: true,
cancelable: true,
button: options.button ?? 0,
clientX: options.clientX,
clientY: options.clientY,
});
Object.assign(event, { pointerId: options.pointerId ?? 1 });
target.dispatchEvent(event);
return event;
}
describe('childMotionDebugInput', () => {
test('maps A, D and Space keys to movement and jump actions', () => {
const target = createTarget();
const actions: ChildMotionDebugAction[] = [];
const controller = createChildMotionDebugInputController({
target,
onAction: (action) => actions.push(action),
now: () => 120,
});
const leftEvent = dispatchKeyboard(target, { key: 'a', code: 'KeyA' });
dispatchKeyboard(target, { key: 'D', code: 'KeyD' });
dispatchKeyboard(target, { key: ' ', code: 'Space' });
expect(leftEvent.defaultPrevented).toBe(true);
expect(actions).toEqual([
{
kind: 'move',
direction: 'left',
source: 'keyboard',
occurredAtMs: 120,
},
{
kind: 'move',
direction: 'right',
source: 'keyboard',
occurredAtMs: 120,
},
{
kind: 'jump',
source: 'keyboard',
occurredAtMs: 120,
},
]);
controller.dispose();
});
test('ignores repeated or unrelated keyboard events', () => {
const unrelatedEvent = new KeyboardEvent('keydown', {
key: 'x',
code: 'KeyX',
});
const repeatEvent = new KeyboardEvent('keydown', {
key: 'a',
code: 'KeyA',
repeat: true,
});
expect(resolveKeyboardDebugAction(unrelatedEvent)).toBeNull();
expect(resolveKeyboardDebugAction(repeatEvent)).toBeNull();
});
test('maps left mouse drag to a left hand trajectory', () => {
const target = createTarget();
const actions: ChildMotionDebugAction[] = [];
const controller = createChildMotionDebugInputController({
target,
onAction: (action) => actions.push(action),
now: () => 240,
});
dispatchPointer(target, 'pointerdown', {
button: 0,
clientX: 10,
clientY: 20,
pointerId: 7,
});
dispatchPointer(target, 'pointermove', {
clientX: 18,
clientY: 24,
pointerId: 7,
});
dispatchPointer(target, 'pointerup', {
clientX: 22,
clientY: 28,
pointerId: 7,
});
expect(actions).toEqual([
{
kind: 'hand_trace',
hand: 'left',
phase: 'start',
pointerId: 7,
point: { x: 10, y: 20 },
path: [{ x: 10, y: 20 }],
source: 'pointer',
occurredAtMs: 240,
},
{
kind: 'hand_trace',
hand: 'left',
phase: 'move',
pointerId: 7,
point: { x: 18, y: 24 },
path: [
{ x: 10, y: 20 },
{ x: 18, y: 24 },
],
source: 'pointer',
occurredAtMs: 240,
},
{
kind: 'hand_trace',
hand: 'left',
phase: 'end',
pointerId: 7,
point: { x: 22, y: 28 },
path: [
{ x: 10, y: 20 },
{ x: 18, y: 24 },
{ x: 22, y: 28 },
],
source: 'pointer',
occurredAtMs: 240,
},
]);
controller.dispose();
});
test('maps right mouse drag to a right hand trajectory and prevents context menu', () => {
const target = createTarget();
const actions: ChildMotionDebugAction[] = [];
const controller = createChildMotionDebugInputController({
target,
onAction: (action) => actions.push(action),
now: () => 360,
});
const pointerDown = dispatchPointer(target, 'pointerdown', {
button: 2,
clientX: 30,
clientY: 40,
pointerId: 9,
});
dispatchPointer(target, 'pointermove', {
clientX: 44,
clientY: 48,
pointerId: 9,
});
dispatchPointer(target, 'pointercancel', {
clientX: 48,
clientY: 52,
pointerId: 9,
});
const contextMenuEvent = new MouseEvent('contextmenu', {
bubbles: true,
cancelable: true,
button: 2,
});
target.dispatchEvent(contextMenuEvent);
expect(pointerDown.defaultPrevented).toBe(true);
expect(contextMenuEvent.defaultPrevented).toBe(true);
expect(actions.map((action) => action.kind)).toEqual([
'hand_trace',
'hand_trace',
'hand_trace',
]);
expect(actions[0]).toMatchObject({
hand: 'right',
phase: 'start',
point: { x: 30, y: 40 },
});
expect(actions[2]).toMatchObject({
hand: 'right',
phase: 'cancel',
point: { x: 48, y: 52 },
});
controller.dispose();
});
test('can be disabled or disposed without emitting debug actions', () => {
const target = createTarget();
const onAction = vi.fn();
const controller = createChildMotionDebugInputController({
target,
onAction,
});
controller.setEnabled(false);
expect(controller.isEnabled()).toBe(false);
dispatchKeyboard(target, { key: 'a', code: 'KeyA' });
dispatchPointer(target, 'pointerdown', {
button: 0,
clientX: 10,
clientY: 20,
});
expect(onAction).not.toHaveBeenCalled();
controller.setEnabled(true);
dispatchKeyboard(target, { key: 'd', code: 'KeyD' });
expect(onAction).toHaveBeenCalledTimes(1);
controller.dispose();
dispatchKeyboard(target, { key: ' ', code: 'Space' });
expect(onAction).toHaveBeenCalledTimes(1);
});
});

287
src/services/child-motion-demo/childMotionDebugInput.ts Normal file
View File

@@ -0,0 +1,287 @@
export type ChildMotionDebugMoveDirection = 'left' | 'right';
export type ChildMotionDebugHand = 'left' | 'right';
export type ChildMotionDebugHandTracePhase = 'start' | 'move' | 'end' | 'cancel';
export type ChildMotionDebugPoint = {
x: number;
y: number;
};
export type ChildMotionDebugMoveAction = {
kind: 'move';
direction: ChildMotionDebugMoveDirection;
source: 'keyboard';
occurredAtMs: number;
};
export type ChildMotionDebugJumpAction = {
kind: 'jump';
source: 'keyboard';
occurredAtMs: number;
};
export type ChildMotionDebugHandTraceAction = {
kind: 'hand_trace';
hand: ChildMotionDebugHand;
phase: ChildMotionDebugHandTracePhase;
pointerId: number;
point: ChildMotionDebugPoint;
path: ChildMotionDebugPoint[];
source: 'pointer';
occurredAtMs: number;
};
export type ChildMotionDebugAction =
| ChildMotionDebugMoveAction
| ChildMotionDebugJumpAction
| ChildMotionDebugHandTraceAction;
type ChildMotionDebugActionPayload =
| Omit<ChildMotionDebugMoveAction, 'occurredAtMs'>
| Omit<ChildMotionDebugJumpAction, 'occurredAtMs'>
| Omit<ChildMotionDebugHandTraceAction, 'occurredAtMs'>;
export type ChildMotionDebugInputTarget = Pick<
EventTarget,
'addEventListener' | 'removeEventListener'
>;
export type ChildMotionDebugInputOptions = {
target: ChildMotionDebugInputTarget;
onAction: (action: ChildMotionDebugAction) => void;
enabled?: boolean;
now?: () => number;
preventContextMenu?: boolean;
};
export type ChildMotionDebugInputController = {
dispose: () => void;
isEnabled: () => boolean;
setEnabled: (enabled: boolean) => void;
};
type ActiveHandTrace = {
hand: ChildMotionDebugHand;
path: ChildMotionDebugPoint[];
};
const DEFAULT_POINTER_ID = 1;
export function createChildMotionDebugInputController(
options: ChildMotionDebugInputOptions,
): ChildMotionDebugInputController {
const { target, onAction, now = () => Date.now() } = options;
const preventContextMenu = options.preventContextMenu ?? true;
const activeHandTraces = new Map<number, ActiveHandTrace>();
let enabled = options.enabled ?? true;
const emit = (action: ChildMotionDebugActionPayload) => {
onAction({
...action,
occurredAtMs: now(),
});
};
const handleKeyDown = (event: Event) => {
if (!enabled) {
return;
}
const action = resolveKeyboardDebugAction(event);
if (!action) {
return;
}
event.preventDefault();
emit(action);
};
const handlePointerDown = (event: Event) => {
if (!enabled) {
return;
}
const hand = resolvePointerHand(event);
if (!hand) {
return;
}
event.preventDefault();
const pointerId = readPointerId(event);
const point = readPointerPoint(event);
const trace: ActiveHandTrace = {
hand,
path: [point],
};
activeHandTraces.set(pointerId, trace);
emit({
kind: 'hand_trace',
hand,
phase: 'start',
pointerId,
point,
path: trace.path,
source: 'pointer',
});
};
const handlePointerMove = (event: Event) => {
if (!enabled) {
return;
}
const pointerId = readPointerId(event);
const trace = activeHandTraces.get(pointerId);
if (!trace) {
return;
}
event.preventDefault();
const point = readPointerPoint(event);
trace.path = [...trace.path, point];
activeHandTraces.set(pointerId, trace);
emit({
kind: 'hand_trace',
hand: trace.hand,
phase: 'move',
pointerId,
point,
path: trace.path,
source: 'pointer',
});
};
const finishPointerTrace = (
event: Event,
phase: Extract<ChildMotionDebugHandTracePhase, 'end' | 'cancel'>,
) => {
if (!enabled) {
return;
}
const pointerId = readPointerId(event);
const trace = activeHandTraces.get(pointerId);
if (!trace) {
return;
}
event.preventDefault();
const point = readPointerPoint(event);
const path = [...trace.path, point];
activeHandTraces.delete(pointerId);
emit({
kind: 'hand_trace',
hand: trace.hand,
phase,
pointerId,
point,
path,
source: 'pointer',
});
};
const handlePointerUp = (event: Event) => finishPointerTrace(event, 'end');
const handlePointerCancel = (event: Event) =>
finishPointerTrace(event, 'cancel');
const handleContextMenu = (event: Event) => {
if (enabled && preventContextMenu) {
event.preventDefault();
}
};
target.addEventListener('keydown', handleKeyDown);
target.addEventListener('pointerdown', handlePointerDown);
target.addEventListener('pointermove', handlePointerMove);
target.addEventListener('pointerup', handlePointerUp);
target.addEventListener('pointercancel', handlePointerCancel);
target.addEventListener('contextmenu', handleContextMenu);
return {
dispose: () => {
activeHandTraces.clear();
target.removeEventListener('keydown', handleKeyDown);
target.removeEventListener('pointerdown', handlePointerDown);
target.removeEventListener('pointermove', handlePointerMove);
target.removeEventListener('pointerup', handlePointerUp);
target.removeEventListener('pointercancel', handlePointerCancel);
target.removeEventListener('contextmenu', handleContextMenu);
},
isEnabled: () => enabled,
setEnabled: (nextEnabled: boolean) => {
enabled = nextEnabled;
if (!enabled) {
activeHandTraces.clear();
}
},
};
}
export function resolveKeyboardDebugAction(
event: Event,
):
| Omit<ChildMotionDebugMoveAction, 'occurredAtMs'>
| Omit<ChildMotionDebugJumpAction, 'occurredAtMs'>
| null {
const keyboardEvent = event as KeyboardEvent;
if (keyboardEvent.repeat) {
return null;
}
const normalizedKey = keyboardEvent.key?.toLocaleLowerCase('en-US') ?? '';
const normalizedCode = keyboardEvent.code ?? '';
if (normalizedKey === 'a' || normalizedCode === 'KeyA') {
return {
kind: 'move',
direction: 'left',
source: 'keyboard',
};
}
if (normalizedKey === 'd' || normalizedCode === 'KeyD') {
return {
kind: 'move',
direction: 'right',
source: 'keyboard',
};
}
if (
keyboardEvent.key === ' ' ||
keyboardEvent.key === 'Spacebar' ||
normalizedCode === 'Space'
) {
return {
kind: 'jump',
source: 'keyboard',
};
}
return null;
}
function resolvePointerHand(event: Event): ChildMotionDebugHand | null {
const button = (event as MouseEvent).button;
if (button === 0) {
return 'left';
}
if (button === 2) {
return 'right';
}
return null;
}
function readPointerId(event: Event) {
const pointerId = (event as PointerEvent).pointerId;
return typeof pointerId === 'number' ? pointerId : DEFAULT_POINTER_ID;
}
function readPointerPoint(event: Event): ChildMotionDebugPoint {
const mouseEvent = event as MouseEvent;
return {
x: mouseEvent.clientX,
y: mouseEvent.clientY,
};
}

1
src/services/child-motion-demo/index.ts Normal file
View File

@@ -0,0 +1 @@
export * from './childMotionDebugInput';

19
src/services/input-devices/index.ts Normal file
View File

@@ -0,0 +1,19 @@
export {
createRuntimeDragInputController,
type RuntimeDragInputControllerOptions,
type RuntimeDragInputMove,
type RuntimeDragInputPress,
type RuntimeDragInputRelease,
type RuntimeDragInputSession,
type RuntimeInputDeviceKind,
type RuntimeInputPoint,
} from './runtimeDragInputController';
export {
createRuntimeInputPointFromClient,
createRuntimeInputPointFromNormalized,
readRuntimeInputElementBounds,
resolveRuntimeInputGridCell,
type RuntimeInputBounds,
type RuntimeInputGridCell,
type RuntimeInputGridSpec,
} from './runtimeInputGeometry';

161
src/services/input-devices/runtimeDragInputController.test.ts Normal file
View File

@@ -0,0 +1,161 @@
import { describe, expect, test, vi } from 'vitest';
import {
createRuntimeDragInputController,
createRuntimeInputPointFromNormalized,
resolveRuntimeInputGridCell,
} from './index';
describe('runtime drag input controller', () => {
test('pointer-like short press remains a tap', () => {
const onTap = vi.fn();
const onDrop = vi.fn();
const controller = createRuntimeDragInputController({
dragThresholdPx: 12,
onTap,
onDrop,
});
controller.press({
targetId: 'piece-1',
inputId: 'pointer:1',
deviceKind: 'pointer',
point: { clientX: 10, clientY: 10 },
});
controller.move({
inputId: 'pointer:1',
point: { clientX: 14, clientY: 14 },
});
controller.release({
inputId: 'pointer:1',
point: { clientX: 14, clientY: 14 },
});
expect(onTap).toHaveBeenCalledTimes(1);
expect(onTap).toHaveBeenCalledWith(
expect.objectContaining({ targetId: 'piece-1', dragging: false }),
);
expect(onDrop).not.toHaveBeenCalled();
});
test('device adapters can force continuous drag semantics', () => {
const onDragStart = vi.fn();
const onDragMove = vi.fn();
const onDrop = vi.fn();
const controller = createRuntimeDragInputController({
dragThresholdPx: 100,
onDragStart,
onDragMove,
onDrop,
});
controller.press({
targetId: 'piece-1',
inputId: 'mocap:hand',
deviceKind: 'mocap',
point: { clientX: 10, clientY: 10 },
});
controller.move({
inputId: 'mocap:hand',
point: { clientX: 11, clientY: 11 },
forceDragging: true,
});
controller.release({
inputId: 'mocap:hand',
point: { clientX: 12, clientY: 12 },
});
expect(onDragStart).toHaveBeenCalledTimes(1);
expect(onDragMove).toHaveBeenCalledTimes(1);
expect(onDrop).toHaveBeenCalledTimes(1);
expect(onDrop).toHaveBeenCalledWith(
expect.objectContaining({ deviceKind: 'mocap', dragging: true }),
);
});
test('device adapters can force drop on release without converting to tap', () => {
const onTap = vi.fn();
const onDrop = vi.fn();
const controller = createRuntimeDragInputController({
dragThresholdPx: 100,
onTap,
onDrop,
});
controller.press({
targetId: 'piece-1',
inputId: 'mocap:hand',
deviceKind: 'mocap',
point: { clientX: 10, clientY: 10 },
});
controller.release({
inputId: 'mocap:hand',
point: { clientX: 10, clientY: 10 },
forceDrop: true,
});
expect(onDrop).toHaveBeenCalledTimes(1);
expect(onDrop).toHaveBeenCalledWith(
expect.objectContaining({
targetId: 'piece-1',
forceDrop: true,
dragging: false,
}),
);
expect(onTap).not.toHaveBeenCalled();
});
test('input-scoped cancel keeps unrelated active sessions alive', () => {
const onCancel = vi.fn();
const onDrop = vi.fn();
const controller = createRuntimeDragInputController({
dragThresholdPx: 1,
onCancel,
onDrop,
});
controller.press({
targetId: 'piece-1',
inputId: 'pointer:1',
deviceKind: 'pointer',
point: { clientX: 10, clientY: 10 },
});
controller.cancel('mocap:hand');
controller.move({
inputId: 'pointer:1',
point: { clientX: 20, clientY: 20 },
});
controller.release({
inputId: 'pointer:1',
point: { clientX: 20, clientY: 20 },
});
expect(onCancel).not.toHaveBeenCalled();
expect(onDrop).toHaveBeenCalledTimes(1);
expect(onDrop).toHaveBeenCalledWith(
expect.objectContaining({ inputId: 'pointer:1', targetId: 'piece-1' }),
);
});
});
describe('runtime input geometry', () => {
test('normalised device coordinates map into client coordinates and grid cells', () => {
const point = createRuntimeInputPointFromNormalized(0.75, 0.25, {
left: 20,
top: 10,
width: 200,
height: 100,
});
expect(point).toEqual({
clientX: 170,
clientY: 35,
normalizedX: 0.75,
normalizedY: 0.25,
});
expect(resolveRuntimeInputGridCell(point, { rows: 4, cols: 4 })).toEqual({
row: 1,
col: 3,
});
});
});

168
src/services/input-devices/runtimeDragInputController.ts Normal file
View File

@@ -0,0 +1,168 @@
export type RuntimeInputDeviceKind = 'pointer' | 'mocap' | 'keyboard' | 'unknown';
export type RuntimeInputPoint = {
clientX: number;
clientY: number;
normalizedX?: number;
normalizedY?: number;
};
export type RuntimeDragInputSession<TTargetId extends string = string> = {
targetId: TTargetId;
inputId: string;
deviceKind: RuntimeInputDeviceKind;
startPoint: RuntimeInputPoint;
currentPoint: RuntimeInputPoint;
dragging: boolean;
forceDrop: boolean;
};
export type RuntimeDragInputPress<TTargetId extends string = string> = {
targetId: TTargetId;
inputId: string;
deviceKind: RuntimeInputDeviceKind;
point: RuntimeInputPoint;
};
export type RuntimeDragInputMove = {
inputId: string;
point: RuntimeInputPoint;
forceDragging?: boolean;
};
export type RuntimeDragInputRelease = {
inputId: string;
point: RuntimeInputPoint;
forceDrop?: boolean;
};
export type RuntimeDragInputControllerOptions<
TTargetId extends string = string,
> = {
dragThresholdPx?: number;
onPress?: (session: RuntimeDragInputSession<TTargetId>) => void;
onDragStart?: (session: RuntimeDragInputSession<TTargetId>) => void;
onDragMove?: (session: RuntimeDragInputSession<TTargetId>) => void;
onDrop?: (session: RuntimeDragInputSession<TTargetId>) => void;
onTap?: (session: RuntimeDragInputSession<TTargetId>) => void;
onCancel?: (session: RuntimeDragInputSession<TTargetId>) => void;
};
const DEFAULT_DRAG_THRESHOLD_PX = 8;
function clonePoint(point: RuntimeInputPoint): RuntimeInputPoint {
return { ...point };
}
function shouldStartDragging(
session: RuntimeDragInputSession,
point: RuntimeInputPoint,
thresholdPx: number,
forceDragging = false,
) {
if (session.dragging || forceDragging) {
return true;
}
return (
Math.hypot(
point.clientX - session.startPoint.clientX,
point.clientY - session.startPoint.clientY,
) >= thresholdPx
);
}
export function createRuntimeDragInputController<
TTargetId extends string = string,
>(initialOptions: RuntimeDragInputControllerOptions<TTargetId> = {}) {
let options = initialOptions;
let session: RuntimeDragInputSession<TTargetId> | null = null;
const setOptions = (
nextOptions: RuntimeDragInputControllerOptions<TTargetId>,
) => {
options = nextOptions;
};
const cancel = (inputId?: string) => {
if (inputId && session?.inputId !== inputId) {
return;
}
const activeSession = session;
session = null;
if (activeSession) {
options.onCancel?.(activeSession);
}
};
const press = (input: RuntimeDragInputPress<TTargetId>) => {
cancel();
session = {
targetId: input.targetId,
inputId: input.inputId,
deviceKind: input.deviceKind,
startPoint: clonePoint(input.point),
currentPoint: clonePoint(input.point),
dragging: false,
forceDrop: false,
};
options.onPress?.(session);
return session;
};
const move = (input: RuntimeDragInputMove) => {
if (!session || session.inputId !== input.inputId) {
return null;
}
const wasDragging = session.dragging;
session = {
...session,
currentPoint: clonePoint(input.point),
dragging: shouldStartDragging(
session,
input.point,
options.dragThresholdPx ?? DEFAULT_DRAG_THRESHOLD_PX,
input.forceDragging,
),
};
if (!wasDragging && session.dragging) {
options.onDragStart?.(session);
}
if (session.dragging) {
options.onDragMove?.(session);
}
return session;
};
const release = (input: RuntimeDragInputRelease) => {
if (!session || session.inputId !== input.inputId) {
return null;
}
const completedSession = {
...session,
currentPoint: clonePoint(input.point),
forceDrop: input.forceDrop === true,
};
session = null;
if (completedSession.dragging || completedSession.forceDrop) {
options.onDrop?.(completedSession);
} else {
options.onTap?.(completedSession);
}
return completedSession;
};
return {
cancel,
getSession: () => session,
move,
press,
release,
setOptions,
};
}

142
src/services/input-devices/runtimeInputGeometry.ts Normal file
View File

@@ -0,0 +1,142 @@
import type { RuntimeInputPoint } from './runtimeDragInputController';
export type RuntimeInputBounds = {
left: number;
top: number;
width: number;
height: number;
};
export type RuntimeInputGridSpec = {
rows: number;
cols: number;
};
export type RuntimeInputGridCell = {
row: number;
col: number;
};
function isFiniteNumber(value: number) {
return Number.isFinite(value);
}
function clamp01(value: number) {
return Math.min(1, Math.max(0, value));
}
function hasUsableBounds(
bounds: RuntimeInputBounds | null | undefined,
): bounds is RuntimeInputBounds {
return Boolean(
bounds &&
isFiniteNumber(bounds.left) &&
isFiniteNumber(bounds.top) &&
isFiniteNumber(bounds.width) &&
isFiniteNumber(bounds.height) &&
bounds.width > 0 &&
bounds.height > 0,
);
}
export function readRuntimeInputElementBounds(
element: Element | null | undefined,
): RuntimeInputBounds | null {
if (!element) {
return null;
}
const rect = element.getBoundingClientRect();
if (!rect.width || !rect.height) {
return null;
}
return {
left: rect.left,
top: rect.top,
width: rect.width,
height: rect.height,
};
}
export function createRuntimeInputPointFromClient(
clientX: number,
clientY: number,
bounds?: RuntimeInputBounds | null,
): RuntimeInputPoint {
if (!hasUsableBounds(bounds)) {
return { clientX, clientY };
}
return {
clientX,
clientY,
normalizedX: (clientX - bounds.left) / bounds.width,
normalizedY: (clientY - bounds.top) / bounds.height,
};
}
export function createRuntimeInputPointFromNormalized(
normalizedX: number,
normalizedY: number,
bounds?: RuntimeInputBounds | null,
): RuntimeInputPoint {
const x = clamp01(normalizedX);
const y = clamp01(normalizedY);
if (!hasUsableBounds(bounds)) {
return {
clientX: x,
clientY: y,
normalizedX: x,
normalizedY: y,
};
}
return {
clientX: bounds.left + x * bounds.width,
clientY: bounds.top + y * bounds.height,
normalizedX: x,
normalizedY: y,
};
}
export function resolveRuntimeInputGridCell(
point: RuntimeInputPoint,
grid: RuntimeInputGridSpec,
bounds?: RuntimeInputBounds | null,
): RuntimeInputGridCell | null {
if (grid.rows <= 0 || grid.cols <= 0) {
return null;
}
const normalizedX =
typeof point.normalizedX === 'number'
? point.normalizedX
: hasUsableBounds(bounds)
? (point.clientX - bounds.left) / bounds.width
: null;
const normalizedY =
typeof point.normalizedY === 'number'
? point.normalizedY
: hasUsableBounds(bounds)
? (point.clientY - bounds.top) / bounds.height
: null;
if (
normalizedX === null ||
normalizedY === null ||
!isFiniteNumber(normalizedX) ||
!isFiniteNumber(normalizedY) ||
normalizedX < 0 ||
normalizedX > 1 ||
normalizedY < 0 ||
normalizedY > 1
) {
return null;
}
return {
row: Math.min(grid.rows - 1, Math.floor(normalizedY * grid.rows)),
col: Math.min(grid.cols - 1, Math.floor(normalizedX * grid.cols)),
};
}

81
src/services/useMocapInput.test.ts Normal file
View File

@@ -0,0 +1,81 @@
import { describe, expect, test } from 'vitest';
import { parseMocapPacket, resolveMocapPalmCenter } from './useMocapInput';
describe('resolveMocapPalmCenter', () => {
test('优先用手腕和四个 MCP 点加权计算掌心派生点', () => {
const center = resolveMocapPalmCenter([
{ name: 'wrist', x: 0.1, y: 0.2 },
{ name: 'index_mcp', x: 0.3, y: 0.4 },
{ name: 'middle_mcp', x: 0.5, y: 0.6 },
{ name: 'ring_mcp', x: 0.7, y: 0.8 },
{ name: 'pinky_mcp', x: 0.9, y: 1 },
{ name: 'index_finger_tip', x: 1, y: 1 },
]);
expect(center?.x).toBeCloseTo(0.44);
expect(center?.y).toBeCloseTo(0.54);
});
test('可用掌心点少于三个时不返回掌心坐标', () => {
expect(
resolveMocapPalmCenter([
{ name: 'wrist', x: 0.1, y: 0.2 },
{ name: 'index_mcp', x: 0.3, y: 0.4 },
]),
).toBeNull();
});
});
describe('parseMocapPacket', () => {
test('解析手部数据时优先把 primaryHand 定位到掌心而不是腕部或指尖', () => {
const command = parseMocapPacket({
hands: [
{
state: 'open_palm',
x: 0.01,
y: 0.02,
landmarks: [
{ name: 'wrist', x: 0.1, y: 0.2 },
{ name: 'index_mcp', x: 0.3, y: 0.4 },
{ name: 'middle_mcp', x: 0.5, y: 0.6 },
{ name: 'ring_mcp', x: 0.7, y: 0.8 },
{ name: 'pinky_mcp', x: 0.9, y: 1 },
],
},
],
});
expect(command.primaryHand?.x).toBeCloseTo(0.44);
expect(command.primaryHand?.y).toBeCloseTo(0.54);
expect(command.primaryHand).toEqual(
expect.objectContaining({
state: 'open_palm',
source: 'palm_center',
}),
);
});
test('缺少足够掌心关键点时退回 wrist landmark，再退回 hand 直出坐标', () => {
const landmarkFallback = parseMocapPacket({
hands: [
{
state: 'grab',
x: 0.9,
y: 0.8,
landmarks: [{ name: 'wrist', x: 0.25, y: 0.75 }],
},
],
});
expect(landmarkFallback.primaryHand).toEqual(
expect.objectContaining({x: 0.25, y: 0.75, source: 'landmark'}),
);
const directFallback = parseMocapPacket({
hands: [{ state: 'grab', x: 0.9, y: 0.8 }],
});
expect(directFallback.primaryHand).toEqual(
expect.objectContaining({x: 0.9, y: 0.8, source: 'direct'}),
);
});
});

372
src/services/useMocapInput.ts
View File

@@ -3,14 +3,29 @@ import {useEffect, useMemo, useRef, useState} from 'react';
export type MocapConnectionStatus = 'idle' | 'connecting' | 'connected' | 'error';
export type MocapHandState = 'open_palm' | 'grab' | 'unknown';
export type MocapHandSide = 'left' | 'right' | 'unknown';
export type MocapHandSource = 'palm_center' | 'direct' | 'landmark';
export type MocapHandInput = {
x: number;
y: number;
state: MocapHandState;
side: MocapHandSide;
source?: MocapHandSource;
};
export type MocapBodyCenterInput = {
x: number;
y: number;
};
export type MocapInputCommand = {
actions: string[];
primaryHand?: {
x: number;
y: number;
state: MocapHandState;
} | null;
hands?: MocapHandInput[];
primaryHand?: MocapHandInput | null;
leftHand?: MocapHandInput | null;
rightHand?: MocapHandInput | null;
bodyCenter?: MocapBodyCenterInput | null;
parseWarnings?: string[];
};
@@ -32,9 +47,19 @@ export type UseMocapInputOptions = {
reconnectDelayMs?: number;
};
type MocapLandmarkRecord = Record<string, unknown>;
const DEFAULT_MOCAP_SERVICE_URL = 'http://127.0.0.1:8876';
const DEFAULT_RECONNECT_DELAY_MS = 1200;
const MAX_RAW_PACKET_PREVIEW_LENGTH = 360;
const PALM_CENTER_WEIGHTS = [
['wrist', 0.25],
['index_mcp', 0.2],
['middle_mcp', 0.25],
['ring_mcp', 0.2],
['pinky_mcp', 0.1],
] as const;
const MIN_PALM_CENTER_POINT_COUNT = 3;
function buildRawPacketPreview(rawData: unknown): string {
const rawText = typeof rawData === 'string' ? rawData : JSON.stringify(rawData);
@@ -57,89 +82,301 @@ function normalizeCoordinate(value: unknown) {
return Math.min(1, Math.max(0, numericValue));
}
function resolvePrimaryHand(hands: unknown) {
if (!Array.isArray(hands)) {
function resolveNormalizedPoint(value: unknown) {
if (Array.isArray(value)) {
const x = normalizeCoordinate(value[0]);
const y = normalizeCoordinate(value[1]);
if (x === null || y === null) {
return null;
}
return {x, y};
}
if (!value || typeof value !== 'object') {
return null;
}
for (const hand of hands) {
if (!hand || typeof hand !== 'object') {
continue;
}
const handRecord = hand as {state?: unknown; landmarks?: unknown; x?: unknown; y?: unknown};
const state = normaliseHandState(handRecord.state);
const directX = normalizeCoordinate(handRecord.x);
const directY = normalizeCoordinate(handRecord.y);
if (directX !== null && directY !== null) {
return {x: directX, y: directY, state};
}
if (!Array.isArray(handRecord.landmarks)) {
continue;
}
const landmarks = handRecord.landmarks as Array<Record<string, unknown>>;
const landmark =
landmarks.find((item) => item?.name === 'wrist') ?? landmarks[0];
const x = normalizeCoordinate(landmark?.x);
const y = normalizeCoordinate(landmark?.y);
if (x === null || y === null) {
continue;
}
return {x, y, state};
const pointRecord = value as {x?: unknown; y?: unknown};
const x = normalizeCoordinate(pointRecord.x);
const y = normalizeCoordinate(pointRecord.y);
if (x === null || y === null) {
return null;
}
return null;
return {x, y};
}
function resolveHandLike(record: unknown) {
if (!record || typeof record !== 'object') {
return null;
}
const handRecord = record as {state?: unknown; landmarks?: unknown; x?: unknown; y?: unknown};
const state = normaliseHandState(handRecord.state);
const directX = normalizeCoordinate(handRecord.x);
const directY = normalizeCoordinate(handRecord.y);
if (directX !== null && directY !== null) {
return {x: directX, y: directY, state};
}
if (!Array.isArray(handRecord.landmarks)) {
return null;
}
const landmarks = handRecord.landmarks as Array<Record<string, unknown>>;
const landmark =
landmarks.find((item) => item?.name === 'wrist') ?? landmarks[0];
function resolveLandmarkCoordinate(landmark: MocapLandmarkRecord | undefined) {
const x = normalizeCoordinate(landmark?.x);
const y = normalizeCoordinate(landmark?.y);
if (x === null || y === null) {
return null;
}
return {x, y, state};
return {x, y};
}
function normaliseHandState(state: unknown): MocapHandState {
if (state === 'grab' || state === 'open_palm') {
return state;
export function resolveMocapPalmCenter(
landmarks: Array<MocapLandmarkRecord>,
): {x: number; y: number} | null {
const landmarksByName = new Map(
landmarks
.filter((landmark) => typeof landmark?.name === 'string')
.map((landmark) => [String(landmark.name), landmark]),
);
const weightedPoints = PALM_CENTER_WEIGHTS.map(([name, weight]) => {
const point = resolveLandmarkCoordinate(landmarksByName.get(name));
return point ? {...point, weight: Number(weight)} : null;
}).filter((point): point is {x: number; y: number; weight: number} =>
Boolean(point),
);
if (weightedPoints.length < MIN_PALM_CENTER_POINT_COUNT) {
return null;
}
const weightTotal = weightedPoints.reduce((sum, point) => sum + point.weight, 0);
if (weightTotal <= 0) {
return null;
}
return {
x: weightedPoints.reduce((sum, point) => sum + point.x * point.weight, 0) / weightTotal,
y: weightedPoints.reduce((sum, point) => sum + point.y * point.weight, 0) / weightTotal,
};
}
function normaliseHandSide(side: unknown): MocapHandSide {
if (typeof side !== 'string') {
return 'unknown';
}
const normalized = side.trim().toLocaleLowerCase('en-US');
if (
normalized === 'left' ||
normalized === 'l' ||
normalized === 'left hand' ||
normalized === 'left_hand' ||
normalized === 'left-hand' ||
normalized === '左' ||
normalized === '左手'
) {
return 'left';
}
if (
normalized === 'right' ||
normalized === 'r' ||
normalized === 'right hand' ||
normalized === 'right_hand' ||
normalized === 'right-hand' ||
normalized === '右' ||
normalized === '右手'
) {
return 'right';
}
return 'unknown';
}
function parseMocapPacket(packet: unknown): MocapInputCommand {
function normalizeMocapAction(value: unknown) {
if (typeof value !== 'string') {
return null;
}
const normalized = value
.trim()
.toLocaleLowerCase('en-US')
.replace(/\s+/gu, '_')
.replace(/-/gu, '_');
return normalized || null;
}
function addMocapActions(actions: Set<string>, value: unknown) {
if (Array.isArray(value)) {
value.forEach((item) => addMocapActions(actions, item));
return;
}
if (value && typeof value === 'object') {
const actionRecord = value as Record<string, unknown>;
addMocapActions(actions, actionRecord.action);
addMocapActions(actions, actionRecord.actions);
addMocapActions(actions, actionRecord.gesture);
addMocapActions(actions, actionRecord.gestures);
addMocapActions(actions, actionRecord.event);
addMocapActions(actions, actionRecord.name);
addMocapActions(actions, actionRecord.type);
return;
}
const normalized = normalizeMocapAction(value);
if (normalized) {
actions.add(normalized);
}
}
function normaliseHandState(state: unknown): MocapHandState {
if (typeof state !== 'string') {
return 'unknown';
}
const normalized = state
.trim()
.toLocaleLowerCase('en-US')
.replace(/\s+/gu, '_')
.replace(/-/gu, '_');
if (
normalized === 'grab' ||
normalized === 'grabbing' ||
normalized === 'close' ||
normalized === 'fist' ||
normalized === 'closed_fist' ||
normalized === 'closed'
) {
return 'grab';
}
if (
normalized === 'open_palm' ||
normalized === 'open_palm_up' ||
normalized === 'open' ||
normalized === 'palm' ||
normalized === 'hand_open'
) {
return 'open_palm';
}
return 'unknown';
}
function resolveMocapHand(record: unknown, fallbackSide: MocapHandSide) {
if (!record || typeof record !== 'object') {
return null;
}
const handRecord = record as {
state?: unknown;
landmarks?: unknown;
x?: unknown;
y?: unknown;
side?: unknown;
handedness?: unknown;
label?: unknown;
hand?: unknown;
};
const state = normaliseHandState(handRecord.state);
const detectedSide = normaliseHandSide(
handRecord.side ??
handRecord.handedness ??
handRecord.label ??
handRecord.hand,
);
const side = detectedSide === 'unknown' ? fallbackSide : detectedSide;
if (Array.isArray(handRecord.landmarks)) {
const landmarks = handRecord.landmarks as Array<MocapLandmarkRecord>;
const palmCenter = resolveMocapPalmCenter(landmarks);
if (palmCenter) {
return {...palmCenter, state, side, source: 'palm_center' as const};
}
const landmark =
landmarks.find((item) => item?.name === 'wrist') ?? landmarks[0];
const fallbackPoint = resolveLandmarkCoordinate(landmark);
if (fallbackPoint) {
return {...fallbackPoint, state, side, source: 'landmark' as const};
}
}
const directX = normalizeCoordinate(handRecord.x);
const directY = normalizeCoordinate(handRecord.y);
if (directX !== null && directY !== null) {
return {x: directX, y: directY, state, side, source: 'direct' as const};
}
return null;
}
function resolveHands(packetRecord: Record<string, unknown>) {
const resolvedHands: MocapHandInput[] = [];
const packetHands = packetRecord.hands;
if (!Array.isArray(packetHands)) {
const leftHand = resolveMocapHand(
packetRecord.leftHand ?? packetRecord.left_hand,
'left',
);
const rightHand = resolveMocapHand(
packetRecord.rightHand ?? packetRecord.right_hand,
'right',
);
if (leftHand) {
resolvedHands.push(leftHand);
}
if (rightHand) {
resolvedHands.push(rightHand);
}
return resolvedHands;
}
for (const hand of packetHands) {
const resolvedHand = resolveMocapHand(hand, 'unknown');
if (resolvedHand) {
resolvedHands.push(resolvedHand);
}
}
return resolvedHands;
}
function resolveBodyCenter(packetRecord: Record<string, unknown>) {
const generalRecord =
packetRecord.general && typeof packetRecord.general === 'object'
? (packetRecord.general as Record<string, unknown>)
: null;
const bodyCandidates = [generalRecord?.body, packetRecord.body];
for (const bodyCandidate of bodyCandidates) {
if (!bodyCandidate || typeof bodyCandidate !== 'object') {
continue;
}
const bodyRecord = bodyCandidate as Record<string, unknown>;
const center = resolveNormalizedPoint(
bodyRecord.center_norm ?? bodyRecord.centerNorm ?? bodyRecord.center,
);
if (center) {
return center;
}
}
return null;
}
export function parseMocapPacket(packet: unknown): MocapInputCommand {
if (!packet || typeof packet !== 'object') {
return {actions: [], parseWarnings: ['packet 不是对象']};
}
const packetRecord = packet as {hands?: unknown};
const primaryHand = resolvePrimaryHand(packetRecord.hands);
const packetRecord = packet as Record<string, unknown>;
const hands = resolveHands(packetRecord);
const primaryHand = hands[0] ?? null;
const leftHand = hands.find((hand) => hand.side === 'left') ?? null;
const rightHand = hands.find((hand) => hand.side === 'right') ?? null;
const bodyCenter = resolveBodyCenter(packetRecord);
const actions = new Set<string>();
const parseWarnings: string[] = [];
if (!Array.isArray(packetRecord.hands)) {
addMocapActions(actions, packetRecord.actions);
addMocapActions(actions, packetRecord.action);
addMocapActions(actions, packetRecord.gesture);
addMocapActions(actions, packetRecord.gestures);
addMocapActions(actions, packetRecord.event);
addMocapActions(actions, packetRecord.name);
addMocapActions(actions, packetRecord.type);
if (!Array.isArray(packetRecord.hands) && hands.length === 0 && !bodyCenter) {
parseWarnings.push('缺少 hands 数组');
} else if (!primaryHand) {
} else if (!primaryHand && !bodyCenter) {
parseWarnings.push('hands 中没有可用坐标');
}
if (primaryHand?.state === 'grab') {
@@ -148,13 +385,22 @@ function parseMocapPacket(packet: unknown): MocapInputCommand {
if (primaryHand?.state === 'open_palm') {
actions.add('open_palm');
}
for (const hand of hands) {
if (hand.state !== 'unknown') {
actions.add(hand.state);
}
}
if (primaryHand && primaryHand.state === 'unknown') {
parseWarnings.push('手势 state 未识别');
}
return {
actions: Array.from(actions),
hands,
primaryHand,
leftHand,
rightHand,
bodyCenter,
parseWarnings,
};
}