.hermes/shared-memory/pitfalls.md

@@ -95,7 +95,7 @@
 
 - 现象：本地 `npm run dev` 因 `3101` 已占用、重复发布 SpacetimeDB wasm 编译太慢，或只想检查 `spacetime-module` 语法而被完整联调链路拖慢。
 - 原因：`npm run dev` 默认同时启动 SpacetimeDB standalone、发布 `server-rs/crates/spacetime-module`、启动 Rust `api-server`、主站 Vite 与后台 Vite；并非每个阶段都需要完整重启和重新发布。
-- 处理：`3101` 已被可复用 standalone 占用时使用 `npm run dev -- --skip-spacetime`；未修改 `spacetime-module` 时使用 `npm run dev -- --skip-publish`；只查模块语法时执行 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml`。
+- 处理：`npm run dev` 启动后会把实际 SpacetimeDB URL 记录到 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`。下次启动即使没有传 `--skip-spacetime`，脚本也会先检查 `spacetime.pid` 对应进程和该 URL 是否在线；在线则直接复用现有宿主。`3101` 已被可复用 standalone 占用时也可显式使用 `npm run dev -- --skip-spacetime`；未修改 `spacetime-module` 时使用 `npm run dev -- --skip-publish`；只查模块语法时执行 `cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml`。
 - 验证：`--skip-spacetime` 后脚本复用现有 `http://127.0.0.1:3101`；`--skip-publish` 后不再进入 publish 阶段；`cargo check -p spacetime-module --manifest-path server-rs/Cargo.toml` 能完成 Rust 语法和类型检查。
 - 关联：`docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md`、`scripts/dev-rust-stack.sh`。
 

docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md

@@ -35,13 +35,14 @@ npm run dev:rust
 
 1. 检查 `cargo`、`node` 与 `spacetime` CLI。
 2. Windows Git Bash 下如 `server-rs/.spacetimedb/local/bin/current/spacetimedb-cli.exe` 不存在，先把本机 `spacetime` 所在安装目录的 `bin/` 与 `spacetime.exe` 同步到 `server-rs/.spacetimedb/local/`。
-3. 启动 `spacetime --root-dir=server-rs/.spacetimedb/local start --edition standalone --listen-addr 127.0.0.1:3101`，确保本地数据库与 SpacetimeDB 内部日志不会落到开发者全局目录。
-4. 等待 SpacetimeDB 就绪：优先接受 `spacetime --root-dir=server-rs/.spacetimedb/local server ping http://127.0.0.1:3101` 输出中的 `Server is online:`；如果 Windows 下 SpacetimeDB CLI `2.1.0` 对已经监听的 standalone 仍打印 `502 Bad Gateway`，脚本会兜底请求 `http://127.0.0.1:3101/v1/ping`，只有该健康端点返回 `2xx` 时才放行。不能只依赖 CLI 退出码，因为 CLI 在 `502 Bad Gateway` 时也可能返回退出码 `0`。
-5. 执行 `spacetime publish <本地数据库名> --server http://127.0.0.1:3101 --module-path server-rs/crates/spacetime-module --build-options="--debug" -c=on-conflict --yes`，确保 publish 仍由 SpacetimeDB CLI 负责构建和发布模块，同时使用 debug 构建参数降低本地开发等待时间；当前开发阶段允许新版模块表结构变化且发生 schema 冲突时清除旧模块数据。
-6. 注入 `GENARRATIVE_API_*` 与 `GENARRATIVE_SPACETIME_*` 后启动默认 debug profile 的 `cargo run -p api-server`；直接运行 `api-server` 时，如未显式设置 `GENARRATIVE_SPACETIME_DATABASE`，服务端也会向上查找 `spacetime.local.json` 作为本地默认库名。
-7. 等待 `http://127.0.0.1:<api-port>/healthz` 返回 HTTP 响应后再启动 Vite，避免前端初始化请求早于 Rust `api-server` 监听完成并在终端刷出 `ECONNREFUSED 127.0.0.1:<api-port>`。
-8. 注入 `RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET` 后启动 Vite。
-9. 任一子进程退出时，脚本回收其余子进程。
+3. 启动 SpacetimeDB 前先检查 `server-rs/.spacetimedb/local/data/spacetime.pid`：如果 pid 对应进程仍存在，且同目录 `dev-rust-spacetime-url` 中记录的 URL 可被 `spacetime server ping` 判定在线，则直接复用该宿主；如果 URL 记录缺失，会依次尝试从 `logs/dev-rust-spacetime-start.log` 和 `logs/spacetime-standalone.log` 中解析最近一次监听地址兜底。否则按正常流程重新启动。
+4. 正常启动 `spacetime start --data-dir server-rs/.spacetimedb/local/data --listen-addr 127.0.0.1:3101`，确保本地数据库与 SpacetimeDB 内部日志落在项目数据目录中；启动成功后把实际 URL 写入 `server-rs/.spacetimedb/local/data/dev-rust-spacetime-url`。
+5. 等待 SpacetimeDB 就绪：优先接受 `spacetime server ping http://127.0.0.1:<spacetime-port>` 输出中的 `Server is online:`；如果 Windows 下 SpacetimeDB CLI `2.1.0` 对已经监听的 standalone 仍打印 `502 Bad Gateway`，脚本会兜底请求 `http://127.0.0.1:<spacetime-port>/v1/ping`，只有该健康端点返回 `2xx` 时才放行。不能只依赖 CLI 退出码，因为 CLI 在 `502 Bad Gateway` 时也可能返回退出码 `0`。
+6. 执行 `spacetime publish <本地数据库名> --server http://127.0.0.1:3101 --module-path server-rs/crates/spacetime-module --build-options="--debug" -c=on-conflict --yes`，确保 publish 仍由 SpacetimeDB CLI 负责构建和发布模块，同时使用 debug 构建参数降低本地开发等待时间；当前开发阶段允许新版模块表结构变化且发生 schema 冲突时清除旧模块数据。
+7. 注入 `GENARRATIVE_API_*` 与 `GENARRATIVE_SPACETIME_*` 后启动默认 debug profile 的 `cargo run -p api-server`；直接运行 `api-server` 时，如未显式设置 `GENARRATIVE_SPACETIME_DATABASE`，服务端也会向上查找 `spacetime.local.json` 作为本地默认库名。
+8. 等待 `http://127.0.0.1:<api-port>/healthz` 返回 HTTP 响应后再启动 Vite，避免前端初始化请求早于 Rust `api-server` 监听完成并在终端刷出 `ECONNREFUSED 127.0.0.1:<api-port>`。
+9. 注入 `RUST_SERVER_TARGET`、`GENARRATIVE_RUNTIME_SERVER_TARGET` 后启动 Vite。
+10. 任一子进程退出时，脚本回收其余子进程。
 
 Vite 代理覆盖范围：
 

scripts/dev-rust-stack.sh

@@ -191,6 +191,111 @@ port_from_listen_addr() {
   echo "${listen_addr##*:}"
 }
 
+spacetime_url_record_path() {
+  local data_dir="$1"
+  echo "${data_dir}/dev-rust-spacetime-url"
+}
+
+spacetime_start_log_path() {
+  local data_dir="$1"
+  echo "${data_dir}/logs/dev-rust-spacetime-start.log"
+}
+
+spacetime_standalone_log_path() {
+  local data_dir="$1"
+  echo "${data_dir}/logs/spacetime-standalone.log"
+}
+
+read_spacetime_pid() {
+  local data_dir="$1"
+  local pid_file="${data_dir}/spacetime.pid"
+
+  if [[ ! -f "${pid_file}" ]]; then
+    return 1
+  fi
+
+  local pid
+  pid="$(tr -cd '0-9' <"${pid_file}" | head -c 20)"
+  if [[ -z "${pid}" ]]; then
+    return 1
+  fi
+
+  echo "${pid}"
+}
+
+try_reuse_existing_spacetime() {
+  local data_dir="$1"
+  local url_file
+  url_file="$(spacetime_url_record_path "${data_dir}")"
+
+  local existing_pid
+  local recorded_url=""
+  if ! existing_pid="$(read_spacetime_pid "${data_dir}")"; then
+    return 1
+  fi
+
+  if ! kill -0 "${existing_pid}" 2>/dev/null; then
+    echo "[dev:rust] 发现过期 spacetime.pid: ${existing_pid}，将重新启动 SpacetimeDB。"
+    return 1
+  fi
+
+  if [[ ! -f "${url_file}" ]]; then
+    local start_log
+    start_log="$(spacetime_start_log_path "${data_dir}")"
+    if [[ -f "${start_log}" ]]; then
+      local logged_addr
+      logged_addr="$(sed -n 's/^.*Starting SpacetimeDB listening on \([^[:space:]]\+\).*$/\1/p' "${start_log}" | tail -n 1)"
+      if [[ -n "${logged_addr}" ]]; then
+        recorded_url="http://${logged_addr}"
+      fi
+    fi
+    if [[ -z "${recorded_url}" ]]; then
+      local standalone_log
+      standalone_log="$(spacetime_standalone_log_path "${data_dir}")"
+      if [[ -f "${standalone_log}" ]]; then
+        local standalone_addr
+        standalone_addr="$(sed -n 's/^.*Starting SpacetimeDB listening on \([^[:space:]]\+\).*$/\1/p' "${standalone_log}" | tail -n 1)"
+        if [[ -n "${standalone_addr}" ]]; then
+          recorded_url="http://${standalone_addr}"
+        fi
+      fi
+    fi
+    if [[ -z "${recorded_url}" ]]; then
+      echo "[dev:rust] 发现运行中的 SpacetimeDB pid=${existing_pid}，但未找到 URL 记录: ${url_file}。"
+      return 1
+    fi
+  else
+    recorded_url="$(head -n 1 "${url_file}" | tr -d '\r' | xargs)"
+  fi
+
+  if [[ -z "${recorded_url}" ]]; then
+    echo "[dev:rust] SpacetimeDB URL 记录为空: ${url_file}。"
+    return 1
+  fi
+
+  if is_spacetime_ready "${recorded_url}"; then
+    SPACETIME_SERVER="${recorded_url}"
+    SPACETIME_PORT="$(port_from_listen_addr "${recorded_url}")"
+    SPACETIME_REUSED_EXISTING=1
+    echo "[dev:rust] 复用已启动 SpacetimeDB: ${SPACETIME_SERVER} (pid=${existing_pid})"
+    return 0
+  fi
+
+  echo "[dev:rust] pid=${existing_pid} 存在，但 URL 不在线: ${recorded_url}，将重新启动 SpacetimeDB。"
+  return 1
+}
+
+record_spacetime_server_url() {
+  local data_dir="$1"
+  local server="$2"
+  local url_file
+  url_file="$(spacetime_url_record_path "${data_dir}")"
+
+  mkdir -p "${data_dir}"
+  printf '%s\n' "${server}" >"${url_file}"
+  echo "[dev:rust] 记录 SpacetimeDB URL: ${url_file} -> ${server}"
+}
+
 is_spacetime_ready() {
   local server="$1"
   local output
@@ -372,6 +477,7 @@ SKIP_PUBLISH=0
 PRESERVE_DATABASE=0
 MIGRATION_BOOTSTRAP_SECRET=""
 MIGRATION_BOOTSTRAP_SECRET_MODE="auto"
+SPACETIME_REUSED_EXISTING=0
 PIDS=()
 NAMES=()
 
@@ -547,35 +653,38 @@ echo "[dev:rust] api timeout: ${API_SERVER_TIMEOUT_SECONDS}s"
 
 if [[ "${SKIP_SPACETIME}" -ne 1 ]]; then
   mkdir -p "${SPACETIME_ROOT_DIR}" "${SPACETIME_DATA_DIR}"
-  SPACETIME_ROOT_OWNER="$(describe_spacetime_root_owner "${SPACETIME_DATA_DIR}")"
-  if [[ -n "${SPACETIME_ROOT_OWNER}" ]]; then
-    echo "[dev:rust] 当前 data-dir 已被其他 SpacetimeDB 实例占用，无法再次启动。" >&2
-    echo "[dev:rust] 如需复用，请传入占用实例实际端口并追加 --skip-spacetime；如需重启，请先停止下列进程。" >&2
-    echo "${SPACETIME_ROOT_OWNER}" >&2
-    exit 1
+  if ! try_reuse_existing_spacetime "${SPACETIME_DATA_DIR}"; then
+    SPACETIME_ROOT_OWNER="$(describe_spacetime_root_owner "${SPACETIME_DATA_DIR}")"
+    if [[ -n "${SPACETIME_ROOT_OWNER}" ]]; then
+      echo "[dev:rust] 当前 data-dir 已被其他 SpacetimeDB 实例占用，无法再次启动。" >&2
+      echo "[dev:rust] 如需复用，请确认 ${SPACETIME_DATA_DIR}/dev-rust-spacetime-url 记录了实例 URL，或传入占用实例实际端口并追加 --skip-spacetime；如需重启，请先停止下列进程。" >&2
+      echo "${SPACETIME_ROOT_OWNER}" >&2
+      exit 1
+    fi
+
+    SPACETIME_START_LOG="$(spacetime_start_log_path "${SPACETIME_DATA_DIR}")"
+    mkdir -p "$(dirname -- "${SPACETIME_START_LOG}")"
+    : >"${SPACETIME_START_LOG}"
+    echo "[dev:rust] 启动 spacetimedb"
+    (
+      cd "${SERVER_RS_DIR}"
+      # 当目标端口被占用时，SpacetimeDB 会询问是否使用最近的可用端口；
+      # 这里直接发送回车接受默认建议，再从启动日志解析实际监听端口。
+      printf '\n' | spacetime \
+        start \
+        --data-dir "${SPACETIME_DATA_DIR}" \
+        --listen-addr "${SPACETIME_HOST}:${SPACETIME_PORT}" \
+        --non-interactive
+    ) 2>&1 | tee "${SPACETIME_START_LOG}" &
+    PIDS+=("$!")
+    NAMES+=("spacetimedb")
+
+    SPACETIME_LISTEN_ADDR="$(wait_for_spacetime_listen_addr "${SPACETIME_START_LOG}" "${SPACETIME_TIMEOUT_SECONDS}" "${PIDS[0]:-}")"
+    SPACETIME_PORT="$(port_from_listen_addr "${SPACETIME_LISTEN_ADDR}")"
+    SPACETIME_SERVER="http://${SPACETIME_HOST}:${SPACETIME_PORT}"
+    echo "[dev:rust] spacetime actual: ${SPACETIME_SERVER}"
+    record_spacetime_server_url "${SPACETIME_DATA_DIR}" "${SPACETIME_SERVER}"
   fi
-
-  SPACETIME_START_LOG="${SPACETIME_DATA_DIR}/logs/dev-rust-spacetime-start.log"
-  mkdir -p "$(dirname -- "${SPACETIME_START_LOG}")"
-  : >"${SPACETIME_START_LOG}"
-  echo "[dev:rust] 启动 spacetimedb"
-  (
-    cd "${SERVER_RS_DIR}"
-    # 当目标端口被占用时，SpacetimeDB 会询问是否使用最近的可用端口；
-    # 这里直接发送回车接受默认建议，再从启动日志解析实际监听端口。
-    printf '\n' | spacetime \
-      start \
-      --data-dir "${SPACETIME_DATA_DIR}" \
-      --listen-addr "${SPACETIME_HOST}:${SPACETIME_PORT}" \
-      --non-interactive
-  ) 2>&1 | tee "${SPACETIME_START_LOG}" &
-  PIDS+=("$!")
-  NAMES+=("spacetimedb")
-
-  SPACETIME_LISTEN_ADDR="$(wait_for_spacetime_listen_addr "${SPACETIME_START_LOG}" "${SPACETIME_TIMEOUT_SECONDS}" "${PIDS[0]:-}")"
-  SPACETIME_PORT="$(port_from_listen_addr "${SPACETIME_LISTEN_ADDR}")"
-  SPACETIME_SERVER="http://${SPACETIME_HOST}:${SPACETIME_PORT}"
-  echo "[dev:rust] spacetime actual: ${SPACETIME_SERVER}"
 fi
 
 if [[ "${SKIP_PUBLISH}" -ne 1 ]]; then

src/components/puzzle-runtime/PuzzleRuntimeShell.test.tsx

@@ -25,6 +25,25 @@ vi.mock('../ResolvedAssetImage', () => ({
   ResolvedAssetImage: () => null,
 }));
 
+const mocapMock = vi.hoisted(() => ({
+  state: 'grab',
+  x: 0.42,
+  y: 0.58,
+}));
+
+vi.mock('../../services/useMocapInput', () => ({
+  useMocapInput: () => ({
+    status: 'connected',
+    latestCommand: {
+      actions: [mocapMock.state],
+      primaryHand: {x: mocapMock.x, y: mocapMock.y, state: mocapMock.state},
+      parseWarnings: [],
+    },
+    rawPacketPreview: {text: '{"hands":[{"state":"grab"}]}', receivedAtMs: 1},
+    error: null,
+  }),
+}));
+
 function createAuthValue() {
   return {
     user: null,
@@ -138,6 +157,150 @@ const clearedRun: PuzzleRunSnapshot = {
   },
 };
 
+test('拼图界面显示 mocap 连接状态和最近动作调试信息', () => {
+  renderPuzzleRuntime(
+    <PuzzleRuntimeShell
+      run={{
+        ...clearedRun,
+        currentLevel: {
+          ...clearedRun.currentLevel!,
+          status: 'playing',
+        },
+      }}
+      onBack={vi.fn()}
+      onSwapPieces={vi.fn()}
+      onDragPiece={vi.fn()}
+      onAdvanceNextLevel={vi.fn()}
+    />,
+  );
+
+  const debugPanel = screen.getByTestId('puzzle-mocap-debug');
+  expect(within(debugPanel).getByText('mocap: connected')).toBeTruthy();
+  expect(within(debugPanel).getByText('动作: grab')).toBeTruthy();
+  expect(within(debugPanel).getByText('手势: grab @ 0.42, 0.58')).toBeTruthy();
+  expect(within(debugPanel).getByText('解析: 无')).toBeTruthy();
+  expect(within(debugPanel).getByText(/原始:/)).toBeTruthy();
+});
+
+test('拼图界面在 mocap open_palm 时显示体感光标', () => {
+  mocapMock.state = 'open_palm';
+  mocapMock.x = 0.42;
+  mocapMock.y = 0.58;
+  renderPuzzleRuntime(
+    <PuzzleRuntimeShell
+      run={{
+        ...clearedRun,
+        currentLevel: {
+          ...clearedRun.currentLevel!,
+          status: 'playing',
+          startedAtMs: Date.now(),
+          remainingMs: 300_000,
+          timeLimitMs: 300_000,
+        },
+      }}
+      onBack={vi.fn()}
+      onSwapPieces={vi.fn()}
+      onDragPiece={vi.fn()}
+      onAdvanceNextLevel={vi.fn()}
+    />,
+  );
+
+  const cursor = screen.getByTestId('puzzle-mocap-cursor');
+  expect(cursor).toBeTruthy();
+  expect(cursor).toHaveStyle({left: '42%', top: '58%'});
+  mocapMock.state = 'grab';
+});
+
+test('抓握时会触发拖拽提交并在松开时落子', () => {
+  mocapMock.state = 'grab';
+  mocapMock.x = 0.34;
+  mocapMock.y = 0.34;
+  const onDragPiece = vi.fn();
+  const playingRun: PuzzleRunSnapshot = {
+    ...clearedRun,
+    currentLevel: {
+      ...clearedRun.currentLevel!,
+      status: 'playing',
+      startedAtMs: Date.now(),
+      remainingMs: 300_000,
+      timeLimitMs: 300_000,
+      board: {
+        ...clearedRun.currentLevel!.board,
+        allTilesResolved: false,
+        pieces: clearedRun.currentLevel!.board.pieces.map((piece) =>
+          piece.pieceId === 'piece-0'
+            ? {...piece, currentRow: 0, currentCol: 0}
+            : piece,
+        ),
+      },
+    },
+  };
+
+  const { container } = renderPuzzleRuntime(
+    <PuzzleRuntimeShell
+      run={playingRun}
+      onBack={vi.fn()}
+      onSwapPieces={vi.fn()}
+      onDragPiece={onDragPiece}
+      onAdvanceNextLevel={vi.fn()}
+    />,
+  );
+
+  const piece = container.querySelector('[data-piece-id="piece-0"]') as HTMLElement | null;
+  if (!piece) {
+    throw new Error('缺少测试拼图片');
+  }
+  const board = container.querySelector('[data-testid="puzzle-board"]') as HTMLElement | null;
+  if (!board) {
+    throw new Error('缺少测试棋盘');
+  }
+  board.getBoundingClientRect = () => ({
+    x: 0,
+    y: 0,
+    left: 0,
+    top: 0,
+    right: 300,
+    bottom: 300,
+    width: 300,
+    height: 300,
+    toJSON: () => ({}),
+  } as DOMRect);
+
+  act(() => {
+    dispatchPointerEvent(piece, 'pointerdown', {
+      pointerId: 11,
+      clientX: 40,
+      clientY: 40,
+    });
+  });
+  act(() => {
+    dispatchPointerEvent(piece, 'pointermove', {
+      pointerId: 11,
+      clientX: 70,
+      clientY: 70,
+    });
+  });
+  act(() => {
+    dispatchPointerEvent(piece, 'pointermove', {
+      pointerId: 11,
+      clientX: 140,
+      clientY: 140,
+    });
+  });
+  act(() => {
+    dispatchPointerEvent(piece, 'pointerup', {
+      pointerId: 11,
+      clientX: 140,
+      clientY: 140,
+    });
+  });
+
+  expect(onDragPiece).toHaveBeenCalledTimes(1);
+  expect(onDragPiece).toHaveBeenCalledWith(
+    expect.objectContaining({pieceId: 'piece-0'}),
+  );
+});
+
 test('通关后显示结算弹窗、排行榜和下一关按钮', () => {
   vi.useFakeTimers();
   const onAdvanceNextLevel = vi.fn();

src/components/puzzle-runtime/PuzzleRuntimeShell.tsx

@@ -23,6 +23,7 @@ import type {
   SwapPuzzlePiecesRequest,
 } from '../../../packages/shared/src/contracts/puzzleRuntimeSession';
 import { useResolvedAssetReadUrl } from '../../hooks/useResolvedAssetReadUrl';
+import { useMocapInput } from '../../services/useMocapInput';
 import { CHROME_ICONS, getNineSliceStyle, UI_CHROME } from '../../uiAssets';
 import { useAuthUi } from '../auth/AuthUiContext';
 import { PixelIcon } from '../PixelIcon';
@@ -283,6 +284,12 @@ type PuzzleHintDemoState = {
   offsetYPercent: number;
 };
 
+type PuzzleMocapCursorState = {
+  x: number;
+  y: number;
+  state: string;
+};
+
 function triggerPuzzlePiecePressHapticFeedback() {
   if (typeof navigator === 'undefined') {
     return;
@@ -367,6 +374,10 @@ export function PuzzleRuntimeShell({
     pieceId: string;
     groupId: string | null;
   } | null>(null);
+  const [mocapCursor, setMocapCursor] = useState<PuzzleMocapCursorState | null>(
+    null,
+  );
+  const mocapDragRef = useRef<{pieceId: string} | null>(null);
   const [dismissedClearKey, setDismissedClearKey] = useState<string | null>(
     null,
   );
@@ -397,6 +408,18 @@ export function PuzzleRuntimeShell({
   const { resolvedUrl: resolvedCoverImage } = useResolvedAssetReadUrl(
     currentLevel?.coverImageSrc ?? null,
   );
+  const mocapInput = useMocapInput({enabled: runtimeStatus === 'playing'});
+  const mocapActionsLabel =
+    mocapInput.latestCommand?.actions.length
+      ? mocapInput.latestCommand.actions.join(', ')
+      : '无';
+  const mocapHandLabel = mocapInput.latestCommand?.primaryHand
+    ? `${mocapInput.latestCommand.primaryHand.state} @ ${mocapInput.latestCommand.primaryHand.x.toFixed(2)}, ${mocapInput.latestCommand.primaryHand.y.toFixed(2)}`
+    : '无';
+  const mocapParseWarningLabel = mocapInput.latestCommand?.parseWarnings?.length
+    ? mocapInput.latestCommand.parseWarnings.join('；')
+    : '无';
+  const mocapRawPacketLabel = mocapInput.rawPacketPreview?.text ?? '未收到';
 
   useEffect(() => {
     currentLevelRef.current = currentLevel;
@@ -850,6 +873,49 @@ export function PuzzleRuntimeShell({
     return { row, col };
   };
 
+  const resolveMocapTargetCell = (x: number, y: number) => ({
+    row: Math.min(board.rows - 1, Math.max(0, Math.floor(y * board.rows))),
+    col: Math.min(board.cols - 1, Math.max(0, Math.floor(x * board.cols))),
+  });
+
+  const handleMocapInputCommand = () => {
+    const hand = mocapInput.latestCommand?.primaryHand;
+    if (runtimeStatus !== 'playing' || isInteractionLocked || !hand) {
+      mocapDragRef.current = null;
+      setMocapCursor(null);
+      return;
+    }
+
+    setMocapCursor({x: hand.x, y: hand.y, state: hand.state});
+    if (hand.state === 'grab') {
+      if (mocapDragRef.current) {
+        return;
+      }
+      const sourceCell = resolveMocapTargetCell(hand.x, hand.y);
+      const sourcePiece = pieceByCell.get(`${sourceCell.row}:${sourceCell.col}`) ?? null;
+      if (!sourcePiece || sourcePiece.mergedGroupId) {
+        return;
+      }
+      mocapDragRef.current = {pieceId: sourcePiece.pieceId};
+      setSelectedPieceId(sourcePiece.pieceId);
+      triggerPuzzlePiecePressHapticFeedback();
+      return;
+    }
+
+    const draggingPiece = mocapDragRef.current;
+    if (!draggingPiece) {
+      return;
+    }
+    const targetCell = resolveMocapTargetCell(hand.x, hand.y);
+    mocapDragRef.current = null;
+    setSelectedPieceId(null);
+    onDragPiece({
+      pieceId: draggingPiece.pieceId,
+      targetRow: targetCell.row,
+      targetCol: targetCell.col,
+    });
+  };
+
   const handlePiecePointerUp = (
     pieceId: string,
     event: React.PointerEvent<HTMLDivElement>,
@@ -973,7 +1039,6 @@ export function PuzzleRuntimeShell({
     isClearResultReady;
   const isInteractionLocked =
     isBusy || runtimeStatus !== 'playing' || Boolean(propDialog);
-
   const handleBackRequest = () => {
     if (hideExitControls) {
       return;
@@ -1085,6 +1150,10 @@ export function PuzzleRuntimeShell({
     }
   };
 
+  useEffect(() => {
+    handleMocapInputCommand();
+  }, [mocapInput.latestCommand?.primaryHand]);
+
   return (
     <div
       className={`puzzle-runtime-shell ${embedded ? 'relative h-full min-h-0 w-full' : 'fixed inset-0 z-[100]'} flex justify-center`}
@@ -1445,6 +1514,21 @@ export function PuzzleRuntimeShell({
                 />
               </div>
             ) : null}
+            {mocapCursor ? (
+              <div
+                data-testid="puzzle-mocap-cursor"
+                className={`pointer-events-none absolute z-[70] flex h-8 w-8 -translate-x-1/2 -translate-y-1/2 items-center justify-center rounded-full border-2 ${
+                  mocapCursor.state === 'grab'
+                    ? 'border-amber-200 bg-amber-400/90 text-amber-950'
+                    : 'border-cyan-200 bg-cyan-300/90 text-cyan-950'
+                } shadow-[0_10px_24px_rgba(15,23,42,0.25)]`}
+                style={{left: `${mocapCursor.x * 100}%`, top: `${mocapCursor.y * 100}%`}}
+              >
+                <span className="text-[10px] font-black leading-none">
+                  {mocapCursor.state === 'grab' ? '抓' : '手'}
+                </span>
+              </div>
+            ) : null}
             {mergeFlash ? (
               <div
                 key={mergeFlash.key}
@@ -1472,6 +1556,19 @@ export function PuzzleRuntimeShell({
               已选择
             </div>
           ) : null}
+          <div
+            data-testid="puzzle-mocap-debug"
+            className="w-[min(92vw,34rem)] rounded-[0.9rem] border border-white/20 bg-slate-950/70 px-3 py-2 font-mono text-[10px] leading-4 text-white shadow-[0_12px_32px_rgba(15,23,42,0.25)] backdrop-blur"
+          >
+            <div>mocap: {mocapInput.status}</div>
+            <div>动作: {mocapActionsLabel}</div>
+            <div>手势: {mocapHandLabel}</div>
+            <div>解析: {mocapParseWarningLabel}</div>
+            <div className="max-h-20 overflow-auto break-all text-white/75">
+              原始: {mocapRawPacketLabel}
+            </div>
+            {mocapInput.error ? <div>错误: {mocapInput.error}</div> : null}
+          </div>
           {canShowNextAction ? (
             <button
               type="button"

src/services/useMocapInput.ts

@@ -0,0 +1,244 @@
+import {useEffect, useMemo, useRef, useState} from 'react';
+
+export type MocapConnectionStatus = 'idle' | 'connecting' | 'connected' | 'error';
+
+export type MocapHandState = 'open_palm' | 'grab' | 'unknown';
+
+export type MocapInputCommand = {
+  actions: string[];
+  primaryHand?: {
+    x: number;
+    y: number;
+    state: MocapHandState;
+  } | null;
+  parseWarnings?: string[];
+};
+
+export type MocapRawPacketPreview = {
+  text: string;
+  receivedAtMs: number;
+};
+
+export type UseMocapInputResult = {
+  status: MocapConnectionStatus;
+  latestCommand: MocapInputCommand | null;
+  rawPacketPreview: MocapRawPacketPreview | null;
+  error: string | null;
+};
+
+export type UseMocapInputOptions = {
+  enabled: boolean;
+  serviceUrl?: string;
+  reconnectDelayMs?: number;
+};
+
+const DEFAULT_MOCAP_SERVICE_URL = 'http://127.0.0.1:8876';
+const DEFAULT_RECONNECT_DELAY_MS = 1200;
+const MAX_RAW_PACKET_PREVIEW_LENGTH = 360;
+
+function buildRawPacketPreview(rawData: unknown): string {
+  const rawText = typeof rawData === 'string' ? rawData : JSON.stringify(rawData);
+  return rawText.length > MAX_RAW_PACKET_PREVIEW_LENGTH
+    ? `${rawText.slice(0, MAX_RAW_PACKET_PREVIEW_LENGTH)}…`
+    : rawText;
+}
+
+function buildMocapStreamUrl(serviceUrl: string) {
+  const url = new URL('/stream', serviceUrl);
+  url.protocol = url.protocol === 'https:' ? 'wss:' : 'ws:';
+  return url.toString();
+}
+
+function normalizeCoordinate(value: unknown) {
+  const numericValue = Number(value);
+  if (!Number.isFinite(numericValue)) {
+    return null;
+  }
+  return Math.min(1, Math.max(0, numericValue));
+}
+
+function resolvePrimaryHand(hands: unknown) {
+  if (!Array.isArray(hands)) {
+    return null;
+  }
+
+  for (const hand of hands) {
+    if (!hand || typeof hand !== 'object') {
+      continue;
+    }
+
+    const handRecord = hand as {state?: unknown; landmarks?: unknown; x?: unknown; y?: unknown};
+    const state = normaliseHandState(handRecord.state);
+    const directX = normalizeCoordinate(handRecord.x);
+    const directY = normalizeCoordinate(handRecord.y);
+    if (directX !== null && directY !== null) {
+      return {x: directX, y: directY, state};
+    }
+    if (!Array.isArray(handRecord.landmarks)) {
+      continue;
+    }
+
+    const landmarks = handRecord.landmarks as Array<Record<string, unknown>>;
+    const landmark =
+      landmarks.find((item) => item?.name === 'wrist') ?? landmarks[0];
+    const x = normalizeCoordinate(landmark?.x);
+    const y = normalizeCoordinate(landmark?.y);
+    if (x === null || y === null) {
+      continue;
+    }
+
+    return {x, y, state};
+  }
+
+  return null;
+}
+
+function resolveHandLike(record: unknown) {
+  if (!record || typeof record !== 'object') {
+    return null;
+  }
+
+  const handRecord = record as {state?: unknown; landmarks?: unknown; x?: unknown; y?: unknown};
+  const state = normaliseHandState(handRecord.state);
+  const directX = normalizeCoordinate(handRecord.x);
+  const directY = normalizeCoordinate(handRecord.y);
+  if (directX !== null && directY !== null) {
+    return {x: directX, y: directY, state};
+  }
+  if (!Array.isArray(handRecord.landmarks)) {
+    return null;
+  }
+
+  const landmarks = handRecord.landmarks as Array<Record<string, unknown>>;
+  const landmark =
+    landmarks.find((item) => item?.name === 'wrist') ?? landmarks[0];
+  const x = normalizeCoordinate(landmark?.x);
+  const y = normalizeCoordinate(landmark?.y);
+  if (x === null || y === null) {
+    return null;
+  }
+
+  return {x, y, state};
+}
+
+function normaliseHandState(state: unknown): MocapHandState {
+  if (state === 'grab' || state === 'open_palm') {
+    return state;
+  }
+  return 'unknown';
+}
+
+function parseMocapPacket(packet: unknown): MocapInputCommand {
+  if (!packet || typeof packet !== 'object') {
+    return {actions: [], parseWarnings: ['packet 不是对象']};
+  }
+
+  const packetRecord = packet as {hands?: unknown};
+  const primaryHand = resolvePrimaryHand(packetRecord.hands);
+  const actions = new Set<string>();
+  const parseWarnings: string[] = [];
+  if (!Array.isArray(packetRecord.hands)) {
+    parseWarnings.push('缺少 hands 数组');
+  } else if (!primaryHand) {
+    parseWarnings.push('hands 中没有可用坐标');
+  }
+  if (primaryHand?.state === 'grab') {
+    actions.add('grab');
+  }
+  if (primaryHand?.state === 'open_palm') {
+    actions.add('open_palm');
+  }
+  if (primaryHand && primaryHand.state === 'unknown') {
+    parseWarnings.push('手势 state 未识别');
+  }
+
+  return {
+    actions: Array.from(actions),
+    primaryHand,
+    parseWarnings,
+  };
+}
+
+export function useMocapInput({
+  enabled,
+  serviceUrl = DEFAULT_MOCAP_SERVICE_URL,
+  reconnectDelayMs = DEFAULT_RECONNECT_DELAY_MS,
+}: UseMocapInputOptions): UseMocapInputResult {
+  const [status, setStatus] = useState<MocapConnectionStatus>('idle');
+  const [latestCommand, setLatestCommand] = useState<MocapInputCommand | null>(
+    null,
+  );
+  const [rawPacketPreview, setRawPacketPreview] = useState<MocapRawPacketPreview | null>(
+    null,
+  );
+  const [error, setError] = useState<string | null>(null);
+  const reconnectTimerRef = useRef<number | null>(null);
+  const websocketRef = useRef<WebSocket | null>(null);
+
+  const streamUrl = useMemo(() => buildMocapStreamUrl(serviceUrl), [serviceUrl]);
+
+  useEffect(() => {
+    if (!enabled || typeof WebSocket === 'undefined') {
+      setStatus('idle');
+      return;
+    }
+
+    let cancelled = false;
+
+    const connect = () => {
+      if (cancelled) {
+        return;
+      }
+      setStatus('connecting');
+      setError(null);
+
+      const websocket = new WebSocket(streamUrl);
+      websocketRef.current = websocket;
+      websocket.onopen = () => {
+        if (!cancelled) {
+          setStatus('connected');
+        }
+      };
+      websocket.onmessage = (event) => {
+        if (cancelled) {
+          return;
+        }
+        try {
+          const rawText = String(event.data);
+          setRawPacketPreview({
+            text: buildRawPacketPreview(rawText),
+            receivedAtMs: Date.now(),
+          });
+          setLatestCommand(parseMocapPacket(JSON.parse(rawText)));
+        } catch (parseError) {
+          setError(parseError instanceof Error ? parseError.message : 'mocap 数据解析失败');
+        }
+      };
+      websocket.onerror = () => {
+        if (!cancelled) {
+          setStatus('error');
+          setError('mocap 连接异常');
+        }
+      };
+      websocket.onclose = () => {
+        if (cancelled) {
+          return;
+        }
+        setStatus('error');
+        reconnectTimerRef.current = window.setTimeout(connect, reconnectDelayMs);
+      };
+    };
+
+    connect();
+
+    return () => {
+      cancelled = true;
+      if (reconnectTimerRef.current !== null) {
+        window.clearTimeout(reconnectTimerRef.current);
+      }
+      websocketRef.current?.close();
+    };
+  }, [enabled, reconnectDelayMs, streamUrl]);
+
+  return {status, latestCommand, rawPacketPreview, error};
+}