diff --git a/docs/technical/ADMIN_CONSOLE_SERVICE_DESIGN_2026-04-23.md b/docs/technical/ADMIN_CONSOLE_SERVICE_DESIGN_2026-04-23.md index 789eb88a..184b3737 100644 --- a/docs/technical/ADMIN_CONSOLE_SERVICE_DESIGN_2026-04-23.md +++ b/docs/technical/ADMIN_CONSOLE_SERVICE_DESIGN_2026-04-23.md @@ -154,47 +154,11 @@ claims 设计： 2. 当前 `SpacetimeDB server/database` 配置。 3. `SpacetimeDB` 数据库基础信息。 4. 当前 schema 表清单。 -5. 首批关键表的行数统计。 +5. schema 表清单对应的逐表行数统计。 -首批关键表固定覆盖： +表统计必须以 SpacetimeDB schema 返回的表名为唯一来源，`schemaTableNames` 的数量必须与 `tableStats` 的行数一致。后台服务只对 schema 中符合安全 SQL 标识符格式的表名发起 `SELECT COUNT(*)`，不提供任意 SQL 输入能力。 -1. `runtime_setting` -2. `runtime_snapshot` -3. `user_browse_history` -4. `profile_dashboard_state` -5. `profile_wallet_ledger` -6. `profile_played_world` -7. `profile_save_archive` -8. `story_session` -9. `story_event` -10. `battle_state` -11. `inventory_slot` -12. `quest_record` -13. `quest_log` -14. `treasure_record` -15. `npc_state` -16. `custom_world_profile` -17. `custom_world_gallery_entry` -18. `custom_world_agent_session` -19. `custom_world_agent_message` -20. `custom_world_agent_operation` -21. `custom_world_draft_card` -22. `big_fish_creation_session` -23. `big_fish_agent_message` -24. `big_fish_asset_slot` -25. `big_fish_runtime_run` -26. `puzzle_work_profile` -27. `puzzle_agent_session` -28. `puzzle_agent_message` -29. `puzzle_runtime_run` -30. `ai_task` -31. `ai_task_stage` -32. `ai_text_chunk` -33. `ai_result_reference` -34. `asset_object` -35. `asset_entity_binding` - -返回中的计数失败项必须带错误信息，不能静默吞掉。 +返回中的计数失败项必须带错误信息，不能静默吞掉。SpacetimeDB private 表或当前身份不可见的表可能在 `/sql` 下返回 `no such table` / `marked private`，这类项统一展示为“不可统计（private 或当前身份不可见）”，不作为整页读取失败处理。 ## 8. API 调试设计 diff --git a/docs/technical/ADMIN_WEB_CONSOLE_TECHNICAL_SOLUTION_2026-04-30.md b/docs/technical/ADMIN_WEB_CONSOLE_TECHNICAL_SOLUTION_2026-04-30.md index ab0a0294..a0a0ac71 100644 --- a/docs/technical/ADMIN_WEB_CONSOLE_TECHNICAL_SOLUTION_2026-04-30.md +++ b/docs/technical/ADMIN_WEB_CONSOLE_TECHNICAL_SOLUTION_2026-04-30.md @@ -253,9 +253,13 @@ export interface ProfileInviteCodeAdminResponse { 后端读取 SpacetimeDB schema 时必须请求 `/v1/database/{database}/schema?version=9`。SpacetimeDB 2.x schema HTTP API 缺少 `version` query 会返回 `400 missing field version`，后台页面只能展示读取异常，不能拿到真实表名。 +`schemaTableNames` 与 `tableStats` 必须采用同一份 schema 表清单生成，不能再用硬编码关键表白名单补齐统计项。后台右上角显示的表数量必须等于统计表格实际行数；schema 读取失败时两者均为空，并通过 `fetchErrors` 暴露读取失败原因。 + 后端读取表行数时必须按 SpacetimeDB 2.x `/sql` 响应解析：接口返回 statement result 数组，单条结果内的 `schema.elements` 描述列名，`rows` 是按列顺序排列的数组行，例如 `rows: [[0]]`。后台服务不能再假设响应是 `{ rows: [{ row_count: 0 }] }` 的对象行形状；为了兼容小版本差异，可保留对象行兜底解析。 -`tableStats` 中单表失败必须展示 `errorMessage`，不能让整页变成空白。 +`tableStats` 中单表失败必须展示 `errorMessage`，不能让整页变成空白。SpacetimeDB private 表或当前身份不可见的表在 `/sql` 下可能返回 `no such table` / `marked private`，后台服务必须将这类错误归一为“不可统计（private 或当前身份不可见）”，避免把预期的访问边界展示成原始 HTTP 400 故障。 + +线上如果大量表都显示“不可统计（private 或当前身份不可见）”，优先检查 `api-server` 启动环境中的 `GENARRATIVE_SPACETIME_TOKEN` / `GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN` 是否存在且属于目标库 owner。Jenkins 覆盖发布包时必须保留部署目录已有运行 token；只带迁移 token 不能让后台概览读取 private 表。 ### 4.6 API 调试 contract diff --git a/docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md b/docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md index c56afa75..1ab4d5fa 100644 --- a/docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md +++ b/docs/technical/RUST_LOCAL_AND_REMOTE_DEPLOYMENT_SCRIPTS_2026-04-22.md @@ -190,7 +190,7 @@ cd build/ ./stop.sh ``` -如果后续通过 Jenkins 的部署脚本把发布包覆盖到固定部署目录，部署阶段默认只替换 `web/`、`api-server`、`spacetime_module.wasm`、`migration-bootstrap-secret.txt`、`scripts/`、`.env*`、`start.sh`、`stop.sh`、`web-server.mjs`、`README.md` 等发布产物；后台管理前端位于 `web/admin/`，随 `web/` 一并覆盖。文件产物使用普通复制，`web/`、`scripts/` 等目录产物递归复制，不会删除部署目录中的 `.spacetimedb/`、`logs/`、`run/`、`deploy-state/`、`database-migrations/` 这类运行态目录。 +如果后续通过 Jenkins 的部署脚本把发布包覆盖到固定部署目录，部署阶段默认只替换 `web/`、`api-server`、`spacetime_module.wasm`、`migration-bootstrap-secret.txt`、`scripts/`、`.env*`、`start.sh`、`stop.sh`、`web-server.mjs`、`README.md` 等发布产物；后台管理前端位于 `web/admin/`，随 `web/` 一并覆盖。文件产物使用普通复制，`web/`、`scripts/` 等目录产物递归复制，不会删除部署目录中的 `.spacetimedb/`、`logs/`、`run/`、`deploy-state/`、`database-migrations/` 这类运行态目录。Jenkins 覆盖 `.env.local` 时会保留目标部署目录已有的 `GENARRATIVE_SPACETIME_TOKEN` / `GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN`，避免后台表统计在部署后失去读取 private 表所需的 owner 身份。 安全边界： @@ -211,7 +211,7 @@ cd build/ 1. Ubuntu x86_64。 2. 已安装 `node`，用于运行发布包内的 `web-server.mjs`。 3. 已安装 `spacetime` CLI，`start.sh` 会启动本地 SpacetimeDB 并发布 wasm。 -4. 业务密钥通过目标服务器环境变量或发布包同目录 `.env.local` 提供。 +4. 业务密钥通过目标服务器环境变量或发布包同目录 `.env.local` 提供；后台概览如果需要统计 private 表，`GENARRATIVE_SPACETIME_TOKEN` 必须是目标库 owner 或具备等效读取权限的 token。 ## 4. 与 M7 的关系 diff --git a/scripts/deploy-rust-remote.sh b/scripts/deploy-rust-remote.sh index c8dfdfef..5776afce 100644 --- a/scripts/deploy-rust-remote.sh +++ b/scripts/deploy-rust-remote.sh @@ -1255,12 +1255,12 @@ cat >"${TARGET_DIR}/README.md" <<'EOF' - \`GENARRATIVE_WEB_HOST\` / \`GENARRATIVE_WEB_PORT\` - \`GENARRATIVE_API_HOST\` / \`GENARRATIVE_API_PORT\` / \`GENARRATIVE_API_LOG\` - \`GENARRATIVE_SPACETIME_HOST\` / \`GENARRATIVE_SPACETIME_PORT\` -- \`GENARRATIVE_SPACETIME_SERVER_URL\` / \`GENARRATIVE_SPACETIME_DATABASE\` +- \`GENARRATIVE_SPACETIME_SERVER_URL\` / \`GENARRATIVE_SPACETIME_DATABASE\` / \`GENARRATIVE_SPACETIME_TOKEN\` - \`GENARRATIVE_SPACETIME_ROOT_DIR\`：默认使用发布目录下的 \`.spacetimedb/\`，同时承载本地 SpacetimeDB 运行数据与 CLI 身份。 - \`GENARRATIVE_SPACETIME_TIMEOUT_SECONDS\`：等待 SpacetimeDB 就绪的秒数，默认 \`60\`。 - \`GENARRATIVE_SPACETIME_MIGRATE_ON_CONFLICT\`：默认 \`true\`，普通发布遇到 schema 冲突时自动导出、清库发布、导入回灌；设为 \`false\` 时保留原始发布失败。 - \`GENARRATIVE_SPACETIME_MIGRATION_DIR\`：自动迁移 JSON 输出目录，默认 \`database-migrations//\`。 -- OSS、LLM、短信、微信等业务密钥仍通过目标服务器环境变量或同目录 \`.env.local\` 管理。 +- OSS、LLM、短信、微信、SpacetimeDB owner token 等业务密钥仍通过目标服务器环境变量或同目录 \`.env.local\` 管理；后台表统计读取 private 表时需要 \`GENARRATIVE_SPACETIME_TOKEN\` 对目标库有 owner 权限。 - 迁移引导密钥由构建发布包时随机生成，构建日志和服务器 \`start.sh\` 发布日志都会显示同一份密钥。 EOF replace_placeholder_in_file "${TARGET_DIR}/README.md" "__GENARRATIVE_BUILD_NAME__" "${BUILD_NAME}" diff --git a/scripts/jenkins-deploy-release.sh b/scripts/jenkins-deploy-release.sh index 91d49534..1187e051 100644 --- a/scripts/jenkins-deploy-release.sh +++ b/scripts/jenkins-deploy-release.sh @@ -13,7 +13,8 @@ usage() { 3. 把指定发布目录中的白名单产物复制覆盖到部署目录，后台前端随 web/admin/ 一并覆盖。 4. 如指定 --clear-database，则以清库模式执行新版本 start.sh。 5. 默认允许新版本 start.sh 在 schema 冲突时自动导出、清库发布、导入回灌。 - 6. 最后执行新版本 start.sh。 + 6. 覆盖 .env.local 时保留目标机已有 SpacetimeDB 运行 token，供 api-server 后台概览读取 private 表统计。 + 7. 最后执行新版本 start.sh。 参数: --source-dir 必填，待部署的发布目录，例如 build/123 @@ -179,6 +180,8 @@ MIGRATION_EXPORT_TOKEN="" MIGRATION_IMPORT_TOKEN="" PRESERVED_MIGRATION_EXPORT_TOKEN="" PRESERVED_MIGRATION_IMPORT_TOKEN="" +PRESERVED_SPACETIME_TOKEN="" +PRESERVED_SPACETIME_MAINCLOUD_TOKEN="" DEPLOY_ITEMS=( ".env" ".env.local" @@ -364,6 +367,8 @@ fi normalize_release_env_files "${SOURCE_DIR}" PRESERVED_MIGRATION_EXPORT_TOKEN="$(read_env_value "GENARRATIVE_SPACETIME_MIGRATION_EXPORT_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" PRESERVED_MIGRATION_IMPORT_TOKEN="$(read_env_value "GENARRATIVE_SPACETIME_MIGRATION_IMPORT_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" +PRESERVED_SPACETIME_TOKEN="$(read_env_value "GENARRATIVE_SPACETIME_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" +PRESERVED_SPACETIME_MAINCLOUD_TOKEN="$(read_env_value "GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" if [[ -x "${DEPLOY_DIR}/stop.sh" ]]; then echo "[jenkins-deploy] 先停止旧版本: ${DEPLOY_DIR}" @@ -424,6 +429,16 @@ elif [[ -n "${PRESERVED_MIGRATION_IMPORT_TOKEN}" ]] \ && [[ -z "$(read_env_value "GENARRATIVE_SPACETIME_MIGRATION_IMPORT_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" ]]; then write_env_override "${DEPLOY_DIR}/.env.local" "GENARRATIVE_SPACETIME_MIGRATION_IMPORT_TOKEN" "${PRESERVED_MIGRATION_IMPORT_TOKEN}" fi +if [[ -n "${PRESERVED_SPACETIME_TOKEN}" ]] \ + && [[ -z "$(read_env_value "GENARRATIVE_SPACETIME_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" ]] \ + && [[ -z "$(read_env_value "GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" ]]; then + write_env_override "${DEPLOY_DIR}/.env.local" "GENARRATIVE_SPACETIME_TOKEN" "${PRESERVED_SPACETIME_TOKEN}" +fi +if [[ -n "${PRESERVED_SPACETIME_MAINCLOUD_TOKEN}" ]] \ + && [[ -z "$(read_env_value "GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" ]] \ + && [[ -z "$(read_env_value "GENARRATIVE_SPACETIME_TOKEN" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" ]]; then + write_env_override "${DEPLOY_DIR}/.env.local" "GENARRATIVE_SPACETIME_MAINCLOUD_TOKEN" "${PRESERVED_SPACETIME_MAINCLOUD_TOKEN}" +fi DEPLOY_DATABASE="$(read_env_value "GENARRATIVE_SPACETIME_DATABASE" "${DEPLOY_DIR}/.env" "${DEPLOY_DIR}/.env.local")" if [[ -z "${DEPLOY_DATABASE}" ]]; then diff --git a/server-rs/crates/api-server/src/admin.rs b/server-rs/crates/api-server/src/admin.rs index c54b4867..00153464 100644 --- a/server-rs/crates/api-server/src/admin.rs +++ b/server-rs/crates/api-server/src/admin.rs @@ -39,43 +39,6 @@ const BLOCKED_DEBUG_HEADERS: &[&str] = &[ "transfer-encoding", "expect", ]; -// 数据库概览首版只统计受控白名单表，禁止后台页面直接输入任意 SQL。 -const DATABASE_OVERVIEW_TABLES: &[&str] = &[ - "runtime_setting", - "runtime_snapshot", - "user_browse_history", - "profile_dashboard_state", - "profile_wallet_ledger", - "profile_played_world", - "profile_save_archive", - "story_session", - "story_event", - "battle_state", - "inventory_slot", - "quest_record", - "quest_log", - "treasure_record", - "npc_state", - "custom_world_profile", - "custom_world_gallery_entry", - "custom_world_agent_session", - "custom_world_agent_message", - "custom_world_agent_operation", - "custom_world_draft_card", - "big_fish_creation_session", - "big_fish_agent_message", - "big_fish_asset_slot", - "puzzle_work_profile", - "puzzle_agent_session", - "puzzle_agent_message", - "puzzle_runtime_run", - "ai_task", - "ai_task_stage", - "ai_text_chunk", - "ai_result_reference", - "asset_object", - "asset_entity_binding", -]; // SpacetimeDB 2.x 的 schema HTTP API 要求显式传入 BSATN JSON 版本。 // 后台总览只读取表名，固定使用当前 CLI 2.1.0 兼容的版本参数即可。 const SPACETIME_SCHEMA_VERSION_QUERY: &str = "version=9"; @@ -283,7 +246,7 @@ async fn fetch_database_overview(state: &AppState) -> AdminDatabaseOverviewPaylo .ok() .flatten(); - let mut schema_table_names = schema + let schema_table_names = schema .as_ref() .and_then(|value| value.tables.as_ref()) .map(|tables| { @@ -300,31 +263,33 @@ async fn fetch_database_overview(state: &AppState) -> AdminDatabaseOverviewPaylo .unwrap_or_default(); let mut table_stats = Vec::new(); - for table_name in DATABASE_OVERVIEW_TABLES { + for table_name in &schema_table_names { + if !is_safe_spacetime_table_name(table_name) { + table_stats.push(AdminDatabaseTableStatPayload { + table_name: table_name.clone(), + row_count: None, + error_message: Some("表名不适合 SQL 统计".to_string()), + }); + continue; + } + let sql = format!("SELECT COUNT(*) AS row_count FROM {table_name}"); match fetch_spacetime_sql_count(&client, server_root, database, token, &sql).await { Ok(row_count) => table_stats.push(AdminDatabaseTableStatPayload { - table_name: (*table_name).to_string(), + table_name: table_name.clone(), row_count: Some(row_count), error_message: None, }), Err(error) => { table_stats.push(AdminDatabaseTableStatPayload { - table_name: (*table_name).to_string(), + table_name: table_name.clone(), row_count: None, - error_message: Some(error), + error_message: Some(normalize_table_count_error(&error)), }); } } } - for table_name in DATABASE_OVERVIEW_TABLES { - if !schema_table_names.iter().any(|name| name == table_name) { - schema_table_names.push((*table_name).to_string()); - } - } - schema_table_names.sort(); - AdminDatabaseOverviewPayload { database_identity: database_info .as_ref() @@ -345,6 +310,27 @@ fn build_spacetime_schema_url(server_root: &str, database: &str) -> String { format!("{server_root}/v1/database/{database}/schema?{SPACETIME_SCHEMA_VERSION_QUERY}") } +// 表名来自 schema，但进入 SQL 前仍做最小标识符校验，避免未来 schema 来源变化时扩大风险面。 +fn is_safe_spacetime_table_name(table_name: &str) -> bool { + let mut chars = table_name.chars(); + let Some(first) = chars.next() else { + return false; + }; + if !(first == '_' || first.is_ascii_alphabetic()) { + return false; + } + chars.all(|ch| ch == '_' || ch.is_ascii_alphanumeric()) +} + +// private 表在 SpacetimeDB SQL 下会表现为不可见，后台只展示可理解状态，不暴露整段 HTTP 噪音。 +fn normalize_table_count_error(error: &str) -> String { + let normalized = error.to_ascii_lowercase(); + if normalized.contains("marked private") || normalized.contains("no such table") { + return "不可统计（private 或当前身份不可见）".to_string(); + } + error.to_string() +} + async fn fetch_spacetime_json( client: &Client, url: &str, @@ -662,7 +648,8 @@ fn build_admin_session_payload(session: crate::state::AdminSession) -> AdminSess #[cfg(test)] mod tests { use super::{ - build_body_preview, build_debug_base_url, build_spacetime_schema_url, normalize_debug_path, + build_body_preview, build_debug_base_url, build_spacetime_schema_url, + is_safe_spacetime_table_name, normalize_debug_path, normalize_table_count_error, parse_spacetime_sql_count_response, trim_preview, }; use axum::{http::StatusCode, response::IntoResponse}; @@ -722,6 +709,38 @@ mod tests { ); } + #[test] + fn is_safe_spacetime_table_name_accepts_schema_identifiers() { + assert!(is_safe_spacetime_table_name("runtime_setting")); + assert!(is_safe_spacetime_table_name("_private_table")); + assert!(is_safe_spacetime_table_name("AiTaskStage2")); + } + + #[test] + fn is_safe_spacetime_table_name_rejects_sql_fragments() { + assert!(!is_safe_spacetime_table_name("")); + assert!(!is_safe_spacetime_table_name("bad-name")); + assert!(!is_safe_spacetime_table_name("1bad")); + assert!(!is_safe_spacetime_table_name("runtime_setting;DROP")); + } + + #[test] + fn normalize_table_count_error_hides_private_table_http_noise() { + let error = "HTTP 400：no such table: `runtime_setting`. If the table exists, it may be marked private."; + + assert_eq!( + normalize_table_count_error(error), + "不可统计（private 或当前身份不可见）" + ); + } + + #[test] + fn normalize_table_count_error_keeps_other_errors() { + let error = "SQL 请求失败：connection refused"; + + assert_eq!(normalize_table_count_error(error), error); + } + #[test] fn parse_spacetime_sql_count_response_accepts_statement_array_rows() { let payload = json!([