diff --git a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md index 1ad4de044..e00c42bc1 100644 --- a/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md +++ b/docs/【开发运维】本地开发验证与生产运维-2026-05-15.md @@ -591,6 +591,8 @@ Jenkins Copy Artifact 必须保持 `Production` 权限模式；产物生产者 `Genarrative-Web-Build` 的主站构建失败若出现 Rollup 报错 `"xxx" is not exported by "src/services/publicWorkCode.ts"`，优先按前端公开作品号工具缺失处理，而不是排查 Jenkins 节点环境。修复时要让 `publicWorkCode.ts` 的 `buildPublicWorkCode` 与 `isSamePublicWorkCode` 成对导出，并补 `src/services/publicWorkCode.test.ts` 覆盖对应玩法前缀；随后用 `npm run build:production-release -- --component web --name <临时名>` 复现 Jenkins web 构建路径。 +`Genarrative-Web-Build` 在运行根 Vitest 前必须分别按根 `package-lock.json` 和 `apps/ai-game-creator-shell/package-lock.json` 执行干净依赖安装，即先执行根 `npm ci`，再执行 `npm ci --prefix apps/ai-game-creator-shell`。根 Vitest 会收集 AI 游戏创作壳的测试，如果只安装根 lockfile，收集阶段会因缺少 `@tauri-apps/api`、`@tauri-apps/plugin-http` 等 Tauri guest 依赖而失败。这些依赖属于 AI 游戏创作壳的独立安装边界，不得为了让 Jenkins 通过而将它们加入根 H5 `package.json`；排查同类报错时先核对两份 lockfile 是否都已安装。 + `Genarrative-Web-Build` 会把 `build//web.tar.gz`、`web.tar.gz.sha256`、`release-manifest.json` 和 `scripts/deploy/production-web-deploy.sh` 直接归档为 Jenkins 构建产物；`Genarrative-Web-Deploy` 只通过 `copyArtifacts` 从指定上游构建复制这些产物和部署脚本，不再在目标机器 checkout Git，再执行随构建归档的 `scripts/deploy/production-web-deploy.sh`。Web 发布不再读取构建机本地缓存目录，也不再通过 release agent `rsync` 回构建机拉取大包；如果 deploy 找不到 `web.tar.gz`，应先检查上游 Web Build 是否按同一 `BUILD_VERSION` 成功归档产物。 `Genarrative-Api-Build` 的 Jenkins 归档产物必须包含 `build//api-server`、`api-server.sha256`、`release-manifest.json`、`build//scripts/deploy/production-api-deploy.sh`、`build//scripts/deploy/maintenance-on.sh`、`build//scripts/deploy/maintenance-off.sh`、`scripts/database-backup-to-oss.mjs`、`scripts/ops/production-health-patrol.mjs`、`scripts/ops/pingora-current-release-audit.mjs`、`scripts/ops/pingora-cutover-status-snapshot.mjs`、`scripts/ops/pingora-cutover-evidence-bundle.mjs`、`scripts/ops/pingora-cutover-command-evidence.mjs`、`scripts/ops/pingora-cutover-evidence-verify.mjs`、`scripts/ops/pingora-cutover-evidence-audit.mjs`、`scripts/check-pingora-direct-preflight.mjs`、`scripts/check-pingora-direct-live.mjs`、`scripts/check-pingora-canary-access-log-parity.mjs`、`scripts/check-production-health-patrol-env.mjs`、`scripts/deploy/pingora-direct-enable.sh`、`scripts/deploy/pingora-direct-rollback.sh`、`deploy/systemd/**`、`deploy/env/**` 和 `deploy/pingora/**`。`deploy/systemd/genarrative-database-backup.service` 从 `/opt/genarrative/current/scripts/database-backup-to-oss.mjs` 执行冷备份，`deploy/systemd/genarrative-health-patrol.service` 从 `/opt/genarrative/current/scripts/ops/production-health-patrol.mjs` 执行巡检；`Genarrative-Api-Deploy` 会从上游 API 构建产物复制并执行 `build//scripts/deploy/production-api-deploy.sh`，同目录的 `maintenance-on.sh` / `maintenance-off.sh` 也必须来自同一 build 产物；部署脚本会先写入 `${RELEASE_ROOT}/.${VERSION}.staging.$$`，把 `release-manifest.json` 校验后复制为 current release 的 `release-manifest.api-server.json`，并把备份脚本、巡检脚本、Pingora 直连启用 / 回退 / 预检 / live smoke / canary access log 对账 / health patrol env 复核 / current release 自审 / 状态快照 / 证据包 / 命令证据 / 证据验真 / 证据根目录审计脚本，以及 `deploy/systemd`、`deploy/env`、`deploy/pingora` 支撑配置全部复制完成后，才用非合并语义提升为 `${RELEASE_ROOT}/${VERSION}` 并用固定替换语义切换 `current` 符号链接，不再在目标机器 checkout Git，也不再执行部署工作区根部脚本。Pingora 直连启用脚本必须能从 `/opt/genarrative/current` 独立执行 preflight 和 direct live smoke，并默认读取 current release 随包 `deploy/systemd/genarrative-pingora-gateway-direct-entry.conf`，不依赖 Jenkins 工作区、源码 checkout 或 `/etc` 参考模板；`plan:pingora-direct-cutover` 必须能用同一组参数生成 current release 切换 / 回退 runbook。`production-api-deploy.sh` 对 release manifest、备份脚本、巡检脚本、env 示例目录和 Pingora 直连依赖都执行 fail-fast，且 `--release-root`、`--current-link`、`--api-env-file` 必须是绝对路径，`--version` 必须以数字或字母开头并只能包含数字、字母、点、下划线和短横线，禁止 `.` / `..` 点目录；发布产物缺少 manifest、manifest 未登记 `api-server`、缺少脚本 / 配置目录、同版本 release 目录已存在、current 路径不是符号链接、提升前 release 目录竞态出现或 staging 构建中失败时会在 current 切换前停止部署，清理 staging 并退出本次打开的维护模式；current 切换后的 Pingora 重启、worker 重启、controller 启动或 readiness 失败仍保留维护模式，避免暴露半发布版本；失败不会留下正式 release 目录，不再从部署机工作区兜底补文件，也不把旧同名 release 目录和新文件混合。如果 API 发布后 current release 中缺少这些脚本或目录，应先检查 `Genarrative-Api-Build` 的 `archiveArtifacts` 和 `Genarrative-Api-Deploy` 的 `copyArtifacts` 过滤器是否仍包含 `build//release-manifest.json`、`build//scripts/deploy/production-api-deploy.sh`、`build//scripts/deploy/maintenance-on.sh`、`build//scripts/deploy/maintenance-off.sh`、`build//scripts/database-backup-to-oss.mjs`、`build//scripts/ops/production-health-patrol.mjs`、`build//scripts/ops/pingora-current-release-audit.mjs`、`build//scripts/ops/pingora-cutover-status-snapshot.mjs`、`build//scripts/ops/pingora-cutover-evidence-bundle.mjs`、`build//scripts/ops/pingora-cutover-command-evidence.mjs`、`build//scripts/ops/pingora-cutover-evidence-verify.mjs`、`build//scripts/ops/pingora-cutover-evidence-audit.mjs`、`build//scripts/check-pingora-direct-preflight.mjs`、`build//scripts/check-pingora-direct-live.mjs`、`build//scripts/check-pingora-canary-access-log-parity.mjs`、`build//scripts/check-production-health-patrol-env.mjs`、`build//scripts/deploy/pingora-direct-enable.sh`、`build//scripts/deploy/pingora-direct-rollback.sh`、`build//deploy/systemd/**`、`build//deploy/env/**` 与 `build//deploy/pingora/**`，不要只在部署机工作区手工补文件。本机用 `npm run check:production-api-release` 通过临时 `CARGO_TARGET_DIR` 和假 `api-server` / `pingora-gateway` release binary 验证 `build-production-release.sh --component api-server --skip-api-build` 会把这些文件打进 API release，并验证显式 `--include-pingora-gateway --skip-pingora-gateway-build` 时发布包包含 `pingora-gateway`、`pingora-gateway.sha256` 和 manifest 登记；`npm run check:pingora-production-release-build` 则用假 `api-server` 和真实 `cargo build -p pingora-gateway --release --target x86_64-unknown-linux-gnu` 验证显式 include 路径能构出可执行网关二进制、checksum 和 manifest 登记；再用 `npm run check:production-api-deploy` 通过临时 release、fake `systemctl` / `curl` 验证从发布产物内执行 `production-api-deploy.sh` 会把这些文件复制到 current release，并验证缺少 release manifest、manifest 未登记 `api-server`、Pingora manifest / 二进制漂移、`--require-pingora-gateway` 缺少 Pingora、缺少数据库备份脚本、健康巡检脚本、健康巡检 env 复核脚本、current release 自审脚本、状态快照脚本、证据包脚本、证据验真脚本、证据根目录审计脚本、canary access log 对账脚本、env 示例目录或 direct live smoke 脚本时都会在 current 切换前失败并退出本次打开的维护模式，还会验证相对 release root / current link / api env file、点目录或点开头 version 被拒绝、失败时不留下 staging / 正式 release 目录、同版本 release 目录已存在、current 路径不是符号链接、提升前 release 目录竞态出现时拒绝覆盖 / 合并，以及 current 切换后的 readiness 失败会保留维护模式。Pingora 影子网关在 `Genarrative-Api-Build`、`Genarrative-Api-Deploy` 和 `Genarrative-Full-Build-And-Deploy` 中默认随 release 构建、归档、复制并用 `--require-pingora-gateway` 硬校验；只有显式取消 `INCLUDE_PINGORA_GATEWAY` 时才允许 API release 不带 `pingora-gateway` / `pingora-gateway.sha256`。本地 CLI 仍保留显式 `npm run build:production-release -- --component api-server --include-pingora-gateway`，用于在需要 Pingora 的手工发布包里登记 manifest 和 checksum；Jenkins 会先检查 `cmake`、C 编译器和 C++ 编译器，避免进入 Cargo 后才因 `libz-ng-sys` 构建依赖缺失失败。发布包包含 Pingora 时，deploy 会在提升 release 前读取 `systemctl cat genarrative-pingora-gateway.service` 和其 `EnvironmentFile`，拒绝 direct-entry `CAP_NET_BIND_SERVICE`、拒绝非 `127.0.0.1:18081` 的 shadow listen、拒绝 `TLS_LISTEN` / `HTTP_REDIRECT_LISTEN`，确认仍是本机 shadow 高端口后才切换 current；切换后执行 `systemctl restart genarrative-pingora-gateway.service` 并复核 active，让 shadow / canary 机器加载新网关二进制。该自动拉起不会启用公网 `80/443` 直连入口；已经进入 direct-entry 状态的机器应走正式直连 runbook 或先回退到 shadow。 diff --git a/jenkins/Jenkinsfile.production-full-build-and-deploy b/jenkins/Jenkinsfile.production-full-build-and-deploy index 64b032762..de098683f 100644 --- a/jenkins/Jenkinsfile.production-full-build-and-deploy +++ b/jenkins/Jenkinsfile.production-full-build-and-deploy @@ -22,7 +22,7 @@ pipeline { string(name: 'SOURCE_BRANCH', defaultValue: 'master', description: '源码分支，默认 master 最新提交') string(name: 'COMMIT_HASH', defaultValue: '', description: '可选，指定属于 SOURCE_BRANCH 的 Git commit') string(name: 'BUILD_VERSION', defaultValue: '', description: '发布版本号，留空则使用 Jenkins BUILD_NUMBER') - booleanParam(name: 'RUN_NPM_CI', defaultValue: true, description: 'Web 构建前是否执行 npm ci') + booleanParam(name: 'RUN_NPM_CI', defaultValue: true, description: 'Web 构建前是否按根 lockfile 与 AGC 独立 lockfile 执行 npm ci') string(name: 'NOTIFICATION_EMAILS', defaultValue: '', description: '本次运行追加通知邮箱；会与 Jenkins Secret Text 凭据 genarrative-notification-emails 合并发送') string(name: 'MIGRATION_BOOTSTRAP_SECRET_CREDENTIAL_ID', defaultValue: 'genarrative-spacetime-bootstrap-secret-dev-file', description: '必填：Stdb module 构建与发布共用的 Jenkins Secret File 凭据 ID') booleanParam(name: 'INCLUDE_PINGORA_GATEWAY', defaultValue: true, description: 'API release 是否构建、归档并部署 Pingora 影子网关；release 默认必须包含') diff --git a/jenkins/Jenkinsfile.production-web-build b/jenkins/Jenkinsfile.production-web-build index 200483d69..1413d6113 100644 --- a/jenkins/Jenkinsfile.production-web-build +++ b/jenkins/Jenkinsfile.production-web-build @@ -22,7 +22,7 @@ pipeline { string(name: 'COMMIT_HASH', defaultValue: '', description: '可选，指定属于 SOURCE_BRANCH 的 Git commit') string(name: 'BUILD_VERSION', defaultValue: '', description: '发布版本号，留空则使用 Jenkins BUILD_NUMBER') string(name: 'NOTIFICATION_EMAILS', defaultValue: '', description: '本次运行追加通知邮箱；会与 Jenkins Secret Text 凭据 genarrative-notification-emails 合并发送') - booleanParam(name: 'RUN_NPM_CI', defaultValue: true, description: '构建前是否执行 npm ci') + booleanParam(name: 'RUN_NPM_CI', defaultValue: true, description: '构建前是否按根 lockfile 与 AGC 独立 lockfile 执行 npm ci') booleanParam(name: 'PUBLISH_AFTER_BUILD', defaultValue: false, description: '构建成功后是否触发 Web 发布') string(name: 'DEPLOY_JOB_NAME', defaultValue: 'Genarrative-Web-Deploy', description: 'Web 发布流水线作业名') choice(name: 'DEPLOY_TARGET', choices: ['development', 'release'], description: 'PUBLISH_AFTER_BUILD=true 时的逻辑部署目标；development 使用当前 Linux 开发/构建/开发部署 agent') @@ -91,6 +91,7 @@ pipeline { script { if (params.RUN_NPM_CI) { sh 'bash -lc "npm ci"' + sh 'bash -lc "npm ci --prefix apps/ai-game-creator-shell"' } } sh ''' diff --git a/scripts/check-production-ops-guardrails.mjs b/scripts/check-production-ops-guardrails.mjs index ee0975dc9..8caf8b132 100644 --- a/scripts/check-production-ops-guardrails.mjs +++ b/scripts/check-production-ops-guardrails.mjs @@ -7376,6 +7376,38 @@ if ((fullPipelineMaintenanceHoldCalls?.length ?? 0) !== 2) { ); } +const webBuildContent = readFileSync( + 'jenkins/Jenkinsfile.production-web-build', + 'utf8', +); +const webBuildStageOffset = webBuildContent.indexOf("stage('Build Web')"); +const webArchiveStageOffset = webBuildContent.indexOf( + "stage('Archive')", + webBuildStageOffset, +); +const webBuildStageContent = + webBuildStageOffset >= 0 && webArchiveStageOffset > webBuildStageOffset + ? webBuildContent.slice(webBuildStageOffset, webArchiveStageOffset) + : ''; +const webNpmCiBlock = `if (params.RUN_NPM_CI) { + sh 'bash -lc "npm ci"' + sh 'bash -lc "npm ci --prefix apps/ai-game-creator-shell"' + }`; +const webNpmCiBlockOffset = webBuildStageContent.indexOf(webNpmCiBlock); +const webTestOffset = webBuildStageContent.indexOf('npm run test'); +const webNpmCiCalls = webBuildStageContent.match(/\bnpm ci(?:\s|["'])/gu); +if ( + webNpmCiBlockOffset < 0 || + webTestOffset < 0 || + webNpmCiBlockOffset >= webTestOffset || + (webNpmCiCalls?.length ?? 0) !== 2 +) { + failed = true; + console.error( + '[check:production-ops] Web Build 必须在 RUN_NPM_CI 条件块内依次安装根 lockfile 与 apps/ai-game-creator-shell 独立 lockfile，并在 npm run test 前完成；RUN_NPM_CI=false 时两次安装必须一起跳过。', + ); +} + const exitMaintenanceStageOffset = fullPipelineContent.indexOf( "stage('Exit Maintenance')", );