增强 Gitea CI 网络稳定性
Project CI / Repository checks (push) Failing after 51s
Project CI / Native shell tests (push) Failing after 52s
Project CI / Frontend tests (push) Successful in 3m13s
Project CI / Backend tests (push) Successful in 3m46s

为所有 npm ci 增加整命令级有界重试。

补齐 AI 游戏创作 npm 与 Cargo 依赖缓存闭合验证。

更新可信 CI 镜像、缓存漂移告警和 Runner 运维文档。

记录 Actions Gateway CONNECT 断连崩溃的处理与验证边界。
This commit is contained in:
2026-08-07 19:22:33 +08:00
parent 6c53eda71c
commit 9c2f02db25
10 changed files with 286 additions and 16 deletions
+6 -6
View File
@@ -67,7 +67,7 @@ jobs:
echo "SPACETIME_SCHEMA_BASE_REF=${base_ref}" >> "${GITHUB_ENV}" echo "SPACETIME_SCHEMA_BASE_REF=${base_ref}" >> "${GITHUB_ENV}"
- name: Install npm dependencies - name: Install npm dependencies
run: npm ci run: bash scripts/ci-npm-ci-with-retry.sh
- name: Run repository lint gates - name: Run repository lint gates
run: npm run lint run: npm run lint
@@ -101,10 +101,10 @@ jobs:
run: GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1 bash scripts/check-gitea-ci-job-image.sh run: GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1 bash scripts/check-gitea-ci-job-image.sh
- name: Install npm dependencies - name: Install npm dependencies
run: npm ci run: bash scripts/ci-npm-ci-with-retry.sh
- name: Install AI game creator dependencies - name: Install AI game creator dependencies
run: npm ci --prefix apps/ai-game-creator-shell run: bash scripts/ci-npm-ci-with-retry.sh --prefix apps/ai-game-creator-shell
- name: Run frontend and script tests - name: Run frontend and script tests
run: npm run test run: npm run test
@@ -159,7 +159,7 @@ jobs:
echo "SPACETIME_SCHEMA_BASE_REF=${base_ref}" >> "${GITHUB_ENV}" echo "SPACETIME_SCHEMA_BASE_REF=${base_ref}" >> "${GITHUB_ENV}"
- name: Install npm dependencies - name: Install npm dependencies
run: npm ci run: bash scripts/ci-npm-ci-with-retry.sh
- name: Prepare server-rs Rust dependencies - name: Prepare server-rs Rust dependencies
shell: bash shell: bash
@@ -204,10 +204,10 @@ jobs:
run: GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1 bash scripts/check-gitea-ci-job-image.sh run: GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1 bash scripts/check-gitea-ci-job-image.sh
- name: Install npm dependencies - name: Install npm dependencies
run: npm ci run: bash scripts/ci-npm-ci-with-retry.sh
- name: Install AI game creator dependencies - name: Install AI game creator dependencies
run: npm ci --prefix apps/ai-game-creator-shell run: bash scripts/ci-npm-ci-with-retry.sh --prefix apps/ai-game-creator-shell
- name: Prepare native Rust dependencies - name: Prepare native Rust dependencies
shell: bash shell: bash
+3 -3
View File
@@ -67,13 +67,13 @@ Linux Docker Engine 若要从宿主机 CLI 连到容器内服务,直接用 `ht
```bash ```bash
bash scripts/gitea-ci-job-image.sh build bash scripts/gitea-ci-job-image.sh build
bash scripts/gitea-ci-job-image.sh verify bash scripts/gitea-ci-job-image.sh verify
bash scripts/gitea-ci-job-image.sh export /仓库外受控路径/genarrative-gitea-project-ci-20260723.1.tar.zst bash scripts/gitea-ci-job-image.sh export /仓库外受控路径/genarrative-gitea-project-ci-20260807.1.tar.zst
bash scripts/gitea-ci-job-image.sh load-runner bash scripts/gitea-ci-job-image.sh load-runner
``` ```
默认构建 tag 为 `genarrative/gitea-project-ci:20260723.1`。脚本通过 NUL 分隔白名单 tar 流只发送 Dockerfile、checkout 脚本、npm manifests/lock Cargo manifests/lock;当前构建 context 约 `1.638 MB`,不会把业务源码、素材或本地私密文件发送给 Docker daemon。镜像除固定工具链外,还按当前根 `package-lock.json``server-rs/Cargo.lock` 和桌面壳 `Cargo.lock` 预热 npm / Cargo 下载缓存;`cargo fetch --locked` 最多执行 5 次整命令级有界重试,再以断网 `cargo fetch --locked` 验证缓存闭合,不包含 `node_modules` 或 Cargo `target``build` 完成后会自动运行环境校验,`load-runner` 还会比对宿主和 runner 内层的完整 Image ID,并在内层执行 bwrap 与 Chrome headless canary。当前验证镜像约 `1.788 GB`,完整 Image ID 为 `sha256:c04b114b1f145072c9df7842c4c974e1bb2eaaf391d95d84c9212a460546b7d5`。执行这些命令不要求必须使用 root,但执行账号必须有权访问宿主 Docker API 并管理 runner 容器;没有该权限时交给 runner 运维人员执行。 默认构建 tag 为 `genarrative/gitea-project-ci:20260807.1`。脚本通过 NUL 分隔白名单 tar 流只发送 Dockerfile、checkout 脚本、根与 AI 游戏创作壳的 npm manifests/lock,以及 server-rs、桌面壳和 AI 游戏创作壳的 Cargo manifests/lock;当前构建 context 约 `2.13 MB`,不会把业务源码、素材或本地私密文件发送给 Docker daemon。镜像除固定工具链外,还按上述五份 lock 预热 npm / Cargo 下载缓存;`cargo fetch --locked` 最多执行 5 次整命令级有界重试,再分别以断网 `cargo fetch --locked` 验证缓存闭合,不包含 `node_modules` 或 Cargo `target``build` 完成后会自动运行环境校验,`load-runner` 还会比对宿主和 runner 内层的完整 Image ID,并在内层执行 bwrap 与 Chrome headless canary。当前验证镜像约 `1.85 GB`,完整 Image ID 为 `sha256:8b4b30f5a096522942947927b06cf47bdb1a1016dde9a3573f9780e79d8e40cf`。执行这些命令不要求必须使用 root,但执行账号必须有权访问宿主 Docker API 并管理 runner 容器;没有该权限时交给 runner 运维人员执行。
runner 配置保留原 `ubuntu-latest` 映射,另外增加 `genarrative-ci:docker://sha256:c04b114b1f145072c9df7842c4c974e1bb2eaaf391d95d84c9212a460546b7d5`。内层 Docker 数据必须持久化,`force_pull` 保持 `false`;该精确 Image ID 在内层不存在时 job 应直接失败,不回退到浮动 tag 或现场拉取。四个 job 使用镜像内 `genarrative-gitea-checkout` 直接从当前 Gitea 拉取事件 commit,带 5 次有界重试,不再运行时下载 GitHub checkout action;随后以 `GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1` 执行 `scripts/check-gitea-ci-job-image.sh`,同时校验工具链、缓存锁命中状态、bwrap 和 Chrome headless。它们仍各自运行干净的 `npm ci` 以校验当前 lockfile 并隔离 PR 依赖,但使用镜像内 npm cache 和 `prefer-offline`;锁文件新增依赖时允许经受控网络补齐,本阶段不启用共享 Actions cache。 runner 配置保留原 `ubuntu-latest` 映射,另外增加 `genarrative-ci:docker://sha256:8b4b30f5a096522942947927b06cf47bdb1a1016dde9a3573f9780e79d8e40cf`。内层 Docker 数据必须持久化,`force_pull` 保持 `false`;该精确 Image ID 在内层不存在时 job 应直接失败,不回退到浮动 tag 或现场拉取。四个 job 使用镜像内 `genarrative-gitea-checkout` 直接从当前 Gitea 拉取事件 commit,带 5 次有界重试,不再运行时下载 GitHub checkout action;随后以 `GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1` 执行 `scripts/check-gitea-ci-job-image.sh`,同时校验工具链、五份缓存锁命中状态、bwrap 和 Chrome headless。锁不匹配时校验会输出 `partial` 和醒目的 Actions warning,提示在可信分支落地后刷新镜像。各 job 仍运行干净的 `npm ci` 以校验当前 lockfile 并隔离 PR 依赖,但统一通过 `scripts/ci-npm-ci-with-retry.sh` 最多执行 3 次整命令级有界重试,并使用镜像内 npm cache 和 `prefer-offline`;锁文件新增依赖时允许经受控网络补齐,本阶段不启用共享 Actions cache。
更新顺序固定为: 更新顺序固定为:
+29
View File
@@ -15,6 +15,7 @@ ENV CARGO_HTTP_MULTIPLEXING=false \
COPY server-rs /tmp/genarrative-cargo-cache/server-rs COPY server-rs /tmp/genarrative-cargo-cache/server-rs
COPY apps/desktop-shell/src-tauri /tmp/genarrative-cargo-cache/desktop-shell COPY apps/desktop-shell/src-tauri /tmp/genarrative-cargo-cache/desktop-shell
COPY apps/ai-game-creator-shell/src-tauri /tmp/genarrative-cargo-cache/apps/ai-game-creator-shell/src-tauri
RUN find /tmp/genarrative-cargo-cache -name Cargo.toml -exec dirname {} \; \ RUN find /tmp/genarrative-cargo-cache -name Cargo.toml -exec dirname {} \; \
| while IFS= read -r crate_dir; do \ | while IFS= read -r crate_dir; do \
@@ -39,12 +40,16 @@ RUN find /tmp/genarrative-cargo-cache -name Cargo.toml -exec dirname {} \; \
} \ } \
&& cargo_fetch_with_retry /tmp/genarrative-cargo-cache/server-rs/Cargo.toml \ && cargo_fetch_with_retry /tmp/genarrative-cargo-cache/server-rs/Cargo.toml \
&& cargo_fetch_with_retry /tmp/genarrative-cargo-cache/desktop-shell/Cargo.toml \ && cargo_fetch_with_retry /tmp/genarrative-cargo-cache/desktop-shell/Cargo.toml \
&& cargo_fetch_with_retry /tmp/genarrative-cargo-cache/apps/ai-game-creator-shell/src-tauri/Cargo.toml \
&& CARGO_NET_OFFLINE=true cargo fetch --locked \ && CARGO_NET_OFFLINE=true cargo fetch --locked \
--target x86_64-unknown-linux-gnu \ --target x86_64-unknown-linux-gnu \
--manifest-path /tmp/genarrative-cargo-cache/server-rs/Cargo.toml \ --manifest-path /tmp/genarrative-cargo-cache/server-rs/Cargo.toml \
&& CARGO_NET_OFFLINE=true cargo fetch --locked \ && CARGO_NET_OFFLINE=true cargo fetch --locked \
--target x86_64-unknown-linux-gnu \ --target x86_64-unknown-linux-gnu \
--manifest-path /tmp/genarrative-cargo-cache/desktop-shell/Cargo.toml \ --manifest-path /tmp/genarrative-cargo-cache/desktop-shell/Cargo.toml \
&& CARGO_NET_OFFLINE=true cargo fetch --locked \
--target x86_64-unknown-linux-gnu \
--manifest-path /tmp/genarrative-cargo-cache/apps/ai-game-creator-shell/src-tauri/Cargo.toml \
&& rm -rf /tmp/genarrative-cargo-cache && rm -rf /tmp/genarrative-cargo-cache
FROM ${RUNNER_IMAGE} FROM ${RUNNER_IMAGE}
@@ -123,21 +128,31 @@ COPY --from=rust-dependency-cache /usr/local/cargo /usr/local/cargo
COPY --from=rust-dependency-cache /usr/local/rustup /usr/local/rustup COPY --from=rust-dependency-cache /usr/local/rustup /usr/local/rustup
ARG NPM_LOCK_SHA256 ARG NPM_LOCK_SHA256
ARG AGC_NPM_LOCK_SHA256
ARG SERVER_RUST_LOCK_SHA256 ARG SERVER_RUST_LOCK_SHA256
ARG DESKTOP_RUST_LOCK_SHA256 ARG DESKTOP_RUST_LOCK_SHA256
ARG AGC_RUST_LOCK_SHA256
COPY package.json package-lock.json /usr/local/share/genarrative-ci/npm/ COPY package.json package-lock.json /usr/local/share/genarrative-ci/npm/
COPY apps/ai-game-creator-shell/package.json apps/ai-game-creator-shell/package-lock.json /usr/local/share/genarrative-ci/agc-npm/
COPY server-rs/Cargo.lock /usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock COPY server-rs/Cargo.lock /usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock
COPY apps/desktop-shell/src-tauri/Cargo.lock /usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock COPY apps/desktop-shell/src-tauri/Cargo.lock /usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock
COPY apps/ai-game-creator-shell/src-tauri/Cargo.lock /usr/local/share/genarrative-ci/locks/ai-game-creator-shell.Cargo.lock
COPY deploy/container/gitea-ci-checkout.sh /usr/local/bin/genarrative-gitea-checkout COPY deploy/container/gitea-ci-checkout.sh /usr/local/bin/genarrative-gitea-checkout
RUN test -n "${NPM_LOCK_SHA256}" \ RUN test -n "${NPM_LOCK_SHA256}" \
&& test -n "${AGC_NPM_LOCK_SHA256}" \
&& test -n "${SERVER_RUST_LOCK_SHA256}" \ && test -n "${SERVER_RUST_LOCK_SHA256}" \
&& test -n "${DESKTOP_RUST_LOCK_SHA256}" \ && test -n "${DESKTOP_RUST_LOCK_SHA256}" \
&& test -n "${AGC_RUST_LOCK_SHA256}" \
&& printf '%s %s\n' \ && printf '%s %s\n' \
"${NPM_LOCK_SHA256}" \ "${NPM_LOCK_SHA256}" \
/usr/local/share/genarrative-ci/npm/package-lock.json \ /usr/local/share/genarrative-ci/npm/package-lock.json \
| sha256sum --check --strict \ | sha256sum --check --strict \
&& printf '%s %s\n' \
"${AGC_NPM_LOCK_SHA256}" \
/usr/local/share/genarrative-ci/agc-npm/package-lock.json \
| sha256sum --check --strict \
&& printf '%s %s\n' \ && printf '%s %s\n' \
"${SERVER_RUST_LOCK_SHA256}" \ "${SERVER_RUST_LOCK_SHA256}" \
/usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock \ /usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock \
@@ -146,6 +161,10 @@ RUN test -n "${NPM_LOCK_SHA256}" \
"${DESKTOP_RUST_LOCK_SHA256}" \ "${DESKTOP_RUST_LOCK_SHA256}" \
/usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock \ /usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock \
| sha256sum --check --strict \ | sha256sum --check --strict \
&& printf '%s %s\n' \
"${AGC_RUST_LOCK_SHA256}" \
/usr/local/share/genarrative-ci/locks/ai-game-creator-shell.Cargo.lock \
| sha256sum --check --strict \
&& chmod 0755 /usr/local/bin/genarrative-gitea-checkout \ && chmod 0755 /usr/local/bin/genarrative-gitea-checkout \
&& npm ci \ && npm ci \
--ignore-scripts \ --ignore-scripts \
@@ -154,6 +173,13 @@ RUN test -n "${NPM_LOCK_SHA256}" \
--prefer-offline \ --prefer-offline \
--prefix /usr/local/share/genarrative-ci/npm \ --prefix /usr/local/share/genarrative-ci/npm \
&& rm -rf /usr/local/share/genarrative-ci/npm/node_modules \ && rm -rf /usr/local/share/genarrative-ci/npm/node_modules \
&& npm ci \
--ignore-scripts \
--no-audit \
--no-fund \
--prefer-offline \
--prefix /usr/local/share/genarrative-ci/agc-npm \
&& rm -rf /usr/local/share/genarrative-ci/agc-npm/node_modules \
&& npm cache verify && npm cache verify
RUN install -m 0755 /usr/local/cargo/bin/rustup /usr/local/bin/rustup \ RUN install -m 0755 /usr/local/cargo/bin/rustup /usr/local/bin/rustup \
@@ -162,6 +188,8 @@ RUN install -m 0755 /usr/local/cargo/bin/rustup /usr/local/bin/rustup \
done done
ENV CARGO_HOME=/usr/local/cargo \ ENV CARGO_HOME=/usr/local/cargo \
GENARRATIVE_GITEA_CI_AGC_NPM_LOCK_SHA256=${AGC_NPM_LOCK_SHA256} \
GENARRATIVE_GITEA_CI_AGC_RUST_LOCK_SHA256=${AGC_RUST_LOCK_SHA256} \
GENARRATIVE_GITEA_CI_DESKTOP_RUST_LOCK_SHA256=${DESKTOP_RUST_LOCK_SHA256} \ GENARRATIVE_GITEA_CI_DESKTOP_RUST_LOCK_SHA256=${DESKTOP_RUST_LOCK_SHA256} \
GENARRATIVE_GITEA_CI_NPM_LOCK_SHA256=${NPM_LOCK_SHA256} \ GENARRATIVE_GITEA_CI_NPM_LOCK_SHA256=${NPM_LOCK_SHA256} \
GENARRATIVE_GITEA_CI_SERVER_RUST_LOCK_SHA256=${SERVER_RUST_LOCK_SHA256} \ GENARRATIVE_GITEA_CI_SERVER_RUST_LOCK_SHA256=${SERVER_RUST_LOCK_SHA256} \
@@ -178,6 +206,7 @@ ENV CARGO_HOME=/usr/local/cargo \
ARG IMAGE_REVISION=uncommitted ARG IMAGE_REVISION=uncommitted
LABEL org.opencontainers.image.vendor="GenarrativeAI" \ LABEL org.opencontainers.image.vendor="GenarrativeAI" \
org.opencontainers.image.version="2026.08.07.1" \
org.opencontainers.image.source="https://git.genarrative.world/GenarrativeAI/Genarrative" \ org.opencontainers.image.source="https://git.genarrative.world/GenarrativeAI/Genarrative" \
org.opencontainers.image.revision="${IMAGE_REVISION}" \ org.opencontainers.image.revision="${IMAGE_REVISION}" \
org.opencontainers.image.base.name="docker.gitea.com/runner-images:ubuntu-latest@sha256:58ea92624c7c09582e05594d95488331045053d3a3f34cf09649f2a32313a614" \ org.opencontainers.image.base.name="docker.gitea.com/runner-images:ubuntu-latest@sha256:58ea92624c7c09582e05594d95488331045053d3a3f34cf09649f2a32313a614" \
@@ -12,6 +12,12 @@
!server-rs/crates/*/ !server-rs/crates/*/
!server-rs/crates/*/Cargo.toml !server-rs/crates/*/Cargo.toml
!apps/ !apps/
!apps/ai-game-creator-shell/
!apps/ai-game-creator-shell/package.json
!apps/ai-game-creator-shell/package-lock.json
!apps/ai-game-creator-shell/src-tauri/
!apps/ai-game-creator-shell/src-tauri/Cargo.toml
!apps/ai-game-creator-shell/src-tauri/Cargo.lock
!apps/desktop-shell/ !apps/desktop-shell/
!apps/desktop-shell/src-tauri/ !apps/desktop-shell/src-tauri/
!apps/desktop-shell/src-tauri/Cargo.toml !apps/desktop-shell/src-tauri/Cargo.toml
+10 -3
View File
@@ -4008,11 +4008,18 @@
- 现象:`Repository checks``Frontend tests``Backend tests``Native shell tests` 都从全新 job 容器开始,apt、setup-node、rustup 和原生系统库在不同 job 里重复安装;后端与原生壳的安装时间可达数分钟,并把软件源和代理瞬时失败放大为四份。 - 现象:`Repository checks``Frontend tests``Backend tests``Native shell tests` 都从全新 job 容器开始,apt、setup-node、rustup 和原生系统库在不同 job 里重复安装;后端与原生壳的安装时间可达数分钟,并把软件源和代理瞬时失败放大为四份。
- 原因:Gitea Actions job 彼此隔离,上一个 job 在容器内安装的包不会自动进入下一个 job;把同一套不随 PR 变化的工具链写在 workflow step 中,必然每次重做。 - 原因:Gitea Actions job 彼此隔离,上一个 job 在容器内安装的包不会自动进入下一个 job;把同一套不随 PR 变化的工具链写在 workflow step 中,必然每次重做。
- 处理:用 `deploy/container/gitea-ci-job.Dockerfile` 预装 Node 22、Rust 1.96、`rustfmt`、Chrome、`bwrap``rg``ffmpeg``clang/lld` 和 Tauri / 后端系统依赖,并按锁预热根 npm、server-rs 与桌面壳 Cargo 下载缓存。四个 job 统一 `runs-on: genarrative-ci`,先用镜像内脚本直接从 Gitea checkout,再以 runtime 模式运行 `scripts/check-gitea-ci-job-image.sh`,同时检查缓存锁、工具链、完整 bwrap 与 Chrome headless。`RUSTUP_AUTO_INSTALL=0``rust-toolchain.toml` 变更时先重建镜像,不把下载 fallback 放回 job。 - 处理:用 `deploy/container/gitea-ci-job.Dockerfile` 预装 Node 22、Rust 1.96、`rustfmt`、Chrome、`bwrap``rg``ffmpeg``clang/lld` 和 Tauri / 后端系统依赖,并按锁预热根与 AI 游戏创作壳 npm、server-rs、桌面壳与 AI 游戏创作壳 Cargo 下载缓存。四个 job 统一 `runs-on: genarrative-ci`,先用镜像内脚本直接从 Gitea checkout,再以 runtime 模式运行 `scripts/check-gitea-ci-job-image.sh`,同时检查五份缓存锁、工具链、完整 bwrap 与 Chrome headless。`RUSTUP_AUTO_INSTALL=0``rust-toolchain.toml` 变更时先重建镜像,不把下载 fallback 放回 job。
- 依赖边界:每个 job 仍必须各自执行 `npm ci`,让当前 lockfile 和 PR 依赖在干净环境中验证;区别是命中镜像 cache 时只做本地解包,锁新增依赖时才走受控网络。不要把 `node_modules` 或 Cargo `target` 烘进镜像,也不要向不受信任 PR 挂载跨 job 可写 cache。 - 依赖边界:每个 job 仍必须各自执行 `npm ci`,让当前 lockfile 和 PR 依赖在干净环境中验证;区别是命中镜像 cache 时只做本地解包,锁新增依赖时才走受控网络。不要把 `node_modules` 或 Cargo `target` 烘进镜像,也不要向不受信任 PR 挂载跨 job 可写 cache。
- 锁漂移边界:runtime 校验输出 `server_rust_cache_lock=partial` 说明镜像内 Cargo lock 与当前 checkout 不同,不代表新增 crate 已经缓存。必须在新镜像中以 `--network none`当前 lock 执行真实 `cargo fetch/build --offline``cargo metadata --no-deps` 不会证明依赖 archive 可用,不能作为替代。 - 锁漂移边界:runtime 校验输出任一 `*_cache_lock=partial` 说明镜像内 lock 与当前 checkout 不同,不代表新增依赖已经缓存;必须同时输出 Actions warning,提示可信分支落地后刷新镜像。必须在新镜像中对 server-rs、桌面壳和 AI 游戏创作壳当前 lock 执行真实 `cargo fetch --locked --offline``cargo metadata --no-deps` 不会证明依赖 archive 可用,不能作为替代。
- 构建网络边界:`CARGO_NET_RETRY` 只覆盖部分 crate 下载,registry `config.json` / index TLS 握手仍可能直接终止整次 fetch。Dockerfile 对每个 `cargo fetch --locked` 再做最多 5 次整命令级有界重试,最终仍执行断网 fetch,不能降低为无锁重试或省略离线闭合验证。 - 构建网络边界:`CARGO_NET_RETRY` 只覆盖部分 crate 下载,registry `config.json` / index TLS 握手仍可能直接终止整次 fetch。Dockerfile 对每个 `cargo fetch --locked` 再做最多 5 次整命令级有界重试,最终仍执行断网 fetch,不能降低为无锁重试或省略离线闭合验证。
- 验证:workflow 不再出现 GitHub checkout action、apt、setup-node 或 rustup 安装 step;镜像`--network none` 下能按当前 npm / Cargo lock 完成依赖准备,四个 job 的环境校验、干净 `npm ci` 和原有测试门禁仍全部执行。 - 验证:workflow 不再出现 GitHub checkout action、apt、setup-node 或 rustup 安装 step;镜像能按五份当前 lock 完成缓存闭合,四个 job 的环境校验、经 3 次整命令级有界重试保护的干净 `npm ci` 和原有测试门禁仍全部执行。
## Gitea Actions HTTPS CONNECT 隧道必须双向收束 socket2026-08-07
- 现象:CI 的 `npm ci` 高频出现 `ECONNRESET / network aborted`Cargo 则出现 crates.io TLS EOF、连接超时或下载失败;同一出口 gateway 容器看似健康,却累计自动重启数百次,日志反复出现 `Socket.ondata -> Writable.write -> write EPIPE -> Unhandled 'error' event`
- 原因:HTTPS CONNECT 建立后使用 `upstreamSocket.pipe(clientSocket)` 与反向 pipe,但只监听 upstream `error`;客户端在 DNS 等待、下载或 job 清理期间关闭连接时,pipe 继续向已断开的 client socket 写入,未处理的 EPIPE 会让 Node 进程退出。`unless-stopped` 自动拉起和浅层 healthcheck 会掩盖崩溃,所有并发 npm / Cargo 隧道同时被 reset。
- 处理:CONNECT 一开始就为 client socket 注册 `error / close`,解析完成后为 upstream socket注册同样的双向销毁处理;DNS 返回、写 200 和开始 pipe 前都检查 client 是否已销毁。任一端 error、close 或 timeout 都幂等 destroy 两端,不把普通客户端 reset 写成错误日志。不要用进程级 `uncaughtException` 吞掉问题,也不要只增加 npm/Cargo 重试掩盖 gateway 崩溃。
- 验证:在独立 canary 和正式 gateway 上分别并发制造至少 500 次“CONNECT 后立即断开”,随后确认容器仍运行、restart count 不增加、日志无 EPIPE;再通过同一 proxy 对 npm registry 与 crates index 建立完整 TLS 隧道。切换前仍须确认 Gitea 无活跃 run 且 Runner 内层无 job 容器。
## Gitea CI 预构建镜像不能只靠 tag 判断内容 ## Gitea CI 预构建镜像不能只靠 tag 判断内容
@@ -243,20 +243,20 @@ PR checkout 必须保留完整 Git 历史,并把 PR base SHA 传给 `SPACETIME
当前 `genarrative-station` 使用 Gitea `1.26.4` 和基于 Gitea Runner `2.0.0-dind-rootless` 的固定 digest 修补镜像。Runner 2.0.0 会先把 `systempaths=unconfined` 解析为空 `MaskedPaths` / `ReadonlyPaths`,再被 `mergo.WithOverride` 当成 empty value 丢失;站点修补只在 merge 后保留这两个显式空 slice,不改其它 runner 行为。真实 job inspect 必须看到 `MaskedPaths=[]``ReadonlyPaths=[]``SecurityOpt=[seccomp=unconfined]``Privileged=false`、无 CapAdd 且 `Binds=[]`。外层 runner 以 `rootless` 用户运行,`privileged=false`、不增加 `CAP_SYS_ADMIN`,只映射 `/dev/net/tun`,内部 Docker 只监听私有 Unix socketrunner 配置保持 `docker_host: "-"``valid_volumes: []``bind_workdir: false``force_pull: false`,防止内部 Docker socket 或宿主 bind mount 进入 job。job 只连接 `gitea-actions` internal network`genarrative-station` 由只转发 `/git` 到 Gitea 的内部 gateway 解析,公网依赖只经拒绝私网、保留地址和 metadata 的 80/443 egress proxy;绕过 proxy 的公网和 Postgres/Redis 数据网都必须不可达。完整 bwrap canary 需要 rootless DinD 外层的 rootlesskit AppArmor/userns 边界,以及内层 job 的 namespace/proc 挂载支持;相关 `seccomp/systempaths` 放宽只允许存在于这个无宿主 socket 的 rootless DinD 内层,禁止复制回控制宿主 rootful Docker 的 runner。 当前 `genarrative-station` 使用 Gitea `1.26.4` 和基于 Gitea Runner `2.0.0-dind-rootless` 的固定 digest 修补镜像。Runner 2.0.0 会先把 `systempaths=unconfined` 解析为空 `MaskedPaths` / `ReadonlyPaths`,再被 `mergo.WithOverride` 当成 empty value 丢失;站点修补只在 merge 后保留这两个显式空 slice,不改其它 runner 行为。真实 job inspect 必须看到 `MaskedPaths=[]``ReadonlyPaths=[]``SecurityOpt=[seccomp=unconfined]``Privileged=false`、无 CapAdd 且 `Binds=[]`。外层 runner 以 `rootless` 用户运行,`privileged=false`、不增加 `CAP_SYS_ADMIN`,只映射 `/dev/net/tun`,内部 Docker 只监听私有 Unix socketrunner 配置保持 `docker_host: "-"``valid_volumes: []``bind_workdir: false``force_pull: false`,防止内部 Docker socket 或宿主 bind mount 进入 job。job 只连接 `gitea-actions` internal network`genarrative-station` 由只转发 `/git` 到 Gitea 的内部 gateway 解析,公网依赖只经拒绝私网、保留地址和 metadata 的 80/443 egress proxy;绕过 proxy 的公网和 Postgres/Redis 数据网都必须不可达。完整 bwrap canary 需要 rootless DinD 外层的 rootlesskit AppArmor/userns 边界,以及内层 job 的 namespace/proc 挂载支持;相关 `seccomp/systempaths` 放宽只允许存在于这个无宿主 socket 的 rootless DinD 内层,禁止复制回控制宿主 rootful Docker 的 runner。
CI job 镜像由 `deploy/container/gitea-ci-job.Dockerfile` 定义:Ubuntu job base 固定为 `sha256:58ea92624c7c09582e05594d95488331045053d3a3f34cf09649f2a32313a614`Rust stage 固定为 `sha256:19817ead3289c8c631c73df281e18b59b172f6a31f4f563290f69cddd06c30e9`Node `22.23.1` 发行包执行 SHA-256 校验,Google Linux 主签名指纹固定,Chrome 固定为 `150.0.7871.181-1`。构建脚本以 NUL 分隔白名单 tar 流只发送 Dockerfile、checkout 脚本 npm / Cargo manifests/lock;当前 context 约 `1.638 MB`。镜像按根 npm 锁、server-rs 锁桌面壳锁预热下载缓存,不包含 `node_modules` 或 Cargo `target``cargo fetch --locked` 在 Cargo 自身重试之外再执行最多 5 次整命令级有界重试,处理 registry index 握手失败,最终仍以断网 `cargo fetch --locked` 关闭验证。当前验证镜像约 `1.788 GB`,默认 tag 为 `genarrative/gitea-project-ci:20260723.1`,完整 Image ID 为 `sha256:c04b114b1f145072c9df7842c4c974e1bb2eaaf391d95d84c9212a460546b7d5`runner 标签保留 `ubuntu-latest`,并将 `genarrative-ci` 映射到 `docker://sha256:c04b114b1f145072c9df7842c4c974e1bb2eaaf391d95d84c9212a460546b7d5`。内层 Docker 数据必须持久化;`force_pull: false` 表示只使用这个已装载的精确内容,Image ID 缺失时 job 必须失败关闭,不得回退浮动 tag 或临时连 registry。 CI job 镜像由 `deploy/container/gitea-ci-job.Dockerfile` 定义:Ubuntu job base 固定为 `sha256:58ea92624c7c09582e05594d95488331045053d3a3f34cf09649f2a32313a614`Rust stage 固定为 `sha256:19817ead3289c8c631c73df281e18b59b172f6a31f4f563290f69cddd06c30e9`Node `22.23.1` 发行包执行 SHA-256 校验,Google Linux 主签名指纹固定,Chrome 固定为 `150.0.7871.181-1`。构建脚本以 NUL 分隔白名单 tar 流只发送 Dockerfile、checkout 脚本,以及根、AI 游戏创作壳、server-rs 与桌面壳所需的 npm / Cargo manifests/lock;当前 context 约 `2.13 MB`。镜像按根 npm 锁、AI 游戏创作壳 npm 锁、server-rs 锁桌面壳锁和 AI 游戏创作壳 Cargo 锁预热下载缓存,不包含 `node_modules` 或 Cargo `target``cargo fetch --locked` 在 Cargo 自身重试之外再执行最多 5 次整命令级有界重试,处理 registry index 握手失败,最终仍分别以断网 `cargo fetch --locked` 关闭验证。当前验证镜像约 `1.85 GB`,默认 tag 为 `genarrative/gitea-project-ci:20260807.1`,完整 Image ID 为 `sha256:8b4b30f5a096522942947927b06cf47bdb1a1016dde9a3573f9780e79d8e40cf`runner 标签保留 `ubuntu-latest`,并将 `genarrative-ci` 映射到 `docker://sha256:8b4b30f5a096522942947927b06cf47bdb1a1016dde9a3573f9780e79d8e40cf`。内层 Docker 数据必须持久化;`force_pull: false` 表示只使用这个已装载的精确内容,Image ID 缺失时 job 必须失败关闭,不得回退浮动 tag 或临时连 registry。
镜像更新命令: 镜像更新命令:
```bash ```bash
bash scripts/gitea-ci-job-image.sh build bash scripts/gitea-ci-job-image.sh build
bash scripts/gitea-ci-job-image.sh verify bash scripts/gitea-ci-job-image.sh verify
bash scripts/gitea-ci-job-image.sh export /仓库外受控路径/genarrative-gitea-project-ci-20260723.1.tar.zst bash scripts/gitea-ci-job-image.sh export /仓库外受控路径/genarrative-gitea-project-ci-20260807.1.tar.zst
bash scripts/gitea-ci-job-image.sh load-runner bash scripts/gitea-ci-job-image.sh load-runner
``` ```
执行账号只要有权访问宿主 Docker API 并管理 runner 容器即可,不强制使用 root;无该权限时由 runner 运维人员执行。更新顺序必须是 `build/verify -> export 仓库外镜像归档与 SHA-256 sidecar -> load-runner -> 确认无活跃 job -> 备份当前 config -> 增加或替换 label -> docker restart --timeout 660 gitea-runner``--timeout 660` 只是停止宽限,不是 drain APIrootless DinD supervisor 可能同时停止内层 dockerd,因此重启前必须确认 Gitea 没有 `in_progress` run 且内层 `docker ps` 为空。config 和镜像归档只保存到仓库外受控位置,不在文档、仓库或日志中记录注册信息。重启后先重跑真实 PR 的四个 job,复核隔离边界并确认全部通过,再清理旧镜像。回滚时先把 workflow 的 `runs-on` 改回 `ubuntu-latest`,再恢复 config 备份并重启 runner。 执行账号只要有权访问宿主 Docker API 并管理 runner 容器即可,不强制使用 root;无该权限时由 runner 运维人员执行。更新顺序必须是 `build/verify -> export 仓库外镜像归档与 SHA-256 sidecar -> load-runner -> 确认无活跃 job -> 备份当前 config -> 增加或替换 label -> docker restart --timeout 660 gitea-runner``--timeout 660` 只是停止宽限,不是 drain APIrootless DinD supervisor 可能同时停止内层 dockerd,因此重启前必须确认 Gitea 没有 `in_progress` run 且内层 `docker ps` 为空。config 和镜像归档只保存到仓库外受控位置,不在文档、仓库或日志中记录注册信息。重启后先重跑真实 PR 的四个 job,复核隔离边界并确认全部通过,再清理旧镜像。回滚时先把 workflow 的 `runs-on` 改回 `ubuntu-latest`,再恢复 config 备份并重启 runner。
四个 job 先运行镜像内 `genarrative-gitea-checkout`,再以 `GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1` 执行 `scripts/check-gitea-ci-job-image.sh`,校验 Node 主版本、仓库 Rust toolchain、受信任 PATH、缓存锁命中状态、原生命令、pkg-config 依赖、完整 bwrap sandbox 和 Chrome headless。`RUSTUP_AUTO_INSTALL=0`,因此仓库 `rust-toolchain.toml` 变更必须先更新镜像,不能让 job 现场下载。每个 job 仍独立运行 `npm ci`,以当前 lockfile 为准验证 PR 依赖;`NPM_CONFIG_PREFER_OFFLINE=true` 且网络重试为 10 次,命中镜像 cache 时只做干净解包,lock 变化时允许补齐差量。不在镜像内烘入 `node_modules`,也不挂载跨 PR 可写缓存。任何 job 的 sandbox canary 失败都必须停止,不允许跳过。Cargo 通过受控 proxy 下载 lock 差量时继续关闭 HTTP multiplexing 并设置 `CARGO_NET_RETRY=10` 四个 job 先运行镜像内 `genarrative-gitea-checkout`,再以 `GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1` 执行 `scripts/check-gitea-ci-job-image.sh`,校验 Node 主版本、仓库 Rust toolchain、受信任 PATH、五份缓存锁命中状态、原生命令、pkg-config 依赖、完整 bwrap sandbox 和 Chrome headless。运行时发现锁不匹配时必须输出对应 `*_cache_lock=partial` 和 Actions warning,提示可信分支落地后刷新镜像,不能把陈旧缓存误报为闭合。`RUSTUP_AUTO_INSTALL=0`,因此仓库 `rust-toolchain.toml` 变更必须先更新镜像,不能让 job 现场下载。每个 job 仍独立运行 `npm ci`,以当前 lockfile 为准验证 PR 依赖;统一通过 `scripts/ci-npm-ci-with-retry.sh` 做最多 3 次整命令级有界重试,同时保留 `NPM_CONFIG_PREFER_OFFLINE=true` 和 npm 自身 10 次 fetch retry。命中镜像 cache 时只做干净解包,lock 变化时允许补齐差量。不在镜像内烘入 `node_modules`,也不挂载跨 PR 可写缓存。任何 job 的 sandbox canary 失败都必须停止,不允许跳过。Cargo 通过受控 proxy 下载 lock 差量时继续关闭 HTTP multiplexing并设置 `CARGO_NET_RETRY=10`
站点 stack 仍由宿主受控目录管理,`.env`、runner 注册文件和数据库凭据不进入仓库。Compose 必须在 helper/container 内把该目录挂到与宿主相同的绝对路径再执行;挂载到不同路径会让相对 bind source 被 Docker daemon 解析到错误的宿主目录并启动空数据。升级或 runner 迁移前先停止 Gitea 写入,并把 Gitea 冷快照、数据库导出、compose/env 与 runner config/.runner 保存到仓库外受控备份位置。备份文件、绝对宿主配置和注册 token 不得提交 Git,也不在共享文档中记录具体路径或注册内容。 站点 stack 仍由宿主受控目录管理,`.env`、runner 注册文件和数据库凭据不进入仓库。Compose 必须在 helper/container 内把该目录挂到与宿主相同的绝对路径再执行;挂载到不同路径会让相对 bind source 被 Docker daemon 解析到错误的宿主目录并启动空数据。升级或 runner 迁移前先停止 Gitea 写入,并把 Gitea 冷快照、数据库导出、compose/env 与 runner config/.runner 保存到仓库外受控备份位置。备份文件、绝对宿主配置和注册 token 不得提交 Git,也不在共享文档中记录具体路径或注册内容。
+17
View File
@@ -36,6 +36,7 @@ verify_cache_lock() {
fi fi
if [[ "${GENARRATIVE_GITEA_CI_CHECK_RUNTIME:-0}" == '1' ]]; then if [[ "${GENARRATIVE_GITEA_CI_CHECK_RUNTIME:-0}" == '1' ]]; then
printf '%s_cache_lock=partial\n' "${cache_name}" printf '%s_cache_lock=partial\n' "${cache_name}"
printf '::warning title=CI dependency cache is partial::%s lock differs from the prebuilt image; refresh the trusted CI image after this lock change lands.\n' "${cache_name}"
return return
fi fi
echo "${cache_name} cache lock does not match the verification checkout." >&2 echo "${cache_name} cache lock does not match the verification checkout." >&2
@@ -43,22 +44,34 @@ verify_cache_lock() {
} }
npm_lock_path="${repo_root}/package-lock.json" npm_lock_path="${repo_root}/package-lock.json"
agc_npm_lock_path="${repo_root}/apps/ai-game-creator-shell/package-lock.json"
server_rust_lock_path="${repo_root}/server-rs/Cargo.lock" server_rust_lock_path="${repo_root}/server-rs/Cargo.lock"
desktop_rust_lock_path="${repo_root}/apps/desktop-shell/src-tauri/Cargo.lock" desktop_rust_lock_path="${repo_root}/apps/desktop-shell/src-tauri/Cargo.lock"
agc_rust_lock_path="${repo_root}/apps/ai-game-creator-shell/src-tauri/Cargo.lock"
if [[ ! -f "${npm_lock_path}" ]]; then if [[ ! -f "${npm_lock_path}" ]]; then
npm_lock_path='/usr/local/share/genarrative-ci/npm/package-lock.json' npm_lock_path='/usr/local/share/genarrative-ci/npm/package-lock.json'
fi fi
if [[ ! -f "${agc_npm_lock_path}" ]]; then
agc_npm_lock_path='/usr/local/share/genarrative-ci/agc-npm/package-lock.json'
fi
if [[ ! -f "${server_rust_lock_path}" ]]; then if [[ ! -f "${server_rust_lock_path}" ]]; then
server_rust_lock_path='/usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock' server_rust_lock_path='/usr/local/share/genarrative-ci/locks/server-rs.Cargo.lock'
fi fi
if [[ ! -f "${desktop_rust_lock_path}" ]]; then if [[ ! -f "${desktop_rust_lock_path}" ]]; then
desktop_rust_lock_path='/usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock' desktop_rust_lock_path='/usr/local/share/genarrative-ci/locks/desktop-shell.Cargo.lock'
fi fi
if [[ ! -f "${agc_rust_lock_path}" ]]; then
agc_rust_lock_path='/usr/local/share/genarrative-ci/locks/ai-game-creator-shell.Cargo.lock'
fi
verify_cache_lock \ verify_cache_lock \
npm \ npm \
"${GENARRATIVE_GITEA_CI_NPM_LOCK_SHA256:-}" \ "${GENARRATIVE_GITEA_CI_NPM_LOCK_SHA256:-}" \
"${npm_lock_path}" "${npm_lock_path}"
verify_cache_lock \
agc_npm \
"${GENARRATIVE_GITEA_CI_AGC_NPM_LOCK_SHA256:-}" \
"${agc_npm_lock_path}"
verify_cache_lock \ verify_cache_lock \
server_rust \ server_rust \
"${GENARRATIVE_GITEA_CI_SERVER_RUST_LOCK_SHA256:-}" \ "${GENARRATIVE_GITEA_CI_SERVER_RUST_LOCK_SHA256:-}" \
@@ -67,6 +80,10 @@ verify_cache_lock \
desktop_rust \ desktop_rust \
"${GENARRATIVE_GITEA_CI_DESKTOP_RUST_LOCK_SHA256:-}" \ "${GENARRATIVE_GITEA_CI_DESKTOP_RUST_LOCK_SHA256:-}" \
"${desktop_rust_lock_path}" "${desktop_rust_lock_path}"
verify_cache_lock \
agc_rust \
"${GENARRATIVE_GITEA_CI_AGC_RUST_LOCK_SHA256:-}" \
"${agc_rust_lock_path}"
for command_name in \ for command_name in \
bwrap \ bwrap \
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
max_attempts="${GENARRATIVE_CI_NPM_CI_ATTEMPTS:-3}"
base_delay_seconds="${GENARRATIVE_CI_NPM_CI_RETRY_DELAY_SECONDS:-5}"
if [[ ! "${max_attempts}" =~ ^[1-9][0-9]*$ ]]; then
echo 'GENARRATIVE_CI_NPM_CI_ATTEMPTS must be a positive integer.' >&2
exit 2
fi
if [[ ! "${base_delay_seconds}" =~ ^[0-9]+$ ]]; then
echo 'GENARRATIVE_CI_NPM_CI_RETRY_DELAY_SECONDS must be a non-negative integer.' >&2
exit 2
fi
for attempt in $(seq 1 "${max_attempts}"); do
if npm ci "$@"; then
exit 0
fi
if [[ "${attempt}" -eq "${max_attempts}" ]]; then
echo "npm ci failed after ${max_attempts} attempts." >&2
exit 1
fi
sleep "$((attempt * base_delay_seconds))"
done
+15 -1
View File
@@ -4,7 +4,7 @@ set -euo pipefail
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)" repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
dockerfile_context_path="deploy/container/gitea-ci-job.Dockerfile" dockerfile_context_path="deploy/container/gitea-ci-job.Dockerfile"
image_tag="${GENARRATIVE_GITEA_CI_IMAGE_TAG:-genarrative/gitea-project-ci:20260723.1}" image_tag="${GENARRATIVE_GITEA_CI_IMAGE_TAG:-genarrative/gitea-project-ci:20260807.1}"
runner_container="${GENARRATIVE_GITEA_RUNNER_CONTAINER:-gitea-runner}" runner_container="${GENARRATIVE_GITEA_RUNNER_CONTAINER:-gitea-runner}"
write_build_context_file_list() { write_build_context_file_list() {
@@ -14,6 +14,10 @@ write_build_context_file_list() {
deploy/container/gitea-ci-checkout.sh \ deploy/container/gitea-ci-checkout.sh \
package.json \ package.json \
package-lock.json \ package-lock.json \
apps/ai-game-creator-shell/package.json \
apps/ai-game-creator-shell/package-lock.json \
apps/ai-game-creator-shell/src-tauri/Cargo.toml \
apps/ai-game-creator-shell/src-tauri/Cargo.lock \
server-rs/Cargo.toml \ server-rs/Cargo.toml \
server-rs/Cargo.lock \ server-rs/Cargo.lock \
apps/desktop-shell/src-tauri/Cargo.toml \ apps/desktop-shell/src-tauri/Cargo.toml \
@@ -61,6 +65,10 @@ case "${command_name}" in
deploy/container/gitea-ci-checkout.sh \ deploy/container/gitea-ci-checkout.sh \
package.json \ package.json \
package-lock.json \ package-lock.json \
apps/ai-game-creator-shell/package.json \
apps/ai-game-creator-shell/package-lock.json \
apps/ai-game-creator-shell/src-tauri/Cargo.toml \
apps/ai-game-creator-shell/src-tauri/Cargo.lock \
server-rs/Cargo.toml \ server-rs/Cargo.toml \
server-rs/Cargo.lock \ server-rs/Cargo.lock \
apps/desktop-shell/src-tauri/Cargo.toml \ apps/desktop-shell/src-tauri/Cargo.toml \
@@ -74,10 +82,14 @@ case "${command_name}" in
)" )"
npm_lock_sha256="$(sha256sum "${repo_root}/package-lock.json")" npm_lock_sha256="$(sha256sum "${repo_root}/package-lock.json")"
npm_lock_sha256="${npm_lock_sha256%% *}" npm_lock_sha256="${npm_lock_sha256%% *}"
agc_npm_lock_sha256="$(sha256sum "${repo_root}/apps/ai-game-creator-shell/package-lock.json")"
agc_npm_lock_sha256="${agc_npm_lock_sha256%% *}"
server_rust_lock_sha256="$(sha256sum "${repo_root}/server-rs/Cargo.lock")" server_rust_lock_sha256="$(sha256sum "${repo_root}/server-rs/Cargo.lock")"
server_rust_lock_sha256="${server_rust_lock_sha256%% *}" server_rust_lock_sha256="${server_rust_lock_sha256%% *}"
desktop_rust_lock_sha256="$(sha256sum "${repo_root}/apps/desktop-shell/src-tauri/Cargo.lock")" desktop_rust_lock_sha256="$(sha256sum "${repo_root}/apps/desktop-shell/src-tauri/Cargo.lock")"
desktop_rust_lock_sha256="${desktop_rust_lock_sha256%% *}" desktop_rust_lock_sha256="${desktop_rust_lock_sha256%% *}"
agc_rust_lock_sha256="$(sha256sum "${repo_root}/apps/ai-game-creator-shell/src-tauri/Cargo.lock")"
agc_rust_lock_sha256="${agc_rust_lock_sha256%% *}"
( (
cd "${repo_root}" cd "${repo_root}"
write_build_context_file_list \ write_build_context_file_list \
@@ -86,8 +98,10 @@ case "${command_name}" in
--pull=false \ --pull=false \
--build-arg "IMAGE_REVISION=${image_revision}" \ --build-arg "IMAGE_REVISION=${image_revision}" \
--build-arg "NPM_LOCK_SHA256=${npm_lock_sha256}" \ --build-arg "NPM_LOCK_SHA256=${npm_lock_sha256}" \
--build-arg "AGC_NPM_LOCK_SHA256=${agc_npm_lock_sha256}" \
--build-arg "SERVER_RUST_LOCK_SHA256=${server_rust_lock_sha256}" \ --build-arg "SERVER_RUST_LOCK_SHA256=${server_rust_lock_sha256}" \
--build-arg "DESKTOP_RUST_LOCK_SHA256=${desktop_rust_lock_sha256}" \ --build-arg "DESKTOP_RUST_LOCK_SHA256=${desktop_rust_lock_sha256}" \
--build-arg "AGC_RUST_LOCK_SHA256=${agc_rust_lock_sha256}" \
--file "${dockerfile_context_path}" \ --file "${dockerfile_context_path}" \
--tag "${image_tag}" \ --tag "${image_tag}" \
- -
+171
View File
@@ -7,6 +7,61 @@ const workflow = readFileSync(
resolve(process.cwd(), '.gitea/workflows/project-ci.yml'), resolve(process.cwd(), '.gitea/workflows/project-ci.yml'),
'utf8', 'utf8',
); );
const imageBuildScript = readFileSync(
resolve(process.cwd(), 'scripts/gitea-ci-job-image.sh'),
'utf8',
);
const imageCheckScript = readFileSync(
resolve(process.cwd(), 'scripts/check-gitea-ci-job-image.sh'),
'utf8',
);
const npmCiRetryScript = readFileSync(
resolve(process.cwd(), 'scripts/ci-npm-ci-with-retry.sh'),
'utf8',
);
const imageDockerfile = readFileSync(
resolve(process.cwd(), 'deploy/container/gitea-ci-job.Dockerfile'),
'utf8',
);
const imageDockerignore = readFileSync(
resolve(
process.cwd(),
'deploy/container/gitea-ci-job.Dockerfile.dockerignore',
),
'utf8',
);
const jobNames = [
'repository-checks',
'frontend-tests',
'backend-tests',
'native-shell-tests',
] as const;
function jobSection(jobName: (typeof jobNames)[number]) {
const jobStart = workflow.indexOf(` ${jobName}:`);
expect(jobStart).toBeGreaterThanOrEqual(0);
const nextJobOffset = workflow
.slice(jobStart + 1)
.search(/^ [a-z][a-z0-9-]+:$/m);
return workflow.slice(
jobStart,
nextJobOffset < 0 ? undefined : jobStart + 1 + nextJobOffset,
);
}
function stepSection(jobName: (typeof jobNames)[number], stepName: string) {
const job = jobSection(jobName);
const stepStart = job.indexOf(` - name: ${stepName}`);
expect(stepStart).toBeGreaterThanOrEqual(0);
const nextStepOffset = job.slice(stepStart + 1).search(/^ - name: /m);
return job.slice(
stepStart,
nextStepOffset < 0 ? undefined : stepStart + 1 + nextStepOffset,
);
}
function backendStepIndex(stepName: string) { function backendStepIndex(stepName: string) {
const backendJobStart = workflow.indexOf(' backend-tests:'); const backendJobStart = workflow.indexOf(' backend-tests:');
@@ -20,6 +75,122 @@ function backendStepIndex(stepName: string) {
} }
describe('project CI workflow', () => { describe('project CI workflow', () => {
it('keeps every job on the isolated preinstalled CI image boundary', () => {
expect(workflow.match(/^ runs-on: genarrative-ci$/gm)).toHaveLength(4);
expect(workflow).not.toContain('actions/checkout');
expect(workflow).not.toContain('actions/setup-node');
expect(workflow).not.toMatch(/^\s+run: .*\b(?:apt|rustup)\b/m);
for (const jobName of jobNames) {
const job = jobSection(jobName);
const checkout = job.indexOf('genarrative-gitea-checkout');
const validateImage = job.indexOf(
'GENARRATIVE_GITEA_CI_CHECK_RUNTIME=1 bash scripts/check-gitea-ci-job-image.sh',
);
const installDependencies = job.indexOf(
'- name: Install npm dependencies',
);
expect(checkout).toBeGreaterThanOrEqual(0);
expect(validateImage).toBeGreaterThan(checkout);
expect(installDependencies).toBeGreaterThan(validateImage);
}
expect(imageDockerfile).toMatch(
/^ARG RUST_IMAGE=[^\s]+@sha256:[a-f0-9]{64}$/m,
);
expect(imageDockerfile).toMatch(
/^ARG RUNNER_IMAGE=[^\s]+@sha256:[a-f0-9]{64}$/m,
);
});
it('retries every root and AI game creator npm clean install as a bounded whole command', () => {
for (const jobName of jobNames) {
const install = stepSection(jobName, 'Install npm dependencies');
expect(install).toContain('bash scripts/ci-npm-ci-with-retry.sh');
expect(install).not.toContain('--prefix');
}
for (const jobName of ['frontend-tests', 'native-shell-tests'] as const) {
const install = stepSection(
jobName,
'Install AI game creator dependencies',
);
expect(install).toContain(
'bash scripts/ci-npm-ci-with-retry.sh --prefix apps/ai-game-creator-shell',
);
}
expect(npmCiRetryScript).toContain(
'max_attempts="${GENARRATIVE_CI_NPM_CI_ATTEMPTS:-3}"',
);
expect(npmCiRetryScript).toContain(
'base_delay_seconds="${GENARRATIVE_CI_NPM_CI_RETRY_DELAY_SECONDS:-5}"',
);
expect(npmCiRetryScript).toContain(
'for attempt in $(seq 1 "${max_attempts}"); do',
);
expect(npmCiRetryScript).toContain('if npm ci "$@"; then');
expect(npmCiRetryScript).toContain(
'if [[ "${attempt}" -eq "${max_attempts}" ]]; then',
);
});
it('builds and verifies image caches against both AI game creator locks', () => {
const npmManifest = 'apps/ai-game-creator-shell/package.json';
const npmLock = 'apps/ai-game-creator-shell/package-lock.json';
const rustManifest = 'apps/ai-game-creator-shell/src-tauri/Cargo.toml';
const rustLock = 'apps/ai-game-creator-shell/src-tauri/Cargo.lock';
for (const [path, expectedCount] of [
[npmManifest, 2],
[npmLock, 3],
[rustManifest, 2],
[rustLock, 3],
] as const) {
expect(imageBuildScript.split(path)).toHaveLength(expectedCount + 1);
expect(imageDockerignore).toContain(`!${path}`);
}
expect(imageBuildScript).toContain(
'--build-arg "AGC_NPM_LOCK_SHA256=${agc_npm_lock_sha256}"',
);
expect(imageBuildScript).toContain(
'--build-arg "AGC_RUST_LOCK_SHA256=${agc_rust_lock_sha256}"',
);
expect(imageDockerfile).toContain('ARG AGC_NPM_LOCK_SHA256');
expect(imageDockerfile).toContain('ARG AGC_RUST_LOCK_SHA256');
expect(imageDockerfile).toContain(
'--prefix /usr/local/share/genarrative-ci/agc-npm',
);
expect(
imageDockerfile.match(
/--manifest-path \/tmp\/genarrative-cargo-cache\/apps\/ai-game-creator-shell\/src-tauri\/Cargo\.toml/g,
),
).toHaveLength(1);
expect(imageDockerfile).toContain(
'cargo_fetch_with_retry /tmp/genarrative-cargo-cache/apps/ai-game-creator-shell/src-tauri/Cargo.toml',
);
expect(imageDockerfile).toContain(
'GENARRATIVE_GITEA_CI_AGC_NPM_LOCK_SHA256=${AGC_NPM_LOCK_SHA256}',
);
expect(imageDockerfile).toContain(
'GENARRATIVE_GITEA_CI_AGC_RUST_LOCK_SHA256=${AGC_RUST_LOCK_SHA256}',
);
expect(imageCheckScript).toContain(npmLock);
expect(imageCheckScript).toContain(rustLock);
expect(imageCheckScript).toContain(
'${GENARRATIVE_GITEA_CI_AGC_NPM_LOCK_SHA256:-}',
);
expect(imageCheckScript).toContain(
'${GENARRATIVE_GITEA_CI_AGC_RUST_LOCK_SHA256:-}',
);
expect(imageCheckScript).toContain(
'::warning title=CI dependency cache is partial::',
);
});
it('prepares locked server-rs dependencies before the first Cargo build gate', () => { it('prepares locked server-rs dependencies before the first Cargo build gate', () => {
const prepareDependencies = backendStepIndex( const prepareDependencies = backendStepIndex(
'Prepare server-rs Rust dependencies', 'Prepare server-rs Rust dependencies',