AGC 壳 Rust 套件分片并行,客户端 Rust 关键路径压到 7 分钟以内
Project CI / AI game creator shell Rust crates (push) Successful in 3m17s
Project CI / Native shell tests (push) Successful in 6m1s
Project CI / Backend tests (push) Successful in 6m39s
Project CI / Frontend tests (push) Successful in 3m59s
Project CI / Repository checks (push) Successful in 2m37s
Project CI / AI game creator shell web tests (push) Successful in 2m37s
Project CI / AI game creator shell Rust tests (push) Has been cancelled
Project CI / AI game creator shell Rust crates (push) Successful in 3m17s
Project CI / Native shell tests (push) Successful in 6m1s
Project CI / Backend tests (push) Successful in 6m39s
Project CI / Frontend tests (push) Successful in 3m59s
Project CI / Repository checks (push) Successful in 2m37s
Project CI / AI game creator shell web tests (push) Successful in 2m37s
Project CI / AI game creator shell Rust tests (push) Has been cancelled
run 2097 显示 AI game creator shell Rust tests 是客户端 CI 的关键路径(15 分 27 秒): 前置 5 分 30 秒(checkout + npm ci + Cargo fetch)、编译 1 分 39 秒、AGC 壳 bin target 的 2466 条单测一条命令串行 507 秒、agent-run smoke 51 秒。 - 新增 apps/ai-game-creator-shell/scripts/run-rust-shell-test-shards.mjs:cargo test --no-run 编译一次拿到测试可执行文件,用 --list 取全部用例名,排序后按 index % shards 切 4 片,每片作为独立进程执行 <bin> --exact <名单> --test-threads=1,并给每片独立 TMPDIR。片并集必须等于 --list 全集且互斥(assertShardsCoverEveryTest),改分片规则 不会静默漏跑。串行口径只从「整套串行」放宽到「片内串行」:当年 libtest 线程并行会 互相干扰的是进程内后台锁与异步终态,进程分片不共享这些状态,因此不必放宽断言。 - package.json:ai-game-creator-shell:check:rust 拆成 :rust:crates(agent-runtime-core / orchestration / platform-llm / shared-contracts)与 :rust:shell(分片运行器),聚合 脚本保持同序,ai-game-creator-shell:check 与本地全量入口语义不变。 - check-native-shells.mjs:分组由五个变六个(新增 agc-rust-crates、agc-rust-shell, 移除 agc-rust);smoke 与壳测试同组以共享 AGC 壳的依赖预热。 - project-ci.yml:新增第 7 个 job AI game creator shell Rust crates(预热 server-rs 与 两个无锁独立 crate 后跑四条 crate 测试);AI game creator shell Rust tests 只保留壳 分片与 agent-run smoke,并只预热 AGC 壳自己那份 manifest(其 Cargo.lock 的 path 依赖 已覆盖 platform-llm / platform-agent / agent-runtime-core / shared-contracts)。 这两个 job 只用 cargo 与 node 内建模块,因此都不再执行 npm ci(各省 1~3 分钟)。 - project-ci-workflow.test.ts:新增纯 cargo job 免 npm ci、分片运行器覆盖校验、crate 级 job 预热顺序等断言;同步运维文档、development-workflow、decision-log、pitfalls。 验证:分片运行器本地以 agent-runtime-core(7 条 → 2/2/2/1)与 platform-llm(146 条 → 49/49/48)验证分片、--exact 与片 TMPDIR 隔离,参数负例 exit 1;vitest scripts/project-ci-workflow.test.ts 12 passed;contract 分组、AGC 与 mobile 壳 check-config、eslint、prettier、check:encoding(13400 文件)、check:doc-index 全绿。 Gitea master 分支保护需补 Project CI / AI game creator shell Rust crates (pull_request)。 Co-authored-by: DotCraft <273930855+dotcraft-ai@users.noreply.github.com>
This commit is contained in:
@@ -85,14 +85,18 @@ const aiGameCreatorViteConfigSource = fs.readFileSync(
|
||||
// - contract:纯源码 / 契约 / 清单断言,不需要任何构建产物。
|
||||
// - shells:微信壳、Expo 移动壳、Tauri 桌面壳与 H5 HostBridge 的现役运行时门禁。
|
||||
// - agc-web:AI 游戏创作壳的前端门禁(typecheck 与壳内测试,不触碰 Cargo)。
|
||||
// - agc-rust:AI 游戏创作壳的 Rust 门禁(共享 / 平台 crate 测试、串行壳测试和
|
||||
// 会用 `src-tauri/Cargo.toml` spawn `cargo` 的 agent-run smoke)。
|
||||
// - agc-rust-crates:AGC 壳依赖的共享 / 平台 crate 测试(server-rs workspace 加两个
|
||||
// 无锁独立 crate),只需 server-rs 侧的依赖预热。
|
||||
// - agc-rust-shell:AGC 壳自身的 Rust 门禁(bin target 单测按名单分片并行、片内保持
|
||||
// `--test-threads=1`)与会用 `src-tauri/Cargo.toml` spawn `cargo run` 的 agent-run
|
||||
// smoke,只需 AGC 壳自己的依赖预热。
|
||||
// - release:发布构建 smoke 和依赖发布产物的落盘断言。
|
||||
const nativeShellGateGroups = [
|
||||
'contract',
|
||||
'shells',
|
||||
'agc-web',
|
||||
'agc-rust',
|
||||
'agc-rust-crates',
|
||||
'agc-rust-shell',
|
||||
'release',
|
||||
];
|
||||
const requestedNativeShellGroups = readRequestedNativeShellGroups(
|
||||
@@ -2310,8 +2314,9 @@ const steps = [
|
||||
},
|
||||
// AI 游戏创作壳原先一步串完 typecheck、壳内测试、共享 / 平台 crate 测试和
|
||||
// 串行壳测试,CI 因此只有一条 10 分钟以上的长尾。这里按同一组命令切成
|
||||
// web 与 rust 两段,顺序与 `npm run ai-game-creator-shell:check` 完全一致,
|
||||
// 但允许 CI 并行执行;本地全量运行仍然是 web -> rust -> smoke 原顺序。
|
||||
// web、rust-crates、rust-shell 三段,顺序与 `npm run ai-game-creator-shell:check`
|
||||
// 完全一致,但允许 CI 把三段放进不同的 job 并行执行;本地全量运行仍然是
|
||||
// web -> rust(crates -> shell) -> smoke 原顺序。
|
||||
{
|
||||
group: 'agc-web',
|
||||
label: 'ai-game-creator-shell-check-web',
|
||||
@@ -2319,15 +2324,21 @@ const steps = [
|
||||
args: ['run', 'ai-game-creator-shell:check:web'],
|
||||
},
|
||||
{
|
||||
group: 'agc-rust',
|
||||
label: 'ai-game-creator-shell-check-rust',
|
||||
group: 'agc-rust-crates',
|
||||
label: 'ai-game-creator-shell-check-rust-crates',
|
||||
command: npmCommand,
|
||||
args: ['run', 'ai-game-creator-shell:check:rust'],
|
||||
args: ['run', 'ai-game-creator-shell:check:rust:crates'],
|
||||
},
|
||||
// agent-run smoke 会用 `src-tauri/Cargo.toml` spawn `cargo`,因此与 Rust 段同组,
|
||||
// 保证它落在已经预热 AGC Cargo 依赖的 job 里。
|
||||
{
|
||||
group: 'agc-rust',
|
||||
group: 'agc-rust-shell',
|
||||
label: 'ai-game-creator-shell-check-rust-shell',
|
||||
command: npmCommand,
|
||||
args: ['run', 'ai-game-creator-shell:check:rust:shell'],
|
||||
},
|
||||
// agent-run smoke 会用 `src-tauri/Cargo.toml` spawn `cargo run`,因此与壳测试同组,
|
||||
// 保证它落在已经预热 AGC 壳 Cargo 依赖的 job 里。
|
||||
{
|
||||
group: 'agc-rust-shell',
|
||||
label: 'ai-game-creator-shell-agent-run-smoke',
|
||||
command: npmCommand,
|
||||
args: ['run', 'ai-game-creator-shell:agent-run:smoke'],
|
||||
|
||||
@@ -42,8 +42,18 @@ const jobNames = [
|
||||
'native-shell-tests',
|
||||
'ai-game-creator-shell-web-tests',
|
||||
'ai-game-creator-shell-rust-tests',
|
||||
'ai-game-creator-shell-rust-crates',
|
||||
] as const;
|
||||
|
||||
// 这两个 job 是纯 cargo 门禁:AGC 壳的 Rust 套件与 agent-run smoke 只用 cargo 与
|
||||
// node 内建模块(smoke 脚本只 import `node:*`),crate 级测试只跑 cargo 命令,
|
||||
// 都不需要 node_modules。省掉两份 `npm ci`(各 1~3 分钟)是把客户端 Rust 关键路径
|
||||
// 压到 7 分钟以内的前提,因此这里显式允许它们不装 npm 依赖。
|
||||
const jobsWithoutNpmInstall: readonly string[] = [
|
||||
'ai-game-creator-shell-rust-tests',
|
||||
'ai-game-creator-shell-rust-crates',
|
||||
];
|
||||
|
||||
const rootPackageJson = JSON.parse(
|
||||
readFileSync(resolve(process.cwd(), 'package.json'), 'utf8'),
|
||||
) as { scripts?: Record<string, string> };
|
||||
@@ -58,7 +68,8 @@ const nativeShellGateGroupScripts = {
|
||||
contract: 'npm run check:native-shells:contract',
|
||||
shells: 'npm run check:native-shells:shells',
|
||||
'agc-web': 'npm run check:native-shells:agc-web',
|
||||
'agc-rust': 'npm run check:native-shells:agc-rust',
|
||||
'agc-rust-crates': 'npm run check:native-shells:agc-rust-crates',
|
||||
'agc-rust-shell': 'npm run check:native-shells:agc-rust-shell',
|
||||
release: 'npm run check:native-shells:release',
|
||||
} as const;
|
||||
|
||||
@@ -130,6 +141,10 @@ describe('project CI workflow', () => {
|
||||
|
||||
expect(checkout).toBeGreaterThanOrEqual(0);
|
||||
expect(validateImage).toBeGreaterThan(checkout);
|
||||
if (jobsWithoutNpmInstall.includes(jobName)) {
|
||||
expect(installDependencies).toBe(-1);
|
||||
continue;
|
||||
}
|
||||
expect(installDependencies).toBeGreaterThan(validateImage);
|
||||
}
|
||||
|
||||
@@ -160,8 +175,12 @@ describe('project CI workflow', () => {
|
||||
);
|
||||
});
|
||||
|
||||
it('retries the single root workspace clean install in every job as a bounded whole command', () => {
|
||||
it('retries the single root workspace clean install in every job that needs node_modules', () => {
|
||||
for (const jobName of jobNames) {
|
||||
if (jobsWithoutNpmInstall.includes(jobName)) {
|
||||
expect(jobSection(jobName)).not.toContain('ci-npm-ci-with-retry.sh');
|
||||
continue;
|
||||
}
|
||||
const install = stepSection(jobName, 'Install npm dependencies');
|
||||
expect(install).toContain('bash scripts/ci-npm-ci-with-retry.sh');
|
||||
expect(install).not.toContain('--prefix');
|
||||
@@ -381,36 +400,75 @@ describe('project CI workflow', () => {
|
||||
expect(workflow).not.toMatch(/^ {8}run: npm run check:native-shells$/mu);
|
||||
});
|
||||
|
||||
it('splits the AI game creator shell gates into web and Rust jobs', () => {
|
||||
it('splits the AI game creator shell gates into web, shell shard, and shared crate jobs', () => {
|
||||
const webJob = jobSection('ai-game-creator-shell-web-tests');
|
||||
expect(webJob).toContain('run: npm run check:native-shells:agc-web');
|
||||
expect(webJob).not.toContain('cargo fetch');
|
||||
|
||||
const rustJob = jobSection('ai-game-creator-shell-rust-tests');
|
||||
expect(rustJob).toContain('run: npm run check:native-shells:agc-rust');
|
||||
expect(rustJob).toContain('server-rs/Cargo.toml');
|
||||
expect(rustJob).toContain(
|
||||
'run: npm run check:native-shells:agc-rust-shell',
|
||||
);
|
||||
expect(rustJob).toContain(
|
||||
'apps/ai-game-creator-shell/src-tauri/Cargo.toml',
|
||||
);
|
||||
expect(rustJob).toContain('cargo fetch --locked');
|
||||
// 壳 job 只预热 AGC 壳自己那份锁定依赖:server-rs 那份归 crate 级 job。
|
||||
expect(rustJob).not.toContain('server-rs/Cargo.toml');
|
||||
|
||||
// 拆开的 web / rust 两段必须还是原 `ai-game-creator-shell:check` 的同一条命令序列。
|
||||
const cratesJob = jobSection('ai-game-creator-shell-rust-crates');
|
||||
expect(cratesJob).toContain(
|
||||
'run: npm run check:native-shells:agc-rust-crates',
|
||||
);
|
||||
expect(cratesJob).toContain('server-rs/Cargo.toml');
|
||||
expect(cratesJob).toContain('cargo fetch --locked');
|
||||
|
||||
// 拆开的 web / rust 两段必须还是原 `ai-game-creator-shell:check` 的同一条命令序列,
|
||||
// rust 段再拆成 crate 级与壳分片两段后在聚合脚本里保持同序。
|
||||
expect(rootPackageJson.scripts?.['ai-game-creator-shell:check']).toBe(
|
||||
'npm run ai-game-creator-shell:check:web && npm run ai-game-creator-shell:check:rust && npm run ai-game-creator-shell:agent-run:smoke',
|
||||
);
|
||||
expect(rootPackageJson.scripts?.['ai-game-creator-shell:check:web']).toBe(
|
||||
'npm run ai-game-creator-shell:typecheck && npm run test -- apps/ai-game-creator-shell/tests',
|
||||
);
|
||||
expect(
|
||||
rootPackageJson.scripts?.['ai-game-creator-shell:check:rust'],
|
||||
).toContain(
|
||||
'cargo test --locked --manifest-path apps/ai-game-creator-shell/src-tauri/Cargo.toml -- --test-threads=1',
|
||||
expect(rootPackageJson.scripts?.['ai-game-creator-shell:check:rust']).toBe(
|
||||
'npm run ai-game-creator-shell:check:rust:crates && npm run ai-game-creator-shell:check:rust:shell',
|
||||
);
|
||||
expect(
|
||||
rootPackageJson.scripts?.['ai-game-creator-shell:check:rust:crates'],
|
||||
).toContain(
|
||||
'cargo test --locked -p shared-contracts --manifest-path server-rs/Cargo.toml game_creation_app',
|
||||
);
|
||||
expect(
|
||||
rootPackageJson.scripts?.['ai-game-creator-shell:check:rust:shell'],
|
||||
).toBe(
|
||||
'node apps/ai-game-creator-shell/scripts/run-rust-shell-test-shards.mjs --shards=4',
|
||||
);
|
||||
// 整套用例不再作为一条 `cargo test -- --test-threads=1` 命令串行跑:CI 与本地都走
|
||||
// 分片运行器,片内串行、片间并行。
|
||||
expect(workflow).not.toContain('-- --test-threads=1');
|
||||
});
|
||||
|
||||
it('prefetches the excluded standalone Rust crates before the AI game creator shell Rust gates', () => {
|
||||
it('keeps the shell shard runner exhaustive over discovered tests', () => {
|
||||
const shardRunner = readFileSync(
|
||||
resolve(
|
||||
process.cwd(),
|
||||
'apps/ai-game-creator-shell/scripts/run-rust-shell-test-shards.mjs',
|
||||
),
|
||||
'utf8',
|
||||
);
|
||||
// 分片必须对 `--list` 的全集做覆盖与互斥校验,否则改分片规则会静默漏跑门禁。
|
||||
expect(shardRunner).toContain('function assertShardsCoverEveryTest(');
|
||||
expect(shardRunner).toContain('must be exhaustive');
|
||||
expect(shardRunner).toContain('must be disjoint');
|
||||
expect(shardRunner).toContain("'--test-threads=1'");
|
||||
expect(shardRunner).toContain('TMPDIR: shardTmpDirectory');
|
||||
expect(shardRunner).toContain("'--exact'");
|
||||
});
|
||||
|
||||
it('prefetches the excluded standalone Rust crates before the shared crate gates', () => {
|
||||
const standaloneStep = stepSection(
|
||||
'ai-game-creator-shell-rust-tests',
|
||||
'ai-game-creator-shell-rust-crates',
|
||||
'Prepare standalone Rust crate dependencies',
|
||||
);
|
||||
for (const manifest of [
|
||||
@@ -424,11 +482,11 @@ describe('project CI workflow', () => {
|
||||
expect(standaloneStep).toContain('cargo fetch \\');
|
||||
expect(standaloneStep).not.toContain('cargo fetch --locked');
|
||||
|
||||
const rustJob = jobSection('ai-game-creator-shell-rust-tests');
|
||||
const cratesJob = jobSection('ai-game-creator-shell-rust-crates');
|
||||
expect(
|
||||
rustJob.indexOf('Prepare standalone Rust crate dependencies'),
|
||||
cratesJob.indexOf('Prepare standalone Rust crate dependencies'),
|
||||
).toBeLessThan(
|
||||
rustJob.indexOf('run: npm run check:native-shells:agc-rust'),
|
||||
cratesJob.indexOf('run: npm run check:native-shells:agc-rust-crates'),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user