给真机夹具加 restart 用例:回合进行中杀进程后部分历史与宿主事实
Project CI / Backend tests (push) Has been cancelled
Project CI / AI game creator shell Rust lane 2/2 (push) Has been cancelled
Project CI / AI game creator shell Rust smoke (push) Has been cancelled
Project CI / AI game creator shell Rust crates (push) Has been cancelled
Project CI / Native shell tests (push) Has been cancelled
Project CI / Frontend tests (push) Has been cancelled
Project CI / Repository checks (push) Has been cancelled
Project CI / AI game creator shell web tests (push) Has been cancelled
Project CI / AI game creator shell Rust lane 1/2 (push) Has been cancelled

- direct-execution-production-fixture:新增 restart 场景(契约 → 部分助手文本 + 阻塞原生工具 → 工具真开始后 SIGTERM 杀进程),断言部分文本与工具卡片按顺序落盘、被杀工具卡片是最后一行且无回执行、账本不伪造成 completed/executorStopped
- 复验:completed,passes,mcp,mcp-write,native,deadline,restart 共 7/7 passed
- 变异验证:把"等 400ms 再杀"改成"等 20s"(让回合自然跑完)后同一条用例立即变红(AGC must be killed mid-turn)
- 记录事实:CLI/无前端宿主不写用户消息(GUI 由前端写),夹具断言据此只校验部分文本与工具卡片顺序
- 文档:DirectProject 里程碑补该轮核对与剩余 UI 侧边界;pitfalls 记 CLI 不写用户消息这一坑
This commit is contained in:
kdletters
2026-09-28 23:32:02 +08:00
parent 4415c7a5ff
commit c388803729
3 changed files with 145 additions and 35 deletions
@@ -150,6 +150,63 @@ async function readLedger(host) {
return null;
}
/**
* `restart` 用例判据:回合进行中杀掉进程后
* - 已落盘的用户消息、部分助手文本与工具卡片必须按原顺序保留;
* - 宿主不得为这一轮伪造终态(不能出现最终助手消息,账本不能是 completed / executorStopped)。
* 对应的里程碑条款是「回合进行中终止并重启进程后重进项目:已落盘的部分文本与工具卡片按原顺序出现,
* 且界面不显示忙碌态」里**宿主侧可证的那一半**(界面渲染仍要真实客户端窗口)。
*/
async function assertRestartKeepsPartialHistory(project, ledger, exit) {
assert(
exit.code !== 0 || exit.signal,
'restart: AGC must be killed mid-turn, got ' + JSON.stringify(exit),
);
const history = await fs
.readFile(path.join(project, '.agent/conversations/project.jsonl'), 'utf8')
.catch(() => '');
const rows = history.split('\n').filter(Boolean);
const raw = rows.join('\n');
// 注意:CLI / 无前端宿主不写用户消息(GUI 是前端先调 append_direct_project_conversation_message),
// 所以这里只断言"部分文本 + 工具卡片按原顺序落盘",用户消息的持久化属于 GUI 路径。
const order = ['RESTART-PARTIAL-TEXT', 'exec_command'].map((needle) =>
raw.indexOf(needle),
);
assert(
order.every((index) => index >= 0) && order[0] < order[1],
'restart: partial text and tool card must stay in order; got ' +
JSON.stringify({ order, rows: rows.length }),
);
assert(
!raw.includes('Fixture actions finished'),
'restart: killed turn must not fabricate the final assistant message',
);
// 被杀的工具卡片必须是最后一条:既证明卡片已落盘,也证明它没有产生任何回执
// (有回执就说明这一轮跑到了工具结束之后,不再是"进行中终止")。
const lastRow = rows[rows.length - 1] ?? '';
assert(
lastRow.includes('restart-slow') && lastRow.includes('exec_command'),
'restart: the killed tool card must be the last persisted row; got ' +
lastRow.slice(0, 200),
);
assert(
!rows.some(
(row) => row.includes('restart-slow') && row.includes('"output"'),
),
'restart: the killed tool must not have produced an output row',
);
assert.notEqual(
ledger.phase,
'completed',
'restart: killed turn must not claim completion',
);
assert.notEqual(
ledger.executorStopped,
true,
'restart: killed turn must not report a fully stopped executor',
);
}
function ownFixtureTree(directory) {
if (process.platform !== 'win32') return;
assert(path.resolve(directory).startsWith(path.resolve(root) + path.sep));
@@ -260,6 +317,7 @@ async function runScenario(name) {
'patch',
'deadline',
'native-session',
'restart',
].includes(name),
'unknown fixture case',
);
@@ -469,18 +527,42 @@ async function runScenario(name) {
),
],
]
: [
[register('never.txt')],
[native('build', 'console.log(1+1);')],
[native('test', 'console.log(2+2);')],
[native('fail', 'process.exit(1);')],
[
native(
'forbidden-after-budget',
writes('forbidden.txt', 'must not run'),
),
],
];
: name === 'restart'
? [
[register('started.txt')],
[
{
type: 'message',
id: 'fixture-restart-partial',
role: 'assistant',
content: [
{
type: 'output_text',
text: 'RESTART-PARTIAL-TEXT',
},
],
},
native(
'restart-slow',
writes('started.txt', 'started') +
'setTimeout(()=>{' +
writes('late.txt', 'must not run after kill') +
'},15000);',
),
],
]
: [
[register('never.txt')],
[native('build', 'console.log(1+1);')],
[native('test', 'console.log(2+2);')],
[native('fail', 'process.exit(1);')],
[
native(
'forbidden-after-budget',
writes('forbidden.txt', 'must not run'),
),
],
];
const requests = [];
const responses = [];
const httpArrivals = [];
@@ -696,12 +778,29 @@ async function runScenario(name) {
child.stderr.on('data', (data) => {
stderr = (stderr + data.toString('utf8')).slice(-2 * 1024 * 1024);
});
// restart:等工具真的开始跑(marker 落盘)再杀进程,保证"回合进行中终止"是真事,
// 而不是在模型请求还没回来时就把它掐掉。
const restartKillWatch =
name === 'restart'
? (async () => {
const marker = path.join(project, 'started.txt');
const deadline = Date.now() + 30_000;
while (Date.now() < deadline && !(await exists(marker))) {
await sleep(20);
}
assert(await exists(marker), 'restart: tool never started');
// 留一点时间让这一轮的工具卡片落进项目历史,再终止进程。
await sleep(400);
child.kill();
})()
: null;
const timeout = setTimeout(() => child.kill(), 90_000);
const exit = await new Promise((resolve, reject) => {
child.once('error', reject);
child.once('exit', (code, signal) => resolve({ code, signal }));
}).finally(() => clearTimeout(timeout));
if (planWatch) clearInterval(planWatch);
if (restartKillWatch) await restartKillWatch;
await planObservationTask;
server.closeAllConnections();
await new Promise((resolve) => server.close(resolve));
@@ -829,28 +928,32 @@ async function runScenario(name) {
': Direct Responses requests must allow multiple tool calls without changing the selected model',
);
assert(ledger, name + ': missing host-authoritative ledger');
assert.equal(exit.code, 0, name + ': CLI failed; inspect result.json');
assert.equal(
report.markers.forbidden,
false,
name + ': effect ran after host terminal',
);
assert.equal(
ledger.value.usedPasses,
1,
name + ': ordinary commands must share one pass',
);
assert.equal(
ledger.value.phase,
['completed', 'patch', 'native-session'].includes(name)
? 'completed'
: 'exhausted',
);
assert.equal(
ledger.value.executorStopped,
true,
name + ': missing full executor exit proof',
);
if (name === 'restart') {
await assertRestartKeepsPartialHistory(project, ledger.value, exit);
} else {
assert.equal(exit.code, 0, name + ': CLI failed; inspect result.json');
assert.equal(
report.markers.forbidden,
false,
name + ': effect ran after host terminal',
);
assert.equal(
ledger.value.usedPasses,
1,
name + ': ordinary commands must share one pass',
);
assert.equal(
ledger.value.phase,
['completed', 'patch', 'native-session'].includes(name)
? 'completed'
: 'exhausted',
);
assert.equal(
ledger.value.executorStopped,
true,
name + ': missing full executor exit proof',
);
}
if (name === 'completed') assert.equal(report.markers.ready, true);
if (isMcp) {
assert.equal(
@@ -84,4 +84,10 @@ AGC 项目开发对话的显示与恢复只依赖两项输入:**项目对话
- **第二层已修:CLI / 单回合宿主的收尾不再被记成本轮未完成**。定位过程(都靠插桩,不是推测):① 不是 codex 版本漂移(换 0.147.0 相同);② 不是 stdin 被忽略(改 `pipe` 相同);③ 不是 `Drop for ExecutionBinding`(插桩后一次没打印);④ app-server 不是崩溃(1242 字节 stderr 全是 ProgramData/模型元数据/PowerShell snapshot 之类 WARN);⑤ **是宿主自己关的**——关闭原因日志显示 `reason=宿主执行预算或交付收尾`(`execution.rs:938` 的 `shutdown_and_report()` 先置 `closed` 再关 app-server,关进程会给在途回合发 `TransportClosed`)。消费端(`mod.rs:4015`)只看 session 阶段、没看 `closed`,而 `fail_turn()` 内部无论如何都会 `session.interrupt(reason)`,于是正常收尾被改写成 `interrupted`、回执换成「执行通道已断开」。
修复:`fail_turn()` 在 `is_closed()` 时只把原因追加进终态说明(新增 `ExecutionSession::append_terminal_note()`),**不改阶段**;用户主动终止与真实通道故障保持原口径。
**复验(本里程碑现在有了可复跑的真机证路)**:`node apps/ai-game-creator-shell/scripts/direct-execution-production-fixture.mjs --agc-exe apps/ai-game-creator-shell/src-tauri/target/debug/genarrative-ai-game-creator-shell.exe --cases completed,passes,mcp,mcp-write,native,deadline` → **6/6 passed**(真 AGC CLI → 真 app-server → loopback Responses,无账号无付费);`completed` 的账本 `phase=completed`、`executorStopped=true`,`terminalReport` 变成宿主真实交付报告。定向单测 `codex_app_server::execution` 13 passed、`direct_execution` 26 passed。
- **两条运行时验收仍继续未勾选**:它们的判据里有「界面不显示忙碌态」「页面重进(含切走再切回)能恢复运行中回合并允许终止」,本质要真实客户端窗口;上面的夹具只能证「真实 app-server 下的回合收尾与历史持久化」,证不了 UI 投影。
- **再次核对(2026-09-28,同一夹具加 `restart` 用例)**:给 `direct-execution-production-fixture.mjs` 新增 `restart` 场景——受控契约注册 → 一轮里同时下发**部分助手文本**与一个**阻塞的原生工具** → 等工具真的开始(`started.txt` 落盘)后 `SIGTERM` 杀掉 AGC 进程。判据(全部实跑通过,`--cases completed,passes,mcp,mcp-write,native,deadline,restart` → **7/7 passed**):
- 进程确实死在回合进行中:`exit={code:null, signal:'SIGTERM'}`;
- `.agent/conversations/project.jsonl` 里 `RESTART-PARTIAL-TEXT` 与工具卡片(`exec_command`)**按原顺序**保留,且被杀的工具卡片是**最后一行**、没有对应回执行(说明它没跑到工具结束之后);
- 宿主没有伪造终态:账本 `phase=working`(不是 `completed`)、`executorStopped=false`;
- **变异验证**:把"等 400ms 再杀"改成"等 20s 再杀"(让这一轮自然跑完)后,同一条用例立刻以 `restart: AGC must be killed mid-turn, got {"code":0,"signal":null}` 变红,证明这些断言确实在区分「进行中终止」和「已完成」。
- 顺带记录一个事实:CLI / 无前端宿主**不写用户消息**(GUI 是前端先调 `append_direct_project_conversation_message`),所以用例只断言部分文本与工具卡片的顺序;用户消息的持久化属于 GUI 路径。
- **两条运行时验收仍继续未勾选**:上面这条覆盖了「已落盘的部分文本与工具卡片按原顺序出现、且宿主不假装完成」的宿主侧证据;但判据里的「**界面不显示忙碌态**」「页面重进(含切走再切回)能恢复运行中回合并允许终止」仍要真实客户端窗口,夹具证不了 UI 投影。
@@ -6119,4 +6119,5 @@ Cocos Creator 根目录由 `package.json.creator.version` 与普通 `assets/`
修复:①`fail_turn()` 在 `is_closed()`(宿主自己收尾)时只把原因追加进终态说明、**不改阶段**,新增 `ExecutionSession::append_terminal_note()`(`interrupt()` 同款写法但不设 `Interrupted`);用户主动终止(`host_stop_requested`)与真实通道故障仍保持原口径。
复验(真 AGC CLI → 真 app-server → loopback Responses):`node apps/ai-game-creator-shell/scripts/direct-execution-production-fixture.mjs --agc-exe apps/ai-game-creator-shell/src-tauri/target/debug/genarrative-ai-game-creator-shell.exe --cases completed,passes,mcp,mcp-write,native,deadline` → **6/6 passed**;`completed` 用例的账本从 `phase=interrupted` 变成 `phase=completed`、`executorStopped=true`,`terminalReport` 变成宿主真实交付报告(「本轮已完成宿主验收…已通过:产物 marker.txt」),不再是「执行通道已断开」。定向单测:`cargo test … codex_app_server::execution` 13 passed、`cargo test … direct_execution` 26 passed。
- **保留的诊断开关**:关闭原因日志留在 `GENARRATIVE_AGC_DIRECT_DEBUG=1` 下(与既有的 `agent.direct_codex.stderr bytes=` 同一开关),以后排查「谁关掉的」不用再插桩。
- **顺带发现(2026-09-28)**:同一个夹具新增的 `restart` 用例证明——CLI / 无前端宿主**不会把用户消息写进 `.agent/conversations/project.jsonl`**(GUI 是前端先调 `append_direct_project_conversation_message`,所以总有至少一条用户条目)。写依赖"历史里一定有用户消息"的夹具断言前先确认是走 GUI 还是 CLI 路径,否则会得到假失败。
- **顺带记一条环境陷阱(2026-09-28 已修)**:`apps/ai-game-creator-shell/src-tauri/resources/codex/win-x64/` 下曾有两个 codex 二进制——`bin/codex.exe` 是**真正被解析**的那份(0.155.1),而包根目录那份 `codex.exe` 是 0.147.0 的旧残留(tauri 的 Windows 资源映射只引用 `bin/` 等路径),检查都查不出来,却会让本地核对误判「应用跑的是 0.147.0」。根因是 `src-tauri/build.rs` 的 `stage_codex_target()` 只按布局拷贝、从不清理目录,旧布局的组件会永久留在随包资源目录里。现在加了 `prune_stale_codex_components()`:拷贝前删掉不在本轮布局、也不在 `manifest.json`/`NOTICE.md` 白名单里的文件并收掉空目录;实测重建后根目录 `codex.exe` 被清掉、六个声明组件与清单/声明保留。